dynare issueshttps://git.dynare.org/Dynare/dynare/-/issues2020-02-17T14:50:00Zhttps://git.dynare.org/Dynare/dynare/-/issues/1706Fix wrong computation in third-order approximation in pruned_state_space.m2020-02-17T14:50:00ZJohannes Pfeifer Fix wrong computation in third-order approximation in pruned_state_space.mMattermost discussion 06/02/20Mattermost discussion 06/02/204.6Willi Mutschlerwilli@mutschler.euWilli Mutschlerwilli@mutschler.euhttps://git.dynare.org/Dynare/dynare/-/issues/1718Auxillary particle filter --- number_of_state_variables is undefined2020-03-30T10:09:49ZArtur ZmanovskiiAuxillary particle filter --- number_of_state_variables is undefinedIn `particles/src/auxiliary_particle_filter.m` under specific options (`options_.particle.pruning=1`, `resampling` in `estimation()` is `systematic/generic`), a chunk of code is executed (lines 136--139), which has undefined variable (`number_of_state_variables` --- at lines 137--138) and causes error.In `particles/src/auxiliary_particle_filter.m` under specific options (`options_.particle.pruning=1`, `resampling` in `estimation()` is `systematic/generic`), a chunk of code is executed (lines 136--139), which has undefined variable (`number_of_state_variables` --- at lines 137--138) and causes error.4.7Stéphane Adjemianstepan@adjemian.euStéphane Adjemianstepan@adjemian.euhttps://git.dynare.org/Dynare/dynare/-/issues/1715Clean root folder of /tests2020-03-07T17:21:34ZJohannes Pfeifer Clean root folder of /testsAll integration tests should be in subfolders. The current structure is a messAll integration tests should be in subfolders. The current structure is a mess4.7https://git.dynare.org/Dynare/dynare/-/issues/1713Decide minimal MATLAB version requirement for Dynare 4.72020-03-04T08:02:42ZSébastien VillemotDecide minimal MATLAB version requirement for Dynare 4.7We need to decide what will be the minimal version of MATLAB required to run Dynare 4.7.
For Dynare 4.6, our policy has been to support MATLAB versions that are at most 10 years old.
Assuming that Dynare 4.7 is released in 2021, keeping the same policy would imply raising the minimal MATLAB version to R2011a or R2011b (depending on the exact release date).
But we could also decide to change our policy and support less versions. For example, we could go for a 5-years windows, which would imply R2016a or R2016b. Or we could choose something between 5 and 10 years.
Relevant to this discussion is the list of [MATLAB incompatibilities across versions](https://git.dynare.org/Dynare/dynare/-/wikis/MATLAB-Versions). Here are the main benefits that would bring certain requirements:
- *R2013a*: we could get rid of the hack needed to support `intersect(…, 'stable')`
- *R2014a*: we could easily install the minimal required MATLAB version on modern GNU/Linux systems (currently we need a hack to create an artificial `eth0` device with the correct MAC address)
- *R2016a*: we could drop the support for 32-bit versions, which would halve the size of the Windows installer and simplify the build process
- *R2016b*: we could use automatic broadcasting in many places, instead of the obscure `bsxfun` syntaxWe need to decide what will be the minimal version of MATLAB required to run Dynare 4.7.
For Dynare 4.6, our policy has been to support MATLAB versions that are at most 10 years old.
Assuming that Dynare 4.7 is released in 2021, keeping the same policy would imply raising the minimal MATLAB version to R2011a or R2011b (depending on the exact release date).
But we could also decide to change our policy and support less versions. For example, we could go for a 5-years windows, which would imply R2016a or R2016b. Or we could choose something between 5 and 10 years.
Relevant to this discussion is the list of [MATLAB incompatibilities across versions](https://git.dynare.org/Dynare/dynare/-/wikis/MATLAB-Versions). Here are the main benefits that would bring certain requirements:
- *R2013a*: we could get rid of the hack needed to support `intersect(…, 'stable')`
- *R2014a*: we could easily install the minimal required MATLAB version on modern GNU/Linux systems (currently we need a hack to create an artificial `eth0` device with the correct MAC address)
- *R2016a*: we could drop the support for 32-bit versions, which would halve the size of the Windows installer and simplify the build process
- *R2016b*: we could use automatic broadcasting in many places, instead of the obscure `bsxfun` syntax4.7https://git.dynare.org/Dynare/dynare/-/issues/1707Macro commands that take no arguments have spurious parentheses appended to t...2020-02-17T14:50:00ZSébastien VillemotMacro commands that take no arguments have spurious parentheses appended to their synopsisSee for example the definition of `@#else`, `@#endif`, `@#endfor`, `@#echomacrovars` in the reference manual.See for example the definition of `@#else`, `@#endif`, `@#endfor`, `@#echomacrovars` in the reference manual.4.7Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/-/issues/1704Update partial information code2020-03-02T16:14:50ZSébastien VillemotUpdate partial information codeFirst version located in https://git.dynare.org/Yang/dynareFirst version located in https://git.dynare.org/Yang/dynare4.7Sébastien VillemotSébastien Villemothttps://git.dynare.org/Dynare/dynare/-/issues/1703det_cond_forecast() depends on oo_.dr.state_var but this is not always available2020-02-11T10:08:29ZMichelJuillarddet_cond_forecast() depends on oo_.dr.state_var but this is not always available``det_cond_forecast()`` depends on ``oo_.dr.state_var`` but this variable is computed by ``dyn_first_order_solver``. So it isn't available for a purely backward or purely forward model``det_cond_forecast()`` depends on ``oo_.dr.state_var`` but this variable is computed by ``dyn_first_order_solver``. So it isn't available for a purely backward or purely forward model4.7MichelJuillardMichelJuillardhttps://git.dynare.org/Dynare/dynare/-/issues/1702det_cond_forecast() argument checking is broken2020-02-11T09:44:04ZMichelJuillarddet_cond_forecast() argument checking is brokenIf one calls det_cond_forecast with two arguments only as in the manual example, the function fails with error
```
impossible case
```
that is triggered near line 208If one calls det_cond_forecast with two arguments only as in the manual example, the function fails with error
```
impossible case
```
that is triggered near line 2084.7MichelJuillardMichelJuillardhttps://git.dynare.org/Dynare/dynare/-/issues/1700Proposal for a generalized solver2020-01-30T09:53:09ZMichelJuillardProposal for a generalized solverModel inversion and deterministic conditional forecast solve the same mathematical problem: solving a (non-)linear system where the identity of the unknown variables change from period to period. I suggest to create a unique function that would solve such problems.
``general_solver`` would take at least the following arguments:
* y: endogenous variables (matrix)
* x: exogenous variables (matrix) [possible concatenation of exo_det variables to the right]
* model: model (function handle)
* i_endo_flip: flip variables indices in endogenous variables (vector)
* i_exo_flip: flip variables indices in exogenous variables (vector)
### Conventions
1. there are as many endogenous flip variables as exogenenous or exo det variables
1. the endogenous and exogenous flip variables are in the same order (not necessary to be contiguous but may help)
1. given the indices vector of flip variablesm implicit pairs of endogenous/exogenous variables are formed
1. If the observation of an endogenous variable is a NaN, the endogenous variables plays it usual role of endogenous variable
1. If the observation of an endogenous variables is a valid number, the endogenous variable is treated as exogenous and the corresponding flip exogenous variables is treated as endogenous
1. The presence of a NaN for an endogenous variable not declared as a flip variable is an error
1. The corner case of unconditional forecast should be accepted for ease of use
1. The solution is written back in ``y`` and ``x`` irrespective of the actual role of a variable in a given period
### Operations
``general_solver`` dispatches la résolution du nouveau problème selon
1. purely backward linear model
1. purely backward nonlinear model
1. purely forward linear model (if we have the algorithm...)
1. purely forward nonlinear model
1. general linear model
1. general nonlinear model
1. bytecode and or block model
and according to the options selecting a particular algorithmModel inversion and deterministic conditional forecast solve the same mathematical problem: solving a (non-)linear system where the identity of the unknown variables change from period to period. I suggest to create a unique function that would solve such problems.
``general_solver`` would take at least the following arguments:
* y: endogenous variables (matrix)
* x: exogenous variables (matrix) [possible concatenation of exo_det variables to the right]
* model: model (function handle)
* i_endo_flip: flip variables indices in endogenous variables (vector)
* i_exo_flip: flip variables indices in exogenous variables (vector)
### Conventions
1. there are as many endogenous flip variables as exogenenous or exo det variables
1. the endogenous and exogenous flip variables are in the same order (not necessary to be contiguous but may help)
1. given the indices vector of flip variablesm implicit pairs of endogenous/exogenous variables are formed
1. If the observation of an endogenous variable is a NaN, the endogenous variables plays it usual role of endogenous variable
1. If the observation of an endogenous variables is a valid number, the endogenous variable is treated as exogenous and the corresponding flip exogenous variables is treated as endogenous
1. The presence of a NaN for an endogenous variable not declared as a flip variable is an error
1. The corner case of unconditional forecast should be accepted for ease of use
1. The solution is written back in ``y`` and ``x`` irrespective of the actual role of a variable in a given period
### Operations
``general_solver`` dispatches la résolution du nouveau problème selon
1. purely backward linear model
1. purely backward nonlinear model
1. purely forward linear model (if we have the algorithm...)
1. purely forward nonlinear model
1. general linear model
1. general nonlinear model
1. bytecode and or block model
and according to the options selecting a particular algorithm4.7https://git.dynare.org/Dynare/dynare/-/issues/1698Allow checking linearity for perfect foresight models2020-02-14T15:05:47ZJohannes Pfeifer Allow checking linearity for perfect foresight modelsThe linearity check underlying `model(linear)` is based on the Hessian of the model. But we don’t compute the Hessian for a perfect foresight simulation, hence the check is skipped. See line 799 of `preprocessor/src/ModFile.cc`.
That can be problematic in case of nonlinearities like a ZLB constraint in an otherwise linear model. The solver assumes that the Jacobian is the same at every period, and the constraint is not enforced.
For very large perfect foresight models, we actually don’t want to put the Hessian in the generated dynamic file, because compiling it can be very time consuming. Hence the fix is not straightforward.
I would suggest to add a way of testing this. One option would be trigger the Hessian computation with a `debug` option or something like this. For example, we could use the already present command line option `debug`.The linearity check underlying `model(linear)` is based on the Hessian of the model. But we don’t compute the Hessian for a perfect foresight simulation, hence the check is skipped. See line 799 of `preprocessor/src/ModFile.cc`.
That can be problematic in case of nonlinearities like a ZLB constraint in an otherwise linear model. The solver assumes that the Jacobian is the same at every period, and the constraint is not enforced.
For very large perfect foresight models, we actually don’t want to put the Hessian in the generated dynamic file, because compiling it can be very time consuming. Hence the fix is not straightforward.
I would suggest to add a way of testing this. One option would be trigger the Hessian computation with a `debug` option or something like this. For example, we could use the already present command line option `debug`.4.7https://git.dynare.org/Dynare/dynare/-/issues/1693Various improvements in Sphinx doc2020-01-08T16:24:40ZHoutan BastaniVarious improvements in Sphinx doc- see if we can use https://pypi.org/project/sphinxcontrib-matlabdomain/ instead of `MatComm` and `MatlabVar` defined in `doc/manual/utils/dynare_dom.py`
- if not, add new domain entry to differentiate between MATLAB commands and MATLAB functions
- find way to fix output of Matlab Commands so the options conform to the true type used for Dynare Command options
- code block:
- many general problems with highlighting: e.g. `end` is highlighted when it is MATLAB code but the corresponding `for` is not highlighted
- despite being in `doc/manual/utils/dynare_lex.py`, `var` is not highlighted in code blocks
- ideally find a programmatic way to fill `doc/manual/utils/dynare_lex.py` from the `rst` files instead of having to update it by hand- see if we can use https://pypi.org/project/sphinxcontrib-matlabdomain/ instead of `MatComm` and `MatlabVar` defined in `doc/manual/utils/dynare_dom.py`
- if not, add new domain entry to differentiate between MATLAB commands and MATLAB functions
- find way to fix output of Matlab Commands so the options conform to the true type used for Dynare Command options
- code block:
- many general problems with highlighting: e.g. `end` is highlighted when it is MATLAB code but the corresponding `for` is not highlighted
- despite being in `doc/manual/utils/dynare_lex.py`, `var` is not highlighted in code blocks
- ideally find a programmatic way to fill `doc/manual/utils/dynare_lex.py` from the `rst` files instead of having to update it by hand4.7https://git.dynare.org/Dynare/dynare/-/issues/1688fix `basic_plan`, `flip_plan` in doc2020-01-28T14:01:41ZHoutan Bastanifix `basic_plan`, `flip_plan` in docThe calling structure for both of these functions runs off the end of the pageThe calling structure for both of these functions runs off the end of the page4.7Stéphane Adjemianstepan@adjemian.euStéphane Adjemianstepan@adjemian.euhttps://git.dynare.org/Dynare/dynare/-/issues/1684New WITH_TREND() operator for epilogue block2019-12-13T17:29:05ZSébastien VillemotNew WITH_TREND() operator for epilogue blockImplement a new operator (something like `WITH_TREND()` that could be used in the epilogue block, and that would add back the trend on the computed variables. In this way, the symbolic manipulations would be done in the preprocessor, which is much more easier to do than in MATLAB.
Initially discussed in #1648Implement a new operator (something like `WITH_TREND()` that could be used in the epilogue block, and that would add back the trend on the computed variables. In this way, the symbolic manipulations would be done in the preprocessor, which is much more easier to do than in MATLAB.
Initially discussed in #16484.7https://git.dynare.org/Dynare/dynare/-/issues/1680Adapt evaluate_planner_objective.m to higher order2019-12-12T10:30:23ZJohannes Pfeifer Adapt evaluate_planner_objective.m to higher orderSupersedes https://git.dynare.org/Dynare/dynare/issues/564 and is related to https://git.dynare.org/Dynare/dynare/issues/1678Supersedes https://git.dynare.org/Dynare/dynare/issues/564 and is related to https://git.dynare.org/Dynare/dynare/issues/16784.7https://git.dynare.org/Dynare/dynare/-/issues/1675Migrate to Dragonfly parallel toolkit2020-02-12T10:23:03ZSébastien VillemotMigrate to Dragonfly parallel toolkitThe embedded parallel toolkit should be replaced by the [Dragonfly](https://github.com/DragonflyTeam/dragonfly) toolkit.
A branch called `dragonfly` has been created, with the toolkit under `matlab/modules/dragonfly` (see ee3971ad6364910851ff06da9729d9b4a084ca4b).The embedded parallel toolkit should be replaced by the [Dragonfly](https://github.com/DragonflyTeam/dragonfly) toolkit.
A branch called `dragonfly` has been created, with the toolkit under `matlab/modules/dragonfly` (see ee3971ad6364910851ff06da9729d9b4a084ca4b).4.7Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/-/issues/1674Modification to shock_decomposition and plot_decomposition interfaces to incl...2020-01-21T17:47:09ZDóra Kocsisdora@dynare.orgModification to shock_decomposition and plot_decomposition interfaces to include flexibility with datesInclude the following options for dates when shock_decomposition is triggered:
* date of the start of the decomposition, default: start of the estimation sample (some people may want to start the decomposition with the smoothed value at the end of the sample). Proposed option name: `init_date = INTEGER`.
* date of the start of the graph, default: start of the estimation sample. Proposed option name: `graph_init_date = INTEGER`.Include the following options for dates when shock_decomposition is triggered:
* date of the start of the decomposition, default: start of the estimation sample (some people may want to start the decomposition with the smoothed value at the end of the sample). Proposed option name: `init_date = INTEGER`.
* date of the start of the graph, default: start of the estimation sample. Proposed option name: `graph_init_date = INTEGER`.4.7Dóra Kocsisdora@dynare.orgDóra Kocsisdora@dynare.orghttps://git.dynare.org/Dynare/dynare/-/issues/1671Refactoring initval_file and histval_file2020-01-29T10:48:45ZMichelJuillardRefactoring initval_file and histval_file``initval_file`` and ``histvfal_file`` should be more flexible and have functionalities similar to option ``datafile``in ``estimation``
# Usage
## initval_file
- is used only in perfect foresight
- provides a guess value for computing the solution
- in absence of ``histval`` or ``histval_file`` provides the initial conditions for the simulation (we don't want to require users to upload two files for initial conditions and guess values
## histval_file
- is used for stochastic and perfect foresight models
- provides initial conditions
# Current implementation
## initval_file
- accept ``.m``, ``.mat``, ``.xls(x)``, ``.csv`` files
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
## histval_file
- accept only files prepared by ``smoother2histval`` that uses odd format
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
- note that ``histval`` sets a ``dseries`` object
# Proposal
1. have the same interface for ``initval_file`` and ``histval_file``
2. add an option ``dseries`` to be able to pass a ``dseries`` object
3. handle references to ``varexo_det`` variables
4. use the ``dseries`` interface to read files
5. ``initval_file`` and ``histval_file```set ``dseries`` object
6. modify ``make_y_`` ``make_ex_`` accordingly
7. handle auxiliary variables in a consistent manner (see #1004)
# Breaking changes
1. Documented behavior of ``initval_file``: None
2. Documented behavior of ``histval_file``: None
3. Documented behavior of ``perfect_foresight_setup``: the format of files designated in option ``filename`` has changed
4. Function ``initvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``initvalf()`` sets a result ``dseries`` on output
5. Function ``histvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``histvalf()`` sets a result ``dseries`` on output
* ``histvalf.m``doesn't recognize the old ``smoother2histval`` file format anymore.
6. Function ``smoother2histval.m``
* when an output file is requested ``smoother2histval()`` creates a ``.mat`` file containing a single ``dseries``
# Questions
1. Is it a problem to have an optional ``dseries`` input in an instruction called ``initval_file`` and no file strictly speaking of?
WORK IN PROGRESS``initval_file`` and ``histvfal_file`` should be more flexible and have functionalities similar to option ``datafile``in ``estimation``
# Usage
## initval_file
- is used only in perfect foresight
- provides a guess value for computing the solution
- in absence of ``histval`` or ``histval_file`` provides the initial conditions for the simulation (we don't want to require users to upload two files for initial conditions and guess values
## histval_file
- is used for stochastic and perfect foresight models
- provides initial conditions
# Current implementation
## initval_file
- accept ``.m``, ``.mat``, ``.xls(x)``, ``.csv`` files
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
## histval_file
- accept only files prepared by ``smoother2histval`` that uses odd format
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
- note that ``histval`` sets a ``dseries`` object
# Proposal
1. have the same interface for ``initval_file`` and ``histval_file``
2. add an option ``dseries`` to be able to pass a ``dseries`` object
3. handle references to ``varexo_det`` variables
4. use the ``dseries`` interface to read files
5. ``initval_file`` and ``histval_file```set ``dseries`` object
6. modify ``make_y_`` ``make_ex_`` accordingly
7. handle auxiliary variables in a consistent manner (see #1004)
# Breaking changes
1. Documented behavior of ``initval_file``: None
2. Documented behavior of ``histval_file``: None
3. Documented behavior of ``perfect_foresight_setup``: the format of files designated in option ``filename`` has changed
4. Function ``initvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``initvalf()`` sets a result ``dseries`` on output
5. Function ``histvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``histvalf()`` sets a result ``dseries`` on output
* ``histvalf.m``doesn't recognize the old ``smoother2histval`` file format anymore.
6. Function ``smoother2histval.m``
* when an output file is requested ``smoother2histval()`` creates a ``.mat`` file containing a single ``dseries``
# Questions
1. Is it a problem to have an optional ``dseries`` input in an instruction called ``initval_file`` and no file strictly speaking of?
WORK IN PROGRESS4.7MichelJuillardMichelJuillardhttps://git.dynare.org/Dynare/dynare/-/issues/1670Mac Package: see if we can use Homebrew `binutils` package to provide `as` an...2019-11-28T14:46:11ZHoutan BastaniMac Package: see if we can use Homebrew `binutils` package to provide `as` and `ld` and thus not install CLTThe idea is that if we can use Homebrew to provide `as` and `ld` then we can avoid installing Command Line Tools. I am not hopeful because `install_name_tool`, which is used to rename library paths when installing to a non-default directory is installed via Command Line Tools. The thing to check is whether this is used only when installing to non-default directories or whether it is always used. If it is always used, then CLT must not be required (though, it is located in `/Library/Developer/CommandLineTools/usr/bin/install_name_tool`)The idea is that if we can use Homebrew to provide `as` and `ld` then we can avoid installing Command Line Tools. I am not hopeful because `install_name_tool`, which is used to rename library paths when installing to a non-default directory is installed via Command Line Tools. The thing to check is whether this is used only when installing to non-default directories or whether it is always used. If it is always used, then CLT must not be required (though, it is located in `/Library/Developer/CommandLineTools/usr/bin/install_name_tool`)4.7Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/-/issues/1665Implement bridge sampler for computing marginal data density2019-11-26T16:25:08ZJohannes Pfeifer Implement bridge sampler for computing marginal data density4.7https://git.dynare.org/Dynare/dynare/-/issues/1663datafile option in perfect_foresight_setup: incomplete documentation and not ...2020-01-02T18:21:21ZMichelJuillarddatafile option in perfect_foresight_setup: incomplete documentation and not flexible enoughthe datafile option in perfect_foresight solver requires a text file without variable names, variables being order in order of VAR statement. The file name must end with ``_endo.dat``.
- [x] make loading data for guess value more flexible
- [x] honor INITVAL_FILE command or deprecate it
- [ ] make possible to use guess values with simul_backward-modelthe datafile option in perfect_foresight solver requires a text file without variable names, variables being order in order of VAR statement. The file name must end with ``_endo.dat``.
- [x] make loading data for guess value more flexible
- [x] honor INITVAL_FILE command or deprecate it
- [ ] make possible to use guess values with simul_backward-model4.7