dynare issueshttps://git.dynare.org/Dynare/dynare/issues2020-02-21T10:13:22Zhttps://git.dynare.org/Dynare/dynare/issues/1709Automatic detrending2020-02-21T10:13:22ZFerhatMihoubiAutomatic detrendingFor the moment the deterministic or stochastic trends have to be declared using the trend_var (or log_trend_var) commands and the option deflator (or log_deflator) in var command to declare the trended variables.
The purpose of the new feature is to find and remove automatically the trends. To do so, the proposal is to add :
- an option in var command to indicate if the variable is expressed in logarithm or not (the default being not in logarithm). For instance *var(log) list of variables*.
- a command that will add to all endogenous variables a growth factor (additive or multiplicative depending on the previously declared status) and the equations that put in relation all the growth factors to theirs endogenous variables. This step should be performed in the preprocessor to produce a new dynamic model containing all the endogenous variables and the growth factor associated to all the endogenous variables.
- during the execution the complete new dynamic model should be solved to compute the growth factors and the steady state values of the endogenous variables.
This command could be *detrend* for instance.For the moment the deterministic or stochastic trends have to be declared using the trend_var (or log_trend_var) commands and the option deflator (or log_deflator) in var command to declare the trended variables.
The purpose of the new feature is to find and remove automatically the trends. To do so, the proposal is to add :
- an option in var command to indicate if the variable is expressed in logarithm or not (the default being not in logarithm). For instance *var(log) list of variables*.
- a command that will add to all endogenous variables a growth factor (additive or multiplicative depending on the previously declared status) and the equations that put in relation all the growth factors to theirs endogenous variables. This step should be performed in the preprocessor to produce a new dynamic model containing all the endogenous variables and the growth factor associated to all the endogenous variables.
- during the execution the complete new dynamic model should be solved to compute the growth factors and the steady state values of the endogenous variables.
This command could be *detrend* for instance.https://git.dynare.org/Dynare/dynare/issues/1706Fix wrong computation in third-order approximation in pruned_state_space.m2020-02-17T14:50:00ZJohannes Pfeifer Fix wrong computation in third-order approximation in pruned_state_space.mMattermost discussion 06/02/20Mattermost discussion 06/02/204.6Willi Mutschlerwilli@mutschler.euWilli Mutschlerwilli@mutschler.euhttps://git.dynare.org/Dynare/dynare/issues/1707Macro commands that take no arguments have spurious parentheses appended to t...2020-02-17T14:50:00ZSébastien VillemotMacro commands that take no arguments have spurious parentheses appended to their synopsisSee for example the definition of `@#else`, `@#endif`, `@#endfor`, `@#echomacrovars` in the reference manual.See for example the definition of `@#else`, `@#endif`, `@#endfor`, `@#echomacrovars` in the reference manual.4.7Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/issues/1660Update documentation of dseries2020-02-17T09:27:42ZSébastien VillemotUpdate documentation of dseriesSince dseries have been substantially rewritten since 4.5, the documentation in the reference needs to be updated.Since dseries have been substantially rewritten since 4.5, the documentation in the reference needs to be updated.4.7Stéphane Adjemianstepan@adjemian.euStéphane Adjemianstepan@adjemian.euhttps://git.dynare.org/Dynare/dynare/issues/1698Allow checking linearity for perfect foresight models2020-02-14T15:05:47ZJohannes Pfeifer Allow checking linearity for perfect foresight modelsThe linearity check underlying `model(linear)` is based on the Hessian of the model. But we don’t compute the Hessian for a perfect foresight simulation, hence the check is skipped. See line 799 of `preprocessor/src/ModFile.cc`.
That can be problematic in case of nonlinearities like a ZLB constraint in an otherwise linear model. The solver assumes that the Jacobian is the same at every period, and the constraint is not enforced.
For very large perfect foresight models, we actually don’t want to put the Hessian in the generated dynamic file, because compiling it can be very time consuming. Hence the fix is not straightforward.
I would suggest to add a way of testing this. One option would be trigger the Hessian computation with a `debug` option or something like this. For example, we could use the already present command line option `debug`.The linearity check underlying `model(linear)` is based on the Hessian of the model. But we don’t compute the Hessian for a perfect foresight simulation, hence the check is skipped. See line 799 of `preprocessor/src/ModFile.cc`.
That can be problematic in case of nonlinearities like a ZLB constraint in an otherwise linear model. The solver assumes that the Jacobian is the same at every period, and the constraint is not enforced.
For very large perfect foresight models, we actually don’t want to put the Hessian in the generated dynamic file, because compiling it can be very time consuming. Hence the fix is not straightforward.
I would suggest to add a way of testing this. One option would be trigger the Hessian computation with a `debug` option or something like this. For example, we could use the already present command line option `debug`.4.7https://git.dynare.org/Dynare/dynare/issues/1173Support estimation under optimal policy2020-02-12T16:18:04ZJohannes Pfeifer Support estimation under optimal policySee http://www.dynare.org/phpBB3/viewtopic.php?f=1&t=8071
See http://www.dynare.org/phpBB3/viewtopic.php?f=1&t=8071
4.7Sébastien VillemotSébastien Villemothttps://git.dynare.org/Dynare/dynare/issues/1675Migrate to Dragonfly parallel toolkit2020-02-12T10:23:03ZSébastien VillemotMigrate to Dragonfly parallel toolkitThe embedded parallel toolkit should be replaced by the [Dragonfly](https://github.com/DragonflyTeam/dragonfly) toolkit.
A branch called `dragonfly` has been created, with the toolkit under `matlab/modules/dragonfly` (see ee3971ad6364910851ff06da9729d9b4a084ca4b).The embedded parallel toolkit should be replaced by the [Dragonfly](https://github.com/DragonflyTeam/dragonfly) toolkit.
A branch called `dragonfly` has been created, with the toolkit under `matlab/modules/dragonfly` (see ee3971ad6364910851ff06da9729d9b4a084ca4b).4.7Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/issues/569Integrate OccBin2020-02-11T17:11:43ZHoutan BastaniIntegrate OccBinhttps://github.com/lucashare/occbin
https://www2.bc.edu/matteo-iacoviello/research_files/TOOLKIT_PAPER.pdf
https://github.com/lucashare/occbin
https://www2.bc.edu/matteo-iacoviello/research_files/TOOLKIT_PAPER.pdf
4.7Marco RattoMarco Rattohttps://git.dynare.org/Dynare/dynare/issues/1703det_cond_forecast() depends on oo_.dr.state_var but this is not always available2020-02-11T10:08:29ZMichelJuillarddet_cond_forecast() depends on oo_.dr.state_var but this is not always available``det_cond_forecast()`` depends on ``oo_.dr.state_var`` but this variable is computed by ``dyn_first_order_solver``. So it isn't available for a purely backward or purely forward model``det_cond_forecast()`` depends on ``oo_.dr.state_var`` but this variable is computed by ``dyn_first_order_solver``. So it isn't available for a purely backward or purely forward model4.7MichelJuillardMichelJuillardhttps://git.dynare.org/Dynare/dynare/issues/1702det_cond_forecast() argument checking is broken2020-02-11T09:44:04ZMichelJuillarddet_cond_forecast() argument checking is brokenIf one calls det_cond_forecast with two arguments only as in the manual example, the function fails with error
```
impossible case
```
that is triggered near line 208If one calls det_cond_forecast with two arguments only as in the manual example, the function fails with error
```
impossible case
```
that is triggered near line 2084.7MichelJuillardMichelJuillardhttps://git.dynare.org/Dynare/dynare/issues/1704Update partial information code2020-02-04T15:51:15ZSébastien VillemotUpdate partial information codeFirst version located in https://git.dynare.org/Yang/dynareFirst version located in https://git.dynare.org/Yang/dynare4.7Sébastien VillemotSébastien Villemothttps://git.dynare.org/Dynare/dynare/issues/1700Proposal for a generalized solver2020-01-30T09:53:09ZMichelJuillardProposal for a generalized solverModel inversion and deterministic conditional forecast solve the same mathematical problem: solving a (non-)linear system where the identity of the unknown variables change from period to period. I suggest to create a unique function that would solve such problems.
``general_solver`` would take at least the following arguments:
* y: endogenous variables (matrix)
* x: exogenous variables (matrix) [possible concatenation of exo_det variables to the right]
* model: model (function handle)
* i_endo_flip: flip variables indices in endogenous variables (vector)
* i_exo_flip: flip variables indices in exogenous variables (vector)
### Conventions
1. there are as many endogenous flip variables as exogenenous or exo det variables
1. the endogenous and exogenous flip variables are in the same order (not necessary to be contiguous but may help)
1. given the indices vector of flip variablesm implicit pairs of endogenous/exogenous variables are formed
1. If the observation of an endogenous variable is a NaN, the endogenous variables plays it usual role of endogenous variable
1. If the observation of an endogenous variables is a valid number, the endogenous variable is treated as exogenous and the corresponding flip exogenous variables is treated as endogenous
1. The presence of a NaN for an endogenous variable not declared as a flip variable is an error
1. The corner case of unconditional forecast should be accepted for ease of use
1. The solution is written back in ``y`` and ``x`` irrespective of the actual role of a variable in a given period
### Operations
``general_solver`` dispatches la résolution du nouveau problème selon
1. purely backward linear model
1. purely backward nonlinear model
1. purely forward linear model (if we have the algorithm...)
1. purely forward nonlinear model
1. general linear model
1. general nonlinear model
1. bytecode and or block model
and according to the options selecting a particular algorithmModel inversion and deterministic conditional forecast solve the same mathematical problem: solving a (non-)linear system where the identity of the unknown variables change from period to period. I suggest to create a unique function that would solve such problems.
``general_solver`` would take at least the following arguments:
* y: endogenous variables (matrix)
* x: exogenous variables (matrix) [possible concatenation of exo_det variables to the right]
* model: model (function handle)
* i_endo_flip: flip variables indices in endogenous variables (vector)
* i_exo_flip: flip variables indices in exogenous variables (vector)
### Conventions
1. there are as many endogenous flip variables as exogenenous or exo det variables
1. the endogenous and exogenous flip variables are in the same order (not necessary to be contiguous but may help)
1. given the indices vector of flip variablesm implicit pairs of endogenous/exogenous variables are formed
1. If the observation of an endogenous variable is a NaN, the endogenous variables plays it usual role of endogenous variable
1. If the observation of an endogenous variables is a valid number, the endogenous variable is treated as exogenous and the corresponding flip exogenous variables is treated as endogenous
1. The presence of a NaN for an endogenous variable not declared as a flip variable is an error
1. The corner case of unconditional forecast should be accepted for ease of use
1. The solution is written back in ``y`` and ``x`` irrespective of the actual role of a variable in a given period
### Operations
``general_solver`` dispatches la résolution du nouveau problème selon
1. purely backward linear model
1. purely backward nonlinear model
1. purely forward linear model (if we have the algorithm...)
1. purely forward nonlinear model
1. general linear model
1. general nonlinear model
1. bytecode and or block model
and according to the options selecting a particular algorithm4.7https://git.dynare.org/Dynare/dynare/issues/1671Refactoring initval_file and histval_file2020-01-29T10:48:45ZMichelJuillardRefactoring initval_file and histval_file``initval_file`` and ``histvfal_file`` should be more flexible and have functionalities similar to option ``datafile``in ``estimation``
# Usage
## initval_file
- is used only in perfect foresight
- provides a guess value for computing the solution
- in absence of ``histval`` or ``histval_file`` provides the initial conditions for the simulation (we don't want to require users to upload two files for initial conditions and guess values
## histval_file
- is used for stochastic and perfect foresight models
- provides initial conditions
# Current implementation
## initval_file
- accept ``.m``, ``.mat``, ``.xls(x)``, ``.csv`` files
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
## histval_file
- accept only files prepared by ``smoother2histval`` that uses odd format
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
- note that ``histval`` sets a ``dseries`` object
# Proposal
1. have the same interface for ``initval_file`` and ``histval_file``
2. add an option ``dseries`` to be able to pass a ``dseries`` object
3. handle references to ``varexo_det`` variables
4. use the ``dseries`` interface to read files
5. ``initval_file`` and ``histval_file```set ``dseries`` object
6. modify ``make_y_`` ``make_ex_`` accordingly
7. handle auxiliary variables in a consistent manner (see #1004)
# Breaking changes
1. Documented behavior of ``initval_file``: None
2. Documented behavior of ``histval_file``: None
3. Documented behavior of ``perfect_foresight_setup``: the format of files designated in option ``filename`` has changed
4. Function ``initvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``initvalf()`` sets a result ``dseries`` on output
5. Function ``histvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``histvalf()`` sets a result ``dseries`` on output
* ``histvalf.m``doesn't recognize the old ``smoother2histval`` file format anymore.
6. Function ``smoother2histval.m``
* when an output file is requested ``smoother2histval()`` creates a ``.mat`` file containing a single ``dseries``
# Questions
1. Is it a problem to have an optional ``dseries`` input in an instruction called ``initval_file`` and no file strictly speaking of?
WORK IN PROGRESS``initval_file`` and ``histvfal_file`` should be more flexible and have functionalities similar to option ``datafile``in ``estimation``
# Usage
## initval_file
- is used only in perfect foresight
- provides a guess value for computing the solution
- in absence of ``histval`` or ``histval_file`` provides the initial conditions for the simulation (we don't want to require users to upload two files for initial conditions and guess values
## histval_file
- is used for stochastic and perfect foresight models
- provides initial conditions
# Current implementation
## initval_file
- accept ``.m``, ``.mat``, ``.xls(x)``, ``.csv`` files
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
## histval_file
- accept only files prepared by ``smoother2histval`` that uses odd format
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
- note that ``histval`` sets a ``dseries`` object
# Proposal
1. have the same interface for ``initval_file`` and ``histval_file``
2. add an option ``dseries`` to be able to pass a ``dseries`` object
3. handle references to ``varexo_det`` variables
4. use the ``dseries`` interface to read files
5. ``initval_file`` and ``histval_file```set ``dseries`` object
6. modify ``make_y_`` ``make_ex_`` accordingly
7. handle auxiliary variables in a consistent manner (see #1004)
# Breaking changes
1. Documented behavior of ``initval_file``: None
2. Documented behavior of ``histval_file``: None
3. Documented behavior of ``perfect_foresight_setup``: the format of files designated in option ``filename`` has changed
4. Function ``initvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``initvalf()`` sets a result ``dseries`` on output
5. Function ``histvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``histvalf()`` sets a result ``dseries`` on output
* ``histvalf.m``doesn't recognize the old ``smoother2histval`` file format anymore.
6. Function ``smoother2histval.m``
* when an output file is requested ``smoother2histval()`` creates a ``.mat`` file containing a single ``dseries``
# Questions
1. Is it a problem to have an optional ``dseries`` input in an instruction called ``initval_file`` and no file strictly speaking of?
WORK IN PROGRESS4.7MichelJuillardMichelJuillardhttps://git.dynare.org/Dynare/dynare/issues/1688fix `basic_plan`, `flip_plan` in doc2020-01-28T14:01:41ZHoutan Bastanifix `basic_plan`, `flip_plan` in docThe calling structure for both of these functions runs off the end of the pageThe calling structure for both of these functions runs off the end of the page4.7Stéphane Adjemianstepan@adjemian.euStéphane Adjemianstepan@adjemian.euhttps://git.dynare.org/Dynare/dynare/issues/444Document nonlinear filters on a wiki page2020-01-24T16:06:31ZStéphane Adjemianstepan@adjemian.euDocument nonlinear filters on a wiki pageWe need to describe and discuss all the available options in `options_.particle` (there is still no interface for these options because the set of options is not yet stable).
We need to describe and discuss all the available options in `options_.particle` (there is still no interface for these options because the set of options is not yet stable).
Stéphane Adjemianstepan@adjemian.euStéphane Adjemianstepan@adjemian.euhttps://git.dynare.org/Dynare/dynare/issues/1674Modification to shock_decomposition and plot_decomposition interfaces to incl...2020-01-21T17:47:09ZDóra Kocsisdora@dynare.orgModification to shock_decomposition and plot_decomposition interfaces to include flexibility with datesInclude the following options for dates when shock_decomposition is triggered:
* date of the start of the decomposition, default: start of the estimation sample (some people may want to start the decomposition with the smoothed value at the end of the sample). Proposed option name: `init_date = INTEGER`.
* date of the start of the graph, default: start of the estimation sample. Proposed option name: `graph_init_date = INTEGER`.Include the following options for dates when shock_decomposition is triggered:
* date of the start of the decomposition, default: start of the estimation sample (some people may want to start the decomposition with the smoothed value at the end of the sample). Proposed option name: `init_date = INTEGER`.
* date of the start of the graph, default: start of the estimation sample. Proposed option name: `graph_init_date = INTEGER`.4.7Dóra Kocsisdora@dynare.orgDóra Kocsisdora@dynare.orghttps://git.dynare.org/Dynare/dynare/issues/1693Various improvements in Sphinx doc2020-01-08T16:24:40ZHoutan BastaniVarious improvements in Sphinx doc- see if we can use https://pypi.org/project/sphinxcontrib-matlabdomain/ instead of `MatComm` and `MatlabVar` defined in `doc/manual/utils/dynare_dom.py`
- if not, add new domain entry to differentiate between MATLAB commands and MATLAB functions
- find way to fix output of Matlab Commands so the options conform to the true type used for Dynare Command options
- code block:
- many general problems with highlighting: e.g. `end` is highlighted when it is MATLAB code but the corresponding `for` is not highlighted
- despite being in `doc/manual/utils/dynare_lex.py`, `var` is not highlighted in code blocks
- ideally find a programmatic way to fill `doc/manual/utils/dynare_lex.py` from the `rst` files instead of having to update it by hand- see if we can use https://pypi.org/project/sphinxcontrib-matlabdomain/ instead of `MatComm` and `MatlabVar` defined in `doc/manual/utils/dynare_dom.py`
- if not, add new domain entry to differentiate between MATLAB commands and MATLAB functions
- find way to fix output of Matlab Commands so the options conform to the true type used for Dynare Command options
- code block:
- many general problems with highlighting: e.g. `end` is highlighted when it is MATLAB code but the corresponding `for` is not highlighted
- despite being in `doc/manual/utils/dynare_lex.py`, `var` is not highlighted in code blocks
- ideally find a programmatic way to fill `doc/manual/utils/dynare_lex.py` from the `rst` files instead of having to update it by hand4.7https://git.dynare.org/Dynare/dynare/issues/1663datafile option in perfect_foresight_setup: incomplete documentation and not ...2020-01-02T18:21:21ZMichelJuillarddatafile option in perfect_foresight_setup: incomplete documentation and not flexible enoughthe datafile option in perfect_foresight solver requires a text file without variable names, variables being order in order of VAR statement. The file name must end with ``_endo.dat``.
- [x] make loading data for guess value more flexible
- [x] honor INITVAL_FILE command or deprecate it
- [ ] make possible to use guess values with simul_backward-modelthe datafile option in perfect_foresight solver requires a text file without variable names, variables being order in order of VAR statement. The file name must end with ``_endo.dat``.
- [x] make loading data for guess value more flexible
- [x] honor INITVAL_FILE command or deprecate it
- [ ] make possible to use guess values with simul_backward-model4.7https://git.dynare.org/Dynare/dynare/issues/1657add shock decomposition for forecasts done after estimation2019-12-20T16:32:54ZSébastien Villemotadd shock decomposition for forecasts done after estimation@MichelJuillard has some preliminary codes.@MichelJuillard has some preliminary codes.4.7Dóra Kocsisdora@dynare.orgDóra Kocsisdora@dynare.orghttps://git.dynare.org/Dynare/dynare/issues/1513Allow selecting proper training sample for endogenous_prior2019-12-20T16:32:01ZJohannes Pfeifer Allow selecting proper training sample for endogenous_priorCurrently, we simply use `Y=data';`, but it is straightforward to include different dataCurrently, we simply use `Y=data';`, but it is straightforward to include different data4.7