dynare issueshttps://git.dynare.org/Dynare/dynare/-/issues2020-05-14T14:46:58Zhttps://git.dynare.org/Dynare/dynare/-/issues/1671Refactoring initval_file and histval_file2020-05-14T14:46:58ZMichelJuillardRefactoring initval_file and histval_file``initval_file`` and ``histvfal_file`` should be more flexible and have functionalities similar to option ``datafile``in ``estimation``
# Usage
## initval_file
- is used only in perfect foresight
- provides a guess value for computing the solution
- in absence of ``histval`` or ``histval_file`` provides the initial conditions for the simulation (we don't want to require users to upload two files for initial conditions and guess values
## histval_file
- is used for stochastic and perfect foresight models
- provides initial conditions
# Current implementation
## initval_file
- accept ``.m``, ``.mat``, ``.xls(x)``, ``.csv`` files
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
## histval_file
- accept only files prepared by ``smoother2histval`` that uses odd format
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
- note that ``histval`` sets a ``dseries`` object
# Proposal
1. have the same interface for ``initval_file`` and ``histval_file``
2. add an option ``dseries`` to be able to pass a ``dseries`` object
3. handle references to ``varexo_det`` variables
4. use the ``dseries`` interface to read files
5. ``initval_file`` and ``histval_file```set ``dseries`` object
6. modify ``make_y_`` ``make_ex_`` accordingly
7. handle auxiliary variables in a consistent manner (see #1004)
# Breaking changes
1. Documented behavior of ``initval_file``: None
2. Documented behavior of ``histval_file``: None
3. Documented behavior of ``perfect_foresight_setup``: the format of files designated in option ``filename`` has changed
4. Function ``initvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``initvalf()`` sets a result ``dseries`` on output
5. Function ``histvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``histvalf()`` sets a result ``dseries`` on output
* ``histvalf.m``doesn't recognize the old ``smoother2histval`` file format anymore.
6. Function ``smoother2histval.m``
* when an output file is requested ``smoother2histval()`` creates a ``.mat`` file containing a single ``dseries``
* ``smoother2histval.m`` saves now M_.orig_maximum_lag`` observations instead of ``oo_.maximum_lag`` in order to be able to recompute auxiliary variables
* It will not be possible to use a file saved with an previous version of ``smoother2histval``
# Questions
1. Is it a problem to have an optional ``dseries`` input in an instruction called ``initval_file`` and no file strictly speaking of?
WORK IN PROGRESS``initval_file`` and ``histvfal_file`` should be more flexible and have functionalities similar to option ``datafile``in ``estimation``
# Usage
## initval_file
- is used only in perfect foresight
- provides a guess value for computing the solution
- in absence of ``histval`` or ``histval_file`` provides the initial conditions for the simulation (we don't want to require users to upload two files for initial conditions and guess values
## histval_file
- is used for stochastic and perfect foresight models
- provides initial conditions
# Current implementation
## initval_file
- accept ``.m``, ``.mat``, ``.xls(x)``, ``.csv`` files
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
## histval_file
- accept only files prepared by ``smoother2histval`` that uses odd format
- modifies directly ``oo_.endo_simul``, ``oo_.exo_simul``, but not ``oo_.exo_det_simul``
- note that ``histval`` sets a ``dseries`` object
# Proposal
1. have the same interface for ``initval_file`` and ``histval_file``
2. add an option ``dseries`` to be able to pass a ``dseries`` object
3. handle references to ``varexo_det`` variables
4. use the ``dseries`` interface to read files
5. ``initval_file`` and ``histval_file```set ``dseries`` object
6. modify ``make_y_`` ``make_ex_`` accordingly
7. handle auxiliary variables in a consistent manner (see #1004)
# Breaking changes
1. Documented behavior of ``initval_file``: None
2. Documented behavior of ``histval_file``: None
3. Documented behavior of ``perfect_foresight_setup``: the format of files designated in option ``filename`` has changed
4. Function ``initvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``initvalf()`` sets a result ``dseries`` on output
5. Function ``histvalf.m``
* ``M_`` and ``options`` are added to inputs
* the old ``filename`` input triggers an error
* function ``histvalf()`` sets a result ``dseries`` on output
* ``histvalf.m``doesn't recognize the old ``smoother2histval`` file format anymore.
6. Function ``smoother2histval.m``
* when an output file is requested ``smoother2histval()`` creates a ``.mat`` file containing a single ``dseries``
* ``smoother2histval.m`` saves now M_.orig_maximum_lag`` observations instead of ``oo_.maximum_lag`` in order to be able to recompute auxiliary variables
* It will not be possible to use a file saved with an previous version of ``smoother2histval``
# Questions
1. Is it a problem to have an optional ``dseries`` input in an instruction called ``initval_file`` and no file strictly speaking of?
WORK IN PROGRESS4.7MichelJuillardMichelJuillardhttps://git.dynare.org/Dynare/dynare/-/issues/1670Mac Package: see if we can use Homebrew `binutils` package to provide `as` an...2020-10-20T13:02:00ZHoutan BastaniMac Package: see if we can use Homebrew `binutils` package to provide `as` and `ld` and thus not install CLTThe idea is that if we can use Homebrew to provide `as` and `ld` then we can avoid installing Command Line Tools. I am not hopeful because `install_name_tool`, which is used to rename library paths when installing to a non-default directory is installed via Command Line Tools. The thing to check is whether this is used only when installing to non-default directories or whether it is always used. If it is always used, then CLT must not be required (though, it is located in `/Library/Developer/CommandLineTools/usr/bin/install_name_tool`)The idea is that if we can use Homebrew to provide `as` and `ld` then we can avoid installing Command Line Tools. I am not hopeful because `install_name_tool`, which is used to rename library paths when installing to a non-default directory is installed via Command Line Tools. The thing to check is whether this is used only when installing to non-default directories or whether it is always used. If it is always used, then CLT must not be required (though, it is located in `/Library/Developer/CommandLineTools/usr/bin/install_name_tool`)4.7https://git.dynare.org/Dynare/dynare/-/issues/1669Restore index of functions and commands in the manual2019-12-03T14:45:18ZSébastien VillemotRestore index of functions and commands in the manualIn the texinfo manual, there used to be an index for functions and commands (see https://www.dynare.org/manual/Command-and-Function-Index.html#Command-and-Function-Index).
This has been lost in the conversion to sphinx, and should therefore be restored.
By the way, if this is technically feasible, and index of variables (fields of `M_`, `oo_`, …) would also be useful.In the texinfo manual, there used to be an index for functions and commands (see https://www.dynare.org/manual/Command-and-Function-Index.html#Command-and-Function-Index).
This has been lost in the conversion to sphinx, and should therefore be restored.
By the way, if this is technically feasible, and index of variables (fields of `M_`, `oo_`, …) would also be useful.4.6Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/-/issues/1668Tighten tolerance for first iteration in solve1()2019-12-12T17:59:24ZMichelJuillardTighten tolerance for first iteration in solve1()Currently solve1() uses the same tolerance level (``tolf``) to decide to start Newton iterations or to terminate them. I suggest the the tolerance above which to start Newton iterations should be tighter.
It is mainly useful for the simulation of purely backward models. When the model is stationary, we expect the trajectory to converge asymptotically towards the steady state. This occurs as long as at least one iteration Newton takes place in each period. If the tolerance criteria is too lax, the simulation remains constant from too early a point.
**I suggest to use the square of ``tolf`` at the start of the computation**
For reference, csolve() always performs one Newton iteration. Matlab's fsolve() and lmmcp use a single tolerance level.Currently solve1() uses the same tolerance level (``tolf``) to decide to start Newton iterations or to terminate them. I suggest the the tolerance above which to start Newton iterations should be tighter.
It is mainly useful for the simulation of purely backward models. When the model is stationary, we expect the trajectory to converge asymptotically towards the steady state. This occurs as long as at least one iteration Newton takes place in each period. If the tolerance criteria is too lax, the simulation remains constant from too early a point.
**I suggest to use the square of ``tolf`` at the start of the computation**
For reference, csolve() always performs one Newton iteration. Matlab's fsolve() and lmmcp use a single tolerance level.https://git.dynare.org/Dynare/dynare/-/issues/1667Problem with command line options passed on the first line of a `.mod` file w...2020-01-23T17:58:43ZHoutan BastaniProblem with command line options passed on the first line of a `.mod` file when they are used in dynare.mWhen command line options are passed on the first line of a `.mod` file within `// --+ options: +--`, they are parsed by the preprocessor and ignored by `dynare.m`.
This ignores the options `nopathchange`, `nopreprocessoroutput`, `onlymacro`, `onlyjson`, and `nolog`. An attempt to fix this was made with a regex for `nolog` but it is buggy (e.g. a command line argument such as `savemacro=,nolog,` would result in no Dynare log being created).When command line options are passed on the first line of a `.mod` file within `// --+ options: +--`, they are parsed by the preprocessor and ignored by `dynare.m`.
This ignores the options `nopathchange`, `nopreprocessoroutput`, `onlymacro`, `onlyjson`, and `nolog`. An attempt to fix this was made with a regex for `nolog` but it is buggy (e.g. a command line argument such as `savemacro=,nolog,` would result in no Dynare log being created).4.6Sébastien VillemotSébastien Villemothttps://git.dynare.org/Dynare/dynare/-/issues/1666option for tolerance value in Modified Harmonic Mean estimator2019-12-12T11:58:40ZMarco Rattooption for tolerance value in Modified Harmonic Mean estimatorcurrently there is an hard-coded threshold set to 0.01 for the relative difference between the marginal likelihood computed withing the 1st and 9th deciles.
It might be helpful to allow modifying it with an explicit option, something like:
`options_.Modified_Harmonic_Mean_Tolerance`
with default 0.01currently there is an hard-coded threshold set to 0.01 for the relative difference between the marginal likelihood computed withing the 1st and 9th deciles.
It might be helpful to allow modifying it with an explicit option, something like:
`options_.Modified_Harmonic_Mean_Tolerance`
with default 0.014.6Johannes Pfeifer Johannes Pfeifer https://git.dynare.org/Dynare/dynare/-/issues/1665Implement bridge sampler for computing marginal data density2019-11-26T16:25:08ZJohannes Pfeifer Implement bridge sampler for computing marginal data density4.7https://git.dynare.org/Dynare/dynare/-/issues/1664Implement option to use LMMCP for steady state computation2019-11-09T17:37:42ZMichelJuillardImplement option to use LMMCP for steady state computationIntroducing mixed complementarity problems in steady state computation may be useful to exclude parts of the definition space where no solution exists. It may also help when one doesn't know whether an occasionally binding constraints bites at the steady state or not, depending on the value of the parameters.Introducing mixed complementarity problems in steady state computation may be useful to exclude parts of the definition space where no solution exists. It may also help when one doesn't know whether an occasionally binding constraints bites at the steady state or not, depending on the value of the parameters.https://git.dynare.org/Dynare/dynare/-/issues/1663datafile option in perfect_foresight_setup: incomplete documentation and not ...2020-11-13T14:22:21ZMichelJuillarddatafile option in perfect_foresight_setup: incomplete documentation and not flexible enoughthe datafile option in perfect_foresight solver requires a text file without variable names, variables being order in order of VAR statement. The file name must end with ``_endo.dat``.
- [x] make loading data for guess value more flexible
- [x] honor INITVAL_FILE command or deprecate it
- [ ] make possible to use guess values with simul_backward-modelthe datafile option in perfect_foresight solver requires a text file without variable names, variables being order in order of VAR statement. The file name must end with ``_endo.dat``.
- [x] make loading data for guess value more flexible
- [x] honor INITVAL_FILE command or deprecate it
- [ ] make possible to use guess values with simul_backward-model4.7https://git.dynare.org/Dynare/dynare/-/issues/1662Add support for linear models in perfect_foresight_problem MEX2019-11-27T13:11:54ZSébastien VillemotAdd support for linear models in perfect_foresight_problem MEXIn that case, the Jacobian needs not be recomputed for each period.In that case, the Jacobian needs not be recomputed for each period.Sébastien VillemotSébastien Villemothttps://git.dynare.org/Dynare/dynare/-/issues/1661`fast` option to dynare does not recompile every time the model changes2019-10-08T07:17:30ZHoutan Bastani`fast` option to dynare does not recompile every time the model changesThe temporary terms of the equations are not printed in the buffer on which the checksum is calculated. Hence, if the model is changed in a term that then becomes a temporary term, the checksum does not change.The temporary terms of the equations are not printed in the buffer on which the checksum is calculated. Hence, if the model is changed in a term that then becomes a temporary term, the checksum does not change.4.6Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/-/issues/1660Update documentation of dseries2021-01-21T15:53:26ZSébastien VillemotUpdate documentation of dseriesSince dseries have been substantially rewritten since 4.5, the documentation in the reference needs to be updated.Since dseries have been substantially rewritten since 4.5, the documentation in the reference needs to be updated.4.7Stéphane Adjemianstepan@adjemian.euStéphane Adjemianstepan@adjemian.euhttps://git.dynare.org/Dynare/dynare/-/issues/1659Filter out non-positive definite Hessians before Laplace approximation2019-11-21T14:06:17ZJohannes Pfeifer Filter out non-positive definite Hessians before Laplace approximationWe do not check whether the normal approximation underlying the Laplace approximation to the marginal data density makes any sense given the Hessian at hand. That results in wrong output, because the log determinant will be complex and Matlab only prints the real-valued parts.
See https://forum.dynare.org/t/hessian-matrix-at-the-mode-is-not-positive-definite-but-the-marginal-density-is-displayed-anyway/14510/3
Proposed solution: skip the computation of the Hessian is not positive semi-definite and return NaN.We do not check whether the normal approximation underlying the Laplace approximation to the marginal data density makes any sense given the Hessian at hand. That results in wrong output, because the log determinant will be complex and Matlab only prints the real-valued parts.
See https://forum.dynare.org/t/hessian-matrix-at-the-mode-is-not-positive-definite-but-the-marginal-density-is-displayed-anyway/14510/3
Proposed solution: skip the computation of the Hessian is not positive semi-definite and return NaN.4.6Johannes Pfeifer Johannes Pfeifer https://git.dynare.org/Dynare/dynare/-/issues/1658Write a howto on forecasting2019-09-20T13:06:55ZSébastien VillemotWrite a howto on forecastinghttps://git.dynare.org/Dynare/dynare/-/issues/1657add shock decomposition for forecasts done after estimation2021-01-21T15:56:53ZSébastien Villemotadd shock decomposition for forecasts done after estimation@MichelJuillard has some preliminary codes.@MichelJuillard has some preliminary codes.4.8Sébastien VillemotSébastien Villemothttps://git.dynare.org/Dynare/dynare/-/issues/1656Provide documentation for the upgrading to 4.62020-01-10T15:47:45ZSébastien VillemotProvide documentation for the upgrading to 4.6Between 4.5 and 4.6, many structures in `M_`, `options_` and elsewhere which used to be character vectors are now cell arrays. Other things have changed in a backward-incompatible way.
We need to compile the list of all those structures, and document that change in the release notes.
Between 4.5 and 4.6, many structures in `M_`, `options_` and elsewhere which used to be character vectors are now cell arrays. Other things have changed in a backward-incompatible way.
We need to compile the list of all those structures, and document that change in the release notes.
4.6https://git.dynare.org/Dynare/dynare/-/issues/1655Move created LaTeX-files to subfolder2019-11-21T14:15:56ZJohannes Pfeifer Move created LaTeX-files to subfolderSee https://git.dynare.org/Dynare/preprocessor/commit/0988a1f755be01b00aab7a458c89d9b63800875d#note_8528See https://git.dynare.org/Dynare/preprocessor/commit/0988a1f755be01b00aab7a458c89d9b63800875d#note_85284.6Johannes Pfeifer Johannes Pfeifer https://git.dynare.org/Dynare/dynare/-/issues/1654Efficient computation of products of matrix arrays2019-07-03T10:43:38ZWilli Mutschlerwilli@mutschler.euEfficient computation of products of matrix arraysHi,
while extending the identification toolbox I often run into the need to compute a 2d matrix with a 3d array. That is, assume `A is [m x n]`, `B is [n x o x p]` and I need to compute `C(:,:,j)=A*B(:,:,j)`. Currently, I simply run a for loop (as p is dimension of parameters and hence not so large), i.e. the minimal matlab code looks like
```
for j=1:size(B,3)
C(:,:,j) = A*B(:,:,j);
end
```
Is there a more efficient way/trick to do so?Hi,
while extending the identification toolbox I often run into the need to compute a 2d matrix with a 3d array. That is, assume `A is [m x n]`, `B is [n x o x p]` and I need to compute `C(:,:,j)=A*B(:,:,j)`. Currently, I simply run a for loop (as p is dimension of parameters and hence not so large), i.e. the minimal matlab code looks like
```
for j=1:size(B,3)
C(:,:,j) = A*B(:,:,j);
end
```
Is there a more efficient way/trick to do so?https://git.dynare.org/Dynare/dynare/-/issues/1653kstate2020-05-04T16:22:56ZWilli Mutschlerwilli@mutschler.eukstateHi,
I was wondering if somebody can explain to me the structure of kstate? I am aware that using this is depreciated, as this is a reminiscence of old dynare versions where we did not create auxiliary variables for leads and lags greater than one, but still I find it is used in several functions dealing with e.g. the perturbation solution or identification toolbox.
In any case, I still have problems to get the hang of this. So maybe someone can help me out, given the following simple example:
```
var Y C K A;
varexo eps_A;
parameters alph betta rhoA sigA;
alph = 0.35; betta = 0.99; rhoA = 0.9; sigA = 0.6;
model;
C^(-1)=alph*betta*C(+1)^(-1)*A(+1)*K^(alph-1);
K=A*K(-1)^alph-C;
log(A)=rhoA*log(A(-1))+sigA*eps_A;
Y = A*K(-1)^alph;
end;
```
As I understand, my state variables are K and A, i.e.
- the declaration order is Y C K A
- the DR order is Y K A C
- lead_lag_incidence is equal to
| | Y | C | K | A |
|------|---|---|---|---|
| t-1 | 0 | 0 | 1 | 2 |
| t | 3 | 4 | 5 | 6 |
| t+1 | 0 | 7 | 0 | 8 |
where the number corresponds to the corresponding column number in the dynamic files (i.e. derivative wrt the variable)
Now, the kstate variable is given by:
| | | | |
| ------ | ------ | -----|--|
| 3 | 3 | 4 | 0|
| 4 | 3 | 3 | 0|
| 2 | 2 | 0 | 1|
| 3 | 2 | 0 | 2|
but I do not understand this. As far as I see, there is only one comment in set_state_space.m:
```
% composition of state vector
% col 1: variable; col 2: lead/lag in z(t+1);
% col 3: A cols for t+1 (D); col 4: A cols for t (E)
```
but what is the meaning of z(t+1), A, D and E or to which state space representation do these correspond to?
Thanks!Hi,
I was wondering if somebody can explain to me the structure of kstate? I am aware that using this is depreciated, as this is a reminiscence of old dynare versions where we did not create auxiliary variables for leads and lags greater than one, but still I find it is used in several functions dealing with e.g. the perturbation solution or identification toolbox.
In any case, I still have problems to get the hang of this. So maybe someone can help me out, given the following simple example:
```
var Y C K A;
varexo eps_A;
parameters alph betta rhoA sigA;
alph = 0.35; betta = 0.99; rhoA = 0.9; sigA = 0.6;
model;
C^(-1)=alph*betta*C(+1)^(-1)*A(+1)*K^(alph-1);
K=A*K(-1)^alph-C;
log(A)=rhoA*log(A(-1))+sigA*eps_A;
Y = A*K(-1)^alph;
end;
```
As I understand, my state variables are K and A, i.e.
- the declaration order is Y C K A
- the DR order is Y K A C
- lead_lag_incidence is equal to
| | Y | C | K | A |
|------|---|---|---|---|
| t-1 | 0 | 0 | 1 | 2 |
| t | 3 | 4 | 5 | 6 |
| t+1 | 0 | 7 | 0 | 8 |
where the number corresponds to the corresponding column number in the dynamic files (i.e. derivative wrt the variable)
Now, the kstate variable is given by:
| | | | |
| ------ | ------ | -----|--|
| 3 | 3 | 4 | 0|
| 4 | 3 | 3 | 0|
| 2 | 2 | 0 | 1|
| 3 | 2 | 0 | 2|
but I do not understand this. As far as I see, there is only one comment in set_state_space.m:
```
% composition of state vector
% col 1: variable; col 2: lead/lag in z(t+1);
% col 3: A cols for t+1 (D); col 4: A cols for t (E)
```
but what is the meaning of z(t+1), A, D and E or to which state space representation do these correspond to?
Thanks!https://git.dynare.org/Dynare/dynare/-/issues/1652Crash in bytecode2019-12-03T14:45:18ZSébastien VillemotCrash in bytecodeI attach an example ([surprise_solution.zip](/uploads/17158ceefc6fbf59f6810d17cbcdc084/surprise_solution.zip)) that crashes bytecode (just by running `surprise_solution_2019_05_20.mod`).
The problem can be reproduced both with unstable and 4.5.
*Note:* comes from https://forum.dynare.org/t/matlab-shutdown-deterministic-solution-stack-solve-algo-1/13882I attach an example ([surprise_solution.zip](/uploads/17158ceefc6fbf59f6810d17cbcdc084/surprise_solution.zip)) that crashes bytecode (just by running `surprise_solution_2019_05_20.mod`).
The problem can be reproduced both with unstable and 4.5.
*Note:* comes from https://forum.dynare.org/t/matlab-shutdown-deterministic-solution-stack-solve-algo-1/138824.6Sébastien VillemotSébastien Villemot