dynare issueshttps://git.dynare.org/Dynare/dynare/-/issues2023-10-02T15:40:12Zhttps://git.dynare.org/Dynare/dynare/-/issues/1859New interface for dynamic and static files representing the model2023-10-02T15:40:12ZSébastien VillemotNew interface for dynamic and static files representing the modelThe preprocessor currently creates `dynamic` and `static` files for representing the model (forgetting about the `bytecode` case which is sligthly different), which have the following two characteristics:
- the Jacobian matrix that they ...The preprocessor currently creates `dynamic` and `static` files for representing the model (forgetting about the `bytecode` case which is sligthly different), which have the following two characteristics:
- the Jacobian matrix that they return is a dense one
- in the `dynamic` file, the column indices for the dynamic (endogenous) variables are organized so as to avoid columns of zeros; said otherwise, if there are `n` variables, there are less than `3×n` columns (for endogenous) in the dynamic Jacobian. This design decision makes sense since the matrix is dense, but it complicates the computations with the Jacobian matrix.
The proposal is to change the design and have the `dynamic` and the `static` files:
- return a sparse Jacobian
- for the `dynamic` file, use exactly `3×n` columns for the endogenous in the Jacobian (and also for higher order derivatives)
It is expected that having a sparse Jacobian will improve performance, at least for large models.
Since the `static` and `dynamic` files are used at many places in the code, the transition cannot be done overnight, and the idea is to have the old and the new interface coexist for some time. The old interface would be removed at some point (unless it turns out that there is a real use case for dense Jacobians).
The interaction with block decomposition also needs to be taken into account when doing this transition. Currently, when block decomposition is requested, the preprocessor outputs `static` and `dynamic` files that have the same name as in the case without block decomposition, but a different interface. This is bad design. The preprocessor should use different names for those files when using block decomposition. We could even go as far as always providing the two versions of those files (with and without block decomposition), so that the `block` option of the `model` block would no longer affect preprocessor output (but only the algorithms used for computations). An even further step could be to always use block decomposition for perfect foresight, and drop the other code paths (but we probably don’t want to do that for the stochastic case). In particular, this could help with large backward models, for which we only need (dynamic) derivatives with respect to contemporaneous variables, an optimization which is already automatically done when using block decomposition.
Steps:
- [x] Change the preprocessor so that it produces the new `static` and `dynamic` files in parallel to the old ones
- [x] Change the preprocessor so that it always outputs the block decomposed files (with the new interface) in parallel to the non-block decomposed ones? (see also preprocessor#41)
- [x] Start using the new interface in the MATLAB/Octave code
- [ ] Drop the perfect foresight algorithms that do not use block decomposition? (i.e. `block` becomes the default, and only choice, for perfect foresight). This would probably require #1858 to be done first.
- [ ] Drop the old interface (if it is confirmed that dense Jacobians have no real use case)Sébastien VillemotSébastien Villemothttps://git.dynare.org/Dynare/dynare/-/issues/1858Use MEX to create stacked Jacobian in two-boundaries blocks with block decomp...2022-09-13T15:50:11ZSébastien VillemotUse MEX to create stacked Jacobian in two-boundaries blocks with block decompositionThe `perfect_foresight_problem` MEX was introduced so as to speed-up the construction of the stacked Jacobian for deterministic problems (when there are both leads and lags).
However, when doing block decomposition, there is no equivale...The `perfect_foresight_problem` MEX was introduced so as to speed-up the construction of the stacked Jacobian for deterministic problems (when there are both leads and lags).
However, when doing block decomposition, there is no equivalent MEX for constructing the stacked Jacobian for two-boundaries blocks. This is an obvious opportunity for optimization.
It remains to be seen how much can be factorized with the existing `perfect_foresight_problem` MEX. The bytecode case may also need special treatment.https://git.dynare.org/Dynare/dynare/-/issues/1187allow to skip 2nd order param derivs2019-06-19T15:37:50ZMarco Rattoallow to skip 2nd order param derivsFor large models, computing the analytic Hessian just produces a too big `_params_derivs.m` file which is untractable.
For running identification or only computing scores, 2nd order derivs would be useless.
@houtanb Would it be possibl...For large models, computing the analytic Hessian just produces a too big `_params_derivs.m` file which is untractable.
For running identification or only computing scores, 2nd order derivs would be useless.
@houtanb Would it be possible to tell the pre-processor to only compute 1st order derivatives, i.e. stop writing the `param_derivs` file when it reaches
`if nargout >= 3 ...`
?
2nd order terms could be just defined as empty, and routines (`dsge_likelihood`) possibly asking for 2nd order ones will trap this and just stop.
4.5Houtan BastaniHoutan Bastanihttps://git.dynare.org/Dynare/dynare/-/issues/330ReshapeMatFiles2016-06-14T20:57:38ZMarco RattoReshapeMatFilesReshapeMatFiles is currently used only for posterior IRF's. For large models it is a huge bottle-neck which is probably useless. The approach followed in prior_posterior_statistics.m is to use the pm3 utility that takes the data needed f...ReshapeMatFiles is currently used only for posterior IRF's. For large models it is a huge bottle-neck which is probably useless. The approach followed in prior_posterior_statistics.m is to use the pm3 utility that takes the data needed for plots from the non-reshaped files. Such a procedure is in the end much less time consuming than the ReshapeMatFile.
The current pm3 utility is not easily extended for irfs (matrices are 4-dimensional for irf's).
However I tried to write a pm4 utility for 4-D matrices which looks interesting.
How about removing the call in posteriorIRF to:
ReshapeMatFile
posteriorIRF_core2 (the plot utility for bayesian irf's which uses the reshaped files)
add making a new call to a pm4.m utility that replicates the behaviour of pm3 to 4-D matrices?
Marco RattoMarco Rattohttps://git.dynare.org/Dynare/dynare/-/issues/270Determinant computation in marginal_density.m looks sub-optimal2015-08-06T13:33:30ZSébastien VillemotDeterminant computation in marginal_density.m looks sub-optimalIn marginal_density.m, the determinant is computed (at three places) using the det() function. This function is known to be less precise than a Cholesky-based determinant computation when the determinant is close to zero.
Stéphane think...In marginal_density.m, the determinant is computed (at three places) using the det() function. This function is known to be less precise than a Cholesky-based determinant computation when the determinant is close to zero.
Stéphane thinks that the Cholesky was used before, so we need to understand why this has changed, and therefore whether we can go back to the Cholesky.
Issue reported by Gilles Bélanger (Ministère des Finances du Québec)
https://git.dynare.org/Dynare/dynare/-/issues/38Allow k-order and estimation DLL to use bytecode for the model evaluation2016-06-10T13:10:36ZSébastien VillemotAllow k-order and estimation DLL to use bytecode for the model evaluationCurrently, k-order DLL and estimation DLL only work with use_dll option.
This constraint could be easily relieved by calling the M-file of the model from within the DLL, using mexCallMATLAB().
For the bytecode, a similar trick could be...Currently, k-order DLL and estimation DLL only work with use_dll option.
This constraint could be easily relieved by calling the M-file of the model from within the DLL, using mexCallMATLAB().
For the bytecode, a similar trick could be done, provided that support is added for 2nd and 3rd order derivatives.
From the implementation side, the idea would be to create an super class of DynamicModelDll.
This change needs to be documented in the reference manual when implemented.
https://git.dynare.org/Dynare/dynare/-/issues/23Add Estimation DLL2016-06-10T12:50:14ZSébastien VillemotAdd Estimation DLLFor DSGE-net.
For DSGE-net.
4.5https://git.dynare.org/Dynare/dynare/-/issues/8Benchmarks with various BLAS libraries2023-06-06T07:56:03ZSébastien VillemotBenchmarks with various BLAS librariesWe should test the 5 options:
- reference BLAS
- generic ATLAS
- customized ATLAS
- generic OpenBLAS
- customized OpenBLAS
The tests should be performed on a few hardware and on a few interesting models. The results should be put on a W...We should test the 5 options:
- reference BLAS
- generic ATLAS
- customized ATLAS
- generic OpenBLAS
- customized OpenBLAS
The tests should be performed on a few hardware and on a few interesting models. The results should be put on a Wiki page.
https://git.dynare.org/Dynare/dynare/-/issues/6Possible optimization of decision rules at first order2023-06-06T07:56:02ZSébastien VillemotPossible optimization of decision rules at first orderIn dr.pdf, at the end of section 4.1, we have:
g^-_y = Z'_11 T^{-1}_11 S_{11} (Z'_{11})^{-1}
But dr1.m used to contain the following formula:
g^-_y = (T^{-1}_11 X)^{-1} S_{11} X
which is probably more efficient given that X=Z_{11}+Z_...In dr.pdf, at the end of section 4.1, we have:
g^-_y = Z'_11 T^{-1}_11 S_{11} (Z'_{11})^{-1}
But dr1.m used to contain the following formula:
g^-_y = (T^{-1}_11 X)^{-1} S_{11} X
which is probably more efficient given that X=Z_{11}+Z_{12}g⁺_y (see equation 10), because it saves an inversion
We should make benchmarks between the 2 formulas, because Ondra thinks that for performance reasons, the first method is better, even though this is not obvious.Sébastien VillemotSébastien Villemot