Fix identification issues around steady state file

The issue popped up in https://forum.dynare.org/t/mode-check-plots-with-flat-lines/16020/13 with the attached files. From what I can see, the issue is parameters being updated in the steady state file. Identification in 4.7 seems unable to handle this. If I am not mistaken (@rattoma should know better), this is a regression as in the past, we resorted to simulation instead of theoretical derivatives which are now unavailable.

Model.mod

SOEMData.xlsx

Model_steadystate.m

added bug identification / sensitivity analysis labels

@JohannesPfeifer thank you. this is indeed a regression. I did not notice that this situation is no longer trapped, I guess since 4.6? Surely we should resort to numerical derivatives in this situation. steady state files are still widely used that also endogenized parameters, so we need to handle this. Do I get the problem correctly?

@rattoma I don't know what the original approach was in case there are endogenized parameters. But in 4.6, we do not take care of it.

@JohannesPfeifer thank you. I did not spot this detail, which might have snipped out when identification routines were fully revised/rewritten.

@JohannesPfeifer it is not a regression, it is a bug since 4.5 at least. I'll try with a pull request

mentioned in merge request !1732 (merged)

mentioned in commit 616d2b0f

mentioned in commit 120275f2

@wmutschl I noted since version 4.6/4.7, dynare does not compute correctly the jacobian of moments for Model.mod in this issue (and in principle for many others of course). namely, the observation error shocks are obviously identified (SE_eps_y_ME and SE_eps_im_ME), while dynare 4.6/4.7 provides

MOMENTS (ISKREV, 2010):
    !!!WARNING!!!
    The rank of J (Jacobian of first two moments) is deficient by 2 (rank(J) = 3 < 5 with tol = robust)!

Dynare 4.5, instead, with the same MOMENTS Jacobian analysis, correctly provides:

All parameters are identified in the model (rank of H).


All parameters are identified by J moments (rank of J)

could you pls look into this?

Hi, I found the bug. It was in this line 1059 of pruned_state_space.m which at first-order is just the normal state space system:

[indzerosrow,indzeroscol] = find(abs(dVar_y(:,:,jpV)) < 1e-12); %find values that are numerical zero
dVar_y(indzerosrow,indzeroscol,jpV) = 0;

Somehow this was not the right approach to overwrite the very small values in the array dVar_y. I fixed this and also found another bug, when one uses instead autocorrelations. I am in the process of rerunning all mod files in the testsuite both with dynare 4.5 and the 4.6 one to make sure there is no other regression or wrong computation. So far it looks good and I will merge the bugfixes soon.

@wmutschl That sounds good. Is there a way to add a test against future regressions?

We already have that with the kim2.mod test case. There the parameter s is updated in the steady state model block and we automatically switch to options_.analytic_derivation_mode= -2. Should we also add a steady_state_file here to cover both cases?

Of course this is not a severe case here, as s has no impact on the actual model equations, only on the steady state.

My question was why the testsuite did not flag any issues like the wrong Jacobian/falsely detected identifiability. I guess the design flaw is that output of identification is only provided in the command window or graphically, so we cannot check the results easily.

Actually, @rattoma could you have another look at the kim2.mod in the testsuite of the unstable version. When I run it with dynare 4.6 or unstable the analytic_derivation_mode is changed to -2 and in the command line I get the message that s changes the steady state. When I run it with dynare 4.5, however, the analytic_derivation_mode is not changed. But we do want to change this right?

@wmutschl many thanks. Indeed the latest implementation is more correct, since s is endogenized in steady state and is a function of estimated param alph. I think my latest push made this check more robust. thank you!

I agree. I also like that you put an informative message there, so users are aware now. I will add a testcase to !1733 (merged) to make sure the option is triggered.

One more thing. If I remove the estimated_params block, then we default to check identification for all model parameters and all stderr parameters. However, in dynare_estimation_init.m the check for endogenous parameters in a steady state file/block is only triggered if estim_params_.np > 0 (which is 0 if we don't have an estimated_params block). So I will add another option that triggers this in this case.

excellent, good catch

My question was why the testsuite did not flag any issues like the wrong Jacobian/falsely detected identifiability.

I think it is possible. We store the output (and options) in the identification folder. So, for instance regarding the steady_state parameter change, we can load kim2/identification/kim2_prior_mean_identif.mat and check whether store_options_ident.analytic_derivation_mode==-2. I will add this to the kim2.mod testfile once @rattoma gives his okay that the current implementation is correct.

Regarding the Jacobians, we could save the "correct" ones into mat files and run a test against this.

I was not talking about the options-setting. Rather, it is a fundamental difference if a mod-file suddenly suggests that some parameters are not identified at all anymore. That should be easy to test based on e.g. the identification strength.

Alright, I have finished the bug fixes and also provide some unit tests whether the results are correct.

Agreed, that is why it is good to also have the Qu and Tkachenko and Komunjer and Ng tests that should (but not always) come to the same conclusion. I think about how to include a real check in the testsuite.

Btw my fixes are on this branch https://git.dynare.org/wmutschl/dynare/-/tree/identification_bug_fixes

mentioned in merge request !1733 (merged)