Potentially change use of options_.drop in simulations

Currently, options_.drop is used for both IRFs and moments. However, the simulated series generated with simult_ do not drop periods. I think that this is not desirable and we should also drop periods at the beginning unless the user wants to start at the deterministic steady state.

For linear models, this would not matter. But for models solved at higher order, we would then not knowingly output simulated series that for sure have not yet converged to the ergodic distribution.

I don't understand: simulated series don't converge. You probably mean "drawn from the ergodic distribution."

Currently, simult_.m returns simulated series conditional on given initial conditions (y0) and a sequence of pseudo random shocks. IMHO, if the user wants to use the simulated series to do econometrics, assuming that the series are drawn from the ergodic distribution of the approximated model, it is her responsibility to simulate more periods than needed and drop the initial ones in her use of the simulated series.

I think it is clearer to explain that way and it is better to make the user conscious of the fact that simulated series start necessarily from some fixed initial conditions.

Johannes Pfeifer writes:

Currently, options_.drop is used for both IRFs and moments. However, the simulated series generated with simult_ do not drop periods. I think that this is not desirable and we should also drop periods at the beginning unless the user wants to start at the deterministic steady state.

For linear models, this would not matter. But for models solved at higher order, we would then not knowingly output simulated series that for sure have not yet converged to the ergodic distribution.

---

Reply to this email directly or view it on GitHub: https://github.com/DynareTeam/dynare/issues/396

Michel Juillard

Yes, I meant they are not drawn from the ergodic distribution. Perhaps it's only me, but I find the current use of options_.drop confusing. We feel that by default we should not use the first 100 periods for any type of computation (IRFs, moments, etc.). But although we think that we should discard a burnin, we don't do this automatically when providing simulated series, but leave this up to the user.

I see the problem that there are cases where we have a given y0 we want to start from. But if endo_histval is empty, we always start at the deterministic steady state and I cannot think about any application where I want a simulation where the initial draws are not from the ergodic distribution. Thus, I would prefer the default setting to be the other way round.

Moreover, error message like

STOCH_SIMUL error: The horizon of simulation is shorter than the number of observations to be dropped 

are confusing. If a user specifies for example periods=50, he wants a simulation of 50 periods. Instead, we output nothing, because drop is 100. Unless he sets periods=100 and manually drops the first 50 periods, there is no way to obtain this.

Put differently, if we drop the first options_.drop periods by default for simulations, a user can automatically generate any number of simulated periods from the ergodic distribution by appropriately setting drop and periods. If we leave the current behavior, the user can only generate any number of simulated periods if he sets drop to 0 and accepts that the resulting draws will start at initial conditions and are not from the ergodic distribution.

If we think we should leave the current behavior, we should make it more explicit in the manual that drop only affects subsequent calculations based on simulations, but not the simulation themselves.

I would like the opinion of the other developers on this issue.

In any case, simult.m or simult_.m is also used for forecasts, where we don't want to drop anything. So, we would need to be extra careful where we put the truncation of the simulated series.

The inconvenient with changes in default is that existing *.mod files will not have the same behavior anymore. So that can only be done on the occasion of a major version update.

I agree that anyway, more detailed explanations are in order in the manual

Johannes Pfeifer writes:

Yes, I meant they are not drawn from the ergodic distribution. Perhaps it's only me, but I find the current use of options_.drop confusing. We feel that by default we should not use the first 100 periods for any type of computation (IRFs, moments, etc.). But although we think that we should discard a burnin, we don't do this automatically when providing simulated series, but leave this up to the user.

I see the problem that there are cases where we have a given y0 we want to start from. But if endo_histval is empty, we always start at the deterministic steady state and I cannot think about any application where I want a simulation where the initial draws are not from the ergodic distribution. Thus, I would prefer the default setting to be the other way round.

Moreover, error message like

STOCH_SIMUL error: The horizon of simulation is shorter than the number of observations to be dropped 

are confusing. If a user specifies for example periods=50, he wants a simulation of 50 periods. Instead, we output nothing, because drop is 100. Unless he sets periods=100 and manually drops the first 50 periods, there is no way to obtain this.

Put differently, if we drop the first options_.drop periods by default for simulations, a user can automatically generate any number of simulated periods from the ergodic distribution by appropriately setting drop and periods. If we leave the current behavior, the user can only generate any number of simulated periods if he sets drop to 0 and accepts that the resulting draws will start at initial conditions and are not from the ergodic distribution.

If we think we should leave the current behavior, we should make it more explicit in the manual that drop only affects subsequent calculations based on simulations, but not the simulation themselves.

---

Reply to this email directly or view it on GitHub: https://github.com/DynareTeam/dynare/issues/396#issuecomment-17779613

Michel Juillard

I was thinking about leaving simult and simult_ unaltered. We only would have to call them with options_,drop + options_.periods. This would not break any other behavior, but would at the same time alleviate the need for additionally dropping periods in disp_moments.

As far as I can see, the issue mostly involves stoch_simul. Thus, in stoch_simul we could just change the assignment oo_.endo_simul = ys; to oo_.endo_simul = ys(:,options_.drop+1:end);

and in disp_moments change y = y(ivar,options_.drop+1:end)'; to

y = y(ivar,:)';

Let's see what the others think.

Like @MichelJuillard , I also feel a bit uneasy with changing the behavior of stoch_simul with respect to the length of simulated series. I would rather prefer a clear explanation about what is going on, in particular the fact that the drop option does not remove the burnin from simulated series when directly used by the user.

@sebastien-villemot If I may ask, what is the disadvantage you see?

The rationale is simply that it is a bad thing to break old code, especially if the breakage is hard to detect. In this case, if the user has some code manipulating the simulated series, he would not get a clear error message (such as "This option is now deprecated"), but would instead get a cryptic error message, or maybe even no error message (and therefore wrong results, which is quite bad).

I see, but my suggestion was to not change the behavior of the code, except for internally dropping the drop-periods for simulations. This would not break existing code. If periods=100, we would still return a series of 100 points (with the difference of now internally simulating 200 periods and dropping the first 100 as we already do for moment computation and getting rid of the annoying error message quoted above). If someone has written code that manually drops the first n periods from endo_simul, he would now drop the first n+100 periods. But this would not provide wrong results. He would still draw from the ergodic distribution, only at a later stage.

Or do you mean if someone uses stoch_simul to access simult_ starting from the steady state? In this case you are right that the new default output would cut off the transition to the ergodic distribution.

My feeling was that there are more people out there erroneously including the burning periods in their computations than people explicitly wanting to access the burnin phase.

But I am also fine with just making the manual more explicit and keeping the current behavior.

Yes, I was thinking about people who access oo_.endo_simul and are expecting (for whatever reason) that it starts from the steady state, as it currently does.

I agree with @sebastien-villemot, I do not think that we should change the behavior of the drop option with respect to the simulated time series. I also think that there is no possible confusion if the user reads the entry for the drop option in the manual :

Number of points dropped at the beginning of simulation before computing the summary statistics.