Skip to content

Make sure options_.first_obs is properly set

Johannes Pfeifer requested to merge JohannesPfeifer:dseries_first_obs into master

In dynare_estimation_1.m we restore compatibility with the old data interface via

%set options for old interface from the ones for new interface
if ~isempty(dataset_)
    options_.nobs = dataset_.nobs;
    options_.first_obs=double(dataset_.init);
end

This is problematic, because

  1. options_ is global and first_obs is then not NaN anymore, affecting subsequent calls. A short solution would be to reset options_.first_obs at the end of the function, but I am not sure, this is the best solution.
  2. The setting of first_obs does not correctly work if the data is a dseries that does not start with 1. To see the problem, consider a dseries dataset that starts in 1950Q1. options_.first_obs.double(dataset_.init) will then be 1950 instead of 1. This will for example invalidate the
trend_addition=trend_coeff*[DynareOptions.first_obs:DynareOptions.first_obs+ntobs-1];

in compute_trend_coefficients.m, which relies on first_obs being an integer with observations starting at 1. 3. makedataset.m contains the line

DynareDataset = DynareDataset(FIRSTOBS:lastobs);

so we always cut the sample. But this implies that we lose all information on the first period originally contained in the dataset and therefore prevents recovering the implied options_.first_obs afterwards. Thus, we need to do it before calling makedataset.m. @stepan-a I am not sure how makedataset is intended to work, but it seems a solution might be to condition the setting of first_obs=1 if NaN was encountered:

if isnan(options_.first_obs)
    options_.first_obs=1;
end
[dataset_, dataset_info, newdatainterfaceflag] = makedataset(options_, options_.dsge_var*options_.dsge_varlag, gsa_flag);

%set options for old interface from the ones for new interface
if ~isempty(dataset_)
    options_.nobs = dataset_.nobs;
end

A subsequent call to shock_decomposition,m will call evaluate_smoother.m, which contains

persistent dataset_ dataset_info

if isempty(dataset_) || isempty(bayestopt_) || (options_.nobs ~= dataset_.nobs)
    [dataset_,dataset_info,xparam1, hh, M_, options_, oo_, estim_params_,bayestopt_] = dynare_estimation_init(var_list, M_.fname, [], M_, options_, oo_, estim_params_, bayestopt_);
else 
% set the qz_criterium
    options_=select_qz_criterium_value(options_);
end

There are two problems here. 5. I do not understand why we use persistent variables. This strikes me as error prone. We only build the dataset the first time or when nobs changes. But what if nobs does not change, but first_obs changes? 6. Upon entering options_.first_obs is not NaN anymore, but 1950. This causes trouble in makedataset.m where we have

if isnan(DynareOptions.first_obs)
    firstobs = DynareDataset.init;
else
    firstobs = DynareDataset.dates(DynareOptions.first_obs);
end

We go to the else branch and try to select a dseries element. But element 1950 does not exist.

The attached commit seemingly fixes the issues except for the problem with persistent. The xlsx-file for the integration test is attached data_uav.xlsx

Merge request reports