Fix bug for mh_recover that prevents using mh-files already created
There seems to be a bug hidden in metropolis_hastings_initialization.m
when using mh_recover
.
We have at the end
NumberOfSavedMhFiles = NumberOfSavedMhFilesInTheCrashedBlck+LastFileNumberInThePreviousMh;
if NumberOfSavedMhFiles
load([BaseName '_mh' int2str(NumberOfSavedMhFiles) '_blck' int2str(fblck) '.mat']);
ilogpo2(fblck) = logpo2(end);
ix2(fblck,:) = x2(end,:);
end
That means that the starting values for the chain are taken from the last mh-file on the disk. In the unit test at #911 , there are 4 in the current block and the crash happened after file number 3 (as the fourth file is deleted). In this case, we start the MCMC with the last draw in mh-file number 3. But we do not set Fline
, nruns
, and NewFile
to account for the presence of three quarters of the MCMC. Instead, we run the full chain again, but starting at the endpoint of the last crashed chain.
Moreover, I think we are not accounting for the state of the random number generator when using mh_recover. My proposal is to save the last seed of each mh-file in record
(instead of always only keeping the very last one for use in load_mh_file
, saved to record.LastSeeds
in random_walk_metropolis_hastings_core
). We could then continue after the respective last mh-file on the disk at this state by changing record.InitialSeeds(curr_block)
together with Fline
, nruns
, and NewFile
in metropolis_hastings_initialization.m