Integration with DBnomics

Allow the construction of series using identifiers or other criteria.

Inspiration could be taken from the API of the rdbnomics package.

added enhancement label

Note that since R2016b, MATLAB has the jsondecode function for parsing JSON files, which is quite efficient. For older versions of MATLAB and for Octave, a possibility is to use the JSON library which is under contrib/jsonlab/ in Dynare, but which is not very efficient. Still another possibility would be to create an ad hoc MEX file.

We use jsondecode at the ECB, it's much faster. But if we write a mex file for reading json files, I think we could improve on the java based jsondecode routine (also I am not sure this routine works if the -nojvm Matlab's option is used). We will probably need, for other purposes like exploiting estimation/simulation results from a GUI, a routine to write JSON files.

Regarding the integration with dbnomics, I think we should separate things. We have two problems

1. How to download JSON/CSV files from dbnomics?
2. How to put the data in dseries objects?

For 1. the only issue is to deal with the various ways of downloading remote files, depending on the engine (old/new Matlab, or Octave). With recent versions of Matlab, we can use the webread routine which directly puts the content of the downloaded JSON file (through dbnomics API) into a structure. I think this routine uses jsondecode. I had some examples for this, but something must have changed in dbnomics, and I cannot get it work (I don't find anymore the data in the structure).

For 2. Once we have the data in a structure, we just have to put them in a dseries. The only issue is how we deal with the periods (which are not coded using the same conventions in dbnomics and dseries). For instance, 2020Q1 in dseries will be 2020-01-01 in dbnomics (I don't remember the order). So we need a routine to translate the periods (actually only the first period) when instantiating.

I would separate 1. and 2. First by developing a general matlab class to fetch data from dbnomics. And then adding routines to dseries class to fill dseries objects with these routines.

Yes, after discussing with @MichelJuillard and Thomas Brand, we are precisely going to separate the work in two parts: a generic MATLAB/Octave plugin for DBnomics (that would also be distributed by the DBnomics project), and dseries routines that would use this plugin.

Wouldn't it be better to include the conversion routines to dseries under the mdbnomics package? Just as DBnomics is offering a conversion to xts in their R plugin. So that we offer a full package which converts the final cell array to a more convenient time series setting if the user prefers, making dseries as a dependency of mdbnomics. This way we also offer more visibility for dseries to potential new users who come across the MATLAB plugin via the DBnomics website.

What do you think about this @MichelJuillard @stepan-a?

@DoraK I think it would be simpler not to add dseries as a dependency of the mdbnomics package. By the way, where can we have a look on the code?

mentioned in commit DoraK/dseries@198023a2

closed via merge request !38 (merged)

mentioned in commit af4d2cc3