[![pipeline status](https://git.dynare.org/DoraK/mdbnomics/badges/master/pipeline.svg)](https://git.dynare.org/DoraK/mdbnomics/commits/master)

This MATLAB toolbox comes with routines to access DBnomics time series from MATLAB.

The package is compatible with MATLAB 2019b and following versions.
Octave compability will follow after the release of Octave 6 (an implementation to webread/webwrite/jsondecode is required).

## Installation

The toolbox can be installed by cloning the Git repository:

    ~$ git clone https://git.dynare.org/DoraK/mdbnomics.git

or downloading a zip archive:

    ~$ wget https://git.dynare.org/DoraK/mdbnomics/-/archive/master/mdbnomics-master.zip
    ~$ unsip mdbnomics-master.zip
    -$ mv mdbnomics-master mdbnomics

## Usage

Add the `mdbnomics/src` folder to the MATLAB path, and run the following command (on MATLAB) prompt:

    >> initialize_mdbnomics()

which, depending on your system, will add the necessary subfolders to
the MATLAB path.

You are then ready to go. A full documentation will come soon.

## Capabilities

### Fetch one time series by ID
First, let's assume that we know which series we want to download.
A series identifier (ID) is defined by three values, formatted like this: `provider_code/dataset_code/series_code`.
The `fetch_series` function is used to construct the cell array.
For example, to fetch the time series `EA19.1.0.0.0.ZUTN` from the
[\"Unemployment rate\" [ZUTN] dataset](https://db.nomics.world/AMECO/ZUTN)
belonging to the [AMECO provider](https://db.nomics.world/AMECO).

Example:

    >> df_id = fetch_series('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN');

The returned data is stored in the `df_id` variable. Its type is a cell array. To display the first 3 rows of the array as a table
(including the column headers), type:

    >> tab_id = cell2table(df_id(2:end,:), 'VariableNames', df_id(1,:));
    >> tab_id(1:3,:)
       
       3�16 table
       x_frequency    provider_code    dataset_code                               dataset_name                                    series_code                                 series_name                             original_period        period        original_value    value    freq                    unit                      geo        Frequency                      Unit                        Country   
        ___________    _____________    ____________    __________________________________________________________________    _____________________    ____________________________________________________________    _______________    ______________    ______________    _____    _____    ___________________________________    ________    ____________    _____________________________________    _____________

        {'annual'}       {'AMECO'}        {'ZUTN'}      {'Unemployment rate: total :- Member States: definition EUROSTAT'}    {'EA19.1.0.0.0.ZUTN'}    {'Annually � (Percentage of active population) � Euro area'}       {'1960'}        {'1960-01-01'}        {'NA'}         NaN     {'a'}    {'percentage-of-active-population'}    {'ea19'}    {'Annually'}    {'(Percentage of active population)'}    {'Euro area'}
        {'annual'}       {'AMECO'}        {'ZUTN'}      {'Unemployment rate: total :- Member States: definition EUROSTAT'}    {'EA19.1.0.0.0.ZUTN'}    {'Annually � (Percentage of active population) � Euro area'}       {'1961'}        {'1961-01-01'}        {'NA'}         NaN     {'a'}    {'percentage-of-active-population'}    {'ea19'}    {'Annually'}    {'(Percentage of active population)'}    {'Euro area'}
        {'annual'}       {'AMECO'}        {'ZUTN'}      {'Unemployment rate: total :- Member States: definition EUROSTAT'}    {'EA19.1.0.0.0.ZUTN'}    {'Annually � (Percentage of active population) � Euro area'}       {'1962'}        {'1962-01-01'}        {'NA'}         NaN     {'a'}    {'percentage-of-active-population'}    {'ea19'}    {'Annually'}    {'(Percentage of active population)'}    {'Euro area'}
    
    >>
    

In such cell array, you will always find at least those columns:
* `x_frequency`: (harmonized frequency generated by DBnomics)
* `provider_code`
* `dataset_code`
* `dataset_name`
* `series_code`
* `series_name`
* `original_period`: the `period` as returned by DBnomics
* `period`: the first day of `original_period`
* `original_value` (`str` or `float`): the observation value as returned by DBnomics, where not available values are represented by `'NA'`
* `value` (`float` or `NaN`): the observation value as returned by DBnomics, where not available values are represented by `NaN`

Followed by dimensions columns, corresponding to the dimensions of the dataset:
* dimensions labels: `freq`, `unit`, `geo`
* dimensions values labels: `Frequency`, `Unit`, `Country`

### Fetch two time series by ID
Again, let's assume that we know which series we want to download.
We can reuse the `fetch_series` function, this time with two series codes.
For example, to fetch the time series `EA19.1.0.0.0.ZUTN` and `DNK.1.0.0.0.ZUTN` from the
[\"Unemployment rate\" [ZUTN] dataset](https://db.nomics.world/AMECO/ZUTN)
belonging to the [AMECO provider](https://db.nomics.world/AMECO).

Example:

    >> df_ids = fetch_series('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN'});

### Fetch time series by code mask
The code mask notation is a very concise way to select one or many time series at once.
It is not compatible with all the providers. In particular, only the providers from the following list accept code mask:
* BIS
* ECB
* Eurostat
* FED
* IMF
* IMF-WEO
* INSEE
* OECD
* WTO

Given 3 dimensions 'frequency', 'country' and 'indicator', the user can select:
* one time series by giving its code: `'M.FR.PCPIEC_IX'`
* many series by enumerating dimensions codes: `'M.FR+DE.PCPIEC_IX'` is equivalent to `{'M.FR.PCPIEC_IX', 'M.DE.PCPIEC_IX'}`
* many series by skipping a dimension, repeating '.' in the code mask: `'M..PCPIEC_IX'` is equivalent to `{'M.country1.PCPIEC_IX', 'M.country2.PCPIEC_IX', ..., 'M.countryN.PCPIEC_IX'}`

Examples:

    >> df_code_mask1 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
    >> df_code_mask2 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', '.FR.PCPIEC_WT');
    >> df_code_mask3 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M..PCPIEC_IX+PCPIA_IX', 'max_nb_series', 400);

### Fetch time series by dimension
Searching by dimension is a less concise way to select time series than using the code mask, but it's universal:
some fetchers are not compatible with the code mask notation. The following example fetches many series from the
["Doing Business" [DB]](https://db.nomics.world/WB/DB) dataset of the [World Bank](https://db.nomics.world/WB) provider, selecting for time series about France, Italy and Spain (`country` dimension),
and the indicator "Procedures required to start a business - Women (number)" (`indicator` dimension).

Example:

    >> df_dims = fetch_series('provider_code', 'WB', 'dataset_code', 'DB', 'dimensions', '{"country":["ES","FR","IT"],"indicator":["IC.REG.COST.PC.FE.ZS.DRFN"]}');

### Fetch time series by API link
When the dimensions, provider, dataset or series codes are unknown, the user can:
* go to the page of a dataset on DBnomics website (eg: [Doing Business](https://db.nomics.world/WB/DB))
* select some dimensions by using the input widgets of the left column
* click on `Copy API link` in the menu of the `Download` button
* use the `fetch_series_by_api_link` function

Example:

    >> df_link = fetch_series_by_api_link('https://api.db.nomics.world/v22/series/WB/DB/ENF.CONT.COEN.ATDR-AE?observations=1');

### Fetch time series from the cart
On the [cart page](https://db.nomics.world/cart) of the DBnomics website, click on "Copy API link" and copy-paste it as an argument of the fetch_series_by_api_link function.
Please note that when you update your cart, you have to copy this link again, because the link itself contains the IDs of the series in the cart.

Example:

    >> df_cart = fetch_series_by_api_link('https://api.db.nomics.world/v22/series?series_ids=AMECO%2FZUTN%2FEA19.1.0.0.0.ZUTN&observations=1');

### Fetch time series with different frequencies

Example:

    >> df_multi_freq = fetch_series('series_ids', {'BEA/NIUnderlyingDetail-U001BC/S315-A',...
                                                   'BEA/NIUnderlyingDetail-U001BC/S315-Q',...
                                                   'BEA/NIUnderlyingDetail-U001BC/S315-M'});

### Transform time series
The routines can interact with the [Time Series Editor](https://editor.nomics.world/) to transform time series by applying filters to them.
Available filters are listed on the [filters page](https://editor.nomics.world/filters).
The Time Series Editor is usable via a web interface ([example with AMECO/ZUTN/EA19.1.0.0.0.ZUTN](https://editor.nomics.world/series?source=dbnomics&series_id=AMECO/ZUTN/EA19.1.0.0.0.ZUTN))
but you can call it directly from MATLAB. The user is also able to chain many filters.
Here is an example of how to interpolate two annual time series with a monthly frequency, using a spline interpolation.

Example:

    >> filters_ = '[{"code": "interpolate", "parameters": {"frequency": "monthly", "method": "spline"}}]';
    >> df_filter = fetch_series('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'dbnomics_filters', filters_);

The first row of the final cell array changes when filters are used:
* `period_middle_day`: the middle day of `original_period` (can be useful when you compare graphically interpolated series and original ones)
* `filtered` (`bool`): True if the series is filtered
* `series_code`: same as before for original series, but the suffix `_filtered` is added for filtered series
* `series_name`: same as before for original series, but the suffix `(filtered)` is added for filtered series