Skip to content
Snippets Groups Projects
Select Git revision
  • master default
  • octave_fix
2 results
Compare
user avatar
bugfix: fetch_series.m, fetch_series_by_api_link.m
Dóra Kocsis authored
db439cff
History
user avatar db439cff
History
Name Last commit Last update
archive
contrib
src
tests
.gitignore
.gitlab-ci.yml
.gitmodules
COPYING.txt
Makefile
README.md
README.md

pipeline status

This MATLAB toolbox comes with routines to access DBnomics time series from MATLAB.

The package is compatible with MATLAB 2019b and following versions. Octave compability will follow after the release of Octave 6 (an implementation to webread/webwrite/jsondecode is required).

Installation

The toolbox can be installed by cloning the Git repository:

~$ git clone https://git.dynare.org/DoraK/mdbnomics.git

or downloading a zip archive:

~$ wget https://git.dynare.org/DoraK/mdbnomics/-/archive/master/mdbnomics-master.zip
~$ unsip mdbnomics-master.zip
-$ mv mdbnomics-master mdbnomics

Usage

Add the mdbnomics/src folder to the MATLAB path, and run the following command (on MATLAB) prompt:

>> initialize_mdbnomics()

which, depending on your system, will add the necessary subfolders to the MATLAB path.

You are then ready to go. A full documentation will come soon.

Capabilities

Fetch one time series by ID

First, let's assume that we know which series we want to download. A series identifier (ID) is defined by three values, formatted like this: provider_code/dataset_code/series_code. The fetch_series function is used to construct the cell array. For example, to fetch the time series EA19.1.0.0.0.ZUTN from the "Unemployment rate" [ZUTN] dataset belonging to the AMECO provider.

Example:

>> df_id = fetch_series('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN');

The returned data is stored in the df_id variable. Its type is a cell array. To display the first 3 rows of the array as a table (including the column headers), type:

>> tab_id = cell2table(df_id(2:end,:), 'VariableNames', df_id(1,:));
>> tab_id(1:3,:)
   
   3×16 table
   x_frequency    provider_code    dataset_code                               dataset_name                                    series_code                                 series_name                             original_period        period        original_value    value    freq                    unit                      geo        Frequency                      Unit                        Country   
    ___________    _____________    ____________    __________________________________________________________________    _____________________    ____________________________________________________________    _______________    ______________    ______________    _____    _____    ___________________________________    ________    ____________    _____________________________________    _____________

    {'annual'}       {'AMECO'}        {'ZUTN'}      {'Unemployment rate: total :- Member States: definition EUROSTAT'}    {'EA19.1.0.0.0.ZUTN'}    {'Annually – (Percentage of active population) – Euro area'}       {'1960'}        {'1960-01-01'}        {'NA'}         NaN     {'a'}    {'percentage-of-active-population'}    {'ea19'}    {'Annually'}    {'(Percentage of active population)'}    {'Euro area'}
    {'annual'}       {'AMECO'}        {'ZUTN'}      {'Unemployment rate: total :- Member States: definition EUROSTAT'}    {'EA19.1.0.0.0.ZUTN'}    {'Annually – (Percentage of active population) – Euro area'}       {'1961'}        {'1961-01-01'}        {'NA'}         NaN     {'a'}    {'percentage-of-active-population'}    {'ea19'}    {'Annually'}    {'(Percentage of active population)'}    {'Euro area'}
    {'annual'}       {'AMECO'}        {'ZUTN'}      {'Unemployment rate: total :- Member States: definition EUROSTAT'}    {'EA19.1.0.0.0.ZUTN'}    {'Annually – (Percentage of active population) – Euro area'}       {'1962'}        {'1962-01-01'}        {'NA'}         NaN     {'a'}    {'percentage-of-active-population'}    {'ea19'}    {'Annually'}    {'(Percentage of active population)'}    {'Euro area'}

>>

In such cell array, you will always find at least those columns:

  • x_frequency: (harmonized frequency generated by DBnomics)
  • provider_code
  • dataset_code
  • dataset_name
  • series_code
  • series_name
  • original_period: the period as returned by DBnomics
  • period: the first day of original_period
  • original_value (str or float): the observation value as returned by DBnomics, where not available values are represented by 'NA'
  • value (float or NaN): the observation value as returned by DBnomics, where not available values are represented by NaN

Followed by dimensions columns, corresponding to the dimensions of the dataset:

  • dimensions labels: freq, unit, geo
  • dimensions values labels: Frequency, Unit, Country

Fetch two time series by ID

Again, let's assume that we know which series we want to download. We can reuse the fetch_series function, this time with two series codes. For example, to fetch the time series EA19.1.0.0.0.ZUTN and DNK.1.0.0.0.ZUTN from the "Unemployment rate" [ZUTN] dataset belonging to the AMECO provider.

Example:

>> df_ids = fetch_series('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN'});

Fetch time series by code mask

The code mask notation is a very concise way to select one or many time series at once. It is not compatible with all the providers. In particular, only the providers from the following list accept code mask:

  • BIS
  • ECB
  • Eurostat
  • FED
  • IMF
  • IMF-WEO
  • INSEE
  • OECD
  • WTO

Given 3 dimensions 'frequency', 'country' and 'indicator', the user can select:

  • one time series by giving its code: 'M.FR.PCPIEC_IX'
  • many series by enumerating dimensions codes: 'M.FR+DE.PCPIEC_IX' is equivalent to {'M.FR.PCPIEC_IX', 'M.DE.PCPIEC_IX'}
  • many series by skipping a dimension, repeating '.' in the code mask: 'M..PCPIEC_IX' is equivalent to {'M.country1.PCPIEC_IX', 'M.country2.PCPIEC_IX', ..., 'M.countryN.PCPIEC_IX'}

Examples:

>> df_code_mask1 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
>> df_code_mask2 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', '.FR.PCPIEC_WT');
>> df_code_mask3 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M..PCPIEC_IX+PCPIA_IX', 'max_nb_series', 400);

Fetch time series by dimension

Searching by dimension is a less concise way to select time series than using the code mask, but it's universal: some fetchers are not compatible with the code mask notation. The following example fetches many series from the "Doing Business" [DB] dataset of the World Bank provider, selecting for time series about France, Italy and Spain (country dimension), and the indicator "Procedures required to start a business - Women (number)" (indicator dimension).

Example:

>> df_dims = fetch_series('provider_code', 'WB', 'dataset_code', 'DB', 'dimensions', '{"country":["ES","FR","IT"],"indicator":["IC.REG.COST.PC.FE.ZS.DRFN"]}');

Fetch time series by API link

When the dimensions, provider, dataset or series codes are unknown, the user can:

  • go to the page of a dataset on DBnomics website (eg: Doing Business)
  • select some dimensions by using the input widgets of the left column
  • click on Copy API link in the menu of the Download button
  • use the fetch_series_by_api_link function

Example:

>> df_link = fetch_series_by_api_link('https://api.db.nomics.world/v22/series/WB/DB/ENF.CONT.COEN.ATDR-AE?observations=1');

Fetch time series from the cart

On the cart page of the DBnomics website, click on "Copy API link" and copy-paste it as an argument of the fetch_series_by_api_link function. Please note that when you update your cart, you have to copy this link again, because the link itself contains the IDs of the series in the cart.

Example:

>> df_cart = fetch_series_by_api_link('https://api.db.nomics.world/v22/series?series_ids=AMECO%2FZUTN%2FEA19.1.0.0.0.ZUTN&observations=1');

Fetch time series with different frequencies

Example:

>> df_multi_freq = fetch_series('series_ids', {'BEA/NIUnderlyingDetail-U001BC/S315-A',...
                                               'BEA/NIUnderlyingDetail-U001BC/S315-Q',...
                                               'BEA/NIUnderlyingDetail-U001BC/S315-M'});

Transform time series

The routines can interact with the Time Series Editor to transform time series by applying filters to them. Available filters are listed on the filters page. The Time Series Editor is usable via a web interface (example with AMECO/ZUTN/EA19.1.0.0.0.ZUTN) but you can call it directly from MATLAB. The user is also able to chain many filters. Here is an example of how to interpolate two annual time series with a monthly frequency, using a spline interpolation.

Example:

>> filters_ = '[{"code": "interpolate", "parameters": {"frequency": "monthly", "method": "spline"}}]';
>> df_filter = fetch_series('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'dbnomics_filters', filters_);

The first row of the final cell array changes when filters are used:

  • period_middle_day: the middle day of original_period (can be useful when you compare graphically interpolated series and original ones)
  • filtered (bool): True if the series is filtered
  • series_code: same as before for original series, but the suffix _filtered is added for filtered series
  • series_name: same as before for original series, but the suffix (filtered) is added for filtered series