Commit 6e28c166 authored by Dóra Kocsis's avatar Dóra Kocsis

add mdb_options to initialize_mdbnomics, rename fetch_series

parent 921c7c1e
......@@ -33,14 +33,14 @@ You are then ready to go. A full documentation will come soon.
### Fetch one time series by ID
First, let's assume that we know which series we want to download.
A series identifier (ID) is defined by three values, formatted like this: `provider_code/dataset_code/series_code`.
The `fetch_series` function is used to construct the cell array.
The `mdbnomics` function is used to construct the cell array.
For example, to fetch the time series `EA19.1.0.0.0.ZUTN` from the
[\"Unemployment rate\" [ZUTN] dataset](https://db.nomics.world/AMECO/ZUTN)
belonging to the [AMECO provider](https://db.nomics.world/AMECO).
Example:
>> df_id = fetch_series('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN');
>> df_id = mdbnomics('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN');
The returned data is stored in the `df_id` variable. Its type is a cell array. To display the first 3 rows of the array as a table
(including the column headers), type:
......@@ -77,14 +77,14 @@ Followed by dimensions columns, corresponding to the dimensions of the dataset:
### Fetch two time series by ID
Again, let's assume that we know which series we want to download.
We can reuse the `fetch_series` function, this time with two series codes.
We can reuse the `mdbnomics` function, this time with two series codes.
For example, to fetch the time series `EA19.1.0.0.0.ZUTN` and `DNK.1.0.0.0.ZUTN` from the
[\"Unemployment rate\" [ZUTN] dataset](https://db.nomics.world/AMECO/ZUTN)
belonging to the [AMECO provider](https://db.nomics.world/AMECO).
Example:
>> df_ids = fetch_series('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN'});
>> df_ids = mdbnomics('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN'});
### Fetch time series by code mask
The code mask notation is a very concise way to select one or many time series at once.
......@@ -106,9 +106,9 @@ Given 3 dimensions 'frequency', 'country' and 'indicator', the user can select:
Examples:
>> df_code_mask1 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
>> df_code_mask2 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', '.FR.PCPIEC_WT');
>> df_code_mask3 = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M..PCPIEC_IX+PCPIA_IX', 'max_nb_series', 400);
>> df_code_mask1 = mdbnomics('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
>> df_code_mask2 = mdbnomics('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', '.FR.PCPIEC_WT');
>> df_code_mask3 = mdbnomics('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M..PCPIEC_IX+PCPIA_IX', 'max_nb_series', 400);
### Fetch time series by dimension
Searching by dimension is a less concise way to select time series than using the code mask, but it's universal:
......@@ -118,34 +118,34 @@ and the indicator "Procedures required to start a business - Women (number)" (`i
Example:
>> df_dims = fetch_series('provider_code', 'WB', 'dataset_code', 'DB', 'dimensions', '{"country":["ES","FR","IT"],"indicator":["IC.REG.COST.PC.FE.ZS.DRFN"]}');
>> df_dims = mdbnomics('provider_code', 'WB', 'dataset_code', 'DB', 'dimensions', '{"country":["ES","FR","IT"],"indicator":["IC.REG.COST.PC.FE.ZS.DRFN"]}');
### Fetch time series by API link
When the dimensions, provider, dataset or series codes are unknown, the user can:
* go to the page of a dataset on DBnomics website (eg: [Doing Business](https://db.nomics.world/WB/DB))
* select some dimensions by using the input widgets of the left column
* click on `Copy API link` in the menu of the `Download` button
* use the `fetch_series_by_api_link` function
* use the `mdbnomics` function with the `api_link` parameter.
Example:
>> df_link = fetch_series_by_api_link('https://api.db.nomics.world/v22/series/WB/DB/ENF.CONT.COEN.ATDR-AE?observations=1');
>> df_link = mdbnomics('api_link', 'https://api.db.nomics.world/v22/series/WB/DB/ENF.CONT.COEN.ATDR-AE?observations=1');
### Fetch time series from the cart
On the [cart page](https://db.nomics.world/cart) of the DBnomics website, click on "Copy API link" and copy-paste it as an argument of the fetch_series_by_api_link function.
On the [cart page](https://db.nomics.world/cart) of the DBnomics website, click on "Copy API link" and copy-paste it as an argument of the `mdbnomics` function.
Please note that when you update your cart, you have to copy this link again, because the link itself contains the IDs of the series in the cart.
Example:
>> df_cart = fetch_series_by_api_link('https://api.db.nomics.world/v22/series?series_ids=AMECO%2FZUTN%2FEA19.1.0.0.0.ZUTN&observations=1');
>> df_cart = mdbnomics('api_link', 'https://api.db.nomics.world/v22/series?series_ids=AMECO%2FZUTN%2FEA19.1.0.0.0.ZUTN&observations=1');
### Fetch time series with different frequencies
Example:
>> df_multi_freq = fetch_series('series_ids', {'BEA/NIUnderlyingDetail-U001BC/S315-A',...
'BEA/NIUnderlyingDetail-U001BC/S315-Q',...
'BEA/NIUnderlyingDetail-U001BC/S315-M'});
>> df_multi_freq = mdbnomics('series_ids', {'BEA/NIUnderlyingDetail-U001BC/S315-A',...
'BEA/NIUnderlyingDetail-U001BC/S315-Q',...
'BEA/NIUnderlyingDetail-U001BC/S315-M'});
### Fetch the available datasets of a provider
When fetching series from DBnomics, the user needs to give a provider and a dataset before specifying correct dimensions.
......@@ -223,7 +223,7 @@ Here is an example of how to interpolate two annual time series with a monthly f
Example:
>> filters_ = '[{"code": "interpolate", "parameters": {"frequency": "monthly", "method": "spline"}}]';
>> df_filter = fetch_series('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'dbnomics_filters', filters_);
>> df_filter = mdbnomics('series_ids', 'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'dbnomics_filters', filters_);
The first row of the final cell array changes when filters are used:
* `period_middle_day`: the middle day of `original_period` (can be useful when you compare graphically interpolated series and original ones)
......
......@@ -18,6 +18,8 @@ function initialize_mdbnomics()
% along with Dynare. If not, see <http://www.gnu.org/licenses/>.
% Get the path to the mdbnomics toolbox.
global mdb_options
mdbnomics_src_root = strrep(which('initialize_mdbnomics'), 'initialize_mdbnomics.m', '');
% Set the subfolders to be added in the path.
......@@ -58,4 +60,10 @@ if matlab_ver_less_than('9.1')
addpath([mdbnomics_src_root '/../contrib/jsonlab']);
end
mdb_options.api_base_url = 'https://api.db.nomics.world';
mdb_options.editor_base_url = 'https://editor.nomics.world';
mdb_options.api_version = 22;
mdb_options.editor_version = 1;
assignin('caller', 'mdb_options', mdb_options);
assignin('caller', 'mdbnomics_src_root', mdbnomics_src_root);
function df = fetch_series(varargin) % --*-- Unitary tests --*--
% function fetch_series(varargin)
function df = mdbnomics(varargin) % --*-- Unitary tests --*--
% function mdbnomics(varargin)
% Download time series from DBnomics.
% Returns a cell array.
%
% Examples:
% Fetch one series:
% fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
% fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', '.FR.PCPIEC_WT');
% mdbnomics('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
% mdbnomics('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', '.FR.PCPIEC_WT');
%
% Fetch all the series of a dataset:
% fetch_series('provider_code', 'AMECO', 'dataset_code', 'UVGD', 'max_nb_series', 500);
% mdbnomics('provider_code', 'AMECO', 'dataset_code', 'UVGD', 'max_nb_series', 500);
%
% Fetch many series from different datasets:
% fetch_series('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN', 'IMF/CPI/A.AT.PCPIT_IX'});
% mdbnomics('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN', 'IMF/CPI/A.AT.PCPIT_IX'});
%
% Fetch many series from the same dataset, searching by dimension:
% fetch_series('provider_code','AMECO', 'dataset_code', 'ZUTN', 'dimensions', '{"geo":["dnk"]}');
% mdbnomics('provider_code','AMECO', 'dataset_code', 'ZUTN', 'dimensions', '{"geo":["dnk"]}');
%
% Fetch series given an "API link" URL.
% "API link" URLs can be found on DBnomics web site (https://db.nomics.world/) on dataset or series pages using "Download" buttons.
% mdbnomics('api_link', 'https://api.db.nomics.world/v22/series?series_ids=AMECO%2FZUTN%2FEA19.1.0.0.0.ZUTN&observations=1');
%
% POSSIBLE PARAMETERS
% provider_code [string] the code of the dataset provider.
......@@ -26,6 +30,7 @@ function df = fetch_series(varargin) % --*-- Unitary tests --*--
% max_nb_series [integer] maximum number of series requested by the API. If not provided, a default value of 50 series will be used.
% api_base_url [string] the base URL used for API requests. If not provided, a default value of: 'https://api.db.nomics.world/v22/' will be used.
% dbnomics_filters [char] filters to apply on the requested series. If provided it must be a string formatted like: '[{"code": "interpolate", "parameters": {"frequency": "monthly", "method": "spline"}}]'.
% api_link [char] fetch series given an "API link" URL.
%
% OUTPUTS
% df
......@@ -52,8 +57,10 @@ function df = fetch_series(varargin) % --*-- Unitary tests --*--
% You should have received a copy of the GNU General Public License
% along with Dynare. If not, see <http://www.gnu.org/licenses/>.
default_api_base_url = 'https://api.db.nomics.world/v22/';
default_editor_base_url = 'https://editor.nomics.world/api/v1/';
global mdb_options
default_api_base_url = sprintf('%s/v%d/', mdb_options.api_base_url, mdb_options.api_version);
default_editor_base_url = sprintf('%s/api/v%d/', mdb_options.editor_base_url, mdb_options.editor_version);
p = inputParser;
validStringInput = @(x) ischar(x) || iscellstr(x);
......@@ -65,6 +72,7 @@ p.addParameter('series_ids', '',validStringInput);
p.addParameter('max_nb_series', NaN, @isnumeric);
p.addParameter('api_base_url', default_api_base_url, validStringInput);
p.addParameter('dbnomics_filters', '', @ischar);
p.addParameter('api_link', '', @ischar);
p.KeepUnmatched = false;
p.parse(varargin{:});
......@@ -85,7 +93,7 @@ end
series_base_url = [p.Results.api_base_url 'series'];
if isa(p.Results.dimensions, 'function_handle') && isempty(p.Results.series_code) && isempty(p.Results.series_ids)
if isempty(p.Results.provider_code) || isempty(p.Results.dataset_code)
if (isempty(p.Results.provider_code) || isempty(p.Results.dataset_code)) && isempty(p.Results.api_link)
error('When you don''t use dimensions, you must specifiy provider_code and dataset_code.');
end
api_link = sprintf('%s/%s/%s?observations=1', series_base_url, p.Results.provider_code, p.Results.dataset_code);
......@@ -116,12 +124,17 @@ if ~isempty(p.Results.series_ids)
end
api_link = sprintf('%s?observations=1&series_ids=%s', series_base_url, series_ids);
end
if ~isempty(p.Results.api_link)
api_link = p.Results.api_link;
end
df = fetch_series_by_api_link(api_link, p.Results.dbnomics_filters, p.Results.max_nb_series, default_editor_base_url);
end
%@test:1 % test_fetch_series_by_code
%$ try
%$ df = fetch_series('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'series_code', 'EA19.1.0.0.0.ZUTN');
%$ df = mdbnomics('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'series_code', 'EA19.1.0.0.0.ZUTN');
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -141,7 +154,7 @@ end
%@test:2 % test_fetch_series_by_code_mask
%$ try
%$ df = fetch_series('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
%$ df = mdbnomics('provider_code', 'IMF', 'dataset_code', 'CPI', 'series_code', 'M.FR+DE.PCPIEC_IX+PCPIA_IX');
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -160,7 +173,7 @@ end
%@test:3 % test_fetch_series_by_code_mask_with_plus
%$ try
%$ df = fetch_series('provider_code', 'SCB', 'dataset_code', 'AKIAM', 'series_code', '"J+K"+"G+H".AM0301C1');
%$ df = mdbnomics('provider_code', 'SCB', 'dataset_code', 'AKIAM', 'series_code', '"J+K"+"G+H".AM0301C1');
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -179,7 +192,7 @@ end
%@test:4 % test_fetch_series_by_dimension
%$ try
%$ df = fetch_series('provider_code','WB','dataset_code','DB', 'dimensions', '{"country":["ES","FR","IT"],"indicator":["IC.REG.COST.PC.FE.ZS.DRFN"]}');
%$ df = mdbnomics('provider_code','WB','dataset_code','DB', 'dimensions', '{"country":["ES","FR","IT"],"indicator":["IC.REG.COST.PC.FE.ZS.DRFN"]}');
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -198,7 +211,7 @@ end
%@test:5 % test_fetch_series_by_id
%$ try
%$ df = fetch_series('series_ids','AMECO/ZUTN/EA19.1.0.0.0.ZUTN');
%$ df = mdbnomics('series_ids','AMECO/ZUTN/EA19.1.0.0.0.ZUTN');
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -218,7 +231,7 @@ end
%@test:6 % test_fetch_series_by_ids_in_different_datasets
%$ try
%$ df = fetch_series('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'BIS/cbs/Q.S.5A.4B.F.B.A.A.LC1.A.1C'});
%$ df = mdbnomics('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN', 'BIS/cbs/Q.S.5A.4B.F.B.A.A.LC1.A.1C'});
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -244,7 +257,7 @@ end
%@test:7 % test_fetch_series_by_ids_in_same_dataset
%$ try
%$ df = fetch_series('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN',...
%$ df = mdbnomics('series_ids', {'AMECO/ZUTN/EA19.1.0.0.0.ZUTN',...
%$ 'AMECO/ZUTN/DNK.1.0.0.0.ZUTN'});
%$ t(1) = 1;
%$ catch
......@@ -267,7 +280,7 @@ end
%@test:8 % test_fetch_series_of_dataset
%$ try
%$ df = fetch_series('provider_code', 'AMECO', 'dataset_code', 'ZUTN');
%$ df = mdbnomics('provider_code', 'AMECO', 'dataset_code', 'ZUTN');
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -287,7 +300,7 @@ end
%@test:9 % test_fetch_series_with_filter_on_one_series
%$ try
%$ filters_ = '[{"code": "interpolate", "parameters": {"frequency": "monthly", "method": "spline"}}]';
%$ df = fetch_series('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'series_code', 'DEU.1.0.0.0.ZUTN', 'dbnomics_filters', filters_);
%$ df = mdbnomics('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'series_code', 'DEU.1.0.0.0.ZUTN', 'dbnomics_filters', filters_);
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -312,7 +325,7 @@ end
%@test:10 % test_fetch_series_with_max_nb_series
%$ try
%$ df = fetch_series('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'max_nb_series',20);
%$ df = mdbnomics('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'max_nb_series',20);
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -332,7 +345,7 @@ end
%@test:11 % test_fetch_series_with_na_values
%$ try
%$ df = fetch_series('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'series_code', 'DEU.1.0.0.0.ZUTN');
%$ df = mdbnomics('provider_code', 'AMECO', 'dataset_code', 'ZUTN', 'series_code', 'DEU.1.0.0.0.ZUTN');
%$ t(1) = 1;
%$ catch
%$ t = 0;
......@@ -351,3 +364,22 @@ end
%$
%$ T = all(t);
%@eof:11
%@test:12 % test_fetch_series_by_api_link
%$ try
%$ df = mdbnomics('api_link', 'https://api.db.nomics.world/v22/series/BIS/long_pp?limit=1000&offset=0&q=&observations=1&align_periods=1&dimensions=%7B%7D');
%$ t(1) = 1;
%$ catch
%$ t = 0;
%$ end
%$
%$ if t(1)
%$ t(2) = dassert(length(unique(df(2:end,2))),1);
%$ t(3) = dassert(df(2,2), {'BIS'});
%$ t(4) = dassert(length(unique(df(2:end,3))),1);
%$ t(5) = dassert(df(2,3), {'long_pp'});
%$ t(6) = dassert(length(unique(df(2:end,5))),23);
%$ end
%$
%$ T = all(t);
%@eof:12
......@@ -32,8 +32,7 @@ function datasets = mdbnomics_datasets(varargin) % --*-- Unitary tests --*--
% You should have received a copy of the GNU General Public License
% along with Dynare. If not, see <http://www.gnu.org/licenses/>.
api_base_url = 'https://api.db.nomics.world';
api_version = 22;
global mdb_options
p = inputParser;
validStringInput = @(x) ischar(x) || iscellstr(x);
......@@ -55,7 +54,7 @@ end
datasets = struct();
for i = 1:numel(provider_code)
pc = provider_code{i};
provider_page = sprintf('%s/v%d/providers/%s', api_base_url, api_version, pc);
provider_page = sprintf('%s/v%d/providers/%s', mdb_options.api_base_url, mdb_options.api_version, pc);
provider_info = webread(provider_page);
provider_info = provider_info.category_tree;
code = [];
......
......@@ -34,8 +34,7 @@ function dimensions = mdbnomics_dimensions(varargin) % --*-- Unitary tests --*--
% You should have received a copy of the GNU General Public License
% along with Dynare. If not, see <http://www.gnu.org/licenses/>.
api_base_url = 'https://api.db.nomics.world';
api_version = 22;
global mdb_options
p = inputParser;
validStringInput = @(x) ischar(x) || iscellstr(x);
......@@ -79,7 +78,7 @@ dimensions = struct();
for i = 1:numel(provider_code)
pc = provider_code{i};
dc = dataset_code{i};
dataset_page = sprintf('%s/v%d/datasets/%s/%s', api_base_url, api_version, pc, dc);
dataset_page = sprintf('%s/v%d/datasets/%s/%s', mdb_options.api_base_url, mdb_options.api_version, pc, dc);
dataset_info = webread(dataset_page);
dataset_name = sprintf('%s_%s', pc, dc);
......
......@@ -41,8 +41,7 @@ function series = mdbnomics_series(varargin) % --*-- Unitary tests --*--
% You should have received a copy of the GNU General Public License
% along with Dynare. If not, see <http://www.gnu.org/licenses/>.
api_base_url = 'https://api.db.nomics.world';
api_version = 22;
global mdb_options
p = inputParser;
validStringInput = @(x) ischar(x) || iscellstr(x);
......@@ -101,7 +100,7 @@ series = struct();
for i = 1:numel(provider_code)
pc = provider_code{i};
dc = dataset_code{i};
dataset_page = sprintf('%s/v%d/series/%s/%s', api_base_url, api_version, pc, dc);
dataset_page = sprintf('%s/v%d/series/%s/%s', mdb_options.api_base_url, mdb_options.api_version, pc, dc);
if exist('db_query', 'var')
dataset_page = sprintf('%s?q=%s', dataset_page, db_query{i});
end
......
......@@ -37,7 +37,10 @@ function df = fetch_series_by_api_link(api_link, varargin) % --*-- Unitary tests
%
% You should have received a copy of the GNU General Public License
% along with Dynare. If not, see <http://www.gnu.org/licenses/>.
default_editor_base_url = 'https://editor.nomics.world/api/v1/';
global mdb_options
default_editor_base_url = sprintf('%s/api/v%d/', mdb_options.editor_base_url, mdb_options.editor_version);
p = inputParser;
validStringInput = @(x) ischar(x) || iscellstr(x);
......
......@@ -30,15 +30,14 @@ function providers = mdbnomics_providers(varargin) % --*-- Unitary tests --*--
% You should have received a copy of the GNU General Public License
% along with Dynare. If not, see <http://www.gnu.org/licenses/>.
api_base_url = 'https://api.db.nomics.world';
api_version = 22;
global mdb_options
p = inputParser;
p.addParameter('code', false, @islogical);
p.KeepUnmatched = false;
p.parse(varargin{:});
providers_url = sprintf('%s/v%d/providers', api_base_url, api_version);
providers_url = sprintf('%s/v%d/providers', mdb_options.api_base_url, mdb_options.api_version);
response = webread(providers_url);
if p.Results.code
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment