Inconsistent behavior of dseries2Q() depending on the input shape

Hello everyone,

I'm encountering a suprising issue with dseries2Q() where its behavior seems to be inconsistent depending on the form of the inputed data while the function "sees" there seems to be an issue with said data. Specifically:

I'm using it to transform monthly data to quarterly frequency using the 'arithmetic average' method and these three inputs give different results:

Prepararing my data

toto_m = dseries([1 2 3 4 5 NaN]', dates('2020m10'):dates('2021m3'), 'toto');
titi_m = dseries([1 2 3 4 5]', dates('2020m10'):dates('2021m2'), 'titi');
tata_m = dseries([1 2 3 4]', dates('2020m10'):dates('2021m1'), 'tata');

Applying the function

toto_q = dseries2Q(toto_m, 'arithmetic-average');
titi_q = dseries2Q(titi_m, 'arithmetic-average');
tata_q = dseries2Q(tata_m, 'arithmetic-average');

I'm having an issue with titi and tata the most and think what the function does on toto is the right thing to do when facing incomplete data for 2021Q1. Although it seems to do it for the wrong reason (aka computing the average on some numbers + NaN gives NaN).

• In the toto case: the function gives a NaN for 2021Q1, seemingly because it computes the average on [4, 5, NaN], and does not warn about imcomplete data.

• In the tata case: the function happily provides the data in 2021M1 as the number for 2021Q1 while warning that the data is imcomplete for that quarter.

• In the titi case: the function averages 2021M1 and 2021M2 to give 2021Q1, with a warning about imcomplete data for 2021Q1 too.

Idealy, I do think the function should always refuse to compute the average on incomplete quarters (because it would be inventing data else?) and give a NaN instead.

added Madysson in dynare label

changed title from Inconsistant behavior of dseries2Q() depending on the input shape to Inconsistent behavior of dseries2Q() depending on the input shape

added bug in dynare label

moved from dynare#1928 (moved)

I added a warning if data are missing (with NaN values for the observations). I also added a second output argument (a flag equal to true iff some data are missing).

The commits are 979a570a and 02c3c672

@Ugo Do you consider the issue to be closed?

I'm afraid not. Unless I'm missing something, the code I get from dynare-7-unstable-2024-06-20-1930-d51934a0 (7zip archive) does not appear to work as intended:

If I try to get @stepan-a's new output, the 'flag', for the calls provided as examples in this ticket, I get the following:

K>> toto_q = dseries2Q(toto_m, 'arithmetic-average');

Warning: Not all subperiods are available in 2021Q1. In dseries2Q (line 60) [...]

That's fine, and an improvement. But, trying:

K>> [toto_q, myflag] = dseries2Q(toto_m, 'arithmetic-average');

Warning: Not all subperiods are available in 2021Q1. In dseries2Q (line 60) In MainBaseSource (line 693) In MainBase (line 12) In MainBaseCond (line 56) Output argument "wflag" (and maybe others) not assigned during call to "dseries2Q".

(in red in the Command window, suggesting an error.)

Am I missing something obvious ?

@stepan-a As discussed this morning: we now get too many warnings when the dseries contains many NaNs in the future.

As shown in this example:

tutu_m = dseries([1 NaN NaN NaN NaN NaN NaN]', dates('2020m10'), 'tutu');
tutu_q = dseries2Q(tutu_m, 'arithmetic-average');

Gives me:

Warning: Not all subperiods are available in 2020Q4. 
> In dseries2Q (line 60) 
Warning: Not all subperiods are available in 2021Q1. 
> In dseries2Q (line 60) 
Warning: Not all months are available in 2021Q2. 
> In dseries2Q (line 57) 
Warning: Not all subperiods are available in 2021Q2. 
> In dseries2Q (line 60)  

We would only need to be warned once (for the first incomplete period? or a general warning about periods starting to misss at that date).