Interface for irf matching

changed milestone to %6.x

mentioned in merge request dynare!2191 (merged)

removed milestone %6.x

I find that too cumbersome. Most of the time, people will specify full IRFs for a shock with a diagonal weighting. I would propose to allow for just one block following the perfect foresight shocks block syntax:

matched_irfs;
shock eR; var y;
periods 1:10;
values 1,2,3,4,5,5,4,3,2,1;
weights 20; 
shock eA; var c;
periods 4;
values 0.01;
weights (1/empIrfsCov(3,3))
shock eA; var i;
periods 1;
values (empIrfs(3,1));
weights 10;
end;

translating to M_.matched_irfs.values with

{'y'}    {'eR'}    {[1:10]}    {[1,2,3,4,5,5,4,3,2,1]}
{'c'}    {'eA'}    {[4]}    {[0.01]}
{'i'}    {'eA'}    {[4]}    {[empIrfs(3,1)]}

and M_.matched_irfs.weights with

{'y'}    {'eR'}    {[1:10]}    {'y'}    {'eR'}    {[1:10]}    {[20]}
{'c'}    {'eA'}    {[4]}    {'c'}    {'eA'}    {[4]}    {[(1/empIrfsCov(3,3))]}
{'i'}    {'eA'}    {[2]}    {'i'}    {'eA'}    {[2]}    {[10]}

We should then also have a matched_irfs_weights block for setting additional fields of M_.matched_irfs.weights:

matched_irfs_weights;
i(1), eA, y(2), eR, 10;
y(1:10), eR, y(2:5), eR, 20;
y(2:20), eR, y(6:10), eR, (WEIGHTMAT);
end;

translating to

{'i'}    {'eA'}    {[1]}    {'y'}    {'eR'}    {[2]}    {[10]}
{'y'}    {'eR'}    {[1:10]}    {'y'}    {'eR'}    {[2:5]}    {[20]}
{'y'}    {'eR'}    {[2:20]}    {'y'}    {'eR'}    {[6:10]}    {[WEIGHTMAT]}

For multiple blocks, concatenation should be the default, with both blocks also taking the overwrite option.

I see your point and do like the intuitiveness of your proposal. We can very well go for this one; I don't really have a better idea here and was just aiming for something simple and clearly structured (without flexibility). Anyways, I do see the following difficulties:

Implementation in preprocessor: This might be quite hard to implement in the preprocessor due to its flexibility.
MATLAB code complexity: The MATLAB code would need to catch all cases as periods, values, and weights might have different lengths (integers, vectors) or elements might be missing (weights). Values would then need to be duplicated or initialized. In principle, this can all be done, but I wonder if having too much flexibility in the interface might actually confuse users and is more error-prone? But you have more insight into this.
Existing irf_calibration: We already have irf_calibration which is very similar to this blocks, so we would have a discrepancy here.
Cumbersome process: Regarding my proposal being cumbersome: the same applies to irf_calibration. That's why in the manual, we provide examples of how to use the preprocessor (particularly @#for) to make this less cumbersome.
Personal Preference on Syntax: Personally, I don't like the way we specify the shocks block by using semicolons all the time and only via the keywords the preprocessor knows when an entry is actually finished. I prefer having everything just separated by commas, with a semicolon being the end of this entry/line (e.g., as in estimated_params or irf_calibration). But this is just my preference.

Maybe @rattoma or @MichelJuillard can also provide a suggestion on what they find more intuitive and less cumbersome for such blocks given their experience in designing interfaces.

I don't see the high degree of complexity. It's deliberately similar to the shocks block. But here the respective entries should be passed directly to a structure without further processing or consistency checks.
That type of consistency checking would need to be done in any case and is easier to do and maintain in Matlab than in the preprocessor. In the original proposal, weights can also be missing. The only different part relative to shocks is the implicit expansion from a scalar to a vector or matrix if the length of values or weights does not coincide with the one in periods. But these are straightforward dimension checks in Matlab: error out whenever matrix dimensions are not conformable unless we are filling a vector/matrix with a scalar.
I have rarely seen people use irf_calibration and would not consider it a meaningful precedent.
irf_calibration usually does not involve a large number of matched IRFs. I understand the point about macroprocessing, but it's still very offputting for average users.
That's true. The tradeoff is between the simplicity of specifying everything just with commas separating the entries and necessarily requiring two blocks.

An alternative may be the convex combination:

matched_irfs;
eR, y(1:10), 1,2,3,4,5,5,4,3,2,1, weights 20; 
eA, c(4), 0.01, weights (1/empIrfsCov(3,3));
eA, i(1) (empIrfs(3,1)), weights 10;
end;

Alternatively, we may require a semicolon before the weights keyword. However, that new proposal may be harder to implement as it deviates more from existing code and would probably put more burden on @sebastien.

I came to prefer functional expressions with keyword names. Keywords may have default values. It is more verbose but I find it easier to read. Something like

matched_irfs(shock = eR, var = y, periods = 1:10, values = [1,2,3,4,5,5,4,3,2,1], weights = 20); 
matched_irfs(shock = eA, var = c, periods = 4, values = 0.01, weights = (1/empIrfsCov(3,3)));
matched_irfs(shock = eA, var = i, periods = 1, values = (empIrfs(3,1)), weights = 10);

The weight keyword could have a default value of 1.

First, I don’t think the complexity of the implementation should be the deciding factor in this discussion (and in any case, all the proposals have more or less the same complexity).

I think I tend to prefer @JohannesPfeifer’s proposal, which is the most Dynare-ish syntax (I would just replace shock by varexo, to avoid introducing a new keyword).

However @wmutschl’s proposal seems also acceptable to me.

If I understand correctly, the choice between the two depends essentially on whether off-diagonal weighting elements will be frequently specified or not. If the weighting matrix is diagonal in most cases, @JohannesPfeifer’s syntax is more readable and more compact, since it has everything in the same place (the weight being just next to the value). On the other hand, that syntax is a bit cumbersome for off-diagonal elements (which are specified differently than diagonal elements).

Lastly, I find @MichelJuillard’s syntax interesting but too far from the existing Dynare syntax, which is not functional for declarative elements.

I forgot to mention that I’m referring to @JohannesPfeifer’s initial proposal, not the amended one (which is not very Dynare-ish, and which I don’t like much).

An additional issue I see with @MichelJuillard's proposal is that it's not clear which commands belong together and need to be concatenated. If there is only one experiment in the mod-file, that may be fine. But what about overwriting or clearing previous commands?

Okay, so let's go with the following:

matched_irfs block:

matched_irfs;
var y;
varexo eR;
periods 1:10;
values 1,2,3,4,5,5,4,3,2,1;
weights 20;

varexo eA;
var c;
periods 4;
values 0.01;
weights (1/empIrfsCov(3,3));

var R;
varexo eR;
periods 1 2 3;
values (empIrfs(4,1)),(empIrfs(4,2)),(empIrfs(4,5));
weights (1/empIrfsCov(4,4)) (1/empIrfsCov(5,5)) (1/empIrfsCov(6,6));

varexo eA;
var i;
periods 1,3,4;
values (empIrfs(3,1)),(empIrfs(3,2)),(empIrfs(3,5));
end;

How exactly do we determine when a new entry starts? Whenever the preprocessor finds a new var or varexo?

And for fine-tuning the off-diagonal elements of the weighting matrix (this will almost never be used):

matched_irfs_weights;
i(1), eA, y(2), eR, 10;
y(1:10), eR, y(2:5), eR, 20;
y(2:20), eR, y(6:10), eR, (WEIGHTMAT);
end;

Yes, a var or varexo keyword triggers a new entry (as is done in shocks, even though there is only var).

I will implement this next week, so that we can have it in 6.x.

@wmutschl So if I understand correctly, if the weights keyword is absent from a stanza, it should default to 1, as suggested by @MichelJuillard?

Yes, the default should be 1, but I would rather implement that on the Matlab end by initializing the weighting matrix with eye. The preprocessor should simply leave the fourth column entry of M_.matched_irfs.values empty.

This is already the current implementation in the MATLAB code: the weighting matrix is the identity and only if the user specified anything then we'll adjust that entry in the MATLAB code. Ideally, the keyword ´weight´ is optional. And yes, let's use ´weight´ and not ´weights´.

Actually weights may be better, because it’s symmetrical with periods and values, the idea being that you can specify several of each.

Indeed!

Also, I would rather have the preprocessor put all the weights in the same matrix for consumption by the MATLAB side. Mixing values and weights make sense at the interface level, but not so much at the MATLAB level.

In any case, we can easily adjust the MATLAB side of things later. The important decision at this stage is the interface.

If one of the blocks has the overwrite option, should this reset only the specific block (matched_irfs or matched_irfs_weights), or both?

I would suggest only the specific blocks and we will take care of conflicts between matched_irfs and matched_irfs_weights in the MATLAB code. Actually, the code already does that; that is, the idea being that matched_irfs is the more important block. So say if a user first specified both blocks using three observables and then overwrites only the matched_irfs block and uses 2 observables, then we will disregard any entries in matched_irfs_weights corresponding to the third observable.

Do you want the preprocessor to check that symbol names of endogenous and exogenous are valid, and then to translate them into integer indices?

From a user perspective, the advantage is that the error message will be more precise (line and column of invalid symbol).

Another possibility is to have the preprocessor do the check, but still pass the symbol as strings and not integer indices.

Interface for irf matching

Designs

Child items ...

Activity