diff git a/dynarepreprocessorwjson.rst b/dynarepreprocessorwjson.rst
index 4847f5cec262b42a07a8fbb6e7c6478558be2ec9..1a717ef8f642cd3e1c178cbf2b9d182fb13a8437 100644
 a/dynarepreprocessorwjson.rst
+++ b/dynarepreprocessorwjson.rst
@@ 1,7 +1,7 @@
Using JSON Output from the Dynare Preprocessor
##############################################
:date: 20200124
+:date: 20200123
:tags: Dynare, Preprocessor, JSON, MATLAB
:category: Dynare
:slug: dynarepreprocessorwjson
@@ 459,28 +459,29 @@ Together, these functions return matrices corresponding to the observed
variable, ``Y``, the regressors, ``X``, the constant term, ``lhssub``, and the
first and last periods of the observed data (``fp`` and ``lp``).
Parsing proceeds as follows in ``parse_ols_style_equation.m``. First, the
function arguments are verified (lines 3866). After this, we know that the LHS
contains either a ``VariableNode`` or a ``UnaryOpNode``.
+Parsing proceeds as follows in ``parse_ols_style_equation.m``. After, the
+function arguments are verified at the beginning of the function (lines 3866),
+we know that the LHS contains either a ``VariableNode`` or a ``UnaryOpNode``.
.. note:: Several different types of nodes that can appear in the AST,
corresponding to the types of operations that are available in a
``.mod`` file equation. The nodes are:
 #. ``NumConstNode``: nonnegative integers or doubles
+ #. ``NumConstNode``: e.g. nonnegative integer or double
#. ``VariableNode``: endogenous, exogenous, or parameter
#. ``UnaryOpNode``: unary operation on a node, e.g. ``log``, ``abs``,
unary minus, ...
#. ``BinaryOpNode``: binary operation on a node, e.g. arithmetic
 operations, ``min``, ``max``, ``=``, comparison operators, ...
+ operations, ``max``, comparison operators
#. ``TrinaryOpNode``: trinary operation on a node, e.g. ``normcdf``,
``normpdf``
#. ``ExternalFunctionNode``: external function node
Assured that the type of the LHS is valid for OLS, the value of ``Y`` is set by
evaluating the LHS of the specified equation. Given that all equations are
represented as ``BinaryOpNode``'s (the two arguments being the LHS, ``arg1``,
and the RHS, ``arg2``), the call to evaluate the LHS of the equation is:
+represented as ``BinaryOpNodes`` (the two arguments being the LHS, ``arg1``,
+and the RHS, ``arg2``), the call to evaluate the LHS of the equation is as
+follows:
.. codeblock:: MATLAB
:linenos: inline
@@ 554,19 +555,20 @@ this node given the ``dseries`` contained in ``ds``.
end
In the evaluation of the observed variable (``X`` in this local function, ``Y``
in the main routine), what's important to us is lines 313342 as we know it's
either a ``VariableNode`` or a ``UnaryOpNode``. Regardless of the type of node
it is, it's evaluated using the ``dseries`` (``ds``) and the corresponding
``dseries`` vector is returned. In our case, the interest rate ``r`` is
evaluated on line 317 by looking up its value in ``ds``.
+in the main routine), what's important to us is lines 313342. In particular,
+our OLS routine handles an observable variable declared as an endogenous or
+exogenous variable, with or without a unary operator applied to it. Given the
+case it falls in (``VariableNode`` or ``UnaryOpNode``), the LHS (``node``) is
+evaluated using the ``dseries`` (``ds``) and the corresponding ``dseries``
+vector is returned.
Though our work for ``Y`` was easy, computing the matrix of regressors and the
vector of constants will be a bit more involved.
+Though our work for ``Y`` was easy, the work to compute the matrix of
+regressors and the vector of constants, will take a bit more work.
Back in the main function, before we can create the matrix of regressors ``X``,
we decompose the RHS (``arg2``) of the equation into additive terms, storing
them in a cell array called ``terms``. We do this by calling the locally
defined function:
+defined function ``decomposeAdditiveTerms``:
.. codeblock:: MATLAB
:linenos: inline
@@ 601,10 +603,11 @@ As you can see, ``decomposeAdditiveTerms`` is a recursive, treetraversal
function that breaks down terms separated by ``+`` or ````, storing them in
the return value, ``terms``. Each cell in the return value is comprised of a
pair of elements: the root node of the subtree representing the additive node
and the sign preceding this node (``1`` or ``1``).
+and the sign in preceding this node. That sign will be used in the construction
+of the matrix, setting the sign of the data accordingly.
Given the equation we want to estimate (shown here from
``Smets_Wouters_2007.mod`` as a reminder):
+``Smets_Wouters_2007.mod`` as a reminder),
.. include:: sw2007/Smets_Wouters_2007.mod
:code:
@@ 625,78 +628,13 @@ Hence, in this case, 4 nodes of ``terms`` contain ``BinaryOpNode``'s
(corresponding to the binary operation ``*``) and one term contains a
``VariableNode``.
To understand how the terms are stored, one need only look at the following
output for the first term, ``crpiMcrpiXcrr*pinf``:

.. codeblock:: text

 >> terms

 terms =

 1x5 cell array

 {1x2 cell} {1x2 cell} {1x2 cell} {1x2 cell} {1x2 cell}

 >> terms{1}

 ans =

 1x2 cell array

 {1x1 struct} {[1]}

 >> terms{1}{1}

 ans =

 struct with fields:

 node_type: 'BinaryOpNode'
 op: '*'
 arg1: [1x1 struct]
 arg2: [1x1 struct]

 >> terms{1}{1}.arg1

 ans =

 struct with fields:

 node_type: 'VariableNode'
 name: 'pinf'
 type: 'endogenous'
 lag: 0

 >> terms{1}{1}.arg2

 ans =

 struct with fields:

 node_type: 'VariableNode'
 name: 'crpiMcrpiXcrr'
 type: 'parameter'
 lag: 0

Here, the second element of ``terms{1}`` is ``1`` as it is not preceded by a
minus sign. The node itself is a ``BinaryOpNode`` as it represents the
multiplication of an endogenous variable (the first argument of the node) and a
parameter (the second argument of the node).

.. note:: The elements of ``terms`` are not necessarily in the same order as
written in the equation in the ``.mod`` file. This is because, for
efficiency reasons (e.g. node sharing), the AST created by the
 preprocessor does not guarantee this ordering. By the same token, the
 elements of a node are not guaranteed to be in the same order as they
 appear in the ``.mod`` file, as you can see above where the first
 argument of the ``BinaryOpNode`` in ``terms{1}{1}`` is the endogenous
 variable ``pinf`` whereas ``pinf`` appears second in the
 multiplication as written in the ``.mod`` file:
 ``crpiMcrpiXcrr*pinf``.
+ preprocessor does not guarantee this ordering.
Now that we have ``terms`` set, we can construct the regressor matrix ``X`` by
entering the loop on line 75 of ``parse_ols_style_equation.m``:
+Now that we have ``terms`` set, we can enter the loop in the main function of
+``parse_ols_style_equation.m`` that constructs the regressor matrix ``X``:
.. codeblock:: MATLAB
:linenos: inline
@@ 780,47 +718,44 @@ entering the loop on line 75 of ``parse_ols_style_equation.m``:
X = [X Xtmp];
end
Entering the loop, we set 3 variables: ``Xtmp``, ``node_to_parse``, and
``node_sign``. ``Xtmp`` is where we will construct the regressor column to be
appended to ``X`` at the end of each loop. ``node_to_parse`` and ``node_sign``
are simply the corresponding parts of the pair stored in each ``terms`` cell,
as described above.
+When the loop starts, certain variables are set. ``Xtmp`` is where we will
+construct the regressor column to be appended to ``X`` at the end of each
+loop. ``node_to_parse`` and ``node_sign`` are simply the corresponding parts of
+the pair stored in each ``terms`` cell, as described above.
With these variables set, we take different actions, depending on the type of
node encountered in ``node_to_parse``. The following subsections explain those
actions.
Condition 1: VariableNode (lines 7997)
```````````````````````````````````````
+VariableNode
+````````````````
If ``node_to_parse`` is a ``VariableNode`` it is an additive variable in the
equation. If it was declared as a parameter in the ``.mod`` file, then it's the
intercept of the equation. It is thus stored in ``Xtmp`` with the value of the
parameter at every period. If it's a lone exogenous variable that is not
present in ``ds``, we treat it as the residual. If there are more than one such
variable, then it's an error. If it's an endogenous variable or an exogenous
variable present in ``ds``, we add it to the ``dseries`` ``lhssub``, a
+parameter at every period. If it's a lone exogenous variable, we treat it as
+the residual. If there are more than one such variable, then it's an error. If
+it's an endogenous variable, we add it to the ``dseries`` ``lhssub``, a
``dseries`` that will be subtracted from the LHS before returning. If the type
of the ``node_to_parse`` doesn't fall into any of these categories, parsing
ends with an error.
Condition 2: UnaryOpNode (lines 98101)
```````````````````````````````````````
+UnaryOpNode
+```````````````
If ``node_to_parse`` is a ``UnaryOpNode``, we evaluate it and add it to the
``lhssub`` variable to be subtracted from the LHS.
Condition 3: BinaryOpNode with division operator (lines 102108)
````````````````````````````````````````````````````````````````
+BinaryOpNode with division operator
+```````````````````````````````````
In this case, if a parameter is found in this expression, we end with a parsing
error. Otherwise, we evaluate the node and add it to ``lhssub``.
Condition 4: BinaryOpNode with multiplication operator (lines 109137)
``````````````````````````````````````````````````````````````````````
+BinaryOpNode with multiplication operator
+`````````````````````````````````````````
In this case, we parse the ``node_to_parse``, by calling the local function
``parseTimesNode``:
+In this case, we parse the ``node_to_parse``, by calling the local function ``parseTimesNode``
.. codeblock:: MATLAB
:linenos: inline
@@ 856,24 +791,23 @@ In this case, we parse the ``node_to_parse``, by calling the local function
end
end
This function returns the ``dseries`` vector/matrix ``X`` corresponding to the
regressors found. It can handle parsing expressions of additivelyseparated
parameters multiplied by an expression of variables. Each of the additively
separated parameters in ``pterms`` corresponds to one column of the returned
vector/matrix ``X``.
+This function handles returns the ``dseries`` vector/matrix ``X`` corresponding
+to the regressors found. This function can handle parsing expressions of
+additivelyseparated parameters multiplied by an expression of variables. Each
+of the additively separated parameters in ``pterms`` corresponds to one column
+of the returned vector/matrix ``X``.
Returning from this function, we have ``Xtmp`` corresponding to the evaluated
variables and ``names`` corresponding to the parameter names. In the code that
follows (lines 113137), we add any columns of ``Xtmp`` that were multiplied by
a constant to ``lhssub`` and then remove this column from ``Xtmp``.
+follows, we add any columns of ``Xtmp`` that were multiplied by a constant to
+``lhssub`` and then remove this column from ``Xtmp``.
Condition 5: Otherwise (line 139)
`````````````````````````````````
+Otherwise
+`````````
If none of the previous 4 conditions were satisfied, we have encountered a
parsing error.
+We have encountered a parsing error.
End of loop (lines 141150)
+End of loop, lines (143151)
````````````````````````````
At the end of the loop, we combine the temporary vector/matrix of regressors,
@@ 913,8 +847,7 @@ assigned to the variables ``fp`` and ``lp``:
lp = min(lp, lhssub.lastobservedperiod);
end
Finally, the ``Y``, ``X``, and ``lhssub`` datasets are adjusted given ``fp``
and ``lp``:
+Finally, the ``Y``, ``X``, and ``lhssub`` datasets are adjusted given ``fp`` and ``lp``:
.. codeblock:: MATLAB
:linenos: inline
@@ 950,12 +883,12 @@ the first and last observed period of the estimation range for each equation.
[Y, lhssub, X, fp, lp] = common_parsing(ds(ds_range), ast, true, param_names);
We loop over these cell arrays, running our estimation for each equation:
:math:`\hat{\beta} = (X'X)^{1}X'Y`. [#]_ The output is set to the ``ols``
field of the standard Dynare MATLAB/Octave output structure, ``oo_``. Each
subfield of ``oo_.ols`` corresponds to an equation tag that was either
provided by the user or created by ``dyn_ols.m``. In this case, they will be
saved to ``oo_.ols.taylor_rule``. Furthermore, the estimated parameter values
are set in ``M_``.
+:math:`\hat{\beta} = (X'X)^{1}X'Y`. [#]_ The output is set to the standard
+Dynare MATLAB/Octave output structure ``oo_`` in the ``ols`` field. Each
+subfield of ``ols`` corresponds to an equation tag that was either provided by
+the user or created by ``dyn_ols.m``. In this case, they will be saved to
+``oo_.ols.taylor_rule``. Furthermore, the estimated parameter values are set in
+``M_``.
.. codeblock:: MATLAB
:linenos: inline