explore parallelization of code
Various performance improvements can be made in the preprocessor by parallelizing calls, given the functionality provided by the thread
and future
libraries.
Examples of places where processes can be run in parallel is:
- writing of separate files (not the main driver file, but rather the dynamic and static files, epilogue file, params_deriv file, etc...)
- computing pass of static file, dynamic file
- checkPass and computingPass in loop around statements
An exploratory commit was made here: https://git.dynare.org/houtanb/preprocessor/commit/6a22b5c07052306cb5ba617121f2fb335274b6f7 Only the writing of files and the computation passes have been parallelized. More could be done...
It should be noted that, from what I have read, async
should be preferred to thread
as it's both more efficient and avoids some of the pitfals of thread
.
When run on a large model, it saved 17% preprocessing time. The gain in this commit is rather small in nominal terms (just one second), so given the preprocessor is already fast, perhaps this optimization is not worth the effort beyond being interesting to work on....