Skip to content
Snippets Groups Projects
New look: Off

X-13ARIMA-SEATS routines can crash depending on the current directory

    • Closed Issue created by Pierre Aldama @PierreAldama

    While working on building our dataset (in our case, for seasonal adjustment), we observed that X-13ARIMA-SEATS routines can crash when the current directory is different from the path of the Matlab script where we call the X13 routines. This behavior seems to be eratic: it occurs while we loop over a loop of variables and it does not always crash on the same variable.

    In the attached example, I replicate this crash with random data on which I want to do seasonal adjustment. I loop over an arbitrary number of variables (N=30). To trigger the crash, one needs to change the current directory of Matlab to any subfolder.

    %% Create a subfodler
mkdir SomeFolder
addpath('./SomeFolder')

%% To trigger the bug set bug = true
bug = true ;
if bug 
    cd('./SomeFolder') 
else
    cd(cwd)
end

    In this case, the loop will eventually stop. Sometimes, it can stop for the first variable, sometimes it can iterate until a higher number of variable, for a reason we cannot explain.

    Variable 7 
 
ans is a dseries object:
 
       | d11    
1999Q1 | 0.39353
1999Q2 | 0.91905
1999Q3 | 0.6416 
1999Q4 | 0.721  
2000Q1 | 0.92627
2000Q2 | 0.42445
2000Q3 | 0.51803
2000Q4 | 0.3232 
2001Q1 | 0.32554
2001Q2 | 0.3497 
       |        
2021Q2 | 1.1398 
2021Q3 | 0.5587 
2021Q4 | 0.23827
2022Q1 | 0.53768
2022Q2 | 0.46911
2022Q3 | 0.77198
2022Q4 | 0.35521
2023Q1 | 0.4205 
2023Q2 | 0.61482
2023Q3 | 0.21813
2023Q4 | 1.1208 
 
Variable 8 
Reference to non-existent field 'd11'.

Error in x13/subsref (line 88)
                o = builtin('subsref', o.(S(1).subs), S(2));

Error in BugX13 (line 50)
    o.results.d11

    BugX13.7z

    Edited

    Designs

      Child items ...

      2. Activity

      • Pierre Aldama changed the description

        changed the description

      • Ugo Dubois added Madysson label

        added Madysson label

        • Stéphane Adjemian
          Stéphane Adjemian @stepan-a ·
          Owner

          @PierreAldama I am not able to reproduce this issue on my machine (I tried to increase NVar up to 300, but it works flawlessly). I suspect that this problem is specific to your environment. For some reason, when you decide to work in another directory (but I am not sure if that is really the cause of the error), the run method (that is the call to x13) fails or does not return what it should in o. We should return a message when something goes wrong with the run method. Here, your code fails later, but it's hard to understand why.

        • Pierre Aldama
          Pierre Aldama @PierreAldama ·
          Author

          Thanks for looking into it! Indeed, I experienced it again today and it wasn't due to the change of directory (i.e. my example could not reproduce it either). It only diseappeared after restarting my computer (also tried to restart Matlab before that, but it didn't do anything).

        • Stéphane Adjemian
          Stéphane Adjemian @stepan-a ·
          Owner

          I have just pushed a patch in Dynare to throw an error if the call to X13 fails. Can you test the same example tomorrow with the latest snapshot, and tell me if you see this new error message. Otherwise, it probably means that it's a formatting issue in one of the many files generated by the X13 binary.

        • Pierre Aldama
          Pierre Aldama @PierreAldama ·
          Author

          Hi Stéphane, I have experienced the bug once again this morning:

          Seasonal adjustment of variable f0_s11s12n_3: DONE
Seasonal adjustment of variable f0_s11n_3: DONE
Seasonal adjustment of variable f0_s12n_3: DONE
Seasonal adjustment of variable f0_s13n_3: DONE
Seasonal adjustment of variable f0_s14n_3: DONE
Seasonal adjustment of variable f0_s15n_3: DONE
Seasonal adjustment of variable f0_s1n_3: DONE
Seasonal adjustment of variable f3_s11r_3: DONE
Seasonal adjustment of variable f3_s11e_3: DONE
Seasonal adjustment of variable f2_s14e_3: DONE
Seasonal adjustment of variable f3_s14e_3: DONE
Seasonal adjustment of variable f4_s14r_3: DONE
Seasonal adjustment of variable f4_s14e_3: DONE
Seasonal adjustment of variable f5_s14e_3: DONE


 X-13ARIMA-SEATS Seasonal Adjustment Program
 Version Number 1.1 Build 59
 Execution began  Jan 11, 2023  09.46.34 

  Reading input spec file from KSdO0MOKOb.spc
  Storing any program output into KSdO0MOKOb.out
  Storing any program error messages into KSdO0MOKOb.err
 

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0xffffffff
#1  0xffffffff
#2  0xffffffff
#3  0xffffffff
#4  0xffffffff
#5  0xffffffff
#6  0xffffffff
#7  0xffffffff
#8  0xffffffff
#9  0xffffffff
#10  0xffffffff
#11  0xffffffff
#12  0xffffffff
#13  0xffffffff
#14  0xffffffff
#15  0xffffffff
#16  0xffffffff
#17  0xffffffff
#18  0xffffffff
#19  0xffffffff
#20  0xffffffff


Error using x13/run (line 40)
X13 failed on KSdO0MOKOb.spc

Error in x13/subsref (line 72)
                    feval(S(1).subs, o);

Error in MainBaseSource (line 1526)
    Temp.run();

Error in run (line 91)
evalin('caller', strcat(script, ';'));

Error in MainBase (line 24)
    run ('MainBaseSource.m')

          I am using: dynare-6-unstable-2023-01-06-1542-f7694208

          (And happy new year to you!)

          Edited by Pierre Aldama
        • Ugo Dubois
          Ugo Dubois @Ugo ·
          Reporter

          Bonjour à tous,

          Je remonte ce sujet : j'ai à présent le bonheur de déclencher le même message d'erreur depuis deux semaines.

          Je ne sais pas à quoi c'est lié étant donné que je ne la rencontrais jamais sous dynare-6-unstable-2023-05-12-1808-3c35f34a, que j'ai commencé à la recontrer sous dynare-6-unstable-2023-05-19-1925-03db691a (mais le retour à la version précédente n'a pas résolu le problème), et que dynare-6-unstable-2023-06-04-0811-b1933d58 ne résout pas le soucis.

          Pour le moment ma seule solution consiste à redémarer le PC, ne lancer -que- dynare et faire tourner nos codes. Si le processus marche une fois dans une instance de matlab, alors il ne semble pas tomber en panne plus tard (ouvrir une nouvelle instance de matlab peut déclancher le problème dans cette seule instance, sans affecter l'autre).

          Edited by Ugo Dubois
        • Johannes Pfeifer
          Johannes Pfeifer @JohannesPfeifer ·
          Developer

          Do you get a Segmentation fault error as before?

        • Ugo Dubois
          Ugo Dubois @Ugo ·
          Reporter

          Yes, it is the exact same error message (see below), I am using Matlab 2021a if that is relevant.

          The full error message Seasonal adjustment of variable f0_s11s12n_3: DONE

          X-13ARIMA-SEATS Seasonal Adjustment Program Version Number 1.1 Build 59 Execution began Jun 6, 2023 09.52.40

          Reading input spec file from I87exIa4x7.spc Storing any program output into I87exIa4x7.out Storing any program error messages into I87exIa4x7.err

          Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

          Backtrace for this error: #0 0xffffffff #1 (closed) 0xffffffff #2 (closed) 0xffffffff #3 (closed) 0xffffffff #4 (closed) 0xffffffff #5 (closed) 0xffffffff #6 (closed) 0xffffffff #7 0xffffffff #8 (closed) 0xffffffff #9 (closed) 0xffffffff #10 (closed) 0xffffffff #11 (closed) 0xffffffff #12 (closed) 0xffffffff #13 (closed) 0xffffffff #14 (closed) 0xffffffff #15 (closed) 0xffffffff #16 (closed) 0xffffffff #17 (closed) 0xffffffff #18 (closed) 0xffffffff #19 (closed) 0xffffffff #20 (closed) 0xffffffff

          Error using x13/run (line 40) X13 failed on I87exIa4x7.spc

          Error in x13/subsref (line 72) feval(S(1).subs, o);

          (Annnd, I should have written my first message in English, I'm sorry.)

        • Sébastien Villemot
          Sébastien Villemot @sebastien ·
          Owner

          It looks like the bug is in x13as, which is a third-party program developed by the US Census Bureau. As a consequence, it is unclear whether we can fix the problem ourselves.

          In order to try to reproduce the bug, and have a better understanding of it, could you send us the .spc file that triggers the crash?

        • Johannes Pfeifer
          Johannes Pfeifer @JohannesPfeifer ·
          Developer

          I can reproduce the problem with cSuLqLWmwF.spc The problem is the o.transform('function','log')-step. The error disappears if I switch to o.transform('function','auto');

          Note that other packages also have the same issue with the x13-binary, see https://github.com/christophsax/seasonal/issues/154

          Edited by Johannes Pfeifer
        • Johannes Pfeifer
          Johannes Pfeifer @JohannesPfeifer ·
          Developer

          Strangely, the issue is not about the spc-file per se. I tried calling x13as directly from the Matlab command line using the string of the x13.run()-command. This yields the expected

          >> !C:\dynare\6-unstable-2023-05-19-1925-03db691a\matlab\modules\dseries\src\..\externals\x13\windows\64\x13as.exe eQlsNHSTaf
 
 X-13ARIMA-SEATS Seasonal Adjustment Program 
 Version Number 1.1 Build 59 
 Execution began  Jun 13, 2023  16.11.40  
 
  Reading input spec file from eQlsNHSTaf.spc 
  Storing any program output into eQlsNHSTaf.out 
  Storing any program error messages into eQlsNHSTaf.err 
  

Program received signal SIGSEGV: Segmentation fault - invalid memory reference.

Backtrace for this error:
#0  0xffffffff
#1  0xffffffff
#2  0xffffffff
#3  0xffffffff
#4  0xffffffff
#5  0xffffffff
#6  0xffffffff
#7  0xffffffff
#8  0xffffffff
#9  0xffffffff
#10  0xffffffff
#11  0xffffffff
#12  0xffffffff
#13  0xffffffff
#14  0xffffffff
#15  0xffffffff
#16  0xffffffff
#17  0xffffffff
#18  0xffffffff
#19  0xffffffff
#20  0xffffffff

          However, using the direct path without the .. works:

          K>> !C:\dynare\6-unstable-2023-05-19-1925-03db691a\matlab\modules\dseries\externals\x13\windows\64\x13as.exe eQlsNHSTaf
 
 X-13ARIMA-SEATS Seasonal Adjustment Program 
 Version Number 1.1 Build 59 
 Execution began  Jun 13, 2023  16.11.58  
 
  Reading input spec file from eQlsNHSTaf.spc 
  Storing any program output into eQlsNHSTaf.out 
  Storing any program error messages into eQlsNHSTaf.err 
  
 
 Execution complete for eQlsNHSTaf.spc at  Jun 13, 2023  16.11.58

          Consequently, the change at JohannesPfeifer/dseries@abe2f13c fixes the issue for me.

          Edited by Johannes Pfeifer
        • Please register or sign in to reply
      • Sébastien Villemot assigned to @sebastien

        assigned to @sebastien

      • Johannes Pfeifer mentioned in commit JohannesPfeifer/dseries@abe2f13c

        mentioned in commit JohannesPfeifer/dseries@abe2f13c

      • Johannes Pfeifer mentioned in merge request dseries!45 (closed)

        mentioned in merge request dseries!45 (closed)

      • Ugo Dubois
        Ugo Dubois @Ugo ·
        Reporter

        Thanks to you both for investigating, I did not provide my file as the codes suddenly did not trigger the bug anymore (without any changes on my side, and I could not find a way to reliably trigger it before either). I'll be sure to get the next unstable and will let you know if I ever get this issue again.

        I think we can close the issue here for now (?)

      • Sébastien Villemot
        Sébastien Villemot @sebastien ·
        Owner

        The problem comes from an invalid memory read in x13as. It essentially means that the program tries to read in a random memory location. Sometimes it reads in an authorized location, and it gets a bogus value (but does not crash); sometimes it reads in a forbidden location, and it crashes (with a segmentation fault). Hence the apparently random nature of the bug. Some small change in the environment can make the crash happen or not (and for this reason, I don’t think that dseries!45 (closed) solves the problem; the path location is most likely just one of the many factors that affect the randomness).

        By investigating with a memory analyzer (valgrind) I found a possible location for the bug. But on the other hand GCC Address Sanitizer (ASAN) does not find any problem. So I’m unsure about the way forward. In any case, as said above, the bug is in a program that is not maintained by the Dynare Team, so fixing it may not be possible for us, or it may require many resources.

        Edited by Sébastien Villemot
        • Sébastien Villemot
          Sébastien Villemot @sebastien ·
          Owner

          @JohannesPfeifer @Ugo As you know, the x13as binary in the Dynare package is compiled by ourselves. In order to rule out a bug/problem in our compilation process, could you try the binary provided by the US Census Bureau and see whether the bug still occurs (keeping in mind that you must test various scenarios, due to the randomness)? It is available there: https://www2.census.gov/software/x-13arima-seats/x13as/windows/program-archives/x13as_ascii-v1-1-b59.zip (do not forget to rename x13as_ascii.exe into x13as.exe and put it in the right location)

          Edited by Sébastien Villemot
        • Johannes Pfeifer
          Johannes Pfeifer @JohannesPfeifer ·
          Developer

          There is a problem with our binary. The Census one works like a charm:

          !C:\dynare\6-unstable-2023-05-19-1925-03db691a\matlab\modules\dseries\src\..\externals\x13\windows\64\x13as_ascii.exe I87exIa4x7
 
 X-13ARIMA-SEATS Seasonal Adjustment Program 
 Version Number 1.1 Build 59 
 Execution began  Jun 14, 2023  08.31.43  
 
  Reading input spec file from I87exIa4x7.spc 
  Storing any program output into I87exIa4x7.out 
  Storing any program error messages into I87exIa4x7.err
        • Sébastien Villemot
          Sébastien Villemot @sebastien ·
          Owner

          That's encouraging. But did you test it in enough different scenarios to make sure that the absence of crash is not just because of a favourable state of nature in the non-deterministic behaviour?

        • Johannes Pfeifer
          Johannes Pfeifer @JohannesPfeifer ·
          Developer

          I have one real-world time series that I need to deseasonalize. And our Dynare binary always crashes with that particular series if the .. is present in the call to the binary name. I haven't encountered other cases yet, but I am investigating.

        • Sébastien Villemot
          Sébastien Villemot @sebastien ·
          Owner

          Ok I'll make the switch to the official X13 binary in our Windows package. Then we'll reassess the situation if someone is still encountering the bug.

        • Sébastien Villemot
          Sébastien Villemot @sebastien ·
          Owner

          I’ve switched to the official binary in commit 39c2cba1. If you still experience crashes with the X13 binary after this change, please report it here.

        • Ugo Dubois
          Ugo Dubois @Ugo ·
          Reporter

          I just switched to the latest unstable (its publishing time appears to match your comment), I'll be sure to report any misbehavior from X13 here if it acts up again.

        • Johannes Pfeifer
          Johannes Pfeifer @JohannesPfeifer ·
          Developer

          @Ugo Did you experience any issues recently? On my machine, the official binary solved the crashes. Otherwise, we would close the issue for now.

        • Ugo Dubois
          Ugo Dubois @Ugo ·
          Reporter

          Thankfully it never came back! I thinks it is ok to close the issue too.

        • Please register or sign in to reply
      • Sébastien Villemot added bug label

        added bug label

      • Sébastien Villemot mentioned in commit f986eac3

        mentioned in commit f986eac3

      • Johannes Pfeifer closed

        closed

      Please register or sign in to reply