The upcoming interview with David Hendry to be published soon in the International Journal of Forecasting reveals a researcher whose work is top tier. One of his accomplishments discussed was his methodology of detecting change in parameters (ie Chow test) and outliers called "Impulse Indicator Saturation"(ISAT) which is available in the R GETS Package. We tested it out and weren't impressed compared to what Autobox does on some examples. Of course, we are always ready to listen, if you have any opinions on this matter. Speak up, we'll listen!
Detecting change in patterns is not only very important for forecasting, but also for what some advertise as "Business Intelligence" where you want to detect unusual behavior in the data. So, who can detect this?
There are two examples used in the package. One is the Nile example(Cobb 1978) and the other a random number generator creating a counter example to show that it doesn't create false positives. The Nile example is well handled as it is in Autobox(comes with Autobox in the \course\Example2.asc folder).
ISAT finds the level shift and the outlier at obs 43, but misses the others that Autobox finds at 7,18,94.
ISAT handles the two examples well enough. However, when we gave it one of our favorite and very OBVIOUS examples that we use to demonstrate changes in parameters it didn't detect the change. We wonder "Why?". Please tell us if you know. :)
The data is 9 years of quarterly GDP with a obvious break in parameters. The following output can be easily reproduced by downloading Autobox and run using the example found in Autobox in the \course\Example16.asc. folder.
Here is the data if you want immediate access to it to try in your tool(we'd love to hear what your tool does with this!). 130.7,137.7,137.7,161.2,173.3,181.1,181.5,180.7,180.7,193,207.8,209.5,211.6,228.4,237.1,233.5,243.1,280.2,297.1,328.2,283.3,330.7,348.9,385.8,326,381.9,395.6,429.1,350.8,406.5,422.2,470,387.3,441.6,453.7,494.1
Here we run the GETS ISAT function
Here are the results of ISAT from the GETS Package. The fit shows that it is not detecting the seasonality and it thinks there are 4 changes??? It doesn't detect the change in parameters(onset of seasonality) and model it. Why?
Autobox identifies a break point an tells the user
Here is the model showing an AR4 with a constant and the detection and deletion of the older data using only data after the break.
Here is the actual, fit and forecast of the truncated series.