Frequently Asked Statistical Questions (INTERVENTION/OUTLIERS)
| QUESTION:Most of your competitors market products that do outlier detection and some sort of adjustment. What is the big deal with AUTOBOX in this regard ?ANSWER:The answer to this question hinges on how we compute the standard deviation for the purposes of determining statistical significance. Consider that the standard deviation of the series could be used to normalize the series to a 0,1 distribution. The standard deviation as computed by our competitors , save one , is based upon the sum of squares of the values around the MEAN . The formula requires that one compute the sum of squares around the EXPECTED VALUE. The MEAN is the expected value if and only if the series is UNCORRELATED or independent of previous values. AUTOBOX correctly computes the standard deviation , thus correctly assesses unusual values. Consider the series 1,9,1,9,1,9,9,9,1,9,.... . The usual , and in this case incorrect , computation of the standard deviation would cause a failure to detect the anomaly. Replacing the MEAN with the EXPECTED VALUE would result in a standard deviation that is smaller and thus the anomaly in the 7th value would be immediately identified.QUESTION:Allright, so your computation of the standard deviation is more correct, explain why I need AUTOBOX to detect level shifts, seasonal pulses and time trends.ANSWER:If one restricts OUTLIER DETECTION to detecting unusual values that are INDPENDENTLY flagged, one can miss important information. One man's signal is another man's noise is the best way that I can put it. If you have an "UNUSUAL" value every December , this is "USUAL" since it represents a predictable process. Software that INDPENDENTLY evaluates each and every value based only upon IT's characteristics can severely miss the boat in picking up quality information and predictable structure. This is then the argument for Seasonal Pulse identification, a unique feature of AUTOBOX. The idea here is to test for instantaneous and immediately disappearing changes in the mean of the errors. If the mean of the errors changes to another "regime" this is called a level or a step shift. For example a series 1,1,1,1,1,2,2,2,2,2,2,2, would exhibit A STEP SHIFT or a change in the mean. AUTOBOX would identify the need for a dummy variable that had 0,0,0,0,0,1,1,1,1,1,1,1 and when estimated the coefficient of response would be a 1 with an intercept of 1. A step or level shift can be seen to be the integration of a pulse , viz. 1 1 0 1 0 1 0 1 0 2 when differenced 1 2 0 2 0 2 0 2 0 2 0 2 0 In the same spirit a trend can be seen to be the integration of a step 1 1 2 1 3 1 4 1 5 1 7 when differenced 2 9 2 11 2 13 2 15 2 17 2 19 2 AUTOBOX can then detect the need for a TIME TREND VARIABLE which of course is nothing more than the integration of a step or the double integration of a pulse.QUESTION:I am intrigued by how AUTOBOX performs intervention detection or as it is commonly known outlier detection. It is clear that the correct computation of the standard deviation using the correct EXPECTED VALUE rather than the simple MEAN would work in detailing the identification scheme. In fact I probably could have it programmed here at my company saving the expense of AUTOBOX but how does it work when it detects SEASONAL PULSES , LEVEL(STEP) SHIFTS and LOCAL TIME TRENDS ? On the surface I need to explain this to my boss but I would also like to know how it's done so that I could program it myself.ANSWER:The heart of the matter is the General Linear model where Y is the variable to be predicted and the form of the X matrix is to be determined empirically. Consider a case where the noise process is uncorrelated. One could construct simple regression models, where the trial X variable was initially 1,0,0,0,0,0,0.....0,0,0,0 and evaluate the error sum of squares. You could then create yet a second model where the X variable was 0,1,0,0,0,0,0,0,0,0,0.......0,0,0,0 and evaluate it in terms of the resultant error sum of squares. If you had 100 observations you would then run 100 regressions and the regression that generated the smallest error sum of squares would then be the MAXIMUM LIKELIHOOD ESTIMATE. This process would be repeated until the most important NEW variable was not statistically significant. This is essentially STEP-WISE forward regressioon, where the X variable is found by a search method. Two important generalizations are needed before you go running off to make our new competitor; 1. The search for the X variable has to be extended to include SEASONAL PULSES, LEVEL SHIFTS and TIME TRENDS and 2. The error process may be other than white noise, thus one has to iteratively construct TRANSFER FUNCTIONS rather than multiple regression. The process gets a little more sticky when you have pre-defined user inputs in the model. Consider the case where
QUESTION:I have a problem in testing the significance of an event Through a panel survey conducted on 115 points of sale (POS) I gather weekly sales data for 3 products. Follows a summary table where: Xi = Weekly total sales (115 POS) per product Yi = Number of POS (counts) which in the corresponding week had the product available on shelf Week 1 Week 2 Week n X1 Y1 X2 Y2 ...... Xn Yn Product A 136 90 142 86 156 99 Product B 645 76 538 84 552 81 Product C 318 103 346 108 301 92 My questions are: 1. How can test if the effect of, e.g., a promotional action caused sales in week 2 for product A (142 US$) to be significantly different from the corresponding level in week 1 (136 US$)? 2. How can I test if the number of POS which in week 2 had product A in stock (86 POS) is significantly smaller than that in week 1 (90 POS)?ANSWER:At one level , your problem is a Transfer Function where you are specifying two cause variables ; 1. The Number of Stores and 2. An indicator variable Z (all zeroes save the week in question which would be inicated by a one). This variable is called an Intervention Variable and is hypothesized in advance. If it were found empirically it would be still called an Intervention Variable bit it would have been detected by Intervention Detection or Outlier Analysis. Briefly the reason that you have to construct a Transfer function is that Sales in a particular week may be effected by sales the previous week or weeks, sales at this point in time a year ago, the number of sites carrying the product last week , the week before that and/or a recent level shift due to some omitted variable or even a trend in sales. All of these things , and more , may be operational thus to identify and measure the increment or lift due to this unique activity one has to allow for the other effects. Failing this one can incorrectly assign significance to what is otherwise caused by lag effects or seasonal processes or local changes in the mean.
where at is White Noise process.
où is white noise. The model presents a formulation of an input series , and the output series . The transfer model is summarized in the polynomial . The problem is to IDENTIFY and ESTIMATE the polynomial .
But before continuing I think that your problem could be slightly restated. The SeriesThe data could be laid out as follows , using Y as the variable to be predicted.
Yi = Weekly total sales (115 POS) per product Xi = Number of POS (counts) which in the corresponding week had the product available on shelf Week 1 Week 2 Week n Y1 X1 Y2 X2 ...... Yn Xn Product A 136 90 142 86 156 99 Construct a Transfer Function between the two exogenous variables (X and Z) making sure that the error process is correctly modelled, that is the ARIMA component. Test to see if there are violations of the Gaussian assumptions; viz. 1. The mean of the errors is zero everywhere. This can be tested via Outlier Detection. Various model augmenation may be required ( Pulse, Seasonal Pulse, Level Shift, Time Trends) to reach this objective. 2. The variance of the errors may not be constant, i.e. the variance may be proportional to the level or it may have had regime changes, where for some period of time the variance may have doubled or whatever. All of the above is premaces on the assumption that the Number of Stores and the Sales for Product 2 have no effecton the Sales of Product 1. If this were not true or had to be tested then the required tools would be Vector ARIMA, where in this case there would be Two endogenous variables (Y1,Y2) and three exogenous series X1,X2 and the hypothesized Z1 variable. Conclusion ------------ To the best of my knowledge AFS is the sole provider of software to deal with either solution. AUTOBOX allows one to identify and model outliers in a Transfer Function. I don't know any other piece of software that does that. Secondly , MTS which is a product of AFS deals with the Vector formulation. Again I believe that it is a unique solution because the only other Vector ARIMA program requires all variables , both endogenous and exogenous to be simultaneously predicted, thus there are no purely exogenous variables. All variables in that system are treated as endogenous. Reference: BOX, G.E.P. AND JENKINS, G.M. (1976). TIME SERIES ANALYSIS: | FORECASTING AND CONTROL, 2ND ED., SAN FRANCISCO: HOLDEN DAY. | Reilly, D.P. (1980). "Experiences with an Automatic | Box-Jenkins Modeling Algorithm," in Time Series Analysis, | ed. O.D. Anderson. (Amsterdam: North-Holland), pp. | 493-508. | | Reilly, D.P. (1987). "Experiences with an Automatic Transfer | Function Algorithm," in Computer Science and Statistics | Proceedings of the 19th Symposium on the Interface, ed. | R.M. Heiberger, (Alexandria, VI: American Statistical | Association), pp. 128-135. | | Tiao, G.C., and Box, G.E.P. (1981). "Modeling Multiple Time | Series with Applications," Journal of the American | Statistical Association, Vol. 76, pp. 802-816. | | Tsay, R.S. (1986). "Time Series Model Specification in the | Presence of Outliers," Journal of the American Statistical | Society, Vol. 81, pp. 132-141. | QUESTION:I want to analyze whether there is a significant trend over time in the annual failure rate of a product. I have 20 years of measurements (i.e., n = 20). As I understand it, an ordinary regression analysis would be inappropriate because the residuals are not independent (i.e., the error associated with a failure rate for 1974 is more highly correlated with the 1975 failure rate than the 1994 failure rate). Is it appropriate to simply divide the data into two groups (the 1st 10 years vs. the 2nd 10 years) and do a between-groups ANOVA? Or is there some other (better) way to analyze these data? Should anyone be so inclined as to do the analysis, here are the data:
ANSWER:"The Heart of the Matter" Your question regarding "testing the differences between two means" where the breakpoint , if any , is unknown and to be found is an application of intervention detection. The problem can , and is in your case, compounded by unusal one-time only values. Another major complication , not in evidence in your problem, is the affect of autocorrelated errors and the need to account for that in computing the "t value or F value" for the test statistic. The answer to your question is1. Don't assume that you need to break the data into two halves or some other guessed breakpoint. Employ Intervention Detection to search for that break point that provides the greatest distinction between the two developed groups. It is possible to pre-specify the test point but it is clear from your question that you were just struggling. In some cases where the analyst knows of an event ( law change , regime change ) it is advisable to pre-specify , but most of the time this is a dangerous proposition as delays or lead effects may be in the data. 2. Adjusting for one time onlies , analgous to Robust Regression, allows one to develop a gaussian process free of one-time anomalies and thus more correctly point to a break point. In this case:
3. After adjusting for these four "unusual values" the autocorrelation of the modified series exhibited the following;
Analysis concludes that there is no evidence of any ARIMA structure. Note that if there was an ARIMA model , not some assumed model like Durbin's would be used to model the noise process for our generalized least squares estimator of the test for the significant difference between the two means of a specified or unspecified group size. Early reserachers in the 1950's (Durbin,Watson,Hildreth,Liu to name a few had to ASSUME the form of the error structure). More modern approaches skillfully IDENTIFY the form of the error process leading to correct model specification. Dated procedures should be treated very carefully as their simplicity can be your demise! One final comment , your question had to with things changing . It is possible that the mean might have changed or the trend might have changed. AUTOBOX speaks to both questions and in this case concluded that the mean had indeed changed. 3. In the final analysis a "t test" is developed for the Hypothesis that
INPUT SERIES X5 I~AL0017 0017 LEVEL The conclusion based on a t value of -3.053 , is that yes there is a statistically significant difference between the two failure rates. This conclusion is obvious when one looks at a plot. Click here to view. QUESTION:Please explain INTERVENTION MODELING when I know a priori the timing and the duration of an event.ANSWER:In this section we discuss a class of models and a model development process for time series influenced by identifiable isolated events. More commonly known as intervention analysis, this type of modeling is transfer function modeling with a stochastic output series and a deterministic input variable. The values of the input variable are usually set to either zero or one, indicating off or on. For instance, a time series disturbed by a single event, such as a strike or a price change, could be modeled as a function of its own past values and a "dummy" variable. The dummy input would be represented by a series of zeroes, with a value of one at the time period of the intervention. A model of this form may better represent the time series under study. Intervention modeling can also be useful for "what if" analysis - that is, assessing the effect of possible deterministic changes to a stochastic time series. There is a major flaw associated with theory based models. It is called specification bias and this model suffers from it. Consider the assumption of a level shift variable starting at time period T. The modeler knows that this is the de jure date of the intervention. For example the date that a gun law went into effect. If the true state of nature is that it took a few periods for the existence of the law to affect the behavior then no noticeable effect could be measured during this period of delay. If you split the data into two mutually exclusive groups based upon theory, the test results will be biased towards no difference or no effect. This is because observations that the user is placing in the second group rightfully belong in the first group, thus the means then are closer than otherwise and a false conclusion can arise. This specification error has to be traded off with the potential for finding spurious significance when faced with testing literally thousands and thousands of hypothesis. QUESTION:Give me the formal presentation of the IMPACT of outliers.ANSWER:
|