252regrext.doc 12/6/99 (Open in Outline View)
K. REGRESSION EXTENSIONS
1. Residual Analysis
a. Presence of Outliers
b. Patterns of Residuals
(i) Nonlinear Equations
(ii) Heteroskedasticity
(iii) Missing Variables
(iv) Autocorrelation
2. Dummy Variables
a. Limits on numbers of Dummy Variables (Overdetermination)
b. Use in ANOVA
3. Nonlinear regression
Semilog, Double log, Reciprocal, Polynomial and Cyclical Forms.
4. Runs test
Consider the sequence + + + - - + + - - or AAABBAABB The Runs Test is a test to see if the sequences of two items, plusses and minuses in the first case and ‘A’s and ‘B’s in the second case is random. If the sequence is not random, the alternation of the two kinds of item tends to follow a predictable pattern. Our null hypothesis is randomness. In the sequences above, let be the total number of items, be the total number of items of the first kind (plusses or ‘A’s), be the total number of items of the second kind (minuses or ‘B’s) and be the total number of runs, that is the number of sequences of one kind. In the sequences above there are two sequences of plusses or ‘A’s and two of minuses or ‘B’s, so that (Note that the sequence ABA is three runs in itself). In the sequences above, we can also see that and .
To test the null hypothesis of randomness for a small sample, assume that the significance level is 5% and use the table entitled 'Critical values of in the Runs Test.’ For and , the top part of the table gives a critical value of 2 and the bottom part of the table gives a critical value of 9. This means that we reject the null hypothesis if or In this case, since we do not reject the null hypothesis.
For a larger problem (if and are too large for the table), follows the normal distribution with and . Example: Assume that the significance level is 5% and that we find that , and Then and . So Since this value of is not between , we reject
5. Durbin-Watson test
This is a test for randomness of regression residuals . The alternative hypothesis is (first-order) autocorrelation. The computer will print , and compute the Durbin-Watson statistic, . Use a Durbin-Watson table to fill in the diagram below.
0 ? 2 ? 4
+ + + + + + +
For example, if for a significance level of and we want a 2-sided test for a regression with a sample size of observations with independent variables and ,we go to a Durbin-Watson table for There we find for these values of and that and , so that and Our diagram is now
0 0.79 ? 1.70 2 2.30 ? 3.21 4
+ + + + + + +
Because is between 0.79 and 1.70, it is in an area marked with a question mark, and we cannot be sure whether to reject If, however, were between 0 and 0.79 or 3.21 and 4 we could reject the null hypothesis, while if it were between 1.70 and 2.30 we could say that we cannot reject the null hypothesis.
We can also do a one-sided test for an alternative hypothesis of positive autocorrelation by looking up critical values for and saying that we reject the null hypothesis if is below , that we do not reject it if is above and that we are not sure if is between and . If the alternative hypothesis is negative autocorrelation, we use the same critical values but we reject the null hypothesis if is above , we do not reject it if is above and we are not sure if is between and . For example, if and is , and we again have a regression with a sample size of observations with independent variable, a one sided 5% table gives us and Thus, since our computed is below , we reject the null hypothesis and conclude that there is positive autocorrelation. The most accessible D-W table can be found through D-W table.