Econ 301.02TASKIN

Econometrics

Notes on the violation of the Gauss Markov assumptions

Heteroskedasticity:

One of the important assumption of classical regression model is that the variance of the error term is constant, ie. , and the violation of this assumption is known as the heteroskedasticty problem in the regression anaylsis.

When the error term does not have a constant variance the problem can be stated as:

What might be the causes of heteroskedasticity:

  • In some cases the dependent variable shows a larger variability at different levels of the explanatory variable. Examples of this phenomenon can be observed in the analysis of consumption behavior or saving behavior. In most cases, there is little variability in the desired consumption amounts at low levels of disposable income but larger variability in the desired consumption at higher disposable income levels. The similar observation may be made for saving behavior. At high income levels the spending on necessities make up a lower percentage of total spending and hence the amount of spending beyond that level can be different for each high income family. A similar observation can be made in the analysis of firm dividends explained by the level of profits. There is low variability in the amounts of dividends distributed by firms with low profits but high variability in the dividends of high profit firms.
  • Heteroscedasticity may be the results of outlier observations.
  • Other forms of misspecification can also produce heterscedasticity. For example is a variable is omitted, the resulting error terms may exhibit a pattern similar to the omitted variable. For example if your are estimating a demand function and you missed the price of other variables that is an either a substitute good or a complementary good, the errors in the misspecified model will behave as the omitted price variable. As long as the variable omitted is not linearly correlated with the explanatory variables included into the regression, the estimates will be unbiased.
  • Another form of misspecification which looks like a heteroscedastic errors is the choice of wrong functional form. If the relationship studied is a quadratic relationship but if the squared variables are not included in to the estimation, then model will exhibit heteroscedastic errors.
  • The systematic measurement errors in one direction in the variables also may lead to heteroscedastic error results.

Consequences of heteroscedasticity:

  • The unbiasedness property of the OLS estimators does not change. OLS esimates are still unbiased but they are not efficient. Therefore B.L.U.E. does not hold. OLS estimators may not have the minimum variance among all the linear unbiased estimators.

Eg. More distant observations from the true line will have a larger weight in the determination of the slope coefficients, and hence tend to have unbalanced influence on the result.(this is clearly the mistake that the weighted least square estimation tagests to correct).

  • Standard errors or variances are biased and have a different formula than the one that is used in the OLS with well behaved error terms.

Eg. In the simple model: , the variance of the slope estimator is under the conditions of the standard assumptions. However, when the errors are htereoscedastic, then the variance becomes . Hence the reported OLS results without ay correction is the first one, when in fact the correct one is the second formula.

  • The following t stats are also incorrect and and any test that follows the reported that uses these variance estimates such as the conventionally computed t or F statistics or confidence interval estimates are also wrong.

The standard estimate of is also wrong, hence does not hold.

Detection of the presence of the heteroscedasticity problem:Does the problem of heretoskedasticity exists in a data that you are working with?

  • Visual tests:
  • Goldfeld and Quant test:
  • White’s heteroskedasticity test:
  • Breush-Pagan test:

Correction of the heteroscedasticity problem:

  1. Generalized Least Square estimation: (Weighted Least Square estimation)

Here the objective is to minimize the weight of the large variance

observation and maximize the variance of the small variance observation.

If you have information on , then the corrected estimation will be where each observation is weighted by the inverse of this true error variance, i.e.

The corrected Weighted Least Square estimation will be:

  • However, the information on iis impossible to obtain. Hence, the above GLS method can be used if there is a proxy for .\There may be a relationship between the error term ., hence ,and , or or any other variable which may not be in the model. The starting point of finding this relationship will be at the stages of detection of the heteroscedasticity with either visual methods or methods that examine the residual of the initial OLS estimation and the possible set of variables.

e.g. (1)If you think that the form of heresockedasticity is as follows:

,

which essentially says that there is a proportional relationship between the variance and the variable by a constant factor of .

With this case the weighted relationship will be:

e.g. (2) If your judgment indicates that the form of heresockedasticity is as follows:

,

which essentially says that there is a proportional relationship between the variance and the square of the variable by a constant factor of .

With this case the weighted relationship will be:

  • If you had been able to know the true value of the ,, then the results of the weighted least square would have been B.L.U.E.
  • Since this is not possible, then the estimation with the proxy for ,the estimators are consistent.
  • Another examples of a generalized least square estimation information is when you have information that there are two different values of .

e.g. (3) You know that or you were able to observe that for your sample that covers the period 1971- 2010, the first sub period of 1971-1990 have low variance but the subperiod of 1991-2010 has a larger variance for the error term. It is possible to do weighted least squares here.

If you think that first sub perioEstima d the variance of the error term is and it is lower that the variance of the error term in the second sub period, . The correction that you will do should follow the following steps:

  1. Estimate the equation with OLS and obtain the residuals.
  2. Compute and , by using the residual values separately for each subperiod.
  3. Estimate the GSL (WLS) with the following equation:

Autocorrelation (Serial Correlation)

When error terms from different time periods are correlated. This problem occurs usually in time series data. Most of the time the errors in adjacent time periods are correlated. This violates the assumption of unrelated disturbance terms belonging to different periods, i.e. ..

This becomes . Since we are using a time series data, it is common to use (t subscript rather than i), and the presence of autocorrelation can bedespicted as .

Since there are many different forms of correlation is possible between the error terms with the above too general form, it is customary to specify a mechanism that generates the error terms to create an autocorrelated error terms. One such mechanism is the following:

The

The is known as the coefficient of autocorrelation. The et is the white noise error term with all the desired properties.

This process that creates the is known as the first –order autoregressive scheme (AR1) .[1]

The properties of such error terms will be :

What might be the causes of autocorrelation:

  • If there is an inertia in the economic series, especially in macroeconomic series. The sluggish adjustment in the economic series creates this pure autocorrelated error terms.
  • If there is an excluded explanatory variable in the model, the effect of this variable will be observed as a systematic factor in the error term. Usually this type of autocorrelation problem is corrected by including the omitted variable.
  • Incorrect functional form also gives the autocorrelated error terms. The correction of the functional form in the estimation also corrects the problem.
  • Lagged dependent variable is an explanatory factor, especially in slow adjusting variables. If this lagged dependent is omitted from the model, then the error will contain the effect of this lagged dependent variable and the errors will be correlated.
  • The nontsationarity of the series also will create a correlated error term.s
  • Data transformations may also create correlation in the error terms, if the original errors are uncorrelated.

Consequences of autocorrelation

  • The unbiasedness property of the OLS esimators does not change. OLS esimates are still unbiased but they are not efficient. Therefore B.L.U.E. does not hold. OLS estimators may not have the minimum variance among all the linear unbiased estimators.
  • The estimators are linear unbiased, as well as consistent and asymptotically normally distributed. But they are not efficient.
  • The formula for is not the standard formula which is

Hence if we continue to use the standard variance reported for statistical test such as t or F we, they will be incorrect.

  • Even if the the correctly calculated formula for the variance is used the confidence intervals will still be wider than the confidence intervals which may be commputed with an alternative estimator (such as GLS).
  • The estimated error variance with the standard formula is likely to underestimate the true value of .
  • We are likely to overestimate R2.
  • Even if is correctly estimated , still will underestimate
  • Therefore the usual t and F tests are no longer valid and is likely to give misleading results.

(The relationship between and the true )

Detection of the presence of the autocorrelation problem:Does the problem exits or not?

  • Visual test: Run the OLS estimate of the initial model , obtain the residuals , plot the these residuals to see if the same sign tends to follow each other. (+ residuals following the previous + residuals and – residuals following the – coefficients)
  • Durbin Watson test
  • Conditions that is necessary for the use of Durbin-Watson.
  1. DW can only test for the first order autocorrelation, ie. AR(1),
  2. should have normal distribution,
  3. should be nonstochastic (fixed in repeated samples),
  4. model should have an intercept term,
  5. the model should not include the lagged dependent variable as an explanatory variable since this creates the problem of endogeneity if there is autocorrelated error terms.

The null hypothesis is and if DW is less than dlower critical value, then we can reject the above null hypothesis.

  • t test for the AR(1) term in the following equation:

by using the residuals of the OLS estimates of the original model.

  • Breusch-Godfrey test: Allows for higher order autocorrelation and regressors, such as lagged dependent variable.

Step 1: Run the OLS estimate of the initial model , obtain the residuals ,

Step 2: Run the following regression: ;

And then test the joint significance of the coefficients of autocorrelation terms, with the following distribution:

if the null hypothesis of all p coefficients of autocorrelation terms.

Correction of the autocorrelation problem:

  1. Generalized Least Square estimation: (Weighted Least Square estimation)

When there is pure autocorrelation problem, it is possible to transform the model to get rid of dependency of the error terms. This is one version of Generalized Least Square estimation method.

In a simple regression model

with the error term and ;

The same equation will also hold for the time period t-1and multiplied by :

,

Then the last equation is subtracted from the first one which gives the following equation:

This equation has uncorrelated error terms which satisfies the condition. Hence the above transformations performs the necessary correction for the problem of first order autocorrelation problem. The new equation is with transformed variables and the application of the OLS estimation to this new variables is known as Generalized Least Square estimation. The transformed equation is:

where , ,

In order to perform this estimation, we need a value for . However the true value of is unknown. It is possible to only come up with an estimate of . There are several methods we can use to estimate . (1) method is to use DW statistics and to use the ; (2) method is to estimate the following equation by using the residual from the initial OLS estimate and use the estimated .

The GLS estimation conducted with the estimated values provide consistent estimates of and[2]. This estimation can only be done with (n-1) number of observations, since one observation is lost in the transformation[3]. To avoid any inefficiency due to a loss of observation, especially in cases when the n is not large, another transformation can be conducted for the first observation. This transformation is as follows:

is used for the first observation of and is used for the first observation of .

Endogeneity Problem:

Another important assumption about the error term is that it is uncorrelated with the explanatory variables. Hence, when we assume that the explanatory variables are fixed in repeated sample, ( are not stochastic) then the . With this assumption we can state the Gauss Markov assumptions as .

However, is the assumption of fixed explanatory variables is violated and the explanatory variables are also random variables drawn from a distribution then the same Gauss Markov assumptions has to be stated as:

This is to say that no matter what explanatory variable values are observed the assumptions given the value of x’s still holds. Then the unbiasedess can be stated as

However, there can be cases where the explanatory variable is contemporaneously correlated with the error terms. The cases that lead to this correlations are as follows:

1)Errors in measurement of the explanatory variables,

2)Omitted variables (correlated with the included variable),

3)Jointly determined variables (simultaneity),

4)Lagged dependent variables with autocorrelated error terms,

These create the endogeneity of the explanatory variables. Under these conditions the OLS estimator will be biased and inconsistent, which is known as the endogeneity problem.

The proof of this inconsistency:

OLS estimator of the slope coefficient in the simple linear regression model is given as :

Since , the and hence the estimator is inconsistent.

The solution is to use Instrumental variable estimation technique, ie. IV estimator. You have to find another variable, which can be used as an instrument that is uncorrelated with and closely correlated with. Then the formula for the instrumental estimator is:

This estimator is going to be a consistent estimator.

[1] There can be higher order processes if the error term today is related to just not the the error term last period but the period before that. An equation such as will explain such a phenomena ...

[2]If the true value of had been known, then GSL estimates would have been unbiased

[3] To avoid any inefficiency due to a loss of observation, especially in cases when the n is not large, another transformation can be conducted for the first observation. This transformation is as follows:

is used for the first observation of and for the first observation of .