SIMULTANEOUS EQUATIONS REGRESSION MODEL

INTRODUCTION

The classical linear regression model, general linear regression model, seemingly unrelated regressions model make the following assumption:

The error term is uncorrelated with each explanatory variable.

If this assumption is violated, then the OLS, FGLS and SUR estimators produce biased estimates in small samples, and inconsistent estimates in large samples.

SOURCES OF CORRELATION BETWEEN THE ERROR TERM AND EXPLANATORY VARIABLE

The most important sources of correlation between the error term and an explanatory variable are omitted confounding variables and reverse causation.

Omitted Confounding Variable

Consider the following wage equation,

Y = β1 + β2X + μ,

Y is the worker’s wage, X is the worker’s years of education, and μ is the error term. We want to analyze the effect of education on the wage. Let Z be the worker’s innate ability. Since we omit Z from the equation its effect is included in μ. Workers with more innate ability have higher wages, and therefore larger errors. Also, workers with more innate ability have more education, and therefore higher values of X. Thus, the error term and education are positively correlated. The OLS, FGLS, and SUR estimators will include the effect of innate ability in the estimate of β2. This results in a biased estimate of the effect of education on the wage.

Reverse Causation

Consider the following simple Keynesian model of income determination comprised of two equations: a consumption function, and equilibrium condition

C = a + b Y + 

Y = C + I

C is aggregate consumption; Y is aggregate income; I is exogenous investment; a and b are parameters; and  is an error term that summarizes all factors other than Y that influence C (e.g., wealth, interest rate). Now, suppose that  increases. This will directly increase C in the consumption function. However, the equilibrium condition tells us that the increase in C will increase Y. Therefore,  and Y are positively correlated. The OLS, FGLS, and SUR estimators will produce a biased estimate of the effect of income on consumption because it will capture the reverse effect of consumption on income.

OBJECTIVE

In this section of the course, we will examine statistical models that assume the error term is correlated with an explanatory variable. This can result from either an omitted confounding variable or reverse causation. We will spend most of our time on the simultaneous equations model. This model assumes that the error term is correlated with an explanatory variable because of reverse causation. However, the estimators we develop for the simultaneous equations model can be used for any model for which the error term is correlated with an explanatory variable because of an omitted confounding variable.

INTRODUCTION TO THE SIMULTANEOUS EQUATIONS MODEL

When a single equation is embedded in a system of simultaneous equations, at least one of the right-hand side variables will be endogenous, and therefore the error term will be correlated with at least one of the right-hand side variables. In this case, the true data generation process is not described by the classical linear regression model, general linear regression model, or seemingly unrelated regression model; rather, it is described by a simultaneous equations regression model. If you use the OLS estimator, FGLS estimator, SUR estimator, or ISUR estimator, you will get biased and inconsistent estimates of the population parameters.

Definitions and Basic Concepts

Endogenous variable – a variable whose value is determined within an equation system. The values of

the endogenous variables are the solution of the equation system. More generally, any variable that is correlated with the error term.

Exogenous variable – a variable whose value is determined outside an equation system. More generally, any variable not correlated with the error term.

Structural equation – an equation that has one or more endogenous right-hand side variables.

Reduced form equation – an equation for which all right-hand side variables are exogenous.

Structural parameters – the parameters of a structural equation.

Reduced form parameters – the parameters of a reduced form equation.

THE IDENTIFICATION PROBLEM

Before you estimate a structural equation, you must first determine if it is identified. An equation is identified if you have enough information to get meaningful estimates of its parameters. A meaningful estimate is one that has a useful interpretation. An equation is not identified if you don’t have enough information to get meaningful estimates of its parameters. If an equation is not identified, then estimating its parameters is meaningless. This is because the estimates you obtain will have no useful interpretation.

Example

You want to estimate the price elasticity of demand for a good. You collect annual data on price (P) and quantity bought and sold (Q) for the period 1980 to 2015. You estimate the following equation,

lnQ = γ1 + γ2lnP + μ

where ln designates natural logarithm. The problem is that this equation does not have an identity. It can be either a demand equation, supply equation, or some combination of both. Therefore, γ2 might measure the price elasticity of demand, price elasticity of supply, or some combination of both. Running a regression of lnQ on lnP has no useful interpretation.

Classifying Structural Equations

Every structural equation can be placed under one of three categories.

Unidentified Equation. Not enough information to get a meaningful estimate.

Exactly Identified Equation. Just enough information to get a meaningful estimate.

Overidentified Equation. More than enough information to get a meaningful estimate.

Exclusion Restrictions

The most often used way to identify a structural equation is to use prior information provided by economic theory to exclude certain variables from an equation that appear in a model. This is called obtaining identification through exclusion restrictions. To exclude a variable from a structural equation, we restrict the value of its coefficient to zero. This type of zero fixed value restriction is called an exclusion restriction because it has the effect of omitting a variable from the equation to obtain identification.

Rank and Order Condition for Identification

Exclusion restrictions are most often used to identify a structural equation in a simultaneous equations model. When using exclusion restrictions, you can use two general rules to check if identification is achieved. These are the rank condition and the order condition. The order condition is a necessary but not sufficient condition for identification. The rank condition is both a necessary and sufficient condition for identification. Because the rank condition is more difficult to apply, many economists only check the order condition and gamble that the rank condition is satisfied. This is usually, but not always the case.

Order Condition

The order condition is a simple counting rule that you can use to determine if one structural equation in a system of linear simultaneous equations is identified. Define the following:

G = total number of endogenous variables in the model (i.e., in all equations that comprise the

model).

K = total number of variables (endogenous and exogenous) excluded in the equation being

checked for identification.

The order condition is as follows.

If K = G – 1 the equation is exactly identified

If K > G – 1 the equation is overidentified

If K < G – 1 the equation is unidentified

SPECIFICATION OF A SIMULTANEOUS EQUATIONS MODEL

A simultaneous equation regression model has two alternative specifications: reduced form and structural form. The reduced-form specification is comprised of M reduced-form equations and a set of assumptions about the error terms in the reduced form equations. The reduced-form specification of the model is usually not estimated, because it provides limited information about the economic process in which you are interested. The structural-form specification is comprised of M structural equations and a set of assumptions about the error terms in the structural equations. The structural-form specification of the model is the specification most often estimated. This is because it provides more information about the economic process in which you are interested.

Specification of the Structural Form

A set of assumptions defines the specification of the structural form of a simultaneous equations regression model. The key assumption is that the error term is correlated with one or more explanatory variables. There are several alternative specifications of the structural form of the model depending on the remaining assumptions we make about the error term. For example, if we assume that the error term has non-constant variance, then we have a simultaneous equation regression model with heteroscedasticity. If we assume the errors in one or more equations are correlated, then we have a simultaneous equation regression model with autocorrelation.

ESTIMATION

Single Equation Vs System Estimation

Two alternative approaches can be used to estimate a simultaneous equation regression model are single equation estimation and system estimation.

Single Equation Estimation

Single equation estimation involves estimating either one equation in the model, or two or more equations in the model separately. For example, suppose you have a simultaneous equation regression model that consists of two equations: a demand equation and a supply equation. Suppose your objective is to obtain an estimate of the price elasticity of demand. In this case, you might estimate the demand equation only. Suppose your objective is to obtain estimates of price elasticity of demand and price elasticity of supply. In this case, you might estimate the demand equation by itself and the supply equation by itself.

System Estimation

System estimation involves estimating two or more equations in the model jointly. For instance, in the above example you might estimate the demand and supply equations together. You might do this even if your objective is to obtain an estimate of the price elasticity of demand only.

Advantages and Disadvantages of the Two Approaches

The major advantage of system estimation is that it uses more information, and therefore results in more precise parameter estimates. The major disadvantages are that it requires more data and is sensitive to model specification errors. The opposite is true for single equation estimation.

SINGLE EQUATION ESTIMATION

If the error term is correlated with an explanatory variable, then we cannot find an estimator that is unbiased in small samples. This means we must look for an estimator that has desirable large sample properties. We will consider 4 single equation estimators.

  1. Ordinary least squares (OLS) estimator
  2. Instrumental variables (IV) estimator
  3. Two-stage least squares (2SLS) estimator
  4. Generalized method of moments (GMM) estimator

ORDINARY LEAST SQUARES (OLS) ESTIMATOR

Properties of the OLS Estimator

If the error term is correlated with an explanatory variable, then the OLS estimator is biased in small samples, and inconsistent in large samples. It does not produce maximum likelihood estimates. Thus, it has undesirable small and large sample properties.

Role of OLS Estimator

The OLS estimator should be used as a preliminary estimator. You should initially estimate the equation using the OLS estimator. Then estimate the equation using a consistent estimator. Then compare the OLS estimate and consistent estimate of a parameter to determine the possible direction of the bias. This is because a consistent estimator will have a smaller bias than the inconsistent OLS estimator in any finite sample.

INSTRUMENTAL VARIABLES (IV) ESTIMATOR

The IV estimator involves the following two-step procedure.

  1. Find one instrumental variable for each right-hand side variable in the equation to be estimated. A valid instrumental variable has two properties:
  1. Instrument relevance. It is correlated with the variable for which it is to serve as an instrument.
  2. Instrument exogeneity. It is not correlated with the error term in the equation to be estimated.
  1. Apply the following IV estimator formula. The formula requires matrix algebra.

Comments

Each exogenous right-hand side variable can serve as its own instrumental variable. This is because it is perfectly correlated with itself and is not correlated with the error term by assumption of exogeneity. The best candidates to be an instrumental variable for an endogenous right-hand side variable in the equation to be estimated are exogenous variables that appear in other equations in the model. This is because they are correlated with the endogenous variables in the model via the reduced-form equations, but they are not correlated with the error term in any equation. Oftentimes there will exist more than one exogenous variable that can serve as an instrumental variable for an endogenous variable. In this case, you can do one of two things. 1) Use as your instrumental variable the exogenous variable that is most highly correlated with the endogenous variable. 2) Use as your instrumental variable the linear combination of candidate exogenous variables most highly correlated with the endogenous variable. As we will see later, if we do this we have a more general type of IV estimator called the two-stage least squares estimator.

Relationship Between the IV Estimator and Identification

The following relationship exists between the IV estimator and identification.

  • If the equation is exactly identified, then there are exactly enough exogenous variables excluded from the equation to serve as instrumental variables for the endogenous right-hand side variable(s).
  • If the equation is overidentified, then there are more than enough exogenous variables excluded from the equation to serve as instrumental variables for the endogenous right-hand side variable(s).
  • If the equation is unidentified, then there are not enough exogenous variables excluded from the equation to serve as instrumental variables for the endogenous right-hand side variable(s). In this case, the IV estimator cannot be used.

Properties of the IV Estimator

  • Like all estimators, it is biased in finite samples.
  • It is consistent in large samples.
  • It is not necessarily asymptotically efficient. This is because an endogenous variable can have more than one instrumental variable. Each instrumental variable results in a different IV estimator. The higher the correlation between the endogenous variable and the instrumental variable, the more efficient the IV estimator.
  • If there is heteroscedasticity, then the IV estimator is not efficient in the class of consistent estimators and the estimated standard errors are biased and inconsistent.
  • It is not the maximum likelihood estimator.

TWO-STAGE LEAST SQUARES (2SLS) ESTIMATOR

The 2SLS estimator is a generalization of the IV estimator. It reduces to the IV estimator if the equation is exactly identified. This estimator involves two successive applications of the OLS estimator. This two-stage procedure is as follows.

Stage #1: Regress each right-hand side endogenous variable in the equation to be estimated on all exogenous variables in the simultaneous equation model using the OLS estimator. Calculate the fitted values for each of these endogenous variables.

Stage #2: In the equation to be estimated, replace each endogenous right-hand side variable by its fitted value variable. Estimate the equation using the OLS estimator.

Comments

  • Stage 1 is identical to estimating the reduced-form equation for each endogenous right-hand side variable in the equation to be estimated.
  • The exogenous variables in the stage 1 regression are the instruments. They can be placed under two categories. 1) Identifying instruments. 2) Other instruments. An identifying instrument is any exogenous variable that has been excluded from an equation to identify it. Other instruments are exogenous variables included in the equation that serve as instruments for themselves.
  • The fitted value variable from the stage 1 regression is the linear combination of instruments that has the highest correlation with the endogenous explanatory variable in the structural equation. At least one identifying instrument must be partially correlated with the endogenous explanatory variable; if not, then the fitted value variable will be perfectly correlated with the exogenous variables included in the stage 2 regression and the 2SLS estimator cannot be used.
  • The estimated standard errors obtained from the stage 2 regression are incorrect and must be corrected. This is because the estimate of σ2 = RSS/(T-k) which uses RSS from the second stage estimate is wrong. We need to use RSS from the estimated structural equation. Statistical programs that have a 2SLS procedure make this correction automatically and report the correct standard errors.

Logic of 2SLS Estimator

Suppose Y is the dependent variable, X is the endogenous right-hand side variable, μ is the error term, and Z is instrumental variable. We can decompose the variation in X into 2 parts. One part is correlated with μ. The other part is uncorrelated with μ. To get an unbiased estimate of the effect of X on Y, we need to use the variation in X that is uncorrelated with μ, and eliminate the variation in X that is correlated with μ. To capture the variation in X that is uncorrelated with μ, we use an instrumental variable, Z, that is correlated with X, but uncorrelated with μ. For Z to perform this function, it must be relevant and exogenous. If it is not relevant, then it is not correlated with X, and therefore it does not capture variation in X. If it is not exogenous, then it is correlated with μ, and therefore it captures variation in X that is correlated with μ. How does the 2SLS estimator capture the variation in X uncorrelated with μ, and disregard the variation in X correlated with μ? The stage 1 regression can be written as: X = π0 + π0Z + νt. This regression decomposes the variation in X into 2 parts. 1) The systematic component π0 + π0Z captures the variation in X explained by Z, but not explained by μ. This is because Z is correlated with X but uncorrelated with μ. The error term ν captures the variation in X explained by μ and any additional factors other than Z. However, the true values π0 + π0Z are unknown because the parameters π0 and π0 are unknown. Therefore, we use the predicted values X^ = π0^ + π0^Z from a regression of X on Z using OLS. The stage 2 regression can be written as: Yt = α + βXt^ + εt. OLS yields a consistent estimate of β, because Xt^is not correlated with the error term μt. Note that εt = Yt - α - βXt^, while μt = Yt - α - βXt. To obtain a correct estimate of the standard error of the estimate, we must use the residuals μt^ = Yt– α^ - β^Xt. Statistical programs with a 2SLS command calculate these residuals for you.