Queen's School of Policy Studies

Program Evaluation

SPS 818 - Winter 2004

Assignment 2

Due: March 18, 2004 @ 3:00pm

While in theory, this assignment counts for 5% of the grade; it is a great tool to make sure you are up to date on quasi-experimental analysis. I will assign a grade based on the completed assignment as a whole.

Regression Discontinuity

1. In regression discontinuity designs, describe why it is important to know the functional form of the effects of the assignment variable on the outcome variable. And what are some options for investigating the sensitivity of the results to the functional form?

2. Write down the basic regression discontinuity model with a linear assignment variable effect. Now write down the model that allows the treatment effect to vary by level of assignment (Z). Use the regression coefficient parameters to write down the value of the outcome variable – expressed as parameter estimates from the regression -- in each range of the data; i.e., for those who have been treated and those who have not been treated.

3. Write down the regression equation that would result from merging a pre-/post-intervention design with a regression discontinuity design. What is the a) outcome variable b) treatment variable and c) assignment variable?

Matching

4. What is the principle of matching? What are the potential limitations with this method? Does the propensity score address some of these limitations?

5. In the course we learnt that if a researcher controls for all observables in a regression equation estimated by OLS then the parameter estimate should not suffer from omitted variable bias. In that case why do estimates from matching algorithms differ from OLS estimates of treatment effects?

Pre/Post with Comparison Group

7. Consider a pre-/post-study design with two time periods.

  1. Graphically depict this design. On the y-axis should be the outcome of interest. On the x-axis should be time, with t=0 and t=1. Draw a line between the two points and label this line “treatment group.” The intervention took place between t=0 and t=1.
  2. Now add to the graph outcomes measured at t=0 and t=1 for a control group that did not receive the intervention. Draw a line between the two points and label this “control group.”
  3. Write down the regression equation that represents this pre-/post-intervention with comparison group study design. Make sure you define all right hand side variables. For example, X=0 if …and X=1 if…etc…
  4. Label the distances on the graph used above that correspond to each of the coefficient estimates in your regression equation.
  5. Express the mean of the outcome variable as a function of the coefficient estimates for the following groups:
  6. Treatment group at time 0 ==
  7. Treatment group at time 1 ==
  8. Control group at time 0 ==
  9. Control group at time 1 ==

Instrumental Variables

8. Explain what is an instrumental variable and the properties that it must possess to have a causal interpretation and to be considered a weak instrument.

Panel Data

9. Consider a one-way individual specific fixed effects regression.

a) Explain how this regression removes time invariant unobserved heterogeneity.

b) Briefly explain whether a fixed effects regression can identify i) the effect of gender on individual’s earnings and ii) the effect of the national unemployment rate on an individual’s earnings.

10. In their study of the effectiveness of cardiac catherization, McClellan and Newhouse (1994) used as an instrument the difference in distance to cardiac catherization and regular hospitals. How could you determine whether this instrument is relevant? Is it exogenous?

Internal Validity

11. Consider a study to evaluate the effect on college student grades of dorm room Internet connections. In a large dorm, half the rooms are randomly wired for high speed Internet connections (the treatment group), and final course grades are collected for all residents. Which of the following pose a threat to internal validity, and why?

a)Midway through the year, all the male athletes move into a fraternity and drop out of the study (their final grades are not observed).

b)Engineering students assigned to the control group put together a local area network so that they can share a private wireless Internet connection that they pay for jointly.

c)The political studies majors in the treatment group never learn how to access their Internet accounts.

d) The economics majors in the treatment group provide access to their Internet connection to those in the control group, for a fee.

1