PS 204A: Research Design

PS 204A: Research Design

GSBA 599, Fall 2010

Research Design

ProfessorMathew D. McCubbins, office HOH 612; Gould 421

Time and Venue: Wednesday 2:00-4:50pm ACC 312

Office Hours: Wednesday 11am - 1:50pm

Course Description

This is a course on research design. The course will cover how to design compelling research, the focus of which is on causal inference. We will cover the design of true experiments and contrast them to classroom simulations. Discussing threats to the internal validity of true experiments will allow us to focus in on the same threats to observational studies. We will cover extensively the design of quasi-experiments where the researcher controls neither the assignment of cases into groups nor the administration of the treatment being studied. Our approach will follow the Neyman-Rubin potential outcomes framework. This perspective has become increasingly popular in many fields including statistics, medicine, economics, political science, sociology, law, finance, accounting and marketing. The framework assumes that each unit being studied has two potential outcomes, one if the unit is treated and the other if untreated. A causal effect is defined as the difference between the two potential outcomes, but only one of the two potential outcomes is observed. Rubin and others developed the model into a general framework for causal inference with implications for observational research.

Identifying the causal impact, for example, of some variable T on a dependent variable Y is difficult in the best of circumstances, but faces seemingly insurmountable problems in observational data, where T is not manipulable by the researcher and cannot be assigned randomly. Nevertheless, estimating such an impact or treatment effect is the goal of much research, even much research that carefully states all findings in terms of associations rather than causal effects.

Experimental research designs offer the most plausibly unbiased estimates of the treatment, effect but experiments are frequently infeasible. Four types of quasi-experimental (or observational) research designs offering approaches to causal inference using observational data will be discussed. In rough order of increasing internal validity: 1) ordinary regression and panel methods, including differences in differences, and time-series, cross-section studies, as well as event studies; various matching and reweighting estimators, including propensity score matching (PSM), genetic matching and synthetic controls, instrumental variables (IV) and related methods; and regression discontinuity (RDD) designs, We will carefully describe the kinds of causal inferences one may make from a design, ranging from the population average treatment effect (ATE) to the local average treatment effect (LATE) to the intent to treat effect (ITT) and many kinds in-between.

Text and Reading Material

The primary text for this course is available at Amazon.com:

Trochim, William and James P. Donnelly. 2007. The Research Methods Knowledge Base, 3rd Edition. Cincinnati, OH, Atomic Dog Publishing. An online edition is available for $69.75, and a paperback and online edition for $87.00. I recommend the latter so that you will have the book for future reference.

I highly recommend that you also purchase/download:

Angrist, Joshua D. and Jorn-Steffen Pischke. 2009. Mostly Harmless Econometrics: An Empiricist’s Companion. Princeton University Press.

Cook, Thomas D. and Donald T. Campbell. 1979. Quasi-Experimentation:

Design and Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin.

Guido Imbens and Jeffrey Wooldridge. 2007 Lecture notes to What’s New in Econometrics. NBER. Available at:

All other readings for this course are available through Electronic Reserves at the USC library or on a CD provided by the instructor.

Grading and Course Requirements

Final Essay20%

Class Participation40%

Mid-Course Essay20%

Presentation20%

Total100%

You are expected to read all assigned materials and to be prepared to discuss them at the class meeting for which they are assigned. There are eight weeks of lecture followed by six weeks of student presentation and discussion. Many of these student presentations will be by advanced students preparing their theses. There are two written assignments for the course, due as noted on the syllabus below. Late assignments will not be accepted. Each written assignment will be worth 20% of your total grade, as is the presentation of your mid-term, and the remaining 40% will be based on your participation in classroom discussions.

Statement for Students with Disabilities

Any student requesting academic accommodations based on a disability is required to registerwith Disability Services and Programs (DSP) each semester. A letter of verification for approvedaccommodations can be obtained from DSP. Please be sure the letter is delivered to me (or toTA) as early in the semester as possible. DSP is located in STU 301 and is open 8:30 a.m.–5:00p.m., Monday through Friday. The phone number for DSP is (213) 740-0776.

Statement on Academic Integrity

USC seeks to maintain an optimal learning environment. General principles of academic honestyinclude the concept of respect for the intellectual property of others, the expectation thatindividual work will be submitted unless otherwise allowed by an instructor, and the obligationsboth to protect one’s own academic work from misuse by others as well as to avoid usinganother’s work as one’s own. All students are expected to understand and abide by theseprinciples. Scampus, the Student Guidebook, contains the Student Conduct Code in Section11.00, while the recommended sanctions are located in Appendix A: Students will be referred to the Office of Student Judicial Affairs and Community Standards forfurther review, should there be any suspicion of academic dishonesty. The Review process canbe found at:

Course Outline:

Week 1 (8/25): The Scientific Method

Lecture readings:

  1. Trochim and Donnelly, Chapter 1

Discussion readings:

  1. Schwartz, Thomas. 1980. The Art of Logical Reasoning. New York: Random House. Pp. 3-53, skim rest.
  2. Friedman, Milton, 1953. The Methodology of Positive Economics, in Friedman, Essays in Positive Economics. Chicago, IL: University of Chicago Press.
  3. Satz, Debra, and John Ferejohn. 1999. Rational Choice Theory and Folk Psychology. Unpublished ms.
  4. Weingast, Barry. 1979. A Rational Choice Perspective on Congressional Norms. American Journal of Political Science 23:245-62.

Week 2 (9/2): Methods of Observation and Inference

Lecture Readings:

  1. Curd, Martin and J.A. Cover. 1998. Philosophy of Science: The Central Issues. New York: W.W. Norton. Chapters by Ruse, pp. 38-47, Hempel and Snyder, pp. 445-480.
  2. Trochim and Donnelly, Chapter 3.

Discussion Readings: TBA

Week 3 (9/8): The Theory of Measurement, Sampling, and Scaling

Lecture Readings:

  1. Trochim and Donnelly, Chapters 2, 4, and 5.
  2. Goertz, Gary. 2006. Social Science Concepts: A User’s Guide. Princeton, NJ: Princeton University Press, Chapter 1.
  3. King, Gary, Robert O. Keohane, and Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, NJ: Princeton University Press. Chapters 4-5.

Discussion Readings:

  1. Christenfeld N., Philips, DP, and Glynn LM. 1999. What’s in a Name: Mortality and the Power of Symbols. Journal of Psychosomatic Research, 47, 3:241-54.
  2. Freedom House, Freedom in the World: Methodology. Available at
  3. James H. Fowler and Sangick Jeon. 2008. “The Authority of Supreme Court Precedent,” Social Networks 30: 16-30.
  4. Lake, David A. 2009. Hierarchy in International Relations. Ithaca, NY: Cornell University Press, Chapter 3, pp. 63-92 (note: you may want to skim Chapters 1 and 2).

Week 4 (9/15): Design, Validity, and Disconfirmation

Lecture Readings:

  1. Trochim and Donnelly, Chapter 7.
  2. Cook, Thomas D. and Donald T. Campbell. 1979. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin. Chapters 1 and 2.

Discussion Readings:

  1. Donald T. Campbell et al. 1968. “Analysis of Data on the Connecticut Speeding Crackdown as a Time-Series Quasi-Experiment,” Law and Society Review 3, 1: 55-76.
  2. Bartels, Larry. 2005. Homer Gets a Tax Cut: Inequality and Public Policy in the American Mind. Perspectives on Politics 3, 1:15-31.
  3. Lupia et al. 2005. Were Bush Tax Cut Supporters ‘Simply Ignorant?’ A Second Look at Conservatives and Liberals in ‘Homer Gets a Tax Cut.’ Perspectives on Politics 5, 4:73-784.

Week 5 (9/22): Experimental Design

Lecture Readings:

  1. Trochim and Donnelly, Chapter 9.
  2. Donald R. Kinder and Shanto Iyengar. 1987. News That Matters. Chicago, IL: University of Chicago Press. Chapter 2.
  3. Green, Donald P. and Alan S. Gerber. 2002. Reclaiming the Experimental Tradition in Political Science. In Political Science: State of the Discipline, ed. By Ira Katznelson and Helen V. Milner. New York: W. W. Norton. Pp. 805-32.
  4. (recommended) Cook, Thomas D. and Donald T. Campbell. 1979. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston: Houghton Mifflin. Chapter 8.

Discussion Readings:

  1. Lupia, Arthur and Mathew D. McCubbins. 1998. The Democratic Dilemma. New York: Cambridge University Press. Pp. 1-14, 101-148.
  2. Miguel, Edward and Michael Kremer. 2004. Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities. Econometrica 72: 159-217.
  3. Tomz, Michael. 2007. Domestic Audience Costs in International Relations: An Experimental Approach. International Organization 61:821-840.
  4. Kahneman, Daniel and Amos Tversky. 1979. Prospect Theory: An analysis of decision under risk. Econometrica 47: 263-292.
  5. Sears, David O. 1986. “College Sophomores in the Laboratory: Influences of a Narrow Data Base on Social Psychology’s View of Human Nature.” Journal of Personality and Social Psychology 51: 515-530.

Week 6 (9/29): Quasi-Experimental Designs: Non-Equivalent Groups

Lecture Readings:

  1. Trochim and Donnelly, Chapter 10.
  2. Cook, Thomas D. And Donald T. Campbell. 1979. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin. Chapters 3 and 5.

Discussion Readings:

  1. Crosier, Scott. John Snow: The London Cholera Epidemic of 1854.
  2. Jared Diamond, “A Natural Experiment of History,” in Guns, Germs and Steel. New York: W.W. Norton and Company, 1999, pp. 53-66.
  3. Hyde, Susan. 2997. The Observer Effect in International Politics: Evidence from a Natural Experiment. World Politics 60: 37-63.
  4. TBA

Week 7 (10/4): Quasi-Experimental Designs: Regression Discontinuity and Matching

Lecture Readings:

  1. Trochim and Donnelly, Chapter 10.

Discussion Readings:

  1. Acemoglu, Daron, Simon Johnson, and James Robinson. 2001. The Colonial Origins of Comparative Development. American Economic Review 91: 1369-1401.
  2. Donohue, John, and Steven Levitt. 2001. The Impact of Legalized Abortion on Crime. Quarterly Journal of Economics 116: 379-420.
  3. Lee, D.S., Moretti, E., and M. Butler, (2004), Do Voters Affect or Elect Policies? Evidence from the U.S. House, Quarterly Journal of Economics 119, 807-859.
  4. Cellini, Stephanie Riegg, Fernando Ferreira, and Jesse Rothstein. 2009. The Value of School Facility Investments: Evidence from a Dynamic Regression Discontinuity Design. Quarterly Journal of Economics (forthcoming).

Week 8 (10/13): Design Validity

Lecture Readings:

  1. Trochim and Donnelly, Chapters 11, 12.1.

Discussion Readings:

  1. Den Hartog, Christopher and Nathan W. Monroe. 2005. “The Value of Majority Status: The Effect of Jeffords’s Switch on Asset Prices of Republican and Democratic Firms.” Legislative Studies Quarterly.
  2. Daniel E. Ho, Kosuke Imai, Gary King and Elizabeth A. Stuart. 2007. “Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis 15: 199-236.
  3. Bratton, Michael and Nicolas van de Walle. 1997. Democratic Experiments in Africa: Regime Transition in Comparative Perspective. New York: Cambridge University Press. Chapter 3.

Week 9 (10/20): Differences in Differences; Differences-in-Differences-in-Differences

Lecture Readings: tba

Week 10 (10/27): Instrumental Variables

Lecture Readings: tba

Week 11 (11/3): Propensity Score Matching I

Lecture Readings: tba

Week 12 (11/10): Propensity Score Matching II

Lecture Readings: tba

Week 13 (11/17): Regression Discontinuity Designs

Lecture Readings: tba

Week 14 (11/24): Synthetic Controls

Lecture Readings: tba

Week 15 (11/3): Presentations

Lecture Readings: None.

Supplementary materials available on the Internet:

An excellent introduction to statistics and research design is Statistics at Square One --

see especially Chapter 5 --

Good websites on statistics, econometrics, including free downloadable software for data entry, data analysis, research design, hypothesis testing, document preparation and presentation include:

Online readings on the scientific method:

Useful online articles on qualitative research:

Top View