UNIVERSITY OF SOUTHERN CALIFORNIA

Marshall School of Business

Fall 2009

GSBA 625 – DESIGNING AND RUNNING EXPERIMENTS (Section 16120R)

Instructor: Professor Sarah Bonner

Class Hours: Wed., 9:00 – 11:50, HOH 506

Course Website: http://blackboard.usc.edu

Office: HOH 622

Office Phone: (213) 740-5025

Email Address:

Office Hours: 2:00 – 3:30 TuTh, 12:00 – 2:00 Wed., and by appt.*

Emergencies: Call (213) 740-4321 (for a personal emergency) or (213) 740-9233

or listen to 91.5 KUSC radio (for emergency information)

USC Information: (213) 740-2311

Prerequisites: Departmental approval

Corequisites: None

*Also please feel free to email me at any time. I will respond to you as quickly as I can.

I. COURSE DESCRIPTION

This course provides an overview of the design and implementation of experiments. The first half of the course is devoted to understanding fundamental design issues and the various types of experimental designs available for answering research questions. The second half of the course is devoted to the various issues that arise when actually operationalizing one’s own experiment. These issues include measuring dependent variables, as well as measurement issues related to mediators and covariates; proper manipulation of independent variables; choice, recruitment, and treatment of experimental subjects; and proper experimental procedures including methods for avoiding demand/experimenter expectancy effects and minimizing noise.

II. COURSE PREQUISITE KNOWLEDGE

This course assumes you are familiar with basic principles of research design, as well as with basic psychological concepts since most experimental research in business is based on psychology. Additionally, since examples from business settings are used, some familiarity with basic business issues is assumed. Enrollment in this course is restricted to Ph.D. students.

III. COURSE GOAL AND LEARNING OBJECTIVES; COURSE FORMAT

The overall goal of this course is to introduce you to the key issues related to designing and running experiments. Subsumed under this goal are several specific learning objectives and desired outcomes related to those objectives:

n  You should gain factual knowledge of important terminology related to experiments, including, for example: random assignment, between-subjects design, within-subjects design, repeated measures, fully crossed factorial design, main effect, interaction, planned contrasts, post hoc comparisons, measurement, measurement error, reliability, confound, mediator, moderator, covariate, and manipulation check.

-- The desired outcome for this objective is that you be able to define and describe these concepts.

n  You should learn fundamental principles related to designing and running experiments. These fundamental principles are among the specific skills and competencies needed by experimental researchers. These principles include those related to the choice of a proper and efficient design and the analysis technique that is appropriate for that design, proper manipulation of treatments, proper measurement of dependent and other variables, proper control of noise and correlated omitted variables, proper sampling and treatment of subjects, and proper experimental procedures.

-- The desired outcome for this objective is that you be able to describe these principles.

n  You should learn to apply the above terminology to novel situations. This learning would include the ability to recognize, based on your understanding of terminology, whether something you read employs repeated measures, a fractional factorial design, etc. More important, you should also learn to apply the above principles to novel situations and begin to learn how to apply the above principles to your own experimental research – this would include determining whether another researcher (or you yourself) have chosen a proper design, properly measured or manipulated variables, used proper controls, etc.

-- The desired outcome for these objectives is that you be able to classify “good” (or “present”) vs. “bad” (“absent”) examples of important terminology and principles, e.g., whether a study has properly measured a specific type of dependent variable (e.g., memory recall). An additional desired outcome is that you can begin to properly apply the principles to your own work. You obviously cannot completely learn all there is to know about experiments in a one-semester course. This is why my objective is that you can begin to do these things.

To achieve the above learning objectives, I will employ a combination of background reading, interactive lecture, written work, and prompt feedback. Because research on learning indicates that having an initial conceptual framework substantially improves later learning, I will lecture to establish the framework and clarify the terminology and principles embedded therein. I expect questions during lectures. However, because my learning objectives include fostering your ability to apply terminology and principles, and beginning to learn to apply the principles to your own work, there also will be quite a bit of written work and feedback from me on that work. Research on learning indicates that it is very difficult to gain anything more than a superficial understanding of (i.e., memorize) material without practice and feedback. The best combination of techniques for learning difficult material deeply, then, is the introduction of the framework up front, followed by practice and feedback.

One specific element of most lectures will be what I’ll call “reality checks.” There are two types of reality checks that we will employ. In the first half of the class where we discuss various designs, we will talk about the real-world circumstances under which one might see or use such designs. There are two reasons for using this type of reality check. The first is that people tend to get into ruts with designs because they tend to get in ruts with research questions. Thus, getting into the habit of thinking about the circumstances under which you could use various designs might help you get out of a rut with research questions. Second, some designs have been invented because of real-world problems that experimentalists have faced. For example, fractional factorial designs are an alternative to fully-crossed factorial designs that can help if, for example, there is a restriction on subject time. In other words, experimental designs are not simply a jumbled up bag of tricks from which one pulls a choice, and thinking about the reality of the circumstances under which they are appropriate will reinforce your knowledge of how the designs fit together. In the second half of the course, the reality check sections will relate to the theoretical principles that govern the actual doing of experiments. These principles are sound and strict, but experimentalists have to operate in a world that often confronts them with restrictions, e.g., on money, time, subject availability, and laboratory conditions. As a result, we often are prevented from completely following the principles – we have to make tradeoffs. We will discuss the types of tradeoffs you might be confronted with and the effects of these tradeoffs on your research.

Another element of several classes will be the provision of a study that employs a “clever experiment” as part of your background reading. These studies were nominated by several colleagues who do experimental research, and are clever in various ways. They are included in the hope that you might be able to draw an analogy from the situations in these studies to a situation that you face. You need only skim these articles.

IV. REQUIRED TEXTS, READINGS, AND OTHER TOOLS; RECOMMENDED OR ADDITIONAL READINGS

The required texts for this course are:

n  Rosenthal, R., and Rosnow, R. Essentials of Behavioral Research: Methods and Data Analysis (3rd edition). Boston, MA: McGraw-Hill. 2007. ISBN:

978-0-07-353196-0 or 0-07-353196-0. Please shop around for this book. Used copies are available on the Internet for very low prices.

n  Shadish, W., Cook, T., and Campbell, D. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin. 2002. ISBN: 0-395-61556-9. You should be able to shop around for this book on the Internet as well.

Additionally, you will be assigned readings from psychology and business journals (listed in detail below). To the extent these materials are not available online, they will be distributed to you well ahead of time. You also may want to consider purchasing the following texts for your library at some point (if you do not already own them), as they are among the standard references for (behavioral) experimental researchers:

n  Kerlinger, F., and Lee, H. Foundations of Behavioral Research (4th edition). Forth Worth, TX: Holt, Rinehart and Winston. 2000.

n  Kirk, R. Experimental Design: Procedures for the Behavioral Sciences (3rd edition). Pacific Grove, CA: Brooks/Cole Publishing Company. 1995.

n  Nunnally, J., and Bernstein, I. Psychometric Theory (3rd edition). Boston, MA: McGraw-Hill. 1994.

n  Winer, B., Brown, D., and Michels, K. Statistical Principles in Experimental Design (3rd edition). Boston, MA: McGraw-Hill. 1991.

n  Cohen, J. Statistical Power Analysis for the Behavioral Sciences (2nd edition). Hillsdale, NJ: Lawrence Erlbaum. 1988.

n  Siegel, S., and Castellan, N., Jr. Nonparametric Statistics for the Behavioral Sciences (2nd edition). Boston, MA: McGraw-Hill. 1988.

For each topic we cover, there literally are hundreds of additional readings. Thus, I have not listed additional readings for each session, as this would be overwhelming. However, you should feel free to inquire further about any topic in which you have a particular interest.

V. GRADE COMPOSITION AND GRADING EXPECTATIONS

Total points for this course are 1000. Your letter grade will be determined based on your relative performance (vis-à-vis your peers).

Component / Points
Written assignments
Final exam
Class participation / 500
400
100
Total / 1000

My expectations for performance in each component of this class are as follows. First, with regard to the final exam, the exam will occur during finals period and will be closed book. Students will be allowed to take a make-up exam only under exceptional circumstances. You can expect to have some shorter definitional and recall questions, as well as lengthier qualifying-exam type questions because both types of questions are necessary for me to determine whether you have met the learning objectives of the course (see above). Shorter definitional and recall questions will be similar to those you have seen in undergraduate and masters-level courses. “Qualifying-exam” types of questions typically ask students to recall large amounts of material, to synthesize and organize material in new ways, and to apply the material to novel situations to demonstrate a relatively deep understanding thereof. Grading includes, but is not limited to, the following components: accuracy of recall (including references to particular articles and other sources, if required), completeness of recall, meaningful organization of material (i.e., the answer is not just a jumbled-up listing of concepts and facts), novelty of insights (if required), and appropriate application of material to new situations including, for example, the mapping of abstract concepts to details of the new situation. As performance on these components of the exam decreases, grades will decrease from a maximum of 400 points.

This course also requires FIVE written assignments because of the challenging nature of the learning objectives (see above). There are 500 points allocated to these assignments – the points for each assignment are indicated in the detailed description of the assignments below. I will be grading each assignment on whether you completed all parts of the assignment, the effort you put into the assignment as exhibited by your thoroughness and depth of thought, and, to a lesser extent, accuracy of application of concepts, in cases where we have covered the material pertinent to the assignment in some depth prior to the assignment being due. Additionally, your grade on Assignment 4 will depend on the quality of your presentation in class. In other words, I am not necessarily looking for you to always get a “right answer” unless we clearly have covered the material prior to the assignment, but rather that you are working hard and putting forth your best effort to learn. Assignments are due by the beginning of class on the date indicated, even if you must miss class. I will grade your assignments and provide feedback on them as promptly as possible, typically by the next class period.

Next, 100 points are allocated to general class participation – that related to discussions we have on days when there are no assignments due and/or discussions we have on days when assignments are due but that do not relate to your assignment. My expectations for this participation grade are that you will have read the background readings thoroughly and you will be prepared to discuss their major points and ask and answer questions. Your preparedness can be exhibited voluntarily or involuntarily (i.e., I will call on you). The main criterion for grading participation is effort, as exhibited by thoroughness of preparation and depth of thought. In discussions there often is no “right answer,” so little weight will be placed on an accuracy criterion.

VI. STATEMENT FOR STUDENTS WITH DISABILITIES

Any student requesting academic accommodations based on a disability is required to register with Disability Services and Programs (DSP) each semester. A letter of verification for approved accommodations can be obtained from DSP. Please be sure that the letter is delivered to me as early in the semester as possible. DSP is located in STU 301 and is open 8:30 AM – 5:00 PM, Monday through Friday. The phone number for DSP is (213) 740-0776.

VII. STATEMENT ON ACADEMIC INTEGRITY

USC seeks to maintain an optimal learning environment. General principles of academic honesty include the concept of respect for the intellectual property of others, the expectation that individual work will be submitted unless otherwise allowed by an instructor, and the obligation both to protect one’s own academic work from misuse by others as well as to avoid using another’s work as one’s own. All students are expected to understand and abide by these principles. Scampus, the Student Guidebook, contains the Student Conduct Code in Section 11.00, while the recommended sanctions are located in Appendix A: http://www.usc.edu/dept/publications/SCAMPUS/gov/. Students will be referred to the Office of Student Judicial Affairs and Community Standards for further review, should there be any suspicion of academic dishonesty. The Review process can be found at http://www.usc.edu/student-affairs/SJACS/.

Specifically, the following are my policies on academic integrity for this course. With regard to the final exam, sharing information with other students about the contents of or answers to the exams is considered a violation of the Student Conduct Code and will result in an “F” for the course. Additionally, using reference materials of any sort on the final (unless you are instructed to the contrary) is a violation of the Student Conduct Code and will result in an “F” for the course. With regard to written/discussion assignments and critiques of others’ assignments, my policy is that all these assignments and critiques are to be completed individually. It will be considered a violation of the Student Conduct Code if any discussion about a written/discussion assignment or a critique of another’s written assignment has occurred among students or from a student to, for example, a faculty member (except, of course, the simple transmission of the written assignment by one student to the other student responsible for critiquing it). If you have questions about assignments or critiques, please direct them to me. Students discussing assignments or critiques will receive a -0- for the item that they have discussed.