MEJO890 Experimental Design

Spring 2018

Instructor: Allison Lazard, PhD

Assistant Professor

384, Carroll Hall

School of Media and Journalism

University of North Carolina at Chapel Hill

Meeting Time: Thursday 2-4:45pm in 340A, Carroll Hall.

Office Hours: Tuesday11:30am-1pm and by appointment.

Course Overview: This course will focus on the methodological and design issues in planning an experiment to enable you to fully design and conduct an experiment. Rather than just reading about controlled experiments and field experiments, single factor experiments and factorial designs, manipulation checks, etc., we will walk through the steps in deciding which of these elements is best used in the creation of your own experiment, including making the stimuli and questionnaire, submitting an IRB application, etc. By the end of the semester you will run your own experiment. It is important to note that this class differs from experimental design course offered from other departments, such as psychology, which deal primarily with analyzing data with various statistics. While we will cover this aspect briefly, the inner workings of the statistics, formulas, and calculating them by hand will not be part of this class; you should take a traditional experimental design class to complement the conceptual knowledge gained in this course if desired.

Required Reading:

  1. Bausell, R. B. (1994). Conducting Meaningful Experiments: 40 Steps to Becoming a Scientist. Los Angeles: Sage.
  2. Shadish, W.R., Cook, T. D., & Campbell, D.T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference Belmont, CA: Wadsworth.
  3. Readings posted on the reading list, handed out during class, or available online.

Course Work & Grading:

Assignment / Weight
Reflection papers / 15%
Weekly assignments (8) / 30%
Final paper / 45%
Class participation / 10%

Reflection papers:

You will submit a total of 3 one-page reflection or thought papers that analysis and synthesizes an idea (or ideas) in one or more of the course readings for that week. All reflection papers must be submitted before the beginning of class. No reflection papers will be accepted late, via email, or from another student on your behalf. There are 10 weeks of assigned reading. You may choose to do the reflection papers any week. Only one reflection paper for the week’s assigned reading allowed. The purpose of these papers is to give you the opportunity to think, and more importantly write, about insights gained from the readings so that you will meaningful contribute to the class discussion and further progress on your study. You may critique the reading, write about questions you have, or any other angle you would like. This is not a summary though. Do not summarize any of the readings. Pick a topic and write the whole paper on it. You will get a better grade for one in-depth topic than 2 (or more) superficial ones. This is your opportunity to demonstrate your ability for critical thinking.

Weekly assignments:

There are a total of 8 weekly assignments: 3 ideas, literature review, hypotheses, sampling strategy, stimuli, manipulation check, questionnaire, and IRB application/submission. These are opportunities to build and get feedback on your design prior to conducting the experiment and submitting the final paper.

Final experimental design paper:

The final paper and presentation should demonstrate the culmination of the entire semester’s work. This is a standard research paper suitable for a conference submission. The paper should be no longer than 7000 words(or less if norm in your subfield) and include: introduction, theory and literature review, hypotheses, a complete methods section, results, and a discussion. This paper is expected to be of much higher quality than the weekly assignments. It needs to be clearly and concisely written and suitable to a peer-review submission process. With permission and sufficient rationale, proposal written up to the results section may be accepted.

Grade scale: H 100-95; P94.99-80; L79.99-60; F 59.99-0

Grade categories are defined as:

  • H - The work is intellectually rigorous, shows an exceptional understanding of the material and is error free. Ready for publication with minor revisions.
  • P - The work illustrates a good effort at understanding the material and has few errors. Revise-and-resubmit.
  • L- The work indicates little progress toward gaining an understanding of the material and has substantial errors. Reject.
  • F - The work shows no understanding of the assignment or was not completed in a timely manner.

Schedule: The schedule includes the main topics for each week and the reading required for themeeting. Students are expected to have completed the assignedreadings BEFORE coming to class.Note: Based on your input and our progress, I reserve the right to amend and change thesyllabus, reading schedules, and grading events during the semester.

Week / Dates / Topic, Reading, & Assignments
1 / 1/11 / INTRODUCTION HISTORY OF EXPERIMENTS
Assignment due next Wed (week 2, 1/18): 3 ideas for an experiment you would like to conduct. Clear and concise statement of the problem, why it is important to scholars, practitioners, society. 250 words per idea.
2 / 1/18 / OVERVIEW OF EXPERIMENTS, THEORY, & LITERATURE
Read: SC&C Ch. 1 & 8; Bausell Ch. 1 & 2, Thorson, Wicks, & Leshner 2012
Assignment due this week: 3 ideas
Assignment due in two Wednesdays (week 4, 2/1): Revise 1 approved idea, add ~5 pages of theory and literature review.
3 / 1/25 / EXPERIMENTAL VARIALBES
Read:Baron & Kenny 1986; Spencer, Zanna, & Fong 2005; Stevens 2012; TBD
Assignment due this week: None
4 / 2/1 / VALIDITY & HYPOTHESES
Read: SC&C Ch. 2 &3; Bausell Ch. 3 & 4; Coleman 2006
Assignment due this week: Revised idea with literature review
Assignment due next Wednesday (week 5, 2/8): 3+ Hypotheses and any Research Questions
5 / 2/8 / EFFECT SIZES SAMPLING
Read: Basil, Brown, & Bocarnea 2002; Cohen 1992; Courtright 1996; Crump, McDonnell, & Gureckis 2013; Lang 1996; O’Keefe 2017; VanVoorhis & Morgan 2007
Assignment due this week: 3+ Hs and any RQs
Assignment due next Wednesday (week 6, 2/15): Write up the sampling strategy for your study, including a power analysis.
6 / 2/15 / MEASUREMENT & STIMULI DESIGN
Read:Jackson & Jacobs 1983; O’Keefe 2003; TaoBucy 2007
Assignment due this week: Sampling strategy
Assignment due in two Wednesdays (week 8, 3/1): Design your stimuli.
7 / 2/22 / RANDOM ASSIGNMENT & FACTORIAL DESIGNS
Read:Bausell 5 & 6; Ho 2008; Smith, Levine, Lachlan, & Fediuk 2002
Assignment due this week: None
8 / 3/1 / MANIPULATION CHEKCS, PRETESTS, & PILOT STUDIES
Read:Bausell Ch. 7-9; Arpan, Baker, Lee, Jung, Lorusso, & Smith 2006
Assignment due this week: Stimuli
Assignment due next Wednesday (week 9, 3/8): Write your manipulation check for your study (& pretest if applicable).
9 / 3/8 / QUESTIONNAIRES & PROCEDURES
Read: SC&C Ch. 4 & 5; Coleman, Thorson, & Wilkins 2011
Assignment due this week: Manipulation check (& pretest if applicable)
Assignment due in two Wednesdays (week 11, 3/22): Design your questionnaire and complete IRB application.
10 / SPRING BREAK
11 / 3/22 / ETHICS & WRITING UP METHODS
Read:SC&C Ch. 9; (Optional) Select any two peer-reviewed, experiment design journal articles, analyze and critique the methods section write-up. Is there a pattern that you prefer (e.g., order of stimuli, sample, procedure, etc.)? Is there enough information to replicate the study? Is one method section better than the other? If yes, why? This purpose of this thought paper should be to closely read and think critically about how methods sections are written (well and/or poorly).
Assignment due this week: Questionnaire and IRB application
Assignment due nextWednesday (week 12, 3/29): Submit IRB application
12 / 3/29 / ANALYZING & INTERPRETING DATA
Read: O’Keefe 2007; Levine 2002; Levine 2013
Assignment due this week:Submitted IRB application
13 / 4/5 / WRITING UP RESULTS
Read:(Optional) Select any two experimental design studies, analyze and critique the way the results are written in the same manner as done for methods.
Assignment due this week: None
14 / 4/12 / CONSULTATIONS
Assignment due this week: None
15 / 4/19 / CONSULTATIONS
Assignment due this week: None
16 / 4/26 / FINAL PRESENTATIONS
Prepare a presentation with visuals a la AEJMC or ICA conference. No longer than 10 minutes each. You will be timed. Questions from the audience immediately after each presentation.
Final papers due– Monday, May 7, 2017by 5:00pm

Honor Code: I expect that each student will conduct himself or herself within the guidelines of the University honor system (http://honor.unc.edu). All academic work should be done with the high levels of honesty and integrity that this University demands. You are expected to produce your own work in this class. If you have any questions about your responsibility or your instructor’s responsibility as a faculty member under the Honor Code, please see the course instructor or Senior Associate Dean Charlie Tuggle, or you may speak with a representative of the Student Attorney Office or the Office of the Dean of Students.
Seeking Help: If you need individual assistance, it’s your responsibility to meet with the instructor. If you are serious about wanting to improve your performance in the course, the time to seek help is as soon as you are aware of the problem – whether the problem is difficulty with course material, a disability, or an illness.
Diversity and Inclusion:The School of Media and Journalism adopted diversity and inclusion mission and vision statements in spring 2016 with accompanying goals. It complements the University policy on Prohibiting Harassment and Discrimination. In summary, UNC is committed to providing an inclusive and welcoming environment for all members of our community and does not discriminate in offering access to its educational programs and activities on the basis of age, gender, race, color, national origin, religion, creed, disability, veteran’s status, sexual orientation, gender identity, or gender expression.
Special Accommodations:If you require special accommodations to attend or participate in this course, please let the instructor know as soon as possible. If you need information about disabilities visit the Accessibility Services website at https://accessibility.unc.edu/
ACCREDITATION: The School of Journalism and Mass Communication’s accrediting body outlines a number of values you should be aware of and competencies you should be able to demonstrate by the time you graduate from our program. Learn more about them here:
http://www2.ku.edu/~acejmc/PROGRAM/PRINCIPLES.SHTML#vals&comps
No single course could possibly give you all of these values and competencies; but collectively, our classes are designed to build your abilities in each of these areas. In this class, we will address a number of the values and competencies, with special emphasis on these six bullet dots under "Professional values and competencies" in the link above.

  • Demonstrate an understanding of the history and role of professionals and institutions in shaping communications;
  • Understand concepts and apply theories in the use and presentation of images and information;
  • Think critically, creatively and independently;
  • Conduct research and evaluate information by methods appropriate to the communications professions in which they work;
  • Write correctly and clearly in forms and styles appropriate for the communications professions, audiences and purposes they serve;
  • Apply tools and technologies appropriate for the communications professions in which they work.