1

Meta Analysis

Spring 2005

EPSY 6349.002

Credit: 3 hours

Semester: Spring 2005

Instructor: Hansel Burley, Ph.D.

Office #: Educ 163, moving to 265

Office hrs: Monday, 1-4; Tuesdays, 1-3

Telephone: (806) 742-1997 ext. 224

Meeting time: Thurdays 6-8:50

Room Number: Education 257

E-mail:

Course Overview

This course examines the theoretical base, strategies, issues and research findings related to the application of the set of statistical techniques known as meta-analysis. The course will focus on the issues of traditional versus statistical reviews of the literature and the procedures for completing a statistical review. Students will perform a limited meta-analysis on a topic of interest. This meta-analysis will include a description of the topic, the library search, and the data menu for the study. Also, students will describe the coding procedure, overall analysis and analysis of subcategories of independent variables. Finally students will interpret their findings.

Course Objectives

As a result of this course, participants will be able to:

Demonstrate knowledge of basic meta-analysis techniques and procedures,

Review the literature using quantitative techniques,

Critically evaluate research studies,

Critically evaluate meta-analyses used in various contexts, including educational, medical, physiological, and psychological.

Course Resources

Lipsey, M. (2001). Practical meta-analysis (Applied Social Research Methods). Thousand Oaks, CA: Sage Publications

Wolf, F. (1986). Meta-analysis: Quantitative Methods for Research Systhesis. Thousand Oaks, CA: Sage Publications.

Course Requirements

1. Regular attendance. This includes meeting in class and on-line at prescribed times.

2. Participation in class discussions and completions of assigned tasks and readings.

3. Participate as a secondary reviewer.

5. Each participant will conduct a meta-analysis of a selected topic.

6. I recommend a floppy disk, for data set storage.

Class Activities

Lectures/Discussions–The instructor will provide information or lead discussions regarding meta-analysis topics.

1

Hands-on activities–The instructor will demonstrate and model specific techniques, particularly those dealing with SPSS.

Work in Progress Sharing and Conferencing–Time will be provided for students to conference with colleagues and the instructor concerning

demonstrations and student project.

Each student will write one paper, ready to be submitted to a scholarly journal for publication.

Each student will conduct a seminar related to his or her meta-analysis project.

If there is any student this class who has any disabilities, additional accommodations needs, or any suggestions that would lead to enhancing this course’s learning experiences, please talk with me before or after class, or during my office hours.

Topical Outline

  • Meta-analysis theory
  • Literature searches
  • Calculation of effect sizes, d and r
  • Transformations to d and r from
  • t, F, chi-square, log odds ratios and other measures of effect
  • effect size estimates and estimation
  • using actual data
  • confidence intervals
  • Aggregation of effect sizes, including
  • variance
  • Weights, including Hedges inverse of variance
  • Confidence intervals
  • Forest plots
  • Hedges’ ANOVA analogue for meta-analysis
  • Distribution for effect sizes
  • Omnibus test
  • Tests of homogeneity for subgroups
  • Interpretation
  • Coding and coding sheets
  • Data entry and reliability
  • Basic SPSS
  • Inter-rater reliability
  • Manuscript preparation
  • Meta-analysis intro
  • Meta-analysis methodology
  • Meta-analysis analysis section
  • SPSS training
  • Compute command
  • If and do if statements
  • SPSS matrix and matrix procedures and functions.

Schedule

Topic / Student activity
Week 1: Problems of traditional vs. statistical reviewing and synthesis of research (Validity issues will be detailed) / Brainstorm for topic; locate Internet access
Week 2: Search procedures (Computer searches and standard library branching searches). / Begin searching using indexes
Week 3: Coding Research Studies (Systematic approaches to selecting variables, study characteristics, and coding problems are examined) / Copy a few articles and begin coding
Week 4: Calculating effect sizes (Methods for calculating effect sizes in experimental and correlational studies are examined; transformations of non-traditional or unusual summary statistics are discussed / Interlibrary loan, library express orders need to be in by this day
Week 5: Analysis Techniques (Parametric techniques are reviewed / Locate computer tool for analysis
Week 6: Application of analysis techniques (Using provided data students will practice meta-analysis techniques; mock meta-analysis ) / Begin practicing with computer tool using given data
Week 7 Q & A day (a chance to ask questions about meta-analysis analysis Project / Certify that tool is accurate
Week 8: Use in educational contexts (Educational findings of selected meta-analyses are examined; special attention will be paid to comparisons of meta-analysis findings with traditional reviews of research) / Begin coding studies
Week 9: SPSS Day / Start writing.
Week 10: SPSS Day
Week 11: Writing the final report
Week 12: Student led seminar
Week 13: Student led seminar
Week 14 Student led seminar
Week 15 Student led seminar

Note: SPSS training will be part of nearly each class day.

Assignments

The ultimate assignment of to complete a rough draft of a meta-analysis manuscript that will be submitted for publication.

1. Select a small topic for review. Use library/Internet search to find key articles. Write a minimum four page historical and/or conceptual framework for the field.

2. Submit a data menu for the review will all variables and an initial legend of codes. Decide on the measure of effect size you plan to use.

3. Code all studies in your review. Make allowances for any reliability issues. Write a minimum four page description of the methodology you plan to use for your study.

4. Perform descriptive analyses on at least 3 major variables. Submit printouts and written summaries

1

5. Perform two detailed analyses of subcategories of subcategories of 2 different independent variables. Interpret findings, suggest one area for research in each based on missing or sparse research. Submit output and written summaries

6. After output and written summaries have been written, construct tables, graphs, and summaries in a format ready for publication.

7. Write a discussion section for the paper. Present the final paper to the entire class. Turn in the final copy of the paper.

Grading

Assignment 1 Conceptual Frame: 10 points

Assignment 2 Methodology: 20 points

Assignment 3 Summaries, Tables and Graphs: 30 points

Assignment 4 Final paper : 30 points

Assignment 5 Presentation: 10 points

Criteria for assignments will be given on the day the project parts are assigned. After the instructor get a better sense of student’s SPSS skills, a calendar of due dates for the assignments will be given out.

Reference List

Cook, T. D. et al. (1992). Meta-analysis for explanation: A casebook. New York: Russell Sage Foundation.

Glass, G. V., McGraw, b., & Smith, M.L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.

Hedges, L. V. & Olkin, J. (1985). Statistical methods for meta-analysis. New York: Academic press.

Hunter, J. E. & Schmidt, F. L (1990). Methods of meta-analysis: correcting error and bias in research findings. Newbury Park: Sage.

Light, R. J. (Ed.) Evaluation studies, vol 8. (1983). Beverly Hills, CA: Sage (An edited book of 36 articles on the topic of meta-analysis.

Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly hills, CA: Sage.

Wolf, F. M. (1986). Meta-analysis: quantitative methods for research synthesis. Newbury Park: Sage.