EPS 780: RESEARCH PRACTICUM ON MIXED METHODS ANALYSIS
Associate ProfessorSara Goldrick-Rab
Educational Policy Studies and Sociology
MEETING DETAILS:
Fall and Spring 2011-2012
Class meetings: Wednesdays, 9:15 am-12:00 pm, L150 Education Building
STAFF:
Sara Goldrick-Rab—Email preferred: . I can be reached by phone at 608-265-2141. Office is 211 Education Building.Office hours:Wednesdays, 1:00pm-2:30pm, 211 Education Building
Peter Kinsley—Peter will be the class’s teaching assistant. I have asked him to provide you with methodological help (e.g., advice on statistics or Dedoose) and insights regarding WSLS. Peter can be reached at . He will hold weekly labs on Wednesdays from 12:00pm-1:00pm in Education L177.
Alison Bowman—Alison is the WSLS project manager. You should contact her regarding any logistics questions (IRB, Dedoose password, etc.). She can be reached at . Her office is in 575 Education Sciences.
DESCRIPTION:
This two-semestercourse covers the theory and practice of mixed methods research in the social sciences, with a special emphasis on educational research. It is a practicum in which each student undertakes a project with linked qualitative and quantitative components, with the intention of producing a publishable research paper. The course covers the nuts and bolts of constructing research, at the same time maintaining an overall emphasis on the specific approach known as “mixed methods” research.
A central advantagefor enrolled students is the opportunity to use survey, administrative,and interview data from the first three years of the Wisconsin Scholars Longitudinal Study (WSLS), a randomized trial of need-based financial aid ( That study, directed by Sara Goldrick-Rab and Douglas N. Harris, is housed at the University of Wisconsin-Madison. It is the first statewide, longitudinal, mixed-methods study to examine how and why financial aid affects the undergraduate experience. The rich data included in the study facilitatesthe study of numerous topics not related to financial aid, including: the roles of identity, motivation, and self-efficacy in the educational attainment process; the process of college transfer; racial/ethnic differences in the college experience; the formation of social networks during college; the role of institutional practices and actors in student outcomes;and the influence of health and healthy behaviors on educational attainment. Students are encouraged to think broadly about the potential uses of the WSLS data and develop research questions related to their own areas of training and study. They are also permitted to write papers that compare the WSLS data to other data sources (for example, national datasets).
PREREQUISITES:
Allgraduate students interested in mixed-methods approaches to data analysis are welcome. Prior coursework in higher education is not assumed. I recommend that students have completed at least one course in quantitative methods and in qualitative methods—but this isnotrequired. The course is appropriate for both new and continuing graduate students, as there is space provided for students to work at their own pace, and time built in for instruction in “building block” research skills. The course meets the “education in context” and “measurement” requirements of the Interdisciplinary Training Program. It also counts toward the “policy” and “sociology” concentrations in Educational Policy Studies (and possibly others, according to the preferences of students’ advisers). In the past, students taking the course have made significant progress on a master’s thesis or dissertation proposal during the year.
REQUIREMENTS:
1. I expect every student to commit to attend and actively participate in every class session.
This course has limited enrollment and is intended to be a seminar in which the students and the professor actively engage and interact with one another in a mutually beneficial way. This only works if all students regularly attend. I ask that you make every effort not to miss any class—I have already devoted several weeks to your independent work on papers, allowed time off for the key education conference, and recognized holidays. I also expect you to do the readings assigned in advance of each class. I reserve the option of introducing pop quizzes (to constitute 10% of the final grade) if attendance is uneven or if reading is not done.
I also highly recommend regular attendance at my office hours. Office hours are the most appropriate time during which to seek advising on your paper, which should be an ongoing effort throughout the year.
2. I require compliance with the Wisconsin Scholars Longitudinal Study human subjects and authorship policies.
Through this course you are granted access to a restricted-use dataset that is still being constructed. This is a rare opportunity that could pay dividends if you are successful at writing publishable papers.
Access to this data requires that you:
(a) Complete human subjects training and sign on to the Institutional Review Board paperwork for the project (to be done during the initial weeks of class) and
(b) Sign the class authorship agreement (to be introduced during the first week of class).
No one is required to complete either; however access to the data is not allowed until both are done.
3. I require completion of the course assignments over the two-semester academic year.All of these are intermediate steps en route to the final assignment.
ASSIGNMENTS
The course assignments follow. Please submit all memos and papers via Learn@UW in the form of a PDF. I grade using an iPad and return comments to you that way.
MEMO #1: Statement of Interest(DUE 9/28)
This memo is a 1-2 page (double-spaced) description of a student’s chosen WSLS research topic. Please provide some theoretical and/or policy basis for selecting your topic, an indication of the literature that will form the backdrop for the research (a short bibliography is acceptable).
MEMO #2: Study Motivation and Literature Review (DUE 10/26)
This memo (approximately 6-8 pages) should build on Memo #1, clearly stating the theoretical or policy concern you plan to address (make the case, why is this important?), describe what we need to know (and currently do not) about the issue, and discuss the relevant theory and literature. I will return feedback on Memo #2 by 10/30; you should plan to discuss it in office hours with me during the follow week or two as needed.
MEMO #3: Study Motivation, Literature Review, and Research Questions (DUE 11/16)
This memo (approximately 7-9 pages) should be a revision of Memo #2 and include 2-3 specific research questions that should explicitly motivate a mixed methods analysis.
MEMO #4: Proposed Sample and Methodology (DUE 12/21)
This memo (approximately 5 pages) should outline the sample (from the WSLS and any other dataset you choose) you want to utilize and the details of your proposed methodology. Include estimated sample sizes, planned measures for key concepts (e.g., draft codes), and an overview of your analytic plan.
MEMO #5: Revised Sample and Methodology (DUE 2/8)
Following the feedback I provide as well as the results of the group workshop, revise Memo #4 and resubmit.
MEMO #6: Preliminary Analysis (DUE 3/7)
Provide a set of tables and/or figures that demonstrates where you think the “gold” is in your analysis. Write a brief description of what key concerns or questions you are now facing.
MEMO #7: Draft of Full Paper (DUE 4/18)
Include all sections from prior memos along with a draft set of results and analysis.
PRESENTATION(5/2 and 9)
On a designated day, you will present for 10 minutes a powerpoint describing your study.
FINAL PAPER (DUE 5/16)
The paper should be a full, complete draft that would be acceptable for submission to an academic conference. It should be no more than 30 pages, excluding tables, references, and appendices. It should include references that conform to either APA or Chicago style. Select papers will receive the opportunity to present at the 2012 WSLS conference and/or be included in an edited volume of WSLS papers.
GRADING:
This is a two-semester course requiring activities and deliverables across both semesters. Therefore, at the conclusion of semester 1, students will be given a grade of “pass.” At the end of semester 2, a grade will be assigned that will serve for both semesters (taking the place of the pass, and assigned to semester 2). If this grading approach presents a significant challenge, please notify me at the start of the first semester.
The grading rubric is as follows:
Memo 1: 5% of total grade, graded on scale of 1-5
Memos 2-6:10% of total grade each, graded on scale of 1-20 (total=50%)
Memo 7: 15% of total grade, graded on scale of 1-100
Presentation: 5% of total grade, graded on scale of 1-10
Final Paper: 25% of the total grade, graded on scale of 1-100
I will assign letter grades corresponding to the following structure:
A94-100
B+90-93
B84-89
C+80-83
COURSE MATERIALS:
I strive to keep the costs of this course to a minimum. Except for books (see below) all readings and homework materials are accessible via the course website at Learn@UW. I’ve placed them into a single zipped folder for ease of downloading. Feel free to download them to your laptops and bring laptops to class for note-taking, rather than printing paper. However, I ask that you TURN OFF your internet connection during class unless explicitly instructed otherwise. Email or Facebook during class will not be tolerated.
BOOKS:
The following three books are required. They are available on Amazon, in paperback, for under $25 each. Please obtain copies quickly.
Duncan, Huston, and Weisner. 2009. Higher Ground: New Hope for the Working Poor and Their Children. Russell Sage Foundation.
Lieberson, S. 1987. Making it Count: The Improvement of Social Research and Theory. University of California Press. (Earlier versions are fine)
Yoshikawa, Weisner, and Lowe. 2009. Making It Work: Low-Wage Employment, Family Life, and Child Development. Russell Sage Foundation.
In addition, I urge those of you who have not taken graduate statistics courses to obtain the following book and read it early in the course.
Vickers, Andrew. 2010. What is a p-value anyway? 34 Stories to Help You Actually Understand Statistics. Addison-Wesley.
SOFTWARE:
Dedoose software is used in the class; this is accessible via Web and I am paying all fees associated with your usage and will also work to train you. You will keep that license until the end the calendar year, following the term in which the class concludes, to allow you to revise and publish your paper.Alison Bowman is in charge of initiating your Dedoose account, and any general questions or problems with Dedoose should be directed to her. Questions related to analysis should be directed to Peter Kinsley or myself.
All other software packages you need can be accessed via the Social Sciences Computing Cooperative (SSCC).
DATA:
Depending on whether you have prior experience working with the WSLS, you will either utilize a redacted version of the WSLS quantitative dataset provided via MyWebSpace (password and access information to be provided in class on the designated date) or you will utilize the full dataset through the WSLS website (information provided separately).
In addition, I will make interview transcripts available to you via MyWebspace (password and access information to be provided in class on the designated date) so that you can read interviews in their entirety. However, in addition, interview data is preloaded into Dedoose for your use when coding.
SPEAKERS
We will have several guest speakers during the year who will meet with us in person or via the Bascom Hill Telepresence Classroomto share their perspectives as practitioners of mixed-methods research. It is especially important that you plan to attend class on the days they will join us. The speakers include:
Regina Deil-Amen, Associate Professor of Higher Education, University of Arizona.
Stefanie DeLuca, Associate Professor of Sociology, Johns Hopkins University.
Eli Lieber, Associate Research Psychologist in the UCLA Department of Psychiatry and Biobehavioral Sciences and co-director of the Fieldwork and Qualitative Data Research Laboratory, Center for Culture and Health.
Katherine Magnuson, Associate Professor of Social Work, UW-Madison
Fran Schrag, Professor Emeritus of Educational Policy Studies, UW-Madison
Thomas Weisner, Professor of Anthropology and Psychiatry at UCLA.
SCHEDULE OF READINGS AND ASSIGNMENTS
DATE / TOPICS / DUE ON THIS DATE / NOTES9/7/11 / Course logistics; Overview of higher education research and areas for further research
9/14/11 / Intro to the Wisconsin Scholars Longitudinal Study; Discussion of potential topics for class papers / Complete IRB training; Goldrick-Rab et al., 2009; Goldrick-Rab et al., 2011; listen to assigned audio files / We will review top-level reports in today’s class—continue reviewing them afterward
9/21/11 / Methodology: What it means, why it matters / Leahey, 2008; Lieberson Ch1 / You will have access to interview transcripts by today—begin reading
9/28/11 / Mixed methods research: Defining terms, considering examples (part 1) / MEMO 1 DUE
Johnson, Onwuegbuzie, & Turner, 2007; Kinsley & Goldrick-Rab 2011; Figueiredo-Brown 2011; Carter, 2010
10/5/11 / Mixed methods research: Defining terms, considering examples (part 2) / Higher Ground (whole book)
10/12/11 / Research questions: Formulations & hypothesis generation / Becker, 1996; Lieberson Ch 5; Schrag, 1989; Small, 2009 / Guest Speaker: Fran Schrag
10/19/11 / Mixed methods research: Defining terms, considering examples (part 3) / Making it Work (whole book)
10/26/11 / Designing mixed methods research / MEMO 2 DUE
Creswell & Plano Clark, 2007 Ch 4; Gibson & Duncan, 2005; Yoshikawa et al., 2008
11/2/11 / Coding and creating scales / Weisner, 2011 / Guest Speaker: Tom Weisner
Peter leads class
11/9/11 / Art and science of interviewing (we will discuss audiofiles in class) / Hochschild, 2009 / Access to quant data begins...
11/16/11 / Statistical inference, selection, comparisons / MEMO 3 DUE
Lieberson (Ch 2,-4,6, 7); Murnane &Willett, 2011 Ch 3 / Use Vickers book as needed to supplement
11/23/11 / HAPPY THANKSGIVING
11/30/11 / Dedoose training / Read interview transcripts and visit Dedoose website to look around / Guest speaker Eli Lieber (10-12). Coding of qual data begins!
12/7/11 / Causality and experimentation / Howe, 2004; Lieberson Ch 8-11; Maxwell, 2004
12/14/11 / Workshop on Memo 3 / Read Memo 3 for members in your assigned group
MEMO 4 Due 12/21
12/22/11-1/24/12 / WINTER BREAK / Work on analyzing data to get ahead!
1/25/12 / Workshop on Memo 4 (part 1) / Read Memo 4 for members in your assigned group
2/1/12 / Analyzing mixed methods data / Caracelli & Greene, 1993; Teddlie & Tashakkori, 2009
2/8/12 / Workshop on Memo 4 (part 2) / MEMO 5 DUE
2/15/12 / Strategies for collecting data for mixed methods research, including how to structure interview protocol, survey design, how to work with a team, timeline (community engagement) / Harris & Goldrick-Rab, 2012; Teddlie & Yu, 2007
2/22/12 / Ethical dilemmas in mixed methods research / Leahey, 2007; Fine, 2011
2/29/12 / Writing and displaying mixed methods research / Bryman, 2007; Mertens, 2011; Alise & Teddlie, 2010; Dougherty
Workshop by Eli
3/7/12 / Planning, budgeting, and fundraising for mixed methods research
(how to put proposals and how they are evaluated) / MEMO 6 DUE
Creswell et al., 2011
3/14/12 / Workshop on Memo 6 (group 1) / Read Memo 6 for members in your assigned group
3/21/12 / Conducting mixed methods research / Deil-Amen & DeLuca, 2010; Deil-Amen & Goldrick-Rab, 2010 / Guest speaker Regina Deil-Amen
3/28/12 / Workshop on Memo 6 (group 1) / Read Memo 6 for members in your assigned group
4/4/12 / SPRING BREAK
4/11/12 / Conducting mixed methods research; Research meets Policy / Weiss, 2008 / Guest speaker Katherine Magnuson
4/18/12 / NO CLASS-- AERA / MEMO 7 DUE
4/25/12 / Conducting mixed methods research / DeLuca & Rosenblatt, 2010 / Guest speaker Stefanie DeLuca
5/2/12 / PRESENTATIONS
5/9/12 / PRESENTATIONS / FULL PAPER DUE 5/16
READINGS
Alise, M.A., & Teddlie, C. (2010). A continuation of the paradigm wars? Prevalence rates of methodological approaches across the social/behavioral sciences. Journal of Mixed Methods Research, 4(2), 103-126.
Becker, H. S. (1996). The epistemology of qualitative research. In R. Jessor, A. Colby, & R. Schweder (Eds.), Essays on ethnography and human development(pp. 53-71). Chicago: University of Chicago Press.
Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8-22.
Caracelli, V. J.,Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis,15(2), 195-207.
Carter, P. L. (2010). A venture into mixed methods: race and cultural flexibility among students in different multiracial schools.Teachers College Record.
Creswell, J. W. et al. (2011). Best practices for mixed methods research in the health sciences. Bethesda, MA: National Institutes of Health Office of Behavioral and Social Sciences Research. Retrieved from
Creswell, J. W.,Plano Clark, V. (2011). Choosing a mixed methods design. Designing and conducting mixed methods research(2nd ed.). Thousand Oaks, CA: Sage.
Deil-Amen, R., DeLuca S. (2010). The underserved third: How our educational structures populate an educational underclass. Journal of Education for Students Placed at Risk,15, 27–50.
Deil-Amen, R., & Goldrick-Rab, S. (2010).Institutional transfer and the management of risk in higher education (WISCAPE Working Paper). Madison, WI: Wisconsin Center for the Advancement of Postsecondary Education. Retrieved from
DeLuca, S., Rosenblatt, P. (2010). Does moving to better neighborhoods lead to better schooling opportunities? Parental school choice in an experimental housing voucher program. Teachers College Record, 112(5), 1443-1491.
Figueiredo-Brown, M. R. (2011). Family support of Latino/a college students: How and why does gender matter? (Unpublished working paper). Madison, WI: University of Wisconsin.
Fine, C. (2011, July 30). Biased but brilliant. New York Times. Retrieved from
Gibson, C. M., & Duncan, G. J. (2005). Qualitative/quantitative synergies in a random-assignment program evaluation. InT. Weisner (Ed.), Discovering successful pathways in children’s development: New methods in the study of childhood and family life(pp.283-315). Chicago: University of Chicago Press.
Goldrick-Rab, S., Harris, D. N., Trostel, P. A.(2009). Why financial aid matters (or doesn’t) for college success: Toward a new interdisciplinary perspective. Higher Education: Handbook of Theory and Research, 24, 1-45.
Goldrick-Rab, S. Harris, D. N., Benson, J., & Kelchen, R.(2011). Conditional cash transfers and college persistence: Evidence from a randomized need-based grant program (IRP Discussion Paper 1393-11). Madison, WI: Institute for Research on Poverty. Retrieved from