Research and Evaluation
(APPL.633.185)
Tuesdays, 5:30-8 p.m.
AC 212
Spring, 2017
3 credits
Instructor: Elaine Johnson, Ph.D. Email:
Office: LC 407 Office Hours: T, 4-5 p.m. and by appt.
Phone: 410.837.6683 (email preferred) Fax: (410) 837-4059
Sakai: On your myUB portal, click “UBOnlineSakai” and log in using your net ID. Tab to this course. You will have access to announcements, the syllabus, and other postings. You will also submit papers
by the due date and time to this site, where they are automatically scanned by Turnitin. Also, please bring a hard copy to class on the due date; they are due at the beginning of class.
Catalog Description
This course surveys methods used to investigate questions and acquire knowledge in counseling and psychology, and provides students with the skills necessary to critically evaluate counseling and clinical research literature. Topics include principles and techniques of qualitative and quantitative models, needs assessment, program evaluation, ethical and multicultural considerations in the research process and the role of the mental health counselor as a knowledgeable research consumer.
Course Purpose
The purpose of this course is to prepare future counselors to be informed consumers, rather than producers, of research. It provides a foundation for understanding research findings that are reported in professional outlets, evaluating the evidence base for counseling practices, and evaluating the effectiveness of one's own counseling practice. It may also provide the foundation for more advanced study in statistics and research methods.
Student Learning Outcomes
At the conclusion of this course, students will be able to:
1. Locate research in counseling literature databases and other resources to guide professional practice.
2. Describe basic psychology research methods, statistical analyses, needs assessment and program evaluation methods that are applicable to professional counseling.
3. Explain the basic components of and current controversies surrounding evidence-based practices in counseling.
4. Evaluate the applicability of research methods such as qualitative, quantitative, single-case designs, process research, and outcome research to questions encountered in counseling practice.
5. Critically evaluate research studies for soundness and applicability to professional practice.
6. Employ principles, models, and applications of needs assessment and program evaluation to effect program modifications
7. Design research to improve one’s own counseling effectiveness.
8. Discuss factors involved in culturally competent research.
9. Discuss ethical and legal considerations involved in the research process.
Required Readings
American Counseling Association. (2014). ACA Code of ethics. Alexandria, VA: Author. (May be downloaded from http://www.counseling.org/Resources/aca-code-of-ethics.pdf.
Reichenberg, L.W., & Seligman, L. (2016). Selecting effective treatments: A comprehensive, systemic guide to treating mental disorders. Hoboken, N.J.: John Wiley & Sons. (Selected sections; on reserve in
Langsdale Library)
Rubin, A. (2013). Statistics for evidence-based practice (3rd ed.) Belmont, CA: Brooks/Cole.
ISBN: 978-0-8400-2914-0
Other readings provided by instructor in Sakai "e-reserves Langsdale” or Files Directory
Course Method
As a graduate-level course, students are expected to develop and demonstrate skills in independent learning, including library and electronic research, critical thinking about assigned reading, and professional-level writing skills. Students are expected to spend 2-3 hours of independent preparation for every hour of class time in graduate courses. Thus, students can expect to devote 10-12 hours per week to this and other 3-credit graduate courses.
Classes will be conducted in lecture/discussion formats. Some information will be presented in didactic/lecture format, but students are also expected to come to each class prepared to discuss and apply the concepts from assigned readings and homework assignments.
Course Requirements
Specific Requirements / A / A- / B+ / B / B- / C+ / C / C-Research Journal / 30
Paper 1 / 15
Paper 2 / 15
Paper 3 / 30
Paper 4 / 50
Exam 1 / 100
Exam 2 / 100
Attendance/Participation / 30
Homework / 60
Extra Credit
Optional Take-Home Final
Total / 430 / 400 / 387 / 374 / 357 / 344 / 331 / 314 / 301
Total Percent / 100 / 93 / 90 / 87 / 83 / 80 / 77 / 73 / 70
Requirements Explained
Research Journal. Your research journal is the record of your thinking about a simple question, beginning with “I wonder if ______is related to ______.” Each week, you will think about how one major concept encountered in the Rubin text applies, or doesn’t apply, to your question. Your journal will be handed in at midterm and at the end of the semester. More detail will given in class.
Paper 1. This short paper will consist of your reactions to the first two assigned readings in the course, one by Eysenck and one by Lambert. It should consist entirely of your reactions upon reading them. That is, do not do any further research or reading on either research design or statistics for this paper. It should be no more than two pages in length. You are not expected to any show particular sophistication in your answers. Especially for the last two questions below, just write your initial reactions.
Based on your reading of Eysenck, 1952, address these four questions:
· Among the studies that Eysenck cites, is there an independent variable somewhere? A dependent variable? What might they be?
· What is one type of statistic that is used in at least one of the studies cited?
· What seems sound and/or impressive and/or important about this research? Describe briefly.
· Would you be inclined to use the major findings of this study to inform your counseling practice? Why or why not?
Next, answer all of the same questions in relation to the Lambert, 1992 chapter
Paper 2. Identify a clinical question that is of interest to you (e.g., “I wonder what evidence exists for treatment of autism?”). Spend some time exploring two (more, if you get curious!) of the “registries” listed on page 18 of your CNO reading (CNO, Ch. 1) and see if you can find information about evidence-based treatment for your question of interest. Do a very brief write-up of the type of information available on that website, what you found relative to your question, and be prepared to SHOW and TELL what you learned about this website to the class. No more than 3 pages.
Paper 3. Continue exploring the registries from p. 18, CNO. Use the Cochrane Collaboration (www.Cochrane.org) , Campbell Collaboration (www.campbellcollaboration.org) or one of the others to look for a review of studies that addresses your question. Alternately, let yourself get interested in a review on a related topic that you find. Describe the research question (including independent and dependent variables) that the review addresses, the population(s) that were studied, the type of studies reviewed (qualitative, quantitative, RCT, quasi-experimental, single-subject, etc) and type/s of statistical analyses that were used (from Rubin, Ch. 2). Describe whether and how this review might be useful to you in your clinical work. Finally, consult Reichenberg and Seligman (on reserve in Langsdale) to see what other research-based approaches are suggested for the type of problem that you are investigating. Summarize your findings in a 4-5 page paper.
Paper 4. Using the techniques presented in Class 5, searchout and read ONE complete published counseling/psychotherapy outcome study that is relevant toan area of clinical interest. Write a 5-6 page paper that addresses each of the following:
· a statement of one ofthe research questions that the authors addressed. For that question, what was/were the independent (predictor) variable/s and what is one dependent (criterion) variable?
· a description of how the dependent variable cited above was measured, and whether the measure used a nominal, ordinal, interval or ratio scale
· the hypothesis
· a description of the methodology used in the study (brief summary of the procedures)
· whether the study used an experimental design (was a true experiment), or quasi-experimental, and the implications of this.
· the type(s) of statistical analysis that they used to test the research question
· whether the assumptions and conditions for appropriate use of that type of analysis appear to have been met. What may have been a way to improve the design or analysis?
· whether the study used correlation or regression procedures and implications of this (and the design) for inferring causality
· a brief statement of their findings relative to the author’s research question and how relevant the findings are to your future practice.
· whether there is any mention of cultural/diversity factors that might have influenced this research, and if so, how they were addressed. If not, what cultural/diversity factors would have been good to address?
· Finish your report by giving this study a rating from 1 (flawed) to 10 (strong) on
a. how methodologically sound and
b. clinically significant it appears to be.
· Include the reference for this article in APA style. Attach a copy of the research article to your paper.
Homework is due each week starting on 2.7 on the “Review Questions and Exercises” at the end of each chapter in Rubin. Bring handwritten homework to turn in to each class. Homework must include all calculations or reasoning that you used to arrive at the correct answer (generally given on pp 319-327 of the text). Late homework will not be accepted! Illegible homework will receive zero points. Other “homework” is related to projects that you should be ready to discuss in class.
Exams. Exams 1 and 2 will cover materialpresented and studiedin the preceding classes. These will consist of multiple choice, true/false, matching, and short answer questions.The optional, extra credit, final exam is a separate take-home exam.
Missed exams can be made up only for emergency situations. You must notify me, by phone or e-mail BEFORE the exam, and you must supply documentation (doctor’s note, tow truck receipt, etc.) supporting your emergency’s status. An unexcused missed exam will result in an exam grade of zero.
Attendance in weekly class meetings is required. You will sign in by indicating your arrival time for every class. Also, if you should leave before class ends, you must sign out. If you have not signed in, it is assumed you did not attend the class (!) Your A&P grade will be reduced by 1 point for each late arrival beyond 15 minutes, or anytime you leave class early, by 15 minutes or more. Your A&P grade will be reduced by 3 points for each missed class, unless you have a documented emergency or illness. If you come in late and miss the signup sheet, it is your responsibility to come up after class and fill in your time of arrival. Same with early departures.
Your participation grade will be based on the quality of your interactions with your classmates in meaningful discussions, and the enthusiasm (!) with which you share and discuss your ideas and your homework.
Course Outline
Class
/Date
/ Topic and Activity /Assigned
Reading & Projects
/Homework -- from Rubin or not
1 / 1.31 / Introduction to the courseResearch Journal
Research AND Evaluation
Research in Counseling and
Psychotherapy: Retrospective / Formulate a research question
2 / 2.7 / Paper 1 Due (by Sakai and hard copy)
Evidence-Based Practices
/ Eysenck, 1952 (S)
Lambert, 1992 (S)
Rubin, p. xiii,
Ch 1-2
H,W,&K, (S) Ch 1, pp. 2-6; Ch. 2, pp. 46-49 / Prepare to discuss your paper!
Ch. 1: 1 & 3
Prepare to lead! and to discuss.
3 / 2.14 / Evidence-based practices, continued
Measurement and Design in Research and Evaluation
Reviews of Research: Campbell Collaboration, etc.
Descriptive Statistics : Frequency
Distributions / CNO, Ch. 1, pp 15-27
P&N, pp. 531-532(S)
Rubin, Ch 3
Rubin, Ch. 4 / Prepare to lead and discuss!
Ch. 3: 1, 2
Ch. 4: by last name:
A-B: 1a
D-H: 1b
K-V: 1c&d
All: 2, 5
4 / 2.21 / Paper 2 due
Descriptive Statistics – Continued
Charts and Graphs
Descriptive Statistics, Continued
Central Tendency
Ethics in Research & Evaluation / Rubin, Ch. 5
Rubin, Ch. 6
ACA, Section G / Ch. 5
A-B, 1a
D-H, 1b
K-V, 1c & d
All: 2,3,4
Ch. 6: 1a, 1d, 2
5 / 2.28 / Measures of Dispersion
Types of Distributions
Tentative: Researching Databases in Psychology: Lucy Holman, Langsdale Library, Guest Presenter / Rubin, Ch. 7
Rubin, Ch.8 / Ch. 7: 1-4
Ch 8: 1,2,3
6 / 3.7 / Paper 3 due
Descriptive Statistics, Continued Distributions – Shapes, z’s, Percentiles & Effect Size / Rubin, Ch. 9 / Ch 9: 1,2,3,
A-B: 4 – Article1
D-H: 4 – Article 2
K-V: 5
7 / 3.14 / Introduction to Needs Assessment and Program Evaluation / B,B&R Ch. 1-5
3.21 / Spring Break – No class!!
8 / 3.28 / Exam I
Inferential Statistics I – Probability, Sampling Distributions, Hypothesis Testing and Significance / Rubin, Ch. 10-11 / Ch 10: 1-4
Ch. 11: 1-3
9 / 4.4 / Inferential Statistics II – Types of Errors, Strength and Importance of Relationships / Rubin, Ch. 12-13 / Ch 12: 1-5
Ex Cr: 7 (4)
Ch 13: 1-7
10 / 4.11 / Inferential Statistics III—Data Analysis
t-tests and ANOVA / Rubin, Ch. 14-15 / Ch 14: 1-4
Ex. Cr: 5 (3)
Ch 15: 1-4
Ex. Cr: 5 or 6 (3)
11 / 4.18 / Inferential Statistics III – Data Analysis
Chi Square
Critical Analysis / Rubin, Ch. 16
Even & Robinson / Ch 16: 1-9
Ex. Cr: Written critique of Even & Robinson (1-10 points)
12 / 4.25 / Inferential Statistics III – Data Analysis -- Correlation / Rubin, Ch. 17 / Ch 17: 1-6
13 / 5.2 / Inferential Statistics III – Data Analysis -- Regression / Rubin, Ch. 18 / Ch 18: 1-6
14 / 5.9 / Paper 4 Due
Single System Evaluation
Wrap up! / Rubin, Ch. 19 / Ch 19: 1-4
Ex Cr Either 5 or 6 (4 points)
5.16 / Exam 2 and Final Exam Due
(S) – Reading is posted in Files Director in Sakai; Others in e-reserves: H,W&K = Heppner, Wampold, & Kivlighan, Research Design in Counseling; CNO = Cormier, Nurius, & Osborn, Interviewing and Change Strategies for Helpers;B,B&R = Bond, S. L., S. E. Boyd & K. A. Rapp. "Taking Stock"
The instructor reserves the right to modify this schedule and syllabus at any time during the semester.
UB Policies and Procedures
Academic Integrity
The Academic Integrity Policy for the University of Baltimore provides information regarding plagiarism. Incorrect use of other individuals’ work will likely result in plagiarism charges, which can lead to a failing grade on an assignment, a failing grade in the course, or even suspension from UB. Not understanding the definition of plagiarism or improper attribution are not excuses for failure to abide by originality requirements in this or any other course.