Adler School of Professional Psychology

Adler School of Professional Psychology

Doctoral Written Qualifying Examination Handbook

Carol L. Oster, Psy.D.

July 2002

Disclaimer

This Handbook grew out of the Preparatory Course for the Written Qualifying Examination. The suggestions it contains result from my experience with that course since its inception, my experience as chair of the Academic Standing Committee, and suggestions and recommendations from students. While people differ in their approaches to studying and learning, commonalities do exist in successful and unsuccessful approaches to studying. Use this Handbook as a springboard to develop your own studying strategy. Since this is essentially my extended syllabus for the Preparatory Course for the Written Qualifying Exam, this Handbook represents my own opinions and may not be taken as a statement of School policy. For official School policy, see the School Catalog and official memos.

Introduction

Qualifying examinations are common in higher education in the United States, and are almost unanimously utilized in graduate study. Every Psy.D. program in the U.S. requires students to pass one or more qualifying examinations. Most Ph.D. programs do as well. Exams developed by the various programs differ somewhat in format and in placement in the timeline of the doctoral program. Formats include various forms of “objective” tests, essay tests, skill demonstrations, and oral exams, and various combinations of these. They may be administered partway through the program, at its end, or at intervals along the way.

Regardless of methodology, competency exams share common purposes. Qualifying examinations are particularly critical in professional programs, regardless of the profession. One purpose is to assure the program’s various stakeholders that its graduates are actually competent in the concentration area of the program. Qualifying exams protect the public from incompetent practitioners, assure students that they and their classmates meet some standard of performance so that their degree “means something”, and assure the faculty that the students they graduate are indeed competent. Perhaps the most critical of these functions is protection of the public.

Another purpose of the exams is to provide feedback to the faculty and the program about the success of their efforts to teach the profession’s knowledge, skills, and values. This feedback is also used to inform accrediting agencies about the success of the program, assuring that the requirements for entry into the profession are both rigorous and fair.

From a pedagogical perspective, qualifying examinations “force” students to integrate material necessarily presented in disjointed fashion in chunks called courses. Essay and clinical examinations particularly require both integration and application of knowledge, skills, and values. Review of what has been learned with the demand to demonstrate higher-level mastery itself increases mastery, often doubling or tripling prior gains. The same synergistic boost in knowledge and competence is typically seen in the preparation for licensing examinations.

In the sequence of tasks leading to the doctorate in clinical psychology, successful completion of qualifying examinations assures internship sites that its trainees have achieved some minimum level of knowledge, skill, and values that make them “safe” to take on as advanced student clinicians. It also assures the internship that the student is capable of seeing a project through to completion.

At Adler, the doctoral program requires two qualifying examinations: the Written Qualifying Examination (WQE) and the Clinical Qualifying Examination (CQE). The WQE is based on the first 5 terms of coursework in the regular (weekday) sequence of courses in the doctoral program. The exam is placed early enough in the sequence of courses that students are not overwhelmed with the amount of material to study in preparation, and so that students who are not able to succeed have not invested 100% of tuition and time in the program before they discover they are not able to complete it.

In successfully completing the WQE, students demonstrate that they have learned the skills, knowledge, and attitudes presented in the first, most basic, part of the doctoral curriculum, are able to apply that learning integratively, and can explain what they have learned in writing.

There are two categories of scores given on the WQE. The first is the content score, and the second is the writing score. An essential job function of the clinical psychologist is the constant demand for coherent writing in the public or semi-public eye. Thus, one of the skills students must master to be an effective clinical psychologist is the ability to effectively present their ideas, observations, and plans in writing. The grading criteria in the “writing” category make it clear that the writing score is an assessment of the student’s ability to present a coherent, organized, targeted discussion of an issue.

The second qualifying exam in the doctoral program, the CQE, is designed to follow the WQE by about one year. It provides students with the opportunity to demonstrate advanced knowledge, skills, and attitudes following a year of clinical training, and before the internship. A separate Handbook for that exam exists, and currently can be located online at www.osterpro.homestead.com.

Development of Exam Questions

Exam questions are derived only from coursework covered by the examination. (See WQE Memo.) Members of the core doctoral faculty sometimes develop exam questions in question-writing workshops, and are free to submit potential questions at any time. From the pool of created questions, two questions from each of 5 domains are selected for each administration of the exam. The Deans’ Council reviews proposed questions for each exam administration for appropriateness.

Selection of questions for any particular exam administration is guided by a number of principles. First, variety within and between domains is sought, to achieve breadth of curriculum coverage and to provide students with choice within each domain. An exam that contains too many references to one topic across domains, for example, would both be too narrow to assess competency in the curriculum covered, and disadvantage any student with that particular area of weakness.

Second, questions are reviewed for consistency with the curriculum and with the level of competence expected. Questions that, in the opinion of the deans or other faculty, require advanced mastery of the content areas covered (as taught in our curriculum), are not covered in the courses required for the exam, or are too controversial within the field to allow a clear response are often rejected, although discussing a controversy within the field may itself be the question! For example, questions about psychophysiology would be rejected because that is not covered in the prerequisite courses. Questions about diagnosing adult ADHD from intelligence test results have been rejected as not covered, requiring advanced competency, and being too controversial in the field. However, questions addressing the debate about whether alcoholism or conduct disorders are mental disorders or criminal behavior patterns have been allowed.

Third, questions are reviewed for clarity and for reasonableness. In the judgment of the faculty, given the curriculum and how the topic is covered in the prerequisite courses, can the question be answered within 40 minutes to the level of competency the faculty looks for? Is it too complex, containing too many “parts” to be covered reasonably well within the time limit? Is there sufficient consensus on what a good answer would contain that we “know what we’re looking for”? Is the question phrased clearly enough to provide reasonable direction for the student?

Finally, questions have to vary somewhat across administrations. This is particularly true if there are students making a second or third attempt at the exam, as we do not have students face the same questions/demons twice. But it is also important so that faculty receive varied feedback about the efficacy of the curriculum.

Grading of the Exam

Both students and faculty are protected by a double-blind grading procedure. Unless students make it known, faculty other than Dr. Gralewski are not aware of who is taking the WQE for a particular exam administration. Students select codes to put on their exam booklets, rather than names. After students complete the exam, their handwritten answers are photocopied and then transcribed by individuals hired specifically to do so. The faculty never sees the handwritten answers.

Once all tests have been transcribed, two graders are assigned from among the doctoral core faculty. The assignment is on a rotating basis, except that no student and grader are paired twice. That is, if a student is making a second or later attempt at the exam, new graders are assigned. This prevents a second level of “double jeopardy,” as does the fact that a student will not face the same question twice regardless of how many times they attempt the exam.

A pair of graders is assigned one to four exams to score per exam administration. Each grader reads and grades the exams independently. Then the two graders get together to discuss the grading of the exams and to resolve any differences in grades assigned. When they have agreed upon the grade for each exam, the results are submitted to Dr. Gralewski. In the extremely rare instance when the two graders cannot agree on the pass-fail decision, a third grader, unaware of the impasse, is asked to grade not only the exam in question, but also at least one other exam. “Impasse” graders are not aware that they are serving in this capacity.

Inter-grader reliability has always been high. Current inter-grader reliability is better than .9. In the past 3 years, 81% of all attempts at the exam were successful. (The percentage of all persons passing the exam is higher, as several of these were second or later attempts at the exam.) 83%-87% of content scores were 10 or better (out of 15 possible content points); 82%-87% of writing scores were 4 or better (out of 5 possible content points). The mean of all failing scores was 60%, well below passing (75%).

Scoring Criteria for the exam can be found on the official scoring guideline sheet, found here. There are two parts to the score for each answer. The content score, which makes up ¾ of the possible score for each answer, is based on the adequacy and accuracy of the answer, in terms of correctness, breadth, depth, and pertinence. The writing score, which makes up ¼ of the possible points in a score, is based on the logic with which the answer is written – whether the answer is coherent (“hangs together” well) and well-developed (organized, logical flow of the “argument”).

General Studying Tips

When to Take the Exam

The doctoral program is set out in a specific sequence that introduces basic concepts, skills, and values in the first year or so, and proceeds to advanced knowledge and skills in later terms. The WQE is designed to assess mastery of the first part of the curriculum. Students who follow the sequence of courses as designed, and who take the exam “on time” rather than delaying, take the exam in or at the end of their second year, or after about 55 credits. The courses required for the exam are as follows, by domain. You must have completed every prerequisite course. No exceptions are made.

Assessment: 511, 512, 627, 628, 629

Psychopathology: 644, 645, 646

Intervention: 471, 505, 521

Adlerian Theory & Methods: 401, 515, 522, 535

Foundations: 498, 521, 625, 626

Follow the principles of learning theory to prepare. For example, we know that distributed practice is superior to mass practice in mastering material. That means that practicing consistently over a longer period will result in better mastery than will “cramming” for the exam, and than will brief spurts of studying fit into your schedule on an intermittent basis. (Use this same method for preparing for the licensing exam, too!)

Begin studying for the exam before these courses are completed (study as you go). Most students study specifically for the exam for 3 to 6 months before they plan to take it. Depending on your self-evaluation, you may decide to study longer or shorter than that.

Authors of most study guides (such as the licensing exam study guides) suggest that you plan to be done studying 2-3 days before an exam of this type. This gives you time for last minute brushing up, and allows you time to relax enough to master your anxiety. For example, you could exercise, visit with friends, or see a movie. A good comedy can relieve tension built up by the press of studying. Including some “down time” during the couple of days before the exam allows your final studying sessions time to “incubate” and integrate.

Some students report they “carb up” (eat pasta) the night before the exam, as if they are preparing for a marathon. It’s important to get enough rest. Get to the exam early so you don’t feel rushed. Students arriving from out of town report that getting into town the night before the exam relieves anxiety over possible unforeseen travel problems. Some students report that coming into town a couple days before the exam and strolling Michigan Avenue in the evenings is very relaxing.

Independent and Group Study

Students who do not pass the exam on at least the first 2 attempts often report that they studied alone, or that they began studying with a group, but gave up after only a few meetings. They also report that anxiety is a significant factor in impairing their performance. It would seem that studying alone is hazardous to your mental health and academic success! Not only does Adler allow studying in groups, we strongly encourage it!