Summary – Institutional Effectiveness Report
MSAP – Psychology Program 2004-2005
Submitted by Sam Broughton and John R. Hester
Mission and Goals
Francis Marion University is responsive to the needs of the region by offering the Master of Science in Applied Psychology (MSAP) and proposing program modifications in this professional degree as indicated. Graduates of the MSAP program will develop the knowledge and skills necessary to work as professionals in clinical, school, health, and other community settings as scientist practitioners. The MSAP program adheres to the standards of training of the Council of Applied Master’s Programs in Psychology (CAMPP), is accredited by the Masters in Psychology Accreditation Council (MPAC), and is approved as a specialist-level training program by the National Association of School Psychologists (NASP). Students and graduates of the MSAP program bring scholarship and reflection to their work, and an understanding of diversity in clientele, methodology, and application. MSAP faculty produces scholarship that enhances teaching, involves students, and contributes to the profession of psychology. MSAP faculty consults with and renders academic and practical assistance to local human service agencies, hospitals, and regional schools.
Assessment Activities
The program monitors admission and graduation rates and quantitative characteristics of applicants and matriculated students.
Consistent with our mission and goals, best practices for training master’s level psychologists are monitored from NASP/NCATE for School Psychology, CAMPP and MPAC for clinical/counseling psychology. The evolving standards for licensure of Professional Counselors and Psycho-Educational Specialists by the SC Board of Examiners for Licensure of Professional Counselors, Marriage and Family Therapists, and Psycho-Educational Specialists are monitored, so that graduates will meet didactic training requirements for the appropriate license upon graduation.
Practica are associated with specified applied courses. The number of required practica varies with option. Practica require a specified number of clock hours of practice in association with classroom learning. Students are required to maintain logs of activities and to submit a portfolio of work samples at the end of each practicum.
The performance of all graduates is evaluated during and at the conclusion of their required internships by field supervisors. Interns also provide work logs and samples/portfolios (e.g.: assessment reports, treatment/intervention plans and outcome data, counseling/consultation notes and outcome data, project summaries) that are evaluated by the faculty formatively as part of the internship seminar and summative at the conclusion of internship.
School Psychology graduates complete the Praxis Exam required for certification as a School Psychologist II and licensure as a Psycho-Educational Specialist.
The level of faculty scholarship, community service, and student involvement in faculty research is obtained from activities reported in the annual faculty reports.
The number of school psychology students entering and graduating from the option since its initial NASP/NCATE approval/accreditation in 2001 reached an adequate number during the 2004-2005 academic year to allow an initial validity study of entry criteria and of the internship supervisor rating form.
This year an Alumni Survey was completed for the overall graduate program and a Clinical/Counseling Advisory Committee was organized and a meeting held. A planned employer survey was not conducted.
Assessment Results
During the 2004-2005 academic year (Fall and Spring) 15 newly accepted students enrolled in the program (6 in clinical/counseling and 9 in school psychology). This compares to 18 students that entered the program the previous academic year. Average GRE scores were 501 (GRE-Verbal) and 569 (GRE-Quantitative). The average Verbal score increased 8 points and Quantitative increased 25 over 2003-2004. In a two year span (2002-2003 to 2004-2005) average GRE scores for entering students have increased 81 on Verbal and 49 on Quantitative. This year’s average overall undergraduate GPA was 3.5 and Psychology GPA was 3.3. Such is very similar to last year. Overall, headcount enrollment in the program is 52. Ten students graduated from the degree program during 2004-2005 (7 clinical/counseling and 3 school). This compares to a headcount of 51 and 11 graduates last year. The number of clinical/counseling graduates increased by four and the number of school graduates decreased by five. These changes are thought to be random rather than representing any trends.
Examination of practicum logs and work samples/portfolios showed that all students met or exceeded minimum requirements for acceptable performance in course-related practice settings.
To assess our goal of developing professionals with skills necessary to work as professionals the Department assesses the internship experience. End-of-Internship ratings of school psychology interns by field-based supervisors for 3 interns (all degree seeking students) who completed their one academic year internship in Spring 2005 produced a mean composite rating of 5 on a 5 point scale, with a rating of 5 representing competence at the level of unsupervised practice and 4 representing a requirement of minimal supervision. All three interns received composite ratings of 5. Average Internship Ratings averaged 4.97. Average ratings may represent a more valid measure of internship performance as will be discussed below. One hundred percent of the work samples/portfolio materials submitted for summative evaluation at the end of the internship seminar were rated as satisfactory or higher by the faculty for interns in the school psychology option. Community supervisor rating forms for the 7 clinical/counseling interns who completed internships in 2004-2005 were evaluated and produced a mean overall (composite) rating of 4.9 on a 5 point scale. Six interns received a rating of 5 and one received a 4. As with the school interns, a rating of 5 represents competence at the level of unsupervised practice and 4 represents a requirement of minimal supervision. The average of 4.9 for clinical/counseling interns is up from last year’s 3.8. Concern in last year’s report that the 3.8 might be a downward trend may be unfounded. However, this indictor needs to be monitored carefully in the clinical/counseling program. Generally written comments by supervisors for both school psychology and clinical/counseling interns were positive.
Scores on the Praxis Examination necessary for certification and licensure in school psychology had been received for all 3 students completing internship in the School Psychology Option at the time of this report. The mean score for these 3 completers was 696.67, with individual scores ranging from 660 to 730. A score of 660 is required in South Carolina for certification and licensure. A score of 620 is required for certification in North Carolina. When evaluated against these cut-scores, all 3 graduates met or exceeded the criterion for their state of anticipated employment. All graduates from previous classes who had not achieved passing scores at the time of their graduation achieved passing scores during their first year of employment.
Anecdotal reports received from members of the School Psychology Advisory Committee who participate in and monitor practicum and internship sites continue to indicate satisfaction with the performance of practicum and internship students, the practicum/internship program, and with their district’s relationship with the School Psychology Option. All reporting sites indicated a desire to continue participation for the coming year. At the meeting of the School Psychology Advisory Committee during Spring Semester 2004, support was expressed for changing the degree offered to the specialist rather than the master’s degree. As a follow-up to this support, Drs. Broughton and Bridger visited the School Psychology Specialist program at Winthrop University during fall 2005, and Dr. Broughton visited the Certificate of Advanced Study Program at Appalachian State University during spring 2005 to gather additional information regarding the organization and structure of dual degree specialist and advanced certificate programs. Findings will be incorporated into the Specialist Degree proposal as it moves forward.
The validity study of selected quantitative School Psychology Option entry criteria found the following significant relationships: GRE-V with Praxis scores, .77; GRE-V with final GPA, .51; GRE-Total with Praxis scores, .67; GRE-Total with final GPA, .43; Undergraduate GPA with Praxis scores, .41; Undergraduate GPA with final GPA, .42; Admission Index with Praxis scores, .58; and Admission Index with final GPA, .47. The number of students involved in the majority of these comparisons was 32 (data for all students on some variables was not available at the time of the analysis). Significance levels were set at probabilities of .05 or less. Nonsignificant correlations are not reported. It was concluded that these relationships indicate sufficient validity to merit continued use of current entry measures. Faculty members have expressed concern regarding interpretation of GRE standard scores. Consideration of incorporating percentile scores into entry criteria has been proposed.
Interviews, an oral exam, and a portfolio review for students completing first year course and practicum work in the School Psychology Option was added to the first year review, which previously included only a GPA and transcript review. Results are under study to determine the best uses to be made of this data.
Pre-Internship portfolio and transcript reviews, interviews and oral exams of 4 School Psychology Option students scheduled to begin internship during fall 2005 found all 4 students to have completed prerequisite work and to be ready for internship.
A validity study of the School Psychology Option’s internship supervisor rating form (Field-Based Experience Evaluation Form ) also was undertaken involving the same program completers as for the entry criteria. The following significant relationships were discovered: Average Internship Rating with final GPA, .50; Final GPA with Praxis Scores, .50; Average Internship Rating with Composite Internship Rating, .87. The correlation of the Average Internship Rating with Praxis score (.26) was not significant. These correlations indicate that internship supervisors appear to be rating interns in a valid manner and consistently with other important program indicators. Thus, current evaluation practices will be maintained at this time.
Exit interviews with the 2005 graduates of the School Psychology Option indicated a potential need to review course and program content regarding psychopathology, emotional disability, behavior disorders, and distinguishing social maladjustment from emotional disability, especially as involved during manifestation hearings. A review of course content will be undertaken. On the Internship Experience Evaluation Form, interns gave their overall experience a mean rating of 4.58, where a rating of 5 represents “extremely helpful or more than adequate” and 4 represents “very helpful or very adequate.” The lowest ratings were uniformly obtained across all interns on an item involving the adequacy of the internship contract between the district, university, and intern (average rating = 3.33), where 3 represents “helpful or adequate.” Comments indicated that the syllabus for the internship was more helpful. As a result, the contract form will be reviewed for improvements during the coming year. Consistent with the validity study of the Field-Based Experience Evaluation Form, interns gave the feedback produced by supervisor ratings a mean rating of 4.67, indicating social validity of the supervisor rating form and process.
Faculty scholarship, professional activities and community involvement continued as reflected in the annual reports of individual faculty members. (See undergraduate report)
During the 2004-2005 academic year a survey of MASP alumni was conducted. Surveys were sent to 233 graduates and 63 (27%) completed surveys were returned. Briefly, to summarize the results, graduates rated the quality of the program as favorable. On a five-point scale (4 above average, 5 well above average) the overall rating of the program was at a mean of 4.14. Ratings of interest would include: Professor’s interest in students 4.2, Relevance of courses to interest and needs 4.2, Academic Excellence 4.2, Level of Instruction 4.2, Training in assessment skills, 4.3, Ethical/Professional Skills 4.1. Among the lower rated areas were financial assistance, 3.3, and Computer availability, 3.5. Generally, school psychology graduates were commanding significantly higher salaries than clinical/counseling students.
To complement the School Advisory Committee, a Clinical/Counseling Advisory Committee was organized and a meeting held on September 15, 2004 to discuss current directions of the clinical/counseling program and seek input. Thirteen members of the committee attended as well as eight faculty members. Eleven of the committee members are from public human service agencies that assist in the training of our students and employ our graduates. There was one private practitioner and the director of the FMU Counseling and Testing Center. The Committee was very positive about the quality of training of FMU MSAP students. They did offer several suggestions to include: continue training in diagnosis and treatment of substance abuse disorders, continue to work on professional writing skills, emphasis on cognitive-behavioral therapies, students need to develop proficiency in speaking Spanish, and the program needs to continue to develop avenues to financially assist graduate students.
Program Improvements
As a result of last year’s assessment several improvements were made.
To enhance recruitment efforts graduate brochures were revised and published.
Last year’s Institutional Effectiveness Report listed as a planning goal to add two scholarships available to graduate students. With the assistance of the Provost’s Office two scholarships were added this year. These are available to out-of-state and in-state students based on merit and are targeted at enhancing recruiting and retention efforts. It is too early to assess the efforts but the two incoming students on scholarship have an average GRE of 575 on Verbal a 615 for Quantitative. The GPAs of these two students averaged 3.6 in Psychology and 3.95 overall.
A seventh psychology graduate assistantship was developed by the Counseling and Testing Center. In general the department has intensified efforts to use graduate assistantships to recruit graduate students.
The planned early search for a clinical/counseling faculty member occurred with the highest rated candidate hired before the end of Fall semester. The hire ended a three-year search for a new clinical/counseling faculty member.
MSAP alumni were surveyed as planned and the results were analyzed and reviewed by the Department. The Department held two planning sessions during the year to review the survey results and the feedback from the Clinical/Counseling Advisory Committee meeting. Discussions centered on: means to better prepare students for internship, how to infuse substance abuse training into currently taught classes, and a proposal was developed to move the school psychology option towards a specialist degree.
In response to a favorable reaction by School Psychology Advisory Committee members for the development of a Specialist Degree Proposal, Drs. Broughton and Bridger developed a proposal for converting the 68 semester hour School Psychology Option of MSAP to a Master’s/Specialist combination program. The proposal was presented to the department faculty during the fall 2005 term, and discussed further in the spring. The proposal is to be further developed and advanced to the university level during the coming academic year.