Academic Year 2005-2006 Self-Study Report

University of SouthFlorida

College of Medicine

LCME Institutional Self-Study

Report of Committee Four:

Educational Program for the MD

Study Year 2005-2006

The contents of this report represent the judgments and opinions of the members of this Self-Study Committee. The committee has made every effort to ensure that the information represented is accurate. The LCME administrators have not audited the data in the text of the report and there may be some discrepancies within the database or executive summary as a result. While every attempt has been made to provide and evaluate information accurately and objectively, the committee acknowledges that any errors of fact in this report are unintentional.

(Printed October, 2006)

Table of Contents

I.Executive Summary/Key Issues

II.Responses to Questions in Guide toInstitutional Self-Study

III.Analysis of Recommendations and Deficiencies Relevant to Committee as Identified by Most Recent LCME Review in 1999

IV.Major Changes Since Last LCME Review in 1999

V.Areas of Strength

VI.Areas of Concern and Challenges

VII.Review of Compliance with EstablishedLCMEStandards

VIII.Recommendations, Possible Solutions and Strategies

IX.Issues of Concern Relevant to Other Committees

X.Attachments

XI.Narrative of Process

XII.Database Accuracy

XIII.Committee Membership

I. Executive Summary/Key Issues

Educational Objectives

  • The creation of the Educational Objectives for the College of Medicine involved all stakeholders in the college, including administration, faculty, and students. These deliberations resulted in new college-wide objectives and competencies for medical education, encapsulated by the mnemonic USF CARES. The members reviewed past records of others having undertaken similar tasks. Authoritative and credible documents such as the Medical School Objective Project of the AAMC, the IOM reports on quality in healthcare, CanMeds, the Brown objectives, and the ACGME competencies were studied and adopted for comprehensive educational objectives for undergraduate medical education.
  • USF CARES will provide an effective guide for on-going educational program planning and course evaluation. Each course has its own set of goals and objectives consistent with USF CARES. Student evaluation is now performed at all stages of the curriculum based on USF CARES. Also, individual course or clerkship objectives are examined on the basis of USF CARES. Future course development and program planning will be complemented.
  • There are sufficient patient resources and clinical settings for achieving the college’s clinical learning objectives. The college uses a number of clinical settings (academic and community/private practice settings; inpatient and outpatient/ambulatory settings; primary care/tertiary care settings; etc) and has many physicians and patients with whom students can work during the course of their training. Student clinical experience during their clerkships is evaluated by logbook data, including diagnoses and demographics.

Structure of the Educational Program

  • The educational program provides a general professional education that prepares students for all areas of post-graduate medical education. The College of Medicine has just completed a college-wide curriculum reform process for Years 3 and 4 which was preceded by the creation of a revised, integrated curriculum in Years 1 and 2. In order to ensure that the educational program provided for a general, professional education, consensus was built around identification of common diagnoses, symptoms, and syndromes determined by clerkship directors and clerkship-specific national organizations, national databases and logbook data from previous years. Based on our recent graduate-class distributions within the NRMP (National Resident Matching Program) medical subspecialties, the USFCOM educational program has been successful in preparing students for all medical careers.
  • USF College of Medicine graduates select a wide variety of specialties for residency education; the class of 2004 matched to 17 different specialties, the class of 2005 matched to 19 different specialties, and the class of 2006 matched to 22 different specialties.
  • The medical curriculum provides a balanced and innovative professional education that fosters the development of self-directed learning throughout all four years of the undergraduate medical education.
  • There is consistency of educational quality and of student evaluation when students learn at alternative sites within a course or clerkship. Each course and clerkship has a single course director who informs faculty about the course-specific objectives and grading policies for the course.
  • Faculty use a single evaluation system in each course and clerkship. The implementation of college-wide evaluation unites educational objectives and student evaluation in both formative and summative evaluations. Evaluation forms contain program-wide competency- and objective-achievement ratings as well as clerkship-specific ratings.
  • All content areas required for accreditation are addressed in the curriculum. While the first-and-second year curricula include named courses in anatomy, physiology, microbiology, pathology and pharmacology, the contemporary context of these disciplines is also included in the other required courses during the first two years. The last major revision to the curriculum occurred within the Year 3 and 4 programs with the inaugural iteration starting in June 2005. A principal feature of that revision includes an interdisciplinary/interdepartmental approach to the organization of the required clerkships. In addition to the basic and clinical disciplines, the curriculum throughout the four years includes required behavioral and socioeconomic subjects.
  • Data from the most recent AAMC Medical School Graduation Questionnaire for USF graduates (2005) reflect that USF graduates reported an “appropriate” amount of time was devoted to instruction in all areas surveyed except in nutrition and medical socio-economics issues. These are addressed later in this report.
  • USMLE results for first-time takers from the USF College of Medicine in the three most recently completed academic years demonstrate success rates of 95-99% with the mean scores consistently at or above the national mean for each portion of the examination.
  • During the third year, the amount of time spent in inpatient and ambulatory teaching is balanced with 50% of the clinical assignment in each setting.

Teaching and Evaluation

  • Supervision of the medical student’s clinical experience is the responsibility of core faculty members. With few exceptions, the majority of the faculty on each clerkship are core faculty. Voluntary clinical faculty and residents serve in supplemental roles. Voluntary faculty must have a clinical faculty appointment, and they provide unique learning experiences for the student, such as a rural health experience. All faculty are uniformly given copies of both college (USF CARES) and individual clinical clerkship-learning objectives and receive ongoing feedback from the clerkship director.
  • All faculty working with students on any given clerkship are expected to personally oversee all aspects of the student’s clerkship experience and meet with the student for both educational and feedback purposes. Direct observation of the student is encouraged whenever possible.
  • Residents are an integral part of the teaching team. All residents are given college educational objectives and must attend a program on how to teach and evaluate medical students during their annual orientation. The faculty are expected to monitor the role of residents in the students’ educational program. The core faculty and residents evaluate and suggest areas of improvement for students’ acquired skills and expertise.
  • Each course and clerkship uses multiple methods of evaluation. During the first two years of the curriculum, the more fact-based courses evaluate learning using primarily objective testing methods. Other methods of evaluation measure not only the student’s comprehension of the material, but also his or her ability to research, analyze, synthesize, and communicate information independently. These methods of evaluation include small-group problem-solving, participation in laboratory exercises, written assignments, and topic-based presentations. The third-and-fourth year clerkships evaluate the attainment of clinical skills and reasoning, problem solving, communication skills, and the development of appropriate behaviors and attitudes. This is accomplished by the use of nationally standardized and departmental written examinations, and by faculty assessments of students’ skills, behaviors, and attitudes. The Committee on Curriculum oversees the evaluation methods.
  • A comprehensive system is to assure that core clinical skills defined in the college’s educational objectives, USF CARES, and clerkships have defined core clinical experiences documented by the logbook. These objectives form the basis of the uniform assessment tool used by all faculty in assessing students. There are several formal evaluations of the students’ clinical skills. At the end of the second year, an OSCE is performed for all students. During third year, several of the clinical clerkships require a clinical practice exam (CPX). During the fourth year, the students are videotaped performing a complete history and physical exam.
  • The faculty of the college strongly believe that their graduates have a sound background in problem-solving, clinical-reasoning, and communication skills. The Clerkship Oversight Committee continually evaluates each clerkship’s success and providessufficient clinical experiences for students’ acquisition of required skills.

Curriculum Management

  • The overall curriculum is reviewed during the annual or semi-annual curriculum retreats. These retreats provide an update of national trends and innovations with a detailed review of the college curriculum evaluations, normative test scores, student responses to the AAMC Graduation Questionnaire, all college performance-based examinations, and any other information relevant to curricular assessment. Problem areas are discussed, and corrective plans are formulated. Particular attention is given to content, coordination, and outcome measures.
  • The Committee on Curriculum conducts a formal, highly detailed review of each component of the curriculum every two years or more frequently when problems are detected. This process includes the review of all outcome measures, course materials, student assessment of overall course and faculty effectiveness, course directors’ responses to previous reviews, previous committee recommendations, and actions taken to address any problems. The USMLE sub-score analyses discuss adequacy of the course content as does the performance-based assessments of clinical skills including OSCE and CPX.
  • The Vice Dean for Educational Affairs has had sufficient resources and authority to assure that the educational program can meet its objectives; however, during the self-study process it became apparent that in two areas the process for acquisition of resources has not been optimal. The scheduling of conference rooms for small-group sessions has been difficultbecause of limited centralized control of these resources. The dean has recently directed that the Office of Educational Affairs should assume control of these educational resources and that scheduling be centralized. The second area of concern has been the process associated with staffing the faculty required for the interdisciplinary courses. Since these courses are not the domain of any single department, the departmental chairpersons have not assigned faculty to these courses. This problem will be solved with the implementation of the AIMS program. Under this program, all faculty will be required to devote a certain percentage of effort to stipulated types of educational activities as part of their overall compensation package.
  • Curriculum planning has been one of the particular strengths of the College of Medicine. All four years of the curriculum have been modified over the past seven years. Each major modification arose from discussions that occurred at a Committee on Curriculum Retreat and involved all major stakeholders in a collegial process of determining objectives and planning for their implementation and evaluation.
  • The Committee on Curriculum has limited the number of contact or work hours required of a student per week. For Years 1 and 2, the limit is 28 hours per week of contact plus four hours per week for the Longitudinal Clinical Experience. During Years 3 and 4, the same work-hour limitations that are in force for the residents apply for the students (80 hours per week). As a result of the self-study process, the Committee on Curriculum has determined that the first year of the curriculum has substantially more contact hours than the national average (1132 vs. 830). The departmental chairs, course directors, and faculty involved in the Year 1 curriculum are working with the vice dean for education to reduce the number of contact hours to a level approaching the national average. In addition to reducing the number of contact hours, the proposal will encourage more independent learning experiences and less lecture format.

Evaluation of Program Effectiveness

  • Measures demonstrating that the institutional objectives are being achieved by our students include indicators of effectiveness from both external and internal data. External metrics include results of USMLE and NBME subject examinations, student responses on AAMC Medical School Graduation Questionnaire, NRMP results, specialty choice of graduates, and program director surveys of PGY-1 performance. Internal metrics include student scores on internally developed written examinations, performance-based assessment of clinical skills including OSCE and CPX exams, review of logbook data in Years 3 and 4, student advancement and graduation rates, and a videotaped history and physical for fourth year students. All college-wide evaluation forms link the educational objectives with course-specific competencies.
  • Information about our students and graduates is used to evaluate and improve the college’s educational program. The Committee on Curriculum reviews all of the external and internal data at its annual curricular retreat in conjunction with the senior administration. Areas needing improvement are noted, communicated to course/clerkship director, and re-evaluated the following year. Identified areas of concern are referred to the respective course/clerkships director for immediate attention.

II.Responses to Questions in Guide toInstitutional Self-Study

  1. Describe the level of understanding of the school-wide objectives for the educational program among administrators, faculty members, students, and others in the medical education community. Do these objectives serve as effective guides for educational program planning, and for student and program evaluation?

Committee Response: Administrators

Administration at the College of Medicine was intensively involved in the process of

developing the school-wide objectivesUSF CARES, and has a thorough understanding ofthese goals. Subsequent to the adoption of USF CARES, administrators and faculty members planned, developed, and evaluated all aspects of the undergraduate medical education program.

Faculty members

Members of the COM faculty havereceived copies of USF CARES by e-mail. Faculty based at the MoffittCancerCenterdid not receive email sent to the College of Medicine faculty distribution list (comfac), and may not have received a copy of USF CARES. This oversight has been corrected, and all COM faculty including those at the Moffitt server are now included in electronic mail sent to the comfac distribution list. The USF CARES document was recently re-sent to all faculty to assure that all faculty had received the school-wide objectives. All new faculty will be given a copy of the college’s mission statement and USF CARES as part of the new faculty orientation process.

Although HSC faculty members have received USF CARES, the level of understanding may still be less than thorough. Unless faculty members accept the concept of teaching according to the USF CARES competencies, USF CARES will not have the intended effect. Because all future student evaluations will be basedon USF CARES objectives, faculty members must understand these objectives.

Students

All COM students have received USF CARES by e-mail. Also, USF CARES is presented and discussed during orientation and at the beginning of all courses and clerkships. Each course and clerkship’s objectives are related to the school-wide objectives. Students will be evaluated on the basis of USF CARES during each course and clerkship and will continually gain understanding of the school-wide objectives throughout the four-year curriculum.

Educational program planning

The objectives were developed to serve as effective guides for educational program planning, and that process is ongoing at present. Each course has its own set of goals and objectives that must be integrated with USF CARES. Future course development and program planning will be more easily made with the USF CARES objectives in mind.