Mini Symposium on Peer Observation of Teaching
June 2, 2011
A Sample of Current Peer Observation
of Teaching Activities
Susan Frankl, M.D.
Peer Observation of Teaching Program: A Pilot Study of Clinical Teaching in Office-based Primary Care Practice Sites
HMS Course: Primary Care Clerkship (PCC)
Location: Various Clinical Offices of PCC Preceptors
Description of the Peer Observation of Teaching Activity/Program
Observers: PCC course directors
Observed: Faculty preceptor volunteers
Twenty experienced preceptors were invited to participate in this pilot program; 12 volunteers were observed for 90 minutes while teaching their 3rd year HMS PCC students in their office practices. Observers and preceptors were paired based on their schedules and availability.
Description of how the peer observer provides feedback to the “observee.”
Observers used a standardized observation checklist tool and indicated whether specific teaching behaviors were either observed or not observed. In addition, observers were instructed to provide detailed descriptive comments on each teaching behavior as well as feedback on the preceptor’s strengths and opportunities for improvement. Verbal feedback was also provided to the preceptor immediately following the observation. The completed checklist was then emailed to the preceptor with an invitation to continue the conversation if desired.
Description of training the peer observers received in how to conduct the observations.
Prior to performing the observation, observers participated in a 90-minute observer-training workshop that included practice with a videotaped case.
Description of how receptive the faculty have been to the observations
Faculty participation in the pilot program was voluntary. No reticence was encountered in the pilot group of these faculty volunteers. Pre- and post-observation surveys were sent to all participants by email.
Of those preceptors who have responded to the post-observation survey to date, 100% agreed with the following statements:
- I would be interested in being observed again in the future.
- The observer provided useful feedback about teaching strategies.
- Participating in this program will help me to improve my teaching in the PCC.
- Would you recommend the observation of teaching experience to a colleague? Yes.
Educational outcomes that have resulted from the program
5 out of 6 observers completed the post-observation survey. When asked to choose the top 3 outcomes from of a list of 15 potential results of conducting a peer observation, 100% of them chose “helped me reflect more on my own teaching.”
90-100% of preceptors were observed to:
- establish a positive learning climate
- convey information clearly
- explain their own clinical reasoning
- create opportunities for student autonomy
- demonstrate interviewing skills, PE skills and counseling/shared decision-making
However, only 27% of preceptors were noted to:
- observe their student obtain the patient history
- observe their student perform the physical exam
Future directions for the peer observation of teaching activity/program.
The pilot group of 12 preceptors who have undergone an observation of teaching session will be invited to train as peer faculty observers. Peer observation of teaching in the PCC will then be offered to all PCC preceptors for the next course beginning the fall of 2011. Our goal is to observe each preceptor twice (i.e. in 2 separate clinical sessions) over the 8-month course. Using pre- and post-observation surveys of the participants, we will attempt to determine if faculty find the exercise to be a useful tool for faculty development. Data from the observation checklist tool will be used to determine which teaching skills faculty use most frequently and to identify common areas of opportunities for improvement to inform the content of future faculty development programs.
Appendix A: Observation of Teaching Checklist Form
David Roberts, M.D.
Program: Combined Harvard Pulmonary & Critical Care Fellowship Program
Location: Beth Israel Deaconess Medical Center
Description of the Peer Observation of Teaching Activity/Program.
All fellows in the Combined Harvard Pulmonary & Critical Care Fellowship Program rotating at BIDMC are observed leading a pre-ICU rounds didactic session for medical residents and students. Observers have all received advanced training as part of education fellowships (such as Rabkin and Macy) and are either members of the Pulmonary Division or the Department of Medicine.
Description of how the peer observer provides feedback to the “observee.”
Observers watch the 30 minute didactic session and then provide both direct verbal feedback and fill out a templated Teaching Evaluation Form (Appendix B).
Description of training the peer observers received in how to conduct the observations.
Observers met years ago at the onset of the program to establish goals and objectives and review the evaluation form. As noted above, all observers are graduates of an education fellowship.
Description of how receptive the faculty have been to the observations
Fellows have been incredibly receptive to this program as it appears to be unique within the 3 hospital combined fellowship. Fellows very much appreciate direct feedback on their teaching and it is often quoted as one of the highlights of the rotation.
Educational outcomes that have resulted from the program
With the support and leadership of Dr. Woody Weiss, Pulmonary Division Chief at BIDMC, a peer-peer faculty observation program is underway and faculty have observed each other leading attending rounds in the ICU and on the Consult Service. Additional evening faculty development sessions about teaching skills have also occurred over the last year.
Future directions for the peer observation of teaching activity/program.
With one of our current fellows who is immersed in education research, we are surveying all fellows in our program regarding the opportunities for teaching and direct observation by faculty that they have over their clinical rotations. This IRB-approved study has been completed and was presented at the American Thoracic Society meeting in Denver in May 2011. A follow-up survey of all Pulmonary & Critical Care Fellowship Directors around the country regarding opportunities to learn teaching skills and have direct observation of teaching is currently underway.
Kathleen Finn, MD
Program: Peer Observation of Teaching: MGH Inpatient Clinician Educator Service
Location: MGH Inpatient Teaching Service– General Medicine
Description of the Peer Observation of Teaching Activity/Program.
Our Peer Observation of Teaching (PoT) project occurs among the 11 member Inpatient Clinician Educator Service in the Department of Medicine at MGH. The mission of the clinician educator service is to focus on improving clinical teaching and education. PoT was chosen by the group as a faculty development project to help all members improve their clinical teaching. The 11 members range in experience from 3 to 35 years and all expressed curiosity about each other’s teaching.
Three times a year each clinician educator is required to observe other members of the service. This is done on a rotating basis to that everyone gets a chance to see everyone else. Peer observations occur during routine attending rounds on an inpatient ward service. The ward team typically consists of two attending physicians (“co-attendings”), one or two junior residents, four interns, and 1-3 medical students who care for 18-24 patients on a medicine ward. During the daily 2 hour attending-led teaching rounds, the team generally focuses on the presentation and discussion of newly admitted patients, ranging from one to seven in number. The peer observer is a clinician-educator who is off-service at the time of the observation. The observer and observee discuss beforehand if there is anything specific for which the faculty being observed desire feedback.
Peer observers join the team at the beginning of rounds and introduce themselves and their purpose to all team members. The observer makes it clear that they are not present to evaluate residents or students on the team, but simply to focus on observing the teaching methods and techniques employed by the attending staff. Following introductions, the peer observer remains silent and in the background for the remainder of rounds, taking notes on teaching, but not contributing to the clinical or didactic discussion. Observations occur during all aspects or activities of attending rounds, including conference room discussions, hallway team discussions, and bedside encounters.
Description of how the peer observer provides feedback to the “observee.”
Shortly after the observation session, the observer provides verbal feedback to the observed attending. This session generally takes the form of recounting specific examples from rounds that represent teaching skills and techniques that were perceived as effective as well as offering alternative methods and potential opportunities for improvement. Following verbal feedback, the observer writes a 1-3 page narrative summary of the experience. The written narratives are given to the observed attending and all reports are placed in a shared drive accessible to all the clinician educators. One goal of the written narrative is to encourage the observer to reflect on his or her own teaching through comparison.
Description of training the peer observers received in how to conduct the observations.
There is no formal training of the peer observer in our PoT. It is our belief that the greatest benefit of PoT is for the observer (Of course, the observee does benefit here too). By observing in a “natural state”, the observer will naturally gravitate towards teaching techniques and observations they are ready to see. Each one of us is at a different level in our development as a clinical teacher and therefore our observations will reflect where we are and what we are able to see. A prescribed check-list would restrict that and would limit observations of teaching moments not on the check list. Plus a check-list would also make the PoT feel more evaluative which we were trying to avoid.
There are general themes for observation agreed upon by the group but none are prescribed. Observers are asked to ignore the clinical content and consider areas as team dynamics, bedside teaching techniques, teaching moments, feedback given and specific teaching styles and techniques used. Also, time management, overall engagement of the team in the discussion of patients, and non-cognitive aspects of teaching are considered. Each observer gravitates towards different areas depending on what they are currently working on for their teaching.
Description of how receptive the faculty have been to the observations
Because our service is small and all 11 members agreed to do this, implementation was not difficult. Everyone knows everyone else and it truly feels like a peer observation rather than an evaluative observation. While we can never totally eliminate the discomfort of being watched, the routine use of PoT has made it easier. Plus, once everyone recognized the benefits for the observer, there was more willingness to participate. The biggest challenge is finding the time in our busy schedules to do this.
Educational outcomes that have resulted from the program
- Word has gotten out among non-clinician educator faculty about the benefits of PoT and we have been asked to do PoT of non-clinician educators including the CMO of the hospital.
- We believe as a group we have all dramatically improved our teaching by observing each other. Every year more of the nominees for the clinical teaching award at MGH come from our group, as well as award winners.
- We have published a paper of our experience: Finn K, Chiappa V, Puig A, and Hunt DP. How to become a better clinical teacher: a collaborative peer observation process. Med Teach 2011; 33(2):151-155.
- Our inpatient clinician educator elective for residents includes a similar format of peer observation and reflection.
Future directions for the peer observation of teaching activity/program.
We have being doing this for 5 years and have over 80 narratives. We are planning several qualitative studies on the development of clinical teachers using the narratives.
Randy King, MD. PhD
HMS Course: Molecular and Cellular Basis of Medicine (MCM)
Location: Course Lectures and Tutorials
Description of the Peer Observation of Teaching Activity/Program.
Course director and curriculum fellow attend all lectures and provide feedback to faculty member on each lecture. All lecturers must submit annotated slides well in advance of lecture. We never have a lecturer present material that has not been reviewed in advance.
Course Director sits in on tutorials and provides feedback, especially if problems with particular tutors are identified. The Center for Teaching and Learning (Toni Peters) has been crucial in observing and providing feedback to new tutors in our course.
Description of how feedback is provided to the “observee.”
Feedback is typically provided through multiple forms: informal feedback immediately after lecture, by email, and in one-on-one meetings, depending on the nature of the issues. We provide feedback on notes/slides prior to lecture, allowing us to correct many problems before they even reach the lecture hall. In some cases where it is clear that feedback will not be effective at improving performance, different lecturers are chosen in the following year.
We have implemented a system of “progress notes” in tutorials that provides tutors with session-by-session feedback from students; this has been very effective at getting our tutorials off to a good start and improving tutorial quality.
Description of training the peer observers received in how to conduct the observations.
We have piloted a tutor training system in which prospective tutors act as co-tutors with an experienced tutor. This has been very useful for evaluating the qualifications of tutors as well as providing on-the-job training.
Description of how receptive the faculty have been to the observations
In most cases, peer observation is well-received, and has resulted in an improvement in the quality of lecture or tutorial performance.
Educational outcomes that have resulted from the program
The informal feedback given to lecturers and tutors in our course has helped improve the quality of teaching in both settings in our course.
APPENDIX A: PCC OBSERVATION OF TEACHING CHECKLIST
Preceptor: Date: Observer:
O=observed, NO= not observed
EFFECTIVE AMBULATORY TEACHING TECHNIQUES: / O / NO / Specific Examples/CommentsEstablishes Positive Learning Climate:
- Actively demonstrates enthusiasm, interest in teaching
- Actively demonstrates respect for learner, listens carefully
- Facilitates good relationships with the staff
Communicates Goals of the Session:
- Ask student questions to assess level of knowledge
- Asks student to set own learning goals for the session
- Preceptor states own expectations/goals for the session
Employs Effective Teaching Methods:
- Selects patients appropriate to the goals
- Creates opportunity for independent evaluation of patients
- Teaches is focused, uses good time management
- Orients the student to the specific patient encounter
- Questions student (fact based)
- Questions student (reasoning)
- Allows adequate wait time
- Thinks out loud, explains own clinical reasoning
- Conveys information clearly
- Encourages critical appraisal of the medical literature, EMB
- Encourages student to develop an assessment and/or plan
- Encourages learners to ask questions
- Models sensitive and respectful attitude toward pts
- Demonstrates interview/communication skills
- Demonstrates physical diagnosis skills
- Demonstrates counseling, shared decision-making
- Observes student obtain the patient history
- Observe student perform physical exam skills
- Observe student provide counseling/results/treatment plan
Fosters Reflection
- Debriefs with student to clarify learning points, summarizes
- Provides positive feedback
- Provides formative (corrective) feedback
- Solicits learner feedback
- Encourages reflection and self-directed learning
- Makes explicit plans for future learning
- Facilitates scheduling return visits and opportunities for f/u
- References learning from previous sessions
Preceptor’s goals for the observation:
Is there anything you would particularly like feedback about?
Observer’s notes:
- General comments (include any specific details which may affect the observations such as early or late in the clerkship, patient no shows, etc.)
- Preceptor’s Strengths
- Opportunities to Consider
- Summary of Suggestions for Alternative or Complementary approaches (List bullet points)
- Please rank in order 1-5 (1=best) the teaching skills best demonstrated by this preceptor during this observation
_____ created a safe learning environment
_____ oriented the student to the patient encounter
_____ thought out loud/justified management approaches
_____ asked the student questions to enhance learning
_____ created opportunities for student autonomy in patient encounters
_____ gave feedback to the student
_____ created opportunities for students to ask questions
_____ debriefed encounter to clarify learning points
_____ facilitated longitudinal learning (referencing prior learning, facilitated f/u pt visits)
1
APPENDIX B:
1