Nuts & Bolts Approach to the ACGME Survey – Best Practices

New Program Director’s Workshop

Saadia Akhtar & Hollynn Larrabee

Goals of this Session:

1. Familiarize everyone with the ACGME Annual Survey – what is it and how can we prepare for it effectively

2. Review key changes ACGME made to the survey in 2011 (new questions, changes in emphasis!)

3. Provide tips that may help programs prepare, even months in advance for optimal results

Additional Changes to the 2011 Survey:

1. They have changed the language e.g. “teaching” is now “instruction”

2. They have added faculty “and staff” for a lot of the questions

3. There is no “Not applicable” so they may need to skip certain questions (e.g. duty hour questions 3 & 7)

4. The answer scale often flips back and forth (never vs always) – make sure residents read answer stem carefully

Question 11 & 12- Supervision

·  Emergency Medicine is a 24/7 specialty. Attendings are always present or immediately available in the ED

Question 13- Instruction

·  Attendings can verbalize that “This is a teaching point regarding your case” when they are discussing a case with a resident in the ED during a shift-especially if it is not a medical knowledge teaching point

·  Establish “Teaching Cards”- list of topics that the resident documents to learn about by the end of the shift or by the next shift and could also make the resident report back to that particular attending

·  Establish a “Teaching Resident” for the senior class during which their sole responsibility is to teach the junior residents by bedside instruction, printing out and reviewing relevant articles during the shift, etc

Question 15 and Question 26- Scholarship and Inquiry (Note increased emphasis on this topic this year)

·  Establish a departmental “Research Day” during which all residents and attendings present their research

·  Participate in Hospital-wide or Medical School Research Day

·  Include residents in departmental research meetings or have quarterly resident research meetings

·  Encourage residents to present at regional and national meetings

·  Collaborate on research projects with other specialties within the institution and/or other EM programs

·  Develop a research curriculum with a didactic component and/or individualized program

·  Include evidence based medicine and research didactics as part of your asynchronous learning program

·  Implement a “Research/Scholarly Activity” rotation or elective

Question 16- 20—Evaluations

·  Remind your residents what you do, interns especially may be less familiar with annual evaluations

·  Utilize an anonymous electronic format in order to do evaluations for residents, faculty and program

·  Do evaluations multiple times a year, especially resident evaluations of faculty

·  Highlight any positive changes that have occurred based on resident feedback – see question 20

Question 21-22-- Goals and Objectives

·  Have an electronic version of the Goals and Objectives document available at all times e.g. accessible on a residency website/electronic program –remind residents of this in the weeks prior to the survey

·  Distribute the Goals and Objectives document on a monthly basis to all residents and faculty

Question 23 & 24 - Feedback

·  For each evaluation form, make a mandatory “Comments” section with minimum word count

·  Include for example “Areas of Strengths” and “Suggestions for Improvements” as mandatory fields

·  Assure that residents know how to review their evaluations and either in digital or hard copy form

Question 25- Fatigue and Sleep Deprivation

·  Have multiple sessions on “Fatigue and Sleep Deprivation” during the conference academic year

·  Schedule a didactic lecture on “Fatigue and Sleep Deprivation” right before the ACGME survey goes live

·  Ask about “Fatigue and Sleep Deprivation” during the semiannual meetings, with a documented check list

Question 28- Reference Materials

·  Establish online access to reference materials

·  Subscription to online journals, EBM resources, textbooks

·  Maintain an area within the ED with hard copy references such as common EM related textbooks

·  Establish online access to medical school library including online remote access to library

Question 29- Fear of intimidation/retaliation

·  Offer support via various mechanisms such as: GME office- DIO or Director of GME, House Staff Committee, Corporate Compliance Services, Human Resources, Dean’s Office, Ombudsman for house staff, Chair of Department, Program Director, Associate/Asst Program Director, Chief Residents

·  Place a feedback box in resident lounge or ED for suggestions, comments – this may appeal to less vocal or junior residents

·  Review changes in residency as they relate to resident concerns at monthly/quarterly Chief Resident rounds, also use this time to seek anonymous feedback from residents via the Chiefs

Question 31 and Question 33- Balance between education and clinical demands

·  Establish clear guidelines for residents and other staff in regards to clinical work

·  Institute simple solutions to decrease burden on residents e.g. zone phones for each resident so that they make page and receive phone calls to their direct phone number while on shift

·  Decrease amount of duplication of work e.g. order entry process for labs, radiology tests

Question 32- Ability to learn compromised by other trainees

·  Establish clear guidelines e.g. who does which procedure

·  Allocate appropriate number of patients per resident type/level

Final thoughts on ways to optimize the intent of the survey & it’s results:

·  Consider a mock survey 3-6 months in advance to target weaknesses in the residency

·  Instruct residents about the importance of the survey, how it is used by the RRC

·  Review questions several weeks prior to actual survey – prior to conference, via email, at chief resident rounds, encourage questions and feedback during these sessions

·  In preparation for survey, review the things your residency does locally related to survey questions

·  Consider reserving a computer room for an hour to facilitate participation in the survey

·  Refer residents to FAQ about survey on ACGME website

Finally, remember to use your survey results to focus on areas to improve for the following year. While it is not a perfect tool, it is intended to help programs identify problem areas. Include this in your Annual Program Evaluation and plan for coming year.