Principal Investigator: Watkins, ScottVersion Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
Emergency Checklist Cards for Intraoperative Pediatric Critical Events: Development and Evaluation Using High Fidelity Simulation
Scott C. Watkins M.D.
Assistant Professor
Division of Pediatric Cardiac Anesthesia
Monroe Carell Jr. Children's Hospital at Vanderbilt
2200 Children's Way
Suite 3115
Nashville, Tn 37232
P: 615-936-6808
Shilo Anders, Ph.D.
Center for Research and Innovation in Systems Safety
Vanderbilt University Medical Center
1211 21st Avenue South
Medical Arts Building, Suite 732
Nashville, TN 37211-1212
P: 615-936-6598
Table of Contents:
Study Schema
1.0Background
2.0Rationale and Specific Aims
3.0Inclusion/Exclusion Criteria
4.0Enrollment/Randomization
5.0Study Procedures
6.0Reporting of Adverse Events or Unanticipated Problems involving Risk to Participants or Others
7.0Study Withdrawal/Discontinuation
8.0Statistical Considerations
9.0Privacy/Confidentiality Issues
10.0Follow-up and Record Retention
Appendix A: List of Emergency Checklist Cards
Appendix B: Scenarios and Metrics
Appendix C: Survey
Appendix D: Consent for video taping
1.0Background
Emergencies involving pediatric patients in the operating room are uncommon events that require prompt and coordinated care from the operative team to insure a safe resolution. The incidence of critical events during pediatric procedures is unclear, although rates between 2 and 3% have been reported (1-3). A review of the UK National Reporting and Learning System, reviewed 96,298 critical incidents over a three year period, of which 606 incidents occurred in the pediatric population(2). A review of the first 10,000 anesthetics performed at a children’s hospital in Singapore found a critical incident rate of 2.97% (1). Respiratory events, primarily laryngospasm, were the most common, followed by cardiovascular events, including hypotension from hemorrhage, sepsis and arrhythmias(1). Marcus et al reported 668 incidents from a review of over 28,000 anesthetics, giving a rate of 2.4% at a single hospital in the UK(3). Due to variation in reporting of adverse events, the true incidence is likely under-reported. The majority of critical events in children are related to the cardiovascular and pulmonary/airway systems, with hemorrhage, laryngospasm and adverse medication reactions being most common (4).
The practice of pediatric anesthesia has seen a steady increase in overall safety with the introduction of pulse oximetry and capnography as standard monitors and the advent of safer volatile anesthetics(4). Despite the overall increase in safety, critical events and emergencies still occur, and often practitioners are unprepared to manage these events. The enhanced safety of our practice may be contributing to the lack of preparedness of practitioners, who may go long periods of time before experiencing a critical event. Studies have demonstrated that practitioners skills in advanced life support decay over time (5) and newer training hour guidelines limit exposure of trainees to emergency events. Further, studies have shown that knowledge gaps exist amongst providers in regard to current guidelines for cardiopulmonary resuscitation(6) and that providers often fail to adhere to best practices(7). The irregularity and unpredictability of critical events contributes to knowledge and skill gaps in practitioners’ ability to manage these events.
Checklists have been proposed as solutions to knowledge gaps and failures to adhere to standards of treatment that occur during crisis situations. Ziewacz et al, recently published a study in which they identified 12 frequently occurring operating room crises and created a checklist with evidenced-based best practices for each, for use during intraoperative emergencies in adults(7). They subsequently tested the emergency cards in high fidelity simulator sessions with operating room teams consisting of a surgical attending, surgical resident, anesthesia attending, anesthesia resident, circulating nurses, and surgical technologists. Each team performed scenarios with and without the emergency checklist cards as cognitive aids. The primary outcome was adherence to critical steps. They found that the group that used the checklist had a 6-fold reduction in omission of critical management steps. The test OR teams found the crisis checklists to be “straightforward, usable, and beneficial” (7).
The use of electronic cognitive aids has grown in popularity with the advent of personal digital assistants (PDA) and smart phones. A recent randomized controlled trial compared performance during simulated cardiac arrest scenarios with and without the use of the Resuscitation Council UK’s iResus©application on a smart phone(8). Results demonstrated a significant improvement in performance ona standardized test of advanced lifesupport skills in the group using the smart phone application. In contrast, other studies have demonstrated potential harm with the use of electronic cognitive aids due to selection of the incorrect algorithm (9) and a delay in initiation of chest compressions(10). This highlights the limitation of any cognitive aid, that the user must make the correct diagnosis and select the correct algorithm. Further, the introduction of new technology has the potential to delay the initiation of basic life support therapies. To date, no studies have evaluated the incorporation of cognitive aids into anesthesia information management systems (AIMS).
2.0Rationale and Specific Aims
The types of critical events that occur during pediatric surgery differ from those that occur during adult procedures, with respiratory and airway events being more common in children(3) and myocardial ischemia and ventricular arrhythmias occurring more often in adults. Because of these differences, the Society for Pediatric Anesthesia committee for safety and quality has developeda series of emergency checklist cards tailored for use by pediatric peri-operative teams. The emergency checklists cover a range of potential intraoperative events including cardiovascular, airway, pulmonary, medication reactions and equipment related events. See Appendix A for complete list of emergency cards. The topics chosen are a combination of common life-threatening operating room crises and rare life threatening events affecting the pediatric population. Each emergency card contains evidence based treatment steps and best practices as determined by the panel of pediatric anesthesiologist and from review of relevant literature.
This study will use emergency checklists in simulated pediatric emergencies to test their efficacy and ease of use. Each simulated crisis scenario will contain a set of key processes, consisting of critical and essential steps for the successful management of the life threatening events. We hypothesize that teams will adhere to best practices more often when the emergency checklists (paper or electronic) are used than when no checklist is used in simulated pediatric operating room emergencies. We hypothesize that use of the checklist (paper or electronic) will reduce the amount of time required for teams to make the correct diagnosis. In addition, we believe that an electronic version of the checklist incorporated in the AIMS will further improve adherence and time to diagnosis. During the simulation, participants (residents and fellows) will be asked to complete 6 scenarios related to pediatric OR emergencies either using or not using the checklists. After which they will be asked to talk about what they did, explain any frustrations and complete a questionnaire about their experience. Analysis about performance and subjective data from the questionnaires, observations and retrospective video data will highlight main concerns around pediatric emergencies in the OR and the use and design of the checklists and how it impacts the current workflow and participant response.
3.0Inclusion/Exclusion Criteria
Inclusion:
-Current residents in anesthesiology at our institution
-Fellows in pediatric anesthesiology at our institution.
4.0Enrollment/Randomization
Currently, residents in anesthesiology and fellows in pediatric anesthesiology at our institution participate in pediatric anesthesia simulation for the purposes of training and education. These sessions are held monthly for residents rotating through VCH and approximately every three months for the fellows. Our intent is to recruit and consent from this pool of residents and fellows. The scenarios will be used for the simulation sessions as education tools, regardless of the trainees desire to participate.
Those that consent to the study will be randomized to performing the scenarios with or without the aid of the emergency checklist cards.
5.0Study Procedures
This study consists of a general introduction to the simulation center, simulated emergency OR scenarios, and subjective evaluation. The study will take place in CELA and each session will consist of 3-4 resident or fellow anesthesia providers. As part of their current training rotations, these individuals regularly participate in simulation training and the evaluation of the checklist will be incorporated into this. During the general introduction to the simulation center, participants will be informed about the educational objectives for the simulation training including conduct, schedule and the creation of a positive learning environment. The video consent will also be obtained; participants choosing not to be videotaped can still fully participate in the simulation session without any repercussions.
The researcher will then answer any questions that the participants have and provide a brief overview of about the first patient of the day. One participant will be designated as the lead anesthesiologist and backup anesthesiologists will also be designated. This will rotate such that all participants will be the lead and backup anesthesiologist at least once and exposed to each condition. Because it is a within-study design, if a cognitive aid (checklist) is to be utilized for the scenario, participants will be provided with the aid.
The participant will then continue to the simulation part of the study. There will be 6 scenarios, which are characterized as A through F in the table below (for complete scenario description see Appendix B). The scenarios will be counterbalanced to minimize any potential order effects. Each participant group will experience each condition, the standard practice, the cognitive aid and the electronic cognitive aid (characterized as 1-3 on the table below). This scheme is only illustrated for 9 groups below, so it will be
repeated as necessary to reach our N of 60 participants. The table represents how this will be done. Additionally, each scenario will be tested in each condition (as can be seen by the table below).
Table 1: Sample of counterbalancing for each scheduled session, where 1=no aide, 2= aide, 3=electronic aid and A through F are patient emergencies
Scenario OrderParticipant Group / First / Second / Third / Fourth / Fifth / Sixth
1 / 1A / 1B / 2C / 2D / 3E / 3F
2 / 2A / 2B / 3C / 3D / 1E / 1F
3 / 3A / 3B / 1C / 1D / 2E / 2F
4 / 1C / 1D / 2E / 2F / 3A / 3B
5 / 2C / 2D / 3E / 3F / 1A / 1B
6 / 3C / 3D / 1E / 1F / 2A / 2B
7 / 1E / 1F / 2A / 2B / 3C / 3D
8 / 2E / 2F / 3A / 3B / 2C / 2D
9 / 3E / 3F / 1A / 1B / 2C / 2D
The researcher will brief the participant about the patient’s current condition and any current concerns. The participant will be given the opportunity to ask any questions before starting the simulation. The simulation takes approximately 2 hours to complete all six scenarios. Between each scenario, participants will be allowed a break if necessary and to accommodate the training aspect of the study there will be a debriefing after each scenario. As illustrated in Appendix B,a scoring checklist of key metrics, consisting of critical and essential steps for the successful management of the life threatening events, will be recorded for each simulated scenario. The effect of the cognitive aid will be determined by the accuracy and comprehensiveness of the participant’s responses. For example, did they complete each step required on the checklist. In addition, certain items on the scoring checklist will be timed to determine the impact of the cognitive aids on time to diagnosis and time to correct therapeutic intervention.
The use of videotape is essential for recording the participant’s interactions with the emergency cards. The videotape will capture details that observers might miss, such as facial expression and verbatim comments. Because the study seeks to understand the usefulness and usability of the emergency cards as well as its possible effects on the participant’s information processing and decision making, it is not feasible to capture all of the potential contributing information real time without the ability to review video records of the simulations for data validation, coding, reliability and to inform tool usability.
Following completion of the scenarios, the participant will complete a questionnaire soliciting information about their interactions with the checklist and demographic information (Appendix C). After completion of the scenarios and questionnaires, the participants will be debriefed around their participation in the study and any further evaluation of the checklist and thanked for their time.
6.0Reporting of Adverse Events or Unanticipated Problems involving Risk to Participants or Others
The PI, co-investigator, and their research team will conduct data and safety monitoring. The PI will have ultimate responsibility for data quality and subject safety. In addition, the PI, in consultation with the research team will regularly monitor study progress, goal achievement, and overall research direction.
Data Monitoring. Database security is maintained using a multi-layered approach to both limit access and the ability to alter data. There are strong protections restricting access to the VUMC network. Local installation of proprietary software is required to detect our server on the network. Each authorized user has a unique username and password that allows specific access and editing privileges. Browsing level access only is used except when data entry/editing are explicitly intended. While multiple authorized users can simultaneously view data, only a single specifically authorized user can edit a given record at any time. The database is stored on a secure server, which is also password protected and only accessible to authorized lab personnel, in our locked laboratory offices. Database backups are conducted automatically on a daily, weekly, and monthly basis. The PI will regularly monitor the data collection and analysis process for data appropriateness, comprehensiveness, accuracy, and timeliness.
Participant Safety Monitoring. The risks to subjects, although quite low, are almost exclusively related to potential release of data about clinical performance or adverse events. These are discussed in other sections of this proposal. However, because of the sensitive nature of the research data, the entire project team will continually monitor these issues and evaluate in great detail any concerns raised by study participants.
This study poses minimal risk to the participants because it is being conducted in a simulated environment where participants are being asked to perform tasks that are routine in their everyday work. Thus, while psychological stress, feelings of inadequacy, disclosure about participant performance, and inconvenience of time involved with study participation are the possible risks participants may experience, these risks are minimal in this study.
Colleagues and supervisors will not be made aware of who has participated in the study or participant’s performance. Additionally, the simulation center and researchers will create a non-judgmental atmosphere and low stress- tone of questioning after each scenario by maintaining a professional environment and supporting the participant in their task. Additionally, the researchers will emphasize to the participant that they can withdraw from the study and any time without and negative consequences to their job or reputation. Finally, participants will be reminded that the simulation study is not testing their performance, but usefulness, usability and performance of the new checklist.
During the data collection, one or more of the researchers will be present to mitigate any unforeseen problems. If concerns about the study or participant’s safety should arise, the researcher will halt the study and review study procedures.
Participant performance will be measured in binary fashion using a list of predetermined key steps for the management of each critical event. See AppendixBfor complete list of measurements for each scenario.
Finally, in order to minimize risk of performance disclosure, especially to employers, participant’s personal information will be stored in a locked filling cabinet in a locked laboratory separate from all other study materials. The rest of the study materials will be assigned a random participant number and be identified that way. Participant’s performance characteristics will only be reported for the group as a whole when data is published. Additionally, any videotaped data that is viewed outside of the research lab will be stripped of all participant identifiers using editing technology.
The risks to participants in this study as described above are quite minimal compared to the potential benefits and widespread impact this study. The potential benefits to science and humankind include a better understanding of how cognitive aides such as the emergency cards can impact participant performance in particular situations, especially considering the high consequence of failure when an emergent issue occurs in the OR. It is very difficult to study how healthcare practitioners manage their patients during emergency events using cognitive aides because of the unpredictable nature of emergencies and patient situations. Participants will provide insight as to how to improve healthcare delivery in situations that have been traditionally difficult to study.
7.0Study Withdrawal/Discontinuation
Participants may withdraw previously signed consent forms at any time. In addition, participants may change their selection regarding the extent to which the video data may be used, if at all.
8.0Statistical Considerations
Describe the statistical power of the study, the confidence intervals and the method for analysis. Describe any possible deviations and their statistical impact.
9.0Privacy/Confidentiality Issues
We are requesting a waiver of consent because we do not want to inform participants that we are studying the efficacy of the cognitive aids. Informing them that the emergency checklists are the subject of this investigation might cause them to adhere to these procedures even if they would have taken other actions in the simulation. We will tell them of their participation at the end of the simulation session. A video consent will be obtained at the beginning of the simulation session (Appendix D).
To protect the privacy of the research participants, signing the video consent form and data collection will be completed at the simulation center, which is a limited access area. The data collected will be in the form of electronic spreadsheets, digital video records, and paper surveys; each requires separate handling.