1. Background
Since the initial publication of the World Health Organization (WHO) surgical safety checklist in 2009 (1), the potential for checklists and other cognitive aids to reduce errors during surgical procedures has been recognized worldwide. The impact of checklists in anesthesia has been investigated by a number of studies (2-4), and two recent editorials conclude that there is now “overwhelming evidence” (5), as well as “sufficient justification to warrant widespread adoption” of perioperative crisis checklists “when well crafted” and when “clinicians are wisely prepared” (6). Various institutions and professional anesthesia societies have begun to advocate for the use of such aids (4, 7-9).
Surveys on anesthesia providers’ view on checklists and other cognitive aids are scarce and mostly linked to the exposure of the specific aid tested (2, 3, 10). We also found a lack of data about the ability to perform anesthesia tasks without making errors if the providers relied solely on experience and memory.
In a Norwegian study published in 2010 (11), anesthesia providers were interviewed regarding the use of a pre-induction checklist. Their opinions were summarized by the authors into five key concepts: 1. Checklists might divert attention away from patients; 2. Checklists can significantly interfere with the anesthesia workflow; 3. The departmental leadership has a high influence on the acceptance of checklists; 4. Checklists can improve confidence in unfamiliar settings; and 5. Checklists might reveal insufficient equipment standardizations.
2. Objectives
Within our department a focus group of anesthesia providers (attendings, residents and certified registered nurse anesthetists [CRNA]) was formed to develop and implement crisis and routine checklists into anesthesia care.
In order to understand the current stance of anesthesia providers on checklists and other printed cognitive aids, as well as reasons for potential pushback, we determined the need to conduct a survey among all anesthesia providers within the Department of Anesthesia at the University of California, San Francisco (UCSF). The survey was developed to substantiate commonly overheard opinions and concerns of our providers on checklists. We also included questions about the key concepts from the Norwegian study and the perception our anesthesia providers have on their ability to perform anesthesia tasks without any errors based on experiences and memory alone.
3. Patients and Methods
The study was approved by our local IRB, and the requirement for written informed consent waived (University of California, San Francisco, Committee on Human Research, Office of Research Administration, San Francisco, CA; IRB #13-12367). All faculty, fellows, residents (including interns) and CRNAs with a full-time position within the department were surveyed. Practice locations of those surveyed included the University hospital, the city’s county hospital and trauma center (San Francisco General Hospital), the Veteran’s Affairs hospital, and private institutions (Kaiser Permanente San Francisco, Children’s Hospital Oakland).
A web-based survey (Qualtrics Research Suite, Qualtrics, Provo, UT, www.qualtrics.com) was developed by the departmental checklist focus group. To identify commonly encountered perceptions, concerns, obstacles and facilitators for implementation of checklists in anesthesia the current literature was reviewed. All survey questions were reviewed and vetted independently by the members of the focus group.
Demographic questions included professional status (faculty, fellow, resident or CRNA), and years of clinical experience in anesthesia. The questionnaire was aimed to investigate:
- What anesthesia providers think about the WHO surgical time-out, the most known and widely established perioperative process using a checklist.
- How competent do providers feel to perform anesthesia tasks without any lapses when relying only on memory and experience.
- How anesthesia providers rate the usefulness of specific checklists in different aspects of anesthesia care (equipment preparation, hand-offs, routine and emergency procedures).
- Whether anesthesia providers would feel uncomfortable using checklists publicly (“cheat sheets”).
- The provider’s opinion on potential downsides to using checklists during anesthesia care.
- How the providers would rank factors that would promote the acceptance of checklists in anesthesia.
- How anesthesia providers would view the use of a pre-induction anesthesia checklist if used when the provider were a patient themselves.
The survey questions consistently used the term “printed or electronic aids/checklists” and “printed or electronic aids (checklists, cards, books)”. While the majority of cognitive aids currently studied and advocated for are in fact true checklists, other cognitive aids like algorithms have been used, too (2, 3); however, in style of the recent publications as well as terminology used by professional airline pilots, for sake of easier reading the term “checklist” in this manuscript is used, although it might apply to other printed cognitive aids.
The survey link was emailed to all faculty, residents and CRNAs within the Department of Anesthesia at UCSF.
3.1. Statistics
After the survey period (4 weeks), the data were downloaded from the website and analyzed using IBM SPSS Statistics v22 (IBM Corp., Armonk, New York). To test for differences according to years of experience on binary outcomes (yes/no), we used Pearson Chi-Square test. To compare the answers on Likert scales based on years of experience, we used the independent samples Kruskal-Wallis-Test. For all tests, a p-value of less than 0.05 was considered to be statistically significant.
4. Results
After sending the survey questions to 312 anesthesia providers in our department, a 69% response rate was achieved (Table 1). Regardless of position and experience, checklists for time-outs, crisis situations and for unfamiliar routine situations are valued by the vast majority of anesthesia providers. Less than 50% valued routine checklists, with significantly higher interest by very junior and very experienced providers. Less than a third of the providers are concerned about distraction and delay of patient care, while a strong minority still feels uncomfortable using checklists.
Table 1. Demographics
| By Status | Response Rate a | % of Total Responses |
|---|---|---|
| Completed Surveys | 214/312 (69) | 100 |
| Faculty | 95/147 (65) | 44 |
| Residents/Fellows | 78/104 (75) | 36 |
| CRNA | 41/61 (67) | 19 |
| Years of experience | ||
| Less than 2 y | 22 | |
| 2 – 10 y | 46 | |
| More than 10 y | 32 |
a Values are presented as No (%).
4.1. Perception of the surgical time-out
98% of all respondents considered the surgical time-out to be either important (40%) or very important (58%) for patient safety. Only 0.5% (n = 1) considered the time-out to be unnecessary. No provider felt that the surgical time-out is potentially dangerous.
If the survey participants took the perspective of being a patient, again 98% would expect their surgical team to conduct a time-out but this time 84% “strongly expect” a time-out, and 14% would “somewhat” expect one to happen. For both questions, no significant differences were found for subgroups by years of experience or position.
4.2. Perception of Clinical Competency Without a Checklist
64% of all providers agreed or strongly agreed that they felt competent to always prepare and perform routine and standard anesthesia care based on memory and experience only (see subgroups by years of clinical experience in Figure 1). When asked if the provider felt competent to deal with emergency situations (such as cardiac arrhythmia, cardiac arrest, malignant hyperthermia etc.) based on memory and experience only, 34% agreed or strongly agreed with this statement (Figure 2). Interestingly, when performing routine anesthesia care they do not do very often, only 19% agreed or strongly agreed that they felt competent without any checklists, while 31% disagreed or strongly disagreed (Figure 3).
Figure 1.
“I feel competent that I always can prepare and perform routine and standard anesthesia care without any lapses based on my memory and experience only”. Replies by years of clinical experience (independent samples Kruskal-Wallis Test, P < 0.01).
Figure 2.
“I feel competent that I always can deal with an emergency anesthesia situation (cardiac arrhythmia, arrest, malignant hyperthermia etc.). without any lapses based on my memory and experience only.” Replies by years of clinical experience (independent samples Kruskal-Wallis Test, P < 0.01).
Figure 3.
“I feel competent that I always can prepare and perform routine anesthesia care I do not do very often (peds, OB etc) without any lapses based on my memory and experience only.” Replies by years of clinical experience (independent samples Kruskal-Wallis Test, P < 0.01).
Significant differences were found within the subgroups based on the level of clinical anesthesia experience. Over 2/3 of all providers with less than 2 years of practice did not feel that they could always perform in emergency and unfamiliar routine situations ((P < 0.001 and P = 0.001, respectively); for routine anesthesia, only a third of the least experienced providers felt insecure (P < 0.001).
4.3. Acceptance of Checklists
Survey takers were asked if they would use checklists, if available, while preparing for or performing anesthesia care; their answers were compared according to years of anesthesia experience (< 2 years, 2 - 10 years, > 10 years, Figure 4).
Figure 4.
Acceptance of checklists to prepare for anesthesia and for hand-offs (P value indicates difference according to years of experience; Pearson Chi-Square test).
Very few providers (0 - 6%, depending on years of experience) stated that they did not see any need for checklists at all. Not surprisingly, for most situations the acceptance of checklists was much higher in the group of inexperienced providers (< 2 years). Also, all providers would be much more likely to use checklists in unfamiliar situations such as emergencies. Interestingly though, providers with 2 - 10 years of experience were less likely to use checklists for routine care and routine room preparation than those in one of the other groups.
4.4. Checklists as “Cheat-Sheets”
One reason why providers might refrain from using checklists is the potential perception of them being less skilled because they need to use a “cheat sheet”. In our survey, only 45% of the providers said they feel comfortable using a checklist in front of colleagues or other team members, while 17% are uncomfortable doing so. Providers with fewer years of anesthesia experience were significantly less comfortable using checklists overtly during clinical care, while seasoned providers are rather indifferent or even comfortable (Kruskal-Wallis Test, P = 0.01).
4.5. Side-Effects of Using Anesthesia Checklists During Patient Care
31% of all providers in our survey believe that checklists might significantly delay patient care. In contrast, 71% believe that anesthesia checklists might improve efficiency of the anesthesia workflow. Distraction from patient care concerns 27% of all providers, while 52% do not consider this to be an issue. No significant differences were found by comparing clinical experience or position.
4.6. Impact of Promoting Factors to Use Anesthesia Checklists
When asked about potential factors that would increase the use of checklists in anesthesia, the survey participants rated usability (e.g. integration into workplace and ease of use) highest, while political endorsement (e.g. department leadership, ASA) was ranked as less likely to be helpful (Figure 5).
Figure 5.
Factors Thought to Promote Use of Checklists in Anesthesia
4.7. Patient’s Perspective on a Pre-Anesthesia Induction Checklist
Providers were also asked how they would feel if they were a patient and observed their anesthesiologist using a pre-induction checklist. 63% of all survey takers believe they would have a positive reaction; 26% responded they would feel safer, and that their anesthesia provider would appear to be very professional. 37% did not see any harm using checklists if their provider considered them important. Only 13% would react negatively; 12% responded such an observation would make them nervous and that they would think their provider must lack experience. No significant differences were found by comparing clinical experience or position.
5. Discussion
Our results confirm an interest by all anesthesia providers to use checklists for routine and emergency anesthesia procedures, but less so for routine anesthesia care.
Anesthesia checklists are increasingly promoted by various professional anesthesia organizations (4, 7, 8), and several published studies have consistently demonstrated their potential benefits to improve patient safety in the perioperative setting (2, 3, 8, 10). While checklists are still not considered to be standard of care in anesthesia, we believe the acceptance of such aids depends on the personal perceptions individual providers have, and the culture of the institutions they work in.
Other authors investigating checklists in simulated scenarios and real-life clinical settings have already published results of surveys among their participants (3, 4, 10, 12). However, these surveys had a relatively low numbers of participants (n = 20 - 67), and the surveys focused solely on the specific checklist that was being investigated. Our survey among 312 anesthesia providers in our department was met with a good response rate of almost 70%, so we assume that our results are representative for all our department’s providers.
We were very pleased to see that the surgical time-out, which is not only recommended by the WHO, but also mandated by the Joint Commission, has found widespread acceptance as an important safety process. Our survey did not inquire about the specific checklist used to perform the timeout in order to not bias responses related to the potential use of badly designed checklists. However, we still believe that the responses given do in fact reflect anesthesia provider general opinions on the value of safety steps like checklists. Interestingly, the survey takers opted more strongly for the surgical time-out from the perspective of being a patient, than they did from the perspective of the provider. This phenomenon has previously been reported by Atul Gawande after conducting a survey of the attendees of the 2011 ASA meeting (13). He found that 80% considered the surgical time-out to be useful and important, while the remaining 20% considered the time-out to be rather unimportant or even waste of time. However, in his investigation, half of those who were dismissive about the time-out would want to have a time-out done if they were patients themselves.
The positive approach to checklists in perioperative medicine is the result of a process that has been going on for the last decade. For instance, in 2002 Hart and Owen (10) investigated the potential impact of a pre-induction checklist for elective c-sections requiring general anesthesia, and included a small survey of all 20 trainees. 95% of the participants felt a pre-induction checklist to be useful, and 80% showed interest in continuing using checklists in future simulation training. However, only 40% of all surveyed providers claimed they would like to use such an aid in a real life scenario. This much lower interest in checklists found in the study by Hart and Owen could be explained by the fact that at the time the study was done, such checklists had not been published or promoted previously, and the publication even preceded the implementation of the WHO surgical safety checklist by several years.
The training of airline pilots ingrains the checklist as an irreplaceable safety tool not only for rare emergency situations, but also for the safety of the daily routine flight situation. In contrast, anesthesia providers, like many other health care workers, have traditionally been trained to rely on memory and experience. Our survey confirms that the majority of providers in an academic setting believe they can perform routine anesthesia care without any lapses based on memory and experience; consequentially less than 50% state they would use routine checklists if available. For anesthesia crisis situations, however, our data seem to be contradictory: While 96% of providers surveyed claim they would use crisis management checklists if available, more than 1/3 still agree or strongly agree that they can manage such crisis situations based on memory and experience only. These seemingly conflicting results may reflect a discrepancy between the rational understanding that crisis checklists are necessary, which conflicts with the engrained culture in health care that good providers must be able to execute tasks from memory. The existence of this culture is highlighted by the fact that 11% of providers with less than 2 years of clinical experience believe they could handle crisis situations based on memory and experience only. While there have not been any previous studies surveying such questions, one can only speculate if such views are decreasing.
Only a minority of participants felt uncomfortable using such aids in front of colleagues or operating room team members, and over 70% believe that using checklists might improve efficiency. The importance of a consistent, high quality patient-care transition has also recently gained increased attention (14, 15), and our study found that the majority of providers claim they would use checklists not only for transition of patient-care, but also for shift handoffs.
Less than a third of all providers believe using checklists might delay or even distract from patient care; on the other side, however, our survey revealed that “easy to use” and “thoughtful integration into the anesthesia workplace” were considered much more important factors promoting the usage of such aids than any departmental or organizational endorsement. These are valid facts that must be properly addressed when implementing checklists, especially in anesthesia.
5.1. Limitations
This survey was conducted in a single academic US institution, and results cannot be easily extrapolated to other settings. However, we believe the results do reflect at least the current assessment of checklists by anesthesia providers at academic institutions in the US.
A significant limitation of our survey is the restriction to a single United States institution. We certainly cannot deduce from our results current perceptions on checklists in other countries, and our results might not reflect the current concepts anesthesia providers have on checklists in other departments, especially those in a non-academic setting. However, we still consider our results likely representative at least for other academic US institutions for various reasons. First, our survey included a diverse provider group with various backgrounds. Many of our faculty have been trained abroad, and all CRNAs from our department have had a significant portion of their education and work experience outside our institution. The results include also the perceptions of future residents in their final stage of the intern year, which do not differ from those of our residents in their first year of training. Their opinions would certainly not reflect any departmental “esprit de corps”, but rather that of their diverse medical schools. Furthermore, like many academic anesthesia departments, ours is spread over various hospital institutions (including the Veterans Affairs Hospital, the County Hospital, and private facilities), all of which have established independent sub-cultures that reflect more the culture of the respective institution they work in rather than that of a unique, consolidated departmental concept. Finally, while our department is widely recognized for its research output and educational position within the US, it has so far not been instrumental and acknowledged in developing, implementing and promoting anesthesia checklists or other cognitive aids. Yet, there have been some “grass-root” checklist projects established in a variety of locations and settings within the department that have not been significantly promoted by the departmental leadership. Thus, we conclude that the surveyed provider population should not be considered especially influenced by a strong promoting, academic exposure, nor by lack of exposure to any checklists within the department.
However, we do believe that a nation-wide survey of the perception of anesthesia providers on checklists and other cognitive aids would be of interest, especially to reflect the views of anesthesiologists in non-academic anesthesia practice.
Our survey revealed that almost all anesthesia providers in our institution support the idea of using checklists in certain aspects of anesthesia care. Acceptance rates are highest for emergency situations and for non-routine procedures, and the majority of providers support the use of checklists during patient and shift hand-offs. The necessity of using checklists for non-emergent routine anesthesia care as airline pilots have done for decades in various stages of routine flights still remains only the belief of a minority. However, our survey clearly documents a growing understanding of human failure rates for routine anesthesia care, and the increasing acceptance of the idea that cognitive aids might be beneficial.
Our results also emphasize the importance of high-quality design and thoughtful integration into the clinical workflow as key components for the successful implementation of checklists into routine anesthesia care. We believe that our results will encourage opinion leaders to develop and implement checklists for anesthesia care within their institutions. Links to examples of existing anesthesia checklists are listed in addendum 1.
