Each year there are patients who wake up from surgery to find an operation has been done on the wrong part of their body. These wrong-site surgeries are an example of “never events” – incidents that simply should not happen if all safety measures are taken. Nevertheless, these events take place each year in countries with the highest standards of care, from the United States to England, and Canada is no exception.
Even simple mistakes in the operating room can have profound implications on a patient’s outcomes, and as such there is a constant pursuit of ways to reduce errors. One of the most heralded innovations in recent years has been the World Health Organization’s (WHO) development of a 19-item checklist for surgical procedures.
This innovation has proven extremely popular. Several Canadian provinces, including Ontario and Alberta, have adopted policies designed to make the use of safe surgery checklists a standard practice. Yet the use of checklists remains controversial for some surgical teams, who point to recent evidence suggesting that mandatory checklists did not improve patient safety in Ontario.
The safe surgery checklist
The WHO’s checklist prompts the surgical team to communicate critical information at three time points: before the administration of anesthesia (briefing), before the first incision (timeout), and before the patient is removed from the operating room (debriefing). These include safety checks such as making sure that all the necessary equipment is in place before starting, or ensuring that all members of the team have discussed the patient’s unique needs including allergies and anticipated blood loss.
The aim of checklists is not to challenge the proficiency of a surgical team, but rather to provide a way of improving the reliability of a complex multi-step procedure. The checklist approach was based on the observation that repetitive procedures – from landing an airplane safely to performing a hernia repair – should not be left to the memory of any one person or team to execute perfectly every time. However, the goal of the checklist is not simply “ticking off boxes” but more so about ensuring attention to details that can get missed during a busy day and improving communication among the members of the surgical team, explains Bryce Taylor, the former surgeon-in-chief at the University Health Network.
Evidence for checklists
The effectiveness of the checklist was first evaluated by an international study published in 2009, which analyzed the rates of death and post-operative complications before and after the introduction of the checklist at 8 hospitals across the world. The results were impressive: the proportion of patients experiencing complications declined by more than a third from 11% to 7%, while the percentage of patients who died in the first 30 days after surgery was almost cut in half, from 1.5% to 0.8%. When the authors looked only at the hospitals in high-income countries (Toronto, Seattle, London, Auckland), although the reduction in mortality was no longer statistically significant (0.9% to 0.6%), a significant drop in complications remained (10.3% to 7.1%).
The benefits of the checklist were widely reported in the media, and championed by one of the study’s lead investigators, Harvard’s Atul Gawande, who promoted its use through editorial columns, a popular TED talk, and his book The Checklist Manifesto. In addition, subsequent studies in the Netherlands and the United States seemed to support the checklist’s positive impact.
Checklists in Ontario
Based on these encouraging findings, many health authorities across the world began to require their surgical teams to adopt the checklist as a standard practice. In July 2010, the Ontario Ministry of Health and Long-Term Care instituted a policy requiring public reporting of adherence to safe surgery checklists in hospitals across the province, essentially mandating the use of the checklist. It was hoped that by encouraging its use, at least some of the touted benefits would be seen in the province.
However, although the self-reported compliance to the checklists was high (92-100%), a recent study led by David Urbach, a surgeon at Toronto’s University Health Network, found that the implementation of the mandatory policy appeared to have no significant impact on the rates of surgical complications or death. This finding was disappointing to many who believed the policy would improve outcomes in Ontario.
But, Urbach says he was not particularly surprised by the lack of clinical benefits observed in the study. Given the high standard of care already present in the province’s operating rooms, he was not sure it was realistic to expect the large improvement in patient outcomes that had been reported in previous studies.
The study proved controversial as a number of physicians, including Gawande, took issue with the findings, questioning some components of the study design. Their concerns primarily surrounded the time frame analyzed, the low proportion of high-risk surgeries in the sample and the reliability of the data source in measuring complication rates.
There are others who suggest that the study’s results are not necessarily a reflection of the checklists themselves, but instead of Ontario’s implementation of the policy. There is some evidence to suggest that the top-down approach in which the checklists were introduced by the Ministry was ineffective in that although the self-reported compliance was high, the checklist itself was often not being used as it was intended, if at all.
Critics argue that by not ensuring buy-in from surgical teams before implementation, there is a strong likelihood that the high rates of self-reported compliance are misleading and not accompanied by the true behaviour change necessary to make them effective. As Avery Nathens, the chief of surgery at Sunnybrook Health Sciences Centre, explains, “whether the surgeon-in-chief believed it valuable or not, they had to go ahead and do it, which is probably not the optimal approach”.
An important factor in achieving this buy-in is having a respected person act as a champion of the safety measure at the hospital level. “You need someone who is really interested in the initiative and has the relationships to implement it properly” says Nathens. He notes that it doesn’t necessarily have to be someone in leadership, but when you have someone who is a true believer in the measure, “it’s almost contagious”.
The Alberta experience
Alberta Health Services (AHS) has also adopted the use of safe surgery checklists across its hospital operating rooms. However, the implementation of the policy was notably different from Ontario’s in that it was developed through the Strategic Clinical Network (SCN) for surgery. Under the auspices of AHS, the Surgery SCN brings together surgeons from across the province under the leadership of other surgeons.
For John Kortbeek, the co-chair of the network, safe surgery checklists represented an attempt to address a troubling trend of never-events in the Calgary health zone. In his first year as the head of surgery in the Calgary zone, there were four wrong-site surgeries, an observation he initially assumed was just an anomaly. However, when there were another four events in both the second and third year, he realized that “it was no longer the exception anymore but instead a baseline fundamental issue”.
Despite self-reported adoption being near 100% soon after the introduction of the checklists, an observational audit of compliance revealed that the checklist had only been appropriately completed for 41% of procedures. Based on this concerning finding, AHS continued the audits monthly, collecting the information and returning reports to each hospital. By providing quantitative measurements and feedback on their progress, each site could evaluate their status and identify barriers to improvement, explains Tracy Wasylak, senior program officer of the SCNs.
A particularly innovative component of the audit was the inclusion of “good catches”; times where the checklist served as a reminder of something that had been missed, such as a medication allergy. It was felt that by identifying these occurrences, they could act as an important tool to convince those who were skeptical of the checklist’s benefits. These are events that would not normally be recorded in a systematic manner, so quantifying them served as a way to capture this often unmeasured benefit. To date, these strategies seem to have worked – as of 2014, the audited compliance rate was greater than 90%.
In addition to the audits, much of the success in checklist adoption has been attributed to engaging the surgical teams, from the nurses to the surgeons themselves. By ensuring buy-in from the entire surgical staff, Kortbeek felt that it was easier to change the culture, remarking that “you cannot just dictate policy and expect people to line up”.
Nevertheless, even with this engagement there will always be resistance to changes in policy. For some members of the surgical team who have been practising in a particular way for their entire careers, asking them to change their methods can be difficult.
Others remain concerned about the added time. However, an evaluation at a Calgary hospital demonstrated that the introduction of checklists did not significantly increase the length of surgeries. Furthermore, Doug Hedden, the other co-chair of the surgery SCN, noted that while there were “concerns about making the process more inefficient… there’s nothing more inefficient than having a significant complication”.
So while there was greater confidence in Alberta that the checklists were actually being appropriately completed, has there been any effect on patient outcomes?
At this point, there are no published reports on quantitatively improved outcomes, so it is challenging to make any definitive statements. Nevertheless, the anecdotal evidence accrued thus far appears to point in a positive direction. In addition to the many “good catches” recorded during audits, since the introduction of the checklist in the Calgary zone in 2011, Kortbeek remarked that there hasn’t been a single wrong-site surgery.
Beyond patient outcomes
While there may be some that are skeptical of the clinical benefits, almost all agree that, when implemented properly, the checklists have improved other important factors such as the culture in the operating room.
A survey conducted at the University Health Network revealed that members of the surgical teams generally felt that the checklist improved communication and fostered a greater culture of teamwork. In addition, by ensuring that the entire surgical team is present before the anesthesia is administered, the patients themselves are reassured and feel more involved in their own care.
In the end, by all accounts, the safe surgery checklist is here to stay. “The fact is many people feel it has value, it costs nothing and it’s risk free” explains Nathens. Like many others, he believes surgeons should “carry on doing it and actually do it better” where possible.