Each year there are patients who wake up from surgery to find an operation has been done on the wrong part of their body. These wrong-site surgeries are an example of “never events” – incidents that simply should not happen if all safety measures are taken. Nevertheless, these events take place each year in countries with the highest standards of care, from the United States to England, and Canada is no exception.
Even simple mistakes in the operating room can have profound implications on a patient’s outcomes, and as such there is a constant pursuit of ways to reduce errors. One of the most heralded innovations in recent years has been the World Health Organization’s (WHO) development of a 19-item checklist for surgical procedures.
This innovation has proven extremely popular. Several Canadian provinces, including Ontario and Alberta, have adopted policies designed to make the use of safe surgery checklists a standard practice. Yet the use of checklists remains controversial for some surgical teams, who point to recent evidence suggesting that mandatory checklists did not improve patient safety in Ontario.
The safe surgery checklist
The WHO’s checklist prompts the surgical team to communicate critical information at three time points: before the administration of anesthesia (briefing), before the first incision (timeout), and before the patient is removed from the operating room (debriefing). These include safety checks such as making sure that all the necessary equipment is in place before starting, or ensuring that all members of the team have discussed the patient’s unique needs including allergies and anticipated blood loss.
The aim of checklists is not to challenge the proficiency of a surgical team, but rather to provide a way of improving the reliability of a complex multi-step procedure. The checklist approach was based on the observation that repetitive procedures – from landing an airplane safely to performing a hernia repair – should not be left to the memory of any one person or team to execute perfectly every time. However, the goal of the checklist is not simply “ticking off boxes” but more so about ensuring attention to details that can get missed during a busy day and improving communication among the members of the surgical team, explains Bryce Taylor, the former surgeon-in-chief at the University Health Network.
Evidence for checklists
The effectiveness of the checklist was first evaluated by an international study published in 2009, which analyzed the rates of death and post-operative complications before and after the introduction of the checklist at 8 hospitals across the world. The results were impressive: the proportion of patients experiencing complications declined by more than a third from 11% to 7%, while the percentage of patients who died in the first 30 days after surgery was almost cut in half, from 1.5% to 0.8%. When the authors looked only at the hospitals in high-income countries (Toronto, Seattle, London, Auckland), although the reduction in mortality was no longer statistically significant (0.9% to 0.6%), a significant drop in complications remained (10.3% to 7.1%).
The benefits of the checklist were widely reported in the media, and championed by one of the study’s lead investigators, Harvard’s Atul Gawande, who promoted its use through editorial columns, a popular TED talk, and his book The Checklist Manifesto. In addition, subsequent studies in the Netherlands and the United States seemed to support the checklist’s positive impact.
Checklists in Ontario
Based on these encouraging findings, many health authorities across the world began to require their surgical teams to adopt the checklist as a standard practice. In July 2010, the Ontario Ministry of Health and Long-Term Care instituted a policy requiring public reporting of adherence to safe surgery checklists in hospitals across the province, essentially mandating the use of the checklist. It was hoped that by encouraging its use, at least some of the touted benefits would be seen in the province.
However, although the self-reported compliance to the checklists was high (92-100%), a recent study led by David Urbach, a surgeon at Toronto’s University Health Network, found that the implementation of the mandatory policy appeared to have no significant impact on the rates of surgical complications or death. This finding was disappointing to many who believed the policy would improve outcomes in Ontario.
But, Urbach says he was not particularly surprised by the lack of clinical benefits observed in the study. Given the high standard of care already present in the province’s operating rooms, he was not sure it was realistic to expect the large improvement in patient outcomes that had been reported in previous studies.
The study proved controversial as a number of physicians, including Gawande, took issue with the findings, questioning some components of the study design. Their concerns primarily surrounded the time frame analyzed, the low proportion of high-risk surgeries in the sample and the reliability of the data source in measuring complication rates.
There are others who suggest that the study’s results are not necessarily a reflection of the checklists themselves, but instead of Ontario’s implementation of the policy. There is some evidence to suggest that the top-down approach in which the checklists were introduced by the Ministry was ineffective in that although the self-reported compliance was high, the checklist itself was often not being used as it was intended, if at all.
Critics argue that by not ensuring buy-in from surgical teams before implementation, there is a strong likelihood that the high rates of self-reported compliance are misleading and not accompanied by the true behaviour change necessary to make them effective. As Avery Nathens, the chief of surgery at Sunnybrook Health Sciences Centre, explains, “whether the surgeon-in-chief believed it valuable or not, they had to go ahead and do it, which is probably not the optimal approach”.
An important factor in achieving this buy-in is having a respected person act as a champion of the safety measure at the hospital level. “You need someone who is really interested in the initiative and has the relationships to implement it properly” says Nathens. He notes that it doesn’t necessarily have to be someone in leadership, but when you have someone who is a true believer in the measure, “it’s almost contagious”.
The Alberta experience
Alberta Health Services (AHS) has also adopted the use of safe surgery checklists across its hospital operating rooms. However, the implementation of the policy was notably different from Ontario’s in that it was developed through the Strategic Clinical Network (SCN) for surgery. Under the auspices of AHS, the Surgery SCN brings together surgeons from across the province under the leadership of other surgeons.
For John Kortbeek, the co-chair of the network, safe surgery checklists represented an attempt to address a troubling trend of never-events in the Calgary health zone. In his first year as the head of surgery in the Calgary zone, there were four wrong-site surgeries, an observation he initially assumed was just an anomaly. However, when there were another four events in both the second and third year, he realized that “it was no longer the exception anymore but instead a baseline fundamental issue”.
Despite self-reported adoption being near 100% soon after the introduction of the checklists, an observational audit of compliance revealed that the checklist had only been appropriately completed for 41% of procedures. Based on this concerning finding, AHS continued the audits monthly, collecting the information and returning reports to each hospital. By providing quantitative measurements and feedback on their progress, each site could evaluate their status and identify barriers to improvement, explains Tracy Wasylak, senior program officer of the SCNs.
A particularly innovative component of the audit was the inclusion of “good catches”; times where the checklist served as a reminder of something that had been missed, such as a medication allergy. It was felt that by identifying these occurrences, they could act as an important tool to convince those who were skeptical of the checklist’s benefits. These are events that would not normally be recorded in a systematic manner, so quantifying them served as a way to capture this often unmeasured benefit. To date, these strategies seem to have worked – as of 2014, the audited compliance rate was greater than 90%.
In addition to the audits, much of the success in checklist adoption has been attributed to engaging the surgical teams, from the nurses to the surgeons themselves. By ensuring buy-in from the entire surgical staff, Kortbeek felt that it was easier to change the culture, remarking that “you cannot just dictate policy and expect people to line up”.
Nevertheless, even with this engagement there will always be resistance to changes in policy. For some members of the surgical team who have been practising in a particular way for their entire careers, asking them to change their methods can be difficult.
Others remain concerned about the added time. However, an evaluation at a Calgary hospital demonstrated that the introduction of checklists did not significantly increase the length of surgeries. Furthermore, Doug Hedden, the other co-chair of the surgery SCN, noted that while there were “concerns about making the process more inefficient… there’s nothing more inefficient than having a significant complication”.
So while there was greater confidence in Alberta that the checklists were actually being appropriately completed, has there been any effect on patient outcomes?
At this point, there are no published reports on quantitatively improved outcomes, so it is challenging to make any definitive statements. Nevertheless, the anecdotal evidence accrued thus far appears to point in a positive direction. In addition to the many “good catches” recorded during audits, since the introduction of the checklist in the Calgary zone in 2011, Kortbeek remarked that there hasn’t been a single wrong-site surgery.
Beyond patient outcomes
While there may be some that are skeptical of the clinical benefits, almost all agree that, when implemented properly, the checklists have improved other important factors such as the culture in the operating room.
A survey conducted at the University Health Network revealed that members of the surgical teams generally felt that the checklist improved communication and fostered a greater culture of teamwork. In addition, by ensuring that the entire surgical team is present before the anesthesia is administered, the patients themselves are reassured and feel more involved in their own care.
In the end, by all accounts, the safe surgery checklist is here to stay. “The fact is many people feel it has value, it costs nothing and it’s risk free” explains Nathens. Like many others, he believes surgeons should “carry on doing it and actually do it better” where possible.
The comments section is closed.
I work in an operating room and I would argue that the Safe Surgical Checklist offers a false sense of security. I have reviewed the NEJM of article for Ontario and the editorial and comments after. It’s amazing to me how badly people want to deny the evidence that it doesn’t work. The editor of the NEJM used advanced statistical terms to basically say ” look harder, look longer, it probably does have benefit”. Just admit that 11% complication rate quoted in the WHO article is ridiculous. In the original 2009 WHO article the post checklist population had an increase in mortality from massive hemorrhage by a factor of six. SO, if you believe the reduction in overall mortality was a result of the checklist, you have to conclude the increase deaths from bleeding were also a result of the checklist intervention. No one believes that, because that would be silly. Next people on here are saying ” well it wasn’t applied right.. people just need to believe”. Well Peter Pan and Tinkerbell with there pixie dust don’t work in my OR, the delusion that just believing equals evidence based medicine left with Captain Hook. Good intentions don’t equal good results.
The safe surgery checklist was introduced as a compulsory element of our Qmentum accreditation program in 2011, consistent with evidence that demonstrated its use reduced the likelihood of complications following surgery, and may improve surgical outcomes. The checklist represents another tool health care organizations use to provide safe and quality surgical care to patients. UHN’s experience of the use of the checklist to improve communication and create a culture of teamwork is a positive example of what the safe surgery checklist can encourage beyond the achievement of a clinical benefit.
Wendy Nicklin, President and CEO, Accreditation Canada
What evidence are you are you referring too, because the Ontario NEJM article refutes that, as well as the world J Surg 2016 Aug; 40 (8)
I am not a statistician so cannot judge the technical merits or otherwise of the Ontario study of surgical outcomes following mandatory adoption of surgical safety checklists. But I have consulted statisticians on numerous occasions in my professional career and it appears to me that the results obtained, while perhaps not sufficiently STATISTICALLY significant to be conclusive, do suggest there MAY have been some improvement (particularly almost 10% in the “.. adjusted risk of death ..”). In such circumstances, surely, more rigorous analysis is called for – not a conclusion that “.. (it) was not associated with significant reductions in operative mortality or complications.”
Nor do I think that St Mary’s Hospital in London, UK or the University of Washington Medical Center in Seattle and, indeed, the Toronto General Hospital (probably among the finest in the World), who participated in the WHO study and showed significant benefits arising from the adoption of surgical safety checklists, would care for the implication that, prior to this, their standards of care were inferior to that in Ontario’s operating rooms!
However, Atul Gawande rejected the idea of IMPOSING checklists in the WHO study, preferring willing cooperation to ensure the behavioural changes required to succeed. He also noted that there was considerable scepticism at the outset, some of which persisted, but that after three months fully 93% of the operating room STAFF involved would want the checklist used if they were having an operation. What’s good enough for them is good enough for me – and SHOULD be for all Canadian patients!
Well said. I am a Gynecologist and Champion the Surgical check list. However, we should have modern methods of collection, plus emulate the Cockpit environment, with camera etc. I am presenting in Florida this week, at the SLS meeting , my presentation Surgical Check list help or hindrance. I would be happy to send you a copy. Nicholas PAiraudeau
I should be very grateful to receive a copy of your presentation. Thank you very much. I think you should be able to obtain my email address through Healthy Debate (I would prefer not to “publicise” it on line. I am familiar with the basic concept of CRS – Crew or Cockpit Resource Management – developed by NASA and adopted by airlines Worldwide. I think surgical teams could possibly/probably benefit from such familiarisation/training and I imagine this is your theme.)
I think checklists for medicine are an effective way to make sure that quality care is delivered. It has made a difference in primary care when we use check lists or templates in our EMR for chronic disease management. Whenever we have 8 tasks to remember in a care pathway we can use check lists. I think we need to encourage standardization in all areas of medicine.
Physicians are not employees and feel they are not accountable to the MOH ….desire higher autonomy! Pilots and engineers are employees and if not meeting accountability standards can be fired……..we will not get quality improvement in medicine until we have accountability.
Having facilitated change in many organizations, engaging those people involved in making the change is the “most critical ingredient” in adopting change and having successful outcomes.
The analogy of a pilot/co-pilot using check lists before a flight and monitoring systems throughout until landing says it all. And there are the only 2 people in the cockpit.
When followed as designed, and audited effectively, checklists WILL avoid mistakes – i.e. poor or, fatal patient outcomes. Until humans are created “perfect”, incapable of “any” mistakes, checklists are invaluable. The time/efficiency element is a non-starter; once people buy-in.
However, failing to explore/discuss the pros/cons with the people expected to implement the checklist was a MISTAKE – People need to be engaged, not mandated. Maybe management/the health system needs a checklist on “how to engage” people for success!
Agree that buy-in by clinicians/leaders is essential to validate checklists produce their intended results. But also agree, can’t measure for intended results without measuring compliance. As noted above, design of compliance audits is a tricky business (complex and comprehensive – not just a count of completed checklists).
But what about airline and nuclear…?
Did they encounter influential nay-sayers?
Did they spend this much time/effort (and $$$’s) implementing? and
From an outsider’s pov, non-compliance does not appear to be an option in these industries. and
When there is an incident, it seems that checklists are referred to immediately in the post analysis (and often made public).
Why not healthcare?