Entrustable Professional Activities (EPAs) are important in competency-based medical education, offering formative assessment for trainee development. However, some EPAs, including resuscitation of the critically ill or injured pediatric patient, are clinically rare, limiting trainee experience.
Most programs have started to use in-person simulations as a supplement to clinical exposure and complete EPA assessments; however, in-person simulations are costly and resource-intensive. Education programs have thus looked for alternatives to support trainees in demonstrating competence. But could we use emerging technologies such as Immersive Virtual Reality (IVR) to supplement trainee education?
IVR for Formative Assessment
IVR is a type of simulation that uses virtual spaces and characters rather than physical and human resources, mediated by a head-mounted device. It offers flexible scheduling, scenario repetition and automated assessments. However, IVR’s role in postgraduate EPA assessment remains unexplored.
Although the number of research articles related to IVR in medical education has increased over the years, implementation in Canadian training programs, especially for assessment purposes, is scarce. Nonetheless, there is an increased number of IVR companies that offer medical scenarios and tailoring options; hence, programs are exploring partnerships with different companies to support residency training. Additionally, the overall cost of IVR-based simulation training is calculated as lower than in-person (e.g., at the simulation centre) and physical simulations (e.g., using manikins). Hence, programs are considering the cost of purchasing the hardware and software, and any technical support and scenario modifications that might be needed.
To understand important considerations when selecting IVR scenarios for formative assessment in residency training, we reviewed the literature and gathered faculty/fellow feedback on several IVR scenarios containing both technical and non-technical skills. Their experiences and feedback were explored qualitatively.
Some IVR scenarios we explored included supraventricular tachycardia, asthma, neonatal resuscitation, anaphylaxis and septic shock. The thematic analysis highlights potential benefits of IVR for supporting competence gain and completing assessments, with specific considerations that medical educators should undertake.
Trainee level and targeted skills
IVR scenarios follow an algorithmic decision process, aligning well in acute care settings where decisions are typically guided by established protocols. Although this limits flexibility to navigate the scenario (e.g., scenario having preselected set of medications, patient requiring sedative and paralytic medication before intubation), the casting and multi-user capability allows for modifications to teach and assess different skills.
Additionally, IVR has tactile and visual limitations specifically pertaining to psychomotor skills (e.g., performing intubation). Depending on the scenario, IVR can be a valuable tool for supporting step-by-step skill execution and decision-making, individually or collaboratively, particularly for trainees in the early stages of learning.
Automated assessment through IVR
Upon completion, trainees receive automated assessments based on preset criteria to encourage self-debriefing. However, there are many situations in which flexibility in approach exists despite the use of algorithms, impacting the usefulness of the preset assessment criteria. Hence, medical educators should review the criteria to determine relevance and modify the assessment as needed, which can be done in collaboration with the IVR company.
Orientation to the IVR environment
Orientation to the IVR environment, including equipment (e.g., using the handheld devices) and scenario-specific orientation (e.g., familiarity with the patient room), is crucial to reduce cognitive load. For example, in the anaphylaxis scenario, participants needed to understand how to use or prepare the virtual devices and medications to effectively manage and treat patient symptoms, in addition to understanding how the skin rash of the patient would change in the virtual environment once medication is administered. Hence, medical educators should familiarize themselves with the virtual environment and the technology to feel comfortable and support learners.
Objectives of the IVR session
Clearly stating the assessment objectives ensure alignment of purpose and transparency to the trainee. For example, in the anaphylaxis scenario, is the objective to help trainees learn the steps of the physical exam or communicate with the caregiver, or a combination of skillsets?
Overall, IVR is an educationally beneficial adjunct to clinical and in-person simulation settings, especially for rare clinical scenarios. However, the intended use of IVR should align with the trainees’ needs and program’s objectives, carefully considering the technology’s benefits and limitations.
Acknowledgements: We would like to thank Dr. Spencer van Mil for supporting scenario exploration. We would also like to acknowledge the McMaster Education in Anatomy Program and thank them for providing us with access to the IVR hardware as well as technical support.
Conflicts of Interest: This project is supported by funding from the Social Sciences and Humanities Research Council. Specifically, we received funding through the Partnership Engage Grant and partnered with MedVR Education.
