Do high-fidelity, augmented reality simulations improve students’ engagement with simulation sessions?

Due to the small number of students taking part in the study (six), a full statistical analysis of the results was deemed inappropriate. The students in the augmented scenario reported that they felt the simulation felt more realistic. However, in some measures of engagement the non-augmented scenario received higher scores. Many augmentedreality features, especially the video-handover, received positive feedback and were viewed as successful in improving realism and engagement.


Introduction
Simulation is the artificial replication of real-life scenarios, of great utility in fields where consequences of failure Thompson O, Harris R, Godfrey T MedEdPublish https://doi.org/10.15694/mep.2017.000208 Page | 2 can be dangerous, such as medicine. Simulation allows students to train without putting themselves or patients at risk of harm and allows exposure to rare cases that are difficult to come by in the clinical environment (Lateef 2010).
Simulation is also a means to provide training in a multidisciplinary team setting and to develop team-working skills, such as communication, coordination and efficiency (Lateef 2010). These attributes are difficult to teach in a didactic setting, yet are increasingly required as part of medical courses. Indeed, team-working and communication skills comprise a large portion of the outcomes set out by the GMC (2016) in their standards of medical education. Simulation has been shown to increase confidence of medical staff in making judgement decisions in complex medical cases -vitally important in high-pressure scenarios such as pre-hospital and emergency medicine (Garside et al. 2012).

Augmented reality
Simulation mannequins are fantastic teaching aides and can emulate a whole series of clinical signs, such as pulse, breath sounds and pupil constriction/dilatation. However, they are unable to reproduce other key signs such as limb weakness, vomiting or speech disturbance (Garside et al. 2012). In addition, simulation mannequins do not appear convincingly human. Their uncanny appearance reduces empathetic connection and decreases the realism (or fidelity) of the scenario.
A team in Northumbria sought to overcome these failings by concurrently playing videos of patients displaying clinical signs relevant to the case. Where questions were asked, pre-recorded responses from the human patient could be selected by the simulation operator. The project received overwhelmingly positive feedback from learners (Garside et al. 2012). Similar technology has been employed at Sheffield Hallam University, where video-based augmented reality is used to aid in the training of nursing students (Sheffield Hallam University Media Centre 2012).

Emotions and learning
The effect of stress on learning has been long discussed. Certainly, anecdotal evidence from clinicians suggests that particularly stressful situations are ones that are remembered for life. However, review of the literature is confusing, with stress appearing to both inhibit and enhance memory. This comes down to differences between working (shortterm) memory and memory consolidation (laying down long-term memory) (LeBlanc 2009).
When it comes to learning in the context of medical education, it is memory consolidation that is most important. Whilst stress has been demonstrated to inhibit working memory, it seems to have a beneficial effect on memory consolidation. However, this comes with a caveat. In order to boost memory consolidation, the stress must be caused by the situation itself and not external antagonising factors (LeBlanc 2009). Put in the context of simulation: if the stress is caused by the intensity and emotional impact of the scenario, memory consolidation will be enhanced. Conversely, if the stress is external, for example due to anxiety at being observed by colleagues, then the beneficial effect on memory is not seen.

Olfaction
Of all the senses, smell has been most consistently linked with memory and emotion. Olfactory stimulation with an odour such as sun-cream can stimulate retrieval of rich memories of holidays on the beach, in a manner far superior to images or sounds (Gottfried et al. 2004). Memories of smells also appear to persist for far longer than those for other senses. Work in the 70's demonstrated that, whilst memories of images deteriorated after hours and days, those for smells lasted for up to a year and beyond (Engen and Ross 1973). The presence of the olfactory neural pathway within the limbic system of the brain (which contains the emotioncontrolling amygdala and the memory-storing hippocampus) may well explain this phenomenon. Work with fMRI has shown that olfactory stimulation causes increased activity in both the piriform (olfactory) cortex and the hippocampus (Gottfried et al. 2004).

Method
We aimed to produce a simulation session that incorporated augmented reality and high-fidelity components, stimulating as many of the learners' senses as possible, especially their sense of smell. An appropriate scenario was devised and the simulation then run twice; with and without augmented reality adjuncts. Three students participated in each scenario and their engagement with the scenarios was measured.

The Non-Augmented Session
The Scenario The scenario devised was set in the pre-hospital environment. It involved two patients, the (drunk) driver of a car and the pedestrian into whom he had crashed. Both roles were played by actors. The pedestrian displayed a heavily bleeding leg wound (which would require a tourniquet to control) and would begin to display signs of hypovolaemic shock. The driver had an evident head laceration, yet would begin the scenario relatively lucid. Throughout the scenario he would gradually lose consciousness and display signs of raised intracranial pressure. This would test the learners' ability to adapt their management plan.

Equipment
The students carried a backpack containing standard equipment for a paramedic team, with a modified guedel airway and bag valve mask. The guedel had been shortened to comfortably sit in an actor's mouth without them gagging. The bag valve mask ( Figure 1) had several holes drilled into it, so that an actor could be "ventilated" simply by timing voluntary respiration with the students squeezing the bag.
Participating in the scenario with the students was a clinical assistant, who had been briefed in the scenario. They carried with them a tablet installed with the SimMon medical simulation app, allowing them to display observations for the patients (Castle+Anderson 2013). The clinical assistant was also able to facilitate the scenario, for instance by inserting the modified airway.

Moulage
Make-up incorporating liquid latex and fake blood was applied to the actors to simulate the head and leg injuries. A bag of fake blood was attached via a giving set to the pedestrian's leg wound, so that an arterial bleed could be imitated by squeezing the bag. The pedestrian also held a squash ball in his armpit, which could be clenched when the students felt for the radial pulse. This occluded the axillary artery and as such prevented a radial pulse from being felt. Fans of the BBC drama "Sherlock" may well recognise this technique.
The driver was to vomit as the students arrived. This was simulated by mixing Weetabix with water, which could be held in the actor's mouth until necessary, to produce a horribly realistic jet of vomit. In addition, a unilateral blown pupil was created by application of tropicamide eye drops. The students were equipped with radios. They could use these to communicate with one of the scenario facilitators playing the role of Control HQ or a paramedic team en route to the location.

Debrief
After the scenario, the learners were debriefed as a team by a simulation fellow.

The Augmented Session
A number of adaptions were made to the augmented scenario so as to improve its realism. These are organised by the sense they each stimulated:

Vision
As the students arrived at the scene they were met by simulation operator acting as a police officer. He had cordoned off the scene with police tape, and had a blue flashing light on top of his car -an "unmarked police car".
The driver's car was full of empty bottles and cans of alcoholic beverages, offering a visual clue as to the circumstances of the crash.

Hearing
For the augmented scenario there was the additional complication of the car engine still running as the students arrived on scene. As such, the students were met with engine noise and a blaring radio, similar to the state of affairs in many real-life incidents. A Bluetooth speaker concealed behind the "police car" was used to broadcast sounds of a busy road including cars, pedestrians and sirens.
When calling in for an air ambulance, the simulation facilitator on the other end of the radio played helicopter sounds from his phone down the radio, producing the incredibly realistic effect that he was communicating from inside a helicopter.

Olfaction
The predominant olfactory stimulation came from the scent of alcohol on the driver. This was achieved simply by the liberal application of whiskey to the drivers clothing and to the weetabix used to simulate sick.
Initially the plan had been to project the smell of burning rubber. Unfortunately, no method was devised to produce this safely.

Miscellaneous
The students were shown an initial briefing video from a paramedic inside an ambulance, explaining the scenario to be faced and their objectives within the scenario.
A cannulatable patch was produced from tubing, latex and a metal backing plate. This, once attached by tape and liquid latex to the pedestrian's arm, enabled the students to cannulate the patient without requiring a rubber cannulatable arm protruding from the actor's shirt. (Figure 2

Measuring Engagement
Engagement cannot be measured until it is has been defined and there are two principal definitions in existence. The first of these implies contribution and participation within a programme in a manner analogous to medical compliance, evidenced by completing work on time or attending all classes (Chapman 2003). The second defines engagement as a more abstract attribute, based on the quality and intensity of a learner's involvement with the teaching. Signs that demonstrate engagement within this model are more emotional, including enthusiasm, curiosity and interest (Skinner 1993).
For the purposes of this study, the latter definition was preferred. Emotional investment and affective response within the simulation were deemed important. A self-reported questionnaire, drawing on aspects of engagement and emotional involvement, was utilised. It was felt that self-reporting was important, as engagement and immersion are primarily subjective sensations.
Given the similarity between video games and medical simulation, it was perhaps unsurprising that inspiration for this self-reported engagement questionnaire came from the world of gaming. The "Game Engagement Questionnaire" was designed to determine player immersion and engagement with games. It included self-reported questions such as losing track of time, or feeling as though the gaming environment was responsive to the player's Thompson O, Harris R, Godfrey T MedEdPublish https://doi.org/10.15694/mep.2017.000208 Page | 7 actions (Brockmyer and Fox 2013).
To objectively measure emotional investment, the students' heart rate was measured before and after the simulation session, as a surrogate measure for sympathetic activity -itself a surrogate of emotional investment. In addition, questions were added to the questionnaire that were similar to those used in the State-Trait Anxiety Inventory (STAI), a validated tool for assessing anxiety (Spielberger 1977).
The feedback questionnaire consisted of a free-text box and 10 questions utilising a 10-point Likert scale. These included questions measuring immersion (eg. "I lost track of time during the scenario") and emotional engagement (eg. "I felt calm during the scenario"). The questionnaire also measured confidence in handling the scenario before and after the session.
As this project formed part of a service evaluation, no ethical approval was required.

Running the scenario
The scenarios operated almost entirely as intended. In the non-augmented scenario there was a degree of confusion from the students as to whether they should be asking for the results of examination (as had been the case in previous simulation sessions) or reporting what they actually found. This issue was corrected in the subsequent session by clarifying the instructions in the pre-simulation briefing.
In both cases the students identified the medical problems, appropriately managed the patients and organised extraction. Debrief was excellent, with the students actively contributing and identifying strengths and areas for improvement.

Feedback
Unfortunately, a very small number of students took part in this service evaluation (n=6) and so a full statistical analysis was unlikely to produce any meaningful data. As such, all findings discussed below are done so with the stipulation that the results are non-significant.
The data collected shows very little difference between the two groups, partly due to the non-augmented scenario receiving far more positive feedback than expected.
The augmented scenario produced slightly higher levels of realism ("The scenario felt more realistic") and selfreported engagement ("I felt engaged in the scenario"). Students taking part in this scenario also reported that they were less easily distracted, felt the environment was more responsive to their actions, and reported a higher level of enjoyment of the scenario. They displayed a greater increase in confidence in managing the clinical situation before and after the session.
However, the students taking part in the non-augmented scenario appeared more immersed, feeling greater emotional attachment to their patients and losing track of time to a greater extent than those in the augmented group. They also self-reported more anxiety, which was reflected in a greater increase in heart rate during the scenario (the average increase was 61.33bpm compared to 40bpm in the augmented session). Thompson O, Harris R, Godfrey T MedEdPublish https://doi.org/10.15694/mep.2017.000208 Page | 8 The feedback from the free text box was very positive. The realism of the augmented scenario was praised, especially the handover from the paramedic and the use of radio communications. The moulage was also singled out as particularly realistic, with the wounds and vomit appreciated. The only negative feedback received was that the groups would have preferred to have had more training in the correct use of radio before the session started.

Discussion
In many ways, the production of this simulation scenario can be considered a success. Feedback was overwhelmingly positive and the scenarios ran largely as planned. Most markers suggested increased engagement within the augmented session. However, self-reported immersion was lower in this group when compared to the group taking part in the scenario without augmentation.
Some of the uncertainty could come down to differences between students in the two groups. There was variation between the groups in terms of prior exposure to both simulation training and clinical experience. The group taking part in the non-augmented session did so after two days of prior simulation training. This gave them a different reference point to the augmented group, who had received no prior training. This may have been corrected by selecting groups with more similar levels of experience.
Future work could make use of more accurate measurements of stress response. Previous studies have made use of salivary cortisol measurements to gain a more robust insight into the stress elicited by a scenario (LeBlanc 2009). In addition, continuous heart-rate monitoring would be useful, for gaining a complete idea of how heart-rate fluctuates throughout the scenario.
The major problem with this piece of work is that the measured outcomes are merely surrogates for what is really important -memory retention. Producing a scenario that demonstrably increases emotional response, engagement and immersion is of dubious benefit if memory retention is not improved. Future work should involve pre-and postscenario testing of knowledge and skills. Testing these at a much later date (perhaps months later) could truly assess if the simulation augmentation is having its desired effect.
One concern levelled at simulation training in the past has been cost. The installation cost of a fully-equipped simulation suite can be as high as $200,000 (Al-Elq 2010). Operating costs are also high, requiring a trained technician in addition to the clinical staff delivering the teaching.
The simulation session described here is a cheaper alternative. Whilst it was relatively resource-intensive in terms of the time given by clinical staff and those involved in the organisation, not using a SimMan mannequin allows large savings to be made. The scenario required students to play the roles of patients. However, given that the GMC encourages students and medical professionals to gain teaching experience, (GMC 2016) there would likely be no shortage of volunteers from the student body. As such, the only costs of delivering teaching in the manner described are those for the clinical staff who deliver the teaching and a small amount on make-up for moulage. This makes simulation in this manner a relatively low-cost alternative to traditional simulation.

Conclusion
Simulation-based teaching is growing in popularity in medical education due to its ability to allow students to practice without the risk of harm. However, scenarios often feel unrealistic due to the limitations of simulation Thompson O, Harris R, Godfrey T MedEdPublish https://doi.org/10.15694/mep.2017.000208 Page | 9 mannequins. There is a large body of evidence available suggesting that increased engagement with teaching improves learning.
We set out to produce a realistic and immersive augmented-reality simulation session in the hopes of improving engagement with the session. Employing various methods to stimulate all the students' senses, we believe that we succeeded in creating a high-fidelity simulation. However, the results from the feedback questionnaire do not show any significant difference in immersion between the augmented and non-augmented scenario.
On a more positive note, many augmented-reality features, especially the video-handover, received very positive feedback and were viewed as successful in improving realism and engagement.
We believe that augmenting simulation scenarios in this manner has the potential to improve engagement and retention of knowledge and skills. It is a cheaper, more cost-effective alternative to traditional simulation. Further work is required, involving greater numbers of students and more robust methods of measuring engagement and knowledge retention.