The eﬀectiveness of surgical simulation in orthopaedic teaching and training: A literature review

within their programme. One strategy to address this gaining increasing recent is the incorporation of simulation training within orthopaedic training programmes. The aim of this paper is to identify and review the literature which currently exists regarding the eﬀectiveness of surgical simulation in orthopaedic teaching and training. Overall, there are a suﬃcient number of encouraging initial reports to suggest simulator training may potentially be a valuable tool for training future generations of orthopaedic surgeons. Signiﬁcant work is still required however to deﬁnitively validate their use. Additional research is also needed to assess trainee and educator satisfaction with simulators, their cost eﬀectiveness and their role within the higher surgical orthopaedic curriculum. At present, this essential data is still lacking.


Introduction
Surgical training is currently undergoing dramatic changes from the traditional master-apprentice model which has stood for centuries and is facing significant challenges (Akhtar et al., 2014). Changes in the way health care is delivered coupled with the advances in orthopaedic surgical techniques entails it has become more difficult for surgical trainees to develop the operative skills required within their training programme (Stirling, Lewis and Ferran, 2014).
The introduction of the European Working Time Directive (EWTD) has led to working hour restrictions of no more than 48 hours a week on average. At the same time there has been an expansion in the number of higher surgical trainees and a push towards senior delivery of care. Increasing pressures on the NHS has also meant there is an ever increasing conflict between training and provision of service. The combination of these factors has meant there are now far fewer learning opportunities for trainees to develop their skill set within the allocated training time than was the case in the past. The rise in medico-legal claims and costs coupled with a greater emphasis on patient safety has also resulted in fewer operating opportunities for trainees (Sonnadara et al., 2012).
Due to this decline in 'hands on' training opportunities, there has been a drive to develop alternative strategies to try and address this shortfall in clinical experience. One such strategy gaining increasing interest and exposure over recent years is the incorporation of simulation training within surgical teaching programmes (Kalun et al., 2018).
Simulation within medical training has previously been defined as 'any technology orprocess that recreates a contextual background in a way that allows a learner to experience mistakes and receive feedback in a safe environment' (Gaba, 2007). The aim of any simulation is therefore to improve trainee confidence and enhance surgical skills without any compromise in the safety of patients.
Simulation in surgical training is not a completely new concept as the use of cadavers to practice operative techniques has long been established. In recent years however, newer simulation techniques have evolved including prosthetic model, arthroscopic, three-dimensional software and virtual reality simulation.
Satava (Satava, 1993) originally recognised the potential of simulator training in surgery fifteen years ago, describing it's potential benefits in terms of safety and savings in money and time. It was not however widely accepted initially due to the basic nature of the technology, high initial set up costs, low sensory feedback and the absence of literature at the time demonstrating its potential benefit to surgical training. Since then, there have been considerable advances in the technology enabling more realistic surgical simulations. This does not however validate the use of simulators in surgical teaching and training and the necessary research is required to demonstrate their effectiveness (Tay, Khajuria and Gupte, 2014).
General surgery has been quicker to recognise the potential benefits of simulators in teaching and training with simulation training having been demonstrated to successfully reduce the training time required to perform in-vivo Kyriacou S MedEdPublish https://doi.org/10.15694/mep.2020.000150.1 Page | 3 laparoscopic surgery and to reduce the rate of intra-operative errors (Aggarwal et al., 2007). In contrast, orthopaedics has been relatively slow to integrate simulation teaching and training within the post-graduate training curriculum and there are fewer studies demonstrating its effectiveness compared to general surgery (Aim et al., 2016).
The aim of this paper is therefore to identify and review the literature which currently exists regarding the effectiveness of surgical simulation in orthopaedic teaching and training. Based upon the findings of this review, conclusions will be drawn and the potential benefit of incorporating simulator teaching and training into the educational practice delivered by higher surgical training programmes will be discussed.

Methods
An electronic search of the databases MEDLINE, EMBASE, and Cochrane Library Databases was conducted from inception to February 27 2020, restricted to articles published in English. To identify studies exploring the effectiveness of simulation in orthopaedic teaching and training, the following key words were searched for: simulation, simulator, orthopaedic, training, teaching and trainer. This database search was supplemented by a search of the reference lists of included studies and the related articles function offered by each database. Articles were vetted to include potential studies, with the entire paper being read to decide upon the relevance of each study.
Studies were deemed eligible if they reported data which contributed towards determination of the effectiveness of surgical simulation in orthopaedic teaching and training. A formal quality assessment tool was not utilised given the study was a literature and not a systematic review.

Results
There are various types of simulator validity described within the literature which will be considered in turn to assess the effectiveness of surgical simulation in orthopaedic teaching and training (Tay, Khajuria and Gupte, 2014;Thomas et al., 2014). These are: Face validity (fidelity) -Extent to which a simulator mimics what it is intended to simulate Construct validity -Ability of a simulator to discriminate between differing skill levels Concurrent validity -Degree to which simulator performance correlates with another assessment tool Transfer validity -Ability of simulator to demonstrate a learning curve showing better performance with continuous use, which is then also observed in real life settings Face validity: In terms of face validity, improved trainee simulator performance has been demonstrated in studies utilising both low and high fidelity simulators (Butler et al., 2013;Cannon et al., 2014;Hetaimish, Elbadawi and Ayeni, 2016). It has been proposed high fidelity simulators may potentially improve the transfer of skills to the operating theatre compared to low fidelity simulators due to the greater amount of haptic feedback they provide and that they may therefore be more suitable for more experienced surgeons (Stirling, Lewis and Ferran, 2014). No definitive advantage of high fidelity simulators was demonstrated however in a systematic review conducted by Hetaimish et

Construct validity:
Construct validity has been demonstrated in numerous studies thereby showing a positive relationship between increasing surgical experience and better simulator performance. A recent review article (Bartlett et al., 2018) has shown this to be the case in a wide array of simulators from simple fracture fixation models to more complex arthroscopic simulation. Although there is strong evidence that simulators can reliably distinguish between inexperienced and experienced orthopaedic surgeons, there is less evidence that they can currently identify intermediate learners who are in the middle of the learning spectrum (Kalun et al., 2018).

Concurrent validity:
Concurrent validity has been assessed in several studies with various results (Bartlett et al., 2018). Camp et al. (Camp et al., 2016) demonstrated improvement in knee arthroscopy performance on a cadaveric simulator following a period of training on a virtual reality simulator by a cohort of orthopaedic trainees. Similar proof of concurrent validity has been demonstrated in medical students by Banasezek et al. (Banaszek et al., 2017) whose study showed concurrent improvement in arthroscopic skills when learning on both virtual reality and bench top simulators. This is in contrast however to another similar study (Rebolledo et al., 2015) which did not find the same improvement in a cadaveric shoulder simulator following a period of training on a virtual reality simulator.

Transfer validity:
It has been argued the most convincing evidence for the effectiveness of a simulator is when it can be demonstrated that improved simulator performance has a positive impact on 'real life' performance (Thomas et al., 2014) thereby demonstrating transfer validity. There were 4 trials identified in this literature review which focused on the difference teaching orthopedic trainees on arthroscopy simulators made to their arthroscopic skills within the operating theatre (Howells et al., 2008;Cannon et al., 2014;Dunn et al., 2015;Waterman et al., 2016). Different methods of measuring performance were employed in each of the four studies, but all included a validated assessment tool. Time to completion of the technical procedure was also assessed in each of the studies. Economy of movement was assessed in 2 of the papers with the use of motion analysis (Howells et al., 2008;Waterman et al., 2016).
Using these criteria, 3 of these studies (Howells et al., 2008;Cannon et al., 2014;Waterman et al., 2016) found skills acquired from simulation training resulted in a statistically significant measurable improvement in real-life surgery performance in the simulator trained group compared to the non-simulator trained control groups in at least 1 single measure of performance.
There were however significant methodological differences between the studies which makes direct comparison between them difficult to make. With the exception of the simulator training sessions, 3 of the studies (Cannon et al., 2014;Dunn et al., 2015;Waterman et al., 2016) otherwise issued identical training videos and manuals to both the simulator group and control group. The study by Howells et al. (2008) however did not provide any learning resources to the control group which may have contributed to the differences observed from the simulator group.
In addition, the study by Cannon et al. (2014) needed trainees to achieve a maximum score on simulator training prior to progressing to real life surgery. Dunn et al. (2015) and Waterman et al. (2016) simply required four 15 minute simulator sessions under the supervision of an experienced arthroscopist over a period of 3 months prior to performing the procedure in a real life setting. This is in contrast to the study by Howells et al. (2008) who stipulated Kyriacou S MedEdPublish https://doi.org/10.15694/mep.2020.000150.1 Page | 5 a more intensive simulator training regime consisting of three sessions with six arthroscopies being performed at each session over the space of a week. There were also differences in the fidelity of the simulators used in each of the studies again making a direct comparison between the findings difficult (Kalun et al., 2018).
Other than these 3 of studies however (Howells et al., 2008;Cannon et al., 2014;Waterman et al., 2016), there is currently limited data in the literature demonstrating transfer validity in orthopaedic simulators and a recent systematic review (Hetaimish, Elbadawi and Ayeni, 2016) focusing exclusively on knee arthroscopy expressed that many existing publications demonstrate transfer validity between various simulators as opposed to real life settings.
There have been various papers published investigating if and how the ability of a learner advances with increased exposure to simulator training (Bliss, Hanner-Bailey and Scerbo, 2005;McCarthy et al., 2006;Andersen, Winding and Vesterby, 2011;Jackson et al., 2012;Pollard et al., 2012;Sugand et al., 2015;Rahm et al., 2016;Sugand, Mawkin and Gupte, 2016). Measurements including economy of movement and time elapsed when learning hip arthroscopy on a simulator have been used to show the formation of a definite learning curve amongst a cohort of junior doctors with no prior experience of the procedure (Pollard et al., 2012). Similar findings of a learning curve in medical students using an arthroscopic knee simulator have been reported (McCarthy et al., 2006) as well as evidence of progressive trainee advancement when using a virtual reality arthroscopic shoulder simulator (Andersen, Winding and Vesterby, 2011).
Studies have also been conducted which demonstrate the successful retention of skills developed during simulation training (Kalun et al., 2018). Jackson et al. (Jackson et al., 2012) demonstrated an improvement in virtual keyhole soft tissue knee repair was measurable for up to 26 weeks following the original simulator teaching session whilst Bliss et al. (Bliss, Hanner-Bailey and Scerbo, 2005) showed minimal loss of arthroscopic simulator performance 4 weeks after the initial teaching session.

Discussion
Ericsson described 10,00 hours of practice is required to attain mastery of a technical skill (Ericsson, 2008) and is has become apparent this is currently difficult to achieve in orthopaedic training programmes. There is therefore an increased drive for more streamlined and focused surgical training which minimizes risk to the patient. Simulation training may be able to contribute to this by providing a safe environment where operative skills can be developed and refined. The integration of orthopaedic surgical simulation within training programmes may potentially help trainees learn both basic and more advanced operative skills and facilitate a quicker advancement up the learning curve (Akhtar et al., 2014). With continued advances in technology making simulators evermore realistic, it is likely they will play an ever-larger role in orthoapedic education.
In terms of assessing the 'effectiveness' of surgical simulation in orthopaedic teaching and training which was the aim of this literature review, evidence has been identified demonstrating trainees practicing on simulators does lead to an improved performance on simulators (Aim et al., 2016). Although construct validity has clearly been demonstrated with orthopaedic simulators having the ability to discriminate between differing levels of pre-existing surgical expertise (Aim et al., 2016), this in itself does not measure or demonstrate any definite value to trainees or patients (Bartlett et al., 2018).
Despite the fact there are studies demonstrating simulator training in turn leads to an improved performance in theatre (Howells et al., 2008;Cannon et al., 2014;Waterman et al., 2016), these are currently few in number involving a relatively small number of trainees and further research is required before any definitive conclusions on Kyriacou S MedEdPublish https://doi.org/10.15694/mep.2020.000150.1 Page | 6 the transfer validity of simulator training can be drawn. In this respect, orthopaedic surgery is lagging behind other fields such as general surgery where a correlation between simulator and real-life performance has clearly been demonstrated (Gallagher et al., 2013;Aggarwal and Grantcharov, 2015). Further work is also required to establish the frequency, type and length of time of simulator training sessions required to observe real life improvement in performance (Frank et al., 2014).
Another major issue that as yet has not been addressed, is the development of a widely accepted, standardised measure of assessing trainee performance on simulators (Atesok et al., 2017). There is currently a great deal of variation amongst published studies within the literature in terms of the outcome scores and measurements used to evaluate the effectiveness of various simulators making direct comparison difficult (Hetaimish, Elbadawi and Ayeni, 2016). This also makes cumulative statistical meta-analysis of multiple studies impossible. In order to address this issue, a standardised set of outcomes utilised by all research involving the use of simulators is ideally required thereby making comparison between study findings more meaningful.

Conclusion
Overall, there are a sufficient number of encouraging initial reports to suggest simulator training may potentially be a valuable tool for training future generations of orthopaedic surgeons. Significant work is still required however to definitively validate their use. Additional research is also needed to assess trainee and educator satisfaction with simulators, their cost effectiveness and their role within the higher surgical orthopaedic curriculum. At present, this essential data is still lacking and is required for a definitive recommendation regarding the implementation of simulation training into the orthopaedic syllabus to be made.

Take Home Messages
• It is increasingly difficult for orthopaedic trainees to gain adequate operative exposure • Simulators have been proposed as a method of augmenting traditional surgical training methods • Early reports suggest simulation may have an increasingly useful role in Orthopaedic training

Notes On Contributors
Mr Steven Kyriacou MSc, FRCS (Orth), PG Cert Med Ed is an Orthopaedic Fellow with an interest in medical education.