Objective improvement with coronary anastomosis simulation training: meta-analysis

Abstract Background Coronary artery anastomosis training and assessment are vital for patient safety and for conferring a prognostic benefit. A systematic review and meta-analysis were performed to analyse the impact of simulation on coronary anastomosis proficiency in terms of time taken and skill score. Methods This review was conducted in accordance with PRISMA guidelines, searching PubMed, Embase and Cochrane databases on 10 October 2020, using the terms ‘Coronary anastomosis simulation’ or ‘vascular anastomosis simulation’ and ‘anastomosis simulation’. Studies included had objective measurement of scores of before and after simulation. Meta-analysis was performed using RevMan, version 5.4 (Cochrane Library). Results From a pool of 1687 articles, 12 articles evaluating the use of simulation in teaching coronary anastomosis were identified, with objective scores at baseline and after simulation. The 12 papers included 274 subjects. Data on 223 subjects could be extracted for analysis in performing coronary anastomosis in a simulated environment. Eight trials evaluated improvement in time and 12 trials evaluated performance using an objective evaluation score. In comparison with no formal simulation training, simulation was associated with improved skill in a five-point scale (standardized mean difference 1.68 (95 per cent c.i. 1.23 to 2.13; P < 0.001)) and time (mean difference 205.9 s (95 per cent c.i. 133.62 to 278.18; P < 0.001)) in trials included in the meta-analysis. Furthermore, novice cardiothoracic surgeons benefited more from simulation as regards time improvement compared with senior cardiothoracic surgeons (293 versus 120 s improvement; P = 0.003). Fidelity of simulator did not have a significant effect on rates of improvement. Conclusion Simulation-based training in coronary anastomosis is associated with improved time efficiency and overall performance in comparison with no intervention. Further studies are necessary to determine the optimum timing of trainees progressing from simulation training to live operating.


Introduction
Coronary artery bypass grafting surgery has been the cornerstone of cardiac surgery since its development in 1960 by Goetz and colleagues 1 . Results of the ROOBY (Randomized On/Off Bypass) trial highlight the significance of excellent surgical technique, whether on-pump or off-pump, in improving graft patency, with a 16.4 per cent rate of adverse events at 1 year in patients with ineffective revascularization compared with a 5.9 per cent rate in the effective revascularization group 2 . Therefore, it is an axiom that patient safety be at the forefront of educating trainees in performing coronary anastomosis. This paradigm has shifted medical-education models from apprenticeship towards simulation training 3 .
Approaches to simulation in educating trainees in cardiothoracic procedures appears to offer appropriate training without risk to patients or trainees [3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19] . It has been employed widely and has shown efficacy in the education for adult cardiac surgery, thoracic surgery, transcatheter aortic valve implantation, robotics, perfusion, cardiac advanced life support and extracorporeal membrane oxygenation 12 . Training for coronary artery bypass grafting differs amongst consultant trainers and training programmes. Reviewing the evidence with respect to the effectiveness and key steps of coronary anastomosis simulation would permit training bodies to target problem areas and identify remaining research needs.
No previous meta-analysis or review has been published regarding how objective scores and times improve with simulation in coronary artery bypass grafting. The aim of this study was to identify, analyse and summarize comparative studies objectively assessing coronary anastomosis simulation by conducting a systematic review of the literature.

Methods
A systematic review and meta-analysis of coronary artery anastomosis performance comparing simulation to no prior simulation was performed in adherence to PRISMA standards of quality for reporting meta-analysis 20 . The specific aim was to analyse if simulation could improve either objective competency scores or time to task completion. No ethical approval was required for this study.

Eligibility
Studies involving healthcare trainees at any stage in training that were evaluated while using simulation to teach coronary anastomosis in comparison with no intervention (that is, a control arm or preintervention assessment) were included. Single-group pre-test/post-test and two-group randomized and non-randomized trials were included. Exclusion criteria were articles not available in English and those which did not refer to coronary anastomosis simulation training. Studies reporting on health professionals using biological tissue and synthetic material to perform coronary anastomosis in simulation in comparison to no simulation were included.

Study identification
PubMed, Embase and Cochrane databases were searched on 10 October 2020, using the terms 'Coronary anastomosis simulation' or 'vascular anastomosis simulation' and 'anastomosis simulation'. Two independent investigators participated in study selection and data abstraction was performed independently and in duplicate.

Study selection
Study selection involved two stages. In stage 1, all studies evaluating coronary artery simulation were identified. Full-text versions of all publications that could not be excluded with confidence were obtained and reviewed for definite inclusion, independently and in duplicate using the Covidence platform. In stage 2, studies in which an end-to-side vascular anastomosis were performed with the goal of simulating coronary anastomosis, and those in which time and/or objective score improvement were assessed objectively were identified. End-to-end anastomosis simulation or large vessel anastomosis or studies in which coronary anastomosis was not addressed were excluded. When conflict arose, the paper was discussed between two reviewers to clarify inclusion and exclusion criteria. No paper required a third reviewer.

Data extraction
Information was abstracted separately for time improvement and Objective Structured Assessment of Technical skill (OSAT) score improvement. OSAT score improvement was further subclassified into individual segments of arteriotomy, graft orientation, bite/depth of bite, spacing, use of needle holder, use of forceps, needle angles, needle transfer, suture management, knot tying, hand mechanics, use of both hands and economy of time if provided in the text. If surgical skill score or time score was not reported, it was not included in the meta-analysis.

Data synthesis
Studies were grouped according to the comparison arm, that is scores before simulation and those after simulation. To analyse scores homogeneously, scores and standard deviations were converted to a five-point scale, where 1 corresponded with a poor score and 5 corresponded with an excellent score. For quantitative synthesis, results were pooled using meta-analysis, comparing time to completion (in seconds) and five-point score before simulation with time to completion (in seconds) and five-point score after simulation. Qualitative synthesis of trials was also included, identifying curriculum design features, to assess salient themes.

Risk of bias
The Cochrane Risk of Bias tool and the Newcastle-Ottawa quality assessment scale (NOS) for non-randomized cohort studies were applied to determine objectively the quality of each eligible study 21,22 . The Cochrane tool considered bias under the following headings: selection bias, performance bias, detection bias, attrition bias, reporting bias or other bias 22 . For the NOS, points were awarded for patient selection (maximum four points), outcome assessment (maximum three points) and comparability of cohort (maximum two points): each added to a maximum of nine points 21 .

Statistical analysis
Competency scores and time to complete tasks were reported on continuous scales. Mean and standard deviation were extracted and calculated for continuous outcomes. The mean difference was the difference in competency score or time to complete a task before and after simulation. Objective scores were inverted or multiplied as required to match other five-point Likert scales. Owing to inherent differences in study populations and clinical sites, random effects meta-analyses were conducted for all outcomes. The weighted mean differences or standard mean differences with 95 per cent confidence intervals were calculated for continuous variables using the inverse variance method. The Cochrane chi-squared and I 2 statistic were used to examine the heterogeneity among effect estimates in included studies. Statistical heterogeneity among studies was defined as I 2 . 50 per cent. Review Manager version 5.4 (The Nordic Cochrane Centre, The Cochrane Collaboration, Copenhagen, Denmark) was used to perform the meta-analysis and to generate forest plots 23 . P , 0.050 was considered statistically significant.

Literature search
The search strategy identified 1650 articles, demonstrated in Fig. 1. After eliminating duplicates and selecting out studies evaluating simulation in coronary artery anastomosis, 12 articles were identified. These studies evaluated simulation-based coronary anastomosis training in comparison with no intervention and these were included in the meta-analysis and are summarized in Table 1. Out of eleven studies which evaluated an objective score improvement, seven could be used in quantitative meta-analysis. Furthermore, the OSAT scores of each of the steps in coronary anastomosis, if available, were subanalysed in these score-improvement studies. Out of eight studies which evaluated time improvement, six could be used in quantitative meta-analysis. All articles were published in English. None were identified in any other language. Study design is described in Table 1. Risk of bias assessment is summarized in Tables 2 and 3. The majority of outcome data was fully available, ensuring a lack of attrition bias between studies. Heterogeneity between study designs introduced selection and selective reporting bias.

Study characteristics
In total, 223 out of 274 potential participants (178 residents or trainees in surgery and 45 medical students) were assessed in meta-analysis. Further demographics of participants based on age and gender in relation to level of improvement were not reported consistently. No study assessed performance in the context of coronary anastomosis in real patients. The only study which had a control group defined as normal operative intervention was that by Spratt and colleagues 14 .
Simulation modalities varied amongst trials. These included both low-and high-fidelity simulation models. Low-fidelity models were described as three-dimensional multistation cardiovascular simulators 14 , the Arroyo box Simulator, vessel anastomosis simulators with silicone vein grafts (Chamberlain Group, Great Barrington, Massachusetts, USA; Limbs & Things, Savannah, Georgia, USA) and the 'BEAT, YOU-CAN' model 7,9,11,[14][15][16][17] . Such models are made of a range of synthetic materials with 3-mm silicone 'vein grafts' which are suitable for multiple end-to-side anastomoses (Fig. 2). The beating heart models are also silicone based and are connected to an external compressor which simulates the movement of the beating heart. High-fidelity models included bovine hearts and pulsatile pig hearts 5,8,10,15,18 . Five studies used biological tissue, while the rest used low-fidelity or a combination of low-and high-fidelity models. The heterogeneity of these models introduced a risk of bias in analysing the data.
Curriculum design was not uniform across the trials, and length of study period or curriculum ranged from 2.5 days to 6 months. One trial mandated a minimum of 20-30 min of dedicated technical practice each week for 8 weeks, however the authors noted that participants may not have committed to this timetable 14 . Maluf and colleagues' training programme spanned over 6 months using four different models 15 . The Top Gun group undertook 6 weeks of simulation 16 , while Enter and co-workers' medical students had 1 month of training 17 . Nesbitt's team evaluated simulation over 4 months 18 . Fann and colleagues gave residents the task station to practice and evaluated improvement after 1 week 5 . There was an over-riding theme of self-directed learning. Only Enter and colleagues, evaluated the impact of supervision on improving simulation scores in medical students. There was no statistically significant difference in supervised simulation and non-supervised simulation.

Effectiveness in improving time and score in comparison with no formal simulation
Eight studies evaluated time improvement comparing simulation-based training with presimulation score 5,7,9,10,15,16,18,19 . Data from six of these studies could be included in meta-analysis. Four of these studies 5,7,9,10 . had more than one group, which are listed in Fig. 3. For time analysis, there were eleven groups who were assessed prior to and after simulation. A mean difference of 205.9 s was observed (95 per cent c.i. 133.52 to 278.18; P , 0.001) in trainees who used simulation (Fig. 3). Individual effect sizes ranged from 64 to 443 s. There was also presence of heterogeneity: I 2 = 77 per cent. The forest plot favoured simulation training with respect to time.

Identification
Screening Eligibility Included Records identified through database search terms 'coronary anastomosis simulation' or vascular anastomosis simulation and 'anastomosis simulation' EMBASE & PUMBED n = 1650 Abstracts screened n = 1355 Full study texts assessed for eligibility n = 98 Full-text articles assessed for eligibility n = 15 Studies included in qualitative synthesis n = 12 Studies included in quantitative synthesis n = 9 Duplicates excluded n = 295 Studies irrelevant n = 1257 Records excluded n = 83 Full-text articles excluded n = 3 Did not address coronary artery anastomosis  Twelve studies evaluated score improvement, comparing simulation-based training with no intervention. Data from seven studies could be included in meta-analysis. Four studies 5,8,9,17 . evaluating score improvement had more than one group which underwent simulation, and these are listed in Fig. 4. There were 14 groups assessed prior to and after simulation as regards score improvement. Trainees who used simulation had an associated improvement of 1.68 points (95 per cent c.i. 1.23 to 2.13; P , 0.001) (Fig. 4). Heterogeneity was present between studies, with I 2 = 77 per cent and individual effect sizes ranging from 0.51 to 3.14 points. The forest plot favoured simulation training with respect to score improvement.
Furthermore, the use of simulation in improving OSAT score on specific tasks was analysed (Fig. S1). Simulation was associated with significant improvement in all trainee scores in arteriotomy, graft orientation, bite/depth of bite, spacing, use of castro/needle holder, use of forceps, needle angles, needle transfer, suture management, knot tying, hand mechanics, use of both hands, economy of time and configuration of anastomosis. Heterogeneity was present in many of these specific tasks.

Simulation effect on level of training
Nesbitt and colleagues demonstrated that simulation training improved medical students' time to completion and skill score with coronary anastomosis to a level comparable to that of residents, while the senior residents had only a non-significant trend towards improvement 18 . Subgroup analysis demonstrated that simulation in junior and non-cardiothoracic residents was associated with more of a positive impact when compared with senior residents, as regards time improvement (293 versus 120 s, P = 0.003), in a test for subgroup differences (Fig. 3). However, the heterogeneity between groups was high and thus limits  interpretation of this result. There was no significant difference in time improvement when comparing high-and low-fidelity simulator trials. In subgroup analysis of score improvement between high-and low-fidelity simulators, there was no significant difference between groups in subgroup differences (P = 0.67) (Fig. 4). Again, heterogeneity scores made this difficult to interpret.

Discussion
In comparison with no intervention, simulation was consistently associated with better learning outcomes in two broad categories: time and OSAT score improvement. Simulation permits surgical trainees to acquire complex and meticulous skills without jeopardizing patient safety. The results can also be explained anecdotally. The phrase 'practice makes perfect' can be demonstrated graphically, plotting performance against experience, which usually produces a sigmoid curve. Repeated application of learned principles through simulation is key to achieving success in transferring skills from the working and sensory memory to the long-term memory. Issenberg and co-workers described the necessary features for simulation to transform a principle into a lesson learned 24 . This includes provision of feedback, repetitive practice, curriculum integration, appropriate range of difficulty level, multiple learning strategies, simulation that captures clinical variation, a controlled environment, individualized learning, defined outcomes and simulator validity 20 . Simulation, and meta-analysis of simulation, has been carried out in other aspects of cardiothoracic surgery, for example, bronchoscopy 3,12 . Simulation has also shown to be useful in real-life scenarios, with a study by Ledermann and co-workers evaluating junior orthopaedic residents, demonstrating that when trainees use a low-fidelity simulator to perform an arthroscopic partial meniscectomy, their score in the Arthroscopic Surgical Skill Evaluation Tool improved in real-life patients 25 .
When senior trainees utilized coronary artery simulators, it did not appear to have a significant effect in improving scores and performance 14 . A suggested reason for this is that senior trainees have lack of flexibility and time in utilizing simulators due to clinical practice and there may be diminishing marginal benefits from additional practice 7,24 . Anecdotally, the flimsy design of lowfidelity simulators has been a barrier for their use. Novice or junior trainees appeared to benefit from simulation across the trials 17 . The non-significant trend towards improvement noted in Nesbitt and colleagues' study in senior residents may also be explained by achieving a proficiency, which only needs to be practised occasionally 18 . It should be noted that junior and senior trainees may not be distinguishable after multiple simulations given there may be a plateau or a ceiling effect 5,26 .  Regarding curriculum design, the use of instructional videos and homework plans was explored in these studies 5,14,17 . Furthermore, it was shown that using high-fidelity porcine simulators increased trainee interest in that specialty 11 . Only one study evaluated the effect of additive supervision in anastomosis training and this did not appear to have a significant effect in training 17 . However, this study only included 45 medical students and may well have been insufficiently powered 17 . The present study highlights aspects of current teaching methods and the need to create dynamic curricula that create proficient surgeons [27][28][29] .
The transition from simulation to real-life patients has not been explored in the literature in coronary artery anastomosis. However, it has been identified that after 30 anastomoses on the simulator, the learning curve stabilized 5,15,19 . Furthermore, in a study of 15 cardiothoracic surgeons by Bridgewater and colleagues, it was identified that newly appointed consultant cardiothoracic surgeons had an improvement in the mortality rate of their low-risk patients undergoing coronary artery bypass graft after 4 years compared with established surgeons 30 . Studies need to evaluate when it is best for a trainee to progress from lowfidelity to high-fidelity simulators and to patients.
Quantity and quality of studies limit the results of this study, along with clinical and methodological diversity. The search criteria and analysis were limited to studies which focused on coronary anastomosis simulation, however there may be merit in including end-to-side anastomosis simulation utilized in vascular surgery, for example, femoral bypass. There were significant differences in approaches between studies. In particular, the quality and amount of instruction given to anastomosis performers in each of the trials were difficult to compare. Verification of simulation practice at home could not be documented accurately. The reality of having a first assistant is not explored in these studies. The length of time and quality of dedicated practice to simulation was not taken into account in the meta-analysis. OSAT scores were similar and inter-observer reliability was maintained in individual studies, but heterogeneity exists between studies. Outcomes of time and score were determined objectively in individual studies, but a significant limitation was a lack of a control group for simulation groups with test scores before and after simulation. Progression with operative experience could account for improved scores, particularly in the lengthier studies. While heterogeneity between studies exists, a positive impact with the use of simulation is demonstrated throughout. This is the first meta-analysis of the use of simulators in coronary artery anastomosis and it provides a basis for reforming training in cardiac surgery. These data suggest simulation improves steps and time to completion in coronary anastomosis, therefore it is suggested that these steps be focused on during training days and trainees should be encouraged to practise at home. Heterogeneity of studies limited the interpretation of whether Heterogeneity: τ 2 = 1986.90; χ 2 = 7.07, 4 d.f., P = 0.13; I 2 = 43% Test for overall effect: Z = 3.90, P < 0.001 Heterogeneity: τ 2 = 9568.56; χ 2 = 17.13, 5 d.f., P = 0.004; I 2 = 71% Test for overall effect: Z = 5.85, P < 0.001 Heterogeneity: τ 2 = 10 428.27; χ 2 = 44.33, 10 d.f., P < 0.001; I 2 = 77% Test for overall effect: Z = 5.58, P < 0.001 Test for subgroup differences: χ 2 = 8.65, 1 d.f., P < 0.003; I 2 = 88.4% Anand et al. 7 Senior residents Fann et al. 5 Beating heart Fann et al. 5 Non-beating Ito et al. 19 Tavlasoglu et al. 10 Senior CTS residents Anand et al. 7 Senior residents Enter et al. 16 Top gun Malas et al. 9 Instruction video Malas et al. 9 No video Tavlasoglu et al. 10 Junior CTS residents Tavlasoglu et al. 10  Heterogeneity: τ 2 = 7777.14; χ 2 = 7.06, 3 d.f., P = 0.07; I 2 = 58% Test for overall effect: Z = 3.55, P < 0.001 Heterogeneity: τ 2 = 10 428.27; χ 2 = 44.33, 10 d.f., P < 0.00001; I 2 = 77% Test for overall effect: Z = 5.58, P < 0.001 Test for subgroup differences: χ 2 = 0.00, 1 d.f., P = 0.95; I 2 = 0% Anand et al. 7 Junior residents Anand et al. 7 Senior residents Fann et al. 5 Beating heart Fann et al. 5 Non beating Tavlasoglu et al. 10 Junior CTS residents Tavlasoglu et al. 10 Non CTS residents Tavlasoglu et al. 10 Senior CTS residents Enter et al. 16 Top gun Ito et al. 19 Malas et al. 9 Instruction video Malas et al. 9   simulation is more beneficial for junior trainees than for established surgeons, thus more controlled studies should evaluate when when it is best for trainees to transition from simulation to live operating. Future research should look at cost analysis of low-and highfidelity models and consider the provision of these models to surgical trainees. Training bodies could then focus simulation days, taking into account cost and time. Use of faculty time is an important consideration in employing their use. Training bodies should balance the cost of simulation with the inherent benefit of improving interest in the specialty.
Simulation is an accepted form of teaching surgical techniques. Trainers may establish a baseline competence before awarding trainees responsibility in the operating theatre. Demonstrating proficiency in simulation may progress trainees faster rather than the traditional apprenticeship model. Identifying appropriate proficiency levels in which trainees may progress from low-to high-fidelity models and subsequently to patients should be identified with further research. Advanced skills, such as managing unexpected bleeding and utilizing assistance, which has not been evaluated by these simulation studies, may be taught in the apprenticeship model in the operating theatre.
Heterogeneity: τ 2 = 0.54; χ 2 = 56.18, 13 d.f., P < 0.001; I 2 = 77% Test for overall effect: Z = 7.35, P < 0.001 Test for subgroup differences: χ 2 = 0.19, 1 d.f., P = 0.67; I 2 = 0% Wu et al. 8 Non-beating group on non-beating heart Wu et al. 8 Non-beating group on beating heart Wu et al. 8 Beating group on non-beating heart Wu et al. 8 Beating group on beating heart Fann et al. 5 Non-beating Fann et al. 5  Malas et al. 9 No video Malas et al. 9 Instruction video Ito et al. 9 Enter et al. 16 Top Gun Enter et al. 17 Fig. 4 Forest plot comparing no simulation versus simulation in evaluating improvement in skill score Skills evaluated using a five-point scale where 0 = poor score and 5 = excellent score. Subgroup analysis comparing high-fidelity simulator with low-fidelity simulator.