Using Cognitive Task Analysis to train Orthopaedic Surgeons - Is it time to think differently? A systematic review.

Background Working time restraints; senior led care; and a reduction in ‘out of hours’ operating has resulted in less operating time for orthopaedic trainees in the United Kingdom. Therefore, there has been an attempt to overcome these challenges by implementing novel techniques. Cognitive Task Analysis (CTA) focuses on the mental steps required to complete complex procedures. It has been used in training athletes and in general surgery but is new to orthopaedic training. Aim To undertake a systematic review to analyse if CTA is beneficial to train novice surgeons in common orthopaedic and trauma procedures. Materials and methods A systematic review was performed evaluating CTA in trauma and orthopaedic surgery on MEDLINE and EMBASE. Search terms used were: 'Cognitive task’, ‘mental rehearsal’ and ‘Orthop*'']. 33 studies were originally identified. Duplicate studies were excluded (11). Articles not relating to Orthopaedic surgery were excluded (15). The CTA research ranking scale was used to evaluate the impact of the studies included. Results 7 studies were identified as appropriate for inclusion. 264 participants. 178 M, 86F. All studies showed objective or subjective benefits from CTA in orthopaedic training when compared to traditional methods. The majority of the participants highlighted high subjective satisfaction with the use of the CTA tools and reported that they proved to be excellent adjuncts to the traditional apprenticeship model. Conclusion CTA learning tools have demonstrated significant objective and subjective benefits in trauma and orthopaedic training. It is cost effective, easily accessible and allows repeated practice which is key in simulation training.


Introduction
Despite changes to orthopaedic training it is well established that current trainees have significantly less theatre training time as compared to their predecessors [1]. The shift from time-based to competency-based training has worsened the current situation [1]. In an attempt to counter both a reduction in theatre time for trainees and a rising demand for skilled surgeons, the development of simulation as an adjunct to the apprenticeship system has aided trainees to achieve their required training needs.
Simulation is 'a method or technique that is employed to produce an experience without going through the real event' [2]. It occurs in a safe environment and has been shown to improve confidence for surgeons [3]. It is heavily advocated in other specialities such as Emergency Medicine [4], General Surgery [5] and by the Royal Colleges of Surgeons [6].
Cadaveric and virtual reality simulation has found increasing application within orthopaedic training [7][8][9][10]. However, they are expensive and not readily accessible. Karam et al. have shown that in the USA, 25% of training programs do not have a dedicated simulation facility and this is due to lack of sufficient funds [11]. The situation is no different in the UK [12]. In this setting, it is important to have a suitable adjunct to the traditional apprenticeship model.
Cognitive Task Analysis (CTA), which originates from the military, is a modern approach to simulation, placing emphasis on decision making and the thought processes behind each step of key procedures [13]. It allows trainees to learn 'how to do' a technical step, 'why' they are doing the step and any potential errors (and solutions) that they can make in each phase of a surgical procedure. Studies have highlighted that mastering surgery requires a high cognitive and mental ability [14]. Fitts and Posners's model for mastery of skill implies that cognitive staging is the primary mental process needed for learning, followed by associative and automated processes [15]. Skill acquisition by cognitive methods has been shown to produce changes in the brain which engage the primary motor complex [16].
A systematic review evaluating CTA in general surgery (2013) has given an insight towards how CTA can improve surgical performance [13] and in recent years CTA has been increasingly used in trauma and elective orthopaedic training [17][18][19].
The aim of this study is to evaluate whether CTA learning tools are of benefit to trainees in Trauma and Orthopaedic Surgery.

Literature search strategy
We reviewed studies on CTA in Orthopaedics on the Ovid MEDLINE and EMBASE databases. A guide stating the research question, search strategy, inclusion/exclusion criteria and risk of bias was formulated. The search and screening were performed by two of the authors, and disagreements were resolved by consensus. An electronic search was performed on November 17, 2019. No date limitations were placed. Only English articles were selected. Key words and phrases used were: 'Cognitive task' OR 'mental rehearsal' AND 'Orthop*'. Due to the limited published data on CTA all study types and trainee ranges were considered. The studies retrieved from the search were manually reviewed to identify other studies which could be relevant. Also reviewed were reference lists of included studies, study registries, and grey literature. This review has been reported in line with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) (Fig. 1). And AMSTAR-2 (Assessing the methodological quality of systematic reviews) Guidelines.

Selection, inclusion and exclusion criteria
Studies which specifically analysed CTA in Orthopaedic surgery were evaluated. We defined CTA as a method by which knowledge for a procedure can be imparted via step by step protocols. 33 studies were identified. De-duplication reduced this to 22. Articles not relating to Orthopaedic surgery were excluded (15). Following inclusion/exclusion criteria, seven published articles were considered (Fig. 1). Demographics of the seven studies are shown in Table 1.

Scoring of CTA studies
A 5-point scoring system to appraise CTA was developed by Wingfield et al. in 2015 [13]. 'The CTA Research Ranking Scale' assesses journal impact score, study types and number of participants. This was used to assess selected studies prior to further in-depth analysis. Table .2 gives an overview of 'The CTA Research Ranking Scale'.

Participants
264 participants across 7 studies. 178 Male and 86 Female. There was an average of 37.7(Range:14-100) participants per study. Expertise ranged from medical students to senior orthopaedic trainees and consultants. (Table 3) Amer et al. [20] randomized 100 medical students to learn carpal tunnel release surgery via video lectures or by using the Touch Sur-gery™ [20] application. Those who had observed or participated in Carpal tunnel release and used the app previously were excluded. The control group watched the presentation three times in one sitting. The intervention group completed the 'Carpal Tunnel Surgery' module three times on the app. The same standardised test of 21 multiple-choice questions was completed. The intervention group was asked to rate the app using a 5-point Likert rating scale (ranging from very poor to very good).

Summary of cognitive task analysis methodology and learning principles used
Bhattacharyya et al. [17] undertook a randomized controlled trial to evaluate the effectiveness of the Imperial Femoral Intramedullary Nailing Cognitive Task Analysis (IFINCTA) tool. A modified-Delphi technique was used to design a combined written and audio-visual tool in femoral intramedullary nailing. 22 medical students were randomized in two equal groups. The intervention group were taught using the CTA tool and the control group were given a standard operative manual. The students were scored on MCQs and how effectively manual steps were completed using finger swipes on the smartscreen.
Bhattacharyya et al. [18] carried out a randomized control trial to evaluate the effectiveness of CTA in knee arthroscopy. The cognitive task tool was developed and designed using the modified-Delphi method. It utilized written, visual video and audiological information simultaneously to train novices in this procedure. A double blind RCT was then undertaken to analyse its efficacy to train novices in diagnostic knee arthroscopy. An objective assessment using the Validated    14 Orthopaedic interns who evaluated the Touch Surgery app: interns completed a simulated ankle open-reduction, internal-fixation and lag screw fixation prior to attending an annual four-week bootcamp. Participants completed the learning module, and then multiple-choice questions (Pass-mark 70%). They were required to pass the exam and allowed multiple attempts if required. A post-course survey was provided to participants on completion.
Logishetty et al. [19] developed procedural steps for Anterior Approach Total Hip Arthroplasty (AA-THA) via a modified-Delphi technique using 4 expert arthroplasty surgeons. 36 surgical residents were block randomized in two equal groups (residents who had previously observed or performed AA-THA were excluded). The intervention group were given online access to the cognitive tool and the control group were given a standard operation manual for this procedure.
Sugand et al. [22] randomized 27 medical students to evaluate the training effect of the touch surgery application in femoral intramedullary nailing. The students completed a pre-module questionnaire, a test module, of which they had six attempts each and a post-module MCQ. Scores were given for decision making, swipe interactions, and time taken to complete steps. Percentage total scores were calculated.
Sugand et al. [23] attempted to validate Touch Surgery™ for Intramedullary Femoral Nailing (IFN). As per Sugand's previous study [22], the procedure was divided into four modules. Real-time objective performance data was obtained and stored from the participants primary attempt. This was used to assess construct validity. A post-study questionnaire using the Likert scale was used to assess face and content validity. (Table 3) All studies found objective or subjective benefits in using CTA. Objectively, in the study by Amer et al. [20], the intervention group scored on average 89.3% (±6.0%) compared to 75.6% (±8.7%) on the 21 question post-study standardised test.

Assessment and study results
Both studies by Bhattacharyya et al. [17,18] found that the intervention group scored higher than the control on CTA's for f for both knee arthroscopy and femoral nailing. For the arthroscopic Knee CTA, the ASSET score was on average 19.5 ± 3.7 for the intervention group, and 10.6 ± 2.3 for the control. For the INFINCTA tool, intervention group participants reported median post test score improvements of 20% in Patient positioning and preparation, 21% in Femoral Preparation, 10% in Proximal locking and 19% in Distal Locking (in comparison to control group participants).
The study on CTA and AA-THA by Logishetty at al [19] found cognitively trained participants were on average 35% faster, made 69% fewer errors in instrument selection, and required 92% fewer prompts. They also were more accurate with acetabular cup orientation.
Both studies by Sugand et al. [22,23] showed benefits of CTA based simulation for intramedullary femoral nailing. On assessing the effect of CTA on training [22], Sugand found that novice participants improved (following CTA based simulation) by 83% in Patient positioning and preparation, by 94% in Femoral canal preparation, by 90% in Proximal locking 90% and 89% in Distal locking and closure. To demonstrate construct validity [23] experts outperformed novices in each module by 32.5% in Patient positioning and preparation, 31.5% in Femoral canal preparation, by 22.5% in Proximal locking and by 17% Distal Locking and closure. Levin et al. [21] did not contain any objective measure of CTA. Subjectively, participants from Amer et al. [20] rated content validity, quality of graphics, ease of use, and usefulness to surgery preparation as very high. Bhattacharyya et al. [17,18], found that participants agreed the cognitive task analysis learning tool was a useful training adjunct to learning in the operating room. Over 90% of participants found to tool easy to use and enjoyed using it. Levin et al. [21] found that 10/14 participants believed using CTA improved baseline understanding, 9/14 believed learning was accelerated and 8/14 felt the procedure was easier to learn as a result of this.
All participants in the AA-THA study by Logishetty et al. [19] found the CTA tool useful to understand key technical steps, decision making processes, highlighting errors, and easy to use. 34/35 enjoyed using the tool.
When validating Touch Surgery™ for intramedullary femoral nailing, Sugand et al. [23] found that both junior and senior cohorts rated the face validity, quality of graphics, willingness to use the app, usefulness for preoperative rehearsal as good or very good. Experts also rated the content validity as good.

Discussion
The use of CTA within orthopaedic training is relatively novel. 6/6 studies show objective benefits when using CTA (one study did not use objective assessment). They suggest CTA enhances performance and efficiency in orthopaedic training. In the randomized controlled trials by Bhattacharyya et al. [17,18] and Logishetty et al. [19] expert created CTA tools (using the Delphi model) have been shown to clearly improve participants ability to complete procedures in trial scenarios. The creation of CTA tools for individual procedures can open the door to multi-centre collaborations for other procedures and potential inclusion in to training programs. Studies by Sugand et al. [22,23], and Amer et al. [20] support objective and subjective benefits in CTA based simulation training.
Using CTA the trainee is able to work in a safe, protected environment, with minimal restrictions and at a time and place which suits them. It is inexpensive, web-based and accessible that allows repetitive practice which is the cornerstone of simulation training. Trainees can progress faster through the initial phase of the Sigmoid learning curve [24]. This, a concept of mathematical psychology, follows the learner from unfamiliarity to mastery of a skill. Initially progress is slow, however with intentional practice, traction is gained, and one enters the stage of 'hypergrowth' (where learning is exponential) and then subsequent mastery [24]. The aim of CTA is to propel the novice learner into hypergrowth prior to getting sustained theatre experience, thereby improving efficiency of their operating theatre training time. Furthermore, this enhances patient safety as trainees are more equipped with knowledge on the technical skills and potential errors before they perform a procedure for the first time on patients [17,19].
By undertaking CTA procedures, the trainee is enabled to progress through unconscious incompetence, conscious incompetence and potentially reach the stage of conscious competence. They, therefore, enter the learning process on patients at a higher point on the Broadwell learning curve [25].
Studies have estimated that 70% of vital steps can be missed out when taught by experts [13]. This is partially attributed to expert surgeons being in the unconscious competence stage of teaching. CTA provides an opportunity to rectify this flaw in traditional methods of training by being thorough and systematic.
In the current unprecedented situation due to the COVID-19 pandemic, CTA based simulation may find an increasing role in standardised orthopaedic training. This can be practiced remotely, repeatedly, with no human contact. The encouraging results of this systematic review can pave the way for future CTA based learning tools in other areas of orthopaedics, eventually leading to formal CTA learning programs for orthopaedic trainees worldwide.

Limitations
All studies reported thus far have not analysed transfer validity. It is important to evaluate whether the beneficial results viewed in the simulation setting translates to patients in the operating theatre.
There may be a role for the assessment of soft skills such as communication and leadership to be integrated into CTA.
As CTA based simulation is a developing topic there are a limited number of studies available to include in this review. Secondary to this, there is no clear outcome measure to evaluate CTA use between different procedures. Variance in expertise between participants can make comparison between CTA based training and conventional methods difficult. The positive findings of all studies mean publication bias must be considered as a potential limitation. Finally, registration of the work was completed on Research Registry which is a smaller sized registry compared to Prospero.
Despite limitations, CTA is a simulation tool which helps the novice learner develop a standardised level of competence in a procedure prior to doing it for the first time on a patient. In healthcare systems around the world today, the availability of such a tool is highly desirable.

Future work
Future work should focus on transfer validity of CTA to the operating room, creation of CTA based training tools for more procedures, and development of a clear, objective outcome measure. Once established, the incorporation of CTA into formal teaching curriculums would attempt to resolve some of the limitations of surgical training.

Conclusion
The current attempts to use CTA and other simulation methods, are secondary to strained health services, time pressures and new legislation reducing the time orthopaedic trainees can spend in the operating theatre. The innovation of CTA in Orthopaedics is a systematic, cost effective and easily accessible method of training that allows us to tackle the challenges faced by our future surgeons.

Provenance and peer review
Not commissioned, externally peer reviewed.

Ethical approval
No ethical approval was required.

Funding
No funding was required.

Consent
None.