Automated analysis of intraoperative phase in laparoscopic cholecystectomy: A comparison of one attending surgeon and their residents

Highlights • Comparing attending and residents’ intraoperative phase times in laparoscopic cholecystectomy.• Touch Surgery™ Enterprise provided analytics of intraoperative phase times.• Residents’ total times can significantly exceed the attending's.• Residents’ dissection of hepatocystic triangle times can significantly exceed the attending's.• This could suggest training need and validates time as a marker of performance.

OBJECTIVE: This study compares the intraoperative phase times in laparoscopic cholecystectomy performed by an attending surgeon and supervised residents over 10-years to assess operative times as a marker of performance and any impact of case severity on times.
DESIGN: Laparoscopic cholecystectomy videos were uploaded to Touch Surgery TM Enterprise, a combined software and hardware solution for securely recording, storing, and analysing surgical videos, which provide analytics of intraoperative phase times.Case severity and visualisation of the critical view of safety (CVS) were manually assessed using modified 10-point intraoperative gallbladder scoring system (mG10) and CVS scores, respectively.Attending and residents' times were compared unmatched and matched by mG10.
SETTING: Secondary analysis of anonymized laparoscopic cholecystectomy video, recorded as standard of care.
PARTICIPANTS: Adult patients who underwent elective laparoscopic cholecystectomy a single UK hospital.Cases were performed by one attending and their residents.
CONCLUSIONS: Residents' total operative and dissection of HCT times significantly exceeded the attending's unmatched cases and low-severity matched cases which could suggest training need, however, also reflects an expected assessment of competence, and validates time as a marker of performance.( J Surg Ed 80:994À1004.Ó 2023 The Authors.Published by Elsevier Inc. on behalf

INTRODUCTION
Total operative time has been used as a surrogate marker of performance in laparoscopic cholecystectomy. 1,2perative time is influenced by many factors, including patient sex, habitus, underlying pathology, and operative findings.In a retrospective study of 315 laparoscopic cholecystectomies performed by surgical residents in the United States (US), total operative time was significantly longer for junior residents (Post Graduate Year [PGY]1-3) compared to senior residents (PGY4-5), irrespective of the grade of assistant. 1][5][6][7][8][9][10] This process is time consuming and requires teams of expert assessors and as such, until recently, this technique has been used in the research setting only.More recently there have been attempts to digitize a similar process. 11t is now possible to digitally analyze surgical video using artificial intelligence to automatically identify intraoperative phases in laparoscopic cholecystectomy. 12,13sing operative time to examine an operation at a more granular level could assist in identifying aspects of cases that require either more focused training attention and/or further training needs.The aim of this study was to compare the total operative time and intraoperative phase timings of surgical residents and an attending surgeon over a 10-year period.

Dataset
The video dataset consisted of fully anonymized operative laparoscopic cholecystectomy videos by a single attending surgeon.Videos were uploaded to Touch Surgery TM Enterprise and anonymized by the RedactOR TM algorithm to ensure any remaining patient identifiable information was removed.RedactOR TM detects portions of the video where the camera is outside of the patient and pixelates the video stream in real-time on upload to prevent the recording of any potentially identifiable information.Operations were undertaken under the care of a single attending surgeon at a UK district general hospital during their first 10-years of practice.Operations were performed by either the attending or their residents.Only full videos of complete cases were included, there were no exclusions of complete cases.Cases had been identified as either attending or resident cases by the attending surgeon at source and saved as such.Resident cases were defined as pooled cases whereby residents were able to log their cases as either "supervised-trainer scrubbed" (STS), "supervised-trainer unscrubbed but in theatre" (STU) and "performed" (P) using the Joint Committee on Surgical Training and Intercollegiate Surgical Curriculum Program trainee supervision codes (See supplementary table 1). 14,15This would therefore allow the attending to log the case as Training a more junior trainee (T).The individual codes STS/STU/ P were not recorded by the attending.This video dataset is owned by the attending surgeon, there is no patient/ clinical data.Patients provided voluntary, informed consent for their intraoperative video to be used for education and training purposes.

Touch Surgery TM Enterprise
Touch Surgery TM Enterprise by Digital Surgery TM Ltd, a Medtronic company, is a combined software and hardware solution for securely recording, storing, and analysing surgical videos.Uploaded videos are automatically anonymized by the RedactOR TM algorithm and automatically broken down into phases.Touch Surgery TM phase identification, developed by Digital Surgery Ltd, is based on the state-of-the-art phase recognition models in the literature, which have been previously applied to laparoscopic cholecystectomy, 12 cataract 14 and total knee replacement surgery. 15The latest model by Digital Surgery TM Ltd achieves a 96% accuracy in detecting phase transitions in laparoscopic cholecystectomy. 16Qualified annotators, trained on surgically validated guidelines, quality-assured the model outputs.
Touch Surgery TM defines the surgical workflow phases as the following five operative phases, through liaising with key opinion leaders and consulting the literature: The following phases are identified: P1 -Port insertion/gallbladder exposure P2 -Dissection of the Hepatocystic Triangle P3 -Ligation and division of cystic artery and cystic duct P4 -Gallbladder dissection P5 -Specimen removal and removal of ports

Laparoscopic Cholecystectomy Severity Score
Grading scores often require clinical, biochemical, and radiological data in addition to intraoperative findings and are often used to predict conversion to an open operation.The G10 score is a 10-point intraoperative gallbladder scoring system, which uses an interoperative severity grading of cholecystitis as a marker of technical difficulty (Supplementary table 2), where 0 is the least challenging and 10 the most challenging. 171 point is scored for a Body Mass Index (BMI) of more than 30.This point was omitted in our study, due to a lack of clinical information.Total scores, out of a maximum of 9, were considered in this study and are referred to as "modified G10" (mG10).

Visualisation of the Critical View of Safety (CVS)
Appropriate visualization of the CVS prior to ligation and division of the cystic duct and artery are crucial for preventing common bile duct injuries. 18An intraoperative score (Supplementary table 3) was devised, which identifies the three domains required to achieve the CVS: Two structures connected to the gallbladder, cystic plate clearance and hepatocystic triangle (HCT) clearance, with scores of 0-2 awarded in each domain.A score of 4 represented adequate visualization.However, when reviewers were asked if it was safe to divide, there was no significant agreement across adequate and inadequate scores. 19For this study, we used the total score only to indicate the visualization of the CVS.

Data Extraction: Manual Assessment of Surgical Video
After videos were uploaded to Touch Surgery TM Enterprise, the videos were assessed for completeness and manually scored for intraoperative severity using the mG10 and visualisation of the CVS scores.All videos were assessed by GH, a clinical research/training fellow and general surgery resident who holds Membership of the Royal College of Surgeons and APC a consultant surgeon who holds Fellowship of the Royal College of Surgeons.The first 50% of complete video cases were also assessed by an attending surgeon and interrater reliability (IRR) analysis was performed.

Data extraction: Automated Identification of Intraoperative Phase
Touch Surgery TM Enterprise provided a .csvfile with case and phase durations in hours and minutes (hh:mm).
Operative times were converted to minutes (decimal).

Intraoperative Phase Times and mG10 and CVS Score
A higher mG10 score represents a more challenging cases and a higher CVS score represents a better visual visualization of the CVS.In resident cases a positive relationship between rater assessed mG10 and CVS scores (r 2 =0.071 p=0.034) was found, but not in the attending's cases (r 2 =0.023, p=0.136), as shown in Figure 1. Figure 2 shows the intraoperative phase times which have significant positive linear relationships with the mG10 scores.Positive relationships were found between mG10 score and P2 (r 2 =0.098 p=0.012) in resident cases, but not in the attending's cases.In the attending's cases positive linear relationships were found between mG10 score and P1 (r 2 =0.311 p<0.001),P4 (r 2 =0.047 p=0.032) and total operative times (r 2 =0.147 p<0.001).Figure 3 shows intraoperative phase times which have significant relationships with CVS score.In the attending's cases significant negative relationships were found between CVS score and P2 (r 2 =0.101 p=0.001),P3 (r 2 =0.104 p=0.001) and total operatives times (r 2 =0.134 p<0.001).There were no significant relationships found between CVS score and operatives times in resident cases.

Unmatched Cases
Table 1 shows the median, IQR, range and Mann-Whitey U statistics of residents' and the attending's total operative and intraoperative phases times of unmatched cases.Figure 4 shows box and whisker plots for residents' and the attending's total operative and intraoperative phases times of unmatched cases.Significant differences were found between residents' and the attending's total operative times (U=3199, p<0.001),P2 (U=1624 p<0.001) and P4 (U=2295, p=0.010).No significant differences were found between the remaining intraoperative phase times.

Case Matching
Attending and resident cases were matched by mG10 score.No significant differences in intraoperative times were found when cases were matched by mG10 scores of 0, 3 and 4.There were insufficient cases to match for mG10 scores of 5 and 6.Table 2 shows the median, IQR, range and Mann-Whitey U statistics of residents' and the attending's total operative and intraoperative phases times of cases matched by mG10 score.Figure 5 shows the box and whisker plots for residents' and the attending's total operative and intraoperative phases times with matched cases for mG10 scores 1 and 2.
Modified G10 Score of 1 Significant differences were found between residents' and the attending's total operative times (U=180, p=0.001) and phase time, P2 (U=189 p=0.002).No significant differences were found between the remaining intraoperative phase times.
Modified G10 Score of 2 Significant differences were found between residents' and the attending's total operative times (U=99, p<0.001) and phase times, P1 (U=1635 p=0.032) and P2 (U=105,   p<0.001).No significant differences were found between the remaining intraoperative phase times.

Intraoperative events
During manual review of the laparoscopic cholecystectomy videos any intraoperative events which could potentially increase operative times were identified.Cases with total or intraoperative phase times outlying the IQR were reviewed for documented intraoperative events.Five outlying resident cases were identified.One case required considerable adhesiolysis, a further case was prolonged because of a minor cystic duct injury (requiring no further intervention) and finally 3 cases were impacted by

CONCLUSIONS
Touch Surgery TM Enterprise anonymizes, stores, and breaks down recorded procedures into intraoperative phases and timestamps for analysis, allowing the investigation of outlying cases and intraoperative phases.The anonymized video dataset used in this study had no accompanying clinical information, therefore the mG10 score was used in this study to manually assess the clinical severity of each case.This score is influenced by the presence of intra-abdominal adhesions and the appearance of the gallbladder, 17,22 which can be secondary to previous surgery, previous infection (e.g., cholecystitis, cholangitis) and/ or pancreatitis and can present challenges to abdominal access and laparoscopic placement.Chronic and repeated acute infection/inflammation can fibrose the gallbladder wall obscuring dissection planes, particularly between the gallbladder and liver. 23Potentially any of the included cases could have been underscored by 1 point, as BMI was not available.Both intraabdominal and subcutaneous fat can challenge laparoscopic surgery, and neither could be reliably commented on in this study.This clinical information would normally be available to the surgeon prior to operating and may influence an attending's decision to allow their resident to perform whole or part of an operation.This study found that the median mG10 scores of residents' cases was lower than attending cases.However, both media scores were of low severity.This study could be underestimating this difference if the attending's case selection included high BMI cases.The data on the previous experience of the resident, training grade and level of attending supervision was not available and could contribute to type II errors.It is likely that residents with less experience will have more attending supervision and have longer operating times compared to more experienced residents.This study did, however, confirm that over a 10-year period resident total operative time was significantly longer than an attending performed surgery.This was similarly reflected in the important intraoperative phases: dissection of the HCT (P2) and gallbladder dissection (P4).In low-severity (mG10=1,2) matched cases residents perform significantly longer total operative times, laparoscopic port insertion/gallbladder exposure (P1) and dissection of HCT (P2) phase times compared to their attending.This is perhaps not surprising as dissection of the HCT is considered the most crucial phase of laparoscopic cholecystectomy, as isolating the cystic artery and cystic duct, and achieving the CVS are considered crucial to reducing the risk of CBD injury 18 and performing a technically safe and robust operation.HCT phase has been found to have a higher risk of intraoperative error by attendings. 8This study defined and classified surgical errors as consequential and inconsequential. 8Consequential errors are any action or omission that resulted in a negative consequence or increased the time of the surgical procedure by necessitating a corrective action.Inconsequential error was described as an action or omission that increased the likelihood of negative consequence and under slightly different circumstances could have had a consequential effect. 8This delayed video analysis study of 200 laparoscopic cholecystectomy cases performed by attending surgeons in the UK identified significantly higher frequency of surgical errors in task zone 2 (6.5 §5.4), which includes dissection of HCT and ligation and division of cystic structures, compared with task zone 1 (2.9 §2.8, p<0.001) and task zone 3 (5.1 §3.9 p<0.05). 8In the simulated setting surgical residents were found to have an error probability of 7.7% in "dissection of the cystic artery", 24 which was lower than "division of the cystic artery and duct' (15.2% and "separation of the gallbladder from the liver bed" [5.6%]). 24This study of 60 exvivo porcine laparoscopic cholecystectomies performed by 60 surgical residents in their first year of surgical training also showed significant variation in the number of errors enacted by the subjects. 24Whilst, interpreting the findings of these studies in relation to this study should be done with caution, the definitions of intraoperative phases or task zones are not directly comparable, with definitions overlapping across phases (e.g., task zone 2 for attendings in on study 8 includes both "dissection of the cystic artery" "division of the cystic artery and duct' in the other study. 24But the prolonged P2 intraoperative phase time in our study could be because of a higher rate of surgical errors by surgical residents compared to the attending.It is however interesting therefore that surgical residents achieved significantly better visualization of the CVS in higher mG10 cases although it did take residents longer to achieve the CVS in cases potentially because of these surgeons being more cautious in these more challenging surgical cases.For the attending's cases, longer P2 was also related to significantly clearer visualization of CVS.These findings of longer P2 times could also reflect surgeons' knowledge and experience that more challenging cases have a higher risk of bile duct injury and therefore additional care and attention has been spent to achieve the CVS.Making assumptions around cause and effect in these cases is difficult as there remains a huge number of intraoperative and interpersonal confounding factors, therefore, it is important not to label the cause of a prolonged total and P2 intraoperative time phase time as just the result of less developed skill or competence for example.The increase in intraoperative times is undoubtedly multifactorial and beyond the scope of this study to details further but is likely the result of relative inexperience and exposure which could be improved by additional training. It is equally important to consider the matched cases where no significant differences were found between residents' and attending operative times in cases matched by mG10 scores of 0, 3 and 4.There were insufficient cases with higher mG10 scores for analysis, which likely reflects the national disease severity distribution at the time of data collection and/or individual operator patient selection.A mG10 score of 0 reflects the most straightforward of cases.No significant differences were found between resident and attending operative times.Nevertheless, without understanding resident experience and attending supervision in resident cases it is difficult interpret these results.However, it is broadly acknowledged that laparoscopic cholecystectomy presents a spectrum of challenge from cases requiring basic laparoscopic skills to those cases requiring advanced laparoscopic skills and biliary surgical expertise.There was also no significant difference between resident and attending operating times for moderately challenging cases, (e.g., mG10 3-4), Again conclusions are difficult to extrapolate but this could signify that those cases are challenging for everyone and are somewhat related to whether the resident has autonomy throughout the case.This could also suggest that there is an optimal level of case difficulty for training, with a training potential present in cases with a mG10 score of 2-3.
This study also reported a positive linear relationship between mG10 score and time for port insertion/exposure (P1), gallbladder dissection (P4) and total operative time for the Attending's cases.This finding was not seen in the resident group.However, there was a positive linear relationship between mG10 score and dissection of HCT (P2) in the resident case cohort.It is possible that this relationship was not seen in the resident cases due to attending case selection.Conversely, these findings could also suggest that an increase in case severity presents different challenges to residents and attendings or increase anxiety in residents promoting over caution.It is not possible to identify the reasons for the difference in intraoperative phase times from the findings of this study.The findings may support the view that the dissection of HCT is a challenging phase to master and focusing cognitive, technical, and perhaps simulation skills training in this intraoperative phase could improve resident's dissection of HCT phase time, which in turn could improve total operative times.
In this study, residents tended to achieve greater visibility of the CVS in more challenging cases, whilst similar CVS scores were achieved throughout the spectrum of attending's cases.It could be that residents focused more on achieving adequate CVS when cases were more challenging because of the increased associated risk.This may reflect resident's fear and apprehension around this area of dissection.Despite this, there was no significant differences identified between median resident CVS score and those of the attending cases.
The authors acknowledge that this study has limitations.The anonymized video dataset contains no clinical data, nor data on the grade/experience of the operating surgical resident, assisting surgeon or supervision code (STS/STU/P) for the case.Whilst case severity was inferred using the mG10 score, it was not possible to analyse other perioperative risk factors other perioperative factors that influence case severity or postoperative complications.Pooled analysis of all residents was undertaken on the assumption that residents have a similar level of competence and experience and therefore are likely to have the similar training needs and baseline competence.Whilst this may have introduced some type II error, particularly if an intraoperative phase was largely performed by the attending, it does provide a useful comparable dataset from which conclusions can be drawn.
It would not be appropriate to assume the extremes that the resident has full autonomy in all cases, nor performed a negligible proportion of all cases.It is likely that there is a mix which reflects the diversity of supervision that exists in a population of residents and is appropriate for training cases.The JCST and ISCP trainee supervision codes gives clear guidance on the definitions of supervision and the logging of cases; based on the P, STS, STU criteria it is presumed that significant components of each case were performed by the resident.Additionally, If a resident were assisting in a case this would be logged as such, in keeping with the supervision code Assisting (A): trainee scrubbed to assist, but not taking a leading part in the operation itself, e.g. in the deception, anastomosis.). 14,15Unfortunately, the detailed supervision code was not recorded at source.However, this dataset includes all the attending's in their 10-year practice, reducing the risk of a reporting bias and offering a representation of resident case selection at the time.As this video dataset is fully anonymized it is not possible to retrospectively identify the resident involved in the case.This data would have enriched the dataset and allowed more detailed conclusion but is limitation of secondary research.
Nonetheless, this study confirms residents had longer total operating time with a particular focus of additional time in the P2 phase.As discussed, broad conclusions are difficult, but we would suggest that identifying this area as a focus for more selective could result in an improved P2 performance time and reduce total operating time as a result.Utilization of this tool therefore works to identify specific intraoperative areas of training focus with which to target either individual residents if a video logbook was maintained or a broader selective course curriculum.The future utility for improved training programmes and identification of resident surgical strengths and weaknesses holds potential.This video dataset is the work of a single attending and their residents and can only infer interpretations on those included.At present this cannot be generalized to the wider population of residents and trainers.Future research should include prospectively collected video datasets with comprehensive operative and clinical data from multiple units to allow comparisons and gain richer insights.Additional data should be collected on resident experience and supervision codes to allow more granular assessment, which should support the development of deeper conclusions.There remains a significant governance argument for recording and storing all laparoscopic cases as visual operation notes.This would provide a huge potential future database and research resource for further work.

DISCLOSURES
GH received funding from Medtronic (Digital Surgery Ltd is a subsidiary of Medtronic) to undertake this work.
DS is an employee of Digital Surgery Ltd.
The remaining authors have no conflict of interest to declare.
This manuscript is being submitted for consideration of publication in Journal of Surgical Education as an original article.This manuscript has not been submitted for publication elsewhere.Data from this study was presented as SAGES 2021 as a poster presentation.

FIGURE 1 .
FIGURE 1. Scatter plots and linear regression analysis of G10 score and Critical View of Safety (CVS) score for (1a)residents and (1b) attending.* marks statistically significant results.

FIGURE 3 .
FIGURE 3. Scatter plots and linear regression analysis of Critical View of Safety Score (CVS) and operative times for (3.1a)Resident CVS score vs Total operative time.(3.1b) Attending CVS score vs Total operative time.(3.)2a Resident CVS score vs P2 operative time.(3.2b) Attending CVS score vs P2 operative time.(3.3a) Resident CVS Score vs P3 operative time.(3.3b) Attending CVS score v P3 operative time.* marks statistically significant results.

FIGURE 4 .
FIGURE 4. Box and whisker plot showing median, interquartile range and range of total and intraoperative phase operating times for residents and attending for all cases, unmatched.(T Resident, C Attending, P1 Port insertion/gallbladder exposure, P2 Dissection of the Hepatocystic Triangle, P3 Ligation and division of cystic artery and cystic duct, P4 Gallbladder dissection, P5 Specimen removal and removal of ports).* marks statistically significant results.

TABLE 2 .
A Comparison of Consultant and Trainee Total and Intraoperative Phase Operating Times for Cases Matched by Modified G10 (mG10) Score.
iatrogenic intraoperative gallbladder perforations.Similarly, the single outlying attending case resulted from an iatrogenic gallbladder perforation.There were no cases analysed that necessitated conversion to open cholecystectomy.