Intersurgeon Variability in Local Treatment Planning for Patients with Initially Unresectable Colorectal Cancer Liver Metastases: Analysis of the Liver Expert Panel of the Dutch Colorectal Cancer Group

Background Consensus on resectability criteria for colorectal cancer liver metastases (CRLM) is lacking, resulting in differences in therapeutic strategies. This study evaluated variability of resectability assessments and local treatment plans for patients with initially unresectable CRLM by the liver expert panel from the randomised phase III CAIRO5 study. Methods The liver panel, comprising surgeons and radiologists, evaluated resectability by predefined criteria at baseline and 2-monthly thereafter. If surgeons judged CRLM as resectable, detailed local treatment plans were provided. The panel chair determined the conclusion of resectability status and local treatment advice, and forwarded it to local surgeons. Results A total of 1149 panel evaluations of 496 patients were included. Intersurgeon disagreement was observed in 50% of evaluations and was lower at baseline than follow-up (36% vs. 60%, p < 0.001). Among surgeons in general, votes for resectable CRLM at baseline and follow-up ranged between 0–12% and 27–62%, and for permanently unresectable CRLM between 3–40% and 6–47%, respectively. Surgeons proposed different local treatment plans in 77% of patients. The most pronounced intersurgeon differences concerned the advice to proceed with hemihepatectomy versus parenchymal-preserving approaches. Eighty-four percent of patients judged by the panel as having resectable CRLM indeed received local treatment. Local surgeons followed the technical plan proposed by the panel in 40% of patients. Conclusion Considerable variability exists among expert liver surgeons in assessing resectability and local treatment planning of initially unresectable CRLM. This stresses the value of panel-based decisions, and the need for consensus guidelines on resectability criteria and technical approach to prevent unwarranted variability in clinical practice. Supplementary Information The online version contains supplementary material available at 10.1245/s10434-023-13510-7.

Methods. The liver panel, comprising surgeons and radiologists, evaluated resectability by predefined criteria at baseline and 2-monthly thereafter. If surgeons judged CRLM as resectable, detailed local treatment plans were provided. The panel chair determined the conclusion of resectability status and local treatment advice, and forwarded it to local surgeons. Results. A total of 1149 panel evaluations of 496 patients were included. Intersurgeon disagreement was observed in 50% of evaluations and was lower at baseline than followup (36% vs. 60%, p < 0.001). Among surgeons in general, votes for resectable CRLM at baseline and follow-up ranged between 0-12% and 27-62%, and for permanently unresectable CRLM between 3-40% and 6-47%, respectively. Surgeons proposed different local treatment plans in 77% of patients. The most pronounced intersurgeon differences concerned the advice to proceed with hemihepatectomy versus parenchymal-preserving approaches. Eighty-four percent of patients judged by the panel as having resectable CRLM indeed received local treatment. Local surgeons followed the technical plan proposed by the panel in 40% of patients. Conclusion. Considerable variability exists among expert liver surgeons in assessing resectability and local treatment planning of initially unresectable CRLM. This stresses the value of panel-based decisions, and the need for consensus guidelines on resectability criteria and technical approach to prevent unwarranted variability in clinical practice.
Local treatment (e.g., surgery, ablation) is the only potentially curative treatment for patients with colorectal cancer liver metastases (CRLM). Survival rates of 30-50% have been reported for patients with initially unresectable CRLM who received local treatment. 1 Results from clinical trials show that 11-57% of patients with initially unresectable CRLM convert to resectable CRLM after downsizing by systemic therapy. 2 A major complicating factor in the interpretation of these results is the lack of consensus on criteria for (un)resectability of CRLM. Consequently, there is a subjective and therefore variable component in the decision-making process, which is also highly dependent on the multidisciplinary treatment options in different treatment centres. This is illustrated by a previous study in which experienced liver surgeons were asked to choose a treatment strategy in ten different CRLM patients, where disagreement on therapeutic strategies was observed in most cases. 3 Another key issue is that not all patients who are eligible for local treatment of CRLM are referred to dedicated liver centres to be offered this option. [4][5][6][7] Previous studies have shown that local treatment rates differ according to the treatment setting, to the potential detriment of patients who are not treated in liver-dedicated centres. 8,9 Lack of resection criteria and low referral rates could be resolved by using easily accessible online expert panels, which, according to two retrospective studies, results in higher rates of patients eligible for local treatment. 10,11 However, disagreement among experienced liver surgeons was also present in liver expert panels when assessing resectability in clinical trials: 12,13 disagreement was observed in 52% of the resectability assessments as reported by a previous evaluation of the Dutch Colorectal Cancer Group (DCCG) liver expert panel. 13 The online DCCG liver expert panel, consisting of experienced liver surgeons and abdominal radiologists, prospectively assessed (un)resectability in the CAIRO5 study, in which the currently most effective induction regimens in patients with initially unresectable CRLM are compared. 14 The current study is an extension of the previous evaluation and was conducted for three reasons. Firstly, the increased sample size and experience of the DCCG liver expert panel allows for a more robust analysis. Secondly, while the variability in resectability assessments among surgeons at the level of individual patients has been investigated, the general variability remains unknown between individual surgeons in assessing resectability. Thirdly, the DCCG liver expert panel also provides technical local treatment plans for patients evaluated as having resectable CRLM, and the preferences of surgeons for certain strategies have not been examined before. Strategies to achieve clearance of all CRLM include one-stage minor or major liver resections, combinations of local resections with tumour ablation or stereotactic body radiation therapy (SBRT), two-stage resections with or without preoperative portal vein embolisation, and/or associating liver partition and portal vein ligation for staged hepatectomy (ALPPS).
The aim of the current study was to assess the variability among liver surgeons: those participating in the DCCG liver expert panel, in resectability assessments and in local treatment planning in patients with initially unresectable CRLM receiving induction systemic therapy.

Patient Selection
Patients were selected from the CAIRO5 study (NCT02162563), a randomised phase III trial of the DCCG, comparing the currently most effective systemic induction regimens in patients with initially unresectable colorectal cancer liver-only metastases. [14][15][16] Patients randomised between the start of the study in November 2014 and April 2021 were selected for this subset analysis. The CAIRO5 study was conducted in accordance with the standards of Good Clinical Practice and the Declaration of Helsinki. The CAIRO5 study was approved by the local ethics committees and all patients provided written informed consent before the start of the study.

Resectability Assessment
The DCCG online liver expert panel, currently consisting of 15 experienced liver surgeons and 3 abdominal radiologists, evaluated unresectability at baseline and resectability every 2 months during follow-up. Given the lack of consensus on (un)resectability criteria, baseline resectability criteria were established by liver surgeons from the expert panel, resulting in clear entry criteria allowing for a homogeneous study population. CRLM was considered unresectable at baseline if an R0 resection could not be achieved with surgical resection only in one stage. Resectability (or amenability to local treatment) during follow-up was based on more liberal resection criteria, since all established local treatments were allowed (i.e., ablation, two-stage surgery, portal vein embolisation) to achieve clearance of all CRLM while preserving a functional liver remnant. The design of the panel has previously been described in detail. 13 In short, after evaluation by one radiologist, each CT scan with panel radiology report [including patient's age, number of treatment cycles, location and resection (yes/no) of primary tumor] was evaluated by three randomly selected panel surgeons, who voted individually on the following categories: resectable, potentially resectable after further induction systemic treatment, or permanently unresectable. Permanently unresectable was selected when there was expected failure of achieving a complete R0 resection or ablation of all CRLM at any moment during systemic therapy. If no consensus (i.e., same category selected by all three surgeons) was obtained, two additional surgeons were consulted by the panel chair (consecutively T.v.G., J.K., R.S.) and the majority vote was accepted by the panel chair as the final vote. If the vote was 2 vs 2 vs 1, the panel chair determined the vote. During follow-up, patients with permanently unresectable CRLM as the final vote were not re-evaluated by the panel. If panel surgeons voted for resectable CRLM, they were asked to provide a detailed technical plan for their local treatment approach. The following items were included in the technical plan: modality [wedge resection/segmental resection/ablation/(extended) hemihepatectomy] specified per segment, one-or two-stage approach, portal vein embolisation (no/yes + left/right). The panel chair decided on one final technical plan, based on the plans of the other panel surgeons. The panel conclusion was forwarded to the referring hospital, along with the proposed local treatment advice if the CRLM was resectable.

Outcomes
The degree of agreement among surgeons has previously been described in detail. 13 In short, minor disagreement was defined as a panel evaluation in which at least one panel surgeon assessed the CRLM as potentially resectable and at least one other surgeon in the same panel voted for resectable CRLM, or a combination of potentially resectable and permanently unresectable CRLM. Major disagreement was defined as a panel evaluation in which at least one panel surgeon assessed the CRLM as resectable and at least one other surgeon voted for permanently unresectable. Intersurgeon variability per individual patient applies to the differences among surgeons in the assessment of the same patient. Intersurgeon variability in general refers to differences among surgeons considering all patients they assessed. In the latter analyses, surgeons with fewer than 10 observations were excluded and the evaluations of four former panel surgeons were included as well. Local treatment plans were considered similar if all surgeons proposed the same type of treatment as presented in Supplementary Table S1, and different if at least one surgeon proposed a different plan. No distinction was made between which segment was treated with which modality because of the clinical relevance (e.g., if one surgeon proposed a wedge resection of a lesion in segment II and ablation of a lesion in segment IV, and the other surgeon proposed the reverse, the plan was considered similar). Complete local treatment was defined as complete R0/R1 resection or ablation of all CRLM. SBRT for a remaining lesion was allowed to qualify for complete local treatment.

Statistical Analysis
Continuous variables were displayed as median with interquartile range (IQR) and categorical variables as counts and percentages. Differences between groups were analysed using Pearson's chi-square test and Fisher exact test, as appropriate. A p value ≤ 0.05 was considered statistically significant. All analyses were performed in R (version 4.0.3).
The degree of agreement among resectability assessments per evaluation point is presented in Supplementary Fig. S1. Any intersurgeon disagreement (minor or major) was lower at baseline compared with follow-up panel evaluations [179 (36%) vs. 392 (60%), p < 0.001]. Major intersurgeon disagreement was lower at baseline compared with follow-up panel evaluations [4 (1%) vs. 111 (17%), p < 0.001]. No difference was observed in overall disagreement over time, neither when major and minor disagreement were grouped together as any disagreement (p = 0.091), nor when split into minor and major disagreement (p = 0.370) ( Supplementary  Fig. S2).

Intersurgeon Variability per Individual Patient in Technical Local Treatment Planning
The panel considered 324 (66%) patients to have resectable CRLM (Fig. 1). In 75 (23%) of these patients, the panel surgeons proposed a similar local treatment plan, which was adopted by the chair in 91% (68/75). In 249 (77%) of these patients, surgeons proposed different local treatment plans. A majority of surgeons proposing a similar plan was present in 153 (61%) of these patients. The chair adopted the majority's plan in 110/153 (72%) patients, followed one of the other panel surgeons' plans (i.e., minority) in 25/153 (16%) patients and proposed a completely different plan in 18/153 (12%) patients. In the absence of a majority [96/249 (39%) patients], the chair followed one of the proposed plans in 76 patients (79%) and created a new plan in 20 patients (21%).
The differences between the surgeons' plans are detailed in Table 2. The major differences in local treatment were: one or more surgeons proposing a parenchymal-preserving approach with local resection and/or ablation versus one or more surgeons proposing a hemihepatectomy (± local resection/ablation) in a one-stage approach [75/249 patients (30%)] or in a two-stage approach [31/249 patients (12%)], and a one-stage versus two-stage hemihepatectomy (± local resection/ablation) [68/249 patients (27%)].

General Intersurgeon Variability in Resectability Assessments
At baseline, there were 1836 resectability assessments by panel surgeons in 494 patients. In 1400 (76%) resectability assessments surgeons voted for potentially resectable CRLM, in 91 (5%) for resectable CRLM and in 345 (19%) for permanently unresectable CRLM at baseline. Votes for permanently unresectable CRLM at baseline decreased from 33 to 13% in the first to last 20% of resectability assessments, respectively. The resectability assessments per surgeon at baseline are depicted in Fig. 2. Votes per surgeon for resectable CRLM ranged between 0 and 12% (surgeons M and D) and for permanently unresectable CRLM between 3 and 40% (surgeons C and E). During follow-up, there were 2728 resectability assessments by panel surgeons (in 481 patients at first, 144 at second, 27 at third and 3 at fourth follow-up). In 917 (34%) resectability assessments surgeons voted for potentially resectable CRLM, in 1185 (43%) for resectable CRLM and in 628 (23%) for permanently unresectable CRLM during follow-up. Votes for permanently unresectable CRLM during follow-up decreased from 30 to 19% in the first to last 20% of resectability assessments, respectively. The resectability assessments per surgeon during follow-up are depicted in Fig. 3. Votes for resectable CRLM ranged between 27 and 62% (surgeons K and M) and for permanently unresectable CRLM between 6 and 47% (surgeons K and D).

Adherence by the Local Treatment Centre to the Panel Advice
Local treatment of CRLM was performed in 271 of 324 (84%) patients evaluated by the panel as having resectable CRLM and this was complete (R0/R1 resection and/ or ablation) in 235 (87%) patients. CRLM of 16 patients (5%) was considered perioperatively unresectable and 37 (11%) received no local treatment. Reasons for decisions to withhold local treatment are presented in Table 3. Six out of a hundred and seventy patients (4% of all unresectable CRLM) received local treatment against panel advice and 10/170 (6%) patients before the panel assessment of which 12/16 (75%) local treatments were complete. This resulted in a rate of 58% (287/494) for attempted local treatment and 50% (247/494) for complete local treatment.
Local treatment was performed after a median of 6 (range 3-15) cycles of systemic therapy and after a median time of 48 days (range 0-243) after the panel conclusion was reached.
The type of local treatment in patients who received complete local treatment is shown in Supplementary Table S1. The most commonly performed strategy was a combination of local resection and ablation (41%), followed by a one-stage (extended) hemihepatectomy combined with local resection and/or ablation (13%). A twostage approach was performed in 49 patients (21%), of  which 12 were formal ALPPS procedures. Four patients with complete local treatment also received radiotherapy for a lesion which could not be treated with resection or ablation.
In 94/235 (40%) patients, the proposed treatment plan by the panel chair was followed by the local surgeon in the referring hospital. The differences between performed and advised local treatment are shown in Table 4. In 35% of patients, the local surgeon chose a more parenchymalpreserving approach by performing a combination of local resection and/or ablation instead of the advised hemihepatectomy (± local resection/ablation) in a oneor two-stage approach [31/141 (22%) and 18/141 (13%) patients, respectively]. The opposite was observed in 12%, where the panel chair advised a parenchymalpreserving approach while the local surgeon performed a hemihepatectomy (± local resection/ablation) one-or twostaged [9/141 (6%) and 8/141 (6%) patients, respectively].

DISCUSSION
This study showed considerable intersurgeon variability in resectability assessments and in technical local treatment planning for patients with initially unresectable CRLM receiving induction systemic therapy. A plausible explanation for the higher amount of disagreement at follow-up compared with baseline may be that the (un)resectability criteria at baseline were strictly defined but more liberal at follow-up. The increasing disagreement rate from the first to the last follow-up may be explained by the fact that patients with CRLM that are technically challenging to treat remain in the follow-up process. Adherence to resectability assessments was high, with 84% of patients who were considered to have resectable CRLM by the panel actually receiving local treatment. In contrast, the variability in technical treatment plans was high (77%) and adherence to the proposed technical local treatment plans by referring surgeons was low (40%).
The general intersurgeon variability in prospective resectability assessments of multiple surgeons over several years has not been investigated before. The observed differences are to be expected considering, first, the large variation in resection rates for CRLM between hospital types and regions, 8,9,17 and second, the lack of consensus on resectability criteria, with expanding indications and local treatment modalities, and intensification of systemic therapy during recent years, which is also reflected by the decreasing proportion of votes for permanently unresectable CRLM. Limitations in the surgeon's technical capacities or different views on the effectiveness of local ablation in certain subgroups of patients are possible explanations for different views on resectability. Apart from the variability in resectability assessments, a large amount of intersurgeon variability in technical plans for local treatment was observed. The most relevant difference is that some surgeons appeared to have a clear preference for a hemihepatectomy (± local resection/ablation), while others preferred a parenchymal-preserving approach with a combination of local resection and/or ablation. Part of the variability in technical plans may be explained by the preference of some surgeons for local or major liver resection over ablation due to the alleged higher risk of local   recurrence and potentially poorer oncological outcomes. However, unbiased high-quality evidence is lacking, and the non-inferiority of ablation to local resection is being investigated in ongoing randomised controlled trials. 18,19 Numerous strategies can be followed to achieve clearance of CRLM, and it is currently not known which strategy is the most beneficial for patients. A previous systematic review comprising retrospective studies demonstrated better perioperative outcomes without compromising oncological outcomes with the use of a parenchymal-preserving approach compared with a hemihepatectomy. 20 However, these results may not be generalisable due to the inability of ruling out selection bias, and the varying definitions of a parenchymalpreserving approach complicates the interpretation of the results. Upon completion of the CAIRO5 study, 14 it will be possible to evaluate a possible correlation between perioperative and oncological outcomes and the various strategies as proposed by the panel.
The large amount of intersurgeon variability reflects the complexity of defining local treatment strategies for patients with CRLM. Variability in clinical practice may reflect differences between patient characteristics or well-informed preferences of patients. 21 However, the strength of this study is that surgeons were randomly assigned to evaluate patients from a homogeneous trial population, therefore it is unlikely that the observed variability is caused by differences between patients. Hence, the observed variability should be considered unwarranted and efforts should be made to reduce this in order to ensure that all patients have the same probability of receiving curative-intent local treatment regardless of which hospital they are treated in. To reduce the unwarranted variability, consensus guidelines on resection criteria and technical approach are warranted, and the use of an expert panel should be advocated, which is supported by previous studies. [8][9][10][11][12][13]22 The short time between uploading imaging by the referring centres and reaching a panel conclusion shows that the use of an online expert panel does not cause a significant delay in treatment initiation, and is feasible. We suggest simplifying a future liver expert panel by focusing on the resectability assessment (resectable, potentially resectable, permanently unresectable) and either include all proposed plans instead of one final plan formed by the panel chair, or omit the technical treatment plans to reduce the workload without compromising the objective. Proposing technical treatment plans seems of limited value since a high variability among the panel surgeons and a low adherence to the plans by the referring surgeons was observed. The low adherence may partly be explained by the lack of information on volume or function of the future liver remnant for the panel, which may be available to the local surgeon and plays an important role in the choice of the technical approach. Additionally, the treatment plan may be influenced by the preference and condition of a patient or pre-or perioperative new findings.
Future research should be directed towards evaluating whether the level of disagreement and/or the complexity of treatment strategies correlate with clinical outcomes. In addition, further research is needed to determine whether panel evaluations may be supported by biological resection criteria, such as the consensus molecular subtypes and circulating tumour DNA, to select patients with CRLM who will derive the most benefit from local treatment.
ACKNOWLEDGMENT We would like to thank all patients and their families, and hospitals and their research teams, for participating in the CAIRO5 study. We would like to acknowledge the Netherlands comprehensive cancer organisation (IKNL) for their collaboration.
FUNDING The CAIRO5 study is supported by unrestricted scientific grants from Roche and Amgen. The funders had no role in the design, conduct, or submission of the study, nor in the decision to submit the manuscript for publication. We had complete access to all study data that support the manuscript.

DATA AVAILABILITY
The data that support the findings of this study are available from the corresponding author upon reasonable request. OPEN ACCESS This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.