Development, validation and clinical impact of a prediction model for 6-month mortality in older cancer patients: the GRADE

Background: To develop, validate, and assess the clinical impact of a clinical score to predict a 6-month mortality risk among older cancer patients. Results: The mean age was 81.2 ± 6.1 years (women: 54%, various cancers, metastatic cancer: 45%). The score, namely the GRADE, included two geriatric variables (unintentional weight loss, impaired mobility), two oncological variables (cancer site, cancer extension), and exclusively supportive care. Up to a 14% risk of early death, the decision curves suggest that cancer treatment should be instated. Conclusion: We have developed and validated a simple score, easy to implement in daily oncological practice, to predict early death among older cancer patients which could guide oncologists in their treatment decisions. Methods: 603 outpatients prospectively included in the Physical Frailty in Elderly Cancer patients cohort study. We created a multivariate prediction model by evaluating the strength of the individual components of the Geriatric Assessment regarding risk of death at 6 months. Each component was evaluated by univariate analysis and the significant variables (P ≤ 0.20) were carried on as covariates in the multivariate cox proportion hazard analysis. The beta coefficients from the model were used to build a point-based scoring system. Clinical impact was assessed using decision curves.


INTRODUCTION
AGING decision to instate exclusively supportive care [4,5]. Cancer treatment decisions in this setting mainly rely on the Geriatric Assessment (GA) recommended by the International Society of Geriatric Oncology (SIOG) [6]. By assessing patient heterogeneity in terms of social environment, comorbidities, dependency, nutrition, mobility, mood and cognition, the GA detects vulnerabilities that are linked to poor outcomes and treatment complications [6]. In particular, impaired mobility is an independent predictor of early death among older cancer patients [7], assessed among other things by walking speed (also named gait speed) over a short distance of 4 meters [8]. This is a simple clinical and geriatric tool, easy to implement in daily oncological practice, and we have previously shown that a slow gait speed is significantly associated with geriatric impairments, which are in turn predictive of early death at 6 months [8,9]. In contrast, the feasibility of administration of the GA depends on the local availability of geriatric expertise. Four clinical and biological scores predictive of early death alongside some items from the GA have been suggested, but they lack of simplicity, and thus remain difficult to implement in daily practice [10][11][12][13].
In this study, we propose a simple score with five clinical items to predict 6-month mortality risk among older cancer patients, and to guide therapeutic decisions.

Patients and their baseline characteristics
Among the 959 consecutive older cancer in-and outpatients from the PF-EC cohort study, aged ≥ 65 years who were referred for GA up to September 30 2017, we excluded 356 inpatients. A total of 603 outpatients were included, 439 patients for the development cohort, and 164 for the validation cohort ( Figure 1).
In the whole cohort, the mean age was 81.2 ± 6.1 years. The mean time lapse between the initial oncology consultation and the geriatric oncology consultation was 6.6 days ± 2.0. Most patients were West European

AGING
There was no difference between the development and validation subsets for cancer-related and demographic data. Significant differences were found for comorbidities, weight loss, and cognitive impairment. There was a larger proportion of patients presenting weight loss (≥ 5%), cognitive impairment and overall co-morbidities in the development cohort, while there were more severe comorbidities in the validation cohort ( Table 1).

Development of a multivariate prediction model for 6-month mortality
At 6 months, the mortality rate was 17.5% (n=77/439). In univariate analysis, male gender, cancer site, cancer extension, exclusively supportive care, malnutrition (BMI < 21 kg/m 2 and weight loss ≥ 5%), and impaired mobility (gait speed < 0.8 m/s) were significantly associated with 6-month mortality. Age, comorbidities, dependency, depressed mood and cognitive impairment were not associated with 6-month mortality ( Table 2).
The final multivariate prediction model included two geriatric variables, malnutrition (unintentional weight loss) and impaired mobility (slow gait speed); two oncological variables (cancer site, cancer extension), and the decision whether or not to instate exclusively supportive care ( Figure 2, Table 3). There was no interaction between cancer site, cancer extension, and exclusively supportive care. The final scores ranged from 2 to 19 with a median of 9 (6)(7)(8)(9)(10)(11). Four groups at increasing risk were identified: 118 patients (25%) were at low risk (2 to 6), 196 (45%) at medium risk (7 to 10), 102 (23%) at high risk (11 to 14), and 23 (5%) at veryhigh risk (15 and over). Overall, the risk of early death ranged from 2% to 61%. The score was well calibrated ( Figure 3A), and discrimination was good, with a Harrell's C index of 0.75 (0.69-0.81). The Kaplan-Meier plot showed significant discrimination across the four risk groups. In particular, the 6-month risk of death was 2%, 14%, 32% and 61% for the low, medium, high and very high-risk groups respectively ( Figure 3C).

Internal validity and clinical impact of the final multivariate prediction model
In the validation cohort, the 6-month mortality rate was 18.9% (n=31/164). The score ranged from 0 to 18 with a median of 9 (6)(7)(8)(9)(10)(11). 43 patients (26%) were at low risk, 68 (41%) at medium risk, 41 (25%) at high risk, and 12 (7%) at very high-risk. Overall, the risk of early death ranged from 0% to 58%. The score was also well calibrated ( Figure 3B) and discrimination was good, with a Harrell's C index of 0.76 (0.66-0.86), close to that for the development cohort. The Kaplan-Meier plot showed significant discrimination across the four risk groups ( Figure 3D).
In stratified analyses, discrimination showed consistent results, with the best Harrell's C index at 0.81 (0.63-0.98) in the 77-to 81-year-old subset ( Table 4).
The decision curves provided better performances for our scoring system than for the ECOG-PS in accurately identifying the risk thresholds for purpose of therapeutic-decisions ( Figure 4). In particular, at the 2% and 14% risk thresholds for early death our scoring system points the need for cancer treatment and is related to a favorable C/B ratio among older patients. At the 32% risk threshold for early death our scoring system yields a less favorable C/B ratio for cancer treatment among older patients and suggests the need for caution in therapeutic decisions (tailored case by). At 61% risk threshold for early death our scoring system discourages cancer treatment among older patients, since the C/B ratio is not in favor of these patients. One the basis of these considerations, 23/38 (60%) patients in the exclusively supportive care subset would be less likely to be undertreated. Conversely, 26/126 (21%) patients in the active treatment subset would be less likely to be overtreated. Overall, according to the GRADE, 49/164 (30%) patients would be misclassified for the final therapeutic decision.

Practical clinical application
For therapeutic decision purposes, the GRADE should be used in a two-step approach: the sum of the first four variables (i.e. weight, gait speed, cancer site and extension) gives a risk of death at baseline; then the addition of the last variable enables the final decision on whether or not it is exclusively supportive care that is required. This dynamic process is expected to help with weighing up therapeutic decisions, to avoid under-and over-treatment situations. We also created a nomogram using the five clinical variables ( Figure 5) with a free website link to be used by clinicians in their daily practice: https://grade.shinyapps.io/dynnomapp/. In brief, the website gives the patient's expected survival by way of click-check variables.

DISCUSSION
Here, we provided a simple geriatric score namely GRADE to predict 6-month mortality, and to guide oncologists in their therapeutic decisions.
In our study, for all 603 patients in the two cohorts, therapeutic decisions were made in a multidisciplinary consultation meeting, which reflects the real-life situation of older patients with cancer.
Our score is composed of five clinical variables, including cancer site and extension, non-intentional AGING  weight loss ≥5%, gait speed <0.8 m/s and the decision or not to instate exclusively supportive care. This is the main strength of our simple score, since it is easy to perform in daily oncological practice with only a few additional minutes to a normal consultation.
Particularly, in our experience, the mean time to measure gait speed is 69.5 seconds, ideally measured during the time spent between the waiting room and the consultation room. In addition, we found that gait speed < 0.8 m/s was an independent, strong covariate in our

AGING
scoring system. A slow gait speed is a well-known predictive factor for mortality among older adults, and it is strongly associated with frailty and treatment complications [8,[14][15][16]. We have previously shown that a gait speed <1m/s is significantly associated with at least one geriatric impairment on the GA, and thus takes the heterogeneity of ageing with cancer into account [18]. A slow gait speed was also found to be independently associated with early death in the recently published Nice Cancer Ageing Survival (NCAS) score [13]. This score predicts early death at 100 days for older cancer patients, but it is not feasible   AGING in daily practice since it uses the mini nutritional assessment (MNA) instead of weight loss as the nutrition parameter. In a first oncological consultation, MNA scoring would require up to 15 additional minutes [17]. A limitation of our score could be its poorer discrimination compared to the NCAS score [13]. However, we chose a compromise between feasibility in daily practice, and the predictive performances of a survival score of this type. In addition to the NCAS score, other scoring systems have been specifically developed for older cancer patients for the prediction of longer-term mortality. (Table 5).
Another strength of our study is the robust methodology that we used as recommended for the development and validation of a multivariate prediction model (TRIPOD guidelines) [18]. We prospectively enrolled patients and assessed development and internal validity with a non-random split sample of patients. We used a Cox proportional hazard regression model that took into account the time to death and selected the five variables to maximize the likelihood. We validated the clinical impact of our score with decision curves, and we showed better performances than the ECOG-PS for cancer treatment decisions, since it is an unreliable clinical scale in older cancer patients [19]. Our score could also be useful for risk adjustment when a clinical trial on cancer is to be performed among older patients. Finally, our score could be useful in epidemiological studies to compare study populations, and patient outcomes across different health-care organizations.
Among older cancer patients, the therapeutic decision is a complex process that relies on multidisciplinary expertise, cancer-related symptoms, the specific type of treatment for the cancer, and the patient's preference, so that a prognostic score cannot be the sole component in the decision-making process. Our score provides three levels of information for the clinician: the risk of death linked to geriatric health (i.e. weight loss and gait speed) and to the cancer extension; the benefit of treating according to the GRADE (i.e. life extended); and the treatment cost in terms of morbidity (i.e. side effects). Thus, our score accurately identifies four groups at risk for death and increasing cost. Moreover, the originality of the GRADE is that it takes into account the decision whether or not to choose exclusively supportive care for a patient. This is particularly important for therapeutic decision-making, in metastatic settings, when the question of whether to treat or not is a daily challenge to avoid under-or over-treatment of older cancer patients. It is now well-known that older cancer patients can be under-treated, usually with sub-optimal-dose treatments if they are not too frail [21,22]. They can also be overtreated, particularly if exclusively supportive care should be preferred. Our score is helpful in that it contributes to the therapeutic decision using the variable "exclusively supportive care". This is particularly true for medium risk groups for whom a "supportive care" decision would put them in a high-risk group, and therefore a missed chance. We are aware that breaking bad news such as the decision to instate exclusively supportive care (i.e. treatment of symptoms) is difficult for the patient but also for the oncologist [23]. However, even a less aggressive chemotherapy regimen is potentially morbid, and usually non-beneficial in this setting (i.e. metastatic cancer patients with a high or very-high risk according to the GRADE) [24,25].
Let us consider the real case of an 87-year-old man with metastatic lung cancer with a 15% weight loss, and a gait speed ≥ 0.8 m/s. Thus, the GRADE is 10 (medium risk), with a 6-month mortality risk of 14%. If this patient is offered exclusively supportive care, he would be up-graded to 13 in the high-risk group, meaning decreased survival and increased cost. For this reason, he should be offered chemotherapy. Now, let us look at the same patient with a gait speed < 0.8 m/s. Before any treatment decision, the GRADE is 13 (high-risk), with a 6-month mortality risk of 32% and an unfavorable cost/benefit ratio. Even low-dose chemotherapy would lead to a greater cost and morbidity for a small benefit. The decision to instate exclusively supportive care should thus be preferred in this clinical situation.
We encourage active treatment for patients with an estimated risk of early death up to 14%, since in this situation, there is a cost/benefit ratio in favor of active treatment.

CONCLUSION
In this study, we developed and validated a simple score easy to implement in daily oncological practice, to predict early death among older cancer patients and to guide oncologists in their treatment decisions.

Reporting
We followed the Transparent Reporting of a multivariate prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines [18].

Study design and population
Patients were recruited from the Physical Frailty in Elderly Cancer patients (PF-EC) cohort. This prospective, observational two-center cohort study started in November 2013, and is described in detail elsewhere [9]. Briefly, all consecutive older in-and outpatients referred for a geriatric oncology assessment were prospectively included in a registry when a diagnosis of cancer was established and when a frailty was suspected, before any cancer treatment decision.
For the present study, we analyzed all outpatients who presented up to September 30 2017, regardless of tumor site or stage. We excluded inpatients because of the impracticality of conducting mobility tests (e.g. infusions limiting walking measures). The inclusion date was the date of the first geriatric oncology visit.
Informed consent was obtained from the patients before inclusion. The study was approved by the local ethics committee (CLEA-2015-019, Avicenne Hospital, Bobigny, France).

Cancer-related and demographic data
Demographic data (age, sex, region of origin classified as follows: West Europeans, East Europeans, Latin Europeans, North Africans, Sub-Saharan Africans, and Asians), tumor characteristics (site, extension: local, locally advanced i.e. unresectable tumor with no distant metastases, or metastatic disease stricto sensu) and Eastern Cooperative Oncology Group Performance Status (ECOG-PS) were obtained at the first geriatric oncology visit as part of the GA. Cancer treatment modes were categorized as exclusively supportive care or not, and were prospectively collected in the 6-month follow-up according to the treatment finally administered (i.e. supportive care only vs. at least one of the following treatment modes: surgery, chemotherapy, targeted therapy, hormonotherapy, radiotherapy, percutaneous treatment, and intra-arterial treatment). All treatment regimens, including exclusively supportive care, were validated in weekly institutional multidisciplinary meetings.

Outcome
Overall 6-month mortality following the GA was recorded to assess predictors of early death. Vital status AGING was determined from medical records or by telephoning patients or their families.

Statistical analyses
We allocated patients included up to July 30 2016 to the development cohort, and the remaining patients up to September 30 2017 to the validation cohort.
Categorical data are expressed as numbers and proportions, and continuous data as means and standard deviation (SD) or medians and quartiles (25 th -75 th ).
Comparisons of baseline characteristics (cancer, demographic data, and GA-components) between the development and validation subsets were performed using the chi-square test or Fisher's exact test for categorical variables, as appropriate.
We performed a correlation assessment using the Spearman's rho test as appropriate for categorical variables. Multicollinearity between variables was defined as a rho test value ≥ 0.50.

Development of the onco-Geriatric scoRe of eArly DEath (GRADE)
A comparison of baseline characteristics (cancer, demographic data, GA-components) between sixmonth survivors and non-survivors in the development subset was performed using the log rank test. Univariate predictors of six-month non-survival were expressed using non-adjusted Hazard Ratios (HR) and 95% confidence intervals (95%CI). Variables yielding P values under 0.20 in the univariate analysis were considered for inclusion in the multivariate analysis. Because of multicollinearity between the GS and the IADL, the ADL and the ECOG-PS, we choose GS as a covariate of functional status. We therefore created a Cox multivariate proportional-hazard regression model to assess multivariate factors associated with 6-month mortality. Multivariate predictors were expressed by the beta-coefficient with standard error, and by adjusted HR (aHR) with 95%CI. Graphically, multivariate predictors were presented in a forest-plot. Model assumptions were verified. To maximize likelihood while minimizing losses in prediction, a final multivariate prediction model was created with a backward selection procedure according to the lowest Akaïke Information Criterion (AIC) [32]. A scoring of each predictor was performed using Schneeweiss's beta-coefficient point-based scoring system [32]. This scoring system weights by 1 unit more or less with each 0.3 increase or decrease in the beta-coefficient. We categorized this score into four groups of increasing risk on the basis of the lowest AIC value in Cox regression. Graphically, we assessed the calibration slope in the final multivariate prediction model. Discrimination by the final multivariate prediction model was assessed using Harrell's C index with 95%CI. Given the current lack of consensus on the best threshold to consider for the quality of discrimination for survival prediction models, we used a previously described categorization of Harrell's C index discrimination as well: 0.5-0.59 (poor), 0.6-0.69 (moderate), 0.7-0.79 (good), 0.8-0.89 (very good), and ≥ 0.90 (excellent) [33]. Survival curves were plotted according to the Kaplan-Meier method with the final score divided into four groups.

Internal validation
we assessed the calibration and discrimination of the final score in the validation subset with the same methods as described above. Stratified discrimination analyses were performed in age quartiles, for an ECOG-PS ≤ 2 vs > 2, and for non-metastatic vs metastatic cancers. To assess the clinical impact of our AGING score, we used the standardized net benefit (SNB) derived from decision curves [34,35] (Supplementary Methods).
All tests were two-sided, and the threshold for statistical significance was set at P<0.05. The data was analyzed using R statistical software (version 3.4.3, R Foundation for Statistical Computing, Vienna, Austria; http://www.rproject.org). A multivariate imputation by chained equations was used to handle missing data for weight loss (n=9), and GS (n=3), via the MICE package in R. AGING SUPPLEMENTARY MATERIALS

Supplementary Methods
The SNB was first developed to assess the potential population-level impact of incorporating a risk prediction instrument into clinical practice. The assumption is that some patients will experience the outcome (i.e. cases), and some patients will not (i.e. controls). The SNB is defined in terms of true and false positive rates, outcome prevalence, and risk threshold. Thus, the interpretation of the SNB value depends on the risk threshold considered -6-month mortality, in the present study. The SNB is closely related to a Cost/Benefit (C/B) ratio in the treatment of patients.
Here, we assume that treating a control has a cost (C) (e.g., side effects of treatment), whilst there is some expected benefit (B) in treating a case (e.g., life extended, morbidity reduced). As we identified four increasing risk categories for early death (i.e., 2%, 14%, 32%, and 61%), we analyzed the SNB (95% CI), and C/B at these thresholds. We then compared the decision curves with ECOG-PS, one of the most widely used measures for therapeutic decisions in oncology.