Prognostic value of the veterans affairs frailty index in older patients with non‐small cell lung cancer

Abstract Background Older patients with non‐small cell lung cancer (NSCLC) are a heterogeneous population with varying degrees of frailty. An electronic frailty index such as the Veterans Affairs Frailty Index (VA‐FI) can potentially help identify vulnerable patients at high risk of poor outcomes. Methods NSCLC patients ≥65 years old and diagnosed in 2002–2017 were identified using the VA Central Cancer Registry. The VA‐FI was calculated using administrative codes from VA electronic health records data linked with Medicare and Medicaid data. We assessed associations between the VA‐FI and times to mortality, hospitalization, and emergency room (ER) visit following diagnosis by Kaplan–Meier analysis and multivariable stratified Cox models. We also evaluated the change in discrimination and calibration of reference prognostic models after adding VA‐FI. Results We identified a cohort of 42,204 older NSCLC VA patients, in which 55.5% were classified as frail (VA‐FI >0.2). After adjustment, there was a strong association between VA‐FI and the risk of mortality (HR = 1.23 for an increase of four deficits or, equivalently, an increase of 0.129 on VA‐FI, p < 0.001), hospitalization (HR = 1.16 for four deficits, p < 0.001), and ER visit (HR = 1.18 for four deficits, p < 0.001). Adding VA‐FI to baseline prognostic models led to statistically significant improvements in time‐dependent area under curves and did not have a strong impact on calibration. Conclusion Older NSCLC patients with higher VA‐FI have significantly elevated risks of mortality, hospitalizations, and ER visits following diagnosis. An electronic frailty index can serve as an accessible tool to identify patients with vulnerabilities to inform clinical care and research.


| INTRODUCTION
Non-small cell lung cancer (NSCLC) has a median age at diagnosis of 70 years and disproportionately affects older patients, 1,2 affecting a heterogeneous population that includes patients with varying functional status, cognitive impairment, and comorbid conditions. [3][4][5] Limited evidence is available for guiding care and treatment of older cancer patients due to their under-representation in clinical trials. 6 Traditional assessments such as those based on chronological age and performance status may not account for the full range of vulnerabilities experienced by older patients with cancer. 7 In the absence of assessments reflective of their full vulnerabilities, frail patients may be at risk of overtreatment with intensive therapies and fit older patients may be at risk of undertreatment with milder, less efficacious therapies. [8][9][10] Current guidelines have recommended the adoption of geriatric assessments in older adults with cancer to help address this gap. 8,9,11,12 This has not been widely implemented in practice due to limitations in time, resources, and geriatrics expertise. 13 Recently, electronic frailty indices have been developed to assess physiologic vulnerability based on the cumulative number of health deficits ascertained through electronic health record (EHR) and administrative data. 14,15 Electronic indices cannot fully capture elements of a geriatric assessment (e.g., gait speed). However, as these electronic indices depend only on data collected during routine care, they can be automatically computed and implemented at scale. This can facilitate automated screening of vulnerable patients 13 for clinical care and also retrospective identification of frail patients for research, when alternative measures are not widely or consistently recorded. Some forms of a frailty index have already been developed for other settings in oncology, such as for patients who have breast cancer, 16 ovarian cancer, 17,18 urologic malignancies, 19 head-and-neck cancer, 20,21 multiple myeloma, [22][23][24] and surgery. 25 To our knowledge, no study has assessed electronic frailty indices in older NSCLC patients, outside of broad studies in overall cancer populations. 26,27 In this study, we measure frailty and assess its prognostic value by applying an electronic frailty index to older patients with NSCLC in the Veterans Affairs (VA) healthcare system. The VA Frailty Index (VA-FI) was previously developed and validated in a large general cohort of older patients treated at the VA. 28, 46 We evaluate both associations between the VA-FI with mortality, hospitalizations, and emergency room (ER) visits in a large national cohort and its incremental value for predicting these outcomes when incorporated into prognostic models based on traditional predictors.

| Study population and data sources
We conducted a retrospective study primarily using EHR data from the VA Corporate Data Warehouse (CDW), which captures clinical, billing, and operational information for patients treated at VA facilities nationwide. Patients with lung cancer were identified as those with a record in the VA Central Cancer Registry (VACCR) in which "LUNG" was mentioned in the description of the primary tumor site. The VACCR records data on cancer patients from VA facilities nationwide and has been estimated to capture up to 90% of cases. 47 Patients diagnosed with NSCLC were distinguished from those with small cell lung cancer through a manual review of lung cancer histology terms consistent with adenocarcinoma, squamous cell carcinoma, large cell carcinoma, and less common subtypes of NSCLC, as reported through ICD-O (International Classification of Diseases for Oncology) codes in the VACCR. We focused on older patients who were at least 65 years old at the time of diagnosis, which enables incorporation of Medicare data for defining the VA-FI among the population. The population was restricted to those diagnosed within 2002-2017 since the VACCR may have had underreporting of cancer cases in earlier years and Medicare data were available only through 2017. To limit attention to regular users of VA services, we excluded patients who did not have at least one record associated with an ICD code in each of the 3 years prior to diagnosis. We also excluded patients who failed to meet data quality control criteria, such as having the diagnosis date after last follow-up date, having a stage 0 cancer diagnosis, or having an Eastern Cooperative Oncology Group (ECOG) score of 5 (death) at diagnosis. This study was conducted under a protocol approved by the VA Boston Healthcare System Research and Development Committee.

| Baseline covariates
Patient demographics, including age, gender, and race/ ethnicity were obtained from records in the CDW. Histological subtype was classified based on terms reported from ICD-O codes available in the VACCR. The American Joint Committee on Cancer stage group, ECOG performance score, and sites of distant metastasis were ascertained based on records associated with the diagnosis from the VACCR. Performance scores reported as Karnofsky scores were converted to equivalent ECOG scores. 29 Unknown performance scores were encoded as a separate category in the covariate for ECOG in the incremental value analyses. Data in the VACCR were abstracted from patient charts by cancer registrars at VA sites treating patients with cancer. 30 The abstraction process complies with standards of the North American Association of Central Cancer Registries. 31

| VA frailty index (VA-FI)
The VA-FI for each patient was calculated as the proportion out of 31 equally weighted deficits (Table 1) incurred by the patient. These deficits account for impairments across multiple physiologic domains and were ascertained by the presence of specific sets diagnostic and procedure codes in patients' records. In particular, we extracted ICD9/10 (International Classification of Diseases Ninth/ Tenth Revision), Current Procedural Terminology (CPT), and Healthcare Common Procedure Coding System codes within a 3-year period prior to patients' NSCLC diagnosis date from the CDW and linked Medicare and Medicaid data to account for encounters both inside and outside the VA, though Medicaid data were available only through 2014. An updated version of the VA-FI based on translating ICD9 codes used to define deficits in a previous version 28 through the Centers for Medicare and Medicaid Services General Equivalence Mappings 32 and manual review was applied to account for ICD10 codes incurred after October 2015. 46 Patients were classified to be non-frail (0-0.1), pre-frail (0.1-0.2), mildly frail (0.2-0.3), and moderate-to-severely frail (>0.3), using previously published frailty index cutoffs. [33][34][35]

| Mortality, hospitalizations, and ER visits
The primary outcome of the study was all-cause mortality as captured in the Veterans Health Administration Vital Status File, which aggregates mortality information captured from official sources, including reports from VA facilities, death certificates, VA National Cemetery Administration, Medicare, and the Social Security Administration data, among other sources. 36 We considered acute hospitalizations and ER visits at VA facilities as secondary outcomes. Episodes of acute hospitalization visits were defined based on classifying the specialty of inpatient visits using a previously published approach. 37 Specialties related to surgery or other scheduled services, such as urology, orthopedics, transplantation, anesthesiology, podiatry, and vascular, were excluded to focus on unplanned episodes of acute hospitalization. ER visits were ascertained by identifying outpatient visits with associated ER decision support system identifiers. The follow-up was defined to be time from diagnosis to either death or last date of either a billable inpatient or outpatient encounter or a record in the fee treatment table for non-VA care paid by the VA. For non-censored patients, time to mortality, hospitalization, or ER visit was defined by the difference between the date of event and date of diagnosis.

| Statistical analysis
The unadjusted survival for time from diagnosis to mortality and cumulative incidence of the first acute hospitalization and ER visit following diagnosis were estimated by Kaplan-Meier curves, stratifying by frailty status and also by ECOG score or stage. Stratified Cox proportional hazards models 38 for time to mortality, first hospitalization, and first ER visit were fit with VA-FI as a continuous covariate, adjusting for age at diagnosis, gender, race/ethnicity, and sites of distant metastasis as covariates and histological subtype and stage as stratification factors. The incremental value of incorporating VA-FI into prognostic models was assessed by comparing the discrimination and calibration among four sets of models: (1) Stratified Cox models described above excluding the VA-FI ("Baseline" model), (2) The Baseline model adding ECOG as a covariate ("Baseline with ECOG"), (3) The Baseline model adding VA-FI as a covariate ("Baseline with VA-FI"), and (4) The Baseline model adding both ECOG and VA-FI as covariates ("Baseline with both ECOG and VA-FI"). Models (1) and (2) serve as references for estimating the incremental value when adding VA-FI to a model with traditional predictors and for assessing the relative incremental value compared to adding ECOG. Model (4) allows for assessment of incremental value when adding VA-FI to models with ECOG. The cumulative risk of mortality, hospitalization, and ER visits were estimated from these models, 38 for a range of short-and longterm time points (1-month, 1-year, 3 years, 5 years, and 10 years after diagnosis). The discrimination was assessed by estimating the time-dependent cumulative/ dynamic area under the curve (AUC) for each model and landmark time. 39,40 The calibration was assessed by plotting the cumulative risk at each landmark time as estimated by Kaplan-Meier against mean predicted risks, within deciles of the predicted risk, using Nam-D'Agnostino test to test for significant discrepancies. 41 As a sensitivity, the assessments of discrimination and calibration for 1-year time point were repeated in the subset of patients with known ECOG status. For the discrimination and calibration, Cox models were trained in a training set comprised of a random sample of 80% of the study sample and validated in the remaining 20%.

| Patient cohort and baseline characteristics
We identified 42,204 older NSCLC patients who met the inclusion/exclusion criteria of the study ( Table 2). The median follow-up time was 6.53 years following diagnosis, and 85.0% of patients died, 63.0% incurred an acute hospitalization, and 54.2% incurred an ER visit during follow-up. The baseline characteristics are reported in Table 3. The majority of patients were male (98.5%) and White (73.1%), with a substantial proportion being African American (12.6%). The mean age at diagnosis was 74.1 years. The two most common histological subtypes were adenocarcinoma (47.7%) and squamous cell carcinoma (47.9%). Most patients did not have performance status documented (60.5%), and 27.5% of patients had T A B L E 1 Count and proportion of patients having each deficit that contributes to the VA-FI, by frailty status: 0.-0.1 (non-frail), 0.1-0.2 (pre-frail), 0.2-0.3 (mild frail), >0.3 (mod+ frail)

| Association of outcomes with frailty
Patients with higher VA-FI at baseline had significantly lower survival and higher incidences of acute hospitalizations and ER visits ( Figure 1). The 1-year survival ranged from 59.3% (95% CI 58.0%-60.9%) for non-frail patients to 44.7% (43.8%-45.6%) for moderate-to-severely frail patients, with similar differences seen for hospitalization and ER endpoints. The differences in survival persisted for as long as 10 years after diagnosis. The VA-FI still differentiated risk for patients with 0-1 or unknown ECOG scores and patients with stage I-III disease or unknown stage (Figure 2), although differentiation was weaker among those with higher ECOG scores and stage IV disease. Associations between VA-FI with mortality and adverse outcomes held after adjustment for demographics and clinical features (

| Incremental value of VA-FI
Adding VA-FI to baseline prognostic models generally led to statistically significant improvements in timedependent AUCs ( Compared to the Baseline model with ECOG, models that additionally included the VA-FI generally exhibited further improvements in AUC, with the exception of 5-and 10-year hospitalizations. These patterns of improvement were also similar among patients with known ECOG, although the gains from adding VA-FI to baseline was lower than that of adding ECOG in some cases (e.g., short-term mortality outcomes) due to improved performance of models with ECOG (Table S1). Calibration plots exhibited a close degree of agreement between predicted and observed risks for early endpoints that was not reduced by addition of the VA-FI (Figure 3). For mortality and hospitalizations, the predicted risks by 1-year after diagnosis agreed closely with a wide range observed risks regardless of whether VA-FI was included in the models. For ER visits, adding the VA-FI to the Baseline model with ECOG did lead to marginally

T A B L E 2
Sample selection criteria for defining study cohort significant discrepancies using the Nam-D'agnostino test, but the absolute differences were small and unlikely to be clinically significant. For later time points and sensitivity analyses among patients with known ECOG, significant discrepancies were generally detected regardless of whether VA-FI was included (Figures S1 and S2).  Here, we applied the VA-FI to a cohort of NSCLC patients at the VA and found that frailty was prevalent among VA patients with NSCLC according to the VA-FI, with over half of the patients being classified to be at least mildly frail and over a quarter moderate-to-severely frail. These rates of frailty are either similar or higher to rates reported in other studies on frailty in solid tumors, 26,27,42 which may be influenced by the greater burden of chronic conditions among Veterans. 43 Frail patients were at elevated risk of mortality and adverse outcomes, even after adjustment for demographics and standard clinical predictors. This is consistent with studies in other cancers 22,25 and with smaller studies in NSCLC. 4,44 The strong associations suggest that such electronic frailty indices can identify vulnerable patients who are at increased risk of adverse outcomes.
Adding the VA-FI to baseline prognostic models led to improvements in AUCs for discriminating both shortand long-term adverse outcomes. These improvements are generally comparable or higher than improvements from adding ECOG. As ECOG was unknown for a majority of patients, the results on models with ECOG reflect the real-world performance after accounting for its incomplete measurement. The impact of adding the VA-FI to models among those with known ECOG status were similar, with some exceptions such as for short-term mortality outcomes. In contrast to ECOG, the VA-FI can be automatically computed using EHR and administrative data only, requiring no time from clinicians to perform an assessment. It can thus be broadly implemented in EHR systems to systematically screen for frailty among patients who are diagnosed with NSCLC to inform clinical care. The VA-FI also provides a standardized measure of frailty that can be applied retrospectively to EHR and administrative data for research that would not be subject to incomplete documentation. The gains in AUC when adding VA-FI to the baseline with ECOG model also suggest that VA-FI captures complementary prognostic information that would refine estimates of risk, even when ECOG is included in the model. Given the low cost calculating the VA-FI and the potential benefits to correct anticipation of adverse outcomes and early intervention, these refinements to prognostic models from incorporating the VA-FI can potentially be clinically meaningful, though further work would be needed to evaluate the impact of adopting the VA-FI in patient assessments on outcomes.
A major strength of this study is the ability to assess frailty among a large national cohort of older NSCLC patients. This enabled the evaluation of associations and prediction performance with VA-FI in a broad NSCLC patient population. As curated data on traditional oncology assessments were available through the VACCR, we were able to adjust for clinical predictors including stage and ECOG score and compare the performance of prognostic models against reference models based on these standard predictors. Whereas many of the existing studies on frailty indices in cancer evaluate only associations with    outcomes, this study also assessed the changes in discrimination and calibration to demonstrate the expected impact on out-of-sample risk prediction. One limitation of our study was that the prognostic models did not incorporate information on treatments following diagnosis. This would allow for models to estimate predicted outcomes under different treatments and potentially improve the actionability of prediction results, though further work would be needed to account for confounding. 45 Adjustment for treatment in models may either attenuate or inflate the associations between the VA-FI and outcomes, depending on patterns of utilization and the safety and effectiveness of the treatment among frail and non-frail patients. Additionally, hospitalizations and ER visits outside the VA are not captured in VA data. We did not utilize the Medicare and Medicaid data for the secondary outcomes, as definitions that were reliable and known to be consistent with definitions used in the CDW were not available. Nevertheless, the cumulative incidence of hospitalizations and ER visits were within the range of estimates from previous studies and do not appear to be severely underreported. 48,49 Associations with the VA-FI could be attenuated if those with non-VA hospitalizations or ER visits had lower VA-FI scores than others in the cohort. Finally, the VA population differs from non-VA populations in several respects, most notably in the high proportion of male patients. Additional studies would be needed to confirm the generalizability of our findings to other populations.

| CONCLUSION
Among older patients with NSCLC, the VA-FI is significantly associated with times to mortality, hospitalization, and ER visit following diagnosis. Incorporating the VA-FI in prognostic models based on demographics and traditional clinical predictors led to significant improvements in discrimination. These results suggest that electronic indices such as the VA-FI can help identify patients with vulnerabilities that put them at an elevated risk of experiencing adverse outcomes. Further research on how electronic frailty indices can help optimize treatment and care of NSCLC patients is warranted.

SUPPORTING INFORMATION
Additional supporting information may be found in the online version of the article at the publisher's website.