Uncovering Barriers to Screening for Distress in Patients With Cancer via Machine Learning

: BACKGROUND: Psychologic distress and manifest mental disorders are overlooked in 30-50% of patients with cancer. Accordingly, international cancer treatment guidelines recommend routine screening for distress in order to provide psychologic support to those in need. Yet, institutional and patient-related factors continue to hinder implementation. OBJECTIVE: This study aims to investigate factors, which are associated with no screening for distress in patients with cancer. METHODS: Using machine learning, factors associated with lack of distress screening were explored in 6491 patients with cancer between 2011 and 2019 at a large cancer treatment center. Parameters were hierarchically ordered based on statistical relevance. Nested resampling and cross validation were performed to avoid overfitting and to comply with assumptions for machine learning approaches. RESULTS: Patients unlikely to be screened were not discussed at a tumor board, had inpatient treatment of less than 28 days, did not consult with a psychiatrist or clinical psychologist, had no (primary) nervous system cancer, no head and neck cancer, and did have breast or skin cancer. The final validated model was optimized to maximize sensitivity at 83.9%, and achieved a balanced accuracy of 68.9, area under the curve of 0.80, and specificity of 53.9%. CONCLUSION: Findings of this study may be relevant to stakeholders at both a clinical and institutional level in order to optimize distress screening rates.


INTRODUCTION
Current day medical services ought to be evidencebased, efficacious and cost-effective. Screening for distress in patients with cancer is recommended in treatment guidelines globally to provide psychooncological services to those in need. [1][2][3][4] Psychologic needs are present in 30-50% of patients with cancer [5][6][7][8][9][10] and persistent in survivors in 20-40%. 5,11 The distress thermometer and problem list have been established internationally as brief instruments to assess distress and its main physical, emotional, and spiritual sources. 12 Nurses are requested to screen patients repeatedly with these instruments during outpatient visits and in at least biweekly intervals during inpatient treatment. 13 Despite these requirements, screening rates remain low, ranging around 40% in inpatient treatment 14 and 22%-74% in outpatient treatment settings 15-17 even if only a single screening per patient is considered. Organizational structural deficiencies in general (e.g., work load, screening implementation and training), 18 sex, marital status, nationality, and the medical specialization of the treatment teams 14 have been identified as barriers to screening. However, there may be more patient-related factors hindering screening. In addition, there is no hierarchical order of the statistical relevance of the factors reported. The present explorative study aims to focus on patient-related barriers to screening for distress in patients diagnosed and treated at an internationally accredited cancer treatment center. It also aims to provide a clear ordering of barriers to screening based on statistical relevance. Results might help to identify indicators for the identification of patients likely to be overlooked in terms of screening for distress. Such "yellow flags" are important for oncological treatment teams, whose workload often does not allow for routine screening of all patients. Results should also provide specific targets for implementation studies aiming to increase screening rates.
Machine learning is a novel statistical technique geared toward filtering large sets of variables and data for the most significant predictors of an event. It uses computer algorithms (e.g., logistic regression, support vector machines, decision trees, or k-nearest neighbor depending on the data structure) to uncover even subtle linear and nonlinear relationships between factors without a priori biases. In the present study, machine learning was used to identify significant predictors for cancer patients who are not screened for distress.

Source of Data and Data Preparation
The present study was approved by the Ethics Committee of the State of Zurich (Ref. -No. BASEC-NR 2020-01949). Electronic file data of 6491 patients with cancer, diagnosed and treated at the Comprehensive Cancer Center Zurich (C3Z), a subunit of the University Hospital of Zurich, Switzerland, between 2011 and 2019, were reviewed. Forty-seven variables were of sufficient data quality to predict "no screening for distress" (see Appendix for complete list and definition of items). Two of these were eliminated owing to missing values in more than one third of patients. "No screening for distress" was defined as not even a single distress screening during cancer treatment, which involved several outpatient and often also inpatient oncological consultations in almost all cases examined. Data for all 45 predictor variables and the outcome variable "no screening for distress" of the final data set was retrieved from the clinical management software (KISIM, Cistec AG) and an institutional cancer register (OncoStar, IT-Choice). At the C3Z, screening for distress is the nurses' responsibility during every outpatient consultation and in biweekly intervals during inpatient treatment. A clinical decision support tool is installed in the electronic health record reminding nurses and residents to ensure screening for distress after every admission to inpatient treatment. In addition, nursing experts randomly check if screening was administered (both on the wards and in the outpatient setting) and remind the responsible nurses if necessary. Screening is achieved with an ultrashort standardized screening instrument, the distress thermometer, and problem list. 19,20 Both instruments have been implemented since over a decade and nurses are trained in their use in regular intervals. With a distress thermometer score of 5 or more, nurses are requested to consult with treating physicians, who should actively approach patients and refer them to psycho-oncology in house. A psycho-oncology referral means a consultation with a trained psychiatrist or psychologic psychotherapist. See Appendix for data selection process.

Statistical Procedures -Machine Learning
Machine learning techniques were adopted from Günther et al. 21 and adapted for the present study to (1) select the variables most predictive of "no screening for distress" and (2) combine them into a single predictive model that could be used to identify patients who are likely to remain unscreened from their case files. We relegate the details of the machine learning procedures to the Appendix, and note that, only here, we followed established guidelines for model training and validation.

RESULTS
See Table 1 for descriptive statistics on patients studied. Forty-nine percent of patients were women and 51% men; the average age was 63 years.
Performance parameters of the different models calculated during nested resampling, the specific hyperparameters used, and corresponding confidence intervals are provided in the Appendix. Gradient boosting outperformed all other machine learning algorithms.
The 7 variables most predicative of no screening for distress used for further model building were: no psychiatric/psychologic consultation preformed, longest inpatient treatment less than 28 days, breast cancer, skin cancer, no head and neck cancer, no (primary) nervous system cancer, and no tumor board being held. For results of the testing for multicollinearity, absolute, and relative distributions of variables, see Appendix.
The quality of the final model was assessed in a validation step (see Table 2 for results). As expected, the balanced accuracy of 68.9 and area under the curve of 0.80 were slightly lower than the results of the initial training model, but still meaningful. With a sensitivity of 83.9%, most patients who were not screened for distress were identified in the final model. With a specificity of 53.9%, more than half of patients who were screened were detected correctly. Figure 1 presents a one-sided tornado graph ranking the variables identified based on model validation results in terms of their significance in predicting the outcome "no screening for distress." Of all patients examined, 30.6% were screened. In the subgroup of patients with information available on whether or not they were discussed in a tumor board, the screening rate was 32%, in the subgroup with missing information on tumor board discussion the screening rate was 25%.

DISCUSSION
Our study revealed that patients unlikely to be screened were not discussed at a tumor board, had inpatient treatment of less than 28 days, did not consult with a psychiatrist or clinical psychologist, had no (primary) nervous system cancer, no head and neck cancer, and did have breast or skin cancer. With a sensitivity of 84%, the final model can help oncological treatment teams to quickly identify patients likely to be overlooked in terms of distress screening. Future prospective research is required to evaluate whether the list of predictor variables identified here is useful to improve screening rates in clinical settings. Ideally, this could provide anchor points for clinical supervisors and practitioners (e.g., nurses, residents) to optimize the thoroughness of screening. Results may also contribute to tailor clinical decision support tools with respect to patients vulnerable to be overlooked. However, in the past, treatment centers may have been reluctant to install such tools because of user fatigue over relatively inexpensive untailored reminders 22,23 and high costs of design and implementation of more tailored reminders.
Second, institutional factors may have equal statistical significance as patient-related barriers to screening. Results show that patients with breast and skin cancer were unlikely to be screened, while those  with head and neck or nervous system cancer were particularly likely to be screened. We assume that this is owing to differences in institutional factors inherent to the various clinical departments. Future research should explore these factors in more detail to better understand in which terms they are responsible for low screening rates for specific cancer types. In addition, further studies are required to show whether the results are replicable in other cancer centers. Recent studies have identified possible explanations for interinstitutional differences in screening rates. These include lack of staff, competing demands, high staff turn-over, 24 lack of time, and lack of training. 25 In contrast, institutional buy-in, support and recognition of nurses' expertise facilitated implementation of routine screening. 24 Informal inquiry at the treatment center of the present study also revealed, that at the department of gynecology, not all screenings administered by nurses are documented electronically. This may explain why breast cancer is a predictor for lack of screening. Similarly, in a comprehensive review of studies on implementation of distress screening, 3 insufficient availability of psychologic support for patients in need was the greatest barrier to screening, also confirmed by more recent research. 25 With respect to our results, we might speculate that nurses only screened for distress/ documented screening if they could be certain that patients had access to psychologic support and if patients stayed for more than 28 days in the hospital. On the other hand, results also confirmed recent research, 26 that patients not being formally screened are highly unlikely to receive psychologic support. In fact, not consulting with a psychologist or psychiatrist was a statistically significant factor in predicting no screening. In other words, unscreened patients are likely to miss out on psychologic support. This highlights the FIGURE 1. Variable Importance. Brust = breast cancer; Haut = skin cancer present; HOSDUR = longest inpatient treatment 28 days or more; KONSPSY = psychiatric/psychological consultation performed; KopfHals = head and neck cancer; Neuro = (primary) nervous system cancer; Tumorboard = tumor board held.
importance of formal screening to facilitate patients' access to psychologic/psychiatric support. Third, results highlight the relevance of tumor boards, not only to optimize somatic treatment but also to identify patients with distress. Patients not being discussed at a tumor board were less likely to be screened.
Fourth, it is noteworthy that sex, age, marital status, confession, palliative care settings, the number of comorbidities, nationality, language spoken, type of health care insurance, profession, presence of a mental disorder, and prescription of psychopharmacy were no relevant factors when patients were not screened for distress. This seems relieving because prior research identified sex, marital status, and nationality as barriers to screening. 14 These results further highlight the importance of using instruments which are independent from language as a potential barrier to screening such as the distress thermometer and problem list 12 The present study has several limitations. They include generalizability of results because only one cancer treatment center in one country was explored. However, owing to strict institutional standards on documentation of patient-related information, the quality of the data is high and distress screening procedures and instruments are very similar across industrialized nations. 12 While inclusion of various types of cancer in the present study increases generalizability of the results, aspects of certain rare cancers may have been overlooked. Unfortunately, data concerning the number of encounters of patients with a nurse for potential screening are not known. As a result, we were not able to consider the higher probability for being screened in case of more regular encounters. Screening is performed by nurses only. Some patients may have spent weeks on a ward (with daily encounters), while others may have had many outpatient consultations with physicians, but few encounters with a nurse instructed to screen for distress. However, all patients explored were diagnosed and received (by definition of the cancer register from which data were drawn) the most significant portion of their treatment at the center.
Another limitation is the number of missing values. In addition to missings at random, there may be some systemic reasons specific to the treatment center explored here. For instance, whether or not a patient was discussed in a tumor board was not documented homogeneously during the study period. This may have led to some systematic mislabeling as "missing." In fact, the electronic health record keeping has been reformed since forcing all users to document tumor board discussions homogeneously. Screening rates for distress were similar in the subgroups of patients with information on tumor board discussion (32%), without such data (25%) and overall (31%). Similarly, intensive care units use a different health record keeping system to document medication administered. Although required to upload that information to the regular electronic record keeping system, there used to be no control mechanism to ensure compliance. In addition, the medication of patients receiving outpatient treatment only may not have been documented by physicians in some cases owing to time pressureespecially if no medication was prescribed.
In all machine learning analyses there is a residual risk of overfitting despite all precautionary measures (see methods section). Hence, further (prospective) research in other cancer treatment centers should perform studies which verify the results presented here for thorough performance evaluation of the final model. Such research might also evaluate further variables, which were not identifiable at the site studied here. Finally, the present study did not examine interactions between the predictor variables explored.

CONCLUSION
Overall, results of the present study have clinical relevance for those interested in improving distress screening rates from an institutional or supervisor point of view. In addition, they can help nurses and residents responsible for screening to avoid pitfalls, such as overlooking patients with short lengths of stay or those who are not discussed at a tumor board. Future research is needed to explore replicability of the results in other treatment centers to ensure their generalizability and to investigate further details of the institutional factors identified.

AUTHORS CONTRIBUTIONS
M.G., J.S. and S.E. conceived and designed the study. Data collection was performed by J.S. and M.G. Material preparation and analysis were performed by J.K. The first draft of the manuscript was written by M.G.
A.G. and R.v.K. provided senior advice for the study. All authors edited multiple drafts and supervised the statistical analyses. All authors read and approved the final manuscript.