Opioid Nonadherence Risk Prediction of Patients with Cancer-Related Pain Based on Five Machine Learning Algorithms

Objectives Opioid nonadherence represents a significant barrier to cancer pain treatment efficacy. However, there is currently no effective prediction method for opioid adherence in patients with cancer pain. We aimed to develop and validate a machine learning (ML) model and evaluate its feasibility to predict opioid nonadherence in patients with cancer pain. Methods This was a secondary analysis from a cross-sectional study that included 1195 patients from March 1, 2018, to October 31, 2019. Five ML algorithms, such as logistic regression (LR), random forest, eXtreme Gradient Boosting, multilayer perceptron, and support vector machine, were used to predict opioid nonadherence in patients with cancer pain using 43 demographic and clinical factors as predictors. The predictive effects of the models were compared by the area under the receiver operating characteristic curve (AUC_ROC), accuracy, precision, sensitivity, specificity, and F1 scores. The value of the best model for clinical application was assessed using decision curve analysis (DCA). Results The best model obtained in this study, the LR model, had an AUC_ROC of 0.82, accuracy of 0.82, and specificity of 0.71. The DCA showed that clinical interventions for patients at high risk of opioid nonadherence based on the LR model can benefit patients. The strongest predictors for adherence were, in order of importance, beliefs about medicines questionnaire (BMQ)-harm, time since the start of opioid, and BMQ-necessity. Discussion. ML algorithms can be used as an effective means of predicting adherence to opioids in patients with cancer pain, which allows for proactive clinical intervention to optimize cancer pain management. This trial is registered with ChiCTR2000033576.


Introduction
Cancer pain is an important and distressing symptom that tends to increase in frequency and intensity as cancer progresses [1].More importantly, pain is an independent predictor of survival in cancer patients [2].Tus, adequate pain management is critical for improving the quality of life and health outcomes in cancer patients.
Tere is a growing recognition that despite the many clinical guidelines that have been developed to help achieve efective cancer pain management, a large number of patients still have inadequate pain management.A systematic review reported approximately one-third of cancer patients still do not achieve adequate pain relief [3].Te reasons for inadequate pain management in cancer patients are multifaceted, and these barriers exist in all parts of the healthcare system, including healthcare professionals, patients, and healthcare system-related barriers [4].Among the patientrelated factors, one of the greatest barriers is poor adherence to analgesics [5].Analgesics are the cornerstone of cancer pain management [6], and adherence to prescribed analgesic regimens is key to successful cancer pain management.
Adherence to analgesics in cancer pain patients has ranged from 8.9% to 82.0% in diferent studies due to the diferent methods to measure adherence [7].Nonadherence is a multifactorial issue that may be infuenced by a range of patient, disease, condition, social/economic, and healthcare system/team-related factors [7].Terefore, identifying the various factors associated with pain medication adherence and timely interventions may help improve analgesic efcacy and achieve a better quality-of-life outcome in patients with cancer pain.Interventions to improve medication adherence have been intensively studied for decades, but even complex interventions have shown only modest, and sometimes inefective, efects [7][8][9][10].One possible explanation is that, as mentioned above, nonadherence is multifactorial and complete coverage of interventions is difcult to achieve [11].Terefore, the feld of adherence research has turned to new strategies that take an individualized rather than a standardized approach to adherence interventions [12,13].Another problem with the inefectiveness of adherence interventions is that although a variety of factors are known to infuence medication adherence, these factors vary across studies depending on their trial design and region, resulting in little knowledge of predictors of patients' actual analgesic-taking behavior [14,15].Consequently, constructing a patient medication adherence prediction model that allows clinical staf to anticipate in advance how patients will adhere to their pain medication is crucial, if possible, thus allowing them to adopt efective individualized interventions to improve patient adherence and thus analgesic efcacy.
In recent years, artifcial intelligence methods such as machine learning (ML) have been increasingly used in medicine to predict clinical events [16].ML is particularly well suited for analyzing large data sets, computing complex interactions, identifying hidden patterns, and generating actionable predictions in clinical settings.In many cases, ML has been shown to outperform traditional statistical techniques [17][18][19][20].ML models ofer a promising approach for predicting patient adherence to medications, and studies have been conducted in this area in diseases such as diabetes and hypertension with high prediction accuracy [14,[21][22][23][24].However, to our knowledge, there have been no studies using ML methods to predict adherence to opioids the core component of moderate-to-severe cancer pain management in patients with cancer pain [6].
Te study aims to investigate whether an ML approach can accurately predict nonadherence to opioids in patients with cancer-related pain based on a previous cross-sectional study.Tis approach allows (1) simultaneous analysis of dozens and hundreds of clinical variables; (2) automated prediction without additional screening steps, thus reducing the burden on healthcare professionals; and (3) the developed models could be implemented in a clinical setting in the form of a clinical decision support system (CDSS), which in turn could help prescribers design targeted interventions to improve patient outcomes.

Design and Study Population.
Figure 1 shows the study process.Tis study is a secondary analysis of existing data from a parent study as an exploratory cross-sectional study of genetic and clinical factors associated with opioid response in Chinese Han patients with cancer pain [25].Subjects were enrolled from March 2018 to October 2019 at a cancer center in a Grade III Level A hospital in central China.Patients were enrolled according to the following criteria: (1) age >18 years; (2) had a histologically or cytologically diagnosed malignant tumor; (3) experienced moderate-to-severe cancer pain symptoms and prescribed with around-the-clock (ATC) opioid analgesics for at least 72 hours; and (4) voluntarily provided informed consent.Tis study protocol was approved by the Medical Ethics Committee of Tongji Medical College, Huazhong University of Science and Technology (approval number: 2018-S016), and registration was submitted to the China Clinical Trials Registry (registration number: ChiCTR2000033576).

Outcome and Initial
Predictor Selection.Te outcome variable was dichotomous: adherence was defned as good if he/she is in strict accordance with the prescribed dose, frequency, and time of taking opioids at least 80% of the time; otherwise, it was defned as poor.Analgesic prescriptions were collected from medical records and how patients took them was reported by their self-reports.Nonadherence includes taking a lower/higher dose of opioids, self-discontinuation, forgetting to take medications, taking medications early/delayed, and taking other medications that are not on the prescription [26].A total of 43 demographic and clinical-related factors, such as age, gender, diagnosis, Charlson comorbidity index (CCI), pain control, analgesic drugs used, and beliefs of medicine, were included as potential predictors.See Tables 1 and 2 for further details on the predictors.Te measures of included variables had been well-explained elsewhere [25].All categorical variables were transformed using a one-hot encoder before being used to construct the predictive model.

Missing Data Imputation.
Seven of the 43 predictors had missing data, in the order of proportion missing: beliefs about medicines questionnaire (BMQ)-harm (13.39%),BMQ-overuse (13.39%),BMQ-necessity (11.72%),BMQconcerns (11.72%), time since onset of pain symptoms (1.76%), time of tumor diagnosis (0.42%), and SDS score (0.17%) (Table 2).Te missing data were relatively few and were not considered missing completely at random (MCAR); instead, their missingness was assumed to be related to the other measured variables (MAR).We estimate the values of missing data using k-nearest neighbor (k-NN) imputation, which is designed to help estimate missing data by fnding the nearest neighbors of observations with missing data using the Euclidean distance metric and estimating the missing data based on the nonmissing values in the neighbors [27].

Handling of Data
Imbalance.In this study, there was a sample imbalance between good and nonadherence patients, with a ratio close to 8 : 2. ML classifers trained on class-imbalanced data are prone to overpredict the majority class.Tis leads to a larger misclassifcation rate for the 2 Pain Research and Management minority class [28].In this study, the thresholding method was used to reduce the disturbance of imbalance on model training.For binary data, the classifcation threshold is set to 0.5 by default; however, this is often not ideal for unbalanced data.Adjusting the decision threshold is a good strategy to deal with the class imbalance problem [29,30].Te decision thresholds of diferent models were automatically determined according to AUC_ROC.In our study, the optimal decision thresholds for LR, MLP, RF, SVM, and XGBT were 0.498, 1.926e − 17, 0.184, 0.756, and 0.323, respectively.In addition, some of the feature values were unbalanced, such as opioid type, pain causes, and health insurance type.Te data were trained and validated by support vector machine (SVM) and logistic regression (LR), and it was found that the performance indicators before and after the removal of such features were almost the same, thus determining that these features did not contribute much.Similarly, after training by the random forest (RF) algorithm, these features were found to be the least important, which afrmed the previous judgment.Terefore, these few metrics are removed in the later model training to increase the model's robustness.

Data Partition.
Twenty percent of the samples are randomly selected as the test set, and the remaining 80 percent of the samples are used in the model training process using the k-fold cross-validation method, with K taking a value of 10.Te cross-validation technique is used to minimize bias in performance comparison and assessment.Te testing set was used to evaluate the predictive performance of the fve models from the training set.Pain Research and Management performing bagging operations to combine multiple decision trees or models to arrive at a more stable and accurate data prediction, and is mainly used for classifcation and regression [31].XGBoost is based on a sparsity-aware algorithm, a weighted quantile sketch in which weak learners can converge sequentially to the ensemble to achieve a strong learner [32].MLP is based on feed-forward artifcial neural network models, which is important in nonlinear ftting analysis due to its high fault tolerance and adaptivity [33].SVM is an optimal classifcation algorithm that distinguishes between diferent classes of samples in a high-dimensional space and is able to transform the training data into a high-dimensional feature space and obtain a linear optimal solution by separating the hyperplanes with the smallest distances between hyperplane points and the largest margin between classes [34].LR is a widely used classifcation model that fts data into a logistic function, enabling prediction of the probability of event occurrence [35].In each model, the GridSearchCV algorithm was used for hyperparameter tuning with AUC_ROC as the evaluation indicator.Supplementary Table 1 lists the set of hyperparameters and the parameters of the fve models.

Model Training and
Te fve models were evaluated by area under the receiver operating characteristic curve (AUC_ROC), accuracy, specifcity, sensitivity, precision, and F1 score.However, model performance is especially infuenced by data class proportions and predicted class proportions.As the classes become more imbalanced, it is often the case that we can get good accuracy by simply predicting the majority class, and accuracy cannot be correct for random chance levels that vary due to predicted and data class proportions.Tus, in this study, Cohen's kappa was used to test the statistical signifcance of model performance against the chance level [36].Te value of the best model for clinical application was assessed using decision curve analysis (DCA).

Data Collected.
A total of 1195 patients were included in the fnal analysis, of which 945 had good adherence and 250 had poor adherence.Te mean age of the participants was 56.41 ± 11.46 years, 61.7% were male patients, the time since onset of pain symptoms was 3.01 ± 4.50 months, the mean number of analgesics per person was 1.32 ± 0.58, the ATC opioid was oxycodone extended-release tablets in 69.94% of the patients, the mean oral oxycodone equivalent daily dose (OEDD) was 37.61 ± 32.89 mg, and 65.36% of patients had good pain control.Te demographic and clinical characteristics of the patient are outlined in Tables 1 and 2. Univariate analysis showed that 11 factors were associated with opioid adherence.Patients who were adherent to opioids more frequently had higher family income, shorter time to diagnose, shorter time since onset of pain symptoms, using fentanyl transdermal patch, a higher percentage of nonmixed pain, a higher percentage of pain due to treatment, better pain control, higher BMQ-necessity scores, and lower BMQ-concerns, BMQ-harm, and BMQ-overuse scores compared to that nonadherence.

Model Performance Evaluation.
In this study, fve selected models, namely, RF, XGBoost, MLP, SVM, and LR, were trained and validated via 10-fold cross-validation.Te efectiveness of ML models was assessed by accuracy evaluation and consistency evaluation, including ROC_AUC, accuracy, precision, sensitivity, specifcity, F1 score, and Cohen's kappa coefcient.Figure 2 summarizes the prediction performance on test set data for all the models developed.Te AUCs of the four models, LR, SVM, RF, and XGBT, were higher than 0.70, indicating good performance in this classifcation task, especially the LR model, which had an AUC of 0.82, accuracy of 0.82, and the highest F1 score of 0.88.At the same time, Cohen's kappa of the LR model was 0.22 (p < 0.001), which indicated that this model got an accuracy above a certain chance level.Te F1 score integrates the model's ability to distinguish negative samples and identify positive samples, and the higher the F1 score, the more robust the model is.Of note, the goal is to predict patients who are less likely to adhere to their treatment plan so that further interventions can be made with this group of patients before taking the medication to improve adherence rates.Tat is, the prediction of these unpredictable individuals is more important than those adherers.Terefore, the "specifcity" describing the "true negative rate" was the focus of this study.Recognizing this, we can see from Table 3 that the LR model had the highest specifcity value of 0.71 among all models.Terefore, LR is the best model by combining the results of all aspects.
Te DCA of test data demonstrated that the LR model showed a higher net beneft than all patients receiving the intervention or no intervention over most of the threshold probability range (Figure 3).Clinical interventions for

Feature Importance.
To investigate the contribution of each predictive feature in the LR model, we retrieved the coefcients of the elements in the model decision function from each cross-validation fold to serve as the importance of the feature values and summarized the results as shown in Figure 4.When the coefcient is less than zero, it indicates that the greater the value taken for the characteristic, the greater the likelihood that the patient will experience nonadherence, and vice versa.Te larger the absolute value of the coefcient, the greater the contribution of the eigenvalue to the prediction of adherence.It can be seen that BMQ-harm, time since the start of opioids, BMQ-necessity, number of analgesics, and gender were the top fve features contributing to model predictions.Overall, patient features contributed 25.30%, while clinical features contributed 74.70% toward model predictions (Figure 4).

Discussion
In this secondary analysis of an exploratory study of genetic and clinical factors associated with opioid response in Chinese Han Chinese patients with cancer pain [25], we constructed fve ML models to predict opioid nonadherence in cancer pain patients.By comparing the specifcity, sensitivity, accuracy, precision, AUC_ROC, and F1 score of diferent prediction models, it was concluded that LR was a desirable prediction model, capable of making more accurate predictions, and its specifcity for predicting opioid nonadherence in cancer pain patients reached 0.71; i.e., the  Pain Research and Management proportion of truly nonadherent patients was 71% among all patients who were predicted to be nonadherent.While the MLP model performed the worst, DCA shows that the LR model based on demographic and clinical characteristics has high clinical application value in predicting the adherence to opioids in cancer pain patients and is of great clinical signifcance in guiding healthcare providers on whether early intervention is needed.
Although studies related to predictive models of medication adherence were found in patients with tuberculosis [37], heart failure [38], diabetes [14], self-inject medication at home [24], nicotine withdrawal treatment [23], opioid use disorder treatment [39], and initiation of statin therapy [40], the models constructed in these studies had AUCs of accuracy of 76%-82%, and specifcity of 53.6%-78.3%.However, there are no predictive models for opioid adherence in patients with cancer pain, for whom medication adherence is a key factor in treatment outcomes.Te best model obtained in this study, the LR model, had an AUC of 0.82, accuracy of 0.82, and specifcity of 0.71, indicating a good predictive performance compared to the above studies.
Tere are various ML algorithms (e.g., LR, classifcation and regression trees, learning vector quantization, SVM, boosting, and deep neural networks) that are used to develop various clinical event prediction models.LR is a traditional binary classifcation model in ML, and the simplicity and efectiveness of this algorithm have made it widely used in many felds.Logistic regression prediction models were used in 53% of studies of clinical support decision systems in emergency departments [41].Christodoulou et al. included 71 studies comparing the ability of LR and other ML models in predicting dichotomous clinical outcomes such as disease diagnosis or prognosis and showed that the performance was close [42].In the low-risk-of-bias studies, there was no diference on average between the performance of LR models and ML models in terms of AUC, with a mean logit (AUC) diference of 0.00 (−0.18 to 0.18).
Similar to diagnostic metrics, ML prediction models have sensitivity and specifcity.When building predictive models, seeking to increase sensitivity will necessarily reduce the specifcity of the model predictions [43].In clinical practice, our goal is to predict patients who are less likely to adhere to their treatment plan, so that further interventions can be introduced to this group of patients prior to planned medication use to improve adherence rates.Terefore, the "specifcity," which describes the "true negative rate," was the focus of this study.Although 43 demographic and clinical characteristics that may be associated with opioid adherence were included in this study, the specifcity of the best model prediction did not reach 0.8.Tis may be because this study was a secondary analysis and the data used to train the model may not have included other important factors afecting patient adherence, such as caregiver attitudes toward opioids, patient expectations of analgesic treatment, and satisfaction with medical staf.In addition, the patient included in this study had been taking opioids for varying lengths of time, from 72 h to 54 months, which made the prediction of nonadherence more complicated.It is suggested that in the future, prediction models could be developed for diferent phases of treatment (e.g., initial, implementation, and discontinuation of therapy) to achieve more accurate prediction [44].
Our LR prediction model covered many known risk factors for medication adherence, including BMQ, duration of opioids, number of analgesic medications, gender, and employment status.Te BMQ-harm was the most important predictor of nonadherence to opioids in cancer pain patients with an importance coefcient of 0.15.In addition to this, BMQ-concern had an importance coefcient of 0.03.Te BMQ-necessary is the most important positive predictor of patient adherence.Over the past decade, many studies have used BMQ to explore the relationship between patients' medication beliefs and adherence [45].In general, the higher the patient's medication apprehension, the lower the medication adherence [46].In particular, Chinese patients' long-standing misconceptions about pain and analgesics have led to excessive concerns about analgesic side efects and addiction, and reluctance to use opioids.Even after extensive pain education for patients and their caregivers in recent years, they still maintain very high negative beliefs about pain and analgesic medications [47].In contrast, BMQ-necessary was positively associated with adherence.Fischer et al. reported a signifcant positive association between BMQ-necessary and adherence at 3-month followup in patients with COPD (OR � 2.6, 95% CI 1.4-5.1)[48].In addition to the known risk factors, our LR prediction model uncovered some new infuences, such as CCI.No literature was retrieved to confrm the relationship between CCI and medication adherence, and the investigators hypothesized that a higher CCI indicates that the patient has more comorbidities or more serious comorbidities and is more likely to have a combination of multiple medications, and poly-medication is one of the risk factors for nonadherence to analgesic medications [49].
Te development of an adherence prediction model can help identify patients at high risk of opioid nonadherence promptly and can better inform clinical staf of interventions to improve medication adherence, thereby enabling individualized education and care for patients with cancer pain.In actual clinical practice, clinicians generally do not notice treatment nonadherence until after patients have experienced it multiple times, and then take action.If clinicians were able to predict the risk of opioid nonadherence in advance, this would allow them to proactively adopt intervention strategies and focus their attention and eforts on high-risk patients, reducing unnecessary interventions in those patients with high adherence.
Limitations of this study include (1) the small sample size of the dataset used for our study, the limited number of features, and the high imbalance between adherent and samples, resulting in the model constructed that is not yet sufciently stable and has room for further optimization; (2) only 80% was used as threshold to classify patients as adherent or nonadherent, so the stability and applicability of the model under other thresholds cannot be examined; (3) as this study was a secondary analysis, it lacked necessary information to further classify adherence into diferent subtypes, and to develop a comprehensive model to predict diferent types of nonadherence; (4) the monocentric nature of the data is a limitation of the study, and it is unclear whether model performance is extrapolated to other healthcare institutions.In future studies, multicenter standardized data could be assembled for training and validation of the model.
Tere are numerous previous studies on factors infuencing opioid adherence in patients with cancer pain, but few studies on predictive models of opioid adherence in patients with cancer pain.To the best of our knowledge, this study is the frst to develop prediction models based on these risk factors using an ML approach.In our study, fve MLbased modeling algorithms were developed for adherence  When the coefcient is less than zero, it indicates that the greater the value taken for the characteristic, the greater the likelihood that the patient will experience nonadherence, and vice versa.Te larger the absolute value of the coefcient, the greater the contribution of the eigenvalue to the prediction of adherence.
Pain Research and Management prediction, and after comparing the specifcity, sensitivity, accuracy, precision, AUC, and F1 score, we concluded that LR was the most optimal model, and showed a good predictive ability to identify nonadherent patients.Future studies can develop more accurate prediction models for cancer pain patients at diferent treatment stages, include patients from multiple centers and levels of hospitals, increase the representativeness of the sample, and continuously revise and improve the prediction model for opioid adherence in cancer pain patients.In addition, programming the prediction model to design an automated tool for predicting adherence, which may then be used by clinicians, pharmacists, and nurses as a decision support tool could be explored to improve cancer pain management.
In conclusion, this study developed fve ML-based modeling algorithms for opioid nonadherence prediction, with LR being the most desirable model and showing good predictive power for identifying nonadherent patients.Te model identifed several important predictors of opioid nonadherence in patients with cancer pain, and this information may help clinical staf identify patients at high risk of nonadherence and develop targeted intervention strategies to promote patient adherence and improve patient beneft.

Figure 1 :
Figure 1: Flow diagram of the study.

Figure 2 :
Figure 2: ROC curves with mean AUCs from cross-validation of the fve diferent models on test set data.ROC, receiver operating characteristic; AUC, area under curve; RF, random forest; XGBoost, eXtreme gradient boosting; MLP, multilayer perceptron; SVM, support vector machine, LR, logistic regression.

Figure 3 :
Figure 3: DCA of the LR model on the test set data.Te X-axis is the threshold probability, and the Y-axis is the net beneft rate.Te red line indicates the net beneft received by patients when clinical interventions are performed according to the LR model, calculated as true negative diagnosed by the model × the value of beneft from applying clinical interventions to true negative patients − false negative diagnosed by the model × the value of loss from applying clinical interventions to false-negative patients.Te curve "Treat all" indicates the beneft of treating all patients as opioid nonadherent and administering the clinical intervention, and the line "Treat none" indicates the beneft of treating all patients as well as adherent to opioids and not giving the intervention.As shown, LR has a greater beneft than all patients receiving or none of the clinical interventions for most of the threshold probability range.LR, logistic regression; DCA, decision curve analysis.

Figure 4 :
Figure4: Importance of predictor variables for nonadherence in logistic regression model.BMQ, beliefs about medicines questionnaire; CCI, Charlson comorbidity index; SDS, self-rating depression scale; PS, performance status; BMI, body mass index; NRS2002, nutritional risk screening 2002.When the coefcient is less than zero, it indicates that the greater the value taken for the characteristic, the greater the likelihood that the patient will experience nonadherence, and vice versa.Te larger the absolute value of the coefcient, the greater the contribution of the eigenvalue to the prediction of adherence.

Table 1 :
Demographic characters of participants.

Table 2 :
Clinical data of participants.
Pain Research and Management2.7.Additional Statistics.If not stated diferently, categorical variables are reported as count (%), and continuous variables are reported as Mean ± SD when they obey a normal distribution; otherwise, they are expressed as median (interquartile range, IQR).A two-sided level of signifcance of 0.05 was applied to general comparisons.T-test, Mann-Whitney test, Chi-square test, and Cohen's kappa statistic were performed using IBM SPSS statistics 27.0 software and R software (version 4.3).All fve ML models were operated using Python sklearn.