Outcome prediction of electroconvulsive therapy for depression

Introduction: We developed and tested a Bayesian network(BN) model to predict ECT remission for depression, with non-response as a secondary outcome. Methods: We performed a systematic literature search on clinically available predictors. We combined these predictors with variables from a dataset of clinical ECT trajectories (performed in the University Medical Center Utrecht) to create priors and train the BN. Temporal validation was performed in an independent sample. Results: The systematic literature search yielded three meta-analyses, which provided prior knowledge on outcome predictors. The clinical dataset consisted of 248 treatment trajectories in the training set and 44 trajectories in the test set at the same medical center. The AUC for the primary outcome remission estimated on an independent validation set was 0.686 (95%CI 0.513 – 0.859) (AUC values of 0.505 – 0.763 observed in 5-fold cross validation of the model within the train set). Accuracy 0.73 (balanced accuracy 0.67), sensitivity 0.55, specificity 0.79, after temporal validation in the independent sample. Prior literature information marginally reduced CI width. Discussion: A BN model comprised of prior knowledge and clinical data can predict remission of depression after ECT with reasonable performance. This approach can be used to make outcome predictions in psychiatry, and offers a methodological framework to weigh additional information, such as patient characteristics, symptoms and biomarkers. In time, it may be used to improve shared decision-making in clinical practice.


Introduction
Depression is a leading cause of disability according to the World Health Organization, affecting one in six people during their lifetime (Kessler et al., 2005;WHO, 2022). Electroconvulsive therapy (ECT) is the most effective available treatment for severe depression (Lisanby, 2007). In practice, ECT is usually reserved for patients who show insufficient response to antidepressant medications and psychotherapy, in part because of stigma and anticipated cognitive side effects (Leiknes et al., 2012). Although highly effective on a group level, a substantial number of patients show no or insufficient response to ECT. There are several factors associated with response to ECT, including age and presence of psychotic symptoms (van Diermen et al., 2018). However, in current psychiatric practice, neither systematic assessment of these independent predictors, nor assessment of cumulative predictive value of multiple predictors are routinely used in the decision to initiate ECT for individual patients. As a result, treatment outcome on the individual level remains largely unpredictable.
Clinical decision support systems (CDSSs) are computerized tools which provide clinicians individualized information based on various sources of information, for instance demographic characteristics and information from electronic health records (EHRs). CDSSs make use of prediction models or algorithms for systematic assessment of information. CDSSs are used in several clinical specialties, such as in cardiovascular medicine (collaboration SwgaECr 2021; Hageman et al., 2022). In psychiatry, the availability of CDSSs is modest at best, as was illustrated by Koposov and colleagues (Koposov et al., 2017). Bright and colleagues give an overview of clinically implemented CDSSs across all medical specialties in a systematic review and meta-analysis of randomized controlled trials. They found that clinicians are more likely to appoint the appropriate treatment when informed by CDSSs compared to clinicians who did not use these systems, based on 46 studies across diverse venues and systems (OR 1.57, 95%CI 1.35 -1.82) (Bright et al., 2012). A recent Cochrane review by Stacey and colleagues, which assessed the effect of decision aids, reported that patients who used CDSSs were better informed on treatment options, felt more knowledgeable, and were likely to have more accurate risk perceptions (Stacey et al., 2017). A CDSS which can predict the effect of ECT for individual patients could be useful to inform patients and facilitate shared decision making before treatment is initiated. In order to realize this, a prediction model for ECT outcome is required.
In this study, we developed a personalized effect prediction model for the prediction of remission after ECT, and, secondary, ECT nonresponse, using a Bayesian network (BN) model. BNs are a combination of intuitive graphical representations of causal or predictive dependencies between variables, and the corresponding underlying quantitative model (for an insightful tutorial aimed at psychopathology researchers see (Briganti et al., 2022)). The presence and underlying quantitative model of these dependencies can be derived from data, can be obtained from expert opinion, or both (Arora et al., 2019). The aim was to predict the effect of an ECT trajectory using data which was clinically available before ECT was initiated. We used a systematic review to identify predictors of ECT outcome to inform a BN with prior knowledge from literature. Subsequently, we used expert knowledge to further design the BN, and tested its performance in clinical data. Finally, we validated the performance of this prediction model in a validation dataset.

Methods
We used a stepwise approach to create the BN model for ECT outcome prediction: 1) acquiring prior knowledge by performing a literature search for clinically obtainable predictors; 2) Creation and training of a BN using prior knowledge from literature and a clinical dataset; and 3) validation of the trained model in a validation cohort.

Systematic review
To acquire high quality prior knowledge from literature, we performed a systematic review on predictors of ECT outcome, in which we only searched for high quality meta-analyses. We performed a literature search in the online libraries MEDLINE and EMBASE according to the PRISMA guidelines for systematic reviews (Page et al., 2021). The protocol of this review was not registered in advance and is not available for review as such. The question this systematic review addressed was formulated as: "for adult patients undergoing ECT, what are clinical predictors for outcome (response or remission) of ECT". Search terms used were: (ect OR electroconv*) AND predict* AND (remission OR respons* OR outcome*); all published articles available before 16-11-2022 were reviewed. Articles were excluded in screening of title and abstract if they were: non-human, non-English language, when treatment was not ECT and if study design was not a meta-analysis. Systematic reviews without meta-analysis were excluded. When the full-text studies were not available, the authors were contacted. Eligibility criteria for inclusion were studies on predictors of ECT outcomes of which data were readily available at baseline in most patients. These were defined as demographic predictors, clinical assessment predictors, comorbidity predictors, pharmacological or technical ECT aspects predictors. The definition did not include MRI findings predictors, because MRI scans are not performed as standard practice at the start of ECT.
The screening and quality assessment of articles were performed by two independent reviewers (YD and AM), without automation tools. Discrepancies in results were resolved by consensus, or by a third reviewer in case of disagreement (ED). We performed a ROBIS quality assessment to assess the risk of bias in the identified meta-analyses (Whiting et al., 2016). Only studies with an overall low risk of bias were included in model development.
Data were collected from each individual predictor for both remission and response. Standardized mean differences with 95% confidence intervals (95%CI) were collected for continuous predictors, odds ratios (ORs) with 95%CI were collected for dichotomous predictors. Outcome was defined as "remission" or "response", without further specification, in order to include all relevant studies. When two or more meta-analyses provided data for a single predictor and outcome, the authors would, after consensus, only extract data from the most relevant meta-analysis available, based on date of publication and quality assessment. Results of the data extraction were used as priors for the BN model.

Study population
For the BN model, we used individual patient data from patients who were treated with ECT in the University Medical Center Utrecht (UMCU) in the Netherlands between 1 January 2008 and 27 September 2019. We included all patients receiving ECT for a depressive episode, including patients with bipolar and schizoaffective disorders, who had a discharge letter with conclusion of ECT trajectory available. When patients had multiple ECT trajectories, a subsequent trajectory was only included after a time interval of at least 90 days. We retrospectively acquired as many patient characteristics as are routinely collected in standard clinical practice and used these as predictors for the outcomes. These were age, sex, somatic comorbidity, age of onset of symptoms, duration of depressive episode, ECT naivety, co-morbid personality disorder, severity of depression, psychotic features, catatonic features, and diagnostic context of ECT indication (e.g. depressive disorder, bipolar disorder, or schizoaffective disorder). Clinical patient characteristics were extracted from Electronic Health Records (EHRs). Patient data were anonymized using DEDUCE and therefore the institutional medical ethics review board waived informed consent (Menger et al., 2018). Validation of the model was performed in a cohort which consisted of patients who received ECT in the UMCU between 17 July 2018 and 22 October 2021.

Outcomes
The primary outcome was remission after ECT. Remission outcome was assessed by deriving the conclusion from the psychiatrist's discharge letter, stating "remission", indicating an absence of depressive symptoms. This dichotomous outcome has been used in meta-analytic research and has clinical usefulness, because it is informative and understandable for both clinicians and patients (Pagnin et al., 2004).
Non-response was assessed as secondary outcome, and was defined as absence of any amelioration of symptoms of depression. This was also assessed by deriving the conclusion from the discharge letter.

Statistical analysis
To explore the data used for training the model, group means or proportions for the predictor variables were compared between remission and non-remission using the appropriate hypothesis tests, where we used Bonferroni correction to correct for multiple testing. In case of missing data we used multiple imputation, using IterativeImputer (Python) for the training set and MICE for the validation set (R).
To gain insight into the associations and/or causal relations between predictors on multiple levels, a BN was fitted on the UMCU data with the "bnlearn" package in R (Scutari, 2010). Prior to learning the structure of the network, black-and whitelists were created based on the data derived from the meta-analyses combined with expert knowledge from authors YD, MS and ED. Associations on these lists were either by default included (for the whitelist)) in, or excluded (for the blacklist) from the network. Adding this sort of prior knowledge can vastly improve the stability of overparameterized networks (Briganti et al., 2022). On the blacklist, response or remission was excluded as a predictor of other variables in the network, and age and gender were excluded as being dependent on other variables in the network. Somatic comorbidity and cognitive disorder were also excluded as possible direct predictors of non-response or remission. On the whitelist, all predictors except catatonic symptoms, forced care and first ECT trajectory were included as direct predictors of response or remission. Personality disorder was included as a direct predictor of somatic comorbidity, age of onset, relapse, episode duration and psychotic and catatonic symptoms. The structure of the BN was determined through applying the score-based (i. e., aimed at optimizing the predictive performance of the network) "Hill-Climbing" algorithm on the data 100 times through bootstrapping: to improve stability, only dependencies occurring in at least 85% of the bootstrapped networks were included in the final structure (Briganti et al., 2022).
Based on the dependencies found in the BN a hierarchical Bayesian logistical regression model was fitted specifically for predicting response to ECT with the "arm" package in R, as the bnlearn package did not offer fitting such models with prior information. As predictors, all variables found to be associated with response to ECT from the meta-analyses and all variables included as predictors of the outcome variable in the BN were included. Priors on coefficients were chosen to be normal, with mean and standard deviation either estimated through the metaanalysis, or set to 0 and 1 in the absence of prior information; all prior settings can be found in Table 2.
To generate insights into model performance, 5-fold cross-validation was performed and mean accuracy, ROC-AUC (receiver operator characteristics area under the curve) and corresponding 95% confidence intervals were calculated. ROC-AUC can be interpreted as the ability of the predictor to distinguish between true positive and negative cases (or probability that it will do so correctly). The model was subsequently validated in the independent dataset, where ROC-AUC curves, sensitivity, specificity and calibration curves were calculated to assess the external validity of the model. The model creation and validation were in accordance with the TRIPOD statement (Collins et al., 2015). For a comparison between BN and other statistical approaches, we used the R package "caret" to train and evaluate other machine learning techniques: regression (penalized and unpenalized), random forest and boosting (glm, glmnet,rf, xgbTree). We used 10-fold cross validation with an oversampling within the cross validation, according to ROSE sampling. Results were reported in the material.

Systematic review
We found a total of 1638 articles after removing duplicates. Of these, 1614 were excluded after screening of title and abstract. A total of 24 articles were sought for retrieval, of which six were eventually not available. Full text screening for eligibility was performed in 18 articles, of which five were included (van Diermen et al., 2018;Baldwin and Oxlad, 1996;Haq et al., 2015;Havaki-Kontaxaki et al., 2006;Kho et al., 2003). One additional article was found while screening manually for relevant references in the included articles (Heijnen et al., 2010). Flowchart and quality assessment summary are reported in the supplementary material. We included three meta-analyses with an overall low risk of bias (van Diermen et al., 2018;Haq et al., 2015;Heijnen et al., 2010). All meta-analyses showed overlap of included studies. For the predictors psychotic symptoms, age, melancholic symptoms and depression severity, two studies reported data on response outcome (van Diermen et al., 2018;Haq et al., 2015). For these predictors, we only extracted data from the meta-analysis of Van Diermen and colleagues (van Diermen et al., 2018), because this meta-analysis was more recent and was assessed as having an overall lower risk of bias in the ROBIS quality assessment. For the predictor medication failure, data for remission outcome from Heijnen and others (Baldwin and Oxlad, 1996) and data for response outcome was extracted from the meta-analysis from Haq and others (Pagnin et al., 2004). Extracted data of predictors of ECT outcomes were reported in supplementary material.

Training and validation datasets
We included a total of 248 treatment trajectories of patients receiving ECT at the UMCU between 2009 and 2019 in the training dataset, of which 90 (36%) were classified as remission and 63 (25%) as non-response. The validation set consisted of 44 independent treatment trajectories, of which 11 (25%) were classified as remission and 5 (9%) as non-response. Summary statistics (mean values or proportions for patients with and without remission) of both datasets can be found in Table 1. In the training data, nine treatment trajectories had an unknown episode duration. In the independent validation cohort, 23 cases had missing data, for the relapse, episode duration, catatonic symptoms and age of onset predictors. All missing variables were imputed. A total of 32 treatments had missing values for their number of medication switches in the current episode and nine treatments had an unknown episode duration. If any other clinical parameter was missing, the treatment trajectory was excluded from analysis. Age was the only predictor with statistically significant differences after Bonferroni correction between non-responders and patients with response/remission, with higher age being associated with higher remission rate, in both the training set and in the validation set (p = 0.0005 and p = 0.0034 respectively).

Bayesian network model and hierarchical model for predicting remission
The Bayesian network found with the Hill-Climbing algorithms revealed no new direct dependencies between predictor variables and outcome variable remission that were not already present on the whitelist provided by the experts and meta-analysis (supplementary figure 1.) The model containing solely priors from literature had an AUC of 0.63 (95% CI 0.56 -0.70) and an accuracy of 0.63 (balanced accuracy: 0.60) for predicting remission of UMCU patients in the training set. After updating the model coefficients using the data of UMCU patients, the AUC was 0.629 (values 0.505 -0.763 observed in 5-fold cross validation) and the classification accuracy estimated through 5-fold crossvalidation was 0.637 (balanced accuracy: 0.569). The trained hierarchical Bayesian logistic regression model and an overview of priors can be found in Table 2 An overview of data of misclassified cases of remission is given below in Table 3. False negative cases (patients predicted as not achieving remission after ECT while in reality they did), were generally younger, without psychotic symptoms. False positive cases were generally older, with psychotic symptoms, and did not have personality disorders, which were strong predictors in the final model for remission (see Table 2).
These false positives could possibly explain the decreasing trend in the calibration plot in the bins with the highest predicted probabilities of remission, where the model overestimates the success probabilities (see Fig. 1).   Table 3 Group means (for continuous data) or proportions with the corresponding property (for dichotomous data) for misclassified cases in the validation set, split based on false negative or false positive misclassification.

Bayesian network model and hierarchical model for predicting secondary outcome non-response
In the training set, 63 (25%) of trajectories were classified as nonresponders. The AUC for the model for predicting the secondary outcome, ECT non-response, was 0.644 (values 0.603-0.675 observed in 5-fold cross-validation), with a classification accuracy estimated through 5-fold cross-validation of 0.746 (balanced accuracy: 0.542). In the validation set, non-response occurred in 5 out of 44 patients. Validation of the updated model resulted in an AUC of 0.528 (95%CI 0.250-0.807) for predicting (non-)response, with an accuracy of 0.84 (balanced accuracy: 0.562), a sensitivity of 0.20 and a specificity of 0.92. The trained hierarchical Bayesian logistic regression model and an overview of priors for non-response can be found in supplementary files. The four alternative machine learning techniques showed comparable AUCs: 0.545 for unpenalized regression (glm), 0.628 for penalized regression (glmnet), 0.591 for random forest (rf) and 0.614 for boosting (xgbTree). Other metrics were reported in the supplementary material.

Discussion
In this study, we created and temporally validated BN model to predict outcome after ECT for depression, using prior knowledge from literature combined with single center clinical patient data. We found a mean AUC of 0.629 (values 0.505 -0.763 observed in 5-fold crossvalidation) for the training set and an AUC for the validation set of 0.686 (95%CI 0.513-0.859) for predicting remission to ECT. These findings suggest that probability of remission of a depressive episode using ECT can be reasonably well estimated with readily available clinical predictors for individual patients. For non-response, we found a mean AUC of 0.644 and an AUC for the validation set of 0.528 (95%CI 0.250-0.807) High-quality meta-analyses are considered as the highest level of evidence in evidence-based medicine. However, one of the downsides of meta-analyses is that the aggregated data have no direct clinical value to individual patients (Berlin and Golub, 2014). In this study we used the knowledge from the best meta-analyses available in a BN model to create a clinical decision support system which calculates personalized outcome predictions for ECT. Although these methods have been studied before, this study is, to our knowledge, the first to investigate the outcome of ECT using a BN. Previous studies of BNs in psychiatry focused on dementia and cognitive impairment (Jin et al., 2016;Gross et al., 2018;Moreira and Namen, 2018). BNs are mostly used in the fields of cardiology and oncology, but have not yet been adopted as a standard technique in medical decision making. One explanation is that previous publications on BNs mostly emphasized technical aspects instead of clinical usefulness (McLachlan et al., 2020;Kyrimi et al., 2021). We found that the addition of prior information to our model did not influence the AUC for remission, but marginally reduced CI width, from an AUC of 0.686 (95%CI 0.491-0.881) in the no priors model, to an AUC of 0.686 (95%CI 0.513-0.859) in the final model. Based on these findings, including prior information hypothetically decreases the sampling variability in a model, by increasing the number of samples of which data is derived. An additional value of priors is that they can be used as an extra validation of findings in a study cohort. If significant discrepancies are observed, further investigation on bias is warranted. There was no improvement in model performance between BN and other machine learning approaches. This shows that our model is comparable with more standard approaches. Further optimization of performance may require more predictors.
Our findings showed that the presence of psychotic symptoms was a strong predictor for remission, as well as the absence of a personality disorder and the absence of medication failure. These findings were expected because previous studies which identified these variables were used as prior knowledge in our study (van Diermen et al., 2018;Heijnen et al., 2010). Several studies found reduced effectiveness of ECT in patients with personality disorders (Yip et al., 2021;Prudic et al., 2004). Higher age was a statistically significant predictor for remission in our study, which is in line with previous research (van Diermen et al., 2018). The secondary outcome of non-response did not yield significant results.
In the misclassification analyses and calibration plots for both remission and non-response, we found a decreasing trend in the plots in the higher predicted probabilities, resulting in an overestimation of success observed probabilities (Fig. 1). Specificity was relatively high, but several cases were falsely positive, resulting in low sensitivity, as reflected in the low balanced accuracy We infer that the dataset may be confounded. However, because of the small sample size of the validation cohort, we cannot assess to what extent. Exploratory analyses of potential confounders, preferably in a larger validation cohort, may yield additional clinical predictors. Next to clinical and demographical parameters, several previous studies reported on biomarkers as predictors of ECT outcome, including MRI, EEG and genetic findings (Luykx et al., 2022;Levy et al., 2019;Simon et al., 2021). Hypothetically, the Fig. 1. Calibration plots of the model for prediction remission (left) and non-response (right) on the validation set. Patients in the validation set were divided into 5 or 4 equal bins, depending on the probability of remission or non-response as predicted by the model. For those bins, the observed probability of remission or nonresponse and corresponding upper-and lower confidence bounds were estimated based on the patient data, resulting in the figures depicted above. accuracy of our model could be increased by including these predictors. However, the problem with these data is that these are not routinely obtained in clinical practice, and therefore often unavailable for the treatment decision about ECT. Therefore, although we were unable to include biomarkers in the model due to unavailability in our data, the clinical model presented here may be easier to implement in clinical practice than a model based on biomarker data.
Although outcome prediction of ECT may benefit shared decisionmaking, prospective studies are necessary before this model can be implemented as CDSS in standard practice. For example, the subjective experience and needs of individual patients are essential for treatment decisions but were not included here. Also, all factors that may be of influence should be reported in a standardized manner, covering items from all clusters of the biopsychosocial model. Moreover, in our sample, the decision to initiate ECT was already made. This resulted in a selected population of patients who were willing to undergo ECT. To assess clinical usefulness, it is necessary to also analyze the patients who decide not to start ECT, and why this decision is made. Misclassification bias may arise after implementation if treatment decisions are made differently because they are informed by a CDSS, and this adaptive change in decision making is not accounted for. One solution for this potential bias is a stepped-wedge cluster randomized controlled trial, in which the CDSS intervention (and its impact on outcomes) is gradually introduced and evaluated at sites (Hemming et al., 2018). Another factor is the unknown generalizability of findings from our single center study at a university hospital to other treatment settings. We speculate that this could have resulted in an increased severity of depression in our sample, and maybe in other unknown selection biases. An (inter)national, multicenter trial could increase generalizability of our current findings. When looking ahead, a CDSS based on the studies mentioned above could fundamentally alter the care for patients with affective disorders. Currently, ECT is an end-of-the-line treatment, and is initiated when other non-pharmacological and pharmacological treatments have failed. A thoroughly validated CDSS may advise to either start ECT as a first line treatment, or advise to not start ECT at all, given the low chance of success and/or side effects. In the end, a CDSS will provide an advice, it is up to the patient and psychiatrist to make the decision.
Our model did not include adverse effects of ECT. This was due to the fact that adverse effects were not recorded systematically, which may have led to a reporting bias. Adverse effects of ECT consist of amnesia, headache and nausea and occur in most patients during treatment (Andrade et al., 2016). Adverse events may be mild, but can also be a reason to halt ECT prematurely, for example in the case of severe amnesia or delirium. Halting treatment may consequently influence the outcome. We hypothesize that there may also be dependencies between these predictors, and that these could be incorporated to the BN model. Additionally, the inclusion of data generated during each session of ECT, such as seizure duration could be used to predict outcomes more accurately during the treatment. However, this would require a model with repeated measurements, with updated probabilities after each session. This approach could guide psychiatrists and patients in their decision to continue, stop or alter frequency of ECT. We aim to expand our model to include these factors and to further test for generalizability in future work.
We used a systematic review of meta-analyses for the collection of prior knowledge. A downside of this method is missing data of recent studies which are not yet included in a meta-analysis. Another problem was that several studies were included in more than one meta-analysis, and that meta-analyses on the same subject reported different outcomes. We considered risk of bias smallest if we analyzed the searches of multiple research groups and selected the one meta-analysis with the highest quality, with the potential risk of sacrificing some recency of data, and loss of prior information based on single studies. An admission for ECT treatment includes more than only the performing of ECT. Other aspects of the treatment, such as changes in medication, social factors, and psychotherapy may be of influence in the outcome of the treatment. In our study, detailed information specific medication switches, including dosing and timing was not available. Therefore, we de facto studied an admission for ECT in our hospital, including all standard care on pharmacological optimization, social and psychotherapy interventions. A prospective study design, with inclusion of standardized reporting on these factors may provide additional insights. We used clinical discharge letters with the final outcome of ECT to define the outcomes remission and response. Quantitative assessment of depression, for instance using the Hamilton Rating Scale for depression (HRSD) or Montgomery-Åsberg Depression Rating Scale (MADRS), is often used in clinical trials (van Diermen et al., 2018;Hamilton, 1967;Montgomery and Asberg, 1979). Outcomes remission and (partial) response are defined using a reduction of the score by a certain percentage, or below an arbitrary threshold. The potential upside of this approach is that, in theory, treatment can be evaluated objectively. However, there is an ongoing debate about the use of the reliability and validity of depression instruments (Fried et al., 2022). One of the hypothetical downsides of depression instruments is that the score is comprised of several symptom clusters. An equal reduction in scores of two patients after ECT may not resemble the same effect. Additionally, in clinical practice, standardized application of quantitative assessments requires additional time and training of staff. Therefore, we chose to use the most clinically relevant outcome assessment available, the conclusion of the discharge letter. This outcome included both clinician assessment and subjective patient experience. There are downsides to this approach, for there is no standardization, and deriving the outcome requires interpretation. As a result, there may have been both false positives and false negatives. Although discharge letters are far from a gold standard, this outcome does offer an integrated and personalized conclusion. Further conceptual research in validity of depression severity may use these integrative concepts to formulate a standardized and clinically useful outcome measurement. In 23 cases, we had missing data on clinical variables. We used multiple imputation to make optimal use of data. Although multiple imputation is superior to complete case analysis regarding potential bias, it may influence model performance (Steyerberg, 2009). In this study, we used a validation sample from our own hospital setting. In general, external validation usually reduces performance. In our study, we assume that the greatest effect on the performance will be due to a selection bias of the population of our center, a tertiary care university clinic. This selection bias may be based on residential area of patients and reasons for referral from other mental health institutions, which may be correlated with comorbidity, educational level and support from a community. To assess generalizability of our findings, external validation is necessary to account for setting-specific confounders and selection bias. Possibilities for retrospective external validation could be (multicenter) ECT registries. Prospective validation in a multicenter cohort study, as stated above, can yield even more insights.

Conclusion
In this study, we found that a BN model comprised of prior knowledge and clinical data can predict remission of depression after ECT with reasonable performance. This approach can be used to make outcome predictions in psychiatry, and offers a methodological framework to weigh additional information, such as patient characteristics, symptoms and biomarkers. In time, it may be used improve shared decision-making in clinical practice.