Identifying Predictors of Psychological Problems Among Adolescents With Congenital Heart Disease for Referral to Psychological Care: A Pilot Study

Background The lifelong care of patients with congenital heart disease (CHD) typically begins at a young age, giving paediatric cardiologists a unique perspective on the mental health of their patients. Our aim was to describe and predict reported psychological problems among adolescents with CHD. Methods A retrospective review was performed on patients aged 12-17 years who presented to the congenital cardiology clinic during a 1-year timeframe. The presence of psychological problems was collected along with CHD class, clinical history, developmental delay, and patient demographics. We described the prevalence of psychological problems and then, using machine learning algorithms, trained and tested optimal predictive models. Results Of the 397 patients who met inclusion criteria, the lifetime prevalence of any reported psychological problem was 35.5%. The most prevalent reported problems were attention-deficit/hyperactivity disorder (18.9%), anxiety (17.6%), and depression (16.1%). Contrary to our expectations, we could not predict the presence or absence of any psychological problem using routine clinical data. Instead, we found multivariate models predicting depression and attention-deficit/hyperactivity disorder with promising accuracy. Prediction of anxiety was less successful. Conclusions Approximately 1 of 3 adolescents with CHD presented with the lifetime prevalence of 1 or more psychological problems. Congenital cardiac programmes are in a position of influence to respond to these problems and impact their patients’ mental health as part of a comprehensive care plan. The discovered models using routine clinical data predicted specific psychological problems with varying accuracy. With further validation, these models could become the tools of routine recommendations for referral to psychological care.

It is well known that congenital heart disease (CHD) is the most common type of birth defect, with a birth incidence of approximately 1 per 100 births. 1,2Long viewed as a paediatric-specific disease, patients with CHD now regularly survive into adolescence and adulthood as a result of advancements in surgical correction and clinical management. 3ow, it is estimated that 2.4 million people in the United States alone are living with CHD, 12% of whom are diagnosed with severe CHD. 4 This increasing awareness of CHD as a lifespan disease has prompted study into adolescent and adult-onset comorbidities.][7][8][9][10][11][12][13] In the general population, adolescents exhibit a lifetime prevalence of 49.5% for psychological disorders, and mental health services among adolescents are often underutilized. 13,14Clinical presentation of CHD typically occurs in infancy or childhood and the patient is supported by cardiology specialists beginning at a young age, beginning a patient-clinician relationship that will likely last several years or decades.Given the clinical context of this patient population, along with the adolescent concerns about psychological disorders, there is an opportunity for the paediatric cardiology clinic to be a gateway for psychological care.
The question then becomes: how can paediatric cardiologists be empowered to assess their patients' risk of psychological problems, allowing for appropriate psychology referrals, without increasing clinician burden?Part of the answer may be found in 2 emerging and synchronistic areas of medicine: clinical predictive models based on machine learning (ML) and precision medicine. 15There are various types of ML; however, in the clinical context, ML can be defined as computer systems that can learn, either supervised or unsupervised, using algorithms to analyse and draw inferences from patterns found in patient data for descriptive, predictive, or prescriptive purposes.7][18][19][20][21][22][23] In paediatric cardiology, the utility of predictive models is being explored with a focus on cardiac surgery, cardiac imaging, interventional procedures, risk M ethodologie : Nous avons effectu e une revue r etrospective des dossiers de patients âg es de 12 à 17 ans qui se sont pr esent es dans une clinique de cardiologie pour anomalies cong enitales sur une p eriode d'un an.Nous avons recens e toute pr esence de problèmes psychologiques, en plus de la classe de la cardiopathie cong enitale, des ant ec edents cliniques, des retards de d eveloppement et des caract eristiques d emographiques des patients.Nous avons egalement d ecrit la pr evalence des troubles psychologiques; puis, à l'aide d'algorithmes d'apprentissage automatique, nous avons donn e une formation sur des modèles pr edictifs optimaux et les avons test es.R esultats : Chez les 397 patients qui avaient r epondu aux critères d'inclusion, la pr evalence de la survenue d'un problème psychologique au cours de leur vie s' elevait à 35,5 %.Les troubles signal es le plus souvent etaient le trouble d eficitaire de l'attention avec hyperactivit e (TDAH) (18,9 %), l'anxi et e (17,6 %) et la d epression (16,1 %).Contrairement à nos attentes, nous n'avons pas pu pr edire la pr esence ou l'absence de troubles psychologiques à l'aide des donn ees cliniques courantes.Nous avons par contre d ecouvert des modèles multivari es pouvant pr edire la d epression et le TDAH avec une pr ecision prometteuse.Toutefois, les pr edictions relatives à l'anxi et e se sont av er ees moins exactes.Conclusion : La pr evalence de la survenue d'au moins un problème psychologique chez les adolescents atteints de cardiopathie cong enitale au cours de leur vie etait d'environ un adolescent sur trois.Dans le cadre d'un plan de soins complet, les programmes en cardiologie cong enitale se trouvent en position d'influence pour r epondre à ces problèmes et ainsi am eliorer la sant e mentale des patients.Par ailleurs, les modèles que nous avons d ecouverts à l'aide des donn ees cliniques courantes ont permis de pr edire des problèmes psychologiques en particulier, mais avec une pr ecision variable.Après une validation plus pouss ee, nous pourrions recommander ces modèles de façon syst ematique afin d'orienter les patients vers des soins psychologiques.
][39][40][41][42][43] The aim of this study is to describe our population from a representative sampling and to work towards the discovery of a pragmatic set of predictors using ML, which could be readily applied to the congenital cardiology clinic.To this end, we stayed "within the chart" in our data analysis, exploring the predictors of chart-noted psychological problems from patient data that would be readily available to the everyday cardiology practice.With sufficient replication, we hope that this line of research can eventually provide a reliable and valid clinical tool for estimating the likelihood of psychological problems in patients with CHD.

Materials and Methods
A retrospective chart review was performed on patients, 12-17 years of age, who presented to the congenital cardiology clinic of a large tertiary-level medical system located in the Midwestern United States during a 12-month timeframe from 2018 to 2019.
Patients meeting the following criteria were included in the sample population for the chart review: diagnosis of CHD, occurrence of a clinical visit in a 1-year timeframe, and an age of 12-17 years at the time of the clinic visit.Patients were excluded with a documented diagnosis of genetic and/or syndromic abnormalities including but not limited to 1p36 deletion, fetal alcohol syndrome, CHARGE syndrome, cri du chat, Ellis-van Creveld, Holt-Oram, 11q terminal deletion, Marfan, DiGeorge, Noonan, rubella, trisomy 13, trisomy 18, trisomy 21, Turner, VACTERL, Williams, and Wolf-Hirschhorn (additional rare conditions were excluded as applicable).Patients with genetic syndromes were excluded from the sample because of the known associations with developmental delay and are routinely grouped into a different risk category and referred to the neurodevelopmental clinic.
The review was conducted using electronic medical records with covariate extraction from cardiology appointment notes and patient self-report general questionnaires regularly administered during clinic appointments.
The outcome variable is the lifetime presence or absence of a reported psychological problem.The presence of a psychological problem outcome was determined from 3 primary areas within the patient chart: documentation of psychological diagnoses, documentation of psychological problems in cardiology clinic notes, and self-reported patient medical history documentation, which are administered regularly at the beginning of outpatient clinic appointments.These 3 collection methods were used as some patients who seek subspecialty care at the study site receive general care at other institutions.Self-reported patient medical history documents specifically request information regarding the patient's history of depression, anxiety, counselling or psychological testing, learning problems, and reasons for previous overnight hospital admissions.For the purposes of this study, psychological problems were categorized into anxiety, depression, attention-deficit/hyperactivity disorder (ADHD), bipolar disorder, and other.
Covariates include standard patient demographics of age, sex assigned at birth, and race.To evaluate the association of clinical history to reports of psychological problems, several covariates associated with cardiac clinical history were collected.CHD classification was assigned based on the 2018 American Heart Association (AHA) guidelinesdmild, moderate, or severe complexity. 44Collected CHD diagnoses were stratified based on these guidelines.Given the comorbidity of development delay with psychological disorders, a diagnosis of developmental delay was also collected; however, specific diagnoses of developmental delay were not captured.[47] Predictor selection rationale The purpose behind the selection of covariates to be used as predictors was two-fold.First, the availability for data to be readily extracted from nonstandardized clinician reports limited how extensive the list of covariates could be.Secondly, the study goal determined that the focus of the analysis was on clinical covariates that could be easily extracted from within the chart.These 2 factors dictated that the clinical predictive model concentrates on cardiac clinical history, preterm history, and standard demographics.These factors also determined, and consequentially limited, how the study captured the reports of psychological problems.Clinical cardiac covariates were a primary focus of this study as it was posited that the severity of CHD lesion along with interventions, surgeries, and the corresponding course of care may influence psychological outcomes.Other aspects of a patient's history, such as socioeconomic status and sociocultural characteristics, were not included in this pilot study.In addition, externally validated psychological diagnostic evaluations were not available for this sample population.However, given the nature of ML and clinical predictive models, all relevant information of a patient's history would be expected to be included in a final iteration.

Analysis
Sample size.Based on findings by the National Institutes of Mental Health, there was a 50% predicted prevalence of psychological disorders in the study population. 13A sample size calculation using a z-approximation found a necessary sample size of at least 384 for a 0.05 confidence interval.During the study timeframe, 497 unique patients aged 12-17 years with a diagnosis of CHD visited the study site.On completion of the chart review, 100 patients were excluded based on study criteria bringing the final sample to 397 patients.
Descriptive statistics and clinical data.Descriptive statistics and univariate analysis were performed, where continuous variables were presented as mean (AE standard deviation) or median (interquartile range) depending on the normality of distribution, whereas categorical variables were presented as counts and percentages.Differences between the positive or negative psychological problem outcome groups were analysed using the t-test, c 2 test, or Fisher's exact test based on normality.
Variables were re-coded in the following ways: age at first surgery was used to code to a critical CHD exposure variable (cardiac surgery at <1 year of age vs >1 year of age).Because of limitations in the sample size, specific psychological problems were combined to create a binary "presence of any psychological problem" variable.Of note, there were insufficient observations to run models on bipolar and "other" psychological problems in this sample.Because of low numbers of observations in some groups, the race-ethnicity variable was categorized as White, Black, Hispanic, and other.Gestational age, again due to low categorical observation counts, was dichotomized into 2 categoriesd>36 weeks of gestation and <36 weeks of gestationdto represent the history of full-term birth and preterm birth, respectively.
Contingency tables were used to describe the relationship between psychological problem outcomes and the following variables: CHD classification, history of critical CHD, history of cardiac intervention (surgery, catheterization, or either), total number of cardiac surgeries, age at first surgery (both grouped <1 year, 1-5, 5-10, 10-15, 15-18, and by year), total number of cardiac catheterizations, and neurodevelopment delay diagnosis.An additional contingency table was run to assess the proportion of CHD classifications with diagnosed developmental delay.
Machine learning for predictors of psychological problems.We explored the predictors of reported psychological problems using ML techniques: developing models on a random subset of the data (training) and validating against the remaining data (testing).An initial model was run to predict the presence of any reported psychological problem.Then, 3 additional models were created to separately predict the presence of depression, anxiety, and ADHD.For each new model, we randomly sampled 70% of the training dataset to train the ML algorithm.
We used cross-validated logistic LASSO regression for ML, which determines the optimal set of variables.In the interest of discovery, we chose the model that minimized error the most.On the remaining 30% of the data for each model (the testing datasets), the accuracy of each trained model was assessed.This allowed for a practical test of the model that was internal to our dataset, lending some additional confidence to the ability of the discovered models to accurately predict psychological problems.

Results
After the retrospective chart review, there were 397 unique patients meeting inclusion criteria in the study timeframe.Patients were aged 12-17 years with a mean age of 14.5 years.Males comprised the majority of the sample population (61%, n ¼ 241).Racial demographics of the sample population closely mirrored the regional study population demographics.Complete demographic description can be found in Table 1.
The lifetime prevalence of any reported psychological problem in our sample population was 35.5% (n ¼ 141), with the most prevalent disorders being anxiety (17.6%; n ¼ 70), depression (16.1%; n ¼ 64), and ADHD (18.9%, n ¼ 75).The proportion of psychological outcomes reported increased with severity of CHD class, with 30% of simple CHD, 36% of moderate CHD, and 40% of the severe CHD groups reporting psychological problems.However, when cross-tabulated across CHD classifications, the differences in the prevalence of any psychological problem were not found to be statistically significant (P ¼ 0.42).We found some comorbidities, with 2-disorder comorbidities between ADHD, anxiety, and depression ranging from 3.0% (n ¼ 12) to 4.8% (n ¼ 19) of all patients.A total of 2.8% (n ¼ 11) patients had all 3 problems reported in their chart.Although there was some comorbidity, problems reported in any kind of comorbidity with others amounted to approximately 20% (n ¼ 42) of the total number of psychological problems reported in the data.
The reported lifetime prevalence of depression was over twice as high in moderate CHD and severe CHD (18.6% and 17.0%, respectively) when compared with the simple CHD (7.4%).Differences in reported psychological problems across cardiac clinical history did not demonstrate statistical significance in cross-tabulation.Complete descriptive statistics of clinical history can be found in Tables 2 and 3.
The overall prevalence of developmental delay was 12.3% among the sample population.The history of any reported mental disorder diagnoses demonstrated a statistically significant relationship to developmental delay status, with 20% of patients reporting any mental disorder having a history of delay compared with 8% of patients with no psychological problem having a history of delay (P < 0.001).Anxiety and ADHD diagnoses were both found to have statistical associations with developmental delay status (P < 0.0001 and P < 0.001, respectively).More than 25% of reports of anxiety and ADHD had a history of developmental delay.These relationships have been presented in previous literature.How these data and other variables connect to predict the presence of psychological problems remained a question we sought to examine with ML models.

Machine learning for predictors of psychological problems
Presence of any psychological problem.We found no model to predict the presence of any psychological problem that adequately fit the criteria set forth in our cross-validated logistic LASSO regression (Table 1).In our data, a general search for predictors of the presence of any reported psychological problem did not return any models.This diverged from what we expected, which was that there would be general factors indicative of psychopathology in general.We proceeded with attempting ML models for predicting the presence of specific psychological problems.
Anxiety.We first explored predictors for the presence of the specific psychological problem of anxiety using the same ML process.We discovered that the optimal model included one predictor: presence of developmental delay.In a logistic regression, those with the indicated presence of developmental delay in their chart were approximately 6 times more likely to also have anxiety mentioned as a problem or as a listed diagnosis (95% confidence interval from 2.78 to 13.21 times more likely), P < 0.001 (Table 4).
Depression.A set of predictors were discovered by the ML process as optimally predictive of the presence of depression listed in patients' charts.The full model included age as a strong positive predictor, racial/ethnic minority status, fewer cardiac catheterizations, presence of moderate CHD diagnosis (vs simple or complex), and absence of developmental delay (Table 4).Overall, less variance was explained than in the anxiety model.
ADHD.Similar to depression, a set of multiple predictors were discovered by the ML procedure and with a similar level of variance explained.The presence of developmental delay, male sex, known preterm birth, fewer cardiac catheterizations, younger age (though a very small effect), and critical CHD predicted the presence of an ADHD-related problem.
Testing machine-learned models Anxiety.Developmental delay history alone did not perform well as a predictor of anxiety problems in the test data.The area under the curve (AUC) was 0.51.Accuracy was poor at 0.37, where the presence of a delay was somewhat specific for anxiety, 0.76, but not sensitive to it, 0.29.Depression.The discovered model for depression performed better than anxiety, though not highly reliable, with AUC ¼ 0.63 at the optimal cut point (0.18).Accuracy was moderate at 0.64, with balanced sensitivity at 0.63 and specificity at 0.64.
ADHD.The model for ADHD performed best of the 3 discovered models at the optimal cut point (0.21).The AUC was 0.73.Accuracy was good at 0.77, though was less sensitive at 0.56 and more specific at 0.82.

Discussion
This study is an attempt to understand reported psychological problems of adolescent patients with CHD using readily available clinical tools.The main goal is to effect a change in the identification of psychological problems and in referral protocols for adolescent patients in need of mental health services.Of clinical significance, greater than one-third of patients reported 1 or more lifetime psychological problems.This prevalence alone is sufficient to justify using resources to further assess psychological problems in this patient population.Although the prevalence of reported lifetime psychological problems was below the expected 49% for our sample, the observed rates of anxiety, depression, and ADHD in our sample population were higher than the rates reported elsewhere. 48These discrepancies may reflect the exploratory nature of the study and the limitations of the retrospective chart review.
Our initial approach in the search for predictors attempted to characterize patients who have at least one of the listed psychological problems: anxiety, depression, or ADHD.This failed to find any predictors.Turning to each psychological problem individually, we discovered unique sets of predictors for each, which at times directly diverged from one another.Depression and ADHD were the most promising in terms of accuracy of prediction with multivariate optimal models.Anxiety was somewhat more difficult to predict in our model; having a developmental delay appeared to indicate some increased likelihood of anxiety, but that characteristic alone incompletely captured all patients with a listed anxiety problem.Developmental delay was present in all models but interestingly diverged in direction; anxiety and ADHD were more common in the presence of developmental delay, whereas depression was less common.We did not have an explanation for this divergence in direction but were confident that it was not a statistical artefact as these problems were all modelled independently.
This study is an exploratory effort to help cardiologists assess their patients' mental health based on clinical cardiac history.This method requires no specialized examination or questionnaire and uses existing data commonly found in patient charts.In clinics where clinically validated psychological questionnaires are currently administered, a method such as this may augment patient mental health assessment and perhaps encourage intentional patient-centred advocacy towards engaging with psychological care.Multimethod assessments, multiple methods to garner different kinds of information on the same topic, can often increase sensitivity to true positive cases. 35,36The implementation of these chart-based algorithms with patients' responses to self-report questionnaires could offer important redundancy to clinical care where administration of a self-report measure may be overlooked as well as increase the effectiveness of screening as a line of information on the same topic but different in kind.

Strengths and limitations
Our study had several strengths.First, the study site is a large tertiary paediatric medical centre and is the only such centre in the region, giving the institution a large geographic referral base.Secondly, the covariates of sex, age, and racial demographics of the study largely reflect the state's demographics and more specifically the health system's region. 49hird, our methodologies remained "within the chart," and used widely accepted diagnostic criteria.AHA classification is a nationally recognized system, and the covariates are commonly collected as part of a patient's clinical history.Because of these strengths, our results have promising validity though would need to be validated in health care systems external to this dataset before confident implementation in congenital heart clinics.
This study also has recognizable limitations, many of which are commonly found among retrospective studies.Foremost of these limitations is the assessment of reported psychological disorders.Reporting on psychological problems is neither a standard of work nor a standardized process in many congenital cardiac clinics.In part, this is due to the novelty of reporting on noncardiac assessment by the congenital cardiologists.Reporting practices during the study timeframe inhibited a comprehensive review of psychological problems.Consequently, this study primarily relied on patient self-reported medical history and cardiologist notes regarding the history of psychological problems.Further, although there was limited comorbidity among reported psychological problems in our sample, future efforts could build upon the modelling approach presented here with multivariate outcome models allowing for the simultaneous modelling of multiple possible psychological problems instead of modelling each as a univariate outcome.It could then become an empirical matter as to the relative advantages of accounting for comorbidities directly within the statistical modelling.
Lastly, unlike CHD, psychological disorders are not reportable conditions and consequently are not tracked with epidemiologic surveillance in the state of Michigan. 50Hence, accurate surveillance of these problems is inherently difficult and is compounded in vulnerable populations, such as children.Because of these reasons, this study was required to primarily rely on patient self-report, which is not as strong of a measure as psychological outcomes documented by a psychologist.In addition, as not every patient in the sample was subjected to a standardized psychological evaluation, it is likely that some patients have never had a comprehensive assessment.As a result, there may be both patients with subclinical mental health disorders and patients without access to mental health care who would account for undiagnosed psychological disorders.

Future directions
Future research of clinical predictive modelling to determine psychological problems in the CHD population should be aware of several considerations.First, because of largely nonstandard assessment and reporting of psychological problems in the cardiology practice, tracking the impact of ML algorithms on referral patterns and diagnosis after psychological diagnostic evaluation should be prioritized.External validation will provide further data to build the clinical predictive models on.Secondly, the nature of ML and clinical predictive modelling invites large datasets to be used for analysis.Thus, a holistic approach to patient history beyond cardiac clinical history to include factors such as socioeconomic status and socio-cultural factors will assuredly improve the model's predictive outcomes.In addition, this study used the AHA CHD classifications; however, it may be beneficial to increase the granularity of CHD classifications to possibly increase the clinical accuracy of the model as well.To be clear, the model created during this study is not viable for clinical utilization; however, we hope that additional studies may elucidate which aspects of a CHD patient's history have significant influences on his or her psychological outcomes and at what point in a patient's course of care he or she should, if necessary, receive a referral for psychological care.Further iterations of this model are required with multisite validation before real-world application to the paediatric cardiologist's workflow.However, the primary goal of future iterations should continue to be the creation of a pragmatic model that allows for a simple patient referral recommendation in the context of a normal outpatient visit without increasing the burden of work on the provider.

Conclusions
The CHD population is inherently vulnerable to acquiring several comorbidities with age, including psychological problems.Other research has used ML clinical predictive models to identify novel risk factors for psychological problems in specific patient groups. 31,32,35Identification of at-risk patients may allow paediatric cardiologists to refer for psychological care earlier and more consistently, possibly pre-empting a significant psychological event.Anxiety, depression, and ADHD are treatable conditions that, without treatment, can worsen and may negatively impact medical status through functional, relational, and physiological pathways.The study's clinical predictive model demonstrated potential in identifying depression and ADHD among patients with CHD simply based on data collected in the regular course of medical care.The evidence presented in this study gives initial guidance on how to approach mental health assessment as part of a comprehensive CHD care plan.Expanding the clinical predictive algorithm through a multisite study and validating the model with psychological testing could transform comprehensive care for the CHD population.

Ethics Statement
This study adheres to our institutional ethical guidelines and was approved by our institutional review board.

Table 4 .
Machine learning model: relationship of predictors to specific psychological problem outcomes