Trends in mental health clinical research: Characterizing the ClinicalTrials.gov registry from 2007–2018

While the epidemiologic burden of mental health disorders in the United States has been well described over the past decade, we know relatively little about trends in how these disorders are being studied through clinical research. We examined all US interventional mental health trials submitted to ClinicalTrials.gov between October 1, 2007 and April 30, 2018 to identify trends in trial characteristics, comparisons with non-mental health trials, and trial attributes associated with discontinuation and results reporting. International data were excluded to minimize potential confounding. Over this period, mental health and non-mental health trials grew at similar rates, though Industry and US government-funded trials declined and academic medical center/hospital/other (AMC/Hosp/Oth) funded trials grew faster in mental health research. The proportion of trials with safeguards against bias, including blinding and oversight by data monitoring committees (DMCs), decreased. This occurred during growth in the proportion of trials studying behavioral and non-pharmacological interventions, which often cannot be blinded and do not require DMC oversight. There was concurrent decline in pharmaceutical trials. There was significant growth in trials studying Non-DSM (Diagnostic and Statistical Manual-5) conditions (e.g. suicidality and wellness), as well as substance use, anxiety, and neurocognitive disorders. One in 12 trials was discontinued. Trial discontinuation was associated with industry and AMC/Hosp/Oth funders, pharmaceutical interventions, and lack of DMC oversight. Only 29.9% of completed trials reported results to the registry. Decreased results reporting was associated with behavioral interventions, phase 1 trials, and industry and AMC/Hosp/Oth funders. The main implications of these data are that funding is shifting away from traditional government and industry sources, there is increasing interest in non-pharmacological treatments and Non-DSM conditions, and there are changing norms in trial design characteristics regarding safeguards against bias. These trends can guide researchers and funding bodies when considering the trajectory of future mental health research.


Introduction
As of 2001, the World Health Organization published that one in four people worldwide suffers from a mental health disorder across his or her lifetime, and in the United States, 46.6 million people currently suffer from a mental health disorder [1,2]. As of 2016, mental health and substance use disorders accounted for 206.5 million disability-adjusted life years (DALYs), or 8.6% of all DALYs worldwide [3]. As of 2013, 16.7% Americans were prescribed a psychotropic medication, and in that year alone the United States spent $187.8 billion on treating mental health and substance use disorders, only trailing behind expenses to treat diabetes, cardiovascular disease, and back pain [4,5]. However, while significant attention has been paid to the epidemiological impact of mental health disorders, relatively little focus has been placed on characterizing trends in how the research community has responded through clinical research. Specifically, there has been a gap in the literature studying changes in trial features, such as trial funders, trial design, targeted disorders, and interventions types.
To address the need for a public resource that could be used to identify and analyze clinical research, as well as the conditions, diseases, and interventions being researched and how they are studied, the National Institutes of Health created the ClinicalTrials.gov registry in 2000 [6]. In 2007, in accordance with Section 801 of the Food and Drug Administration Amendments Act (FDAAA), all United States non-phase 1 trials involving US Food and Drug Administration (FDA) regulated drug and biological products, as well as non-feasibility trials of FDA regulated devices, were mandated to report to a clinical trials registry [7]. Since then, ClinicalTrials.gov has grown to become one of the largest international registries for clinical research, and currently it contains detailed information on more than 335,000 clinical studies conducted in over 200 countries. It has been analyzed in over 300 research articles to characterize the landscape of clinical research [7,8], and these articles have shed light on publication bias [9], noncompliance with trials registration [10], and selective reporting [11]. They have also identified trends across medical research [12] and within individual fields [12][13][14] of how trials are funded and designed, as well as what medical disorders and interventions are studied.
The field of mental health has only just begun to utilize ClinicalTrials.gov to help answer such questions regarding its clinical trials. The earliest large scale effort was a review of trials between 2007-2010 by Califf and colleagues, which examined Mental Health trials along with trials in Cardiology and Oncology [12]. In 2017, Anand and colleagues identified all trials in the registry relevant to bipolar disorder and observed a disproportional growth of trials in pediatric populations and in trials studying transcranial magnetic stimulation as treatment [15]. In 2019, Arnow and colleagues published a paper in which they identified all mental health trials in the registry from 2007 to 2014, and they evaluated differences in trial characteristics, such as trial design, disorders studied, and interventions tested as stratified by funder type [16]. Their findings included identifying that universities and hospitals funded the majority of mental health trials (64%), followed by governmental agencies (25.6%) and industry (21.5%). They found that the majority of trial characteristics tended to differ by funder. For example, more industry trials studied pharmacotherapy (95.2%) than behavioral interventions (0.9%), whereas government funders studied more behavioral interventions (60.6%) than pharmacotherapies (25.6%). They found that the most commonly studied conditions were mood disorders (40.0%), and that the majority of trials enrolled fewer than 100 participants, were randomized, and employed some form of blinding. Industry-funded trials were notable for enrolling more participants, as well as for using data monitoring committees and masking methods less often.
However, there are several questions that Arnow and colleagues left unanswered. First, while they provided a snapshot of mental health trials registered from 2007 to 2014, they did not provide temporal trends of how these trials changed to assess where the field may be heading. Their assessment of trials also ends in 2014, which limits extrapolation to more contemporary trends. In 2014, the Research Domain Criteria (RDoC) initiative instituted by the National Institute of Mental Health (HIMH) was just starting to shape the landscape of what was studied by government-funded trials [17]. Multiple large pharmaceutical companies had also just dramatically downsized their research arms for mental health, and it remains unclear how these changes shaped the clinical trials landscape in the interim [18,19]. Second, they did not assess how mental health trials differed from non-mental health trials in the registry, which can provide context for interpreting results. Third, while they provided some information about the mental health disorders studied in these trials, the disorder categories they used were broad and often grouped multiple Diagnostic and Statistical  classifications into one category (e.g. "Mood" as provided by ClinicalTrials.gov includes both depressive disorders and bipolar and related disorders). This limits a more nuanced interpretation of how the study of mental health disorders has changed. Lastly, while other fields have utilized the ClinicalTrials.gov database to assess results reporting and trial discontinuation, this has not been done within mental health research.
This study sought to provide clarity for these lingering questions. We performed an analysis of the entire portfolio of United States mental health clinical trials registered in ClinicalTrials. gov from 2007 to 2018. We examined time trends in funder type, study design, disorder categories, interventions, and other trial features by calculating annual growth rates and by stratifying trials into two time periods (early [2007][2008][2009][2010][2011][2012] and late [2013][2014][2015][2016][2017][2018]). We manually parsed the disorders studied using DSM-5 classifications to gain a more nuanced understanding of how the field has changed. We compared mental health and non-mental health trials to gain further perspective on these changes. We also conducted logistic regression and survival analysis to investigate the characteristics of mental health trials associated with trial discontinuation and results reporting to the ClinicalTrials.gov registry.

Data selection and classification
We downloaded records on April 30, 2018 for all 274,029 trials submitted (i.e. some trials may have been submitted but not yet posted) as of April 30, 2018 to ClinicalTrials.gov using the Aggregate Analysis of ClinicalTrials.gov (AACT), a relational database of publicly available ClinicalTrials.gov data [20]. We selected for interventional trials (a category provided within ClinicalTrials.gov) submitted to the registry on or after October 1, 2007 to coincide with the passing of the Food and Drugs Administration Amendments Act (FDAAA) on September 27, 2007 [7]. ClinicalTrials.gov defines 'interventional' trials as "studies in human beings in which individuals are assigned by an investigator based on a protocol to receive specific interventions" [12]. We identified trials relevant to mental health using the Medical Subject Heading (MeSH) terms and Disease Condition terms provided for each trial, as previously described [12][13][14]16]. A psychiatrist reviewed the list of all MeSH and Disease Condition terms in the ClinicalTrials.gov registry, and those terms deemed relevant to mental health were selected and reviewed by another physician (S1 Table). All trials selected through this process were divided among six psychiatrists who manually reviewed the official title and study description to: (i) exclude trials not relevant to mental health; and (ii) categorize the remaining trials according to the disorder index categories in the Section II Diagnostic Criteria and Codes provided by the DSM-5 (S2 Table) [21]. Trials that identified disorders by DSM-IV diagnostic nomenclature were reclassified using equivalent terms in the DSM-5. All psychiatrists reviewed a sample of the same 250 trials to ensure agreement on the labeling criteria. Trial categorizations with any ambiguity were marked and reviewed by another psychiatrist. When appropriate, trials were assigned to more than one DSM-5 category. Trials that did not clearly match any DSM-5 category (e.g. stress, burnout, or suicide) were marked "Non-DSM" conditions. Because requirements for registration and reporting of results to trial registries vary by country, only trials with research sites exclusively within the United States were included in this analysis.

Changes to the initial protocol
We developed a protocol for our analysis that was submitted with our manuscript. It was not pre-registered. We subsequently made several changes to this protocol (detailed in S3 Table). These changes resulted from the helpful direction of our reviewers and in response to the work by Arnow and colleagues, whose paper was published after completion of our initial analysis [16]. In brief, there were eight changes to our initial protocol: 1. We initially analyzed both US and international studies, as this had been done in some other analyses of the registry [7,10,14]. However, because trial registration practices differ significantly by country, there was concern that inclusion of international trials potentially confounded our results (i.e. it was difficult to determine whether observed trends were due to true differences in trial characteristics by region or differences in trial registration). Therefore, our revised analysis was limited to only US trials.
2. Our initial analysis excluded the ClinicalTrials.gov funder category US Fed, which has been done in some other analyses since US Fed comprises only 3.5% of trials in the registry [14]. However, we subsequently combined US Fed-funded trials with NIH-funded trials to form a new funder category called 'US Govt' to better capture changes in US government-funded trials.
3. Our initial analysis included a table of trial characteristics stratified by funder type. However, Arnow and colleagues performed a very similar analysis for mental health trials in ClinicalTrials.gov from 2007-2014 with similar results. Rather than duplicate their work, we discussed other findings from our analysis concerning comparisons of mental health and non-mental health trials in the ClinicalTrials.gov registry.
4. Our initial analysis included enrollment as a trial characteristics to assess over time; however, many trials in the registry only reported anticipated/estimated enrollment. Our reviewers advised that estimated enrollment is an unreliable metric, as many trials do not meet this projected enrollment number. Because studying only the trials that reported actual enrollment would introduce significant bias into our analysis, we removed almost all discussion of enrollment from our revised analysis.
5. We initially clustered Phase 1/2 and Phase 2/3 trials under the phase category 'Not Applicable;' however, our revised analysis grouped these trials with Phase 2 and Phase 3 trials, respectively, as these trials were deemed to have ultimately reached Phase 2 and Phase 3 status.
6. Our initial analysis assessed results reporting within 2 years of trial completion to account for the 12-month reporting period and the mid-point of the available extension time provided for certain trials by the FDAAA Section 801 and the Final Rule [22]. However, our revised analysis included results reporting within the maximum extension time period provided for certain trials by the FDAAA 801 and the Final Rule (i.e. 3 years), as this was deemed to be a more robust analysis.
7. Our initial analysis did not include an assessment of interventions studies, though this was included in the revised analysis.
8. We included the citation of Arnow and colleagues in our revised protocol and manuscript, as their work is fundamental to contextualizing our study and several changes made to our revised analysis [16].

Trial characteristics
We analyzed each trial along the following 13 dimensions:  [16]. Any trial with a sponsor or collaborator that was industry was classified as having an 'industry' funder, and any trial with 'NIH' or 'US Fed' sponsor or collaborating funders was classified as having a 'US government' funder. If trials sponsor/collaborators included both industry and US government funders (n = 112, 1.8% of trials), funding was labeled 'industry,' as we wanted to prioritize the involvement of industry in our analysis of trial characteristics. In their study, Arnow and colleagues identified that the majority of trials with funder labeled 'Other' were funded by universities or hospitals, with the remaining minority including consortiums, foundations, individuals, and community-based associations [16]. For this reason they renamed the ClinicalTrials.gov 'Other' funder category as 'University or Hospital. ' We examined a random sample of 2,500 Other-funded trials in the database and identified 94.7% of these funders as academic institutions or hospitals. We believed the label Other ignores the dominant identity of these agencies and suggests greater heterogeneity than this category comprises. Therefore, we similarly renamed the 'Other' funders in our study as 'Academic Medical Centers/Hospitals/Other.' Given that Arnow and colleagues delineated trial characteristics by funder (of note, in their study they use 'sponsor' and 'funder' interchangeably), we assessed funder only as it related to time trends, results reporting, trial discontinuation, and intervention and disorder categories. When discussing the top fifty sponsors of mental health trials, we used the term 'sponsor' as defined by Clini-calTrials.gov: "Sponsor [is] the organization or person who initiates the study and who has authority and control over the study" [23]. For this reason 'sponsor' is used for

Time trend analysis
To facilitate our exploratory analysis of changes in trial characteristics over time, we separated trials into two time periods, as described above, and assessed for significant differences between periods. To summarize year-to-year changes in trial counts, we calculated average annual growth rates (AAGR) and compound annual growth rates (CAGR). The AAGR is calculated by taking the arithmetic mean of the percent change in a variable each year. While an accurate representation of the yearly growth of a variable, arithmetic means can be easily skewed by outlier values (i.e. dramatic fluctuations in growth). The CAGR is calculated by taking the final variable size compared to the initial variable size and approximating the average theoretical annual growth that would have been needed to reach the final size. Consequently, CAGR lessens the impact of large fluctuations in approximated growth rate. By comparing AAGR and CAGR, one can understand more thoroughly how a variable changed over time.

Early discontinuation and results reporting
We assessed the early discontinuation of mental health trials in our sample. See 'Study status' under Trial Characteristics for how early discontinuation was determined from ClinicalTrial. gov study status categories. A total of 5,818 trials (92.3% of total) were included in this analysis, excluding trials that were withdrawn before initiation or if discontinuation status was unknown.
We examined results reporting to understand dissemination of trial results into the database; however, only a subset of trials are required by the FDAAA to report results into the database. We examined results reporting by 36 months after the trial primary completion date to coincide with FDAAA provisions that relevant trials report results within 12 months, with opportunities for an additional 24-month extension [24,25]. Therefore, we restricted our reporting analysis to mental health trials that reached primary completion by April 30, 2015 (n = 2,223) to ensure a full 36-month reporting window. Because many trials report results outside of this window, we also separately analyzed results reporting without restrictions on the date.

Statistical analysis
We summarized trial data using descriptive statistics. Because ClinicalTrials.gov does not mandate that certain optional fields be completed, approximately 5% of trials had missing dimensions, and, therefore, the total number of trials varies slightly between dimensions. The total number of trials reporting each trial characteristic is labeled in Tables 2, 3 and 4. Because only a small percentage of trials had missing data, these trials were excluded from the logistic and cox regressions. We assessed the statistical significance of monotonic trends over time (i.e. annual growth rates and compound annual growth rates) using post-hoc Mann-Kendall tests to test the null hypothesis that the number of trials did not change over time. We assessed independence between groups over ordinal time using the Cochran-Armitage test. All year-toyear analyses included only years with a full 12-month collection of data (2008-2017). We assessed for differences between the distributions of categorical variables using a two-sided Pearson χ2 test. All analyses were two-sided. We performed time-to-event analysis of early trial stoppage with early discontinuation as events. We censored trials that reached completion without early stoppage or that remained ongoing at the cutoff for analysis (April 30, 2018). We visualized the relationship between trial duration and early discontinuation using Kaplan-Meier curves. We also performed Cox proportional hazards regression and provide individual and adjusted hazard ratios (aHR) for each trial characteristic (e.g. primary objective, trial phase, blinding, etc.). For trials that reached primary completion by April 30, 2015, we analyzed trial characteristics associated with results reporting to the registry using odds ratios (OR) and adjusted ORs (aOR) from univariate and multivariate logistic regressions, respectively.
We chose α = 0.001 as the level at which effect sizes represent meaningful trends and differences between groups given the large dataset and the risk of multiple hypothesis testing. We did not adjust for multiple comparisons in this exploratory analysis, and we expect approximately one out of every thousand tests to produce a significant result due to chance. All analyses were performed using the R statistical programming language, version 3.5.0 [26].

Trial characteristics over time
We identified 6,302 United States mental health trials, which comprised 56.4% (6,302/11,168) of all mental health trials and 10.2% (6,302/61,533) of all US interventional trials in the registry from October 1, 2007 to April 30, 2018 (Fig 1). From 2008 to 2017, the annual number of mental health trials increased from 625 to 757 (CAGR 2.2%, Mann-Kendall p = 0.012) (Fig 2A). This growth was roughly equivalent to that seen in non-mental health interventional trials in the United States during that period (CAGR 2.0%, Mann-Kendall p = 0.032) ( Table 1). Industry-funded mental health trials (184 trials to 90; 29.4% of total to 11.9%; CAGR -9.6%, Mann-Kendall p = 0.012) and US government-funded mental health trials (280 trials to 236; 44.8% of total to 31.2%; CAGR -3.9%, Mann-Kendall p = 0.0013) decreased significantly. Trials funded by academic medical centers, hospitals, and other sources grew dramatically (161 trials to 431; 25.8% of total to 56.9%; CAGR 9.2%, Mann-Kendall p = 0.00035) and drove the overall growth of the field (Fig 2B). Similar trends occurred across all non-mental health interventional trials in the registry, though the decrease in industry and US government funders and the growth of academic medical center, hospital, and other funders were less pronounced (Ind CAGR -4.7%, Mann-Kendall p = 0.0013; US Govt -2.3%%, Mann-Kendall p = 0.0042; AMC/Hosp/Oth CAGR 6.4%, Mann-Kendall p = 0.00017).

Mental health trial characteristics compared to non-mental health trials
United States mental health (MH) and non-mental health (NMH) trials in the registry differed in most characteristics (

Mental health trial sponsorship
The fifty organizations most commonly reported as the sponsor for trials accounted for 53.2% (3,355/6,302) of all US mental health interventional trials in our sample (Table 5). Forty-two (84%) were academic institutions or hospitals, four were pharmaceutical companies, and four were US governmental agencies (though two were sponsor names under the larger umbrella of the United States Veterans Association). When compared by time period (i.e. early [2007][2008][2009][2010][2011][2012] and late [2013][2014][2015][2016][2017][2018]), the number of trials sponsored by three of the top industry sponsors and three of the top US government agencies decreased, whereas the number of trials sponsored by thirty-four of the forty-two top academic medical center/hospital/other funders (81.0%) increased.

Mental health trial discontinuation
A total of 529 trials (8.4% of US mental health trials in our sample), representing an actual enrollment of 18,226 participants, were discontinued. Of the discontinued trials, 331 were terminated after enrollment began, 31 were suspended, and 167 were withdrawn before participant recruitment. Industry funders had the largest proportion of discontinued trials (Ind 11.6%, AMC/Hosp/Oth 10.3%, US Govt 4.5%; p<0.0001). Multivariate regression analysis revealed that intervention studied, funder type, and oversight by a DMC were all related to trial discontinuation (  35; p<0.0001) were more likely than trials funded by US government agencies to be discontinued. Trials with oversight from data monitoring committees (DMC) were less likely to be discontinued compared to trials without DMCs (aHR 0.64, 95% CI 0.53-0.78; p<0.0001). Kaplan-Meier curves (Fig 4) show cumulative incidence of trial discontinuation

PLOS ONE
Trends in mental health clinical research over the past decade within the first five years of the trial start date, stratified by funder. Throughout this period, industry-funded trials had the highest rates of discontinuation, with US government-funded trials demonstrating the lowest rate of discontinuation.

Results reporting of mental health trials to ClinicalTrials.gov
Of the 2,197 trials completed by April 30, 2015, 930 (42.3%) reported results to the registry, and only 644 (29.3%) reported results to the registry within 36 months of completion (i.e. by April 30, 2018), which was the maximum time allowed for certain trials by the FDAAA. In multivariate regression analysis, intervention, trial phase, funder, and disorder category were all associated with results reporting (Table 7). Trials studying pharmaceutical or Other interventions were more likely to report results than trials studying behavioral interventions (Pharm adjusted odds ratio [aOR] 7.47, 95% CI 5.05-11.03; Oth aOR 3.16 95% CI 2.12-4.72; both p<0.0001). Phase 1 trials were less likely to report results than other phase trials or trials in which phase designation was not applicable (Phase 1 aOR 0.25, 95% CI 0.16-0.39, p<0.0001). Trials funded by industry or academic medical center/hospital/other funders were less likely to report results than US government funders (Ind aOR 0.70, 95% CI 0.50-0.97, p = 0.034; AMC/Hosp/Oth aOR 0.73, 95% CI 0.56-0.96, p = 0.022). Many disorder categories were more likely to report results than the least studied disorders, which included disruptive, sexual, personality, somatic, movement, dissociative, gender, and paraphilic disorders. Disorder categories with the most significant results reporting included substance use (aOR

Discussion
This study aims to provide clarity to the landscape of contemporary US mental health clinical trials registered in ClinicalTrials.gov. Our analysis helps us to better understand how mental health trial features, including funders, trial design, and disorders and interventions studied, have changed over time, and how many of these trial features differ from non-mental health trials in the registry. This study also provides insight into trial characteristics that may influence or are at least correlated with trial discontinuation and results reporting to the registry. Mental health trials made up 10.2% of the United States interventional trials registered in ClinicalTrials.gov from October 1, 2007 to April 30, 2018. Industry and US governmentfunded trials demonstrated a significant annual decline (-9.6% and -3.9%, respectively). This  decline was counterbalanced by the 9.2% annual growth of trials funded by academic, hospital, and other funders. The decline in US government and industry funders in clinical research is in accordance with the literature [27,28]. Our data suggest that the decline in US government-funded trials is not unique to psychiatry (Mental Health CAGR -3.9% vs Non-Mental Health -2.3%), which is consistent with reports of a 27% reduction in the number of trials funded by the National Institutes of Health across all medical specialties from 2006 to 2014 [29], as well as a 32% decrease in funding ($110 million in 2011 to $75 million in 2014) for the National Institute of Mental Health [28]. Despite these larger trends in depreciating US government funding, the US Department of Veterans Affairs (VA) has significantly expanded its priority to fund psychological research and has nearly tripled the number of mental health professionals it employed since 2006 [30]. We found VA-funded trials increased overall by 17% between the early and late time periods of our study (118 trials [2007-2012] to 138 trials [2013][2014][2015][2016][2017][2018]), and it was the largest funder of mental health trials in the late period. Of note, VA trial sponsors were listed as either 'US Department of Veterans Affairs' or 'VA Office of Research and Development' in ClinicalTrials.gov during the early period, though the latter sponsor name was used almost exclusively in the late period. We tallied the total number of trials under both names to assess this trend.
The causes of the decrease in industry-funded mental health trials are likely multifactorial and include the increasing cost of developing new drugs, greater time required to bring each drug to market, reduced market exclusivity for new medications, and lower demand for branded drugs by increasingly cost-conscious payers [18,27]. Multiple international pharmaceutical companies have significantly decreased their investments in new treatments for depression, bipolar disorder, and schizophrenia, and some companies, such as GlaxoSmithKline, have closed their psychiatric units altogether [18,19]. Our data show that the number of new industry-funded clinical trials in mental health decreased two-times faster than industryfunded non-mental health trials in the registry (Mental Health CAGR -9.6 vs Non-Mental Health CAGR -4.7%). This suggests that, while industry funders are decreasing across all areas of medicine, industry may be specifically repositioning away from mental health research, or it may be devoting more resources to mental health research outside of the United States. While some have suggested that the reduction in industry-funded mental health research is the result of companies partnering with external collaborators, our analysis, which captures such collaborations as industry-funded, suggests otherwise [19]. While philanthropic support has accounted for less than one percent of funding for mental health research historically [31], shrinking funding from US governmental agencies and industry has pushed mental health researchers to pursue charitable giving as an alternative funding source [32]. This funding realignment has required new research strategies within academic institutions and hospitals, which now account for 19 of the top 20 sponsors of mental health trials. It is important to note, however, that 'funder' is a self-reported category within Clinical-Trials.gov, and there may have been trials that did receive funding from Industry or US government collaborators but did not report these collaborations. Further analysis of the sources of funding for academic medical center/hospital/other trials could potentially be assessed through use of the Secondary ID Numbers provided in ClinicalTrials.gov, which include grant and other funding information in a free-text field. Additional information is also available in the Protocol Registration ClinicalTrials.gov Data Element Definitions. While these approaches were beyond the scope of this study, they are promising areas of further research to clarify this issue of funding sources for trials other than industry or US government agencies.
Our analysis shows that the disorders and interventions studied by trials differed significantly by funder. As might be expected, the largest proportion of industry trials (76.6%) studied pharmaceutical interventions. Industry-funded trials preferentially studied depression, neurodevelopmental, schizophrenia spectrum, and sleep disorders, all disorders with classes of pharmaceuticals that are mainstays of care for these disorders (i.e. antidepressants, stimulants, antipsychotics, and hypnotics). The US governmental agencies preferentially studied substance use and trauma disorders. This is in keeping with the growth of US Department of Veterans Affairs (VA) funding, as these disorders are prevalent among veterans [33]. Academic/hospitals and US government agencies funded the majority of trials that studied conditions not clearly defined by DSM-5 diagnostic criteria (i.e. Non-DSM conditions), and trials studying Non-DSM conditions showed the largest growth of any disorder category from 2007 to 2018. This may reflect the efforts of the National Institute of Mental Health and other funding bodies to adopt the Research Domain Criteria (RDoC) in an effort to move away from studying DSM diagnoses and towards studying brain systems that often cross traditional diagnostic boundaries [34]. Both US government and academic medical center/hospital/other funders studied a large proportion of behavioral interventions (US Govt 50.2%, AMC/Hosp/Oth 43.6%, and 97.1% of all behavioral trials), as well as trials that compared behavioral interventions to pharmaceuticals or other interventions, such as transracial magnetic stimulation. This is consistent with the growing appreciation within mental health that there are often synergistic effects of psychotherapy and psychopharmacology [35].
Our data demonstrate that trial design features of registered US mental health trials changed overtime in multiple key respects. Mental health trials have increasingly been single or non-blinded and are not monitored by data monitoring committees (DMCs). While at face value these changes seem disappointing, they may rather reflect the growth of behavioral intervention trials, many of which cannot be blinded and do not required DMC oversight. US regulations only require DMCs for trials testing new drugs, biologics, or devices, in double blinded studies where there is considerable risk to patients, or when research is conducted in vulnerable populations (e.g. prisoners) [36].
There are likely many reasons why US mental health (MH) trials differed from non-mental health (NMH) trials, and the heterogeneity of the non-mental health category limits its interpretability. However, it is striking that non-mental health trials were more than twice as likely to be funded by industry (NMH 39.7% vs MH 17.6%), and non-mental health trials were less likely to be blinded (NMH 36.9% vs MH 58.5%), randomized (NMH 58.3% vs MH 79.9%), or monitored by a DMC (NMH 41.3% vs MH 44.6%). In 2018, none of the top ten highest grossing pharmaceutical products were for mental health indications, which may reflect why there is such disparity in industry sponsorship between mental health and nonmental health trials [37]. There are many instances, such as in surgical trials, where blinding and randomization may be infeasible or unethical, which likely accounts for the lower proportion of non-mental health trials using these design features [38]. This is also consistent with a prior comparison of mental health trials to oncology and cardiovascular trials [12]. Perhaps most salient to the comparison between mental health and non-mental health trials, it is reassuring that over our study period (comparing 2007-2012 to [2013][2014][2015][2016][2017][2018], the change in percentage of mental health trials reaching completion and using trial design measures to limit bias has not lagged behind non-mental health studies. Although the percentage of mental health trials utilizing double blinding and DMCs has decreased over time, this trend has also occurred in non-mental health trials (Double blinding: Early period 23.1%, Late period 20.1%, p<0.0001; DMCs: Early period 42.7%, Late period 40.2%, p<0.0001).
Even though fewer US mental health trials were discontinued than their non-mental health counterparts, one in twelve registered mental health clinical trials was stopped early over the 127-month period analyzed. Industry funders had the greatest proportion of discontinued trials, and 18,226 participants were enrolled in eventually discontinued trials. There are many justifiable reasons for trial termination, particularly in pilot studies (comprising 17.4% of discontinued trials in our sample), including poor patient accrual and lack of intervention efficacy, though commercial considerations remain controversial [39]. Industry may have less tolerance for risk and a shorter view of return on investment compared to the US government or academic medical center/hospital/other funders [19]. The increasing competitiveness of US government-allotted funding may also select for more rigorously designed and feasible trials than those funded by industry [28]. Together, this suggests that industry funders may have fewer restrictions for initiating trials but also a lower bar for stopping a trial early based on initial findings.
We found that 57.7% of completed trials did not report results to the registry, and only 29.3% reported results within 36 months of completion. This is consistent with prior studies of insufficient registration and results reporting in mental health clinical research [40]. The FDAAA and Final Rule do not mandate that all trials report their results to a registry, which is likely why so few have [6]. Moreover, the Final Rule, which expanded the proportion of trials mandated to report results to a registry, only took effect on January 18, 2017, which is outside the window of the trials studied in our sample. Results reporting has likely increased for trials registered after this date. Early phase trials were significantly less likely to report results, which is consistent with previous analyses using the ClinicalTrial.gov registry [24]. Funding was associated with results reporting, with industry and academic medical center/hospital/other funders less likely to report results to the registry than US government agencies. US government funders may have devoted administrative and research support to comply with reporting, and it is possible that a greater percentage of their trials meet the requirement for mandated reporting by the FDAAA and Final Rule. Intervention type was also associated with results reporting, with trials studying behavioral interventions the least likely to report results to the registry. This is likely because the FDAAA only mandated registration and results reporting for certain trials studying pharmaceuticals or devices, and so there is less incentive for reporting for behavioral trials. It is unfortunate that so few trials report their results to the registry. Dissemination of research findings is crucial for informing clinical practice, and selective reporting can lead to distortion of the field's knowledge base. Completed trials with inaccessible data represent a poor return for finite research resources, as well as a potential failure to meet the legal and ethical obligation that investigators have to trial participants, including as a component of informed consent.
Our study has multiple limitations. First, while ClinicalTrials.gov is one of the largest international trial registries and contains 70% of trials registered in the International Clinical Trials Registry of the World Health Organization, it is not an exhaustive list of all US clinical trials [14]. Second, not all trials, such as phase 1 trials or trials studying non-pharmacologic interventions, were subject to the FDAAA or the Final Rule requirements [6]. There may be other incentives and norms that bias the registration of trials with certain characteristics, and trends identified in the registry may at least in part reflect changes in trial reporting instead of changes in how trials were conducted or designed. Nevertheless, ClinicalTrials.gov is a unique resource that has allowed many medical specialties to assess features and trends in their clinical research, which might otherwise remain unassessable [12][13][14]16]. Third, while our team of psychiatrists made significant efforts to manually review all key words, titles, and study descriptions of trials included in this study to confirm their relevance to mental health, there may have been trials excluded from our analysis due to missing or mislabeled keywords in the registry. We consciously excluded some trials within the neurocognitive disorder category, such as Alzheimer's disease and traumatic brain injury, because we found they overlapped significantly with the neurology literature. However, to our knowledge, we included all other available search terms for the disorders defined by Section II Diagnostic Criteria and Codes in the DSM-5. While there are limitations to a categorical versus a dimensional diagnostic system, as is evidenced by the growing number of trials in the registry that study conditions that do not fit a DSM-5 diagnostic category, the DSM-5 currently provides the most universally used schema by which mental health disorders are organized [41]. Finally, our study chose to look exclusively at US trials registered in ClinicalTrials.gov. Heterogeneity in international legislation and incentives for trial registration that differ by country were thought to likely confound the interpretation of our results if they were included in the sample. Consequently, 43.6% of the mental health trials registered in ClinicalTrials.gov from October 1, 2007 to April 30, 2018 were not analyzed in this study, and our results cannot be generalized beyond United States mental health trials. Future analysis will be needed to see how international mental health trials may differ by region and compare to US mental health trials in the registry. There also remains the need to address other important unanswered questions about mental health trials, such as the extent to which publication bias occurs. Others have been able to address this issue using the ClnicalTrials.gov database [42][43][44].
In summary, this study of the entire portfolio of US mental health trials registered in Clini-calTrials.gov provides clarity to many questions left unanswered by prior analysis of these data [16]. While overall US mental health trials grew at a similar rate to non-psychiatric US trials in the registry, there were significant differences in changes in funders, with more dramatic decreases in industry-and US government-and increase in academic medical center/hospital/ other-funders in mental health trials. Features of trial design that provide safeguards against bias, such as blinding and monitoring by a DMC, decreased over time in registered mental health trials, though this may be due to dramatic growth of trials studying behavioral or Other interventions, which often do not lend themselves to blinding or require DMC oversight. There was also a concomitant decline in the registry of mental health trials studying pharmacotherapies. Shifts have occurred in the mental health disorders studied in the registry, with a notable increase in studies of Non-DSM conditions, which may reflect the adoption of the RDoC initiative put forth by NIMH. Despite ethical obligations and policy incentives, trial discontinuation and lack of results reporting remain issues within mental health research. Altogether, we hope our findings foster discussions and collaborations among mental health providers, funding bodies, and other concerned parties to promote the continued development of diverse, well designed, innovative clinical research to improve the care of our patients suffering from mental health disorders.
Supporting information S1