The use of ecological momentary assessment (EMA; also known as ambulatory assessment and the experience sampling method) in psychological research has greatly increased in the last decade. This no doubt has to do with advancing availability, and declining costs, of mobile technology, including smartphones and wearable devices with biometric sensors. A recent survey of U.S. adults found that roughly 80% own a smartphone (Pew Research Center 2019), and there have been rapid developments in techniques for analyzing the intensive longitudinal data that can be collected from mobile devices (Piccirillo and Rodebaugh 2019; Wright and Woods 2020). Several scholars have suggested that the use of mobile technology will enhance basic psychological research (Miller 2012), knowledge of the mechanisms of psychopathology (Fisher et al. 2017; Trull and Ebner-Priemer 2013), and clinical psychological practice (Brown et al. 2019; Zimmermann et al. 2019), signaling a high degree of enthusiasm for EMA among researchers.

Potential Clinical Benefits of EMA

There are several potential benefits of using EMA in routine clinical practice. At the most basic level, real-time ecological assessment can mitigate the inaccuracy and biases that attend retrospective recall. For example, several studies suggest that individuals have a bias towards recalling their experiences, especially their negative experiences, as more intense and persistent than they actually were (Ben-Zeev et al. 2012, 2009; Ellison et al. 2020; Kelly et al. 2019; Urban et al. 2018; Wenze et al. 2012), and they overestimate the links between their symptoms and stressful circumstances (Ellison et al. 2020; Gloster et al. 2017, 2008). These inaccuracies and biases threaten the accuracy and efficiency of diagnoses and case formulations based on clients’ retrospective accounts. Because EMA produces in-the-moment, timestamped data, these biases and inaccuracies can be reduced.

The ability of smartphones and other mobile sensors to timestamp data can also allow for more advanced inferences about the typical timing and patterning of therapy clients’ experiences. For example, differences in the periodic pattern of affect rhythms in individuals with depression are clinically important (Gaspar-Barba et al. 2009), and EMA is well-suited to capturing these patterns with precision (Miller et al. 2015). More generally, it may be useful to identify these cyclical patterns to gain insight into periodic psychosocial and biological triggers (or consequences) of symptoms for an individual therapy client. There is also evidence that the level of rigidity in periodic fluctuations in symptoms can predict therapy outcome (Fisher and Newman 2016). Other dynamic patterns may also illuminate personality processes and interpersonal dysfunction that are not be captured by traditional assessments (Lewis et al. in press). Thus, EMA can provide accurate information about the timing and general fluctuations of therapy clients’ experiences, which may in turn help therapists achieve more accurate case formulations and better treatment plans.

One of the most often-cited potential advantages to EMA in clinical practice is the ability to construct idiographic models of the covariance of different symptoms, which could then be used to prescribe person-specific interventions (David et al. 2018; Epskamp et al. 2018; Fisher 2015; Mumma et al. 2018). This application of EMA is called “precision diagnosis” (van Os et al. 2013), “precision assessment” (Roche and Pincus 2016), or “personalized network modeling” (Epskamp et al. 2018) and utilizes the multivariate time-series data that results from repeated ratings of multiple variables. By observing the contemporaneous and lagged connections among different experiences, clinicians might be able to target more causally relevant symptoms and thus intervene more effectively. Preliminary reports suggest some benefit from these personalized interventions (Fisher and Boswell 2016; Fisher et al. 2019; Hopwood et al. 2016; Rubel et al. 2018; Zimmermann et al. 2019).

More specialized, and data-intensive, potential applications of EMA in clinical practice include the detection of “early warning signs” of changes in clinical status. Dynamic systems theory (Hayes et al. 2015; Nelson et al. 2017) suggests that transitions in complex systems, including psychological systems, can be forecast by increases in the variance and “inertia” (autocorrelation) in the elements of the system. Therefore, if an individual’s symptoms are monitored longitudinally, clinically important transitions (such as a relapse into substance abuse, the onset of a depressive or manic episode, or “sudden gains” in psychotherapy) could potentially be predicted in advance by monitoring these statistics (van de Leemput et al. 2014). A case study by Wichers et al. (2016), for instance, showed that these early warning signs preceded the onset of a depressive episode for a 57-year-old man whose antidepressant medication was discontinued in a double-blind fashion. This general finding was replicated in a second individual (Wichers et al. 2020). Moreover, a more recent study has shown a general utility of these early warning signs in 328 individuals with mood disorders (Olthof et al. 2020a; b). Thus, using EMA as a monitoring tool in ongoing therapy could potentially provide advance warning of imminent changes in an individual’s clinical status, which could be very helpful in treatment planning and risk management.

The enthusiasm on the part of researchers for the potential applications of EMA to clinical practice, therefore, is understandable. Not only can EMA provide a more ecologically valid way of assessing the everyday experiences of individuals in therapy, it also has advantages over self-report or interview-based measures: (1) EMA reduces or eliminates the need for retrospective recall; (2) the time-series data resulting from EMA can be used to summarize periodic trends in symptoms; (3) multivariate time series data can aid in the creation of idiographic (rather than nomothetic) models of a person’s pathological processes, which may aid in personalized case formulation and selection of interventions; and (4) statistics from EMA data may be able to forecast important shifts in a therapy client’s problems before they occur. However, despite researchers’ enthusiasm for the potential of these EMA applications to aid in routine therapy practice, it is far from certain that these promising technologies will be readily taken up by clinicians, even if the evidence for their utility continues to accumulate.

One reason for caution is that clinicians rarely use the existing diagnostic, case-formulation, and outcome-monitoring tools that have been developed and refined empirically (“evidence-based assessment”; Hunsley and Mash 2009). For example, therapists rarely use semi-structured interviews to diagnose their clients (First et al. 2018) or outcome-monitoring questionnaires to track their progress in therapy (Jensen-Doss et al. 2018). Chief among the reasons for this, according to surveys, are many clinicians’ beliefs that standardized assessment measures are not practical or useful (Hatfield and Ogles 2004, 2007; Gilbody et al. 2002; Jensen-Doss and Hawley 2010), present undue burdens to clients (Bruchmüller et al. 2011), and do not offer benefits over clinical judgment alone (Jensen-Doss and Hawley 2010; Gilbody et al. 2002).

It is certainly possible that clinicians will view EMA applications in a more favorable light than they do these traditional assessment methods. For example, whereas semi-structured interviews assume a nomothetic view of psychopathology, resulting in diagnoses that do not fit many individuals’ problems (Haynes et al. 2009; Molenaar and Campbell 2009), EMA can easily be used to create idiographic models of disorder (Brown et al. 2019; Piccirillo et al. 2019). For this reason, therapists may more readily accept these person-specific models as useful for the individuals they work with and a valuable supplement to clinical judgment. Moreover, the hardware necessary for EMA is increasingly widespread, and EMA software is increasingly customizable and affordable, which may minimize concerns about practicality and burden. On the other hand, it is also possible that EMA applications will join these other standardized assessments on the list of tools that are used enthusiastically by clinical scientists but not by the vast majority of psychotherapists in everyday practice. Knowing more about the readiness of clinicians to adopt EMA-based tools in their everyday practice would aid efforts to disseminate this new technology.

A few recent studies have investigated clinician attitudes toward EMA as an assessment tool in routine psychotherapeutic practice, with mixed results. Zimmermann et al. (2019) provided a specialized, smartphone-based diary application to 35 therapy outpatients in Germany, who used it for at least 14 days. The application was designed to measure adaptive and maladaptive personality trait expressions as well as situational contexts, producing an individualized picture that could be used in treatment planning. After the assessment period, the researchers then surveyed the patients and their therapists about the clinical utility and acceptability of the application. Results suggested that practitioners were skeptical of the benefits of this tool and concerned about the potential burden to clients. (Interestingly, the clients themselves found it easy and not very burdensome to complete the smartphone surveys.) Similarly, Frumkin et al. (2020) assessed symptom-states in 12 outpatients for three weeks using EMA and presented personalized network models to the clients and their therapists. Clients were generally more optimistic than their therapists about the utility of these models. Finally, Bos et al. (2019) conducted interviews and focus groups with 22 outpatients and 21 clinicians (psychologists, psychiatrists, psychiatric nurses, and job coaches) regarding the potential usefulness and consequences of EMA in clinical care. The authors concluded that clinicians were mostly positive on the potential applications of EMA, although they also described some adverse effects and limitations. However, because the interviews and focus groups were conducted by two of the authors, both of whom are known as EMA researchers (and who were known personally to at least some of the participants), it is possible that responses were more optimistic than they might have otherwise been.

A limitation with all three of these studies, however, is that they do not compare therapists’ attitudes about EMA to their attitudes towards other assessment techniques (such as traditional diagnostic assessments) or other resources that they might access to help resolve clinical difficulties as a supplement to usual care. Likewise, these studies did not assess clinicians’ general attitudes towards, or use of, traditional standardized assessments for psychotherapy. Thus, it is unclear whether the clinicians’ views on the usefulness of EMA for therapy are more, or less, positive than their views on the usefulness of other assessment tools. Because the use of standardized assessment techniques by therapists is relatively rare (First et al. 2018; Hatfield and Ogles 2004, 2007; Gilbody et al. 2002; Jensen-Doss and Hawley 2010; Jensen-Doss et al. 2018), knowing more about clinicians’ attitudes and perceptions of EMA relative to their attitudes toward traditional assessment could facilitate the dissemination of EMA-related innovations by researchers and help foster productive partnerships between researchers and clinicians. In particular, it would greatly help to know whether practicing clinicians find the potential applications of EMA more or less appealing than traditional assessment, case-formulation, and outcome-monitoring tools that they might adopt to handle particularly tricky therapy cases; which EMA applications clinicians consider most helpful; and what attitudes and professional variables predict the perceived utility of EMA for psychotherapy.

Current Study

One aim of the current study was to investigate the attitudes of clinicians towards smartphone-based tools for the initial assessment of psychotherapy clients and the ongoing monitoring of therapy progress. More specifically, the current study examines clinicians’ perceptions of the usefulness of EMA for clients who present diagnostic or case-formulation challenges and are not well served by their usual practices. A second aim was to compare these attitudes to clinicians’ attitudes regarding traditional resources for diagnostic assessment, case formulation, and outcome monitoring for these tricky cases. A third aim was to examine the relation between clinicians’ attitudes towards smartphone-based assessment tools and their use of, and general attitudes towards, standardized assessment. A survey of clinical psychologists who practiced psychotherapy with adult clients was conducted to investigate these questions.

Methods

Procedure and Participants

Email appeals were used to recruit survey respondents from June 2017 to December 2019. At the beginning of this period, survey invitations were sent to the listservs of two divisions of the American Psychological Association: Division 12 (the Society of Clinical Psychology) and Division 29 (the Society for the Advancement of Psychotherapy). Then, email appeals were sent in several bursts to the rosters of licensed psychologists in six states: Arkansas, Florida, Nebraska, Nevada, New Jersey, and Rhode Island. These states were selected because their licensure boards keep publicly-available lists of psychology licensees that contain email addresses. Emailed invitations to these licensee lists were sent via Qualtrics survey software, which personalized the salutation of each email to address the recipient by surname.

Emails invited recipients to participate in a “survey study about practices and attitudes regarding diagnosis and assessment.” It also stated that eligible participants had to have completed a terminal professional degree and had to be engaged in at least 5 h of psychotherapy in the United States with adult clients. Respondents who clicked on the included link and completed the survey were offered a 1 in 4 chance of winning a $10 gift card in exchange for participating. Two to three weeks after the first email appeal, non-responders were sent a reminder email.

Survey

Questions About Current Practices in Diagnosis and Outcome Monitoring

Respondents were first asked to characterize their current practices in diagnosis and outcome monitoring. These questions, which were created for the current study, asked whether respondents conducted their own intake interviews and what tools were used for intakes in their practice (unstructured clinical interviews, semi-structured diagnostic interviews, formal personality/intellectual assessment instruments, screening measures, and other methods). Respondents were also asked about their practices in outcome/progress monitoring and what tools they used for outcome monitoring (qualitative information, informal numerical ratings in progress notes, outcome monitoring questionnaires, and other methods). If respondents said that they “routinely” or “sometimes” used tools in these broad categories, skip logic directed them to questions about which specific instruments they used.

Attitudes Toward Standardized Assessment (ASA; Jensen-Doss and Hawley 2010)

Respondents then completed the ASA, a 22-item measure of clinicians’ attitudes toward standardized assessment scales in routine clinical practice. Although it was originally written to assess these attitudes in clinicians who work with children, in the current study stems were modified to refer to therapy with adult clients. Preliminary factor-analytic evidence (Jensen-Doss and Hawley 2010) suggests that the ASA contains 3 subscales. The benefit over clinical judgment subscale consists of items reflecting the belief that the use of standardized assessment improves upon clinical judgment (sample item: “standardized measures don’t tell me anything I can’t learn from just talking to my clients” [this item was reverse-scored]). The psychometric quality subscale reflects the attitude that scales with established reliability and validity are important for clinicians to use (sample item: “standardized measures help with accurate diagnosis”). Finally, the practicality subscale reflects the belief that standardized measures are efficient, readily available, and easy to use (sample item: “standardized measures take too long to administer and score” [reverse-scored]). All items were rated on a 5-point Likert scale from 1 (“strongly disagree”) to 5 (“strongly agree”). In the current study, the total ASA scale score was used; its internal consistency was α = 0.86.

Attitudes Toward Standardized Assessment-Monitoring and Feedback (ASA-MF; Jensen-Doss et al. 2018)

Participants then completed the ASA-MF, an 18-item measure of clinicians’ attitudes toward standardized outcome monitoring and feedback scales. Confirmatory factor analysis suggests the presence of three correlated factors for the ASA-MF (Jensen-Doss et al. 2018). The treatment planning subscale reflects attitudes regarding the utility of standardized outcome monitoring measures in planning changes treatment (e.g., “standardized progress measures help identify when treatment is not going well”). The practicality subscale measures attitudes about the ease of use and efficiency of routine outcome monitoring (sample item: “standardized progress measures can efficiently gather information”). Finally, the clinical utility subscale measures whether clinicians believe that routine outcome monitoring adds to clinical practice as usual (e.g., “standardized progress measures provide more useful information than other assessments like informal interviews or observations”). All items were rated on a 5-point Likert scale from 1 (“strongly disagree”) to 5 (“strongly agree”). The total scale score of the ASA-MF was used; in the current study, its internal consistency was α = 0.93.

Questions About Perceived Usefulness of Resources for a Difficult Therapy Case

Respondents were then asked to think about a recent therapy case that presented difficulties in diagnosis, case formulation, or outcome monitoring (for example, a client for whom the clinical picture was complex or confusing, or whose problems shifted to a great extent, making it difficult to plan or execute interventions effectively). They gave a brief description of what was difficult about this client’s problems and then rated how helpful they would find thirteen resources for resolving the difficulty. The resources included both existing instruments and EMA-based methods. Each resource was accompanied by a graphical depiction and a brief (30–40 word) description. These descriptions and graphics can be found in the Supplementary Material and at https://osf.io/jtmyf/.

Respondents rated the helpfulness of these resources on a 5-point Likert-type scale with anchors “Not at all helpful,” “Somewhat unhelpful,” “Neutral,” “Somewhat helpful,” and “Very helpful.” An additional response option could be selected if the respondent had already used the resource for the client in question; these responses were removed from the data before analysis began. One hundred and forty-two responses (out of 4746; 2.99%) from 86 separate respondents were removed for this reason.

EMA-based resources were as follows: a graphical display of the client’s timestamped symptom-states (i.e., a time plot); a report detailing cyclical/periodic trends in the client’s symptom-states; a report of the idiographic dynamics among the client’s symptom-states; and a report of “early warning signs” of symptomatic worsening or relapse. Non-EMA-based resources were: a report from a set of semi-structured diagnostic interviews; results from the DSM-5 cross-cutting symptom measure and a validated scale measuring well-being and daily functioning; a report from a comprehensive personality and intellectual assessment; attending a seminar or webinar related to the client’s problems; attending a workshop on newly developed assessment instruments relevant to the client’s problems; a report from an assessment of cultural features of the client’s problem and how it relates to the client’s social and cultural context; extra time each week to consult with colleagues or supervisors about the issue; and access to the client’s posts on social media (assuming the client’s permission). This last resource, the use of which is a matter of some debate among therapists (e.g., Fisher and Appelbaum 2017), was included as a way to distinguish respondents’ attitudes toward EMA applications per se from their general attitudes toward electronic media and how their use might impact psychotherapy.

Demographic and Professional Information

Finally, respondents provided basic demographic and professional information about themselves, including their highest degree, state in which their primary practice was located, setting of their practice (e.g., hospital, individual practice, clinic), number of years in practice, theoretical orientation, and whether they were involved in research activity in addition to clinical practice.

Statistical Analyses

In order to compare the helpfulness of EMA-based versus traditional resources for a tricky therapy case, ratings of the EMA-based resources (time plots of momentary symptom levels, idiographic network models, “early warning signs,” and periodic symptom trends) were averaged and compared to averaged ratings of traditional resources (consultation with a colleague, report from a semi-structured diagnostic interview, personality assessment report, cultural formulation, seminar/webinar, workshop, cross-cutting symptom measure report, and an outcome monitoring report) using a paired samples t-test. In order to explore predictors of perceived helpfulness of EMA, these averaged helpfulness ratings were regressed onto attitudinal and professional variables (for continuous predictors) or used as the dependent variables in t-tests and analyses of variance (for categorical predictors).

Results

Appeals to listservs went to a maximum of 4,347 individuals (the number of recipients of the listservs in the month these appeals were sent, assuming no overlap in membership). Direct emails were sent through Qualtrics to 9355 unique email addresses. Of these, 450 failed or were returned as undeliverable. The author also received 191 automated replies stating that the email address was temporarily or permanently not being monitored by the intended recipient. Thus, 13,061 emails were generated. Duplicate respondents were screened by examining cases with identical demographic and professional data; no duplicate cases were identified. In all, 375 individuals completed the survey, with another 289 providing partial data (for instance, providing data about their diagnostic practices, which was presented first, but not about their demographic information, which was presented last).

The response rate is difficult to determine with accuracy due to several factors: overlap between listserv recipient lists and between those contacted by listserv and those contacted by direct email; the number of appeals routed to junk-mail folders or sent to non-monitored email accounts; and the number of respondents who received the email but who were not eligible to complete the survey (for example, because they were retired, did not conduct psychotherapy, or practiced only with children, families, or adolescents). If none of these factors affected the response rate, an estimated 2.9% of recipients would have completed the survey (375 respondents from 13,061 appeals), with another 2.2% of recipients providing partial data. Given the above limitations on the denominator of this estimate, however, these estimates should be seen as a lower bound on the response rate.

Table 1 summarizes the demographic and professional characteristics of the respondents. They were, on average, 51.9 years of age (SD = 12.98) and had been in practice an average of 24 years, including their years in training (SD = 14.78). Respondents conducted an average of 20.51 h of face-to-face therapy per week (SD = 11.08). Seventy-nine respondents of 363 (21.8%) reported involvement in research for the purpose of quality control or service improvement in their practice, and 99 respondents of 357 (27.7%) reported involvement in research with the aim of publishing or presenting findings for a broader audience.

Table 1 Respondent characteristics
Full size table

Diagnostic and Outcome-Monitoring Practices and Attitudes

Most respondents who reported information about their intake practices reported that they conducted their own intake assessments (n = 568 of 642, 88.5%). Only one-fifth who reported on the specific instrument used for diagnosis reported using a semi-structured or structured diagnostic interview routinely (n = 118 of 583, 20.2%), whereas the majority (n = 487 of 596, 81.7%) reported routine use of unstructured clinical interviews to diagnose their clients. It was relatively more common for respondents to report the routine use of standardized screening measures (n = 185 of 589 respondents, 31.4%; depression, anxiety, and substance use were the most common types of pathology screened for) or disorder-specific measures (n = 182 of 593, 30.7%) for the initial assessment. The most common method endorsed for monitoring the outcome or progress of therapy was qualitative information from therapy notes (n = 411 of 508, 80.9%). A minority reported the routine use of a broadband outcome monitoring questionnaire (n = 41 of 504, 8.1%), but relatively more routinely used numerical information recorded in therapy notes to keep track of therapy progress, such as a GAF score (n = 130 of 506, 25.7%). Other methods reported included diary cards (n = 48 of 504, 9.5%) or other symptom questionnaire measures (n = 99 of 502, 19.7%).

Overall, respondents showed a slightly more favorable attitude towards standardized assessment and towards routine outcome monitoring than reported in the original validation samples for the ASA (Jensen-Doss and Hawley 2010) and ASA-MF (Jensen-Doss et al. 2018). Respondents’ scores were slightly above the scale midpoint for the ASA (M = 3.49, SD = 0.47) and ASA-MF (M = 3.28, SD = 0.61), as well as for the clinical benefit (M = 3.15, SD = 0.72), psychometric quality (M = 3.86, SD = 0.46), and practicality (M = 3.40, SD = 0.52) of standardized assessment subscales of the ASA. Attitudes were in a similar range for the clinical utility (M = 3.13, SD = 0.65), treatment planning (M = 3.47, SD = 0.62), and practicality (M = 3.34, SD = 0.77) of routine outcome monitoring, as measured by the ASA-MF.

Helpfulness of Different Resources

The average helpfulness rating of the “extra” resources for the difficult therapy case was 3.60 (SD = 0.69), or roughly halfway between “Neutral” and “Somewhat helpful.” EMA-based resources (M = 3.45, SD = 0.97) were seen as less helpful than traditional resources (M = 3.79, SD = 0.78, t(382) = 8.23, p < 0.001. In general, there was considerable diversity in how helpful respondents thought the different resources would be. Figure 1 shows the mean ratings of each resource, excluding ratings indicating that the respondent already used that resource for the case in question. Numerically, the EMA-based resources were rated as less helpful than every other resource, except for access to the client’s social media accounts (which was rated the least helpful by far; note that this resource was not included in either the EMA-based or traditional category).

Fig. 1
figure 1

Perceived Helpfulness of 13 Resources for Assessment, Case Formulation, and Outcome Monitoring. Note: Resource A = consultation with a colleague; B = semi-structured diagnostic interview report; C = personality assessment report; D = report of a cultural formulation; E = attending a seminar about this diagnosis; F = attending a workshop about this diagnosis; G = report from DSM-5 cross-cutting symptom measure; H = report from a standardized outcome monitoring measure; I = report on “early warning signs” of relapse; J = report on periodic trends in symptoms; K = graphical display of momentary symptom levels; L = personalized symptom network; M = access to client’s social media posts. Green (striped) bars are resources based on EMA data. The horizontal line represents the average helpfulness rating of 3.6. Error bars represent standard errors

Full size image

Predictors of Perceived Helpfulness of EMA

Respondents’ attitudes toward standardized assessment predicted their ratings of the helpfulness of EMA, b = 0.50, t(381) = 4.80, p < 0.001, r = 0.24, as did their attitudes toward standardized outcome monitoring tools, b = 0.57, t(378) = 7.49, p < 0.001, r = 0.36.Footnote 1 Several professional variables also predicted the perceived helpfulness of EMA. Respondents’ number of years in practice negatively predicted their ratings of EMA resources’ helpfulness, b  =− 0.02, t(362) = 4.22, p < 0.001, r = − 0.22. However, the number of therapy hours that respondents engaged in per week did not predict EMA helpfulness ratings, b = − 0.01, t(356) = 1.14, p = 0.26, r = − 0.06, nor did the type of terminal degree held by respondents, t(354) = 0.007, p = 0.99, d = 0.0007. Respondent theoretical orientation did relate to the perceived helpfulness of EMA; those reporting a primary or sole cognitive-behavioral (M = 3.53, SD = 0.97), humanistic (M = 3.47, SD = 0.99), psychodynamic (M = 3.14, SD = 1.00), or other/integrative (M = 3.66, SD = 0.77) orientation differed in how helpful they saw EMA, F(3,354) = 3.12, p = 0.03.

Finally, respondents who engaged in research for the purpose of improving their clinical services (N = 75, M = 3.72, SD = 0.89) showed more favorable attitudes toward EMA than those who did not conduct such research (N = 281, M = 3.40, SD = 0.98), t(354) = 2.58, p = 0.01, d = 0.34. However, the difference in perceived helpfulness between respondents who engaged in research for presentation or publication for scientific audiences (N = 96, M = 3.58, SD = 0.96) and those who did not do such research (N = 254, M = 3.44, SD = 0.97) was not statistically significant, t(348) = 1.15, p = 0.25, d = 0.14.

Discussion

Results of the current study suggested that practicing psychologists are less inclined to view EMA-based resources for diagnosis, assessment, and outcome monitoring as helpful for a tricky therapy case than other resources, including traditional assessment tools, training opportunities, and supervision or consultation with a colleague. The results of the current study have clear implications for the translation of new technologies and statistical techniques into actual clinical practice. In particular, the results clarify the status of new EMA applications with respect to the “science-practice gap” in psychotherapy and clinical assessment. Whereas it might be tempting for researchers to imagine that EMA will avoid some of the perceived shortcomings of existing standardized assessment tools (for example, the view among clinicians that they do not provide useful clinical information or that they force a one-size-fits-all perspective onto individualized case formulation), the current research suggests a more tempered view: the benefits of EMA may not be immediately apparent to practitioners. This possibility is further underscored by the fact that comparatively few respondents in the current study reported the use of traditional standardized assessments (e.g., well-validated semi-structured diagnostic interviews and outcome monitoring questionnaires), which were in turn rated above EMA-based resources in terms of potential utility for a difficult therapy case. In general, the results are broadly consistent with research suggesting that therapists have reservations about the benefits of EMA (Frumkin et al. 2020; Zimmermann et al. 2019).

To be sure, one possible reason for the relatively pessimistic view of the helpfulness of EMA-based tools is that therapists are accurately appraising the evidence base for them. Indeed, EMA and its potential applications to clinical practice are novel technologies, and empirical support for their utility in actual treatment settings has mostly consisted of case studies (e.g., Kroeze et al. 2017; Lewis et al. in press; Schiepek et al. 2016), secondary analyses of existing trials (Fisher and Newman 2016), and uncontrolled studies (e.g., Fisher et al. 2019; Frumkin et al. 2020; Zimmermann et al. 2019). Thus, rigorous, controlled investigations of EMA’s applications in clinical practice are absent so far. In contrast, the evidence for the clinical utility of diagnostic assessment (Zander et al. 2018; Zimmerman 2003), personality and neuropsychological assessment (Poston and Hanson 2010), and routine outcome monitoring (Lambert et al. 2018) is relatively robust and established. Therefore, it is conceivable that clinicians are reserving their positive opinions toward EMA until its applications acquire the quantity, and quality, of research evidence that traditional tools have.

However, correlation analyses showed that therapists with more favorable views of traditional evidence-based assessments were more likely, not less likely, to also see EMA as useful. Thus, at least on average, the low perceived utility of EMA-based resources may not have been driven by a critical take on the extant evidence for EMA’s applications to psychotherapy. Instead, clinicians’ skepticism may partake of similar skeptical attitudes towards standardized assessment and outcome monitoring and may have some of the same components. For example, clinicians who believe that traditional standardized assessment is too burdensome, or does not provide information that would not be accessible to clinical judgment alone, may hold some of the same beliefs about EMA. Future research (for example, with interview or focus-group data) may be helpful in identifying more precisely why clinicians are skeptical about the helpfulness of EMA for their practice.

The current study suggests several recommendations for researchers interested in translating EMA applications into clinical practice. First, on the most practical level, these results suggest which therapists might be most receptive to the use of EMA. Researchers might profitably target therapists who are already conducting research for the purpose of improving the services they provide. These “early adopters” of EMA (Rogers 2003) may then spur other clinicians to change their practices. Clinicians who have not been practicing for very long with relatively favorable views towards traditional, standardized assessment and outcome-monitoring tools may also be more receptive to EMA than other therapists. It may be particularly beneficial to target trainees, who may be more receptive to novel practices (Lushin et al. 2019). It is possible that therapists who have been in practice for a long time see EMA as less helpful because they are simply less familiar with the concept and with the terms relating to its use. This is consistent with the fact that the empirically-supported tools rated as most helpful were probably the most familiar: semi-structured diagnostic interviews and personality assessment instruments. In this respect, it is somewhat remarkable that EMA-based resources were seen as nearly as helpful for tricky therapy cases than these widely established, gold-standard methods. Thus, especially as newer generations of therapists begin to practice, EMA may gradually be received more favorably as a psychotherapeutic tool.

In general, the current study suggests that researchers should pay careful attention to the process of disseminating and implementing these new assessment technologies to maximize their benefit. Several general principles, delineated in the literature on dissemination and implementation (e.g., Gallo and Barlow 2012; Proctor et al. 2009), may be relevant here. For instance, clinicians may need incentives to overcome the initial investment of time and resources required to become acquainted with the new technologies involved in EMA (e.g., smartphone applications and statistics/models) and their potential benefits. In this regard, it would greatly help for researchers to develop automated systems for analyzing and interpreting time-series data in clinically relevant ways so that the results of EMA can be easily understood by clients and clinicians alike, and used in collaborative treatment planning, without the intervention of researchers or data analysts. Stressing the flexible, idiographic nature of EMA’s applications may also make them more appealing to therapists, who are naturally concerned with the wellbeing of individual clients and may appreciate the ability to tailor different research questions and different EMA methods to different individuals. A different strategy would be for researchers and clinicians to work together to design practice-oriented research protocols (e.g., Becker et al. 2009; Koerner and Castonguay 2015; Youn et al. 2019) using EMA that answer questions that clinicians care about. Soyster and Fisher (2019), for instance, provide an example of involving participants in the design of an EMA study. Similar efforts with therapist stakeholders might be very effective in integrating high-quality and clinically useful EMA into clinical practice.

The current study has some limitations that deserve mention. First, response rates to the survey were low. This may have been due, in part, to the use of e-mail to recruit respondents, which produces lower response rates than survey solicitations sent through postal mail (Sebo et al. 2017). The response rate may have been artificially deflated due to the fact that only some e-mail recipients were eligible to complete the survey. For example, the licensee rosters used for recruitment contained a number of individuals who did not practice therapy, such as researchers, administrators, and retirees, or who did not practice outpatient therapy with adults. Nonetheless, the individuals who had the time and inclination to respond may have been a selected sample and thus unrepresentative of the broader population of clinicians in some respects. Because neither the listservs nor the state licensing boards from which we recruited maintain public information about their membership, it is difficult to assess the impact this may have had. The sample is also potentially unrepresentative because the appeal was also sent to professional listservs, which probably led to an oversampling of engaged and scientifically inclined clinicians. Any unrepresentativeness, in turn, may have biased the overall picture of clinicians’ readiness to adopt EMA; it is possible that the descriptive results obtained here may not generalize to the population of psychologists as a whole. The current results should thus be taken as preliminary. A second limitation is that respondents were asked only about a hypothetical case, and offered only a brief description of the EMA-based resources and how they might help. It is thus possible that clinicians who are directly exposed to EMA, or at least to more detailed material about its potential uses, may be more enthusiastic about its use than the sample here. However, this possibility also underscores the need for researchers to make a concerted effort to provide training opportunities for clinicians who might be interested in making use of EMA in their practice. Finally, the current sample consisted almost entirely of psychologists with doctoral degrees. Thus, future research will be needed to investigate whether the current results generalize to therapists with other professional disciplines (e.g., counselors; clinical social workers) or therapists in other countries.

The current study clarifies attitudes toward the use of EMA in psychotherapy among a moderately large sample of psychologists in the United States. Although the perceived helpfulness of EMA-based resources was low in comparison with traditional assessments, consultation/supervision, and other resources, the current study also identified which clinicians might be more receptive to its use in psychotherapy. Assuming the evidence for the clinical applications of EMA continues to accumulate, researchers should not ignore the history of erratic and incomplete implementation of evidence-based practice in the United States. Active efforts to integrate novel technology into practice as usual, targeted dissemination efforts, and research-practice integration may help ensure that the public benefits from these promising new tools.