Introduction

Psychology-focused telehealth, sometimes referred to as telepsychology or telebehavioral health, is the integration of technology (e.g., video, telephone, email) with healthcare services. It became an essential means of ensuring continued patient care as American clinical psychologists rapidly responded to the coronavirus disease 2019 (COVID-19) safety orders by switching to online methods (Pierce et al., 2021; Reilly et al., 2020). Although in-office appointments have re-emerged, many have suggested ongoing use of telehealth (Mishkind et al., 2021). As with any new clinical competency, ethical, legal, safe, and evidence-informed practice requires providers to acquire both general knowledge and applied skills, two separate yet overlapping domains of learning (Edirippulige & Armfield, 2017; Perle, 2021). Despite providers and students reporting interest in telehealth-focused education, clinical psychologists have been identified as having variable amounts of telehealth training informing their practice (Baier & Danzo, 2021; Glueckauf et al., 2018; Sammons et al., 2020). To address this gap, a growing number of psychologists have suggested the utility of adapting field-suggested competencies (e.g., efficacy/effectiveness research, adaptations for assessments and interventions, ethics, legal, data security; Galpin et al., 2020; Perle, 2021) to create telehealth-focused education in doctoral-level training (Glueckauf et al., 2018; Traube et al., 2021). Despite recognition, the number of doctoral-level clinical psychology programs offering telehealth education, as well as the foci of such experiences, remains unclear.

Importance of Doctoral-Level Telehealth Programming

Integrating telehealth education into graduate learning holds many benefits over teaching providers post-degree. First, doctoral-level programming has been suggested as one of the most influential predictors of later clinical practice, with information learned during this time applied throughout one’s entire career (Cook et al., 2009). Activities improve one’s ability and self-efficacy with the technology, increasing comfort and subsequent utilization (Cosh et al., 2021). Information gathered also provides a foundation to be built upon through subsequent self-directed continuing education (Babione, 2010). Finally, telehealth education is believed to assist in addressing mental health disparities throughout the USA, as the knowledge gained can be invaluable if a student later decides to reach a wider population (Domino et al., 2019).

Ongoing Need and Current Study

Given the rapid expansion of telehealth, it seems reasonable that doctoral-level clinical psychology training institutions would seek to evolve with field needs by integrating research-informed telehealth education into their programming. However, given no accreditation requirements to include telehealth topics (Comer, 2021), to date, it remains unclear if programs are integrating such training, and if so, what types. While other healthcare fields (e.g., occupational therapy; Serwe et al., 2020) have begun evaluating graduate-level telehealth education to inform future activities, no known assessment has occurred within psychology. To inform initiatives, it remains prudent to determine current developments and ongoing needs for one of the most influential times in a clinical psychologist’s career (Edirippulige & Armfield, 2017). As such, the current study sought to evaluate doctoral-level clinical psychology training programs throughout the USA prior to and following COVID-19. Given limited literature to guide hypotheses, the current study was predominantly viewed as exploratory. Nevertheless, in line with increased popularity and usage, it was hypothesized that less than 50% of programs will have implemented telehealth-focused education prior to COVID-19, but greater than 50% post-COVID-19.

Method

Participant Recruitment and Procedure

Participants were recruited between September and November 2021 via standardized emails sent to the doctoral-level clinical psychology training program’s (i.e., university/college) website-listed director or chair. They were selected due to either their membership in the Council of University Directors of Clinical Psychology, or being listed in the American Psychological Association’s Graduate Study in Psychology resource. Each email contained a link to a REDCap-hosted study information sheet that contained study-related information, and the surveys. Due to a documented low rate of return for surveys, emails were brief, with a personalized reminder sent to each individual 3 weeks after initial messages. Snowball sampling was also encouraged. At the end of the study, participants could enter their email into a second unlinked survey to receive either a summary of findings and/or be entered into a raffle for an electronically delivered $25 gift card. Each strategy was suggested to foster participation and completion (Couper, 2000). The study was approved by the institutional review board of West Virginia University.

Inclusionary Criteria

Participants were at least 18 years of age, able to fluently read English, had access to internet capabilities, were members of a clinical psychology doctoral program within the USA, and had knowledge of the program’s curriculum (e.g., director/chair).

Measures

Due to no suitable measures being available for current needs, participants completed a researcher-created demographic survey that inquired about variables relevant to data analysis (i.e., participant factors, program-specific information), as well as a telehealth training survey that inquired about relevant educational activities (e.g., amount of training pre-and post-COVID-19, topics covered, duration of telehealth-focused activities, how information was delivered).

Results

A total of 219 emails were sent. While 28 participants initiated the survey, 14 programs representing 12 states completed the study, met inclusion criteria, and were included in the analyses, suggesting an approximately 6.4% response rate. State representation included California (2 programs, 14.3%), Florida (1, 7.1%), Georgia (1, 7.1%), Illinois (2, 14.3%), Indiana (1, 7.1%), Iowa (1, 7.1%), Massachusetts (1, 7.1%), Minnesota (1, 7.1%), Missouri (1, 7.1%), Ohio (1, 7.2%), Texas (1, 7.1%), and West Virginia (1, 7.1%).

Sample Demographics

Participants (Mean age = 49.8, SD = 10.3) predominantly identified as female (n = 9, percentage of respondent = 64.3%), white (14, 100%), and non-Hispanic (13, 92.9%) with self-identified roles as either department chair or equivalent (4, 28.6%), training director (8, 57.1%), or a core faculty member with knowledge of the program’s telehealth training components (2, 14.3%). Degrees offered included Ph.D. (11, 78.6%) and/or PsyD (7, 50.0%). The number of students enrolled in programs ranged between 0 and 5 (2, 14.3%), 6 and 10 (2, 14.3%), 16 and 20 (2, 14.3%), 26 and 30 (2, 14.3%), 31 and 35 (2, 14.3%), 56 and 60 (1, 7.1%), 61 and 65 (1, 7.1%), 66 and 70 (1, 7.1%), and 100 + (1, 7.1%).

Shifts in Telehealth Knowledge and Training

Prior to COVID-19 (i.e., 2019 and earlier), three programs (21.4% of respondents) indicated having some form of telehealth-focused education in their curriculum. Two of the three programs (66.7%) suggested that components were optional, and one (33.3%) indicated that some components were mandatory. Table 1 provides pre-COVID-19 information regarding student training methods, who provided training, and how faculty/staff prepared to train students in telehealth. Table 2 provides pre-COVID-19 information regarding integrated competency areas, as well as topics that programs deemed most essential for student education. Regarding the amount of time devoted to telehealth-focused education, two programs (66.7%) indicated that prior to COVID-19, 0–5 h of education were provided to students, while one (33.3%) indicated 6–10 h. Potentially due to limited guiding literature, among programs implementing telehealth education, zero respondents indicated utilizing a curriculum or competency model to guide the inclusion of the telehealth information into the program.

Table 1 Program-specific factors among programs implementing telehealth-focused education
Full size table
Table 2 Competency areas integrated into student education among programs implementing telehealth-focused education
Full size table

Post-COVID, of the 14 total programs, two (14.3%) indicated maintaining (i.e., without change) telehealth training that was previously implemented prior to COVID-19, eleven (78.6%) indicated integrating new or additional telehealth training, and one (7.1%) indicating never implementing any formal telehealth material. Among the 13 programs implementing some form of telehealth-focused education, nine (81.8%) indicated that the material was mandatory, one (9.1%) indicated that all material was optional, one (9.1%) noted both mandatory and optional components, and three did not provide a response. Table 1 provides post-COVID-19 information regarding student training methods, who provided training, and how faculty/staff prepared to train students in telehealth. Table 2 provides post-COVID-19 information regarding integrated competency areas, as well as topics that programs deemed most essential for student education. Regarding the amount of time devoted to telehealth education, among the eleven that responded to this item, three (27.3%) reported 0–5 h of total telehealth-focused education in their program, three (27.3%) reported 11–15 h, two (18.2%) reported 21–25 h, one (9.1%) reported 36–40 h, and two (18.2%) preferred not to answer or were unsure. Among the eleven respondents, two programs (18.2%) indicated utilizing a telehealth-focused curriculum or competency model to guide the integration of material; however, neither reported details. Each indicated intention to continue offering telehealth as part of their curriculum.

Regardless of their training approach, pre-COVID-19, thirteen of the fourteen total programs (92.9%) indicated a belief that their students did not have adequate knowledge to practice telehealth ethically, legally, safely, or appropriately. Furthermore, five (35.7%) noted a desire that their students received additional telehealth education prior to beginning their clinical internship, fellowship, or career. Post-COVID-19, twelve programs (85.7%) believed that their students had adequate telehealth knowledge to practice ethically, legally, safely, and appropriately. Despite this, nine (64.3%) continued to wish their students had additional telehealth education prior to beginning their clinical internships, fellowships, or careers.

Upon reflection of importance (N = 14), of the eleven responding to the item, one program (9.1%) indicated that telehealth-specific education is more important than other clinical training components, six (54.5%) indicated that it is equally important, two (18.2%) indicated that it is less important, and two (18.2%) indicated that it is significantly less important.

Discussion

The current study sought to evaluate the implementation of telehealth-focused education in doctoral-level clinical psychology training programs. While response rates were suboptimal, given the dearth of similar investigations, current findings representing 12 states and providing novel details are believed to contribute important information regarding trends and ongoing needs to guide field initiatives.

Increased telehealth utilization throughout COVID-19 is believed to have contributed to programs implementing telehealth-focused education. Specifically, findings supported the primary hypothesis, with data indicating that approximately 21% of surveyed programs had some form of telehealth training prior to COVID-19 with the components predominantly being optional educational targets and encompassing approximately 0–5 h of the student’s overall education. However, dedication to telehealth shifted post-COVID-19 with approximately 93% of programs indicating telehealth-focused education integration, with a large proportion being mandatory. Such shifts were also demonstrated through reported programmatic increases in faculty and staff pursing telehealth-focused self-education. As prior literature has suggested that instructors must have adequate up-to-date knowledge to educate students (Callan et al., 2017), the observed increases can be viewed as a positive for noted doctoral-level training programs. In line with increased focus post-COVID-19, the amount of time devoted to telehealth was observed to vary between 0–5 and 36–40 h, with the majority of surveyed programs indicating approximately 11–15 h. Although optimal time commitment is currently unclear, when contrasted to the number of hours needed to learn other clinical competencies (e.g., cognitive-behavior therapy, ethics), 11–15 h can be viewed as a starting place. This foundation can then be enhanced through continuing education across a student’s internship, fellowship, and career in order to fully cover all important aspects of telehealth-based practice. Although increasing the number of hours can be beneficial, 11–15 h can be viewed as a positive and suggestive of attempts for programs to evolve with the field needs. Related to how the education was implemented across the hours, very limited offerings were noted pre-COVID-19, but several programs endorsed offering a range of methods post-COVID-19. For example, post-COVID-19, while significant variability presented, more than 40% of programs endorsed broadly integrating applied hands-on skill practice into general coursework, while 50% or more noted providing hands-on experience in training sites, as well as assigning journal readings. While programs generally demonstrated a mix of didactic and applied knowledge post-COVID-19, the number of programs with specialized coursework remains limited. Given the complexity and breadth, programs could benefit from creating a dedicated telehealth-specialized course to fully encapsulate the subject matter through an integration of both didactic knowledge and hands-on skill practice.

Examination of telehealth competencies included in student education also demonstrated significant changes over time. Prior to COVID-19, overall integration of topics among programs that incorporated some form of telehealth-focused education (n = 3) varied between 0.0 and 66.7% endorsement per item. In line with increases in programs endorsing telehealth-focused education post-COVID-19 (n = 13), endorsed target items also rose with endorsements between 15.4 and 84.6%. Information related to differences between face-to-face and telehealth, methods of tailoring assessments, and interventions for telehealth administration, ethics, safety, and legality were all endorsed among more than 75% of programs. While many endorsed topics were viewed as essential for student education post-COVID-19, it is interesting to see that some topics suggested by the literature as important were identified as less essential by programs. For example, given its prevalence in current practice, it is interesting that so few programs endorsed information and research related to videoconferencing as essential (i.e., < 25%). Similarly, given their focus in practice, it is surprising to find that only approximately 50% of programs implementing telehealth-focused education (n = 13) considered legal considerations, data security, and knowledge of how to tailor assessment or intervention strategies as essential components for student education. Furthermore, it was surprising to find that even the most indicated item of importance, ethical considerations, was endorsed by approximately 75% of programs, suggesting some view it as a non-essential educational target, despite its clear importance for practice. However, it should be noted that reasons for such findings are currently unclear, and it is possible that some programs may see such topics as non-essential only for their training program, as it is an expectation that students receive such information at another stage in their training. Thus, programs may not be suggesting that certain competencies are less important overall, but that they are not primary targets when weighed against other accreditation-required educational objectives.

Implications and Application for Doctoral-Level Training Programs

While it appears that programs began including telehealth components into their doctoral-level education, variability presented in the amount of time and types of techniques used. As programs continue integrating telehealth-focused activities into their curriculum, literature has suggested a variety of strategies to ensure comprehensive education encompassing both didactic knowledge and hands-on experiences. As detailed by Perle (2021), training is believed to require education at multiple stages of student development with a focus on increasing complexity that parallels emerging abilities. Training should begin by integrating foundational telehealth information into core education. Optimal courses can include both general and advanced ethics, assessment, and/or intervention classes. Foundational introductions can be enhanced through comprehensive specialized telehealth-focused coursework that can target mid- to higher-level doctoral students. Whether offered in person or online (e.g., self-directed modules), the class should include didactic instruction; applied practice; graded evaluation of knowledge and skill; and out-of-class readings via telehealth-focused journal articles, websites, and textbooks (see Perle, 2020 for an example of methodology). Information acquired in the specialized class should then be applied via real-world practicum experiences. Practicum should begin with the least amount of difficulty, and high support, before evolving to higher complexity and less support. Such experiences should be supervised via live and scheduled methods in order to review (a) the telehealth components, (b) the clinical components, (c) the integration of the two, and (d) the next clinical steps (Perle, 2021). Finally, each new training component should be evaluated by the doctoral-level program to determine impact. Outcome assessment targets should include changing attitudes toward the technology, as well as satisfaction with novel educational techniques (e.g., coursework, supervision). Assessment should examine proximal and distal student outcomes, including evaluation of whether training increases evidence-informed knowledge and skills, whether it fosters increased future use of telehealth, and whether it increases outreach to underserved areas. Collected information can assist iterative development of a program’s telehealth-focused training to maximize outcomes.

Limitations

While believed to be one of the first to evaluate doctoral-level clinical psychology telehealth training opportunities, the current study is not without recognized issues. First, the current response rate limited analyses to only a fraction of all American training programs, with some non-participatory programs potentially having integrated telehealth-focused education. Furthermore, as the materials were created for this specific study, they have not been psychometrically evaluated. It was also recognized that while materials were estimated to take approximately 10 min to complete, this is still a significant time request for busy academics, potentially contributing to the reduced response rate. Related to the data itself, it should be emphasized that self-report surveys were utilized, suggesting that findings are limited by the participant’s recollection of their program’s educational activities. Finally, the current study did not evaluate how training influenced student’s practices nor did it evaluate relative contributions of training aspects in terms of optimal duration, foci, or sequencing.

Future Directions

Future work should build upon the foundation provided in the current study to further clarify observed trends. This work should first seek to validate and update the measures. More detailed questions can assess factors contributing to a program’s choice to implement telehealth-focused programming. Novel items can also evaluate decision-making processes regarding future implementation of telehealth activities. A larger sample should also be collected. This can facilitate more complex analyses to better understand field initiatives and needs. This larger sampling can also adopt longitudinal designs to both track implementation and monitor proximal and distal effects of the education to determine the degrees of integration into a student’s practice (e.g., improved practice that aligns with field recommendations, increased telehealth use, increased outreach to underserved areas). Finally, the training itself warrants greater evaluation to determine the duration, foci, and sequencing of educational activities to maximize student education. For example, the amount of time, overall, or per topic, needed to ensure competent practice, as well as the most optimal component ordering (e.g., didactic training prior to practicum or during), can assist in establishing ideal training methodologies.

Conclusion

The current study was viewed as a preliminary investigation to assist in clarifying the degree to which doctoral-level clinical psychology training programs have integrated telehealth education into their curriculum pre- and post-COVID-19, as well as the focus of the integrated components. Despite current data demonstrating positive developments that align with the shifting zeitgeist, the current study is viewed as a starting point. Additional research is required to further clarify field offerings, as well as the details of implementation. Ultimately, to ensure an ethical, legal, safe, and evidence-informed practice, graduate programs must continue to adapt and evolve to meet shifting field needs.