Telehealth in a paediatric developmental metropolitan assessment clinic: Perspectives and experiences of families and clinicians

Abstract Background The application of telehealth in the paediatric setting is growing, and yet, limited research has focused on using telehealth in developmental diagnostic assessment and the consumers' perceptions of their telehealth experience. This study explored parents'/carers' and staff experiences of using telehealth as part of the developmental diagnostic assessment. Methods Parents/carers who attended an assessment between June 2020 and July 2021 that incorporated a telehealth component within a hybrid service delivery model were invited to provide feedback about their experience of telehealth appointments at a multidisciplinary developmental assessment service. All parents were invited to complete an online survey, with a sample of families being offered a telephone interview. Staff members were invited to a focus group to explore their experiences of delivering services via telehealth. Data obtained were analysed descriptively and thematically using a mixed method of analysis. Codes were categorized, enabling facilitators and barriers to be explored. Results The use of telehealth in the diagnostic assessment of complex developmental disorders received high levels of acceptance from parents/carers and staff, despite having limitations such as technical issues, difficulties building rapport between families/clinicians and limited direct observations of the child. Telehealth services are perceived to reduce costs and increase flexibility, including increased ability to accommodate family needs. Conclusions Results demonstrated that telehealth is a highly acceptable mode of service in a developmental assessment service. The current study informs the development of a hybrid service delivery model by enhancing facilitators and reducing barriers commonly reported by consumers and provides direction for future research. Patient or Public Contribution Parents or carers of children who attended a tertiary paediatric assessment unit for a diagnostic developmental assessment completed the online survey and were interviewed.


| TELEHEALTH IN PAEDIATRIC OUTPATIENT CONSULTATIONS
There has been increase in research examining the acceptability and effectiveness of telehealth in the paediatric healthcare setting.
Many paediatric outpatient clinics, for example, ophthalmology, orthopaedics and pre-and postoperative clinics, include history taking and clinical interview as part of their service. Though ordinarily conducted face to face, this can be easily delivered via telehealth. Telehealth consultations have been found to be highly accepted by families and clinicians for outpatient appointments in many countries such as Australia, Canada, the United Kingdom and the United States. 3,[5][6][7][8][9][10][11] Patients, clinicians and referrers of paediatric outpatient services have expressed a willingness to use telehealth, with the advantages relating to convenience and cost savings. 4,9 However, the potential technical difficulties, concerns about child participation in the telehealth session, lack of physical interaction and preference for face-to-face appointments were reported to be the main barriers to telehealth uptake. 4,[12][13][14][15] Due to the circumstances surrounding COVID- 19, there has been an increased willingness to use telehealth, 2 recognizing the importance of balancing the need for accessing services while adhering to physical distancing guidelines.
There is a need to address these impediments, which, if mitigated, could increase the ability of clinical services to be delivered via telehealth in the context of increasing acceptability by families and clinicians.

| TELEHEALTH DIAGNOSTIC ASSESSMENT OF DEVELOPMENTAL DISORDERS
Diagnostic assessments for children for whom there are developmental concerns have traditionally been conducted in the face-toface setting and can take many hours to complete. Such assessments require expertise and utilize a multidisciplinary approach to understand the complex and dynamic nature of the child-environment interaction on development. The challenge of using telehealth in diagnosing developmental disabilities is manyfold. Issues specific to telehealth diagnostic processes pertain to missing out nuances of child responses, not being able to completely assess parental psychological and emotional states and not being able to directly play and interact, as well as interruptions due to connectivity and technical glitches. 16 Evidence on the use of telehealth for diagnostic and developmental assessments is slowly emerging, but still limited. Research has shown that diagnostic interviews can be conducted via telehealth such as The Autism Diagnostic Interview-Revised and Developmental, Dimensional and Diagnostic Interview. 17,18 Novel telehealth diagnostic assessments of autism spectrum disorder (ASD), including the Brief Observation of Symptoms of Autism, Tele-ASD-Peds and the Naturalistic Observation Diagnostic Assessment, have been developed and demonstrate promise through preliminary findings. [19][20][21][22] Telehealth-delivered standardized cognitive and language assessments for children have demonstrated high fidelity, reliability and acceptability. 15,23,24 However, further work is needed to understand parents' and clinicians' perceptions and confidence in using telehealth within the clinical setting.

| DEVELOPMENT OF A HYBRID SERVICE MODEL
The onset of the COVID-19 pandemic brought a great deal of uncertainty and difficulty in predicting the return of face-to-face assessments. Consequently, many paediatric developmental diagnostic services have been faced with increasing waitlists of children with developmental, behavioural and psychosocial issues. It has become apparent that to maintain service delivery, new approaches are needed.
The current study was conducted within a tertiary paediatric developmental assessment unit covering the Western Sydney Local Health District and offering a statewide service for children who do not have access to a local developmental assessment service within New South Wales, Australia. As this unit has contributed significantly to telehealth research in assessments of children 15,23,24 with language and learning difficulties predating COVID-19, it has existing infrastructure including technology and trained staff accustomed to the use of telehealth. This meant that during the first COVID-19 lockdown in Australia, the service was able to shift service delivery from face-to-face clinics to telehealth. With restrictions imposed on face-to-face clinical appointments, the team planned to replace faceto-face with telehealth-only consultations for 3 months (from April to June 2020), with a view to an additional face-to-face appointment when restrictions eased. From July 2020, the unit commenced a hybrid clinic model (see Figure 1) developed as a planned response to the pandemic restrictions. Face-to-face appointments were conducted in accordance with the public health and hospital regulations around wearing masks, hand sanitization, sanitization of equipment and limiting the number of people per room, as well as reducing assessment duration to a maximum of 2 hours.

| CURRENT STUDY
Quality improvement initiatives have long been used for the ascertainment of the efficacy, reliability and safety of health services.
As telehealth appointments were introduced with all families before face-to-face assessment, it was critical to ascertain the acceptability from the perspective of the participants. This project, therefore, F I G U R E 1 Service delivery model.
aimed to obtain the perspectives of families and staff members regarding the use of telehealth as part of the service model within a developmental assessment clinic.

| The Plan Do Study Act cycle
To inform future service development, evaluation using the Plan Do Study Act (PDSA) cycle (see Figure 2) was implemented between 14 May 2021 and 15 July 2021.

| Plan
The unit developed a hybrid service delivery model in response to public health restrictions imposed on face-to-face interactions. This study was submitted and approved as a Quality Improvement project by the relevant Clinical Governance Unit (QIE-2021-02-25) to assess the acceptability of the new clinic model and identify enablers and barriers.
The parent/carer survey was developed by the Sydney Children's Hospitals Network to study the experiences and acceptance of patients/families using telehealth in outpatient clinics across the COVID-19 period. The survey contained 24 questions and focused on 5 key areas: the ease of use, usefulness, patient/family experience, technical quality and usage intention. Multiple-choice questions and Likert scales were used in the survey.

| Do
The unit implemented the hybrid service delivery model from July 2020. The hybrid clinic adopted by the unit was conducted over 2-3 separate occasions of service as follows: 1. Conducting multidisciplinary team clinical interview via telehealth.
The Vineland Adaptive Behaviour Scales, Third Edition (Vineland- 25 was also administered if suitable.
2. Contacting school or preschool and allied health professionals between telehealth and face-to-face appointments.
3. Partial completion of the report before the face-to-face assessment. Parents/carers of children aged between 0 and 18 years, who had a telehealth appointment as part of the diagnostic assessment within the unit between June 2020 and July 2021, were given information about the evaluation study. Towards the end of the assessment, verbal consent was obtained from parents/carers to email them the link to the survey. Participation in the survey was optional and anonymous. In addition, parents/carers not requiring an interpreter were asked if they were willing to be contacted by authors D. B. and E. C. at a later stage for a telephone interview to gain further feedback about the hybrid model of care. The interviews were approximately 10-15 min each. All interviews were voicerecorded and followed a script (Table 1), which were then transcribed verbatim. All identifiable information was removed at transcription.
Demographic information regarding the children and families who completed the survey is shown in Table 2.
Staff members were also invited to provide their feedback and experience of the hybrid service delivery model in a focus group. The focus group was led by an independent facilitator who did not work within the unit. Before commencing, all participants (14 staff members in total, comprising 2 paediatricians, 1 paediatric registrar, 2 neuropsychologists, 2 speech pathologists, 2 social workers, 1 clinical nurse consultant, 2 occupational therapists and 2 administrative officers) received an information sheet about the focus group and completed a written consent form. The focus group was conducted over 2 days, with each session lasting 2 hours. The focus group was voice-recorded for later transcription. All identifiable information was removed at transcription.

| Study
The data collected from the parents/carers survey were analysed descriptively. The parents'/carers' interviews and staff focus group feedback were analysed thematically using the Framework approach. 33

| Act
The themes derived from the parents'/carers' interviews and staff focus group by using the Framework approach informed the enablers and barriers of integrating telehealth into the hybrid diagnostic assessment process. Service delivery improvements were implemented according to the themes, which are described in the discussion section. 6 | RESULTS

| Parents'/carers' evaluation survey
The online survey was sent to 61 parents/carers and 27 responses (44.3%) were received. Two surveys were excluded from the current study due to invalid responses, resulting in the inclusion of a total of 25 surveys in the analysis. Results indicated that 56.0% of parents/ carers used telehealth for the first time (see Table 3). Only 12.0% of the telehealth appointments took more than 10 min to set up and three telehealth appointments were unsuccessful. Two telehealth appointments required switching telehealth platforms due to poor visual and audio quality.
The majority of parents/carers agreed that they were involved in decisions about their child's care and treatment (92%), felt that their child was treated respectfully (96%) and all parents/carers reported that the clinicians explained things in a way they could understand. 2 What did you think about the experience?
3 What did you like about the telehealth experience?
4 What did you not like about the telehealth experience?
5 How can we improve our telehealth and clinical services further?
6 In future, would you prefer telehealth or in-person appointments? Are you able to tell us why?
T A B L E 2 Demographic information of families who completed the survey

| Parents'/carers' semi-structured interview
A random sample of 11 parents/carers participated in the semistructured interview (see Table 4). The age of the children ranged from 3 to 17 years (63.6% male). An overall positive experience was reported by families, even though some technical issues were present during the telehealth appointment: 'I didn't really have anything negative besides when there are interruptions that you cannot help it' (Child B). Families also commented that the staff were attentive, polite, thorough and respectful in the appointments: 'Over the phone they were always respectful towards her, they listened to what we were saying' (Child E).

Convenience and flexibility
Families enjoyed the telehealth appointment for its convenience (see Access to service during the pandemic Parents/carers valued the telehealth appointment because it continued to provide access to services amid physical distancing, lockdown and geographical distance: 'Obviously due to COVID it was helpful.
We didn't have to go out in the community and put ourselves at risk.
Yes, so that was definitely helpful' (Child E).

Support to families
Parents/carers indicated that they were being supported by clinicians during the telehealth appointments: 'I wrote a lot of the stuff down like the questions we wanted to get answered and they answered everything we needed to know and put us in the direction that we

Limited direct observation of the child
Parents/carers expressed concerns about completing the assessment with their child using only telehealth. They believed that the clinicians are not able to complete all assessments via telehealth or to fully observe and understand their child when observed through the computer: 'I feel (the clinician) can't really assess, face to face is better so they can really assess my child' (Child H).

Limited experience of using telehealth
Unfamiliarity with the telehealth platform and technology was also reported to be a challenge for this hybrid service delivery model.

| Preference of appointment type
Overall, despite the convenience, flexibility and opportunity to access the service during a pandemic, face-to-face appointments were perceived by parents/carers as being preferable over telehealth for engaging their child There was also no particular trend in the preference of appointment type based on the age of the child, diagnostic outcomes or assessment completed. Nearly half of the families reported that a combination of telehealth and face-to-face appointments was preferable, with the type of appointment dependent on the context of the consultation, availability and participants involved in the appointment.

Convenience
Staff members reported that some families appeared more comfortable with the telehealth appointments as they could remain at home and the reduction in hospital visits was perceived as convenient. The hybrid model was perceived to enable easier service access for families in rural and remote areas (see Figure 3).
F I G U R E 3 Themes of the hybrid service model emerging from parents'/carers' interview and staff focus group.
ESTHER ET AL.

Flexibility
Clinicians indicated that utilizing telehealth promotes flexibility in clinical practice, work location and use of time. Clinicians felt that they could more easily make appropriate arrangements to meet the needs of families: 'People have been able to work from home to do the interviews and some feedbacks. Parents that aren't able to spend the whole day can join in for just important parts or relevant parts of the assessment'. Some staff also commented that the hybrid model allowed them to work from home if they had slight physical symptoms, or were required to isolate, therefore allowing appointments to continue.

Improved clinical service
Clinicians perceived that the use of telehealth increased the engagement of different stakeholders such as caseworkers and other healthcare professionals involved in the care of the child, school staff and parents: 'It allows the dad to be more involved within the feedback session or if split families to be able to attend it, mum and dad in the same space to give feedback to both parents at the same time without being physically together. We have also had good success with foster parents and the caseworker online with us to give feedback. That was fantastic'.
Clinicians felt that they had the opportunity to gather more information after the initial telehealth consultation and therefore were better prepared for the upcoming face-to-face appointment with the families: 'some of the clinicians, particularly the doctors who take the history like that they might have two weeks between history and the appointment to contact schools'.
Comprehensive assessment and better communication between the children's care providers can be achieved.

Improved assessment skills
Staff have noted that over time, there has been improvement to service delivery using the hybrid model of assessment. This has included staff realizing that they have adapted communication styles and assessment techniques when relating to families online, resulting in what has been perceived to be an increase in parent and child engagement during telehealth.

Technical issues
Technology challenges from the hospital and family's end were reported to be a disadvantage of the hybrid delivery service model.
Staff experienced different technical glitches when conducting telehealth appointments with families: 'Often families were in poor internet connection areas. Our internet connection was quite poor as well'. They also reported that some families had bandwidth issues, 'Even sometimes trying to get them on a mobile phone or a telephone could be difficult', or that families were not financially able to upgrade their mobile data plan for more seamless telehealth appointments: 'parents often ran out of money and then they didn't have access to the phone'.

Challenges to engage with families
Clinicians perceived that it is more difficult to get to know the child via telehealth, and that additional time is required to build rapport and trust with families when compared to face-to-face appointments.
Reading body language and subtle behaviours of the child and family members was also more difficult with telehealth, resulting in challenges when building personal connection and providing emotional support: 'I think we underestimate how much or how important that [rapport] is in building a relationship with our families and you know some of the gestures we use or the support we can offer them with a box of tissues or a warm word or stroke their arm or something. We can't do any of that to support them or even asking them questions and then becoming distressed so I miss that physical contact'. Distractions within the home could also affect rapport: 'Parents were often at home stuck with kids not at school so there were a lot of other distractions going on at home'.

Team cohesiveness
Staff members reflected that since the hybrid model divides the

Longer waitlist and increased staff workload
The only appointments where clinicians felt an increase in workload and time were those involving interpreters. However, the administrative staff's workload did increase due to managing multiple bookings and cancellations. The current electronic booking systems did not enable them to alter appointments easily, resulting in more time and effort for rescheduling.

| DISCUSSION
This study was a quality improvement project looking at the Furthermore, our findings highlighted the need to increase staff proficiency and comfort levels using telehealth for developmental diagnostic assessment. The conversational aspects of the assessment process were identified as being more readily transferable to the telehealth environment, with more work needing to be done to determine when telehealth may provide an appropriate alternative option to face-to-face assessment while maintaining high fidelity and reliability over time. Hence, our clinicians have received additional training on diagnostic assessments that can be reliably conducted via telehealth. This is not only beneficial for overcoming pandemicrelated barriers but also for children and families living in rural and/or remote regions of Australia, where access to such services is limited. 34 Due to the diagnostic complexity of the population who attend developmental services, there is no 'one size fits all' approach. While this study focussed on parental and staff perceptions and experiences, ongoing advancements in telehealth for direct developmental assessments will mean that more research is needed to explore online protocols for developmental and autism assessments; the profile of children and families who best suit these assessments; ways to ensure the reliability/validity of the assessments; and increase uptake by referrers and parents/carers. Accordingly, our research team has been developing new projects to further evaluate the acceptability of the telehealth diagnostic assessment and consumers' feedback on the assessment process. There is a need to explore the relationship between sociocultural, educational and practice-related factors that influence comfort levels of clinicians and administrators towards the uptake, utilization and dissemination of telehealth in clinical practice.

| Limitations
There are some limitations to this quality improvement project, which include the small sample size of parents/carers who completed the online telehealth questionnaire and interviews. A larger sample size would draw on a greater range of experiences. Those parents/carers who completed the questionnaire and were interviewed had adequate English skills to effectively communicate with the research team and no interpreter was used. Only those parents who opted for telehealth assessment were included in the study. The latter two factors reflect possible bias in the sampling and therefore may not identify the full range of issues and complexities faced by families from culturally and linguistically diverse groups and the broader population when engaging in telehealth services.
Additionally, the interviews with parents/carers were completed within a range of 1 week to 5 months post telehealth appointments.
This variation and hence reliance on parent/carer memory could influence the quality and specificity of the information obtained.

| CONCLUSION
The importance of being able to connect with and access health services during a time of public health restrictions forbidding prolonged face-to-face contact is reflected in the feedback received from both parents/carers and staff. In this study, factors that contributed to the successful delivery of telehealth services were identified by both parents/carers and clinicians. This mode of service delivery received high acceptance from both groups. The identification of barriers, such as technical issues, limited ability to build rapport between families/clinicians and limited opportunities for direct assessment of the child, provides clinicians with the opportunity to put strategies in place to reduce these barriers and further increase the success of telehealth within this clinical setting.
Further studies and research into effective means of directly observing and using standardized assessments via telehealth would be most valuable to increase the scope, validity and reliability of the use of telehealth within a paediatric diagnostic developmental service. There is also the potential to use telehealth for developmental screening programmes enabling families to access relevant services in a timely manner. With increased evidence supporting the equivalence of telehealth and face-to-face assessments, it is expected that uptake of telehealth by parents and clinicians will increase. Research should also obtain opinions of those who did not opt for telehealth to determine barriers to uptake of telehealth. This study indicates that there are many positive aspects of delivering a service via telehealth, even within a complex paediatric setting. Even when public health restrictions forbidding prolonged social contact cease, there is a place for telehealth service delivery to provide the flexibility, convenience and cost-saving advantages identified by both consumers and clinicians.

AUTHOR CONTRIBUTIONS
All authors have made substantial contributions to the work. Esther