Enabling endpoint development for interventional clinical trials in individuals with Angelman syndrome: a prospective, longitudinal, observational clinical study (FREESIAS)

Abstract

Background

Angelman syndrome (AS) is a rare neurodevelopmental disorder characterized by the absence of a functional UBE3A gene, which causes developmental, behavioral, and medical challenges. While currently untreatable, comprehensive data could help identify appropriate endpoints assessing meaningful improvements in clinical trials. Herein are reported the results from the FREESIAS study assessing the feasibility and utility of in-clinic and at-home measures of key AS symptoms.

Methods

Fifty-five individuals with AS (aged < 5 years: n = 16, 5–12 years: n = 27, ≥ 18 years: n = 12; deletion genotype: n = 40, nondeletion genotype: n = 15) and 20 typically developing children (aged 1–12 years) were enrolled across six USA sites. Several clinical outcome assessments and digital health technologies were tested, together with overnight 19-lead electroencephalography (EEG) and additional polysomnography (PSG) sensors. Participants were assessed at baseline (Clinic Visit 1), 12 months later (Clinic Visit 2), and during intermittent home visits.

Results

The participants achieved high completion rates for the clinical outcome assessments (adherence: 89–100% [Clinic Visit 1]; 76–91% [Clinic Visit 2]) and varied feasibility of and adherence to digital health technologies. The coronavirus disease 2019 (COVID-19) pandemic impacted participants’ uptake of and/or adherence to some measures. It also potentially impacted the at-home PSG/EEG recordings, which were otherwise feasible. Participants achieved Bayley-III results comparable to the available natural history data, showing similar scores between individuals aged ≥ 18 and 5–12 years. Also, participants without a deletion generally scored higher on most clinical outcome assessments than participants with a deletion. Furthermore, the observed AS EEG phenotype of excess delta-band power was consistent with prior reports.

Conclusions

Although feasible clinical outcome assessments and digital health technologies are reported herein, further improved assessments of meaningful AS change are needed. Despite the COVID-19 pandemic, remote assessments facilitated high adherence levels and the results suggested that at-home PSG/EEG might be a feasible alternative to the in-clinic EEG assessments. Taken altogether, the combination of in-clinic/at-home clinical outcome assessments, digital health technologies, and PSG/EEG may improve protocol adherence, reduce patient burden, and optimize study outcomes in AS and other rare disease populations.

Background

Angelman syndrome (AS) is a neurodevelopmental disorder with an estimated incidence of 1 in 22,000 [1,2,3]. AS is caused by the loss of function of the maternally inherited allele of the ubiquitin-protein ligase E3A (UBE3A) gene on Chromosome 15 [4,5,6]. The most common genetic mechanism underlying AS is a deletion on Chromosome 15q11-q13, which encompasses the UBE3A gene, and accounts for approximately 70% of all cases [7]. Other mechanisms include pathogenic variants in UBE3A, imprinting defects (ID), and paternal uniparental disomy (UPD) for Chromosome 15 (jointly referred to here as nondeletion AS) [7,8,9].

AS presents with a broad range of symptoms, including severe-to-profound intellectual disability, lack of speech, ataxia, emotional–behavioral problems, and other medical challenges. These symptoms have a significant impact on individuals with AS and their families, and individuals with AS require 24-h care throughout their lives [10,11,12]. AS has a significant unmet medical need, whereby there are no approved therapies that directly address the core pathophysiology. Disease management is instead focused on symptomatic treatment and supportive assistance.

To support the development of new therapies, appropriate clinical outcome assessments (COAs) and biomarkers are needed to assess meaningful improvements in individuals with AS. Ideally, such assessments should be noninvasive and minimally burdensome [13]. To inform the development and selection of meaningful endpoints for future interventional trials, recent efforts have been made to classify the disease-defining aspects of AS into disease conceptual models based on input from caregivers and clinical experts [11, 14]. Identified AS-defining domains include seizures, sleep disturbance, maladaptive behaviors, impaired expressive communication, poor fine motor skills, poor gross motor skills, impaired cognition, and limited self-care abilities [11].

Besides COAs, digital health technologies (DHTs) offer promise for remote continuous monitoring [15]. However, the utility of DHTs for individuals with AS has not been comprehensively explored. Cortical activity, assessed by electroencephalography (EEG) is a candidate biomarker for UBE3A-related pathophysiology [16,17,18]. Prolonged, overnight video-EEG recordings assure captured sleep, and thus can provide insight into impaired sleep physiology in AS. Prolonged video-EEG recordings can also enable investigators to quantify epileptiform activity in AS, particularly as seizures and interictal epileptiform discharges are sleep potentiated in many patients with epilepsy [19, 20].

Though several studies have described clinical symptoms in AS [21,22,23,24,25,26,27,28,29,30], more comprehensive data are needed for the identification of appropriate COAs to enable endpoint development. In a rare disease population such as AS, innovative, decentralized study designs using a combination of in-clinic, remote, or at-home COAs and DHTs are required to support enrollment, reduce patient burden, and optimize study outcomes.

Here, we describe the 1-year, observational, longitudinal FiRst Endpoint-Enabling Study in Angelman Syndrome (FREESIAS), which was built through a collaborative effort across industry, academia, and together with patient advocacy groups. The primary objective of this study was to evaluate the suitability and feasibility of in-clinic and at-home measures of key AS symptoms for incorporation into AS clinical trials, as well as biomarkers that could capture relevant symptoms and pathophysiology in AS. Remote assessments were also explored to reduce participant and caregiver burden and to increase ecologic validity.

Methods

Study design and procedures

This prospective, observational, longitudinal study (designed following consultations with expert and patient groups) was carried out at six sites in the USA (Boston Children’s Hospital, Boston, Massachusetts; Rady Children’s Hospital, San Diego, California; Rush University Medical Center, Chicago, Illinois; Baylor College of Medicine and Texas Children’s Hospital, Houston, Texas; University of California Los Angeles, Los Angeles, California; University of North Carolina, Chapel Hill, North Carolina) between September 2019 and May 2021 (last study visit). Two clinical visits were planned 12 months apart, as well as a total of three home visits for overnight polysomnography (PSG)/EEG assessments. Data on seizures and sleep were collected continuously via diaries and a sleep mat and in a specific 12-day time window for actigraphy (see Fig. 1). To maintain study adherence, and due to the coronavirus disease 2019 (COVID-19) pandemic, Clinic Visit 2 or Early Withdrawal visits were performed either onsite or remotely. Medical and clinical evaluations such as neurological examinations, weight, height, head circumference measurements, and the assessment of the Bayley Scales of Infant and Toddler Development® – Third Edition (Bayley-III) were conducted in-person. A detailed schedule of activities is in the Additional information (Additional file 1: Schedule of activities). The Bayley-III scale was explored for its potential in investigating the symptoms of AS. To ensure appropriate application, the scale was used according to a standardized administration protocol formulated by a team of psychologists and speech and language pathologists experienced in assessing AS and was applied to all participants regardless of their age [31].

Fig. 1

FREESIAS key domains of interest in AS and study design. A Key symptoms of interest in AS as identified in a previously published AS disease concept model. B FREESIAS study design. The study consisted of two in-clinic visits 12 months apart (Clinic Visit 1 and Clinic Visit 2) and three at-home visits with Home Visit 1 and 3 around the time of Clinic Visit 1 and 2, respectively and Home Visit 2 at 3 months after Home Visit 1. AS Angelman syndrome, COA clinical outcome assessment

Total estimated duration for completion of all clinical scales at Clinic Visit 1 and Clinic Visit 2 was ~ 8 h but was permitted to be completed over 2 consecutive days. Co-enrollment into other nondrug observational studies was permitted. To reduce burden on study participants and their caregivers, synergy with other ongoing nondrug studies was aimed for. The FREESIAS protocol was prospectively designed to be compatible with the ongoing Angelman Syndrome Natural History Study (AS-NHS; NCT04507997). Overlapping assessments were performed once and data were subsequently shared between both AS-NHS and FREESIAS studies. Participants enrolled in FREESIAS for ≥ 6 months were offered priority screening for future AS clinical drug studies run by F. Hoffmann-La Roche Ltd., Biogen, and Ionis Pharmaceuticals Inc. The FREESIAS study was closed after all participants had been in the study for ≥ 6 months. This was also the minimum duration required for participants to be eligible for a priority voucher for screening in future clinical drug trials.

Participants

Eligible participants were children with AS aged 1–12 years, adults with AS, and typically developing children (TDC) aged 1–12 years. Children with AS aged 1–12 years were included as they are the likely target population for future interventional clinical trials. Adults with AS were included given the limited natural history data available for this age group. Age-matched TDC, many of whom were siblings of the participants with AS, were to be included to generate reference data for new assessments such as DHTs. The recruitment proceeded relatively easily, with great interest shown by families of individuals with AS. Published evidence indicates that adolescence is a period characterized by great instability and change in AS symptoms as well as clinical heterogeneity [12, 32]. Therefore, adolescents aged 13–17 years were not included in this study.

Key inclusion criteria included a confirmed molecular diagnosis of AS and a caregiver willing to provide written informed consent, comply with study requirements, and accompany participants to clinic visits. Key exclusion criteria included having an unrelated medical condition that might significantly interfere with AS assessment, and current, planned (i.e., within the study duration), or previous participation (i.e., within 4 weeks) in an investigational drug or device trial. Full inclusion and exclusion criteria are in the Additional information (Additional file 1: Inclusion criteria; Additional file 1: Exclusion criteria). Concomitant use of noninvestigational drug products and nonpharmacologic interventions were permitted, with ideally no change throughout the study duration.

For AS participants, all consent was provided by their respective caregiver independent of the AS participants’ age. Assent was provided for TDC aged 1–4 years, with specific assent forms generated for those aged 5–6 years and 7–12 years that also required signatures from their caregiver.

Clinical outcome assessments focusing on participants

COAs were completed by caregivers or administered by trained evaluators (in clinic or remotely via video conferencing) to the caregiver or participant.

Clinician-reported outcomes (ClinRO) included in the study were the Clinical Global Impression – Severity (CGI-S) scale [33], Vineland Adaptive Behavior Scales® – Third Edition (Vineland-3) [34, 35], Communication Matrix (administered by a trained clinician to caregivers of participants with AS and of TDC) [36], and the Functional Mobility Scale (FMS) [37]. The approach used for CGI-S scale rating is described in the Additional information (Additional file 1: Clinical outcome assessments). Observer-reported outcomes (ObsRO) included Schlaffragebogen für Kinder mit Neurologischen und Anderen Komplexen Erkrankungen (SNAKE; Sleep Questionnaire for Children with Severe Psychomotor Impairment) [38], Composite Sleep Disturbance Index (CSDI) [39, 40], Aberrant Behavior Checklist Second Edition – Community Version (ABC-2-C) [41], Pediatric Quality of Life Inventory™ Generic Core Scales, Version 4.0 (PedsQL™ 4.0 Core) [42], and European Quality of Life 5-Dimensions Questionnaire-Youth Visual Analogue Scale (EQ-5D-Y) [43].

The Bayley-III scale is a standardized developmental assessment for children aged ≤ 42 months although it can be administered to individuals with developmental disabilities beyond the normative age ranges [25, 44]. Due to the substantial impairment of individuals with AS, the Bayley-III was administered outside its normal target age range in this study and used as a performance outcome (PerfO) measure. To reduce cross-site rater-variability, an AS-adapted starting point manual for the Bayley-III was used and is described in the Additional information (Additional file 1: Clinical outcome assessments).

Additional details on the COAs used and a list of the key domains in AS measured by these COAs are in the Additional information (Additional file 1: Clinical outcome assessments). All caregivers were provided an explanation of how to complete the ObsROs by site raters, who themselves were trained by experts.

Clinical outcome assessments focusing on caregivers

To study the impact of AS on the caregiver and family of participants with AS, the following ObsROs were administered: holistic assessment of sleep and daily troubles in parents of children with severe psychomotor impairment (HOST) [45], Pittsburgh Sleep Quality Index (PSQI) [46], Epworth Sleepiness Scale (ESS) [47], Pediatric Quality of Life Inventory™ Family Impact Module (PedsQL™-FIM) [48], and the European Quality of Life 5-Dimensions Questionnaire-Five Levels (EQ-5D-5L) [49].

Digital health technologies

This study tested several DHTs to allow for remote and continuous monitoring of study participants (a detailed description of the DHTs used can be found in the Additional information [Additional file 1: Clinical outcome assessments]). DHTs and accompanying manuals were provided to caregivers at the Baseline Visit.

Using a sponsor-provided smartphone, caregivers of participants with AS were asked to register all seizures that occurred during the study in a trial-specific seizure diary and to complete a trial-specific sleep diary every morning. Caregivers were asked to install Emfit® sleep mats under the mattress in the participants’ beds at home, which used the principle of ballistocardiography to register presence in bed as well as several physiologic, behavioral, and sleep-related parameters. Caregivers were asked to place a wearable activity monitor (actigraph; https://actigraphcorp.com/actigraph-link/) on the participant’s nondominant wrist or ankle, abdomen, or chest. The actigraph was worn for up to 10 days at home preceding and up to 2 days after the first home EEG visit.

Electroencephalography/limited polysomnography

Overnight EEG/limited PSG recordings were performed in the participants’ homes. Data were recorded with Trackit Mk3 (Lifelines Neuro, sampling rate 400 Hz for electrophysiologic signals) and comprised 19 EEG channels (10/20 montage, reference: FC5) [50], and a subset of PSG sensors: one electrocardiogram channel referenced to the EEG reference, two electrooculogram channels (under left eye, and above right eye), two leg electromyography (EMG; left leg, right leg), one abdominal belt, one chin EMG, and one pulse oximeter. Furthermore, the participants were monitored with an infrared camera during sleep to support the data analysis. On the first day of Home Visit 1, 10 min of awake EEG data were analyzed quantitatively following the procedures described by Frohlich et al. (2019) [16] to extract EEG delta-band power.

Statistical analyses

Due to the exploratory objective of the study, the sample size was based on practical considerations rather than statistical power. The split by age was selected based on the prior studies, which revealed greater rate of developmental gains in children with AS aged 1–4 years compared with those aged 5–12 years across different Bayley-III scales. Continuous endpoints were summarized using descriptive statistics such as means, standard deviations (SD), medians and ranges, and categoric endpoints were summarized using proportions.

Results

Recruitment and participant baseline characteristics

A total of 55 participants with AS (aged 1–4 years: n = 16, 5–12 years: n = 27, ≥ 18 years: n = 12) and 20 TDC aged 1–12 years were enrolled over a 12-month period. Seventy-five percent of TDC (n = 15) were co-enrolled with a sibling with AS. Fifty-nine of the 75 participants (79%) were enrolled in the first 6 months of the study (Sept 2019–March 2020), prior to the beginning of the COVID-19 pandemic; the last participant was enrolled in September 2020.

All 75 study participants completed Clinic Visit 1 after screening confirmation (there were no screening failures) and 71 participants completed Clinic Visit 2 (51 in clinic; 20 remote). Four participants with AS discontinued the study prematurely and did not complete Clinic Visit 2 (three participants were identified as lost-to-follow-up and one participant had a major protocol deviation [participant enrolled in an investigational drug trial whilst enrolled in FREESIAS]). The mean (± SD) study duration was 362 ± 86 days, ranging from 222–615 days. Nineteen of 55 participants with AS were co-enrolled in the AS-NHS.

Baseline demographics and genetic characteristics are provided in Table 1. The mean (± SD) age at enrollment for participants with AS was 2.9 (± 0.9), 8.3 (± 2.1), and 24.9 (± 5.9) years for the 1–4 year-old, 5–12 year-old, and ≥ 18 year-old groups, respectively, and 6.5 (± 3.4) years for TDC. In total, 40/55 (73%) had deletion AS and 15/55 (27%) had nondeletion AS. Nondeletion genotypes included UPD (n = 5; 9%), UBE3A mutation (n = 5; 9%), and ID (n = 4; 7%); one participant classified as either UPD or ID as a more specific diagnosis was not obtained.

Table 1 Baseline demographics and genetic characteristics

At baseline, gastrointestinal disorders were reported by 87% of those with AS (48/55), the most common condition being gastroesophageal reflux disease in 60% (33/55) of participants with AS followed by constipation (55%; 30/55). Eye disorders were reported in 47% (26/55) of participants with AS, with strabismus being the most commonly reported (40%; 22/55). Psychiatric disorders were reported in 66% (36/55) of participants with AS including insomnia (15/55) and anxiety (7/55; Table S1). As part of the seizure history assessment a total of 75% (41/55) of participants with AS had epilepsy including generalized epilepsy (34%; 14/41); focal epilepsy (24%; 10/41); and combined generalized and focal epilepsy (39%; 16/41); the epilepsy type of one participant was unknown (Table S2).

Even though no formal anchor was provided (see Discussion), ratings were provided by placing the participant in the context of AS i.e., “mildly ill” when compared to the general AS population known to the expert clinician. The most common CGI-S response for participants with AS aged 1–4 years (31%; 5/16) was a score of 3 (Mildly ill), while the most common response for participants with AS aged 5–12 years (37%; 10/27) was a score of 4 (Moderately ill). Among participants with AS aged ≥ 18 years, an equal number responded with a score of 4 (Moderately ill, 42%; 5/12) and 5 (Markedly ill, 42%; 5/12). Participants with deletion AS were most frequently rated with a score of 4 (35%; 14/40), whereas those with nondeletion AS most frequently received a score of 3 (47%; 7/15). Overall, participants with deletion AS were more frequently rated with a score of 5 (Markedly ill, 28%; 11/40) or 6 (Severely ill, 10%; 4/40), compared with participants with nondeletion AS (Markedly ill, 13%; 2/15 or Severely ill, 0%; 0/15).

Feasibility and adherence of COAs, DHTs, and overnight EEG

Clinical outcome assessments

An overview of the completion rate from all COAs can be found in Table 2. A completion rate of 89–100% was obtained for COAs at Clinic Visit 1 in participants with AS. At Clinic Visit 2, Early Withdrawal, or Remote Visits, the completion rate was 76–91% apart from Bayley-III, which had a lower completion rate of 62%, since it requires in-person administration (which was not possible at some sites in some cases due to COVID-19 pandemic restrictions). TDC demonstrated adherence of 95–100% during both Clinic Visit 1 and 2.

Table 2 Overall completion rates for clinical outcome assessments

Baseline data for all assessments (presented as raw scores, unless stated otherwise) by genotype and age are shown in Table 3. While many assessments showed different levels of mean variability, participants with nondeletion AS tended to have higher scores (i.e., higher performance/less impairment) than participants with deletion AS across assessments. Age-dependent differences were also evident, with those aged 5–12 years generally having higher scores compared with those aged 1–4 years. There were largely no further gains in scores for those aged ≥ 18 years. Cross-sectional comparison of participants with AS aged ≥ 18 years showed similar characteristics to those aged 5–12 years.

Table 3 Baseline COA data by genotype and age

Digital health technologies

Seizure diary

A total of 631 unique seizure events among 18 participants were reported via the seizure diary throughout the study. Some caregivers provided incomplete seizure information. For example, some answers were left blank for recovery time (17%; 105/631 events), type of seizure (14%; 88/631 events), and seizure duration (12%; 74/631 events). A summary of the characteristics of participants and reported seizures is provided in Table S3.

Seizure diary adherence could not be determined as no specific means to track confirmations about the absence of seizures within the app was available. Consequently, it is likely that seizures were underreported. For most participants reporting seizures, there were more seizures reported in the first 90 days of participation than in the last 90 days (see Fig. S1). This suggests that adherence to the seizure diaries declined over time, since there is no obvious reason to assume fewer seizures in the second part of the observation period.

The frequency of seizures reported through the seizure diary was consistent with that reported at baseline in most participants who completed the diary (61%; 11/18). There were 24 individuals out of 41 (59%) with a history of epilepsy reported at baseline who did not report seizures through the seizure diary app nor as an adverse event during the observation period. Three individuals that did not report a history of epilepsy at baseline developed a first seizure during the study (one participant with nondeletion AS age 4.4 years; two participants with deletion AS ages 1.6 and 2.2 years).

Sleep diary

Caregivers were asked to report every morning on the participants’ sleep during the study. Caregiver adherence to the sleep diary was defined as the percentage of days per week within the 52-week observational period in which they completed the diary. The number of participants included in the adherence calculation decreased over the 52 weeks due either to withdrawal or early completion. Mean adherence was 52% (interquartile range [IQR]: 34–72%) for participants with AS and 47% (IQR: 34–70%) for TDC (see Fig. 2A, B). Mean adherence was higher in the first 26 weeks than in weeks 27–52, respectively (participants with AS: 56% [IQR: 42–75%] vs. 48% [IQR: 20–74%]; TDC: 51% [IQR: 43–73%] vs. 42% [IQR: 16–67%]).

Fig. 2

Adherence/completed assessments for sleep diaries, sleep mat, actigraph, and overnight EEG. Adherence to sleep diary completion in participants with AS (A) and TDC (B). Adherence was defined as the percentage of days within the 52-week observational period in which a diary entry was recorded. Adherence to the sleep mat in participants with AS (C) and TDC (D). Shaded area in graphs A–D represents ± 1 SD. Sleep mat usage was defined as the percentage of days per week within the 52-week observational period in which a sleep mat recording was started. The number of participants included in the adherence calculation for both the sleep diary and sleep mat decreased over the 52 weeks due to either withdrawal, rollover to clinical drug trials, or early completion (see “N” above x-axis). E The percentage of nights that the actigraph was used in the Actigraphy Sleep Monitoring Period for those participants for which any data were collected (AS deletion n = 31; AS nondeletion n = 11; TDC n = 13). F Representation of wearing location of the actigraph during the home monitoring period with unknown location (black); right wrist (red); right ankle (orange); left wrist (yellow); left ankle (green); chest (light blue); abdomen (dark blue); and no actigraphy (black). The actigraph was worn for up to 10 days at home preceding and up to 2 days after the first home EEG visit. G Number of participants who completed overnight EEG recordings with usable data of at least 5 h duration at Home Visit 1, 2, and 3. AS Angelman syndrome, EEG Electroencephalogram, SD Standard deviation, TDC Typically developing children

Sleep mat

A sleep mat placed under each participant’s mattress was used to record sleep daily over the course of the study. The actual use period of the sleep mat was defined as the percentage of days per week in which a sleep mat recording was started and was calculated for every week of the 52-week observational period. The number of participants included in the calculation decreased over the 52 weeks due to either withdrawal or early completion (see Fig. 2C, D). The mean (IQR) actual use period was 59% (42–87) for participants with AS and 53% (32–74) for TDC. For participants with AS, mean (IQR) adherence was higher in the first 26 weeks than in weeks 27–52 (63% [43–90] vs. 56% [30–86]); this was not the case for TDC (53% [33–81] vs. 55% [37–76], respectively). Reasons for nonuse were not collected systematically, but the following reasons were identified by the investigational sites: participant interfered with device (n = 5); participant slept in different location (n = 3); caregiver overwhelmed and did not have time (n = 4); sleep mat made bed uncomfortable (n = 3); sleep mat caused participant distress (n = 2); and sleep mat was incompatible with bed (n = 1).

Sleep actigraphy

Sleep actigraphy recordings were planned for each night between 10 days before, up to, and including Home Visit 1, and up to 2 days following Home Visit 1. Due to COVID-19, 20/75 participants did not have a first home visit and therefore were not asked to use the actigraph. Participant adherence to sleep actigraphy was calculated for the remaining 55 participants, defined as the percentage of days within this period on which the actigraph was worn for ≥ 30 min. Mean (IQR) adherence during this period was 56% (0–100) for participants with AS and 44% (2–98) for TDC (see Fig. 2E). Participants were given the option of wearing the actigraph on either the wrist or ankle, or in a pocket on the chest or abdomen and caregivers were asked to enter the wear position in the sleep diary. Some caregivers recorded different positions for different nights (see Fig. 2F). For 16/55 (29%) participants with AS, no actigraph recording was made during Home Visit 1. Reasons for nonadherence were not collected systematically, but the following were noted based on site follow-up with caregivers: not feasible for the caregiver (n = 3); participant did not tolerate actigraph (n = 3); and participant interfered with device (n = 1).

Overnight electroencephalography at home

Ninety out of 225 (40%) of the planned home visits were attempted and led to 85 successful overnight EEG recordings, defined as > 5 h of data per EEG recording; mean (± SD) duration of 14 (± 4.6) hours. The EEG recording period was purposefully ~ 14 h long on an average, as it would start in the afternoon to capture participants’ awake EEG data. Home visits were not carried out for reasons related to the COVID-19 pandemic, e.g., it was legally not possible, the EEG vendor was not able to provide service, or the caregiver did not agree to the home visit due to COVID-19. EEG recordings were obtained from 47/75 participants at Home Visit 1, including 13 TDC and 34 individuals with AS (Fig. 2G; Table S4).

The quality of the scalp EEG data was considered sufficient by EEG experts (neuroscientists and neurologists) in most cases to perform quantitative analyses and to identify waking and sleep background elements, epileptiform discharges (spikes and sharp waves), and epileptic seizures.

FREESIAS data in the context of the AS-NHS data

Comparison of raw scores of Bayley-III baseline data collected in FREESIAS and the AS-NHS [44] was performed only on the 1–12 years age range due to limited data availability on adult individuals with AS in the AS-NHS (Figs. 3A–E; Table S5). For FREESIAS versus AS-NHS data, the mean (± SD) raw score for the Cognitive domain in participants with AS aged 1–4 years was 35.5 (± 7.4) vs. 41.5 (± 10.3) and 49.9 (± 11.1) vs. 51.1 (± 11.4) for those aged 5–12 years. The mean (± SD) raw score for the Receptive Communication domain was 12.9 (± 4.2) vs. 13.8 (± 4.5) in participants with AS aged 1–4 years and 17.4 (± 5.6) vs. 17.6 (± 7.6) for those aged 5–12 years. For the Expressive Communication domain, the mean (± SD) raw score was 9.0 (± 3.7) vs. 10.4 (± 3.8) in participants with AS aged 1–4 years and 11.9 (± 4.3) vs. 12.7 (± 5.2) for those aged 5–12 years. The mean (± SD) raw score for the Fine Motor domain was 24.1 (± 3.8) vs. 26.6 (± 5.4) in participants with AS aged 1–4 years and 29.6 (± 7.3) vs. 32.5 (± 8.8) for those aged 5–12 years, while for the Gross Motor domain, mean (± SD) raw scores were 36.6 (± 10.7) vs. 38.1 (± 9.4) in participants with AS aged 1–4 years and 49.3 (± 6.7) vs. 48.8 (± 6.6) for those aged 5–12 years (Table S5). A time-to-event analysis was performed to evaluate seizure onset age in participants with AS, and the results were compared with the AS-NHS data [26, 44] (see Fig. 3G). Results from the FREESIAS study indicated that participants with deletion AS have an earlier seizure onset and a higher seizure prevalence overall, consistent with previous findings from the AS-NHS study [26, 44].

Fig. 3

Bayley-III profiles obtained using FREESIAS and AS-NHS data. Mean Bayley-III raw scores in participants with AS deletion and nondeletion aged 1–12 years, from FREESIAS and the AS-NHS: A Cognitive domain, B Expressive Communication domain, C Receptive Communication domain, D Fine Motor domain, E Gross Motor domain. F EEG data in participants with AS deletion, nondeletion, and TDC from FREESIAS, AS-NHS, and BCH. Axes are plotted on logarithmic scales. G Kaplan–Meyer analysis of cumulative seizure data, stratified by underlying genotype and data source, in participants with AS deletion and nondeletion from FREESIAS and the AS-NHS. The table indicates the number of participants at risk for each group at 12-month intervals. One FREESIAS participant that presented their first seizure at over 18 years old was excluded from this analysis in order to allow for a direct comparison to the AS-NHS analysis that included pediatric population exclusively. AS Angelman syndrome, AS-NHS Angelman Syndrome Natural History Study, Bayley-III Bayley Scales of Infant and Toddler Development® – Third Edition, BCH Boston Children’s Hospital, EEG Electroencephalogram, TDC Typically developing children

Delta power derived from 10 min of awake EEG recorded in the afternoon of the first day was quantified for baseline recordings. Excess EEG delta power was detected for participants with AS compared with TDC, in line with previous reports (see Fig. 3F). The data also quantitatively matched previous ~ 30-min awake state recordings in participants with AS, thus confirming that usable EEG data were collected in the home setting in the present study [16, 17].

Study expectations and feedback

Insights from caregivers of individuals with AS (N = 54) were obtained from multiple choice feedback questionnaires (Fig. 4; Table S6). Understanding the motivation of caregivers of individuals with AS to participate in a nondrug observational study supports the design and execution of future clinical trials. Based on multiple choice feedback questionnaires collected from these caregivers, key motivators for choosing to participate in the study included: contributing to medical research (93%; 50/54); getting more treatment options (82%; 44/54); gaining a better understanding of AS (76%; 41/54); and raising awareness of AS in the community (41%; 22/54). In caregivers of TDC (N = 20), the main motivating factors to participate in the study included contributing to medical research (95%; 19/20) and getting more treatment options for individuals with AS (60%; 12/20).

Fig. 4

Study expectations and output from feedback questionnaires. EEG electroencephalogram

In total, 74 caregivers completed the questionnaire about the in-clinic visits, including 54 caregivers of individuals with AS and 20 of TDC, although the number of caregivers responding to different parts of the questionnaire differed depending on the question, their previous involvement in clinical research, and willingness to answer. Most of the caregivers of individuals with AS (86%; 37/43) and all of those of TDC (100%; 17/17) expressed that they were satisfied or very satisfied with the experiences of those participants in completing the in-clinic assessments. Most of the caregivers of individuals with AS also deemed the duration of clinic visits for the study to be acceptable (79%; 34/43) and were satisfied or very satisfied (95%; 41/43) with the support given during the clinic visits. Importantly, due to the large number of assessments at the clinical visits, caregivers could choose to perform the assessments over 2 days, which was preferred by a third of all caregivers (35%; 21/60).

In total, 50 caregivers reported on their at-home visit experiences, including 38 caregivers of individuals with AS and 12 of TDC. Home visits were deemed to be a slight (45%; 17/38) or moderate (29%; 11/38) burden.

When reporting on the ease of smartphone use, 28/43 (65%) of the caregivers of individuals with AS were satisfied or very satisfied with it, 33/43 (77%) rated the frequency of smartphone assessments as acceptable, and 42/43 (98%) deemed the time required to complete these assessments to be acceptable. Further answers to the smartphone part of the questionnaire revealed that 59% (22/37) of the respondent caregivers of individuals with AS were very satisfied or satisfied with the experience of the at-home EEG assessment. This may relate to the percentage of caregivers who found the sleep of the child slightly impacted (31%; 11/35) or very/extremely impacted (54%; 19/35). Also, in contrast to the aforementioned slight/moderate burden reported for at-home visits, caregivers of individuals with AS who commented on their preference for at-home vs. in-clinic visits. (n = 40) largely preferred home as the future visit setting (75%; 30/40).

When reporting on the at-home sleep devices, 91% (39/43) of the respondent caregivers of individuals with AS were very satisfied or satisfied with the sleep mat; the score was higher than the actigraph, for which 64% (23/36) of the respondent caregivers of individuals with AS reported that they were satisfied or very satisfied.

In total, 79% (34/43) of caregivers of individuals with AS who commented on their overall experience of the study were satisfied or very satisfied with their experience, with only 9% (4/43) being dissatisfied or very dissatisfied. The main reasons stated for this dissatisfaction included technical issues with some of the DHTs and the request to use digital rather than paper assessments.

Discussion

Feasibility and adherence for clinical outcome assessments and digital health technologies in FREESIAS

In line with the primary objective, this prospective FREESIAS trial demonstrated the feasibility and acceptability of conducting assessments in clinic and at home in participants with AS. The overall adherence results indicate that the key clinical aspects of AS identified by caregivers and clinicians —seizures, sleep, motor function, expressive communication, cognition, self-care, and maladaptive behaviors — can be measured through COAs and DHTs [11, 14]. Adherence and/or uptake was likely negatively impacted by the COVID-19 pandemic and fatigue resulting from the use of certain DHTs. For example, lower adherence for the Bayley-III was observed during Clinic Visit 2 as fewer in-clinic visits occurred due to the COVID-19 pandemic; thus, this may underestimate the Bayley-III adherence in future clinical trials.

The COVID-19 pandemic began approximately 6 months after the start of the trial, which restricted travel and in-person interactions, limited the number of clinic and home visits, and decreased enrollment pace. Amending the protocol to allow for remote COAs enabled adherence to remain high, showcasing investigational site flexibility and successful implementation of a decentralized study setup. However, the resultant reduction in home visits, to less than 50% of those originally planned, proportionately reduced the amount of EEG recordings obtained. Notably, this study illustrates the acceptance and importance of telehealth generated by necessity for remote solutions during the pandemic. Such solutions allowed the participants and their caregivers to continue the data collection while potentially minimizing the risk of them contracting COVID-19. However, for future global clinical trials, telehealth might face local or regional challenges due to absence of supportive IT infrastructure, limited experience using the technologies, or cultural acceptance, hence requiring further feasibility and acceptability assessment [51].

General characteristics relating to medical history, as well as the proportion of participants with deletion and nondeletion AS, were similar to those previously published [52] including the AS-NHS, suggesting that the study was representative of the AS population. Notably, the relatively small sample size of each tested group and the exploratory nature of the observational study objectives deemed any statistical testing inappropriate (Table 1). However, some imbalance can be seen when observing the frequency distribution of deletion vs. nondeletion individuals in the three age subgroups (i.e., deletion in 81% of the 1–4 years group, 63% of the 5–12 years group, and 83% of the ≥ 18 years group), expectedly deeming this a random finding.

Furthermore, consistently with the more severe clinical phenotype previously described [21,22,23,24,25], participants with deletion AS consistently showed lower scores, i.e., more impairment in COAs compared with those with nondeletion AS. In addition, the cross-sectional changes of scores with age were consistent with previously published data [29]. The similarities in results between the FREESIAS and the AS-NHS studies further support the appropriateness of combining the findings to increase the amount of available data, improve the understanding of AS, and support AS clinical trial designs.

Challenges associated with clinical outcome assessments and digital health technologies identified in FREESIAS

No publicly available AS-specific CGI-S scale existed at the start of the study. Therefore, a single question CGI scale was used to characterize the severity of the condition of participants with AS, which lacked a prespecified anchor to ensure intra- and inter-rater reliability. Although participants with deletion AS were more frequently rated as “Markedly ill” (score of 5) and “Severely ill” (score of 6) compared with participants with nondeletion AS, overall, the CGI-S showed little distinction between AS genotypic subgroups. This is potentially due to the heterogeneous nature of AS symptoms but is more likely driven by the aforementioned limitations and suggests that the 1-item scale is not appropriate for future clinical studies in the AS population. An adapted CGI-S that assesses each functional domain, with clearly defined anchors, is likely to be more useful.

For the seizure diary, the absence of an option for caregivers to report if no seizures occurred made it difficult to distinguish between poor adherence versus true absence of seizures; therefore, regular confirmation of absence of seizures should be implemented in future seizure diaries.

Seizure frequency reporting as part of the seizure history at baseline could also be improved in future clinical trials. The categories provided for frequency reporting may have been insufficient for this population. For example, a participant with 385 seizures in 463 days logged in the diaries will report the same frequency category during the clinical visit as a participant with 17 seizures in 360 days, i.e., “ < 1 seizure/day”. More fine-grained seizure frequency bins or reporting the actual number of seizures over an observation period, should be considered. Identifying improved categories on seizure frequency tailored to the AS population, would likely improve overall data quality. A key challenge identified in this study pertained to the acceptance and adherence to DHTs, as well as potential technologic barriers. The feasibility of using digital measures at home is dependent upon access to technology, which potentially biased the inclusion of participants to those who were expected to be largely able to continuously adhere to the study requirements. Providing participants with DHTs, telecommunications technologies, and technical support throughout the study by the Sponsor should be considered as part of the study feasibility process and would ensure that participants are not excluded or discouraged from taking part if they do not have their own technology.

Multidimensional sleep analysis in FREESIAS and at-home overnight electroencephalography

Sleeping difficulties are common in the majority of individuals with AS, which in turn impacts the sleep of their caregivers/families [53]. Sleep behavior of study participants and their caregivers was assessed through standardized questionnaires, a sleep diary, sleep actigraphy, and a sleep mat. The multidimensional approach allowed for comparison of newer DHTs (e.g., sleep mat and actigraph) against more established COAs such as the PSQI and SNAKE. Future comparison between the PSQI, CSDI and ESS, the SNAKE, and HOST may help to determine which COA is the most suitable for future clinical trials.

Adherence for the sleep diary was only considered acceptable, despite high caregiver satisfaction. However, the substantial number of missing inputs from caregivers might pose a challenge to data analysis. Implementing shorter diary completion windows at regular intervals might reduce burden, and in turn, increase adherence and data quality. The sleep mat offered low participant burden and high caregiver satisfaction with similar adherence to the sleep diary and therefore is a viable DHT for use in studies including participants with AS. In comparison, monitoring sleep using an actigraph was more challenging due to lower levels of adherence. To reduce burden, participants were asked to wear the actigraph over the 11-day monitoring windows during sleep only, instead of the 24/7 regimen recommended by the manufacturer, and were given a free choice on wearing location. Despite these adjustments, caregivers of both participants with AS and TDC found the actigraph less convenient than the sleep mat, suggesting that the sleep mat might be more suitable for long-term sleep monitoring in AS.

This study explored at-home assessment with the goal of reducing burden on participants and their caregivers, including pioneering home visits to perform overnight EEG/limited PSG recordings in a multicenter clinical trial setting. Such recordings can, in principle, support a multitude of analyses including quantitative EEG in the awake and sleep state, analyses of sleep structure and elements, and analyses of epileptiform activity and seizures. EEG recorded in the home setting provided usable data in most cases, which were confirmed by recovering the AS phenotype of excess EEG delta power.

Novel strategies to support future rare disease clinical trials

The feedback questionnaire data provided valuable insight into the key motivations for families to join observational studies and assessed the perceived burden of participating in clinical studies. While most caregivers were satisfied with their overall study experience, perceived burdens included technical difficulties, intolerance to assessment, and impact on sleep in participants with AS. However, most caregivers indicated that they would still prefer at-home EEG over in-clinic EEG assessment. The novel insights gained here will collectively support optimization of the design and execution of future clinical trials. Furthermore, these insights also support the use of the novel study-specific seizure diary and sleep diary as well as the application of the customized approach to the limited implementation of PSG in the home setting, with both appearing to be feasible for severely cognitively and behaviorally challenged populations across the age span. These elements can be refined further and implemented more widely to improve quality of data and insights in these domains.

This study represents a precompetitive collaboration between industry, academia, and patient advocacy groups to drive study design and study implementation in AS, with the goal of obtaining valuable information for all parties involved and reducing overall burden for families. Furthermore, the FREESIAS study was amended to provide participants with AS with priority screening for future AS clinical drug trials sponsored by the funding industry partners, upon completing at least 6 months in the FREESIAS study. Based on investigator feedback and the increase in enrollment after publicly announcing this approach, it appeared to be a strong motivating factor for caregivers’ participation and may be a strategy that would benefit future clinical trials in rare diseases with limited patient populations and provide an extended baseline for the subsequent drug trial. With over a third of participants with AS co-enrolled in the AS-NHS, sharing data from overlapping COAs between the FREESIAS and AS-NHS studies was agreed in order to reduce the burden on study participants and their caregivers, and to avoid negatively impacting the ongoing long-term AS-NHS observational study.

Conclusions

This study involved both in-clinic and at-home observations with many moving parts during the COVID-19 pandemic, but despite such challenges generated valuable insights into relevant COAs and DHTs to measure key aspects of AS. Among them, it demonstrated that while participants were highly adherent to the prescribed COAs, the DHTs remain variedly popular. The detailed questionnaires, however, highlighted a generally high acceptance of the employed techniques, and showed that most participants were either satisfied or very satisfied with their overall experience of the study.

The results presented herein pose questions to be addressed through advanced evaluations that would, for example, validate newer COAs through comparing them with older COAs. Such additional longitudinal analyses of these data are already planned and shall be published once ready. Taken together, the current results and future works derived from them may inform the design and strengthen the analysis of future clinical trials in AS and other neurological and neurodevelopmental conditions and rare diseases.