An ecological momentary intervention incorporating personalised feedback to improve symptoms and social functioning in schizophrenia spectrum disorders

This study examined the feasibility and effectiveness of an interactive smartphone application that aimed to improve daily-life social functioning and symptoms in schizophrenia spectrum disorders (SZ) with Experience Sampling Method (ESM) derived personalised feedback.Two groups of outpatients with a diagnosis of SZ were included (one receiving ESM-derived personalised feedback (n = 27) and one without feedback (n = 23)) and used the interactive smartphone application for three weeks. Main outcomes were momentary symptoms and social functioning, as assessed by ESM questionnaires. Additionally, feasibility and user-friendliness of the application were assessed. Theresponse rate was 64% for the ESM questionnaires. In the feedback group, participants indicated that on 49% of the ESM days %they acted on at least one personalised feedback prompt per day. Momentary psychotic symptoms significantly decreased over time only in the feedback group. Momentary loneliness and questionnaire-assessed psychotic symptoms decreased over time, irrespective of feedback. Participants evaluated the SMARTapp as user-friendly and understandable. Momentary personalised feedback may impact momentary psychosis in daily life. Feelings of loneliness and questionnaire-based measured psychotic symptoms may be more responsive to non-specific effects of daily-life self-monitoring, not requiring specific feedback. Ecological momentary interventions offer opportunities for accessible and effective interventions in SZ.


Introduction
Schizophrenia spectrum disorders (SZ) are characterised by social and community dysfunction (Couture et al., 2006;Garrido et al., 2013). In addition to positive and negative symptoms like hallucinations, delusions and anhedonia, difficulty in navigating the social world has a substantial impact on daily-life functioning (Couture et al., 2006;Fett et al., 2011;Velthorst et al., 2016). This is reflected in key characteristics of the disorder, e.g. social withdrawal and poor social interactions (Billeke and Aboitiz, 2013;Penn et al., 1996), as well as difficulties in maintaining relationships with family and friends (Burns and Patrick, 2007;Pinkham and Penn, 2006). Functional and social impairments remain a challenge to treat (Robinson et al., 2004;Wykes et al., 2008). If social functions are targeted in interventions, effects often do not transfer to daily life (Couture et al., 2006;Pos et al., 2019;Roberts and Velligan, 2012), which may be related to the low (social) motivation associated with a diagnosis in the schizophrenia spectrum (Medalia and Saperstein, 2011). Supplementing treatment with support in real life may lead to greater functional improvement (Berry and Haddock, 2008;Bradshaw et al., 2007). This is, for instance, https://doi.org/10.1016/j.psychres.2019.112695 Received 29 July 2019; Received in revised form 10 November 2019; Accepted 16 November 2019 implemented by the Social Cognition and Interaction Training (SCIT) (Penn et al., 2007), which facilitates practice outside the therapy sessions. The SCIT shows promising results on social functioning. An easy, useful, and less resource intensive way to improve social functioning in the context of daily life for patients with a SZ diagnosis may lie in further integration with mobile health applications.
Mobile phone ownership and the willingness to engage with mobile health (mhealth) is growing in populations diagnosed with a mental health disorder and up to 81% of patients diagnosed with schizophrenia own a smartphone Lim and Penn, 2018;Visser et al., 2018). One of the most widely-used and validated methods to monitor experiences and behaviour in the flow of daily life is the Experience Sampling Method (ESM), also called Ecological Momentary Assessment (Delespaul, 1995;Granholm et al., 2011Granholm et al., , 2007Myin-Germeys et al., 2009). In ESM, participants answer a set of questions several times a day at random intervals, which allows for real-time monitoring of behaviour, mood, symptoms and context. Incorporating ESM in mhealth interventions provided promising opportunities in promoting health behaviour in the general population (Heron and Smyth, 2010) and more recently, in psychiatric disorders (Granholm et al., 2007;Hartmann et al., 2015;Kramer et al., 2014;Myin-Germeys et al., 2016;Visser et al., 2018). For example, prodromal symptoms of relapse in schizophrenia were identified successfully through monitoring of fluctuations in momentary symptoms, causing a reduction of the number of hospitalizations by 60% (Španiel et al., 2008). Another mobile intervention study offered pre-scheduled and tailored interventions targeting voices, mood, sleep, social functioning and medication use. After using the application for one month patients showed a decrease in psychotic symptoms, depression and general psychopathology (Ben-Zeev et al., 2014). Others showed that sending automated pre-programmed personalised text messages in response to ESM entries increased social interactions (Granholm et al., 2011). In addition, motivational aspects in daily life were succesfully targeted through a mobile intervention in an early psychosis sample; improving self-reported symptoms of depression, defeatist beliefs, self-efficacy, and showing a marginal increase in motivation and pleasure (Schlosser et al., 2018). While these studies yielded initial evidence of beneficial effects, they did not include an ESM control group. Research shows that patients often experience a therapeutic effect through monitoring of their experiences and behaviour, whether they use mobile devices or a paper and pencil method. Monitoring symptoms in daily life during cognitive behavioural therapy improves the outcome of the treatment Os et al., 2013;Torous and Firth, 2016). One study investigated a single-session intervention augmented by automated prompts on a mobile device in serious mental illness (schizophrenia and bipolar disorder). Three groups were included: with and without personalised cognitive behavioural therapy (CBT) prompts and a treatment as usual (TAU) group. The intervention resulted in modest, yet sustained improvement in general psychopathology, measured by questionnaires, in both CBT groups; with and without automated prompts. Incorporating personalised elements of CBT through automated prompts had an additional positive impact on community functioning and defeatist attitudes (Depp et al., 2018).
To disentangle symptom monitoring effects and personalised feedback effects, the current randomized controlled study included an experimental group that received personalised feedback prompts in response to their answers on the ESM questionnaires and an ESM control group that did not receive such feedback. We were interested in finding out whether using an interactive smartphone application is feasible in a SZ sample and whether providing personalised ESM-derived feedback can ameliorate symptoms and improve social functioning. We tested the corresponding hypotheses: (1) the application would be usable and understandable, and (2) the interactive feedback group compared to the no-feedback group would show larger improvements over time in momentary symptoms and social functioning, as measured by ESM, and symptoms and social functioning, as measured pre-and post-intervention by questionnaire-based measures.

Subjects
Sixty-four individuals with a SZ diagnosis were included. Participants were recruited through: (1) research collaborators; (2) assertive community treatment teams, i.e. GGZ inGeest, GGZ Delfland, Mentrum, Arkin, Altrecht, Dijk en Duin, and Yulius; (3) hospitals, i.e. Amsterdam Medical Centre, University Medical Centre Utrecht; and (4) with the help of patient-and relative associations, i.e. Anoiksis, Ypsilon, Phrenos, and PsychoseNet. Inclusion criteria for all participants were: (a) a SZ diagnosis according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; (American Psychiatric Association, 2013)); (b) age between 18-60 years; (c) an IQ of above 70; (d) able to read and understand Dutch; and (e) the ability and willingness to sign informed consent. This study was approved by the medical research ethics committee of the Medical Centre of the VU University Amsterdam [NL56511.068.16].
Of the 64 participants enrolled, 14 dropped out of the study for a variety of reasons: one participant was excluded because of the wrong diagnosis, and for four participants data was lost due to technical errors during the automated data transfer. There were nine non-completers, three of whom withdrew from the study due to various (personal) reasons, and six of whom completed fewer than 30% of the ESM questionnaires. Some additional information on the subjective experiences of these latter six participants are summarized in supplement A. There were no significant differences between completers and noncompleters on any of the investigated demographic or clinical characteristics (see supplement B - Table 1). The final data analysis of this study included 50 participants.

Measures
The Positive and Negative Syndrome Scale (PANSS; Kay et al. (1987)) was used to assess positive and negative symptoms in the two weeks before testing to get a baseline measure of symptoms. Participants' subclinical self-reported positive and negative psychotic symptoms one week prior to testing were assessed with the Community Assessment of Psychic Experiences (CAPE; (Konings et al., 2006); Stefanis et al. (2002)). This self-report measure is more sensitive to pick up on subtle changes during the three-week intervention period. The Social Functioning Scale (SFS; Birchwood et al. (1990)) was included to assess social functioning in the domains social withdrawal, interpersonal functioning, recreation activities and pro-social activities. Two subtests of the Wechsler Adult Intelligence Scale (WAIS-III; Wechsler et al. (1997)) were used as an indicator of general cognitive ability: the vocabulary subtest, a verbal comprehension task, and the letter-and number span subtest, a working memory task.

The SMARTapp
The SMARTapp (Schizophrenia Mobile Assessment and RealTime feedback application) was made using custom questionnaires which were built on the PsyMate™ platform (www.psymate.eu), which is a platform including a smartphone app, a cloud-based data storage and a reporting module, that allows customized collection of ESM data (thoughts, feelings, and behaviour) in everyday life. Research has shown that patients found the PsyMate™ application user-friendly and that it is easily accessible even for people who are not acquainted with smartphones and its applications (Myin-Germeys et al., 2011). Participants were randomly assigned to either of two groups: (1) one where the SMARTapp provided feedback according to the participants' daily ESM entries, or (2) one where the SMARTapp included only ESM questionnaires without personalised feedback.
All participants completed up to six short ESM questionnaires daily when prompted by a beep, for a duration of three weeks. In the morning, all participants received a medication and morning hygiene reminder. The ESM-beeps occurred semi-randomly between 10:00 and 22:00; within time blocks of 130 min to ensure accurate representation of the flow of their daily lives. Symptoms, social activities and mood were assessed. Furthermore, participants were asked to fill in one additional evening questionnaire before they went to bed (available from 20:00 until 04:00). This questionnaire asked general questions about their day (e.g., "I have been alone for most part of the day"), and whether using the application had influenced their day. Questions were answered on a 7-point Likert scale, by fixed answer choices or with a binary yes/no answer. Items that were used to measure social functioning included questions about social engagement, feelings of exclusion and loneliness (see supplement C - Table 2 for ESM questions). Symptoms were assessed in the domain of psychotic experiences and positive and negative affect (e.g. cheerful, relaxed, irritated, ruminating) as previously used by others (Kramer et al., 2014;Myin-Germeys et al., 2009).

Personalised feedback vs no-feedback group
The SMARTapp was identical for both groups, except that one group received personalised interactive ESM-derived feedback from the application in the form of two tailored prompts a day. The prompts provided suggestions for a certain activity or behaviour change, depending on the previous ESM answers. The application provided feedback in the following categories: (a) psychotic symptoms, (b) social engagement, (c) health behaviour (i.e. sleep, eating), (d) physical activity, and (e) mood and emotion. Feedback-prompts were programmed in such a way that even if ESM questions were answered in a similar fashion, participants did not receive the same prompt twice in a day. In the evening questionnaire, the feedback group was asked whether they acted upon the suggestions or not.

Procedure
All participants received written information by mail or e-mail prior to the first visit. They were asked to complete a personal-items-checklist regarding their favourite activities, coping mechanisms and social contacts, and to bring this list with them to the first (baseline) session. Testing took place at the VU University Amsterdam. Participants first gave written informed consent and then completed a battery of clinical measures (see Fig. 1). Participants who did not own a smartphone (14% across completers and non-completers), were provided with one (model: LG K120E), and for them additional training was provided on how to use the smartphone. The application was personalised for all participants, both with and without feedback, according to the personal preferences of the participant. For instance, participants filled in enjoyable activities, several social contacts, comforting thoughts and relaxing activities. They could access the comforting thoughts and relaxing activities at any time in the application. Other information, i.e. enjoyable activities and social contacts, were used to provide personalised feedback (see supplement D for personal list and coping tips). After this, the different elements in the SMARTapp were explained, as well as the meaning of the questions and response options and participants completed a practice ESM questionnaire together with a researcher. Participants were instructed to carry their phone with them and to complete the ESM questions whenever possible. They received written information about the study to take home.
Participants used the application for a period of 21 days. On day two and day seven participants were contacted by phone to check for technical difficulties and whether they had any additional questions. A contact number was provided for technical support. All data was automatically uploaded to a secure server according to the EU data protection guidelines.
After three weeks participants attended the second session during which they completed the post-measures (see Fig. 1). To make sure that the load of the first session was not too much, we assessed the WAIS in the second session. Participants were then asked about their experiences with the application to assess feasibility and after this they were  Hanssen, et al. Psychiatry Research 284 (2020) 112695 debriefed about the two conditions and the purpose of the study. After revealing their SMARTapp version, participants in the no-feedback group were offered to continue using the application with interactive feedback. All participants were given 150 Euro for study participation.

Data analysis
Statistical analyses were conducted using STATA 14.1 (StataCorp, 2015). To inspect the differences between groups on demographics and clinical characteristics at baseline regression analyses and chi-square tests were used.
For the ESM questions, a mean per beep was calculated for each participant for psychotic symptoms ('suspicious', 'disliked', 'harmed', 'voices', 'apparitions'), positive affect ('cheerful', 'relaxed', 'content'), and negative affect ('irritated', 'sad', and 'ruminating'). These were used as dependent variables, as were social functioning outcomes ('prefer not to be alone', 'feeling excluded' and 'feeling lonely'), the evening question ('I have been alone for the most part of the day') and questionnaire outcomes (CAPE and SFS). Mixed multilevel regression analyses were used to account for repeated observations within subjects (minimum of 38 per participant, 30% of the beeps in 21 days), with group (feedback vs. no-feedback) and time (all ESM questionnaires over time/baselinepost intervention) and their interaction as independent variables. In a similar fashion, logistic multilevel regression analyses were run to examine being alone (yes/no) over time.

Demographics and baseline symptoms
Participant demographic information and clinical characteristics are shown in Table 1. The feedback and the no-feedback group differed in baseline negative symptoms; the feedback group had a lower negative PANSS scale score than the no-feedback group. The CAPE and SFS baseline scores are displayed in Table 2. The feedback group had a significantly higher score on interpersonal functioning at baseline measured with the SFS (b = −0.71, 95%CI [.16, 1.27], p = .01), all other CAPE and SFS baseline scores did not differ between groups (all p ≥ .21).

SMARTapp use
The completers replied to 80 beeps (SD = 22.3) over three weeks (64%). The minimum was 40, maximum 126 (of 126). Including the six non-completers, who were dropped because of too little beeps, an average of 74 beeps (SD = 27.1) were completed (59%). No significant differences were found for completion between the feedback and nofeedback group (p = .76). The same pattern was found including the six non-completers (p = .81). The completion rate for evening questionnaires was 18 (SD = 4.1 or 84% (range 2 to 21). There were no significant differences between the feedback and no-feedback group (p = .40).
At the end of the day, the interactive version of the application asked participants whether they acted on the feedback suggestions. Participants reported that on 49% of the ESM days they followed at least one of the two suggestions they got from the personalised prompts. The percentage of given feedback in each category was: (1) psychotic symptoms 7.5%, (2) social engagement 17.1%, (3) health behaviour 10.9%, (4) recreational or physical activity 44.1%, and (5) mood 20.3%.

Change in momentary symptoms and social functioning
Averages of ESM outcomes per week are displayed in Table 3 For the preference not to be alone or feeling excluded by others there was no group-by-time interaction or main effect of group or time in the model without the interaction (all p ≥ .34), nor was there any effect on being alone measured by the evening questionnaire (all p ≥ .10). There was no group-by-time interaction and, in the model without the interaction, no group effect on loneliness (both p ≥ .48), however, loneliness did decrease significantly over time in both groups

Change in questionnaire-based measures of symptoms and social functioning
We examined the effect of group on questionnaire measures for symptoms and social functioning (for pre-and postscores see Table 2). For CAPE positive symptoms there was no group-by-time interaction or main effect of group in the model without the interaction (both p ≥ .59), however, there was a main effect of time (b = −2.5, 95%CI [.20, .32], p < .01), showing less positive symptoms post-intervention in both groups. For the negative and depressive dimension there were no significant interaction or main effects (all p ≥ .08).
There was no group-by-time interaction for SFS interpersonal functioning (p = .81), however, in the model without the interaction, there was a significant effect of group on the SFS interpersonal functioning subscale (b = .76,95%CI [.20,.32], p < .01), indicating that the feedback group had higher baseline and post-intervention levels of interpersonal functioning, which did not change over time (p = .83). The SFS subscales social withdrawal, prosocial activities or recreational activities did not show any significant change (all p ≥ .13).

Participant evaluation of the application
Participants rated the SMARTapp as easy to use (94%) and appealing (95%), indicated that questions were clear (80%), and generally felt that they could reflect their experiences well through the questions provided by the application (68%). Seventy-four percent of the participants said they used the coping tips, and 54% found them useful (43% neutral, 3% not useful). In the no-feedback group, 38% found the application annoying at some point compared to 73% in the feedback group (significantly different, χ 2 = 5.06, p = .03), for example, some participants indicated that there were too many beeps during the day and that they sometimes felt disturbed in their activities by the beep.

Discussion
This ecological momentary intervention study aimed to investigate whether usage of an interactive smartphone application with personalised feedback was feasible in SZ and whether it would improve psychotic symptoms and social functioning. One group received personalised ESM-derived feedback, while the other group received the ESM questionnaires without any personalised feedback, to disentangle the ESM and feedback effects. The findings indicate good feasibility, with high compliance to the application that was rated as user-friendly and understandable. Receiving personalised feedback was associated with a reduction in momentary psychotic symptoms, measured in daily life, in comparison to the no-feedback group. Regardless of whether participants received feedback or not; feelings of loneliness decreased and psychotic symptoms as measured by the CAPE questionnaire decreased.

Effect of the SMARTapp on symptoms and affect
As hypothesized, momentary psychotic symptoms showed a significant decrease over time in the feedback, but not in the no-feedback group, suggesting a beneficial effect of the provided prompts. While the no-feedback group showed no changes in momentary psychotic symptoms, a positive effect on psychotic symptoms in both groups was found on the CAPE questionnaire, showing that psychotic symptoms declined Table 2 Pre-and post-intervention levels of questionnaire-based measures of symptoms and social functioning and their change over time. Feedback (n = 27) No-Feedback (n = 23) Mean (SD) Mean ( 6.9 (0.9) 6.9 (0.9) 6.2 (1.1) 6.1 (1.9) prosocial activities [max. 66] 17.9 (9.4) 17.4 (9.8) 14.5 (9.2) 13.6 (7.6) recreational activities [max. 45] 21.0 (5.3) 20.8 (4.9) 20.4 (7.7) 19.3 (7.2) ⁎ significance level p < .05 Note. The arrows point to the direction of the effect. Note. The arrows point to the direction of the effect. E. Hanssen, et al. Psychiatry Research 284 (2020) 112695 after three weeks. It may be that the no-feedback group, in retrospect, subjectively rated positive symptoms as being lower in the last 3 weeks, while this was not confirmed by the daily ESM entries, possibly reflecting differences between in the moment and retrospective ratings (Moran et al., 2017). The difference may be related to the reliance on patients' long-term memory about their experiences or feelings in the previous weeks. Prospective measurements better reflect the actual mental states. Accumulated sampled measurements best reflect the mental state during the period. Contrary to our hypothesis, no significant group difference was found for negative symptoms measured by the CAPE. Both groups showed a decline in negative symptoms, although this did not reach significance (p = .075).
Momentary positive or negative affect did not change over time and did not differ between groups. This may be related to relatively high average of positive affect and a low average of negative affect at the beginning of this study (e.g. ceiling and floor effects) (Huppert et al., 2001) or it may be that the application does not impact on affect, which seems to be in line with results from an ecological momentary intervention study in depression (Hartmann et al., 2015).

Effect of the SMARTapp on social functioning
We found a decrease in loneliness over time in both groups, as indicated by ESM entries. We did not find an effect on social engagement (i.e. being alone). This does not support the hypothesis that participants in the feedback group would show greater improvement in social engagement than the no-feedback group. However, the found decrease in loneliness is important, because loneliness ratings amongst individuals with a schizophrenia spectrum disorder are high (up to 80%) (Stain et al., 2012) and loneliness is a significant contributor to quality of life and subjective well-being (Eglit et al., 2018). The decrease in loneliness may be partly explained through use of the application itself, related to the monitoring of experiences or coping tips Os et al., 2013;Torous and Firth, 2016) or by the regular contact with the research team. During the evaluation of the SMAR-Tapp, some participants indicated that bacause of the application 'it felt like someone was there for them' and 'someone listened to them' while using the application. In addition, participants may be more inclined to enrol in a treatment study when they are more symptomatic and therefore, these improvements in loneliness, and in positive symptoms measured by the CAPE, could possibly reflect a relative turn towards the better during the fluctuating course of their illness. Future research with a waitlist control group will be necessary to disentangle these effects.
Overall and distinct domains of social functioning, as measured by the SFS, did not change over time and did not show a differential effect of feedback vs. no-feedback. Other research also failed to find effects of interactive feedback on questionnaire-based assessments of symptoms and functioning (Granholm et al., 2011), but did find an effect on dailylife social engagement in a 12-week intervention. It is possible that questionnaire measures may not be sensitive enough to detect subtle changes in social functioning or that the study period of three weeks was too short to have a significant beneficial effect on social interactions. Integrating more sensitive measures, e.g. performance-based measures of social competence, might be more successful in detecting changes in functioning (Bowie et al., 2008) and it may be helpful to include (social) motivational aspects specifically in a mobile intervention to increase social engagement (Schlosser et al., 2018). In addition, integrating mobile sensing, i.e. acquiring data from the environment through a smartphone, may be useful to detect subtle changes in activity levels in an objective way, through geolocations or telephone calls in patients' daily live context (Ben-Zeev et al., 2015;Seppälä et al., 2019). Future studies including personalised feedback may benefit from incorporating video's as feedback, since studies show that patients prefer video interventions because they are experienced as more personal, engaging, and helpful than written interventions (Ben-Zeev et al., 2018).

Feasibility of the SMARTapp
The mobile phone ownership of participants in this study was high (86%) and in line with previous literature Lim and Penn, 2018;Visser et al., 2018). Results on the feasibility of the application were generally positive and compliance was high (64% of the ESM questionnaires and 84% of the self-initiated evening questionnaires). The completion rates did not change over the three week course (63.5%, 61.3% end 66.7% respectively). Also, patients receiving ESM-derived feedback attempted to apply suggestions to their daily lives. Participants generally found the application easy to use, appealing and the questions clear and easy to understand. The feedback group indicated more annoyance from the application, which may be related to a higher number of beeps in total compared to the no-feedback group causing more irritation and disruptions in daily life. Not all participants indicated a reason for feeling annoyed; therefore we cannot pinpoint the precise causes. However, some participants indicated that they sometimes received feedback that was not relevant at the time that they received it. For example, receiving feedback about contacting someone after being alone for most part of the day may not be relevant anymore if the participant just visited a friend or family member. On the other hand, we speculate that participants' annoyance may increase because they find it hard to find the motivation to call someone or to go and be active, even after receiving a feedback suggestion. Future studies should elucidate what the optimal number of beeps is to foster continuous engagement with the app, but not disturbance. In addition, feedback options may be enriched by advice from the patient community, to ensure more relevant and creative suggestions.

Limitations and future directions
Some limitations must be considered with respect to the study findings. First, the results should be considered as preliminary because of a relatively small study sample, which may not provide sufficient power to pick up on interaction effects. Second, the intervention period of three weeks was relatively short. Mobile interventions may need a longer period of time to be able to promote long-term lifestyle changes rather than in the moment coping strategies (Ben-Zeev et al., 2014). However, one of the biggest advantages of working with ESM data is that through this collection technique subtle changes can be detected that might not be detected by standard questionnaire measures (Delespaul, 1995;Kimhy et al., 2012;Os et al., 2013). Third, multiple topics of symptoms, functioning and health-related behaviour were included in the feedback prompts. Because of this, prompts were not solely directed to symptoms or social behaviour. A stronger focus on feedback targeting social functioning may be more effective in improving functional outcome. Last, the study had no waitlist/treatment as usual control group; as such we cannot compare the results in the current study to treatment as usual (TAU) and are unable to differentiate between ESM without feedback and TAU influences.

Conclusion
This study suggests that mobile applications are feasible and incorporating personalised feedback prompts could be beneficial for individuals with SZ in reducing momentary psychotic symptoms. Decreased feelings of loneliness and questionnaire measured psychotic symptoms for all participants may be related to positive effects of monitoring of symptoms and experiences in daily life, study participation or a natural change for the better. Smartphone-based modalities with personalised feedback offer opportunities for simple and accessible interventions. They also offer a way to empower patients to take an active role in their mental health management. For future studies, it would be of particular interest to investigate whether the close integration of mobile interventions with personalised feedback in existing face-to-face treatments could further improve outcomes.

Declaration of Competing Interest
None.