Effectiveness of Self-Guided Virtual Reality–Based Cognitive Behavioral Therapy for Panic Disorder: Randomized Controlled Trial

Background Virtual reality (VR) is as effective a technique as traditional cognitive behavioral therapy (CBT) and a promising tool for treating panic disorder symptoms because VR exposure can be safer and has better acceptability than in vivo exposure and is more immersive than exposure through imagination. CBT techniques can be delivered more effectively using VR as well. So far, VR has required high-quality devices, but the development of mobile VR technology has improved user availability. At the same time, a well-structured form of VR can be reproduced and used anywhere. This means that VR can be used to provide a self-guided form of treatment and address the high treatment costs of evidence-based therapy and the lack of professional therapists. This study aimed to investigate the potential of self-guided VR as an alternative to high-cost treatment. Objective The main goal of this study was to offer data about the efficacy of a mobile app-based self-led VR CBT in the treatment of panic disorder. Methods A total of 54 subjects with panic disorder were enrolled in this study and randomly assigned to either the VR treatment group or waitlist group. The VR treatment was designed to be total 12 sessions for 4 weeks. The VR treatment consists of 4 steps in which patients are gradually exposed to phobic stimuli while learning to cope with panic symptoms in each stage. The effectiveness of treatment was assessed through the Panic Disorder Severity Scale, Hamilton Rating Scale for Depression, Body Sensations Questionnaire, Albany Panic and Phobia Questionnaire, Anxiety Sensitivity Index, State-Trait Anxiety Inventory, Hospital Anxiety and Depression Scale, Korean Inventory of Social Avoidance and Distress Scale, Korean Inventory of Depressive Symptomatology, and Perceived Stress Scale. In addition, physiological changes using heart rate variability were evaluated. Results In within-group analyses, the VR treatment group exhibited improvements in panic disorder symptoms, anxiety, and depression after 4 weeks, while the waitlist group did not show any significant improvement. Compared to the waitlist group, the VR treatment group showed significantly greater improvements in the Panic Disorder Severity Scale in both completer analysis and intention-to-treat analysis. Heart rate variability in the VR treatment group showed improvement in normalized high frequency from baseline to postassessment with no significant differences in any outcome measure between groups. Conclusions The self-guided, mobile app-based VR intervention was effective in the treatment of panic symptoms and restoring the autonomic nervous system demonstrating the validity of the use of VR for self-guided treatment. VR treatment can be a cost-effective therapeutic approach. Trial Registration ClinicalTrials.gov NCT04985019; https://clinicaltrials.gov/ct2/show/NCT04985019


1a-i) Identify the mode of delivery in the title
Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.

선택해제
Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Self-Guided Virtual Reality-based" 1a-ii) Non-web-based components or important co-interventions in title Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").

선택해제
Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

1a-iii) Primary condition or target group in the title
Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial 선택해제 Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Cognitive Behavioral Therapy for Panic Disorder" Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "54 subjects with panic disorder were enrolled in this study and were randomly assigned to either the VR treatment group or waitlist group. The VR treatment was designed to be total 12 sessions for four weeks. The VR consists of 4 steps in which patients are gradually exposed to phobic stimuli while learning to cope with panic symptoms in each stage. The effectiveness of treatment was assessed through Panic Disorder Severity Scale (PDSS), The Hamilton Rating Scale for Depression (HRSD), Body Sensation Questionnaire (BSQ), the Albany Panic and Phobia Questionnaire (APPQ), The Anxiety Sensitivity Index (ASI), The State-Trait Anxiety Inventory (STAI), the Hospital Anxiety and Depression Scale (HADS), the Social Avoidance and Distress Scale (K-SAD), the inventory for Depressive Symptomatology self-report (KIDS-SR), the Perceived Stress Scale (PSS). In addition, physiological changes using heart rate variability (HRV) were evaluated." 1b-ii) Level of human involvement in the METHODS section of the ABSTRACT Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it)

선택해제
Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The main goal of this study was to offer data about the efficacy of the mobile app-based self-led VR CBT in the treatment of panic disorder." 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional offline trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The effectiveness of treatment was assessed through Panic Disorder Severity Scale (PDSS), The Hamilton Rating Scale for Depression (HRSD), Body Sensation Questionnaire (BSQ), the Albany Panic and Phobia Questionnaire (APPQ), The Anxiety Sensitivity Index (ASI), The State-Trait Anxiety Inventory (STAI), the Hospital Anxiety and Depression Scale (HADS), the Social Avoidance and Distress Scale (K-SAD), the inventory for Depressive Symptomatology self-report (KIDS-SR), the Perceived Stress Scale (PSS). In addition, physiological changes using heart rate variability (HRV) were evaluated." 1b-iv) RESULTS section in abstract must contain use data Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it)

선택해제
Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "In within group analyses, VR treatment group exhibited improvements in PD symptoms, anxiety, and depression after 4 weeks, while waitlist group did not show any significant improvement. Compared to waitlist group, VR treatment group showed significantly greater improvements in PDSS in both completer-analysis and intention-to-treat (ITT) analysis. HRV in VR treatment group showed improvement in normalized HF from baseline to post assessment with no significant differences in any outcome measure between groups." Does your paper address subitem 2a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "In this study, we developed the mobile app-based self-led VR CBT that include comprehensive components of CBT and various VR exposure contents. Patients can learn the CBT techniques by a virtual therapist and are repeatedly exposed to realistic virtual environment. We studied whether this VR app for a self-help treatment can be efficacious for PD. It was hypothesized that individuals using VR app would show greater changes in post-treatment measurement scores than waitlist group. The effectiveness was assessed by the trained psychologists as well as self-reported measurements. In addition, we also measured physiological changes using heart rate variability (HRV) before and after treatments, which was not investigated in previous studies." 2a-ii) Scientific background, rationale: What is known about the (type of) system Does your paper address CONSORT subitem 2b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "In this study, we developed the mobile app-based self-led VR CBT that include comprehensive components of CBT and various VR exposure contents. Patients can learn the CBT techniques by a virtual therapist and are repeatedly exposed to realistic virtual environment. We studied whether this VR app for a self-help treatment can be efficacious for PD. It was hypothesized that individuals using VR app would show greater changes in post-treatment measurement scores than waitlist group. The effectiveness was assessed by the trained psychologists as well as self-reported measurements. In addition, we also measured physiological changes using heart rate variability (HRV) before and after treatments, which was not investigated in previous studies." 3a) Description of trial design (such as parallel, factorial) including allocation ratio 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "After all participants were informed of the purpose of the study and consented to participate, for baseline assessment, a psychiatric interview by a psychiatrist or a clinical psychologist was conducted and a self-rating questionnaire completed. Fifty-four subjects were randomly assigned to either the VR treatment group or waitlist group after completion of the baseline assessment. Considering the dropout rates in previous VR studies were relatively high, we assigned patients to VR groups and waitlist groups in a ratio of 3:2 [40] in order to minimize the imbalance of group size. Randomization was carried out using R (version, 4.0.2) with block sizes of five. The allocation sequence was stratified for gender. Patients were naturally aware of the allocation. But the trained psychologist assessing clinician-administered scales was not able to know the patient's group information from the beginning to the end of the study." Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not changed after trial commencement

선택해제
Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변 Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Sixty-one subjects who were diagnosed panic disorder by a psychiatrist according to DSM-5 diagnostic criteria at a psychiatric outpatient clinic (Yonsei University Gangnam Severance Hospital, Seoul, South Korea) were recruited for this study. To be included, subjects had to meet the diagnostic criteria for panic disorder (with or without agoraphobia) based on the Mini-International Neuropsychiatric Interview (MINI), no change in drug dosage during the study period and age from 19 to 60 years. Patients with a history of major neurological or significant medical illness, or met the diagnostic criteria of current substance misuse were excluded. "

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clarified.

선택해제
Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely webbased trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these. subitem not at all important 1 2 3 4 5 essential 4b) Settings and locations where the data were collected Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Sixty-one subjects who were diagnosed panic disorder by a psychiatrist according to DSM-5 diagnostic criteria at a psychiatric outpatient clinic (Yonsei University Gangnam Severance Hospital, Seoul, South Korea) were recruited for this study. " "The VR treatment aims at learning how to cope with panic symptoms and exposing participants to feared situations in a gradual and planned manner. The mobile application was designed to be used without a supporting therapist, although it can be used alongside a therapist or in a clinical setting if needed. The treatment plan for four weeks was designed to be three times a week, total 12 sessions. Each session takes 15 to 30 minutes depending on modules. All participants completed at least 12 sessions and we allowed them to do more sessions or repeat modules if they want within 4 weeks."

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.

Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Written informed consent was acquired from all participants at the first visit. The study design and protocol were approved by the Institutional Review Board of Yonsei University Gangnam Severance hospital (IRB no. 3-2018-0292)." Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Sixty-one subjects who were diagnosed panic disorder by a psychiatrist according to DSM-5 diagnostic criteria at a psychiatric outpatient clinic (Yonsei University Gangnam Severance Hospital, Seoul, South Korea) were recruited for this study." 4b-i) Report if outcomes were (self-)assessed through online questionnaires Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.

선택해제
Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 5) The interventions for each group with sufficient details to allow replication, including how and when they were actually administered subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-i) Mention names, credential, affiliations of the developers, sponsors, and owners
Mention names, credential, affiliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Conflict of interest" section or mentioned elsewhere in the manuscript).

Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.

선택해제
Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).

Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.

선택해제
Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientific reporting.

Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.

Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

선택해제
Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We did not use reminder at all.

5-xii) Describe any co-interventions (incl. training/support)
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.

선택해제
Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We did not apply any other additional interventions. Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Fifty-four subjects were randomly assigned to either the VR treatment group or waitlist group after completion of the baseline assessment. Considering the dropout rates in previous VR studies were relatively high, we assigned patients to VR groups and waitlist groups in a ratio of 3:2 [40] in order to minimize the imbalance of group size." Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This item is not applicable.
Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Fifty-four subjects were randomly assigned to either the VR treatment group or waitlist group after completion of the baseline assessment. Considering the dropout rates in previous VR studies were relatively high, we assigned patients to VR groups and waitlist groups in a ratio of 3:2 [40] in order to minimize the imbalance of group size. Randomization was carried out using R (version, 4.0.2) with block sizes of five. The allocation sequence was stratified for gender. Patients were naturally aware of the allocation. But the trained psychologist assessing clinician-administered scales was not able to know the patient's group information from the beginning to the end of the study." Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Fifty-four subjects were randomly assigned to either the VR treatment group or waitlist group after completion of the baseline assessment. Considering the dropout rates in previous VR studies were relatively high, we assigned patients to VR groups and waitlist groups in a ratio of 3:2 [40] in order to minimize the imbalance of group size. Randomization was carried out using R (version, 4.0.2) with block sizes of five. The allocation sequence was stratified for gender. Patients were naturally aware of the allocation. But the trained psychologist assessing clinician-administered scales was not able to know the patient's group information from the beginning to the end of the study." 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Fifty-four subjects were randomly assigned to either the VR treatment group or waitlist group after completion of the baseline assessment. Considering the dropout rates in previous VR studies were relatively high, we assigned patients to VR groups and waitlist groups in a ratio of 3:2 [40] in order to minimize the imbalance of group size. Randomization was carried out using R (version, 4.0.2) with block sizes of five. The allocation sequence was stratified for gender. Patients were naturally aware of the allocation. But the trained psychologist assessing clinician-administered scales was not able to know the patient's group information from the beginning to the end of the study."

11a-i) Specify who was blinded, and who wasn't
Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any). Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Fifty-four subjects were randomly assigned to either the VR treatment group or waitlist group after completion of the baseline assessment. Considering the dropout rates in previous VR studies were relatively high, we assigned patients to VR groups and waitlist groups in a ratio of 3:2 [40] in order to minimize the imbalance of group size. Randomization was carried out using R (version, 4.0.2) with block sizes of five. The allocation sequence was stratified for gender. Patients were naturally aware of the allocation. But the trained psychologist assessing clinician-administered scales was not able to know the patient's group information from the beginning to the end of the study." 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".

Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Fifty-four subjects were randomly assigned to either the VR treatment group or waitlist group after completion of the baseline assessment. Considering the dropout rates in previous VR studies were relatively high, we assigned patients to VR groups and waitlist groups in a ratio of 3:2 [40] in order to minimize the imbalance of group size. Randomization was carried out using R (version, 4.0.2) with block sizes of five. The allocation sequence was stratified for gender. Patients were naturally aware of the allocation. But the trained psychologist assessing clinician-administered scales was not able to know the patient's group information from the beginning to the end of the study." https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 35/55 11b) If relevant, description of the similarity of interventions (this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This item is not applicable (we used waitlist group) Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To compare the difference between groups existed at baseline in terms of demographic and clinical variables, independent t-tests for continuous variables and chi-square (χ^2) analyses for categorical variables were conducted. To assess the effects of the intervention on the clinical scales, we used analysis of covariance (ANCOVA) with randomization group as the independent variable and post-scores for each variables as the dependent variables, controlling for baseline values of each outcome. Analyses on the mean differences between the 2 groups and confidence intervals (CIs) were also conducted to compare the 2 groups for each outcome. Effect sizes (Cohen's d) divided by baseline pooled standard deviation were calculated for within-and between-group changes, with 0.2, 0.5, and 0.8 corresponding to small, medium, and large effect sizes [51]. We conducted both completer-analysis and intention-to-treat (ITT) analyses using multiple imputation (MI) with Multivariate imputation by chained equations (MICE) [52]. To examine the association between the usage of VR and changes in the PDSS score, we conducted correlation analysis using the Pearson's method. All statistical analyses were completed with R studio (R version 4.0.2). In all cases, a twotailed p < 0.05 was considered statistically significant. In addition, linear regression analysis was conducted to observe changes in SSQ scores over time and Pearson correlation analysis was conducted to see the relationship between changes in SSQ and changes in clinical symptoms, including anxiety." subitem not at all important 1 2 3 4 5 essential 12b) Methods for additional analyses, such as subgroup analyses and adjusted analyses 12a-i) Imputation techniques to deal with attrition / missing values Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).

Does your paper address subitem 12a-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To compare the difference between groups existed at baseline in terms of demographic and clinical variables, independent t-tests for continuous variables and chi-square (χ^2) analyses for categorical variables were conducted. To assess the effects of the intervention on the clinical scales, we used analysis of covariance (ANCOVA) with randomization group as the independent variable and post-scores for each variables as the dependent variables, controlling for baseline values of each outcome. Analyses on the mean differences between the 2 groups and confidence intervals (CIs) were also conducted to compare the 2 groups for each outcome. Effect sizes (Cohen's d) divided by baseline pooled standard deviation were calculated for within-and between-group changes, with 0.2, 0.5, and 0.8 corresponding to small, medium, and large effect sizes [51]. We conducted both completer-analysis and intention-to-treat (ITT) analyses using multiple imputation (MI) with Multivariate imputation by chained equations (MICE) [52]. To examine the association between the usage of VR and changes in the PDSS score, we conducted correlation analysis using the Pearson's method. All statistical analyses were completed with R studio (R version 4.0.2). In all cases, a twotailed p < 0.05 was considered statistically significant. In addition, linear regression analysis was conducted to observe changes in SSQ scores over time and Pearson correlation analysis was conducted to see the relationship between changes in SSQ and changes in clinical symptoms, including anxiety."

X26) REB/IRB Approval and Ethical Considerations [recommended as subheading under "Methods"] (not a CONSORT item)
subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To compare the difference between groups existed at baseline in terms of demographic and clinical variables, independent t-tests for continuous variables and chi-square (χ^2) analyses for categorical variables were conducted. To assess the effects of the intervention on the clinical scales, we used analysis of covariance (ANCOVA) with randomization group as the independent variable and post-scores for each variables as the dependent variables, controlling for baseline values of each outcome. Analyses on the mean differences between the 2 groups and confidence intervals (CIs) were also conducted to compare the 2 groups for each outcome. Effect sizes (Cohen's d) divided by baseline pooled standard deviation were calculated for within-and between-group changes, with 0.2, 0.5, and 0.8 corresponding to small, medium, and large effect sizes [51]. We conducted both completer-analysis and intention-to-treat (ITT) analyses using multiple imputation (MI) with Multivariate imputation by chained equations (MICE) [52]. To examine the association between the usage of VR and changes in the PDSS score, we conducted correlation analysis using the Pearson's method. All statistical analyses were completed with R studio (R version 4.0.2). In all cases, a twotailed p < 0.05 was considered statistically significant. In addition, linear regression analysis was conducted to observe changes in SSQ scores over time and Pearson correlation analysis was conducted to see the relationship between changes in SSQ and changes in clinical symptoms, including anxiety." Does your paper address subitem X26-i?

X26-i) Comment on ethics committee approval
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study " The study design and protocol were approved by the Institutional Review Board of Yonsei University Gangnam Severance hospital (IRB no. 3-2018-0292). "

x26-ii) Outline informed consent procedures
Outline informed consent procedures e.g., if consent was obtained offline or online (how? Checkbox, etc.?), and what information was provided (see 4a-ii). See [6] for some items to be included in informed consent documents.

Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) 선택해제 RESULTS 13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

내 답변
Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study see Table 1, 2, and Figure 1 Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT flow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study see Table 1, 2, and Figure 1

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement.

선택해제
Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변 Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study see Table 1, 2, and Figure 1 21. 10. 23. 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., significant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" 선택해제 Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

내 답변
Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We finished this trial because the intended number of patients has been already included subitem not at all important 1 2 3 4 5 essential 16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Table 1 shows this information

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.

Does your paper address subitem 15-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study subitem not at all important 1 2 3 4 5 essential 16-i) Report multiple "denominators" and provide definitions Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention.

선택해제
Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To compare the difference between groups existed at baseline in terms of demographic and clinical variables, independent t-tests for continuous variables and chi-square (χ^2) analyses for categorical variables were conducted. To assess the effects of the intervention on the clinical scales, we used analysis of covariance (ANCOVA) with randomization group as the independent variable and post-scores for each variables as the dependent variables, controlling for baseline values of each outcome. Analyses on the mean differences between the 2 groups and confidence intervals (CIs) were also conducted to compare the 2 groups for each outcome. Effect sizes (Cohen's d) divided by baseline pooled standard deviation were calculated for within-and between-group changes, with 0.2, 0.5, and 0.8 corresponding to small, medium, and large effect sizes [51]. We conducted both completer-analysis and intention-to-treat (ITT) analyses using multiple imputation (MI) with Multivariate imputation by chained equations (MICE) [52]. To examine the association between the usage of VR and changes in the PDSS score, we conducted correlation analysis using the Pearson's method. All statistical analyses were completed with R studio (R version 4.0.2). In all cases, a twotailed p < 0.05 was considered statistically significant. In addition, linear regression analysis was conducted to observe changes in SSQ scores over time and Pearson correlation analysis was conducted to see the relationship between changes in SSQ and changes in clinical symptoms, including anxiety." subitem not at all important 1 2 3 4 5 essential 17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval)

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).

선택해제
Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

내 답변
Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 21. 10. 23. 오전 11:24 CONSORT-EHEALTH (V 1. subitem not at all important 1 2 3 4 5 essential 17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended 18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-specified from exploratory 17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a).

Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변 Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 21. 10. 23. 오전 11:24 CONSORT-EHEALTH (V 1. Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We included the information in the manuscript.

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).

Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

내 답변
Does your paper address CONSORT subitem 19? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 21. 10. 23. 오전 11:24 CONSORT-EHEALTH (V 1. Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).

선택해제
Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The present study aimed to investigate the effectiveness of mobile app-based VR for PD in randomized controlled design.

Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study We included this in the limitation and conclusion part

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events. Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "There are several limitations of this study. First of all, our dropout rate was higher than previous VR study [55] and similar with self-help treatment approaches [71][72][73]. As we noted, during the study periods, VR was fully executed by patients, so that there was no other way to increase the treatment adherence of patients except for patient's motivation. This lack of encouragement could be a limitation of fully self-help VR approach, and in the present study, it caused the initial difference in the clinical variable with high drop-out rate. However, we tried to overcome this limitation by using ANCOVA, and by also presenting the ITT analysis results. Second, due to the relatively small sample size, it is difficult to generalize the results even though we tried to provide the more objective physiological evidences as well [74]. It will be necessary to replicate our findings in larger samples. Third, given that we did not perform the follow-up assessment, long-term treatment effects remain unknown. Fourth, HRV was measured only at rest. We would have been more informed if we could also measure HRV in anxiety-provoking states. Finally, it is still not clear whether self-help VR is better than other treatment options since the patients without any intervention was set as the waitlist group in this study."

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations 선택해제 . In this study, many participants completed the whole treatment process although they have never used VR devices, indicating the decent usability of VR. If patients are familiar with smartphone, they can easily overcome some inconvenience in mobilebased VR and no additional training is required. In addition, this study demonstrated that mobile-based VR can be used by patient alone, and also exhibited positive results. It seems obvious that minimizing therapists' effort is related to the cost effectiveness of the treatment [69].
In the context of minimizing therapist intervention, we utilized a virtual therapist who deliver psychoeducational contents and provide guided exposure to encourage the use of CBT techniques. Our result showed the possibility that if VR fits well, patients can benefit from immersive learning experience as well as exposure to feared stimuli under the guidance of virtual therapist. Although its efficacy compared to the VR CBT without the presence of virtual therapist remains to be investigated, virtual therapist seems to be essential components for fully self-guided VR treatment. Additionally, it would be meaningful to try VR protocols including relaxation and breathing techniques, and examine the therapeutic effects in future study. Based on recent studies describing the mechanisms of exposure therapy as inhibitory learning, the exposure to feared stimuli without coping skills may maximize the mismatch between expectations and experience, and may enhance the learning effect rather than the exposure with coping skills [70]. Although traditional CBT protocol including coping skills have been used in this study, our VR protocol can be flexibly modified, so that the recent protocol without coping skills can be applied as well. Then, we could conclude whether it will show better therapeutic effects." 21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study 내 답변 Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Trial Registration: ClinicalTrials.gov NCT04985019; https://clinicaltrials.gov/ct2/show/NCT04985019 Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

X27-i) State the relation of the study team towards the system being evaluated
In addition to the usual declaration of interests (financial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.

선택해제
Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study As a result of using this checklist, did you make changes in your manuscript? * What were the most important changes you made as a result of using this checklist?

내 답변
How much time did you spend on going through the checklist INCLUDING making changes in your manuscript * We spent 2 hours for this As a result of using this checklist, do you think your manuscript has improved? * STOP -Save this form as PDF before you click submit To generate a record that you filled in this form, we recommend to generate a PDF of this page (on a Mac, simply select "print" and then select "print as PDF") before you submit it.
When you submit your (revised) paper to JMIR, please upload the PDF as supplementary file.
Don't worry if some text in the textboxes is cut off, as we still have the complete information in our database. Thank you!