A Web-Based Telehealth Training Platform Incorporating Automated Nonverbal Behavior Feedback for Teaching Communication Skills to Medical Students: A Randomized Crossover Study

Background In the interests of patient health outcomes, it is important for medical students to develop clinical communication skills. We previously proposed a telehealth communication skills training platform (EQClinic) with automated nonverbal behavior feedback for medical students, and it was able to improve medical students’ awareness of their nonverbal communication. Objective This study aimed to evaluate the effectiveness of EQClinic to improve clinical communication skills of medical students. Methods We conducted a 2-group randomized crossover trial between February and June 2016. Participants were second-year medical students enrolled in a clinical communication skills course at an Australian university. Students were randomly allocated to complete online EQClinic training during weeks 1–5 (group A) or to complete EQClinic training during weeks 8–11 (group B). EQClinic delivered an automated visual presentation of students’ nonverbal behavior coupled with human feedback from a standardized patient (SP). All students were offered two opportunities to complete face-to-face consultations with SPs. The two face-to-face consultations were conducted in weeks 6–7 and 12–13 for both groups, and were rated by tutors who were blinded to group allocation. Student-Patient Observed Communication Assessment (SOCA) was collected by blinded assessors (n=28) at 2 time points and also by an SP (n=83). Tutor-rated clinical communications skill in face-to-face consultations was the primary outcome and was assessed with the SOCA. We used t tests to examine the students’ performance during face-to-face consultations pre- and postexposure to EQClinic. Results We randomly allocated 268 medical students to the 2 groups (group A: n=133; group B: n=135). SOCA communication skills measures (score range 4–16) from the first face-to-face consultation were significantly higher for students in group A who had completed EQClinic training and reviewed the nonverbal behavior feedback, compared with group B, who had completed only the course curriculum components (P=.04). Furthermore, at the second face-to-face assessment, the group that completed a teleconsultation between the two face-to-face consultations (group B) showed improved communication skills (P=.005), and the one that had teleconsultations before the first face-to-face consultation (group A) did not show improvement. Conclusions The EQClinic is a useful tool for medical students’ clinical communication skills training that can be applied to university settings to improve students clinical communication skills development.

I.e does the title contain the phrase "Randomized Controlled Trial"? (if not, explain the reason under "other") yes Other: 1a-i) Identify the mode of delivery in the title Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if of翿ine products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms. 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "webbased teleheath training platform "

1a-ii) Non-web-based components or important co-interventions in title
Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study we do not have nonwebbased components 1a-iii) Primary condition or target group in the title Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial subitem not at all important essential Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "for teaching communication skills to medical students" 1b) ABSTRACT: Structured summary of trial design, methods, results, and conclusions NPT extension: Description of experimental treatment, comparator, care providers, centers, and blinding status.

1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT
Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Keep in mind the needs of systematic reviewers and indexers by including important synonyms. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Students were randomly allocated to complete online EQClinic training during weeks 15 (Group A) or were allocated to complete EQClinic training during weeks 811 (Group B). EQClinic delivered a combination of automated visual presentation of students' nonverbal behaviour coupled with human feedback from a standardized patient (SP). ). All students were offered two opportunities to complete facetoface consultations with SPs. The two facetoface consultations were conducted in weeks 67 and 1213, and were rated by tutors who were blind to group allocation. " 1b-ii) Level of human involvement in the METHODS section of the ABSTRACT Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "All students were offered two opportunities to complete facetoface consultations with SPs. " "The two facetoface consultations were conducted in weeks 67 and 12 13, and were rated by tutors who were blind to group allocation. Student Patient Observed Communication Assessment (SOCA) was collected by blinded assessors (n=28) at two time points and also by a standardized patient (SP; n=83)." 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. of翿ine), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials subitem not at all important essential Does your paper address subitem 1b-iii? Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study StudentPatient Observed Communication Assessment (SOCA) was collected by blinded assessors (n=28) at two time points and also by a standardized patient (SP; n=83).
1b-iv) RESULTS section in abstract must contain use data Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study A total of 268 medical students were randomly allocated to two groups (Group A: n=133; Group B: n=135 METHODS 3a) Description of trial design (such as parallel, factorial) including allocation ratio Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "We will use a randomised crossover design in order to conduct an initial test of the efficacy of the EQClinic whereby medical students enrolled in a clinical communication skills course are allocated to one of two groups and asked to complete a teleconsultation using EQClinic at different times of semester. " 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "No content or methodological modifications were made after study commencement."

3b-i) Bug ̅xes, Downtimes, Content Changes
Bug ̅xes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug ̅xes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have in翿uenced study design such as staff changes, system failures/downtimes, etc. [2].
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study no bug fixes to report 4a) Eligibility criteria for participants Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q… 9/35 not in the ms, or brie翿y explain why the item is not applicable/relevant for your study No student recruitment needs to do in this study. Having a tele consultation using EQClinic is compulsory for all students who registered in this course.
"Participants were second year undergraduate medical students from an Australian medical school. All students were enrolled in a communication skills training course provided by the medical school."

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clari̅ed.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study all student register in the course need to use EQClinic 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. of翿ine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasianonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email con̅rmation, phone calls) were used to detect/prevent these.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study No student recruitment needs to do in this study. Having a tele consultation using EQClinic is compulsory for all students who registered in this course.
"Participants were second year undergraduate medical students from an Australian medical school. All students were enrolled in a communication skills training course provided by the medical school. " 4a-iii) Information giving during recruitment Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study Students were asked to sign an online consent form when they first accessed EQClinic.

4b) Settings and locations where the data were collected
Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "The SPs and students were allowed to have the teleconsultation anywhere as long as there was: 1) A web browser on a PC or an Android tablet with external or buildin camera and microphone; 2) A good Internet connection; 3) Good lighting." "A trained tutor was present in the room to observe and assess the performance of the student during the consultation with the SP. The tutors were blinded to condition allocation (Group A or Group B). " 4b-i) Report if outcomes were (self-)assessed through online questionnaires Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "The student and the SP then completed the online assessments. After each interview, the SP assessed the performance of the student using the SOCA Form." "The tutor completed a SOCA Form to assess the student and the SP did not provide any evaluation and feedback for the student on this occasion." 4b-ii) Report how institutional af̅liations are displayed Report how institutional af̅liations are displayed to potential participants [on ehealth media], as af̅liations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Not a required item -describe only if this may bias results) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study The style of EQClinic was followed UNSW medical school web page design guideline.
Institutional affiliations were display at the header of the web pages.

5)
The interventions for each group with suf̅cient details to allow replication, including how and when they were actually administered

5-i) Mention names, credential, af̅liations of the developers, sponsors, and owners
Mention names, credential, af̅liations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Con翿ict of interest" section or mentioned elsewhere in the manuscript).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study EQClinic was developed by the Positive Computing lab at the University of Sydney Australia in collaboration with medical schools at the University of Sydney and University of New South Wales (UNSW) Australia.
In this paper, we did not explicitly mentioned the developer of it. Instead, we cited previous paper.

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results. Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "We have previously described a platform called EQClinic [15,16]. "

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "No content or methodological modifications were made after study commencement."

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not applicable 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing 翿owcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing 翿owcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scienti̅c reporting.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study The detail of system design is presented in paper of reference 15 and 16.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study The application can be accessed through the url: SP: https://ospia.med.unsw.edu.au/sp Student: https://ospia.med.unsw.edu.au/stu

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speci̅c group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study Students were email to access the system through the link on their medical school portal.
"Students were requested, by email, to complete one teleconsultation with a SP through EQClinic. The email described the details of the study and asked them to log into EQClinic to complete the training module. It also informed them that once they finished the training, they could request a consultation time from the slots available on their personal calendar." subitem not at all important essential Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "(i) Weeks 1 to 5, Group A completed a teleconsultation using EQClinic and Group B was blocked from the platform; (ii) In weeks 6 and 7, both groups completed a facetoface consultation. In this period, Group A was still able to access the platform for reviewing feedback only; (iii) Weeks 8 to 11, Group B completed an EQClinic consultation and Group A was blocked from the platform; (iv) In weeks 12 and 13, both groups were asked to complete another facetoface consultation. In this period, Group B was able to access the platform for reviewing feedback only. "

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Interview and assessment components took approximately 40 minutes for a student, and 2530 minutes for an SP to complete. " "Students were emailed to ask to return to the system 24 hours after the consultation to review different kinds of feedback, which included the video recording, the comments from the SP and the automated nonverbal behavior feedback."

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Students were requested, by email, to complete one teleconsultation with a SP through EQClinic. " "After each interview, the SP assessed the performance of the student using the SOCA Form. " "A trained tutor was present in the room to observe and assess the performance of the student during the consultation with the SP. The tutors were blinded to condition allocation (Group A or Group B)."

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Students were emailed to ask to return to the system 24 hours after the consultation to review different kinds of feedback, which included the video recording, the comments from the SP and the automated nonverbal behavior feedback." 5-xii) Describe any co-interventions (incl. training/support) Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study Only technical support were provided for students during the study.
6a) Completely de̅ned pre-speci̅ed primary and secondary outcome measures, including how and when they were assessed Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Tutorrated clinical communications skill in facetoface consultations was the primary outcome and was assessed with the SOCA." "Paired samples ttests were used to compare the SOCA assessment scores for those students that completed EQClinic on their two faceto face consultations. " 6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed Not applicable 6a-ii) Describe whether and how "use" (including intensity of use/dosage) was de̅ned/measured/monitored Describe whether and how "use" (including intensity of use/dosage) was de̅ned/measured/monitored (logins, log̅le analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 6a-ii?

Copy and paste relevant sections from manuscript text
We examined whether a student review their nonverbal behaviour feedback.
6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).

2 3 4 5
subitem not at all important essential Does your paper address subitem 6a-iii?

Copy and paste relevant sections from manuscript text
No qualitative feedback was analyzed in this paper.
6b) Any changes to trial outcomes after the trial commenced, with reasons Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "No content or methodological modifications were made after study commencement." 7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed 7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size Describe whether and how expected attrition was taken into account when calculating the sample size.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study All students enrolled this course had to participate this study.

7b) When applicable, explanation of any interim analyses and stopping guidelines
Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not applicable 8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Participants allocation was assigned via a computer generated random number sequence" 8b) Type of randomisation; details of any restriction (such as blocking and block size) Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not applicable 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study The buildin random number generating function of Microsoft Excel was used.
10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "the administrator of this course" 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment 11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).

2 3 4 5
subitem not at all important essential Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "The tutors were blinded to condition allocation (Group A or Group B). The tutor completed a SOCA Form to assess the student and the SP did not provide any evaluation and feedback for the student on this occasion. " 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator". not in the ms, or brie翿y explain why the item is not applicable/relevant for your study As students have the teleconsultation at different period, they may know the intervention.
11b) If relevant, description of the similarity of interventions (this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not applicable 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Paired samples ttests were used to compare the SOCA assessment scores for those students that completed EQClinic on their two faceto face consultations. " 12a-i) Imputation techniques to deal with attrition / missing values Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Students were asked to sign an online consent form when they first accessed EQClinic."

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study The students log in to EQClinic by redirecting from their medical school portal. EQClinic did not store the password of studens' portal.

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study " Figure 1 shows participant flow through the trial. In the period of week 1 to week 5, 127 (male=46, female=81) out of 133 (95%) of Group A students completed the teleconsultation on EQClinic. In the second period (week 6~7), 166 out of 268 students from both Group A (44%, n =59) and Group B (79%, n =107) completed a facetoface consultation. During weeks 8 to week 11, 130 (male=62, female=68) out of 135 (96%) Group B students completed the teleconsultation using EQClinic. Lastly, 144 students from both Group A (n=109) and Group B (n= 35) completed a facetoface consultation during the period of week 12 and 13. " 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT 翿ow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "In total, 11 students (6 from Group A, 5 from Group B) did not complete the teleconsultation in this study."

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other ̅gures or tables demonstrating usage/dose/engagement.

2 3 4 5
subitem not at all important essential Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the ̅gure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study see Figure 1 14a) Dates de̅ning the periods of recruitment and followup Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study The student were not recruited. Having EQClinic consultation was compulsory for all students who registered in this course.
14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., signi̅cant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study No such events.
14b) Why the trial ended or was stopped (early) Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not in the ms, or brie翿y explain why the item is not applicable/relevant for your study We did not have a demographic table. Instead, we described the information in text.
"Participants were second year undergraduate medical students from an Australian medical school. All students were enrolled in a communication skills training course provided by the medical school. Prior to this study, they were not offered any training about tele consultations in the medical school. " "In the period of week 1 to week 5, 127 (male=46, female=81) out of 133 (95%) of Group A students completed the teleconsultation on EQClinic. " "In the period of week 1 to week 5, 127 (male=46, female=81) out of 133 (95%) of Group A students completed the teleconsultation on EQClinic. "

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Participants were second year undergraduate medical students from an Australian medical school." "In the period of week 1 to week 5, 127 (male=46, female=81) out of 133 (95%) of Group A students completed the teleconsultation on EQClinic. " "During weeks 8 to week 11, 130 (male=62, female=68) out of 135 (96%) Group B students completed the teleconsultation using EQClinic." 16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups 16-i) Report multiple "denominators" and provide de̅nitions Report multiple "denominators" and provide de̅nitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speci̅c pre-de̅ned time points of interest (in absolute and relative numbers per group). Always clearly de̅ne "use" of the intervention. Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study See Figure 1 16-ii) Primary analysis should be intent-to-treat Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Participants were second year undergraduate medical students from an Australian medical school." 17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% con̅dence interval) Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not applicable 17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational de̅nitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is de̅ned (e.g., timeout after idle time) [1] (report under item 6a).

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study No qualitative feedback was reported in this study.

22) Interpretation consistent with results, balancing bene̅ts and harms, and considering other relevant evidence
NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group 22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "EQClinic was incorporated into a medical communication skills teaching curriculum to provide students with additional practice opportunities with SPs. Importantly, students could review their nonverbal communication behaviours. We examined the effects of EQClinic on medical students' learning of communication skills evaluated via the students' assessment (SOCA) scores. Results showed that students who completed a tele consultation using EQClinic and reviewed the nonverbal behavior feedback showed higher SOCA scores in the first facetoface consultation. In addition, students accomplished higher SOCA scores in their second face toface consultation if they completed a teleconsultation between the two facetoface consultations. "

22-ii) Highlight unanswered new questions, suggest future research
Highlight unanswered new questions, suggest future research.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "However, based on the current findings, it remains unclear which of the learning components were most useful to enhancing students' learning. Moreover, although a single exposure to the EQClinic demonstrated a measurable improvement in student's medical communication skills scores, future studies will benefit from an examination of the appropriate 'dose' of EQClinic. This will help determine the necessary exposure needed to provide sustained improvement and generalizable communication skills training. Finally, the growth of collected student data by EQClinic will aid the refinement of rules and models using machine learning algorithms to indicate to students what nonverbal behavior is associated with positive or negative responses and feedback from SPs in their clinical teleconsultations." 20) Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "There are several limitations to our study that should be considered when interpreting these findings. Firstly, the absence of baseline measures limited our ability to observe change over time. Second, all the consultations conducted in this study were limited to a historytaking scenario. In reality, clinicians encounter many different scenarios. For example, when breaking bad news to patients, the clinician has to handle difficulties related to emotions. In addition, all the students in this study were second year medical students who had limited knowledge about communication skills. Future studies may explore whether EQClinic is also useful for senior medical students and professionals. An additional limitation is the relatively low number of students (30%) that completed all components of the study. While the sample was still appropriate for the statistical tests conducted, future investigations will benefit from exploring in greater detail the reason for student nonparticipation." 21) Generalisability (external validity, applicability) of the trial ̅ndings NPT: External validity of the trial ̅ndings according to the intervention, comparators, patients, and care providers or centers involved in the trial

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "Importantly, this study demonstrated that EQClinic was a useful and practical communication skills learning tool that is well suited to medical students within university settings."

21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting
Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study As in this study, EQClinic was cooperated in real teaching curriculum. The routine application setting should be similar to this study.

OTHER INFORMATION
23) Registration number and name of trial registry Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not applicable 24) Where the full trial protocol can be accessed, if available Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study not applicable 25) Sources of funding and other support (such as supply of drugs), role of funders Does your paper address CONSORT subitem 25? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "This project is funded by Brain and Mind Centre at The University of Sydney Australia and Australian Government. RAC is supported by Australian Research Council." X27) Con翿icts of Interest (not a CONSORT item) X27-i) State the relation of the study team towards the system being evaluated In addition to the usual declaration of interests (̅nancial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie翿y explain why the item is not applicable/relevant for your study "We have no conflict of interest to report. "

About the CONSORT EHEALTH checklist
As a result of using this checklist, did you make changes in your manuscript? * yes, major changes yes, minor changes no What were the most important changes you made as a result of using this checklist?
Title, and some detail points How much time did you spend on going through the checklist INCLUDING making changes in your manuscript *