Efficacy of an mHealth Intervention (BRAVE) to Promote Mental Wellness for American Indian and Alaska Native Teenagers and Young Adults: Randomized Controlled Trial

Background Culturally relevant interventions are needed to help American Indian and Alaska Native (AI/AN) teenagers and young adults navigate common risky situations involving family and friends, including drug and alcohol misuse, dating violence, and suicidality. Leveraging We R Native, a multimedia health resource for Native teenagers and young adults, staff of the Northwest Portland Area Indian Health Board designed the BRAVE intervention for Native youth. The program is delivered via SMS text messaging and includes role model videos, mental wellness strategies, links to culturally relevant resources, and social support from family and friends. Objective We aim to conduct a randomized controlled trial of the BRAVE intervention among AI/AN teenagers and young adults (aged 15-24 years) to assess its impact on their physical, mental, and spiritual health; their resilience and self-esteem; and their coping and help-seeking skills. Methods From October to December 2019, we recruited 2334 AI/AN teenagers and young adults nationwide via social media channels and SMS text messages and enrolled 1044 participants. AI/AN teenagers and young adults enrolled in the study received either BRAVE SMS text messages, designed to improve mental health, help-seeking skills, and cultural resilience, or 8 weeks of science, technology, engineering, and math (STEM) SMS text messages, designed to elevate and reaffirm Native voices in STEM and medicine and then received the BRAVE SMS text messages. The impacts of the BRAVE intervention were tested using linear mixed-effect models and linear regressions. Results A total of 833 AI/AN teenagers and young adults were included in the analysis. Individuals in the BRAVE and STEM arms showed significant positive trends over the course of the study for all outcomes, except cultural identity and help-seeking behavior. Mean scores were significantly different for health (P<.001), resilience (P<.001), negative coping (P=.03), positive coping (P<.001), self-efficacy (P=.02), and self-esteem (P<.001). Changes in help-seeking self-efficacy were significant in those exhibiting risky behaviors at baseline to exit (P=.01). Those who reported positive coping scores at baseline also reported better health on average; however, no difference was found in risky drug and alcohol use (P<.001). The number of participants who used SMS text messages to help themselves increased from 69.1% (427/618) at 3 months to 76% (381/501; P<.001) at 8 months. Similarly, the number of participants who used SMS text messages to help friends or family members increased from 22.4% (138/616) at 3 months to 54.6% (272/498) at 8 months. Conclusions This is the first national randomized controlled trial of a mobile health intervention among AI/AN teenagers and young adults to test the efficacy of a mental wellness intervention in relation to STEM career messages. This study provides new insights for supporting the next generation of AI/AN changemakers. Trial Registration ClinicalTrials.gov NCT04979481; https://clinicaltrials.gov/ct2/show/NCT04979481

Is this a full powered effectiveness trial or a pilot/feasibility trial? * Manuscript tracking number * If this is a JMIR submission, please provide the manuscript tracking number under "other" (The ms tracking number can be found in the submission acknowledgement email, or when you login as author in JMIR. If the paper is already published in JMIR, then the ms tracking number is the four-digit number at the end of the DOI, to be found at the bottom of each published article in JMIR) TITLE AND ABSTRACT 1a) TITLE: Identification as a randomized trial in the title

1a-i) Identify the mode of delivery in the title
Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if offline products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.

Clear selection
Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "mHealth Intervention" 1a-ii) Non-web-based components or important co-interventions in title Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").

Clear selection
Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "text messaging" 1a-iii) Primary condition or target group in the title Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial Clear selection subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "American Indian and Alaska Native Teens and Young Adults"

1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT
Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Keep in mind the needs of systematic reviewers and indexers by including important synonyms. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it)

Clear selection
Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study This item is not relevant to this study. 1b-ii) Level of human involvement in the METHODS section of the ABSTRACT Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it)

Clear selection
Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable to this study 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional offline trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Clear selection Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Outcomes were assessed using standardized scales and Likert-type ratings where answers were converted to numeric scales and averages and standard deviations were calculated for each outcome. " 1b-iv) RESULTS section in abstract must contain use data Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it)

Clear selection
Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Mean scores were signi cantly different for health (P<.001), resilience (P<.001), negative coping (P=.027), positive coping (P<.001), self-e cacy (P=.021), and selfesteem (P<.001). Changes in help-seeking self-e cacy were signi cant for those exhibiting risky behavior at baseline to exit (P=.01). Those who reported positive coping scores at baseline also reported better health on average, but no difference in risky drug and alcohol use (P<.001). Does your paper address CONSORT subitem 2b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Your answer 3b) Important changes to methods after trial commencement (such as eligibility criteria), with reasons Does your paper address CONSORT subitem 3a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The aim of this study was to test the e cacy of the BRAVE intervention and test the following hypotheses: • Hypothesis 1: Primary Outcome Measure AI/AN teens and young adults in the intervention arm will report better outcomes than participants in the control arm.

•
Hypothesis 2: Secondary Outcome Measure AI/AN teens and young adults who report better physical, mental, and spiritual health will report greater cultural resilience, identity, and cultural pride. • Hypothesis 3: Secondary Outcome Measure AI/AN teens and young adults who report greater use of coping and mental wellness strategies will report better physical, mental, and spiritual health and less risky drug and alcohol misuse.

•
Hypothesis 4: Secondary Outcome Measure Changes in help-seeking self-e cacy will be greatest among male participants, and young adults over 21 years old." Does your paper address CONSORT subitem 3b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable, nothing was changed

Clear selection
Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Not applicable
Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To enroll in the study, participants were required to complete a pre-survey. Those who met the eligibility requirements and completed the pre-survey were randomized into the study (n=1,044) using a 1:1 allocation ratio. mHealth used Excel software to randomize participants into each study arm, with 8 total waves of enrollment from October to December 2019"

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clarified.

Clear selection
Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Interested teens and young adults were asked to text the keyword BRAVE to 97779, which triggered a series of eligibility and consent text messages. Over 2,330 AI/AN teens and young adults texted BRAVE to learn more about the study." Texting showed computer/internet literacy 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. offline), e.g., from an open access website or from a clinic, and clarify if this was a purely webbased trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email confirmation, phone calls) were used to detect/prevent these.

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.

Clear selection
Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Data were collected using online questionnaires at baseline, 3, 5, and 8 months. In appreciation for their time, participants received a $10 amazon gift code for each survey they completed, up to $40 per person. " , a tool designed to measure mental health stigma. Additional questions collected included standard demographic information, resilience measures, self-esteem, positive and negative coping skills, help-seeking self-e cacy, mental health, and cultural pride. Data from the surveys were collected, stored, and maintained by mHealth."

4b-i) Report if outcomes were (self-)assessed through online questionnaires
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.

Clear selection
Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "All participants were required to complete the baseline survey at the time of enrollment (pre-survey). Enrollees were asked to complete the same survey after the rst set of messages (post-survey #1). At the end of the intervention period, participants crossed over to receive the second set of messages and were asked to complete the survey a third time (post-survey #2)."

Clear selection
Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

5-i) Mention names, credential, affiliations of the developers, sponsors, and owners
Mention names, credential, affiliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Conflict of interest" section or mentioned elsewhere in the manuscript). Does your paper address subitem 5-i?

Clear selection
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study No con icts of interest to report

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.

Clear selection
Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "To reach AI/AN teens and young adults with culturally-relevant health messages, the NPAIHB built We R Native -a holistic, multimedia health resource that reaches over 5,000 viewers per day across its messaging channels. We R Native was designed using a youth-centered approach, where youth were actively involved in selecting, writing, and designing its content. The service includes a website (www.weRnative.org), an Ask Auntie Q&A service, a text message service (text NATIVE to 97779), a YouTube channel, and social media accounts (Facebook, Instagram, Twitter). In 2019, the website had over 200,000 unique users. On average, We R Native's website receives over 28,600 pageviews per month; with 74% visiting from a mobile device. We R Native's text messaging service (MobileCommons) has over 5,850 subscribers. We R Native's Facebook page has over 51,000 followers; with 77% accessing via a mobile device. Given the widespread use of mobile phones by Native youth, mHealth interventions provide a promising tool to reach AI/AN youth, who live across vast geographies in urban and rural communities.
We R Native regularly engages users to make sure its tools and messages are relevant, timely, and culturally appropriate. In 2019, the team interviewed 13 AI/AN teens and young adults who regularly viewed We R Native's channels (website, text message, or social media). Interview participants shared numerous ways We R Native's messages have improved their mental health, cultural connectedness, sense of self-worth, and access to health resources -both for themselves and their loved ones. When asked to re ect on their own mental health concerns, grief and depression were the most common topics discussed by participants, followed by stress and mental wellness skills (e.g. coping mechanisms).
Interviews conducted by the NPAIHB with AI/AN teens and young adults informed the design of the BRAVE intervention, focusing on common coping strategies (i.e. alcohol and drug misuse), preferred wellness strategies (i.e. goal setting and selfcare) and help-seeking skills (i.e. reaching out for help for themselves or a friend), and related protective factors, including cultural resilience, identity, and cultural pride. "

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).

Clear selection
Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable. Does your paper address subitem 5-iv?

Clear selection
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The study team discontinued communication and asked participants to complete the nal survey 90 days later (post-survey #3)." 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing flowcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientific reporting.

Clear selection
Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.

Clear selection
Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "BRAVE is available onlinehttp://www.npaihb.org/brave/" and "Future research may address the recommendations offered by study participants to improve the campaigns for future users, and assess the uptake of the service outside the study setting. Native youth can now text BRAVE to 97779 or STEM to 97779 to receive the text message sequence on their own. "

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of specific group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.

Clear selection
Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability). Does your paper address subitem 5-x?

Clear selection
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

Clear selection
Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "• Hypothesis 1: Primary Outcome Measure AI/AN teens and young adults in the intervention arm will report better outcomes than participants in the control arm.
• Hypothesis 2: Secondary Outcome Measure AI/AN teens and young adults who report better physical, mental, and spiritual health will report greater cultural resilience, identity, and cultural pride.
• Hypothesis 3: Secondary Outcome Measure AI/AN teens and young adults who report greater use of coping and mental wellness strategies will report better physical, mental, and spiritual health and less risky drug and alcohol misuse.

•
Hypothesis 4: Secondary Outcome Measure Changes in help-seeking self-e cacy will be greatest among male participants, and young adults over 21 years old." 6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9].

Clear selection
Does your paper address subitem 6a-i?

Clear selection
Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text Not applicable 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).
Clear selection 6b) Any changes to trial outcomes after the trial commenced, with reasons 7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text "Participants were also asked to provide open ended feedback at the end of the survey on ways to improve the program for others. The most frequent suggestions were to offer more resources, followed by preferred changes to the message delivery schedule. At ve months and eight months, participants provided fewer recommendations for change. The team plans to conduct a qualitative analysis of user feedback to inform future improvements. " Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size
Describe whether and how expected attrition was taken into account when calculating the sample size.
Clear selection 7b) When applicable, explanation of any interim analyses and stopping guidelines 8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Their exposure to We R Native may have in uenced their responses and participation. The initial goal was to enroll 125 participants from the 12 Indian Health Service geographic regions (n=1,500) to explore regional differences in the e cacy of BRAVE. However, BRAVE enrollment was 1,030, an insu cient sample size to explore regional differences. Of the 1,030 enrolled, only 833 were included in the analysis to improve data quality. " Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable 8b) Type of randomisation; details of any restriction (such as blocking and block size) 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Those who met the eligibility requirements and completed the pre-survey were randomized into the study (n=1,044) using a 1:1 allocation ratio. mHealth used Excel software to randomize participants into each study arm, with 8 total waves of enrollment from October to December 2019. " Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Not applicable
Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable 11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).

Clear selection
Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".

Clear selection
Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "AI/AN teens and young adults randomized into the intervention arm were exposed immediately to the BRAVE campaign, containing 8 weeks of text messages with three messages per week. Participants in the control arm received messages designed to elevate and re-a rm Native voices in science, technology, engineering, math and medicine (STEM). The STEM messages were similarly delivered three times per week for 8 weeks. Both arms were designed to include a similar number of messages per week, with a combination of information, role model videos, images, and opportunities for re ection and engagement with the campaign (i.e. reply for more info, provide a Q&A response, click links to access resources). " 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Not applicable
Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study The study team explored carryover effects between BRAVE and STEM arms between study periods. Signi cant differences between BRAVE and STEM arms were compared at baseline, crossover, and follow-up.
Two analyses were conducted to explore differences between the primary outcome measures. First, a two-sided paired t-test was used to assess the treatment effect, comparing BRAVE to STEM arms using the difference in outcome score for each measure. Second, a linear mixed-effect model (using a random effect for subject) was carried out to test for change over time. Both treatment effects and changes in outcomes were considered signi cant at the p<0.05 level. Secondary outcome measures were analyzed using a series of statistical tests. Multiple linear regressions were used to assess differences in baseline and follow-up survey scores. Two models were used to assess associations at baseline and changes in scores from baseline to study completion. A linear mixed-effect model (with random effect for subject at 3, 5, and 8 months) was used to assess changes over time." 12a-i) Imputation techniques to deal with attrition / missing values Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).

Clear selection
Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Not applicable
Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

RESULTS
Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Interested teens and young adults were asked to text the keyword BRAVE to 97779, which triggered a series of eligibility and consent text messages. Over 2,330 AI/AN teens and young adults texted BRAVE to learn more about the study."

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline)

Clear selection
Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable 13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "All teens who completed 100% of the surveys, were aged 15-24 at the time of the pre-survey, and identi ed at AI/AN were included in this report. A total of 833 Native teens and young adults were included in this analysis." Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT flow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The washout period between study arms was one week. In total, 86 participants opted out of the study during the rst arm, and 43 opted out after the crossover, resulting in a 87% retention rate" subitem not at all important 1 2 3 4 5 essential 14a) Dates defining the periods of recruitment and follow-up

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other figures or tables demonstrating usage/dose/engagement.

Clear selection
Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the figure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study diagram included, more information not applicable Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "From September to December 2019, NPAIHB recruited AI/AN teens and young adults via We R Native's social media channels (Facebook, text message, Instagram 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., significant changes in Internet resources available or "changes in computer hardware or Internet delivery resources"

Clear selection
Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

No applicable
Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups
Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study " Table 3. Summary of AI/AN teens and young adults who participated in the study at baseline, 3, 5, 8-months (N=833)."

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.

Clear selection
Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable 16-i) Report multiple "denominators" and provide definitions Report multiple "denominators" and provide definitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at specific pre-defined time points of interest (in absolute and relative numbers per group). Always clearly define "use" of the intervention.

Clear selection
Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

16-ii) Primary analysis should be intent-to-treat
17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval) Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The study design was a RCT with crossover. The primary and secondary outcomes did not show a signi cant difference between time points and statistical analyses revealed no carryover effects. To test the study hypotheses and explore primary and secondary outcomes of interest, the research team compared subjects to themselves and explored differences from pre-to post-survey throughout the intervention." Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "A review of 3, 5, and 8-month mean scores when controlling for treatment order showed a positive and upward trend for nearly all constructs measured, Table 4. In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational definitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is defined (e.g., timeout after idle time) [1] (report under item 6a).

Clear selection
Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Not applicable
Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).

Clear selection
Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study

Not applicable
Does your paper address CONSORT subitem 19? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "Participants were also asked to provide open ended feedback at the end of the survey on ways to improve the program for others. The most frequent suggestions were to offer more resources, followed by preferred changes to the message delivery schedule. At ve months and eight months, participants provided fewer recommendations for change. The team plans to conduct a qualitative analysis of user feedback to inform future improvements. " 22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).

Clear selection
Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "This study investigated the e cacy of a theoretically informed, culturally relevant mHealth intervention for AI/AN teens and young adults. Results did not fully support the proposed hypotheses; there were no measurable treatment effects between the BRAVE or STEM study arms. However, most survey measures improved over time for both groups -including notable improvements in mental health, resilience, coping skills, and better self-esteem -despite delivery in the midst of the global COVID-19 pandemic. There were also signi cant reductions in reported A&D misuse.
At baseline, physical, mental and spiritual health measures were positively associated with cultural identity. This suggests that resilience, identity and cultural pride may be associated with better health for AI/AN teens and young adults.
These ndings are consistent with previous studies that report that AI/AN identity may be protective against a myriad of risk behaviors, including A & D misuse [22].
Second, participants who reported better coping strategies at baseline (i.e. having healthy outlets they use when they're stressed, or taking active steps to improve their mental health and nd balance) also reported better health on average, but no difference in risky behavior than their peers. Additionally, participants who reported a positive change in coping strategies over the course of the study, were also more likely to report a positive change in physical, mental and spiritual health during that

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events. Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "With numerous strengths, there were also limitations that should be noted. First, all data are based on self-report. The sample was recruited largely through We R Native, which is a multimedia health and wellness resource. Their exposure to We R Native may have in uenced their responses and participation. The initial goal was to enroll 125 participants from the 12 Indian Health Service geographic regions (n=1,500) to explore regional differences in the e cacy of BRAVE.

Clear selection
However, BRAVE enrollment was 1,030, an insu cient sample size to explore regional differences. Of the 1,030 enrolled, only 833 were included in the analysis to improve data quality. The washout period of one week between intervention and control arms may not have been su cient to observe an effect in treatment and control conditions. Finally, most participants reported high or favorable scores for survey measures at baseline. This may have contributed to the lack of treatment effect between the two arms. "

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 21-i?

Clear selection
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "BRAVE is the rst study to examine the e cacy of an mHealth intervention among AI/AN teens and young adults in the United States. A major strength of the study is the RCT design, where random allocation allowed for a clean comparison between intervention and control groups. The biggest strength of BRAVE is that it was developed with input from AI/AN teens and young adults, with additional input from tribal health educators and topical experts. BRAVE's focus on culturally-relevant images, language, and the use of role model videos demonstrated a respect for diversity, values, and translation of health communication messaging that AI/AN youth could identify with."

21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting
Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.

Clear selection
Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study Not applicable Does your paper address CONSORT subitem 25? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "The project was also supported by the X27-i) State the relation of the study team towards the system being evaluated In addition to the usual declaration of interests (financial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention. About the CONSORT EHEALTH checklist yes, major changes yes, minor changes no Does your paper address subitem X27-i?

Clear selection
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briefly explain why the item is not applicable/relevant for your study "None declared " As a result of using this checklist, did you make changes in your manuscript? * What were the most important changes you made as a result of using this checklist?

NA
How much time did you spend on going through the checklist INCLUDING making changes in your manuscript *