Published on in Vol 6, No 4 (2018): Oct-Dec

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/9944, first published .
Potential of an Interactive Drug Prevention Mobile Phone App (Once Upon a High): Questionnaire Study Among Students

Potential of an Interactive Drug Prevention Mobile Phone App (Once Upon a High): Questionnaire Study Among Students

Potential of an Interactive Drug Prevention Mobile Phone App (Once Upon a High): Questionnaire Study Among Students

Original Paper

1Department of Addiction, Semmelweis University Faculty of Health Sciences, Budapest, Hungary

2Drug Outpatient Centre, Nyírő Gyula National Institute of Psychiatry and Addictions, Budapest, Hungary

3Budapest Center for Vocational Education and Training in Engineering, Budapest, Hungary

4Creative Technology Lab, Moholy-Nagy University of Art and Design, Budapest, Hungary

5Institute of Psychology, Eötvös Loránd University, Budapest, Hungary

Corresponding Author:

Máté Kapitány-Fövény, PhD

Department of Addiction

Semmelweis University Faculty of Health Sciences

Vas Street 17

Budapest, 1088

Hungary

Phone: 36 20 5221850

Email: m.gabrilovics@gmail.com


Background: In recent years, drug prevention networks and drug education programs have started using Web-based or mobile phone apps as novel prevention tools, testing their efficacy compared with face-to-face prevention.

Objective: The aim of this study was to assess the potential of an interactive app called Once Upon a High (VoltEgySzer).

Methods: The app approaches drug prevention from 6 different aspects, and it addresses youngsters with 6 different modules: (1) interactive comics/cartoons, telling stories of recovery; (2) quiz game; (3) roleplay game; (4) introduction of psychoactive drugs; (5) information on the somatic and psychological effects of psychoactive substances; (6) list of available treatment units, rehabs, and self-support groups in Hungary. Students of 2 vocational schools and 2 high schools filled out a questionnaire at a baseline (T0) and a 2-month follow-up (T1) data collection session. Students of 1 vocational school and 1 high school downloaded the Once Upon a High app (app group), whereas students from the other vocational school and high school did not (nonapp group). The time points of T0 and T1 questionnaires contained demographic variables, items with regard to substance use characteristics for both legal and illegal substances, including novel psychoactive substance, exercise habits, knowledge about psychoactive substances, attitudes toward substance users and validated instruments measuring the severity of tobacco (Fagerström Test for Nicotine Dependence), alcohol (Alcohol Use Disorder Identification Test), cannabis (Cannabis Abuse Screening Test), and synthetic cannabinoid consumption. Beliefs about substance use (Beliefs About Substance Abuse) and perceived self-efficacy (General Perceived Self-Efficacy) were also measured. At T1, members of the app group provided additional evaluation of the app.

Results: There were 386 students who participated in the T0 session. After dropout, 246 students took part in T1 data collection procedure. Alcohol was the most frequently consumed psychoactive substance (334/364, 91.8% lifetime use), followed by tobacco (252/386, 65.3%, lifetime use) and cannabis (43/323, 13.3% lifetime use). Decreased self-efficacy (beta=−.29, P=.04) and increased daily physical exercise frequencies (beta=.04, P<.001) predicted higher frequencies of past month energy drink consumption, whereas elevated past month alcohol consumption was mainly predicted by a decrease in negative attitudes toward substance users (beta=−.13, P=.04) in the regression models. Once Upon a High was found to be effective only in reducing energy drink consumption (beta=−1.13, P=.04) after controlling for design effect, whereas perceived utility of the app showed correlation with a decreasing alcohol use (rS(44)=.32, P=.03). The roleplay module of the app was found to be the most preferred aspect of the app by the respondents.

Conclusions: The Once Upon a High app can be a useful tool to assist preventive intervention programs by increasing knowledge and self-efficacy; however, its efficacy in reducing or preventing substance use needs to be improved and further studied. Additional potential impacts of the app need further testing.

JMIR Serious Games 2018;6(4):e19

doi:10.2196/games.9944

Keywords



Background

The era of contemporary substance use scene is also an era in which Web-based communication influences our daily lives to an increasing extent. Spending an increasing amount of time in virtual spaces is a phenomenon that mostly affects the lifestyle of adolescents [1]. Virtual spaces—such as Web-based fora, blogs, Web markets—serve a significant role as one of the primary sources of gathering either classic or novel psychoactive substance (NPS)-related information as well. The Web-based space not only provides information and means of communication, but Web-based marketing is also responsible for increasing the availability of NPS [2].

Not only the drug market but prevention itself also transformed as one of the many consequences of the rise of information technologies. In recent years, drug prevention networks and drug education programs have started using Web-based or mobile phone apps as novel prevention tools [3]. Such virtual methods may provide an opportunity to access individuals who otherwise might not seek or receive professional help. The anonymity and accessibility of prevention apps often lead to increased self-disclosure with regard to sensitive subjects such as substance consumption [4]. The effectiveness of Web-based interventions providing personalized feedback and mobile phone apps in reducing the intake of psychoactive substances is already supported by empirical evidence [5,6]. According to the findings of previous research, such programs may result in significant decrease in alcohol consumption among heavy drinkers [7], in prolonged smoking cessation as a supplement to nicotine patch therapy [8], or in decrease of marijuana use among college students who show a high level of contemplation to be abstinent [9]. However, there are some contradictory results as well with regard to the efficacy of such programs when implemented in the adolescent population. On the basis of the systematic review of Majeed-Ariss and colleagues [10], some apps may be considered feasible health interventions that may increase self-management of chronic health conditions, but on the other hand, there are several apps available on the app market without any evidence-based background. A Web-based prevention program for ecstasy and NPS [11] was reported to be efficient in reducing adolescents’ intentions to use NPS; it increased knowledge about both ecstasy and NPS. However, changes in lifetime use of ecstasy or NPS did not differ significantly between the intervention and control conditions.

The result that tailored and interactive sites and apps are more effective than static ones [12] is also essential with respect to our project, although this principle applies not only to apps and websites but also to prevention itself. Interactivity may also increase the subjective feeling of self-efficacy and self-directedness, which might improve the commitment to participate in a prevention program. Self-directedness itself was found to be a relevant protective factor against opiate addiction [13], alcohol addiction [14], cannabis addiction [15], and smoking [16].

As opposed to personal, face-to-face prevention, preventive interventions implemented in the virtual space have distinct advantages as they are available any time, are more cost effective in the long run, are better in providing tailored feedback, and are able to reach more members of the target population [17]. Previous studies have also indicated that mobile phone apps combine the benefits of Web-based and computerized preventive interventions as they provide both interactive and static contents with and without an internet connection [18], even if some apps cannot operate properly offline. Nevertheless, updates concerning eventual content change are usually assured by automatic version refreshments in every case.

With respect to the methodological concept of our project, we followed the concept of gamification. Gamification—when utilized in prevention—is usually defined as the mixture of game design elements and traditional prevention techniques in a nongame context [19]. The aim of gamification is to increase the engagement and motivation of the target group while providing a useful method in supporting learning and problem solving. It is not essential to use any digital technology in gamification, but smartphones make the implementation easier and the outreach wider. With regard to the efficacy of gamification in preventing substance use among adolescents, Boendermaker and colleagues [20] found that game elements (authors label them as serious games) can help to motivate youngsters to do a cognitive bias modification training, which might reduce substance consumption. However, research is still lacking to draw further conclusions about the efficacy of gamification.

Objective

The aim of this study was to assess the effectiveness of a mobile phone prevention app, titled Once Upon a High (the original name of the app is VoltEgySzer in Hungarian). We aimed to target adolescents in an age range that may indicate the existing experiences with substance use but without clinically relevant problems. Therefore, Once Upon a High was tested as a novel tool with embedded game elements (ie, as a form of gamification) for mainly the purpose of secondary prevention. Being such an instrument, we expected this app to be effective—among other goals—in reducing or at least maintaining substance use frequencies. The app, besides being a contemporary gadget of providing information on the risks of substance use, was expected to enhance or affect certain skills as well, which might be considered as protective factors against the increasing severity of substance consumption and are as follows: (1) self-efficacy is usually seen as a protective resource in prevention or even treatment of substance use disorders (SUD) [21] and as an important determinant of health behavior in general [22]; (2) increasing physical exercise is similarly recommended as a potential tool to be utilized in substance use prevention for the youth [23,24] or as a beneficial adjunctive treatment of SUD [25], which may lead to elevated abstinence rates or the ease of withdrawal symptoms [26].

In the study, besides the exploratory analysis of the app’s utility, the following hypotheses were tested:

Once Upon a High can be effective in:

H1: Increasing knowledge about the risks of psychoactive substances
H2: Enhancing perceived self-efficacy
H3: Decreasing or maintaining the frequency of current substance use
H4: Decreasing negative attitudes toward substance users
H5: Increasing the frequency of physical exercise

Although H2, H4, and H5 might also be related to primary prevention, we still considered self-efficacy, physical exercise, and negative attitudes toward substance users as potential protective factors against the exacerbation of substance use among active users; and as such, preferable outcomes of secondary prevention too, potentially leading to decreased substance use frequencies. We further hypothesized that increased knowledge (H1), perceived self-efficacy (H2), physical exercise (H5), and decreased negative attitudes toward substance users (H4) might predict decreased frequencies of substance use (H3).


Once Upon a High: Introducing the App

Once Upon a High is an interactive drug prevention app targeting the youth and drawing attention to the importance of prevention in a novel way. The app approaches drug prevention from 6 different aspects, and it addresses youngsters with 6 different modules: (1) interactive comics/cartoons, telling stories of recovery (see Animated Comics section); (2) quiz game (see Quiz Game section); (3) roleplay game (see What If? section); (4) introduction of psychoactive drugs (see Substance Store section); (5) information on the somatic and psychological effects of psychoactive substances (see Trans-formation section); (6) list of available treatment units, rehabs, and self-support groups in Hungary (see Where to go? section). The user may choose to use any of the modules from the initial menu. The app unites the experience of usual Web education and computer games. The applied genres—cartoons, videos, and animation—are anticipated to be attractive to the youth by their nature. The app provides anonymity for its users.

The app was designed and developed in a joint project of Nyírő Gyula Hospital, National Institute of Psychiatry and Addictions and the techLab of the Moholy-Nagy University of Art and Design, Budapest. The development of the app was funded from a national tender (invited by the Hungarian Government’s Ministry of Human Capacities) supporting the establishment of novel drug prevention programs. More information on the technical details are presented in the Multimedia Appendix 1 as well as on the visual rendering of the app. To increase risk perception, yet to avoid overestimation of the dangers of specific psychoactive substances (ie, avoiding autotelic deterrence), the literature basis for each module contained current papers providing high-quality evidence (ie, mainly systematic reviews and meta-analyses or randomized controlled trials). The literature basis for each module is presented under the detailed characteristics of the app.

Detailed Characteristics and Aims of the Modules

Animated Comics: Interactive Recovery Stories

The first module comprises altogether 4 recovery stories, of which there are stories of 2 males (Adam, the beggar; Adam, the prince) and 2 females (Eve, the beggar; Eve, the princess). These 4 comic-style animated tales (Figure 1) introduce common risk factors of adolescent psychoactive substance use (eg, peer pressure, peer recognition for substance use involvement, social isolation, a dysfunctional family background indicated by symptoms such as familial substance use and either sexual, physical, or emotional victimization) based on the findings of former studies [27,28], as well as potential way outs by presenting 4 types of treatments: (1) art therapy, (2) individual psychotherapy, (3) group therapy including self-support groups and as a combination with family counseling, and (4) animal-assisted therapy. The selection of these therapeutic interventions was based on former findings with respect to their efficacy in addiction treatment. Art therapy is commonly used throughout the process of detoxification and rehabilitation, including 12-step programs and self-support groups [29]. The effectiveness of individual psychotherapy and primarily cognitive behavioral therapy in the treatment of SUD was supported by the results of numerous meta-analyses [30,31]. Group counseling and family therapies were highlighted as the most effective interventions in a study comparing the effectiveness of various treatment methods for adolescent substance use [32]. A systematic review and meta-analysis of randomized controlled trials further confirmed the efficacy of animal-assisted therapy in the treatment of SUD [33]. The interactivity of this module means that the dialogues are triggered by the user, and turns are indicated by a slight movement of the main characters. The module also aims to display different socioeconomic backgrounds of substance users by telling the stories of both socially marginalized youth and youngsters coming from high-income families.

Quiz Game

The second module, a quiz-game, contains a database of 50 potential questions, of which the app randomly generates 15 questions every time the user plays the game. The quiz game is structured by 3 difficulty levels (easy, moderate, and hard). Difficulty levels were determined by assigning an adjustment weight to the questions based on the results of 98 high school students who gave either correct or wrong answers to all the 50 questions. Many of the quiz questions reflect common misbeliefs about the prevalence and risks of substance use (eg, by asking: “What percentage of high school students in your country have tried an illegal substance at least once in their lifetime?”). As a trend, adolescents often overestimate peer substance use [34], which might legitimize their own substance consumption.

Review of the current literature resulted in the following relevant areas, addressed in the quiz game: the topics of cannabis-induced psychosis [35], effects of energy drink consumption [36], dangers of adolescent alcohol use [37], peer group influence on adolescent smoking [38], or harms of NPS [39]. Less emphasis was placed on the risks associated with the use of substances with low prevalence among adolescents, such as heroin or other opioids, based on the findings of the most recent ESPAD study [40]. To every question, there are 3 potential answers (A, B, and C solutions), of which only one is correct. After every round, the user receives a feedback on his or her answers, which contains relevant citations from the scientific literature as well.

The quiz game has 2 main goals. First, to provide well-established knowledge of the epidemiology and risks of substance use, with respect to both legal and formerly banned “classic” substances (such as alcohol, tobacco, ecstasy, amphetamines, or cannabis) and NPS (in this module, mainly synthetic cannabinoids and synthetic cathinones). Second, the module aims to promote health-conscious behavior. Those users who are able to achieve a 100% quiz result are offered free admission to gyms, dance schools, laser-tag, and paintball facilities or indoor climbing centers. As such, this module might increase physical exercise frequencies as a potential benefit and, therefore, its application was considered to test our H5 hypothesis.

What If? Roleplay Game and Dialogue System

The third module titled “What if?” is an interactive dialogue system, within the confines of a roleplay situation: What if one of your close acquaintances started to use synthetic cannabinoids? What would you suggest? How would you help? During the roleplay game, the user communicates with his or her assumed acquaintance by following a structured dialogue system, in which the virtual “talk partner” makes a specific statement, whereas the user selects a response from 2 optional sentences offered (see Figure 2). The main task within the game is to persuade the virtual acquaintance to change his lifestyle by reducing his substance use, seeking professional help, or starting drug-free recreational activities. Whenever the user selects the better option, the portrayed room gets brighter and brighter and vice versa, in case of a worse option selection, the room goes darker and darker. If the user is able to convince the virtual partner to make a positive change in his life, the game ends positively. However, in case of bad choices during the dialogue (eg, by being too impatient, hypocritical, or arrogant), the game ends with a game over. In this case, users receive a feedback on how to improve their communication skills and strategy next time.

The roleplay game, therefore, aims to improve communication skills, decrease stigma, and negative attitudes against people (see H4) who use psychoactive substances and to increase willingness to help peers with substance use problems. As such it targets adolescents with or without any former substance use experiences. Communication style of the virtual partner was based mainly on clinical experiences with substance users as well as the feedback of the co-design group (see Appendix for details). The assumption that peer support might be of utmost relevance was supported by literature evidence as adolescents tend to prefer informal sources of help (eg, friends) rather than professional support [41]. Finally, the risks of synthetic cannabinoid use presented in this module were also collected from published data [42].

Figure 1. Visual appearance of the animated comics module.
View this figure
Figure 2. Visual appearance of the What if? roleplay game module.
View this figure
Figure 3. Visual appearance of the Substance-store module.
View this figure
Figure 4. Visual appearance of the Trans-formation module.
View this figure
Substance Store: Information About Psychoactive Drugs

The fourth module titled “Substance-store” (Figure 3) provides information on “classic” substances (cannabis, cocaine, ecstasy and amphetamines, lysergic acid diethylamide [LSD], and heroin) and NPS or recently popularized substances (ketamine, gamma-hydroxybutyrate [GHB], Salvia divinorum, synthetic cannabinoids, and cathinones). Users are offered knowledge of the brief history of the selected psychoactive substance, information on the psychoactive compounds, both desired and unwanted or adverse effects of the substance, as well as the dangers of potential overdose. For each substance, an image helps to recognize the common distribution form. As such, this module’s main goal is to educate the users and provide information. Cultural history of substance use and the desired and adverse effects of specific substances were collected from the literature [43-51].

Trans-Formation: Somatic and Psychological Effects of Psychoactive Substances

“Trans-formation” is closely related to the fourth module (Substance store). In addition to that one, this fifth module presents substance-specific somatic and psychological effects in a longitudinal aspect by offering a selection between immediate, short-term, and long-term effects of specific substances. As impulsivity—and especially sensation seeking—usually rise during adolescence [52], for this age group, it is thought to be more relevant to focus on the immediate and short-term consequences of substance use and not just the long-term risks of recurrent psychoactive substance consumption. All the presented somatic and psychological effects are based on the findings of former studies [42,53-55]. These effects are indicated by the red dots located on a picture of a schematic body with the option to touch a dot on the screen (see Figure 4) and, thus, access detailed description of the effects. Here too, we relied on visual clues and active exploration by the user as learning methodology.

Where to Go? List of Available Treatment Units

The aim of the final module was to create a national register of the available treatment units (both inpatient and outpatient), rehabs, and self-support groups. Thus, the purpose of this module is to help youngsters who are already at-risk substance users in finding nationwide professional help, as close to their place of residence as possible. By overviewing the existing national registries and the websites of self-support groups, we created a searchable map with the contact details of each of the treatment units and meeting points. As a result, altogether, more than 300 locations were registered.

Sample and Procedure

The sample of the study consisted of 386 students of 2 vocational schools and 2 high schools from Budapest, Hungary, to analyze outcome variability by school type as well. To examine the impact of age as a covariate on the outcome measures, 9th (approximately 14- to 15-years-olds), 10th (approximately 15- to 16-years-olds), and 11th (approximately 16- to 17-years-olds) graders participated in the study from each school. The age range was 14 to 18 years. Permission from the schools was obtained as well as from the parents in cases where respondents were younger than 18 years. Written informed consents were asked in every case. Our goal was to assess participants in a natural setting, that is, in their schools, with the involvement of those professionals (teachers and school psychologists) who are involved in the everyday life of these students, to model an as realistic arrangement as possible. Randomization was performed at the school level, where participants were nested in classes. Altogether 4 groups were assessed at 2 different time points.

Students of 1 vocational (VS1) and 1 high school (HS1) downloaded the Once Upon a High app, whereas students from the other vocational (VS2) and high school (HS2) did not. Those students who downloaded the app were, therefore, part of the group hereunder referred to as “app group,” whereas those who did not download the app were members of the group referred to as “nonapp group.” We wanted to avoid assigning students to both app and nonapp groups within the same schools to prevent information sharing between study groups. Schools were selected with the help and involvement of 2 school psychologists who were also involved in monitoring the data collection procedure. Therefore, a nonprobability, convenience sampling method was used. Homeroom teachers were asked to help in organizing occasions of data collection. The app itself had an option to provide further information about its use via a website. Thus, detailed psychoeducation with respect to the use of the app was not necessary. However, homeroom teachers and those colleagues who were responsible for face-to-face data collection answered any questions that might have arisen throughout the study. We additionally provided an email address to receive and answer any incoming questions. We tried to motivate students to participate (and maintain their participation) by ensuring the opportunity of winning recreational activities (ie, free admission to gyms, dance schools, laser-tag and paintball facilities, or indoor climbing centers) in case of a flawless result in the quiz-game module.

As data were collected in 2 phases (baseline and 2-month follow-up data collection), participants received a unique identifier (UID) that helped the pairing of baseline (T0) and follow-up (T1) data. UID was generated by a similar algorithm of treatment demand indicator and consisted of the following 6 characters: (1) the third letter of the participant’s surname, (2) the second letter of the participant’s first name (3) last number of the participant’s birth month, (4) last number of the participant’s birth day, (5) third letter of the maiden surname of the participant’s mother, and (6) second letter of the maiden first name of the participant’s mother. This method was found to be more effective than allowing the students to choose a UID for themselves as they were expected to forget that during the 2 months between the 2 dates of data collection.

Measures

The baseline and follow-up questionnaire was filled by all the participants who remained in the study. Members of the app group responded to additional questions during the follow-up measurement with respect to their experiences with the app. Participants were asked to provide an evaluation on (1) the utility and (2) subjective preference of distinct modules, as well as (3) general impression about the app using a 5-point Likert scale (from 1=not at all, to 5=absolutely). We computed 2 variables (“app preference” and “app usefulness”) as a total score of the evaluative response categories with respect to the distinct modules’ preference and the app’s perceived utility.

The baseline questionnaire contained demographic variables, questions about socioeconomic status, former experiences with other prevention programs, and the family history of either alcohol, tobacco, or illicit substance use. These items were not included in the follow-up questionnaire. Both the baseline and follow-up questionnaire comprised questions with respect to the respondents’ physical exercise and sport habits (including sport types, monthly and daily exercise frequencies, measured in minutes/day), psychoactive substance use experiences (including lifetime, last year, and last month frequencies of the consumption of both legal and illegal “classic” and NPS-type drugs). NPS-type substances included synthetic cathinones (mephedrone, methylenedioxypyrovalerone [MDPV], pentedrone), synthetic cannabinoids, and GHB, as the use of these NPS was found to be relevant among Hungarian school-aged respondents in the ESPAD study [40].

With respect to the measurement of psychoactive substance use, the questionnaire further contained validated instruments such as the 6-item Fagerström Test for Nicotine Dependence (FTND) [56,57], the 10-item Alcohol Use Disorder Identification Test (AUDIT) [58,59], and the 6-item Cannabis Abuse Screening Test (CAST) [60,61]. CAST was also used for the assessment of synthetic cannabinoid use and related problems (and referred to as sCAST). AUDIT, CAST, and sCAST were only included in the first data collection procedure as these instruments screen substance use severity at an annual rate (ie, ask questions about the past 12 months), whereas FTND measures current smoking habits. Beliefs about psychoactive substances were measured by 20-item Beliefs About Substance Abuse [62], including areas such as the ability of controlled substance use or the role of craving in relapse. Knowledge about the risks and prevalence of both legal and illegal classic and NPS-type drugs were measured by 12 items. These true/false or multiple-choice questions (eg, “What is the national prevalence of lifetime cannabis use among high school students in your country?” A=15% to 20%, B=30% to 40%, C=50%, or more) tested the knowledge that the app aimed to provide for its users. A 5-point Likert scale was used for assessing the attitudes toward substance users. Respondents had to evaluate 10 statements on substance users and addiction itself (eg, “Addiction is not a disorder, it is a lack of will-power”), rating how strongly they agree with them (from 1=Absolutely disagree, to 5=Absolutely agree). A total score of negative attitudes was computed. Finally, perceived self-efficacy was measured by the 10-item General Perceived Self-Efficacy Scale [63,64], primarily dealing with self-observed coping skills.

Statistical Analysis

Data were analyzed by SPSS 17 (SPSS Inc) [65]. Descriptive statistics were applied to provide sample characteristics. Mean age and exercise frequencies were compared by independent sample t tests, perceived socioeconomic status as an ordinal variable was compared by Mann-Whitney U test, whereas potential differences in gender distribution and repetition of year rates were analyzed by chi-square statistics. Baseline comparison of the app versus nonapp groups, vocational versus high schools, and compliant respondents versus dropouts was implemented by using independent sample t tests, Mann-Whitney U test, chi-square test, and Fisher exact test. AUDIT, FTND, CAST, and sCAST scores were compared between the app and nonapp groups by using Mann-Whitney U test. Gender differences in substance use frequencies were analyzed by chi-square statistics and Fisher exact test.

To control for the design effect, pre- and postdata comparisons were performed by estimating treatment (intervention) effect with a series of linear regression analysis. Design effect was calculated by average cluster size and intracluster correlation. Changes in past month frequencies of less commonly consumed substances (including both “classic” and NPS-type stimulants and depressants) and the FTND scores at the postdata setting could not be examined by a regression analysis because of low response rates. The evaluation of the app was examined with descriptive statistics. Spearman rank correlation was applied to measure the connection between the repeated use of the distinct modules and the changes in attitudes, self-efficacy, knowledge, and exercise frequencies. The association between the app’s preference and perceived utility and substance use changes was also tested by Spearman rank correlation.


Sample Characteristics

Altogether 386 students participated in the first data-collection (T0) session and 246 students took part in the second data collection procedure (T1). Main reasons of dropouts were lack of motivation to participate further in the study or missing identifiers to pair pre- and postdata. Tables 1 and 2 summarize detailed sample characteristics with respect to respondents with available identifiers.

With respect to both the pre-and posttest setting, the app and nonapp groups differed in age but not in gender distribution. In addition, a significant difference was found with respect to baseline exercise frequencies as members of the nonapp group showed higher means of daily exercise duration. In case of the posttest setting, this difference ceased to be significant. Vocational and high school students differed in perceived socioeconomic status as high school students reported higher living standards (mean 4.8, SD 0.9, U=14,517.5, P=.001) than students of the vocational schools (mean 4.5, SD 0.9). However, the app and nonapp groups did not show significant difference in terms of perceived socioeconomic status and rates of school year repetition because of failure (ie, poor academic performance).

Furthermore, respondents who participated in both data collection phases (ie, compliant participants) were compared with those who dropped out of the study, with regard to gender distribution, age, rates of year repetition because of academic failure, and substance use characteristics (lifetime and past month use of both legal and illegal substances, AUDIT, CAST, sCAST, and FTND scores). On the basis of this analysis, the group of students who dropped out of the study was characterized by a higher rate of male respondents (For N=384, χ21=10.4, P=.001), higher mean age (mean 16.9, SD 0.9, t=−2.72, P=.007), higher rates of lifetime cannabis (For N=323, χ21=4.4, P=.04), ecstasy (Fisher exact test, P=.006), amphetamine (Fisher exact test, P=.001), cocaine (Fisher exact test, P=.001), and LSD (Fisher exact test, P=.005) use; higher rates of last month cannabis (For N=323, χ21=4.7, P=.03), tobacco (For N=252, χ21=4.97, P=.03), LSD (Fisher exact test, P=.02), GHB (Fisher exact test, P=.02), and synthetic cannabinoid (Fisher exact test, P=.03) consumption. There were no differences in AUDIT (U=8296, P=.17), CAST (U=720.5, P=.92), sCAST (U=264.5, P=.31), or FTND (U=264.5, P=.09) scores, or in the rate of year repetition because of academic failure (N=385, χ21=0.2, P=.70).

Baseline Psychoactive Substance Use

The majority of the students have consumed alcohol at least once in their lifetimes (average rate=91.8%, 334/364), and there was no significant difference in the frequencies of lifetime and past month alcohol use between the subgroups (app vs nonapp groups; vocational vs high schools). However, vocational and high school students differed in the rates of lifetime and last month smoking. Although high school students showed a higher rate of lifetime smoking, last month frequencies were higher in vocational schools. Concerning illegal substances, respondents mainly had experiences with cannabis (average lifetime use rate=13.3%, 43/323), ecstasy (average lifetime use rate=3.1%, 10/319), and LSD or magic mushroom (average lifetime use rate=2.5%, 8/319). The nonapp group and participants from vocational schools showed higher rate of lifetime ecstasy consumption. In addition, lifetime use of cocaine was more frequent among the participants of the nonapp group. With respect to NPS-type substances, synthetic cannabinoids were found to be the most commonly used drugs (average lifetime use rate=6.6%, 21/319), whereas MDPV consumption only occurred among the students of vocational schools. Table 3 presents the results of the applied comparative analyses.

In case of baseline AUDIT, FTND, CAST, and sCAST total scores, there were no significant differences between the app and nonapp groups. However, students of the vocational schools showed higher total scores on both the CAST (mean 2.2, SD 4.3, U=532, P=.02) and the FTND (mean 3.3, SD 2, U=118, P<.001) compared with the CAST (mean 0.3, SD 0.7) and FTND scores (mean 1, SD 1.4) of high school students. Considering the severity of baseline substance use, 64 students (22.5%) indicated hazardous or harmful alcohol use (based on AUDIT cut-off: a score of 8 or more), 14 students (17.9%) showed moderate or severe cannabis use (based on CAST cut-off: a score of 3 or more), 3 students (5.8%) indicated moderate or severe synthetic cannabinoid use (based on CAST cut-off: a score of 3 or more), whereas 11 students (20.4%) showed risk for moderate or severe nicotine dependence (based on FTND cut-off: a score of 5 or more). With regard to potential gender differences, male students showed higher rates of lifetime ecstasy use (Fisher exact test, P=.04), whereas higher rates of lifetime smoking were found among female students (For N=384, χ21=11.95, P<.001). In case of AUDIT, FTND, CAST, and sCAST, no gender differences occurred.

Pre- and Postdata Comparisons

Among those participants with available and comparable pre- and postdata, potential changes in psychoactive substance use, knowledge about psychoactive substances, perceived self-efficacy, exercise frequencies, beliefs about substance use, and attitudes toward substance users were analyzed and compared between the app and nonapp group. Multimedia Appendix 2 presents pre- and posttest settings’ description with respect to the outcome measures for both the app and nonapp groups. Intervention or treatment effect was estimated with a series of linear regression analyses, while controlling for the design effect as well. Significant treatment or intervention effect was observed only in the frequency of past month energy drink consumption. Users of the app showed greater decrease in energy drink consumption after the implementation of the intervention. A trend increase in psychoactive substance–related knowledge and physical exercise frequencies could be seen in the app group but, compared with the nonapp group, these differences were not significant and could not be considered relevant.

Table 1. Sample characteristics and group differences at T0 participation.
Sample characteristicsApp groupNonapp groupSignificancea (app vs nonapp group)Effect size, r

VS1bHS1cVS2dHS2e

T0 participation, N1481077160N/AfN/A
Age, mean (SD)16.7 (0.9)16.5 (1)16.9 (1)17.2 (0.7)t=−4.31; P<.001.23
Male participants, n (%)131 (89.1)54 (50.9)70 (98.6)24 (40)χ21=0.1.01
Repetition of year of study because of failure, n (%)13 (8.8)1 (0.9)5 (7)0 (0)χ21=0.5.04
Perceived socioeconomic status, mean (SD)4.5 (0.9)4.9 (0.9)4.5 (1.0)4.6 (1.0)U=15,387.5.06
Exercise frequency (minutes/day), mean (SD)103.9 (91.9)92.8 (51.9)169.5 (250.3)82.9 (47.4)t=−2.12; P=.40.10

aIndependent sample t-test or Mann-Whitney U test or χ2 test or Fisher exact test.

bVS1: vocational school 1.

cHS1: high school 1.

dVS2: vocational school 2.

eHS2: high school 2.

fN/A: not applicable.

Table 2. Sample characteristics and group differences at T1 participation.
Sample characteristicsApp groupNonapp groupSignificancea (app vs nonapp group)Effect size, r

VS1bHS1cVS2dHS2e

T1 participation, N90653853N/AfN/A
Age, mean (SD)16.7 (0.9)16.3 (0.9)16.7 (0.8)17.1 (0.7)t=−3.29; P=.001.22
Male participants, n (%)79 (87.8)33 (50.8)37 (97.4)18 (34)χ21=3.7.12
Repetition of year of study because of failure n (%)9 (10)1 (1.5)2 (5.3)0 (0)P=.22N/A
Perceived socioeconomic status, mean (SD)4.4 (0.9)5 (0.9)4.3 (1.1)4.6 (0.9)U=6230.10
Exercise frequency (minutes/day), mean (SD)153.2 (208.9)88.3 (48.5)191.3 (257.5)84.6 (135.8)t=−0.13.01

aIndependent sample t-test or Mann-Whitney U test or χ2 test or Fisher exact test.

bVS1: vocational school 1.

cHS1: high school 1.

dVS2: vocational school 2.

eHS2: high school 2.

fN/A: not applicable.

Table 3. Psychoactive substance use characteristics at baseline measurement.
Category, substance, and characteristicsApp groupNonapp groupSignificancea (app vs nonapp group)Effect size, r



VS1bHS1cVS2dHS2e

Legal substances, n (%)

Alcohol


Lifetime use123 (92.5)94 (88.7)60 (92.3)57 (95)χ21=0.9.05


Last month65 (56)53 (50.5)33 (55.9)40 (67.8)χ21=2.2.08

Tobacco


Lifetime use77 (52)84 (77.6)42 (59.2)50 (83.3)χ21=2.1.07


Last month use35 (45.5)20 (24.1)16 (38.1)12 (24)χ21=0.4.04
Illegal substances, n (%)

Cannabis


Lifetime use14 (13.1)14 (14.1)10 (17.2)5 (8.5)χ21=0.0.01


Last month use2 (1.9)7 (7.1)4 (6.9)1 (1.7)χ21=0.0.001

Ecstasy


Lifetime use3 (2.8)0 (0)6 (10.3)1 (1.7)P=.04fN/Ag


Last month use1 (0.9)0 (0)2 (3.4)0 (0)P=.56N/A

Amphetamines


Lifetime use2 (1.9)1 (1)3 (5.2)1 (1.7)P=.26N/A


Last month use1 (0.9)0 (0)2 (3.4)0 (0)P=.56N/A

Cocaine


Lifetime use1 (0.9)0 (0)5 (8.6)1 (1.7)P=.01hN/A


Last month use1 (0.9)0 (0)2 (3.4)0 (0)P=.56N/A

LSDi/Magic mushroom


Lifetime use3 (2.8)1 (1)4 (6.9)0 (0)P=.47N/A


Last month use2 (1.9)0 (0)2 (3.4)0 (0)P=.63N/A

Heroin


Lifetime use1 (0.9)0 (0)2 (3.4)0 (0)P=.56N/A


Last month use1 (0.9)0 (0)2 3.4)0 (0)P=.56N/A
NPSj-type substances, n (%)

GHBk


Lifetime use2 (1.9)0 (0)3 (5.2)0 (0)P=.36N/A


Last month use1 (0.9)0 (0)3 (5.2)0 (0)P=.14N/A

Mephedrone


Lifetime use1 (0.9)0 (0)2 (3.4)0 (0)P=.56N/A


Last month use1 (0.9)0 (0)2 (3.4)0 (0)P=.56N/A

Pentedrone


Lifetime use2 (1.9)0 (0)3 (5.3)0 (0)P=.36N/A


Last month use1 (0.9)0 (0)2 (3.5)0 (0)P=.30N/A

MDPVl


Lifetime use5 (4.7)0 (0)3 (5.2)0 (0)P=.99N/A


Last month use1 (0.9)0 (0)2 (3.4)0 (0)P=.56N/A

Synthetic cannabinoids


Lifetime use6 (5.7)4 (4.2)8 (13.8)3 (5.1)P=.16N/A


Last month use1 (0.9)1 (1)4 (6.9)0 (0)P=.19N/A

aχ2 test or Fisher exact test.

bVS1: vocational school 1.

cHS1: high school 1.

dVS2: vocational school 2.

eHS2: high school 2.

fP<.05.

gN/A: not applicable.

hP<.01.

iLSD: lysergic acid diethylamide.

jNPS: novel psychoactive substance.

kGHB: gamma-hydroxybutyrate.

lMDPV: methylenedioxypyrovalerone.

Explaining Changes in Past Month Substance Use

As secondary outcome measures, changes in past month frequencies of psychoactive substance use were examined using the changes in beliefs about substance use, psychoactive substance–related knowledge, perceived self-efficacy, exercise frequencies, and attitudes toward substance users as explanatory variables, applying linear regression models. An increase in past month alcohol consumption was explained by a decrease in negative attitudes toward substance users (beta=−.13, P=.04). Changes in past month tranquillizer misuse or overuse, cannabis use, and FTND scores were not explained by any of the predictor variables. An increase in past month energy drink consumption was explained by an increase in exercise frequency (beta=.04, P<.001) and a decrease in perceived self-efficacy (beta=−.29, P=.04).

Evaluation of the App

Tables 4 and 5 present the results of the app’s evaluation as provided by the respondents of the app group.

Respondents highlighted the overall utility of the app and the majority of the participants thought that the app might be an effective tool in providing preventive knowledge. Fewer respondents considered the app to be a useful instrument in decreasing substance use or promoting exercise. When assessing the connection between the use of the app’s distinct modules and the main outcome measures, results of the Spearman rank correlation analysis indicated that of the 6 modules of the app, repeated use of the “What if?” roleplay module showed a significant correlation with an increase in psychoactive substance–related knowledge (rS(35)=.39, P=.02). A significant correlation was also observed between a decrease in negative attitudes toward substance users and the repeated use of the “Where to go?” module (rS(44)=.31, P=.04). Finally, a significant correlation was found between the app’s perceived general usefulness and a decreasing past month alcohol consumption (rS(44)=.32, P=.03), indicating that those respondents in the app group who found the app useful, showed lower frequencies of alcohol intake at the time of postdata collection.

Table 4. Evaluation of the app’s distinct modules by the respondents.
ModulesUtility, mean (SD)Subjective preference, mean (SD)
Animated comics3.3 (1.3)3.5 (1.4)
Quiz game3.9 (1.2)3.8 (1.3)
What if?3.8 (1.3)3.9 (1.3)
Substance store3.9 (1.3)3.8 (1.3)
Trans-formation3.9 (1.3)3.7 (1.3)
Where to go?3.5 (1.4)3.3 (1.5)
Table 5. General evaluation of the app by the respondents.
General evaluation of the appMean (SD)
“Utilization of the app is easy and clear.”4 (1.2)
“By using the app, I received novel and relevant information.”3.9 (1.2)
“Visual realization fits the topic of substance use.”3.6 (1.4)
“I think that the use of this app can change the user’s attitudes toward both legal and illegal substances.”3.6 (1.4)
“The app is intriguing and entertaining.”3.5 (1.3)
“The structure of the app is diverse and captures the user’s attention.”3.4 (1.3)
“I think that the use of the app might facilitate treatment seeking among substance users.”3.3 (1.3)
“The quality of visual realization is high.”3.3 (1.3)
“I think that the use of this app helps in decreasing the frequency of substance use.”3.3 (1.4)
“The app helps in the destigmatization of substance users.”3.2 (1.4)
“The app helps in the promotion of exercise.”3.1 (1.4)
“The app increases empathy and tolerance.”3.1 (1.4)

Principal Findings

This pilot study identified some potential benefits of the Once Upon a High app and also a considerable amount of flaws that need amendments to further increase the utility of the intervention. Our result that alcohol was the most commonly consumed psychoactive substance is in line with the results of epidemiological studies with respect to the substance use characteristics of the adolescent population [40]. Although we did not assess a representative sample, average lifetime use rate of NPS-type drugs was also comparable with the findings of the ESPAD study. Synthetic cannabinoids were the most popular NPS, especially in one of the vocational schools (VS2). Students of the vocational schools showed higher frequencies of tobacco, ecstasy, and MDPV use as well as higher scores on the CAST and FTND, indicating more severe problems associated with their substance use. As O’Malley and colleagues [66] emphasize in their study, school types and related socioeconomic status may have a significant impact on the students’ psychoactive substance use frequencies. Considering the assessed sample, vocational students indeed reported lower living standards than high school students.

With regard to the secondary outcome measures of the regression models, results indicate that a decrease in negative attitudes toward substance users might be a risk factor for an increasing past month alcohol consumption. This finding suggests that the subgoal of the app to destigmatize the population of substance users (those who consume either legal or illegal substances) might miss the aim: instead of facilitating social tolerance, it may indirectly promote an accepting attitude toward substance use itself. According to Hohman and colleagues [67], attitudes toward psychoactive substances—especially ambivalent attitudes—can affect substance use behavior. An increase in past month energy drink consumption was associated with higher frequencies of exercise. This result is in line with the findings of Larson and colleagues [68], namely, that pursuing sports or energy drink consumption is often related to higher rates of participation in organized sport activities among adolescents; thus, exercise is not exclusively a protective factor with respect to any substance use.

On the basis of both the respondents’ evaluation on the utility of the distinct modules and the pre-and postcomparative results of our pilot study, the Once Upon a High app might be overall ineffective in decreasing psychoactive substance use (with the exception of energy drink consumption), increasing exercise frequencies, or enhancing self-efficacy at least under the conditions of this study. When we, however, assessed repeated use of distinct modules of the app (and not the app in general), the What if? module showed significant correlation with an increase in substance-related knowledge, whereas the use of the Where to go? module was associated with decreased negative attitudes toward substance users. The What if? module was also one of the most preferred parts of the app by the responding students. As the “What if?” module contains gaming elements, the fact that it was found to be the most effective and likeable part of the app, once again highlights the benefits of gamification. Johnson and colleagues [69] also conclude that as a result of their systematic review, gamification can have a positive impact on health behaviors, although current literature lacks efficacy studies with respect to the effects of gamification on substance use. Nevertheless, Fleming and colleagues [70] also emphasize that gamification might have considerable potential for increasing the efficacy of mental health–related Web-based interventions.

In terms of hypothesis testing, our study could only partially confirm the H3 hypothesis, as the app appeared to be effective in decreasing energy drink consumption but not in reducing other substance use frequencies. Considering the emerging concerns about the harmful effects of energy drinks with high content of caffeine, sugar, or other ingredients (eg taurine or guarana) and the increasing prevalence of energy drink consumption among adolescents [71], this result might still be relevant. A correlation between the app’s perceived usefulness and a decreasing frequency of past month alcohol use was also found, indicating a more motivated subgroup of respondents who not just downloaded the app but also found it a utilizable tool. It further needs to be addressed that the majority of the respondents did not show severe (or even moderate) substance use problems as indicated by the cut-off scores of AUDIT, CAST, sCAST, and FTND. Furthermore, those students who dropped out from the study, showed higher frequencies of lifetime and past month substance use. Therefore, it is likely that the app could not exert its preventive effects on those students who would have needed it the most. This high-risk subgroup mainly consisted of males and older students. Assessing a sample consisting of adolescents with problematic substance use might better fit the scope of secondary prevention. On the basis of the characteristics of our sample (ie, the majority did not show at risk or hazardous substance use), it was not entirely possible to draw valid consequences in terms of the app’s secondary prevention efficacy. Although a significant increase in substance use was not found among the app users, the possibility of iatrogenic harms cannot be neglected in case of the app, as it was already noted in case of other prevention programs as well [72,73]. As Dishion and colleagues [72] pointed out, such harms might be dependent on many factors, including the adolescent target group’s homogeneity (eg, containing only at-risk individuals) or the specific phase of adolescence (eg, early vs late adolescence) at the time of the intervention.

Limitations

Our pilot study is not without limitations. We did not use a randomized controlled design. The use of the app was not monitored coherently, even if respondents had to provide answers about their experiences with the app. The sample size for the applied postanalyses—because of the relatively high dropout rate—might be considered to be small for reliable multivariate analysis. Furthermore, a nonprobability, convenience sampling method was used that doubtlessly reduced the external validity of the study. The short period of follow-up time between T0 and T1 occasions (ie, 2 months) could also encumber the valid measurement of change, particularly in terms of substance use. Behavioral and cognitive change can be perceived as the result of a complex process with many unassessed factors in the background, including, but not limited to, family dynamics, peer pressure, other sources of psychoactive substance–related information (eg, internet fora), or motivation. For similar reasons, full causality cannot be assumed in our results.

These first experiences with the Once Upon a High app highlight the relevance of the target population’s motivation or the lack of it. We tried to motivate participants by offering them free recreational activities; however, this strategy failed to fulfill its goals. The offered possibility of costless admission to gyms, paintball facilities, or indoor climbing centers, did not show association with an increased frequency of daily exercise, and the dropout rates were still high. Therefore, other strategies need to be tested in the future with respect to their efficacy in motivating adolescents to maintain the use of this or a similar app. As adolescents susceptibly react to peer opinion and judgments, effective dissemination of such an app might be crucial as it may form its perception in the target population. If prevention apps can be presented as exciting or even fashionable tools that promote and help the self-management of various health conditions, the motivation of adolescents to use and share these might increase. As far as we are concerned, tools of extrinsic rewards (eg, money or presents) do not come into question in case of such prevention programs. On the basis of the findings of El-Hilly and colleagues [74], it is intrinsic motivation that may function as a positive drive to engagement in a prevention program and to a positive health behavior as well, whereas external social influences might rather have a negative impact on the overall efficacy of gamification. Similar results were published by Habgood and Ainsworth [75], indicating that in-game intrinsic motivators are more effective than extrinsic ones in case of educational games.

Conclusions

To conclude, Once Upon a High app can be a useful and feasible tool to assist effective preventive intervention programs, as it was originally developed for such purposes as well, not necessarily as a stand-alone prevention instrument. It is likely that the beneficial effects of the app can be maximized when its use is supplemented with personal discussion. As Majeed-Ariss and colleagues [10] emphasize, similar apps that support personal self-management of chronic health conditions are usually implemented with the involvement of health professionals. School psychologists, health visitors, or social workers can also be involved in such projects to improve the effects of the app as well as to increase the target population’s commitment to use it.

Further research is needed to assess additional potentials of the app, and it is also necessary to find better ways to motivate the target population to repeatedly use this tool. Preventive effects of the app on substance use itself also need to be explored by using probability sampling and larger effect sizes and increasing the heterogeneity (eg, cross-cultural differences) to find established ways to improve the app so that it can be really called and function as a “prevention” app in the end. This pilot study might function as a basis for future research with such purposes.

Acknowledgments

The development of the app was funded by the Hungarian Ministry of Human Capacities (grant number: KAB-ME-C-22756). The study was partially supported by the Hungarian National Research, Development and Innovation Office (Grant number: K111938). The work was completed in the ELTE Institutional Excellence Program (783-3/2018/FEKUTSRAT) supported by the Hungarian Ministry of Human Capacities. The funding institution had no role in the study design or the collection, analysis and interpretation of the data, writing the manuscript, or the decision to submit the paper for publication. The authors would like to thank the following colleagues for their immense work during the development of the app: Mónika Gyuró (graphic design), Dániel Karasz (programmer), Klára Berczi (project management), and Lívia Lukács (creating the app’s logo). The authors also wish to thank members of the co-design group for their contribution.

Conflicts of Interest

None declared.

Multimedia Appendix 1

Technical details of development.

PDF File (Adobe PDF File), 251KB

Multimedia Appendix 2

Pre-and posttest description in point of the outcome measures.

PDF File (Adobe PDF File), 29KB

  1. Inchley J, Currie D, Young T. Growing up unequal: gender and socioeconomic differences in young people's health and well-being - Health behaviour in school-aged children (HBSC) study: International report from the 2013/2014 survey. Geneva: World Health Organisation; 2016.   URL: http:/​/www.​euro.who.int/​__data/​assets/​pdf_file/​0014/​303440/​HSBC-No.​7-Growing-up-unequal-PART-1.​pdf?ua=1 [accessed 2018-10-15] [WebCite Cache]
  2. Gilani F. Novel psychoactive substances: the rising wave of 'legal highs'. Br J Gen Pract 2016 Jan;66(642):8-9 [FREE Full text] [CrossRef] [Medline]
  3. Simonato P, Bersani FS, Santacroce R, Cinosi E, Schifano F, Bersani G, et al. Can mobile phone technology support a rapid sharing of information on novel psychoactive substances among health and other professionals internationally? Hum Psychopharmacol 2017 May;32(3). [CrossRef] [Medline]
  4. Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science 1998 May 8;280(5365):867-873 [FREE Full text] [Medline]
  5. Bickel WK, Christensen DR, Marsch LA. A review of computer-based interventions used in the assessment, treatment, and research of drug addiction. Subst Use Misuse 2011;46(1):4-9 [FREE Full text] [CrossRef] [Medline]
  6. Marsch LA, Bickel WK, Grabinski MJ. Application of interactive, computer technology to adolescent substance abuse prevention and treatment. Adolesc Med State Art Rev 2007 Aug;18(2):342-56, xii. [Medline]
  7. Hansen AB, Becker U, Nielsen AS, Grønbæk M, Tolstrup JS, Thygesen LC. Internet-based brief personalized feedback intervention in a non-treatment-seeking population of adult heavy drinkers: a randomized controlled trial. J Med Internet Res 2012 Jul 30;14(4):e98 [FREE Full text] [CrossRef] [Medline]
  8. Strecher VJ, Shiffman S, West R. Randomized controlled trial of a web-based computer-tailored smoking cessation program as a supplement to nicotine patch therapy. Addiction 2005 May;100(5):682-688. [CrossRef] [Medline]
  9. Lee CM, Neighbors C, Kilmer JR, Larimer ME. A brief, web-based personalized feedback selective intervention for college student marijuana use: a randomized clinical trial. Psychol Addict Behav 2010 Jun;24(2):265-273 [FREE Full text] [Medline]
  10. Majeed-Ariss R, Baildam E, Campbell M, Chieng A, Fallon D, Hall A, et al. Apps and adolescents: a systematic review of adolescents' use of mobile phone and tablet apps that support personal management of their chronic or long-term physical conditions. J Med Internet Res 2015 Dec;17(12):e287 [FREE Full text] [CrossRef] [Medline]
  11. Champion KE, Newton NC, Stapinski LA, Teesson M. Effectiveness of a universal internet-based prevention program for ecstasy and new psychoactive substances: a cluster randomized controlled trial. Addiction 2016 Dec;111(8):1396-1405. [CrossRef] [Medline]
  12. Civljak M, Stead LF, Hartmann-Boyce J, Sheikh A, Car J. Internet-based interventions for smoking cessation. Cochrane Database Syst Rev 2013;7:CD007078. [CrossRef] [Medline]
  13. Zaaijer ER, Bruijel J, Blanken P, Hendriks V, Koeter MW, Kreek MJ, et al. Personality as a risk factor for illicit opioid use and a protective factor for illicit opioid dependence. Drug Alcohol Depend 2014 Dec 1;145:101-105 [FREE Full text] [CrossRef] [Medline]
  14. Basiaux P, le BO, Dramaix M, Massat I, Souery D, Mendlewicz J, et al. Temperament and Character Inventory (TCI) personality profile and sub-typing in alcoholic patients: a controlled study. Alcohol Alcohol 2001;36(6):584-587. [Medline]
  15. Spalletta G, Bria P, Caltagirone C. Differences in temperament, character and psychopathology among subjects with different patterns of cannabis use. Psychopathology 2006 Oct 19;40(1):29-34. [CrossRef] [Medline]
  16. Etter JF. Smoking and Cloninger's temperament and character inventory. Nicotine Tob Res 2010 Jul 16;12(9):919-926. [CrossRef] [Medline]
  17. Stoddard J, Delucchi K, Muñoz R, Collins N, Stable EP, Augustson E, et al. Smoking cessation research via the internet: a feasibility study. J Health Commun 2005;10(1):27-41. [CrossRef] [Medline]
  18. BinDhim NF, McGeechan K, Trevena L. Who uses smoking cessation apps? A feasibility study across three countries via smartphones. JMIR Mhealth Uhealth 2014 Feb 6;2(1):e4 [FREE Full text] [CrossRef] [Medline]
  19. Brigham TJ. An introduction to gamification: adding game elements for engagement. Med Ref Serv Q 2015 Oct;34(4):471-480. [CrossRef] [Medline]
  20. Boendermaker WJ, Prins PJ, Wiers RW. Cognitive Bias Modification for adolescents with substance use problems--Can serious games help? J Behav Ther Exp Psychiatry 2015 Dec;49(Pt A):13-20 [FREE Full text] [CrossRef] [Medline]
  21. Kadden RM, Litt MD. The role of self-efficacy in the treatment of substance use disorders. Addict Behav 2011 Dec;36(12):1120-1126 [FREE Full text] [CrossRef] [Medline]
  22. Holloway A, Watson HE. Role of self-efficacy and behaviour change. Int J Nurs Pract 2002 Apr;8(2):106-115. [Medline]
  23. Collingwood TR, Sunderlin J, Reynolds R, Kohl HW. Physical training as a substance abuse prevention intervention for youth. J Drug Educ 2000;30(4):435-451. [CrossRef] [Medline]
  24. Terry-McElrath Y, O'Malley PM, Johnston LD. Exercise and substance use among American youth, 1991-2009. Am J Prev Med 2011 May;40(5):530-540 [FREE Full text] [CrossRef] [Medline]
  25. Linke SE, Ussher M. Exercise-based treatments for substance use disorders: evidence, theory, and practicality. Am J Drug Alcohol Abuse 2015 Jan;41(1):7-15 [FREE Full text] [CrossRef] [Medline]
  26. Wang D, Wang Y, Wang Y, Li R, Zhou C. Impact of physical exercise on substance use disorders: a meta-analysis. PLoS One 2014 Oct;9(10):e110728 [FREE Full text] [CrossRef] [Medline]
  27. Kilpatrick DG, Acierno R, Saunders B, Resnick HS, Best CL, Schnurr PP. Risk factors for adolescent substance abuse and dependence: data from a national sample. J Consult Clin Psychol 2000 Feb;68(1):19-30. [Medline]
  28. Hemphill SA, Heerde JA, Herrenkohl TI, Patton GC, Toumbourou JW, Catalano RF. Risk and protective factors for adolescent substance use in washington state, the United States and Victoria, Australia: a longitudinal study. J Adolesc Health 2011 Sep;49(3):312-320 [FREE Full text] [CrossRef] [Medline]
  29. Schmanke L. Art Therapy and Substance Abuse. In: Rosal ML, Gussak GE, editors. The Wiley Handbook of Art Therapy. New Jersey, USA: Wiley Blackwell; 2015.
  30. Dutra L, Stathopoulou G, Basden SL, Leyro TM, Powers MB, Otto MW. A meta-analytic review of psychosocial interventions for substance use disorders. Am J Psychiatry 2008 Feb;165(2):179-187. [CrossRef] [Medline]
  31. Magill M, Ray LA. Cognitive-behavioral treatment with adult alcohol and illicit drug users: a meta-analysis of randomized controlled trials. J Stud Alcohol Drugs 2009 Jul;70(4):516-527 [FREE Full text] [Medline]
  32. Tanner-Smith EE, Wilson SJ, Lipsey MW. The comparative effectiveness of outpatient treatment for adolescent substance abuse: a meta-analysis. J Subst Abuse Treat 2013 Feb;44(2):145-158 [FREE Full text] [CrossRef] [Medline]
  33. Kamioka H, Okada S, Tsutani K, Park H, Okuizumi H, Handa S, et al. Effectiveness of animal-assisted therapy: a systematic review of randomized controlled trials. Complement Ther Med 2014 Apr;22(2):371-390. [CrossRef] [Medline]
  34. Simons-Morton B, Kuntsche E. Adolescent estimation of peer substance use: why it matters. Addiction 2012 May;107(5):885-886 [FREE Full text] [CrossRef] [Medline]
  35. Marconi A, Di Forti M, Lewis CM, Murray RM, Vassos E. Meta-analysis of the association between the level of cannabis use and risk of psychosis. Schizophr Bull 2016 Dec;42(5):1262-1269 [FREE Full text] [CrossRef] [Medline]
  36. Alsunni AA. Energy drink consumption: beneficial and adverse health effects. Int J Health Sci (Qassim) 2015 Oct;9(4):468-474 [FREE Full text] [Medline]
  37. Marshall EJ. Adolescent alcohol use: risks and consequences. Alcohol Alcohol 2014;49(2):160-164. [CrossRef] [Medline]
  38. Simons-Morton BG, Farhat T. Recent findings on peer group influences on adolescent smoking. J Prim Prev 2010 Aug;31(4):191-208 [FREE Full text] [CrossRef] [Medline]
  39. Gray R, Bressington D, Hughes E, Ivanecka A. A systematic review of the effects of novel psychoactive substances 'legal highs' on people with severe mental illness. J Psychiatr Ment Health Nurs 2016 Jun;23(5):267-281. [CrossRef] [Medline]
  40. ESPAD Group. Results from the European School Survey Project on Alcohol and Other Drugs ESPAD Report 2015. 2015.   URL: http://www.espad.org/sites/espad.org/files/ESPAD_report_2015.pdf [accessed 2018-10-15] [WebCite Cache]
  41. Camara M, Bacigalupe G, Padilla P. The role of social support in adolescents: are you helping me or stressing me out? Int J Adolesc Youth 2014 Mar 27;22(2):123-136. [CrossRef]
  42. Cooper ZD. Adverse effects of synthetic cannabinoids: management of acute toxicity and withdrawal. Curr Psychiatry Rep 2016 Apr 14;18(5). [CrossRef] [Medline]
  43. Karila L, Roux P, Rolland B, Benyamina A, Reynaud M, Aubin HJ, et al. Acute and long-term effects of cannabis use: a review. Curr Pharm Des 2014;20(25):4112-4118. [Medline]
  44. Karch S. Cocaine: history, use, abuse. J R Soc Med 1999 Aug;92(8):393-397 [FREE Full text] [Medline]
  45. Teixeira-Gomes A, Costa VM, Feio-Azevedo R, Bastos Mde L, Carvalho F, Capela JP. The neurotoxicity of amphetamines during the adolescent period. Int J Dev Neurosci 2015 Apr;41:44-62. [CrossRef] [Medline]
  46. Das S, Barnwal P, Ramasamy A, Sen S, Mondal S. Lysergic acid diethylamide: a drug of 'use'? Ther Adv Psychopharmacol 2016 Jun;6(3):214-228 [FREE Full text] [CrossRef] [Medline]
  47. Schwartz RH. Adolescent heroin use: a review. Pediatrics 1998 Dec;102(6):1461-1466. [Medline]
  48. Morgan C, Curran HV, Independent Scientific Committee on Drugs. Ketamine use: a review. Addiction 2012 Jan;107(1):27-38. [CrossRef] [Medline]
  49. Teter CJ, Guthrie SK. A comprehensive review of MDMA and GHB: two common club drugs. Pharmacotherapy 2001 Dec;21(12):1486-1513. [Medline]
  50. Mahendran R, Lim HA, Tan JY, Chua SM, Winslow M. Salvia divinorum: an overview of the usage, misuse, and addiction processes. Asia Pac Psychiatry 2016 Mar;8(1):23-31. [CrossRef] [Medline]
  51. Cottencin O, Rolland B, Karila L. New designer drugs (synthetic cannabinoids and synthetic cathinones): review of literature. Curr Pharm Des 2014;20(25):4106-4111. [Medline]
  52. Romer D. Adolescent risk taking, impulsivity, and brain development: implications for prevention. Dev Psychobiol 2010 Apr;52(3):263-276 [FREE Full text] [CrossRef] [Medline]
  53. Kelly PA. Does recreational ecstasy use cause long-term cognitive problems? West J Med 2000 Aug;173(2):129-130 [FREE Full text] [Medline]
  54. Kapitány-Fövény M, Kertész M, Winstock A, Deluca P, Corazza O, Farkas J, et al. Substitutional potential of mephedrone: an analysis of the subjective effects. Hum Psychopharmacol 2013 Jul;28(4):308-316. [CrossRef] [Medline]
  55. Busardò FP, Jones AW. GHB pharmacology and toxicology: acute intoxication, concentrations in blood and urine in forensic cases and treatment of the withdrawal syndrome. Curr Neuropharmacol 2015 Jan;13(1):47-70. [CrossRef] [Medline]
  56. Heatherton TF, Kozlowski LT, Frecker RC, Fagerström KO. The Fagerström Test for Nicotine Dependence: a revision of the Fagerström Tolerance Questionnaire. Br J Addict 1991 Sep;86(9):1119-1127. [Medline]
  57. Kótyuk E, Kovács-Nagy R, Faludi G. A nikotin dependencia és a dopamin D4-es receptor -521 CT promoter polimorfizmusának asszociációja major depressziós betegcsoportban [Association between nicotine dependence and the -521 promoter polymorfism of the dopamine D4 receptor in patients with major depression]. Neuropsychopharmacol Hung 2009;11:59-67.
  58. Babor T, Higgins-Biddle J, Saunders J. The Alcohol Use Disorders Identification Test, Guidelines for Use in Primary Care, Second Edition.: World Health Organization; 2001.   URL: http:/​/apps.​who.int/​iris/​bitstream/​handle/​10665/​67205/​WHO_MSD_MSB_01.​6a.​pdf;jsessionid=CD724A0054A8DA4018B740DF44D1B5B2?sequence=1 [accessed 2018-10-15] [WebCite Cache]
  59. Gerevich J, Bácskai E, Rózsa S. Az Alkoholhasználat Zavarainak Szűrőtesztje (Alcohol Use Disorders Identification Test, AUDIT). In: Gerevich J, Bácskai E, editors. Korszerű Addiktológiai Mérőeszközök. Budapest: Semmelweis Kiadó; 2012:66.
  60. Legleye S, Karila L, Beck F, Reynaud M. Validation of the CAST, a general population Cannabis Abuse Screening Test. J Subst Use 2009 Jul 12;12(4):233-242. [CrossRef] [Medline]
  61. Gyepesi A, Urbán R, Farkas J, Kraus L, Piontek D, Paksi B, et al. Psychometric properties of the Cannabis Abuse Screening Test in Hungarian samples of adolescents and young adults. Eur Addict Res 2014;20(3):119-128. [CrossRef] [Medline]
  62. Beck AT, Wright FD, Newman CF, Liese BS. Cognitive Therapy of Substance Abuse. New York: Guilford Press; 2001.
  63. Schwarzer R, Jerusalem M. Generalized Self-Efficacy scale. In: Weinman J, Wright S, Johnston M, editors. Measures in Health Psychology: A User's Portfolio. Causal and Control Beliefs. Windsor, UK: NFER-NELSON; 1995:35.
  64. Kopp M, Schwarzer R, Jerusalem M. Hungarian questionnaire in psychometric scales for cross-cultural self-efficacy research. Zentrale Universitats Druckerei der FU Berlin 1993.
  65. SPSS Inc.. SPSS Statistics for Windows. Chicago: SPSS Inc; 2008.
  66. O'Malley PM, Johnston LD, Bachman JG, Schulenberg JE, Kumar R. How substance use differs among American secondary schools. Prev Sci 2006 Dec;7(4):409-420. [CrossRef] [Medline]
  67. Hohman ZP, Crano WD, Siegel JT, Alvaro EM. Attitude ambivalence, friend norms, and adolescent drug use. Prev Sci 2014 Feb;15(1):65-74 [FREE Full text] [CrossRef] [Medline]
  68. Larson N, Dewolfe J, Story M, Neumark-Sztainer D. Adolescent consumption of sports and energy drinks: linkages to higher physical activity, unhealthy beverage patterns, cigarette smoking, and screen media use. J Nutr Educ Behav 2014 May;46(3):181-187 [FREE Full text] [CrossRef] [Medline]
  69. Johnson D, Deterding S, Kuhn KA, Staneva A, Stoyanov S, Hides L. Gamification for health and wellbeing: a systematic review of the literature. Internet Interv 2016 Nov;6:89-106 [FREE Full text] [CrossRef] [Medline]
  70. Fleming TM, Bavin L, Stasiak K, Hermansson-Webb E, Merry SN, Cheek C, et al. Serious games and gamification for mental health: current status and promising directions. Front Psychiatry 2016 Jan;7:215 [FREE Full text] [CrossRef] [Medline]
  71. Harris JL, Munsell CR. Energy drinks and adolescents: what's the harm? Nutr Rev 2015 Apr;73(4):247-257. [CrossRef] [Medline]
  72. Dishion TJ, McCord J, Poulin F. When interventions harm. Peer groups and problem behavior. Am Psychol 1999 Sep;54(9):755-764. [Medline]
  73. Allara E, Ferri M, Bo A, Gasparrini A, Faggiano F. Are mass-media campaigns effective in preventing drug use? A Cochrane systematic review and meta-analysis. BMJ Open 2015 Sep 03;5(9):e007449 [FREE Full text] [CrossRef] [Medline]
  74. El-Hilly AA, Iqbal SS, Ahmed M, Sherwani Y, Muntasir M, Siddiqui S, et al. Game on? Smoking cessation through the gamification of mHealth: a longitudinal qualitative study. JMIR Serious Games 2016 Oct 24;4(2):e18 [FREE Full text] [CrossRef] [Medline]
  75. Habgood MP, Ainsworth SE. Motivating children to learn effectively: exploring the value of intrinsic integration in educational games. J Learn Sci 2011 Apr 19;20(2):169-206. [CrossRef]


AUDIT: Alcohol Use Disorder Identification Test
CAST: Cannabis Abuse Screening Test
FTND: Fagerström Test for Nicotine Dependence
GHB: gamma-hydroxybutyrate
LSD: lysergic acid diethylamide
MDPV: methylenedioxypyrovalerone
NPS: novel psychoactive substance
sCAST: Cannabis Abuse Screening Test adapted for synthetic cannabinoids
SUD: substance use disorders
UID: unique identifier


Edited by G Eysenbach; submitted 25.01.18; peer-reviewed by R Wiers, W Boendermaker; comments to author 08.03.18; revised version received 30.06.18; accepted 07.08.18; published 04.12.18

Copyright

©Máté Kapitány-Fövény, Eszter Vagdalt, Zsófia Ruttkay, Róbert Urbán, Mara J Richman, Zsolt Demetrovics. Originally published in JMIR Serious Games (http://games.jmir.org), 04.12.2018.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Serious Games, is properly cited. The complete bibliographic information, a link to the original publication on http://games.jmir.org, as well as this copyright and license information must be included.