Relative effectiveness of a full versus reduced version of the ‘Smoke Free’ mobile application for smoking cessation: a randomised controlled trial

Introduction

Smartphone applications (apps) are used by many smokers to aid cessation but currently little evidence exists on their effectiveness. The Smoke Free app (smokefreeapp.com) is very popular worldwide, with some 4,000 new downloads per day. When it first became available, it was the subject of a trial with users finding and downloading the app from the app store being randomly assigned to a full version or a reduced version. This provided an opportunity to assess whether the full version was more effective than the reduced version in an effectiveness study closely mirroring the real-world scenario of interest. This paper reports the findings from that trial.

The Smoke Free app was developed using behaviour change techniques (BCTs) found in effective behavioural support programmes for smoking cessation¹. A description of the app is given in Supplementary File 1. Behavioural support delivered via a smartphone could help smokers to stop. Internet-based support has been found in some cases to aid cessation² and smartphone apps can provide this functionality with the added advantage of being readily accessible at almost any time. Two prospective studies of users of smoking cessation apps^3,4, a randomised controlled trial (RCT) comparing an app with a text messaging intervention⁵ and two RCTs comparing mindfulness-based apps with other apps^6,7 found self-reported success rates that were higher than would be expected from unaided cessation. One RCT has found that an app acting as a decision aid for smokers interested in stopping smoking resulted in higher 6-month abstinence rates than an information-only app⁸. Another RCT examined the effectiveness of a set of app components as an aid to cessation in pregnant smokers; engagement with the app was low and no specific components were found to increase short-term self-reported abstinence rates⁹. To date no RCTs have been published comparing apps designed to provide ongoing support for quit attempts with unaided quitting, or more intensive versus less intensive versions of an app.

Evaluating the effectiveness of smoking cessation apps versus unaided cessation in RCTs is complicated by the fact that apps are widely available and participants who are randomised to the unaided quitting condition are likely to be motivated to drop out of the study or use one of the many freely available apps. An alternative is to compare full and reduced versions of an app in which the reduced version is sufficiently credible that participants who are randomised to receive it are not motivated to drop out of the study or seek out another app. That was the approach used in the present study.

Another challenge for RCTs of apps is how to address the problem of loss to follow up. With sufficient resources, high follow-up rates can be obtained in such trials^6,10. However, the methods used can lead to problems of generalizability; study engagement processes that involve face-to-face visits, incentives and contracts on the part of participants that may exclude a substantial proportion of the target population. Moreover, the resources required are prohibitive for the kind of agile, iterative evaluation that is required during the development of these interventions, where evaluations need to be undertaken repeatedly¹¹. Automated outcome assessment using the smartphone is low cost and does not require procedures that may undermine generalizability. Despite the fact that it may result in very low follow-up rates that is the approach used in the current study.

In smoking cessation trials it is common practice to use an intention-to-treat approach with participants lost to follow up considered to have resumed smoking¹². This may bias effect sizes downwards if loss to follow up occurs for reasons other than relapse to smoking¹³. Conversely it may bias effect sizes upwards if the intervention condition leads to higher follow-up rates than the control condition. Only including participants who are successfully followed up may overestimate absolute success rates if participants refuse to engage with follow up because they have resumed smoking, but this would not affect the odds ratio comparing two conditions since this bias would affect both intervention and comparison groups equally. This approach is also immune to bias caused by differential follow-up in intervention and control groups. In practice ‘missing equals smoking’ (MES) and ‘follow-up only’ (FUO) approaches tend to produce very similar odds ratios in smoking cessation RCTs¹⁴, though the percentage point difference between conditions varies considerably. Multiple imputation methods are increasingly being used to estimate values for missing data arising from loss to follow up (e.g. Westmaas et al.¹⁵). However, these are only viable when the proportion of values that are missing is low. To address biases arising from loss to follow up, both the MES and FUO approaches were used in the present study. It may be expected that the true percentage point difference and odds ratios lie somewhere between the estimates provided by these two methods.

Biochemical verification of abstinence is recommended in smoking cessation trials because of psychological pressure to claim abstinence¹². However, this is highly resource intensive and may undermine generalizability to smokers who would use an app but not a more intensive interaction. Also, in a trial where there is no greater psychological pressure to claim abstinence in one condition than another, use of self-report should not bias the estimated effect size. Therefore, this study used self-report for outcome assessment.

Duration of follow up is an important consideration in smoking cessation trials. Conventionally, follow-up at least 6 months after the start of an intervention is considered appropriate for definitive trials while shorter durations are acceptable for proof of concept trials¹². A recent systematic review of continuous abstinence rates in smoking cessation trials has recently found, however, that rates at 6-month and 12-month follow up can be accurately predicted from findings after 12 weeks¹⁶. Loss to follow up may be greater with longer follow up so in the present study participants were followed up 12 weeks after the target quit date.

Thus, this study addressed the question of whether the full version of the Smoke Free app would result in higher 12-week self-reported continuous abstinence rates than a reduced version of the app in smokers downloading the app and using it to set a quit date.

Methods

Results

Table 1 shows participants’ baseline characteristics and Figure 1 shows the numbers allocated to each group and followed up. Participants who received the reduced version of the app were older, smoked more cigarettes per day, started smoking earlier in the day and were more likely to designate a quit date that was after the date of registration, but the differences were small. Complete data are shown in Dataset 1²².

Figure 1. Flow of participants.

Of the participants, 2,114 (7.5%) were followed up (full version 1,213, 8.5%, reduced version 901, 6.5%). In the MES analysis 1.6% (n=234) of the participants in the intervention group and 0.9% (n=124) of the participants in the control group reported as being abstinent from smoking (unadjusted Odds ratio=1.86; 95% CI=1.49-2.31; (p<0.001). In the FUO analysis, 19.3% in the intervention group and 13.8% in the comparison group reported being abstinent (unadjusted Odds Ratio=1.50; 95% CI=1.18-1.90; p<0.001).

Table 2 shows the results from logistic regression analyses with app version and all baseline variables entered together. In both the MES and FUO analyses, participants randomized to the full version of the app had higher odds of reporting successful abstinence with odds ratios almost identical to the unadjusted regression analyses. A number of baseline variables also predicted reported abstinence. Older participants, and in the MES analysis, those with longer time to their first cigarette of the day were more likely to report abstinence, while those using Android (versus iOS devices) and those whose quit date was after the date of registration were less likely to report abstinence.

Discussion

In both the MES and FUO analyses the full version of the Smoke Free app produced higher self-reported abstinence rates than the reduced version 12 weeks after the target quit date. The odds ratios ranged between 1.50 and 1.90 in the two analyses and the percentage point differences ranged between 0.7% and 5.5%.

Even with very low follow-up rates the study found a small but clear advantage to the full version of the app which may be attributed to the inclusion of the daily missions. This is on top of whatever effect the reduced version of the app may have had. Even in the follow-up only analysis the abstinence rates were relatively low, and lower than is found in studies involving face to face support or pharmacotherapy²³. Therefore, this app should not be regarded as a substitute for those forms of support but rather as an alternative in smokers who do not wish to engage with them or do not have access to them. It is possible that this app could increase abstinence rates in smokers using such forms of support but this remains to be tested.

The fact that an intervention effect was found in both the MES and FUO analyses strengthens confidence that it was not due to bias arising from differential loss to follow up. The fact that adjusting for baseline variables did not influence the odds ratios similarly adds confidence that the results were not due to smokers who found it easier to stop being more likely to be followed up in the intervention condition.

Strengths of the current study are the large sample size, the fact that it assessed an app that is very popular and therefore needs to be evaluated, and high generalizability to the population of interest, i.e. smokers finding the app on apps stores. Limitations are the very low follow-up rate, use of self-reported abstinence, the relatively short follow-up duration and the absence of process measures to assess what mediated the intervention effect.

Research continues into apps to support smoking cessation^24–29, with improvements in technology providing new opportunities for intervention content, such as virtual reality and use of wearable devices. The popularity of the Smoke Free app should make it a useful vehicle for testing innovations in smoking cessation support, building on the version of the app tested in this study. Research is needed into improving follow-up rates without compromising generalizability and within the resource constraints operating on companies and research groups seeking to build incrementally on app performance.

In conclusion, the full version of the Smoke Free smartphone app appears to have small effect in improving 12-week abstinence rates in smokers trying to quit. This provides a basis for building a programme of incremental improvement in effectiveness.

Data availability

Dataset 1. Full de-identified data from each study participant, including download dates, quitting dates and all other data input into the app. Data are available in SAV and CSV formats. DOI: https://doi.org/10.5256/f1000research.16148.d218541²².