Effectiveness of contact tracing apps for SARS-CoV-2: an updated systematic review

Search strategy

The Bern COVID-19 Open Access Project (COAP) database was used for identification of relevant research. The COAP database is comprised of research from EMBASE (OVID), MEDLINE (PubMed), BioRxiv and MedRxiv databases, specifically focused on SARS-CoV-2. On June 9^th 2021 the COAP database was searched for scientific literature evaluating the effectiveness of CTAs for SARS-CoV-2 on epidemiological and clinical outcomes. The complete search strategy, as well as background information on the COAP database provided by Bern University, are provided in the Extended data: Supplementary File 1.

Eligibility criteria

Empirical (both observational and experimental) and model-based studies evaluating effectiveness of CTAs for SARS-CoV-2 were eligible for inclusion. Peer-reviewed publications as well as preprint papers were considered.

CTAs were considered when they provided feedback about potential recent exposure to an infected individual, based on proximity measurements (e.g., Bluetooth or GPS). Feedback should be provided directly to the individual through a CTA, although other feedback mechanisms, such as personal devices (e.g., a smartwatch), were also considered. National emergency warning systems using SMS were also included, provided they used proximity data to inform individuals.

All epidemiologically or clinically relevant outcomes that were used to quantify the impact of CTAs were considered, which include but are not limited to: quarantine rate, reproduction number (R), total number of infections, hospitalization rate, and mortality rate related to SARS-CoV-2. Studies investigating other relevant outcomes, such as prevention of outbreaks, a second infection wave, or flattening the curve, were also included. Studies solely assessing (determinants affecting) adoption rate of CTAs (i.e., the proportion of citizens using, and following recommendations provided by, the CTA), temporal change in incidence SARS-CoV-2, or other non-epidemiological or clinical outcomes were excluded.

Study selection

Studies identified in the search were first screened independently on title and abstract by two reviewers. Relevant studies were included for full text screening, and further selection of articles was performed by two independent reviewers. Any discrepancies were discussed and resolved. When consensus was not reached, a third reviewer was consulted to provide the final judgement.

Critical appraisal

Risk of bias was independently assessed by two researchers using separate checklists for empirical and model-based studies. Discrepancies between researchers were discussed and a final verdict was provided by a third reviewer if consensus was not reached. Empirical studies were appraised using a formal scoring method based on the Critical Appraisal Skills Programme (CASP) and Cochrane’s Effective Practice and Organisation of Care (EPOC) checklists^6,7 (Extended data: Supplementary File 2). Risk of bias in model-based research was evaluated by assessing use of empirical input data for the model, number of scenarios analyzed, and transparency of model reporting (Extended data: Supplementary File 3). Information on competing interests and funding were also collected for both empirical and model-based studies.

Data extraction

Data extraction was performed manually by one reviewer and checked by a second reviewer. Descriptive characteristics on type of research, i.e., empirical or model-based, sample size, (simulated) time horizon, study population, CTA properties and intervention, comparator, and epidemiological and clinical outcomes studied, were extracted from all included studies.

Specifically for model-based research, model characteristics (i.e., type of model and distributions used) and values used for important model parameters were collected. Furthermore, CTA specific properties were extracted, such as the method of contact tracing used by these apps. Forward tracing CTAs can only detect the ‘offspring’, i.e., individuals the index case has infected, of an infected individual. Bidirectional tracing CTAs also detect the ‘parents’, i.e., the individual that infected the index case of an infected individual. Models were considered to use bidirectional (as opposed to forward) tracing when, after the index case is detected and registered, all contacts within a period of at least the incubation time are identified, such that the parent of the index case could be found.

Another CTA specific property included the use of 1-step-tracing or sequential tracing. When a CTA-identified individual could only notify their contacts after testing positive themselves, this was considered 1-step-contact tracing. When notified contacts could subsequently also notify their own contacts, creating a cascade, even before that individual has shown symptoms or received a positive test result for SARS-CoV-2, this was considered sequential tracing.

The most important findings regarding effectiveness of CTAs for SARS-CoV-2 on epidemiological and clinical outcomes were extracted, synthesized, and reported narratively. These outcomes were pooled quantitatively whenever it was feasible to do so.

Study selection

A total of 5,123 potential studies were identified by the search. After selection based on title and abstract, 4,941 articles were excluded. Full texts of the 182 remaining studies were assessed, after which 27 articles were included for critical appraisal and data extraction (Extended data: Supplementary File 4). The 155 excluded studies with their reasons for exclusion are summarized in Extended data: Supplementary File 5.

Characteristics of included studies

A total of 27 primary studies were included, of which five were empirical observational (non-randomized) studies, and 22 were model-based studies (Tables 1 and 2).

Two of the 27 studies were published preprints, meaning they had not (yet) gone through the peer review process at the time of data extraction.^8,9 Included studies focused predominantly on the general population, although some analyzed the effectiveness of CTAs for specific populations such as hospital personnel, or school children.^10–19 Especially in model-based studies, results were often presented graphically. Consequently, the effectiveness of CTAs on epidemiological and clinical outcomes was only partly, or not at all, reported in key numerical figures.

The model-based studies typically assessed the effectiveness of CTAs by simulating one or more scenarios based on certain baseline or input values (e.g., proportion of asymptomatic infections). Table 3 provides an overview of characteristics and the most important input parameters used in models of the 22 included studies. Ten of the 22 model-based studies evaluated forward tracing CTAs,^{12,13,15,16,18–23} nine studies analyzed bidirectional tracing CTA,^{8–11,14,24–27} and three used an alternative or unclear method.^17,28,29 Ten studies used a CTA with sequential tracing.^{8–11,14,17,24–27} Nine of these also used bidirectional CTAs, which are more effective than forward tracing CTAs in reducing R, but require quarantining many more contact persons. This is especially the case when a significant number of infections come from asymptomatic individuals (i.e., transmission from a case who does not (yet) have symptoms), who are unaware they have SARS-CoV-2.²⁵

The percentage of CTA adoption was varied in almost all studies, allowing for assessment of the impact of CTAs on epidemiological and clinical outcomes. Average incubation time, i.e., the mean time between infection and symptom onset of SARS-CoV-2, was estimated to be 5 to 6 days for SARS-CoV-2, which was used by the majority of model-based studies.^{9–11,13–16,19,21,22,24,25,27,29} The proportion of asymptomatic SARS-CoV-2 infections, used as input parameter in model-based studies, was estimated at 20% to 50% based on empirical data,^12,13,15,22 but could vary between 18% to 86%.¹⁵ The baseline R value chosen in the model-based studies varied between 1.2¹² and 5.22.²⁴

Furthermore, superspreaders (i.e., individuals that infect numerous other individuals, and consequently have a high individual R) were discussed in context of the SARS-CoV-2 pandemic. Tracing these superspreaders and their contacts using CTAs was more effective compared to manual contact tracing (MCT).²¹ Hence, it is warranted to use bidirectional CTAs to trace these superspreaders, and advise them to immediately enter quarantine on identification.^16,30

Critical appraisal

Risk of bias of the five empirical studies was judged to be unclear for two,^31,32 and high for three^33–35 (Table 4). Confounding variables (such as smoking, work status, and income) were insufficiently taken into account for the three high risk studies, which was due to the explanatory nature of these observational empirical studies. In four out of five studies it was also unclear how missing (outcome) data were dealt with.

Most model-based research was judged to have a low risk of bias (Table 5). Seven of the 22 studies had a high risk of bias due to the lack of use of empirical distributions for variables, the limited number of scenarios analyzed, and insufficient transparency regarding reporting of the model.^{10,11,14,17,19,28,29}

Synthesis of results

Evidence from empirical studies

Five empirical comparative observational studies assessed the effectiveness of CTAs.^31–35 Two studies used empirical data to derive downstream consequences of CTAs using model-based extensions.^32,35

Ballouz et al. studied the time of going to quarantine for users and non-users of the SwissCovid.³¹ Results showed that close contacts of index cases that used a CTA went into quarantine sooner compared to non-users of the CTA [Hazard Ratio (HR) 1.5 (95% confidence interval (CI) 1.2–2.0)]. A second study looking at the SwissCovid CTA used empirical data on the number of authorization codes entered by SARS-CoV-2 positive individuals to model the downstream consequences on quarantine recommendations sent out and the increase in number of positively tested contacts.³⁵ From their model-based assessment they found that at most 1.7-5.1% additional cases of SARS-CoV-2 could have been detected by CTA in addition to MCT.

Another study investigated the introduction and adoption of a ‘Test and Trace’ app by 34,000 individuals living on the Isle of Wight (UK) and compared the estimated value of R in that region to that in the general UK population.³⁴ The CTA marked individuals as positive based on self-reporting of symptoms. Individuals that came in contact with an individual marked as positive were provided with social distancing advice. The study found that R reduced from 1.3 to 0.5 after implementation of the CTA. Within 2 to 3 weeks after implementation, incidence of SARS-CoV-2 diagnoses declined by around 90%.³⁴

A fourth study looked at the effectiveness of a text warning system used in 627,386 individuals who came in contact with an exposed population, and compared this cohort to the general population of Taiwan who did not use such a warning system.²⁰ They showed a reduction in incidence of respiratory syndrome from 19.23 to 16.87 per 1000 individuals. They also showed a reduction in pneumonia incidence from 3.81 to 2.36 per 1000 individuals.²⁰

Finally, Wymant et al. studied the effect of introduction of the NHS COVID-19 CTA on number of cases averted and mortality rate.³² Usage and notification data from 16.5 million users in the UK were used to estimate downstream consequences of NHS CTA use. They used a model-based and a statistical analysis approach and found that respectively 284,000 (95% of simulations ranged between 108,000–450,000) cases, or 594,000 (95% CI 317,000–914,000) cases, could be averted through use of the NHS CTA, relating to 15% or 31% of the total number of cases. Subsequently they determined that 4,200 (95% simulation range 1,600–6,600) deaths, or 8,700 (95% CI 4,700–13,500) deaths could be prevented based on the model-based and statistical approach, respectively. This corresponds to 13% or 27% of the total number of deaths. Approximately one case was averted for each case consenting to notification of their contacts. For every percentage point increase in CTA uptake, the number of cases could be reduced by 0.8% or 2.3% based on the model-based and statistical approach respectively.

Evidence from model-based studies

Effect on R

Effectiveness of a 1-step-contact tracing in reducing R can be approached using the following formula:

Here, R_c is the reproduction number when a CTA is used, R is the reproduction number without the use of a CTA, p is the proportion of the population using the CTA, and f is the combination of other factors that affect effectiveness of notification by the CTA. Such factors include, but are not limited to, delay between CTA notification and testing, delay between testing and test result, delay between reception of test result and entry of that result in the CTA, compliance to interventions (e.g., self-quarantine), and the proportion of infections that occur pre- or asymptomatically. Note that p occurs as a quadratic term, which reflects the fact that both infector and infectee have to use the CTA for the transmission to get traced.

Thirteen of the 22 model-based studies assessed the effect of CTAs on reduction of R.^{9,12–14,16,19,21–23,25–28} CTAs were able to control an ongoing outbreak or epidemic through quicker and more efficient feedback of a positive test result, and by notifying close contacts of a positively tested individual.^13,23,25 This speed and efficiency were not feasible using traditional MCT.¹³ New outbreaks could be controlled (i.e., R_c<1.0) by CTAs, by combining them with quarantine or self-isolation interventions, under the condition that hygiene and social distancing measures are maintained.^12,16,22,28 CTAs were able to reduce R by 0.3 more than traditional MCT, provided that feedback about contact with a positively tested individual is given to all contacts of the index case of the preceding 7 days.²⁵ Another model-based study reported that a CTA with 20% adoption rate reduces R by 17.6% compared to no contact tracing, whereas traditional MCT reduced R by 2.5% compared to no contact tracing.²³ This study also demonstrated that a CTA is able to reduce the R further, even when social distancing has already reduced R to 1.2. In this situation, R can be reduced further by 30% to 0.8 when CTA adoption rate is 80%.²³ Another model-based study determined that 60% adoption rate of a CTA could result in an R below 1.0.¹⁹ In one study, adoption rate of 53% resulted in a 47% reduction in R when the complete household of an individual with a positive test result is advised to be quarantined.¹⁶ The last study looking at effect of CTA on R showed that only at 60% adoption rate of the app a significant beneficial effect on R would become apparent.²⁷ When R is high (e.g., 3.0), and a considerable proportion of individuals is asymptomatic (e.g., 40% of all infections), CTAs need to be combined with other interventions (such as social distancing and random testing) to be able to lower the R below 1.0.²⁷ Potential for CTAs to reduce R is not only dependent on the adoption rate of the app, but also on (effectiveness of) various other factors in the downstream cascade after a positive notification. These include, but are not limited to, the delay between positive notification and testing,²⁷ and delay between receiving a positive test result,²¹ and the proportion of users sharing that result through the CTA. One study found that the percentage of preventable infections by one individual strongly depends on the time delay between CTA notification and the ability to be tested.²³ When there was no delay (i.e., 0 days) 79.9% of infections could be prevented, compared to 41.8% and 4.9% for 3 and 7 day delay, respectively.

Effect on total number of infections

Twelve of the 22 model-based studies assessed the effect of CTAs on reducing the total number of infections.^{8–10,15,17–20,22,24,27,29} Two studies indicated that the success of CTAs in reducing the total number of infections could only be ensured with a high adoption rate of that app.^12,18 Some plateau effect at 60% adoption rate was observed by one study.²⁴ Another study showed that with a high CTA adoption rate of 75%, there would be no more new infections occurring within three months after implementation.¹⁹ It was found that adequate hygiene and social distancing measures are needed to enable CTAs to reduce the total number of infections.^12,15,20,22 Especially in areas where there is low compliance to social distancing, a sufficiently high adoption rate of a CTA is essential to maintain control of an outbreak.¹⁵

The peak number of new infections can, according to one study, be reduced by half with a 50% adoption rate of a CTA,²² whereas another study showed that this could be achieved with an adoption rate as low as 20%.²⁹ Another study demonstrated that at 27% CTA adoption rate, a quarter of all new infections can be prevented.²⁰ However, according to another study that used a similar adoption rate, the number of infections would stabilize, but the epidemic would be maintained by core groups in densely populated areas.²² There may be a period of time of more than two months between implementation of interventions (such as CTAs) and the effect of that implementation on the total number of SARS-CoV-2 infections.¹⁸

Effect on number of hospitalizations

Two of the 22 model-based studies assessed the effect of CTAs on the number of hospitalizations due to SARS-CoV-2 infection. One study found that the peak number of hospitalizations reduced up to 46-78% for a 75% CTA adoption rate.²⁴ Another found that peak height of number of hospitalizations could be reduced by at least 50% for a CTA adoption rate of 60% and over.⁹ A German study did look into the effect of a CTA on the number of days that intensive care unit (ICU) capacity was exceeded.¹⁵ They found in their simulations that – based on the German population, and assuming an ICU capacity of 24.000 beds – a CTA adoption rate of 20% would prevent exceedance of ICU capacity at any point in time. In contrast, if no contact tracing (either manual or digital) would be used, ICU capacity would be exceeded on a quarter of days.

Effect on mortality rate

Five of the 22 model-based studies assessed the effect of CTAs on mortality rate.^{9,15,22,24,29} One study demonstrated that a high adoption rate (80%) of a CTA would result in an 85% reduction in mortality rate, over a period of 500 days.¹⁵ Another study found that a low CTA adoption rate (25%) is associated with a 10% decrease in mortality rate, an average adoption rate (50%) with 25% decrease, and a high adoption rate (75%) with 40-60% decrease.²² For the latter 75% CTA adoption rate, slightly higher rates (56-67%) were observed in another study.²⁴ Other evidence showed that at 40% adoption rate, during the peak of an outbreak, a reduction in number of deaths by 97% could be achieved.²⁹ Ruse et al demonstrated that mortality rate could be reduced by at least 50% for CTA adoption rates of 60% and over, provided testing rates were sufficient.⁹

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Open Science Framework: Supplementary files for Systematic review on Effectiveness of Contact tracing apps for COVID-19, https://doi.org/10.17605/OSF.IO/J5M7D.³⁶

This project contains the following extended data:

Reporting guidelines

Open Science Framework: PRISMA checklist for ‘Effectiveness of contact tracing apps for SARS-CoV-2: an updated systematic review’, https://doi.org/10.17605/OSF.IO/J5M7D.³⁶

Data are available under the terms of the Creative Commons Zero “No rights reserved” data waiver (CC0 1.0 Public domain dedication).