Use of Random Domain Intercept Technology to Track COVID-19 Vaccination Rates in Real Time Across the United States: Survey Study

Background Accurate and timely COVID-19 vaccination coverage data are vital for informing targeted, effective messaging and outreach and identifying barriers to equitable health service access. However, gathering vaccination rate data is challenging, and efforts often result in information that is either limited in scope (eg, limited to administrative data) or delayed (impeding the ability to rapidly respond). The evaluation of innovative technologies and approaches that can assist in addressing these limitations globally are needed. Objective The objective of this survey study was to assess the validity of Random Domain Intercept Technology (RDIT; RIWI Corp) for tracking self-reported vaccination rates in real time at the US national and state levels. RDIT—a form of online intercept sampling—has the potential to address the limitations of current vaccination tracking systems by allowing for the measurement of additional data (eg, attitudinal data) and real-time, rapid data collection anywhere there is web access. Methods We used RDIT from June 30 to July 26, 2021, to reach a broad sample of US adult (aged ≥18 years) web users and asked questions related to COVID-19 vaccination. Self-reported vaccination status was used as the focus of this validation exercise. National- and state-level RDIT-based vaccination rates were compared to Centers for Disease Control and Prevention (CDC)–reported national and state vaccination rates. Johns Hopkins University’s and Emory University’s institutional review boards designated this project as public health practice to inform message development (not human subjects research). Results By using RDIT, 63,853 adult web users reported their vaccination status (6.2% of the entire 1,026,850 American web-using population that was exposed to the survey). At the national level, the RDIT-based estimate of adult COVID-19 vaccine coverage was slightly higher (44,524/63,853, 69.7%; 95% CI 69.4%-70.1%) than the CDC-reported estimate (67.9%) on July 15, 2021 (ie, midway through data collection; t63,852=10.06; P<.001). The RDIT-based and CDC-reported state-level estimates were strongly and positively correlated (r=0.90; P<.001). RDIT-based estimates were within 5 percentage points of the CDC’s estimates for 29 states. Conclusions This broad-reaching, real-time data stream may provide unique advantages for tracking the use of a range of vaccines and for the timely evaluation of vaccination interventions. Moreover, RDIT could be harnessed to rapidly assess demographic, attitudinal, and behavioral constructs that are not available in administrative data, which could allow for deeper insights into the real-time predictors of vaccine uptake–enabling targeted and timely interventions.


Introduction
Accurate and timely COVID-19 vaccination coverage data are vital for informing targeted, effective messaging and outreach and identifying barriers to equitable health service access. The tracking of vaccination rates is needed in conjunction with effective COVID-19 case surveillance and death monitoring, as well as research on predictors of vaccination uptake and vaccine hesitancy, to optimize vaccine coverage in persons at disproportionate risk of COVID-19 and severe outcomes [1][2][3][4][5]. However, gathering such vaccination rate data is challenging, even in high-income countries such as the United States. Bill Gates recently expressed the importance of investment in global pandemic preparedness [6], which should involve assessing and implementing surveillance systems that can be utilized globally.
In the United States, the Centers for Disease Control and Prevention (CDC) rely on multiple external sources for their vaccination tracker [7]. Although considered the gold standard, this approach is labor intensive and limited to the collection of administrative data. In other words, certain demographic, attitudinal, and behavioral constructs are not available. Another approach involves national polling by using traditional recruitment methods (eg, panels and random digit dialing), in which respondents self-report their vaccination status (eg, Kaiser Family Foundation surveys and the Census Household Pulse) [8,9]. These polls have the benefit of collecting extended information, which could allow for deeper insights into the factors that predict vaccine uptake. Although immensely valuable for assessing knowledge, attitudes, and practices, these polls often do not allow for large-scale, rapid data collection, nor do they provide real-time granular results (with data available daily on a variety of population subcategories). Further, neither of these approaches are easily expandable to other countries with differing monitoring systems and access.
To identify targetable risk factors, it is critical to combine the benefits of traditional survey research with the advantages of rapid and continuous data collection by using a method that is scalable and can be incorporated into a global surveillance system. As such, we assessed the validity of Random Domain Intercept Technology (RDIT; RIWI Corp)-a form of online intercept sampling [10]-for tracking self-reported vaccination rates in real-time, with broad reach, and at the national and state levels across the United States.

Procedure
We used RDIT from June 30 to July 26, 2021, to reach a broad sample of US web users. RDIT can be administered anywhere in the world where there is web access. When web users click on a registered but commercially inactive web link or type in a web address for a site that is dormant, they have a random chance of that link being temporarily managed by the company that owns and administers RDIT-RIWI Corp. In this situation, instead of coming across a "this page does not exist" notification, a survey is delivered. Web users then decide whether to anonymously participate and may exit the survey at any time. No incentives are provided for participation.
Upon encountering the landing page, web users chose their preferred language (English or Spanish), and were told that the survey was anonymous and about COVID-19. Web users who chose to participate first reported their age and gender (those aged under 18 years were immediately exited from the landing page). They then reported their vaccination status (the focus of the validation exercise). Following this, those who were vaccinated were administered questions about the vaccine they received, and those who were unvaccinated were given a series of questions about COVID-19 vaccines (intentions, incentives, barriers, etc). All respondents were asked if they had heard recent accounts of COVID-19 vaccination side effects and were asked to provide information on where they receive their news and entertainment. Except for age, gender, and race and ethnicity, the remaining demographic questions were asked at the conclusion of the survey (education, urban or rural living location, political affiliation, and annual household income).

Ethical Considerations
This project was approved as public health practice to inform message development (not human subjects research) by the institutional review boards of Johns Hopkins University and Emory University.

Measures
With regard to COVID-19 vaccination status, respondents were asked "Have you personally received the COVID-19 vaccine?" They were given the following four response options: "Yes, a single-dose vaccine (J&J)"; "Yes, the first of two doses (Moderna or Pfizer)"; "Yes, both doses of a two-dose vaccine (Moderna or Pfizer)"; and "I have not received a vaccine." RDIT captures nonpersonally identifiable state location information from a respondent's IP address, which is instantaneously translated into a unique identifier. For this validation analysis, we compared self-reported adult vaccination rates from RDIT (having received at least 1 dose) to those that were provided in the publicly available CDC COVID-19 Vaccinations in the United States, Jurisdiction data set on July 15, 2021 (midway through RDIT data collection) [11].

Statistical Analysis
RDIT-based vaccination rate estimates (and associated logit transformed 95% CIs) were computed for the full national sample (N=63,853) and by state among those with available state information (n=57,986). A series of correlational analyses and 1-sample t tests (2-tailed) were used to compare RDIT-based estimates to CDC-reported rates.

Respondent Description
By using RDIT, 1,026,850 US web users were exposed to the survey. Of those, 63,853 (6.2%) were aged ≥18 years and reported their vaccination status. Respondents [13,14].

State Comparison
The RDIT-based and CDC-reported state-level estimates were strongly and positively correlated (r=0.90, 95% CI 0.83-0.94; P<.001). RDIT-based estimates were higher than the CDC-reported estimates by a mean of 3% (SD 4.5%, 95% CI 1.7%-4.2%; t 50 =4.71; P<.001). RDIT-based estimates were higher than the CDC's estimates for 37 states and were within 2 and 5 percentage points for 12 and 29 states, respectively. When considering the absolute value of the estimate discrepancies, states with more RDIT respondents were associated with smaller discrepancies (r=−0.43, 95% CI −0.64 to −0.18; P=.001). We observed the largest discrepancy for Alaska (percent difference: 14.5%), which was likely due in part to the small sample size (n=85). The states with RDIT estimates within 1 percentage point of the relative CDC estimate were California; Connecticut; Washington, District of Columbia; Florida; Maryland; Maine; New York; Texas; and Utah (Table  1).

Principal Findings
RDIT provided similar estimates to those provided via the CDC method (the current standard) for vaccination rates at the national and state levels; however, estimates from RDIT are accessible daily at variable magnitudes and have region-targeting capabilities. Although the July 2021 RDIT-based national estimate was higher than the CDC estimate by 2%, it was more comparable to the CDC estimate than those derived from the Census Household Pulse and Delphi-Facebook surveys, which overestimated vaccination coverage by 17% and 14% in May 2021, respectively [15]. At the state level, RDIT-estimated rates strongly correlated with CDC reports, were slightly higher on average, and were within a 5% margin for 57% (29/51) of the states. These findings provide early evidence of the validity of RDIT as a complementary surveillance mechanism for tracking COVID-19 vaccination coverage across the United States.

Limitations
There are limitations to RDIT that should be considered. First, RDIT only reaches the web-using population; nonetheless, RDIT reaches a diverse set of the web-using population, including respondents who are not habitual survey takers, thereby allowing for subgroup analyses as needed. Second, a repeated measures assessment (ie, a follow-up to assess changes in vaccine status per individual) is not possible because RDIT does not collect identifying information. However, population-level changes can be identified. Similarly, without identifying information, opt-in bias is unknown, but one can record and evaluate trends in retention throughout the survey. Additionally, while associated with analytic limitations, the anonymous nature of RDIT is a strength, as participant privacy is prioritized. Although RDIT enables the collection of additional factors that may provide targets for improving vaccine coverage, it is possible for participants to drop out before they provide this information. An analysis of such drop-off can provide further understanding, and researchers can decide whether it is most appropriate to draw insights from all available data or only from respondents who complete the entire question set (in this case, we chose the former). Finally, administrative data, such as the data that inform the CDC vaccination tracker, could be less susceptible to self-reporting bias; however, we found strong correlations between the self-reported vaccination rate estimates and the CDC administrative metrics.

Conclusion
Access to this broad-reaching data stream is potentially less labor intensive than the alternative approaches that are currently used by the CDC, and RDIT-based estimates demonstrate adequate accuracy when compared to CDC estimates. RDIT's real-time nature may be a valuable tool for tracking vaccine uptake; pinpointing localities for targeted, timely interventions; and enabling the rapid evaluation of interventions and messaging campaigns globally. Of course, further investigation is needed to assess the accuracy of RDIT for tracking vaccination status globally. Nonetheless, RDIT could be harnessed to rapidly assess demographic, attitudinal, and behavioral constructs that are not available in administrative data, which could allow for deeper insights into the real-time predictors of COVID-19 vaccine uptake. Such data could be translated into effective interventions to strengthen vaccination coverage.