Can targeted messages reduce COVID-19 vaccination hesitancy? A randomized trial

Highlights • We find no evidence that tailoring public health communication regarding COVID-19 vaccination for broad demographic groups would increase its effectiveness.• A post hoc analysis finds that a vaccine endorsement from Dr. Fauci reduces stated intent to vaccinate among conservatives.• We recommend further research on communicators and endorsers, as well as incentives.


Introduction
Vaccine hesitancy has prolonged the COVID-19 pandemic in the U.S. Overcoming vaccine hesitancy is complicated because the reasons for resisting vaccination can be demographic-specific. For example, hesitancy regarding COVID-19 vaccination is higher among political conservatives and African Americans; some surveys also find increased hesitancy among Latinx people, religious Christians, and parents (Latkin et al., 2021;Milligan et al., 2021;Khubchandani and Macias, 2021;Momplaisir et al., 2021a, b;Tram et al., 2021;Riad et al., 2021a).
We apply theories of social identity to design messaging to reduce vaccine hesitancy among specific population segments. We test whether respondents report greater intent to take a hypothetical vaccine after receiving messages targeted to their demographic segment. Dubé et al. (2015) and Aw et al. (2021) nicely summarize the literature on vaccine hesitancy. Here we discuss factors emphasized in the standard model and in theories referencing one's sense of identity.

Studies on vaccine hesitancy
Prior research on health decisions often uses a rational costs-benefits framework (e.g., Strecher and Rosenstock, 1997;Armitage and Conner, 2001; on COVID-19 specifically, Kreps et al., 2020;Riad et al., 2021b). These approaches highlight: • the seriousness of the disease, • the safety of the vaccine, • the effectiveness of the vaccine, • the vaccination benefits for self and important others, and • the expertise of the source of the message.

Theories of identity
In theories of identity, one learns appropriate behavior for one's identity, typically by observing high-status individuals and the behavior of like people (Akerlof and Kranton, 2000;Stryker and Burke, 2000;Carter and Danielle, 2015). They then prefer to engage in those activities, all else equal.
One definition of social identity involves one's sense of self, derived from perceived membership in social groups. Belonging may provide a sense of identity. Researchers have used group identity to shed light on phenomena such as ethnic and racial conflicts (Sen, 2007), discrimination, political campaigns, and human-capital formation (Coleman, 1961). Charness and Chen (2020) survey the effects of social identity on economic decisions.
Studies of vaccine hesitancy have emphasized that social and identity factors loom large (Aw et al., 2021). For example, "people tend to be more sensitive to social information that is provided to them by prestigious individuals" (Romaniuc et al., 2021). Marketing has long targeted most of the segments we study (e.g., see Podoshen, 2008;Wechsler and Wernick, 1992;Wingrove et al., 2007).
Identity can have effects on both beliefs and preferences (Charness and Chen, 2020).
In terms of belief: • Genes generally affect one's response to drugs. Thus, different groups (such as African Americans) may perceive evidence on vaccine efficacy as more relevant if the trials included a meaningful share of African Americans. • People may place more trust in the benevolence of experts with greater shared identity. • One who sees many like people engaged in an activity may decide that they have relevant information and follow the herd (as in models of information cascades, e.g., Bikhchandani et al., 1992). • Vaccination that speeds the return to an activity a group member valued (e.g., religious services for those who had attended regularly pre-pandemic) is more important. • People more altruistic toward those with aligned identities may be more concerned with how their own vaccination protects these people.
Identity can also affect preferences: • One concerned about status within a group may follow the advice or actions of high-status people in the group. • People may follow their perceptions of typical group behavior ("descriptive norms") or of what the group considers proper behavior ("prescriptive norms"). • If people internalize group norms, they may follow high-status leaders or their perception of common activities, as either can signal the relevant group norms.

Hypotheses
An individual may possess multiple identities-Black or African American, Hispanic or Latina/o/x, religiously observant (prior weekly participation), politically conservative, and an active parent. Consider non-targeted messages that promote COVID-19 vaccination and messages tailored to these specific segments of the population.
Targeted messages may heighten attention to (or the salience of) aspects of the vaccination decision of particular importance to the individual. Religious individuals may focus on the possibility of the return of church services. Black or Latinx individuals may focus on the pandemic's disproportionate impact on their own community.
Relative to generic messages, targeted messages may also carry additional information. Individuals may learn that vaccine trials include genetic diversity. The informational content of an endorsement from someone with shared identity may be more trustworthy. An endorsement from a high-status group member may also convey group norms.
An individual who is a member of any of our five segments of interest may receive treatment of identity-targeted messages that promote COVID-19 vaccination. We hypothesize that the average marginal effect of an additional identity-concordant message has a positive effect on an individual's intent to vaccinate. We further hypothesize that, among conservatives, an endorsement from Donald Trump is more effective than alternatives.

Methods
We conduct a randomized trial with online survey respondents. Following instructions and consent, we survey demographics, ask each respondent to read ten messages carefully to answer an incentivized question regarding message content, and finally elicit vaccination intention. The messages are randomly tailored to each respondent's segments.
For example, a Black respondent might receive a control message with a photo of Dr. Anthony Fauci (who is white), or a targeted message with a photo of COVID-19 vaccine co-developer Kizzy Corbett (who is Black). The text might refer to the average risk of COVID-19, or it might emphasize that African Americans are more likely to suffer from COVID-19.

Messages
Our baseline messages emphasize the health risks of COVID-19 and the safety and benefits of a hypothetical COVID-19 vaccine (Table 1). We randomized message components for specific segments (Appendix A). For respondents eligible for more than one message, we randomized the several message components with equal probability, balanced on segments. Nearly all messages were accompanied by photos. Importantly, either all possible treatments for a given component had corresponding photos or none did.

Danger of COVID-19
All respondents read, "COVID-19 has infected over 30 million Americans, leading to over 500,000 deaths." A random subset of Black and Latinx respondents also read about the higher impact on their The vaccine has been approved by a rigorous FDA process involving tens of thousands of people.

Trial results
This randomized trial found very high effectiveness and almost no serious side effects. Impact COVID-19 has infected over 30 million Americans, leading to over 500,000 deaths. Protection When you get vaccinated, you help protect yourself and the people around you from this virus.

Elders
We must protect our elders and get vaccinated! (Photo of an elder and a child.) Gatherings You can make up for missed get-togethers with friends and family once everyone has been vaccinated. (Photo of a wedding.) Availability The vaccine is available at your doctor's office and local pharmacies.
Note: See Appendix Table 5 for all treatment messages. community. A separate randomization of the religiously observant read that the virus has spread frequently in their place of worship (church, synagogue, mosque, or temple, each with an appropriate photo).

Vaccine safety
All respondents read, "The vaccine has been approved by a rigorous FDA process involving tens of thousands of people. This randomized trial found very high effectiveness and almost no serious side effects." African Americans and Latinx people were randomized to also read that the trial included people from their group. Fig. 1 depicts examples.

Parenting
Parents randomly received, "Children are at risk of long-term damage to their lungs and other organs. Nobody is sure how common or long-lasting this damage will be." A photo of children was included; Latinx parents saw children in a Hispanic parade.

Spillovers to the community
Infectious diseases have large negative externalities in communities. Thus, concern for others can be a major predictor of willingness to vaccinate. Everyone received, "The elderly are most at risk for COVID-19. Unfortunately, some cannot be vaccinated because of health conditions." This was followed with a randomized control message, "We must protect our elders and get vaccinated!" Parents randomly received this instead: "Imagine what you would feel like if you did not vaccinate your child, and then an elderly person in your home became ill." This included a photo of two grandparents playing with grandchildren. Conservatives randomly received this instead: "We share small-town values like caring for our neighbors-especially elders." Finally, a subset of religious respondents read this: "The Bible tells [Our holy books tell] us to care for those most vulnerable." Finally, everyone received the message, "When you get vaccinated, you help protect yourself and the people around you from this virus."

Benefits: Ending social isolation
Our control condition explains, "You can make up for missed gettogethers with friends and family once everyone has been vaccinated." Latinx respondents randomly received an accompanying photo of a quinceañera, celebrating the fifteenth birthday of a young Latina (see Fig. 2). Parents randomly received, "You can make up for missed children's parties and outings with friends and family once everyone has been vaccinated," alongside a photo of a children's party. Religious respondents randomly received, "You can safely attend [place of worship] with friends and family once everyone has been vaccinated," with a photo of the respective place of worship. All respondents then received, "These events will be so much nicer when they are safe." In an additional randomization, some religious conservatives received, "Freedom to go to church is the freedom to worship together, not infect each other!"

Availability
Respondents received a control message: "The vaccine is available at your doctor's office and local pharmacies." Some religious and parents also read that the vaccine is available at their place of worship or their child's school. These locations increase convenience, imply an endorsement by their religious group or school, and suggest that vaccines are normative for that group.

Language
Latinx respondents were randomly treated with Spanish parallel text for all messages received.

Other messages components
We additionally randomized the following treatments: • Conservatives randomly received, "When you get vaccinated, you help protect your body and your mind from this nasty and foreign virus." • Non-Black, non-Latinx conservatives randomly received, "Republican governors from Georgia to Ohio have stressed the economic and human cost of this pandemic."

Recommendations
Each respondent received an endorsement by a famous person such as Dr. Fauci, Donald Trump, Barack and Michelle Obama, a famous religious leader (e.g., the Pope), or a famous entertainer or athlete (e.g., Tom Hanks, LeBron James). Fig. 3 depicts example recommendations. Some endorsers were selected to be concordant on conservatism (e.g., Trump vs. the Obamas), identifying as Latinx (e.g., Hanks vs. Jennifer Lopez), identifying as Black, or religious affiliation (Appendix Table 6 and 7). We chose our recommenders from lists of celebrities from each segment, identifying those with a large social media presence or those recommended by consultants or pilot-survey respondents.
We gave each participant a set of messages they might receive, based on their personal characteristics. We then randomized messages. If neither Black nor Latinx but conservative, the set included Tom Brady. If not both religious and conservative, Tom Hanks. If neither conservative nor religious, LeBron James.

Pre-testing messages
We qualitatively tested messages with experts on each segment. We addressed both comprehension and suitability. We then conducted a quantitative pilot where respondents rated different messages.

The sample
We recruited United States residents through Prolific, which maintains a participant pool for web-based research and facilitates sampling stratified on participant characteristics.
We over-sampled individuals who had told Prolific they (1) identify as Black or African American, (2) identify as Latina/o/x or Hispanic, (3) either voted for Trump in 2020 or self-reported being "conservative" on a political spectrum, or (4) reported at least weekly participation in religious activities pre-pandemic. The screening questions are in Appendix E.
In April 2021, we invited participants who met our selection criteria to take an initial single-question screening survey: "Have you already taken a COVID-19 vaccine dose?" Appendix Table 8 describes respondent demographics.
We restrict our analysis to those without any COVID-19 vaccination who correctly answered an incentivized attention check. At that time, roughly half of American adults had received at least one vaccine dose. Appendix B contains details and a sampling pipeline diagram. We stopped recruiting for the study once enrollment plateaued (Fig. 6).
Appendix F describes consent, instructions, the manipulation check, and debriefing. 1

Outcome measures
Our primary outcome is the reply to: "How likely are you to take the COVID-19 vaccine described above?" Possible responses ranged from "highly unlikely" (coded as 1) to "highly likely" (7). We drop respondents who chose "Don't know/ prefer not to say" (N = 50, 1%). Parents also answered a similar question about vaccinating their child.

Statistical methods
We had intended to enroll 6,500 to 7,000 participants (at least 1,000 per segment). Similar studies (c.f., Freeman et al., 2021;Kreps et al., 2020) have found effects with comparable sample sizes. We were ultimately constrained by the relatively small size of the Prolific participant pool. Attrition during the sampling procedure was minimal (Appendix B).
We implement covariate-adaptive stratified block randomization given our five segments of interest, obtaining 32 strata ("subsegments"). Participants are at risk for multiple randomized treatment components given their subsegment membership. Each possible treatment is assigned with equal probability by Qualtrics survey software, maintaining balance.
Our main test examines willingness to be vaccinated depending on  the number of concordant messages. 2 We include separate intercepts for each subsegment, controlling for the respondent's maximal possible intensity of treatment. Student's t-test is then an exact test with the inclusion of subsegment fixed effects (Bugni et al., 2018). We drop subsegments with fewer than ten respondents (six subsegments, N = 17). We next estimate which message components matter. To reduce the number of tests, we consider bundles of message components-"Population tested in the trials," "Community impact," "Children affected," "Protecting the elderly," "Protection," "Elders," "Gatherings," and "Availability." We test the joint effect of all concordant messages received by each segment: Black or African American, Latinx or Hispanic, conservative, religious, and parents.
Last, our analysis plan pre-specified a test of whether Trump is a particularly effective endorser among conservative respondents.
Latinx individuals were relatively high (5.47), but Black individuals (5.00), the religious (4.89), and parents (4.63) showed lower willingness. Conservatives were the negative outlier (3.66). A full third (33%) of conservatives reported they were "highly unlikely" to accept the vaccine, more than twice the average. Those not in any segment had mean intention-to-vaccinate of 6.29, higher than the focal segments.

Do concordant messages increase likelihood to vaccinate?
Our sample for the experiment included 2,621 respondents who were members of at least one segment (mean membership of 1.62 segments). The mean number of identity-tailored messages possible for a participant was 5.16. Fig. 5 shows histograms of treatment intensity; Appendix C offers additional tabulations. Table 3 contains our primary results. Our analysis uses ordered-logit specifications (similar results using ordinary least squares available upon request). We find no evidence of a relationship between the number of concordant messages received and reporting a greater intention-to-vaccinate. Results for parents' intention-to-vaccinate child are similar in having a positive sign, a small magnitude, and lack of statistical significance.
We then tested which message components matter: if the vaccine was tested on a population including one's own group (pooling Black and Latinx segments); if the gatherings enabled by the vaccine are highly relevant to your group (pooling Latinx, conservative, religious and parent segments); "Impact" messages (including Church impacts); "Elders" messages; "Protection" messages; "Gatherings" messages; and "Availability" messages (Appendix Table 11). Consistent with Table 3, the coefficients are collectively not statistically significant (χ 2 8 = 5.39, p = 0.715). We next tested if concordant messages might matter for a specific segment (Appendix Table 12). There is no evidence that having concordant messages is statistically significantly useful for any of our five segments (χ 2 5 = 1.54, p = 0.908).

Does Trump matter specifically for conservatives?
Conservatives are the most vaccine-hesitant group. We pre-specified one celebrity endorsement as most important-the effect of Trump, who at times recommended vaccination. To reduce the number of subsegments and comparison recommenders, we focus on non-Black, non-Latinx conservatives. Results, with Trump as the baseline recommender, are shown in Table 4.
Conservatives The other possible recommenders were slightly less effective than Trump. The joint test shows Trump is distinct on average from the seven Notes: Standard deviations in parentheses. Intent of 1 corresponds to "highly unlikely" to vaccinate, while 7 is "highly likely." We intentionally over-sampled our demographics of interest, so our sample is not representative, and the means above are unweighted. Many respondents are in more than one segment (e.g., Latinx and Religious and Parent). Because respondents in different segments received different combinations of message elements, the means are not directly comparable. This table uses the sample for descriptive statistics (see Appendix Fig. 7 for the sampling flowchart). * For intent to vaccinate child, the sample is restricted to parents; accordingly "a member of ⩾1 segment" considers only the non-parent segments, as does "a member of no segments." 2 We pre-registered our study with the American Economic Association as AEARCTR-0007478 (Reddinger et al., 2021). We use Stata 17 and R 4.0.2 for analysis. Reddinger et al. (2022) provide data and source code.  In short, the results support the hypothesis of Trump's effectiveness with conservatives. Equally, Tom Brady and the Obamas appear roughly as effective as Trump, even among conservatives.

Summary
We surveyed 3,668 unvaccinated Americans in April 2021 about their likelihood of getting vaccinated, using messages with specific characteristics and celebrity endorsements. Our experiment involved 2,621 participants who were members of at least one of five important demographic segments-Black, Latinx, conservative, religious, and parents-when about half of American adults were unvaccinated.
As others have found, vaccine hesitancy is above average for Black and Latinx respondents and much higher for conservatives.
Contrary to our hypotheses, receiving more concordant messages regarding the vaccine had no detectable effect on stated willingness to vaccinate. Our sample size was large enough to detect effects (c.f., Freeman et al., 2021;Kreps et al., 2020) and Prolific is a well-respected subject pool. While our negative results could reflect methodological issues (limitations listed below), our results suggest any effects are modest at best.
In exploratory tests, no segment had a large benefit from concordant messages. Furthermore, no message element (such as dangers of COVID-19 segment-customized or having a recommender from the same segment) had a large effect.
With caution regarding multiple-hypothesis testing, we find mixed evidence that Trump is a particularly effective recommender for 1032 Notes: 95% confidence intervals in brackets using heteroskedasticity-robust standard errors. Each regression includes subsegment fixed effects. Outcome ranges from 1 (highly unlikely) to 7 (highly likely). Concordant score is the number of message attributes customized for that respondent's segment memberships, plus an additional unit if treated with Spanish parallel text if Latinx. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001.

PANEL B. MARGINS OF COEFFICIENT ON CONCORDANT SCORE
Δ Prob. of each reply: 1 is "highly unlikely to vaccinate," 7 is "highly likely"  Notes: 95% confidence intervals in brackets using heteroskedasticity-robust standard errors. Outcome ranges from 1 (highly unlikely) to 7 (highly likely).
Statistical tests comparing all pairs of recommenders are in Appendix Table 13. All risk sets included recommendations from Trump, Fauci, the Obamas, Johnson, and Brady. Religious Catholics also included the Pope, other religious included Warren, and non-religious included Hanks. † Recommenders and recommender risk sets with fewer than three observations dropped. ‡ Regressions include recommender risk-set fixed effects. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001.
conservatives, and a hint that Dr. Fauci is especially unconvincing for conservatives.

Implications
Despite our findings, it remains sensible to customize messages for segments.
In October 2021, Larsen et al. (2022) treated U.S. counties with a large-scale advertising campaign featuring a COVID-19 vaccine endorsement by Donald Trump on Fox News, finding evidence of increased vaccination at average cost of about $1 per vaccination. Other studies have also found Trump promoting the vaccine has a positive effect on intent (Kreps et al., 2020;Bokemper et al., 2021). While our evidence weakly supports the effectiveness of a Trump endorsement, it is not clearly more effective than all alternatives.
We attribute this discrepancy to the timing of the studies and the impact of the message. Kreps et al. (2020) and Bokemper et al. (2021) found Trump endorsement effective for a hypothetical vaccine during Summer 2020, months before the first emergency use authorization. We sampled unvaccinated respondents in April 2021, when half of U.S. adults had been vaccinated. Our sample was thus more vaccine-hesitant than these other studies by construction. Further, political discourse had galvanized beliefs and attitudes regarding vaccination, reducing the possible effect of our study. The success of the Larsen et al. (2022) trial is likely due to their video's effectiveness, in addition to their larger sample size. 34 If public-service messages like ours cannot overcome most vaccine hesitancy, more costly interventions may nevertheless be cost-effective. For example, perhaps personal communication from friends and family or from a family doctor is more important than marketing messages.
Moving beyond traditional social-marketing approaches, evidence generally supports the effectiveness of monetary incentives and lotteries (Campos-Mercade et al., 2021;Barber and West, 2022). Tying privileges, such as school enrollment or riding commercial airlines, to vaccination status may also motivate some people (Oliu-Barton et al., 2022;Mills and Rüttenauer., 2022).
We finally consider implications for theories of identity, which are supported by both many published studies and introspection. We worry that publication bias may lead to under-reporting of other negative findings. 5 Theories of identity are not always easy to exploit. We need much more research to explore the boundary conditions.

Limitations
The survey only reported on willingness to vaccinate, not vaccination.
In addition, the pool of Prolific respondents was not necessarily representative of their segments. Still, this is not a concern unless the resulting bias is correlated with treatment.
We defined membership in our "conservative" segment as either Trump voters or self-identified conservatives. Some Trump voters are not conservative, and vice versa.
Furthermore, our findings do not reflect the effects of any targeted messaging prior to our trial, since we collected data after half of American adults had already received at least one vaccine dose.
Finally, it is important to test additional message elements, more realistic messaging, more messengers, and in different regions.

Funding statement
The Center on the Economics and Demography of Aging (NIH 2P30AG012839), University of California, Berkeley, provided funding.

Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.  (Colvin, 2021). 4 Note that these authors only find significance at the 80% level with randomization-unit clustering. 5   When you get vaccinated, you help protect your body and your mind from this nasty and foreign virus. Gatherings Baseline You can make up for missed get-togethers with friends and family once everyone has been vaccinated. (Photo of a wedding.) Latinx Instead uses a photo of a quinceañera, a coming-of-age party for a young Latina.

Parent
You can make up for missed children's parties and outings with friends and family once everyone has been vaccinated. (Photo of a child's birthday party.)

Religious
You can safely attend church [synagogue / mosque / temple] with friends and family once everyone has been vaccinated.

Gatherings -Freedom
Christian and conservative Freedom to go to church is the freedom to worship together, not infect each other! Gathering safety All These events will be so much nicer when they are safe. Availability Baseline The vaccine is available at your doctor's office and local pharmacies. Parent Adds: …and your child's school.

Recommendation
All See Table 6 and 7.

Spanish language
Latinx Text of all messages also presented in Spanish, below the English.
Notes: If there is a "baseline" row for an element, then everyone received a message for that element. If there is no baseline message (those with an italicized name in the first column), then half of each eligible segment received a message, and half received no message for that element. If a respondent matched with more than one segment and message for a given element, then they were randomized with equal probability for all eligible messages. "Churches" changed to temples for Buddhists or Mormons, to synagogues for Jews, and to mosques for Muslims. "The Bible" changed to "Holy Books" if religious and not Christian or Jewish. Only those who report practicing at least weekly are at risk of religious messages.

Endorser Endorsement shown Endorser notability
Dr. Anthony Fauci* I have been waiting for this vaccine! I encourage everyone to get vaccinated as soon as possible!* "Director of the U.S. National Institute of Allergy and Infectious Diseases (NIAID) and the chief medical advisor to the president…. The New York Times described Fauci as one of the most trusted medical figures in the United States." Dwayne "The Rock" Johnson* I wish I had access to this vaccine before I was exposed to COVID. I encourage everyone to get vaccinated as soon as possible!* "One of the greatest professional wrestlers of all time …His films have grossed over …$10.5 billion worldwide, making him one of the world's …highest-paid actors." Donald Trump I would recommend the vaccine to everyone, especially people who voted for me and are reluctant. It's a safe vaccine and it's something that works.
Former U.S. president

Barack and Michelle Obama
The COVID vaccine is our best shot at beating this virus, looking out for one another, and getting back to some of the things we miss. Getting vaccinated will save lives-and that life could be yours.
Former U.S. president and first lady Tom Hanks* I wish I had access to this vaccine before I was exposed to COVID. I encourage everyone to get vaccinated as soon as possible!* "One of the most popular and recognizable film stars worldwide …Hanks's films have grossed more than $9.96 billion worldwide." Tom Brady* Get vaccinated so we can get our next season back to normal!* "Brady is widely considered to be the greatest [American football] quarterback of all time." LeBron James* Get vaccinated for our community. It is safe and will save lives!* "Widely considered one of the greatest players in [National Basketball Association] history …selected to the All-NBA Team a record 13 times" Kizzmekia "Kizzy" Shanta Corbett I could never sleep at night if I developed anything-if any product of my science came out-and it did not equally benefit the people that look like me. Period.

"Scientific lead of the [NIH Vaccine Research Center] Coronavirus Team …
propelling …a COVID-19 vaccine" Bad Bunny* I wish I had access to this vaccine before I was exposed to COVID. I encourage everyone to get vaccinated as soon as possible!* "The first Latin urban music artist on the cover of Rolling Stone magazine …Time magazine named him one of the 100 most influential people in the world on their annual list (2020)." Pastor Rick Warren* My flock works to protect our spirits and our bodies. This vaccine is essential for protecting our bodies.* "Founder and senior pastor of Saddleback Church, …the largest church in California …Named by Time as one of the '100 Most Influential People in the World.' …His books have sold over 30 million copies." Pope Francis I believe that morally everyone must take the vaccine. It is the moral choice because it is about your life but also the lives of others.
The Pope

Reverend Raphael Warnock
This pandemic isn't over yet and we must all stay vigilant to protect our community. Follow public health guidance, stay distanced and get the vaccine when you are eligible.  Notes: These demographic characteristics were volunteered to Prolific by the participants prior to our survey experiment; accordingly, these characteristics are underreported. All other tables use demographic characteristics reported in our survey. We recruited only U.S. residents with a 98% approval rate on Prolific. * Excludes any participant who shares an IP (Internet Protocol) address with another participant. ** Includes all participants who reported vaccination status, regardless of the number of surveys attempted, manipulation check, or vaccination intent.   Notes: We collected these demographic characteristics on our experimental survey. Notes: We collected these demographic characteristics on our experimental survey. Notes: 95% confidence intervals in brackets using heteroskedasticity-robust standard errors. Regressions include subsegment fixed effects. Outcome ranges from 1 (highly unlikely) to 7 (highly likely). Concordant messages are the number of message components customized for that respondent's segment memberships. Impact-Churches and Gatherings-Freedom were each a separate randomization, but analyzed as part of the Impact and Gatherings bundles, respectively. For example, everyone received one of the Impact messages, and half the Religious segment also received the Impact-Churches message. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001. Notes: 95% confidence intervals in brackets using heteroskedasticity-robust standard errors. Regressions include subsegment fixed effects. Outcome ranges from 1 (highly unlikely) to 7 (highly likely). Concordant messages are the number of message components customized for that respondent's segment memberships. + p < 0.10, * p < 0.05, ** p < 0.01, *** p < 0.001. Notes: 95% confidence intervals in brackets using heteroskedasticity-robust standard errors. Outcome ranges from 1 (highly unlikely) to 7 (highly likely). † Recommenders and recommender risk sets with fewer than three observations dropped. ‡ Regressions include recommender risk-set fixed effects. + p<0.10, * p<0.05, ** p<0.01, *** p<0.001 After the message intervention, participants were asked the following question as a manipulation check.

F.3. Debriefing
At the end of the experiment, all participants were debriefed with the following messages.
• "It is still important to take safety precautions after being vaccinated." • "These recommendations will change as more people are vaccinated." • "Please follow updates from your public health department and the Centers for Disease Control." • "COVID-19 vaccination site locations vary. Please consult your doctor or local public health department." • "Any quotations in this survey may have been ficticious." • "Thank you for taking our survey." • "Any bonus will be paid within 48 h."