The reception of public health messages during the COVID-19 pandemic

Understanding the reception of public health messages in public-facing communications is of key importance to health agencies in managing crises, pandemics, and other health threats. Established public health communications strategies including self-efficacy messaging, fear appeals, and moralising messaging were all used during the Coronavirus pandemic. We explore the reception of public health messages to understand the efficacy of these established messaging strategies in the COVID-19 context. Taking a community-focussed approach, we combine a corpus linguistic analysis with methods of wider engagement, namely, a public survey and interactions with a Public Involvement Panel to analyse this type of real-world public health discourse. Our findings indicate that effective health messaging content provides manageable instructions, which inspire public confidence that following the guidance is worthwhile. Messaging that appeals to the audience's morals or fears in order to provide a rationale for compliance can be polarising and divisive, producing a strongly negative emotional response from the public and potentially undermining social cohesion. Provenance of the messaging alongside text-external political factors also have an influence on messaging uptake. In addition, our findings highlight key differences in messaging uptake by audience age, which demonstrates the importance of tailored communications and the need to seek public feedback to test the efficacy of messaging with the relevant demographics. Our study illustrates the value of corpus linguistics to public health agencies and health communications professionals, and we share our recommendations for improving the public health messaging both in the context of the ongoing pandemic and for future novel and re-emerging infectious disease outbreaks.


Introduction
Since its outbreak in December 2019, COVID-19 has reinforced the importance of effective and timely public health messaging, which had to be swiftly adapted to reflect new and emerging scientific evidence related to different aspects of the virus and its transmission. The pandemic has brought to light the impact that communication technologies have in fostering misinformation and threatening public health on an international scale ( Naeem et al., 2020 ;Schild et al., 2020 ;United Nations, 2020 ), as well as the effects of social media exposure in exacerbating users' distress ( Gao et al., 2020 ;Holmes et al., 2020 ). Understanding the factors that influence health messaging reception is of paramount importance in supporting effective communications in the future. Over two years into the outbreak, we can now take stock of the challenges that have surrounded public health messaging during the pandemic and how different types of messages have been perceived by the public. This paper reports on the work carried out as part of the project Coronavirus -that is, messaging that encourages audience compliance. One of the 'benchmarks' of effective crisis communication is consistent messaging ( Seeger, 2006 ), however, consistency in messaging often proved challenging during the COVID-19 pandemic as message writers needed to reflect new and emerging evidence about the virus on a continuous basis.
Much of the public health messaging used during the pandemic drew on established health communication strategies, such as encouraging the public to engage in self-limiting behaviours to manage the impact of the virus. Below, we outline three health communication strategies that we examine in more detail in our investigation; these were sometimes combined within single messages throughout the pandemic.
First, 'self-efficacy messaging' provides specific harm-reducing instructions for people affected by underlying health conditions, for people interacting with those who are 'at risk' or, in those cases of a global threatening situation, for the general public. In providing specific information on how to reduce harm, self-efficacy messages offer a sense of control over the risk factors ( Seeger, 2006 , p. 242). As long as the reason for a suggested action is clear, the recommended action is meaningful, and the action has real and apparent utility in reducing the harm ( Seeger, 2006 ), self-efficacy messages can inspire readers' confidence that it is possible to collectively achieve a positive health outcome by adhering to the guidance they contain. Focusing exclusively on personal responsibility can be counter-productive, however, and has been associated with a higher risk of non-compliance (Institute of Medicine, 2015 ).
Next, fear appeals are 'persuasive messages that emphasize the harmful physical or social consequences of failing to comply with the message's recommendations' ( Berry, 2006 , p.109). Effective fear appeals emphasise the severity of the threat; they include evidence to demonstrate audience vulnerability and provide simple actions to counter the threat. The effectiveness of fear appeals is influenced by perceptions of the benefits of taking action, as well as internal (e.g., symptoms) and external (e.g., mass media campaigns) factors (ibid.).
Finally, moralising messages, which have traditionally been aimed at individual behaviours such as smoking, were also adapted for the COVID-19 context to promote the consequences of specific health risks. This messaging approach uses moralised persuasion, appealing to social values to influence social health norms ( Täuber, 2018 ). Psychological research on moral judgements has shown that, if prompted, individuals can interpret any topic in moral terms, which directly impacts on the evaluation of the issue ( Van Bavel et al., 2012 ). In contrast to nonmoral evaluations, moral evaluations are 'elicited faster, [are] more extreme and more universally prescriptive' ( Van Bavel et al., 2012 , p. 12). Moralising health messaging can contribute to othering and stigmatising of (supposed) non-compliant individuals as, once a topic has been construed as a moral matter, individuals are more likely to perceive it as 'a non-negotiable truth' ( Täuber, 2018 , p. 3). Such messaging, therefore, has the potential to jeopardise social togetherness.
Social unity against a health threat is required to achieve higher levels of measure adherence in a population ( Jetten et al., 2020, Tomasini, 2021, West-Oram, 2021. Factors that can challenge efforts to unify the public through encouraging participation in collective action include individuals' financial stability and underlying health conditions, as well as cultural backgrounds and religious beliefs that may be different from the main targeted population. Thus, building social cohesion requires acknowledging the different capabilities of the public ( Fuks et al., 2021 ;West-Oram, 2021 ) to support effective public health messaging.
The provenance of health messaging can also directly impact public adherence to measures. Recent studies have reported that scientific experts are more trusted than political authorities overall ( Coleman et al., 2020 ), making it essential to demonstrate that guidance is informed by scientific research. In the COVID-19 context, low trust in governments has been identified as a determining factor for the low usage of contacttracing apps (Altman et al., 2020; Bunker, 2020 ; Dowthwaite et al., 2021 ), together with a lack of trust in the companies that built the apps and the fear of being tracked (Altman et al., 2020;Dowthwaite et al., 2021 ). Public mistrust can originate from historical issues and perceived institutional racism, suggesting that official health message providers will benefit from collaborating with lay educators, who are respected by targeted audiences ( Crouse and Quinn, 2008 ). Public engagement can help health authorities better understand and address factors that condition trust and compliance, making it possible to implement more supportive health measures ( Fuks et al., 2021 ).

The language of COVID-19: linguistics and communication approaches to the analysis of health communication
Linguists and communication scholars have studied effective health communication extensively throughout the COVID-19 pandemic; notable projects include 'Communicating the pandemic: Improving public communication and understanding', by researchers at Leeds University; 1 'Cultural translation and interpreting of Covid-19 risks among London migrant communities', by researchers at SOAS; 2 and 'Covid-19 languages hub', by Oxford Languages. 3 While approaches differ, most researchers aim to better understand communication strategies and point out helpful lessons for future health threats.
Health communication research on the consumption of health information during the COVID-19 pandemic has involved public engagement methods such as surveys and focus groups. For example, Moss and Konstantinova (2021) find that people cited news fatigue as a reason for engaging less with COVID-19 news over the course of the pandemic. These results support Koh et al.'s (2020) study, which reports that the repetition of public health messaging and updates of deaths and infection rates can result in 'messaging fatigue'. Studies on health information consumption make it possible to differentiate six main groups based on the individuals' engagement with health communication, their confidence in official sources, and their response to the official information ( Coleman et al., 2020 ;Moss and Konstantinova, 2021 ). These groups are defined as follows: individualist risk takers, non-informationseeking sceptics, information-seeking rule-followers, complacently confident, information-seeking critics, and experientially risk-averse. As such, audience profiling can help health message providers better tailor their communications ( Coleman et al, 2020 , pp. 16-23).
Political discourses have received special interest in corpus approaches (e.g., Cavalieri and Corrizzato, 2021 ;Vincent et al., 2021 ;Vincent and Gardner, 2021 ;Power and Crosswaite, 2022 ), together with media representations ( Brglez et al., 2021 ), and public health messages ( Kuiper, 2021 ;Oakey et al., 2021 ). Studying Angela Merkel's communicative strategies, Jaworska (2021a) identifies building interpersonal relationships with the public as key, together with a factual presentation of information and emphasising the need for measure adherence and timely actions. Meanwhile, Yang and Chen (2020) adopt a corpus approach to explore discourses of globalism and nationalism in Chinese official texts, and Tay (2022)  press conferences of the World Health Organisation (WHO) and the Chinese Ministry of Foreign Affairs (CMFA), finding that whilst both adopt a neutral tone overall, the CMFA conveyed some inauthenticity by recurrently avoiding self-reference. Similarly, Sila š ki and Đurovi ć (2022) examine how governmental portrayals of the pandemic in Serbia were adjusted to the political agenda by adopting different metaphorical conceptualisations. Prior to election time, communications contained warrelated metaphors and the virus was presented as a personified 'enemy' to encourage adherence to health measures. Then, to encourage people to go out and vote in national elections, sports-related metaphors replaced the 'war' framing.
Scientific dissemination of COVID-19 information focusses on different audiences and communication types. Luzón (2022) explores the information provided in scientific 'explainers', finding that they serve informative and persuasive purposes, whilst Curry and Pérez-Paredes (2021) offer a contrastive corpus study of the use of stance nouns in COVID-19 related academic blog posts in Spanish and English. In relation to COVID-19 communication aimed at younger audiences, Muelas-Gil (2022) examines text and images in children's storybooks, highlighting the role of metaphor in explaining the pandemic and persuading children to act to avoid catching and spreading the virus. Elsewhere, Kondo (2022) examines online consultations between doctors and elderly patients during lockdowns in Japan, who reported potentially COVID-related symptoms such as 'cough', 'fever', 'fatigue' or 'shortness of breath ' (2022, p. 415). Findings show that empathy and compassion allayed patients' fears over (potential) symptoms and reassured patients that they did not need to visit hospitals in cases where visits were not deemed strictly necessary.
Pandemic representations in the media have been a key area of concern for corpus studies. For example, Dong et al. (2021) contrast media and academia responses to COVID-19, whilst Dayrell et al. (2021) offer a diachronic comparative study of the use of 'face masks' and 'face covering' in Scottish and UK national newspapers. Jaworska (2021b) also compares media representations of COVID-19, finding that different lexical choices made by media outlets across the UK, USA, and Germany influenced responses to the health crises. Elsewhere, a corpus linguistic approach has been applied to examine COVID-19 memorials posted on Remember Me 2020, a Church of England website ( McGlashan, 2021 ). The analysis in McGlashan's study focuses on co-occurring linguistic items and n-grams, showing that the memorials were underpinned by references to love, relationships, time and temporality, loss and absence, and memory.
Social media platforms can reveal public perceptions and understandings of the pandemic. Ho and Chiang's (2022) examination of the scapegoating of Wuhan escapees during the first lockdown on the Chinese microblogging site Weibo, explores how people were dehumanised and vilified via animal-related metaphors. Elsewhere, Vilar-Lluch (2022) studies how the perception of the pandemic changed during different stages of the first wave, as reported in Spanish and British readers' comments on news articles.
Social media research has generated a number of publicly available datasets. Wicke and Bolognesi (2021) have made their corpus of tweets available following their examination of recurrent topics and metaphors in COVID-19-related discourses on Twitter (see below). In relation to investigating the influence of social media platforms on public understanding of the health crisis, Kehoe et al. (2021) introduced TRAC:COVID, an open access online dashboard of Twitter posts (tweets). The tool enables the tracking of COVID-related conversations across Twitter, including the websites shared in the tweets, and offers visual representations of word usage and frequently shared websites. For example, one study of the social reception of the UK health measures reports polarised opinions amongst the British public, though support for official measures was found to be stronger than criticism ( Tkacukova et al., 2021 ). Also using TRAC:COVID, McGlashan et al. (2021) explores the promotion of anti-vaccination discourse finding an association between vaccine hesitancy and the promotion of misinformation and conspiracy theories.
The metaphorical framing of the pandemic has received special consideration. War-related metaphors have played a prominent role in pandemic communications and have been identified on social media (Twitter) in relation to treatment and diagnostics (Wicke and Bolognesi, 2021), in UK politics and news ( Musolff, 2022 ), and children's literature ( Muelas-Gil, 2022 ). The #ReframeCovid initiative ( Olza et al., 2021 ) emerged in response to researchers' concerns over the pervasiveness of war metaphors in official health discourse during the first stages of the pandemic. This open collaborative project makes a multilinguistic and multimodal collection of pandemic-related metaphors available to researchers. This has enabled the exploration of creative metaphor use ( Pérez-Sobrino et al., 2022 ) and a study that highlights the value of fire-based metaphors ( Semino, 2021 ), which were found to be versatile and more effective than the war framing.
These studies show the value of linguistics research in elucidating common health communication strategies and public perceptions of the pandemic and illustrate the value of corpus linguistics as a methodology for examining COVID-19 in its own right; however, it is increasingly common for corpora to be combined with other data sources to understand a topic from different perspectives in a so-called 'methodological triangulation' ( Baker and Egbert, 2016 p. 6). Corpus studies, by themselves, can reveal the strategies used in texts, but not how effective they are. We combine corpus linguistics with public feedback from surveys and public engagement (see 3.0) to examine the reception of key messaging styles used in official COVID-19 health messages. In doing so, we gain a better understanding of communication strategies that may elicit more positive responses and trigger behavioural change.

Material and methods
To explore what makes Coronavirus public health messaging effective, and to understand the reception of established messaging strategies that were utilised in the COVID-19 context, our approach combines corpus linguistic analysis with a public survey and interactions with a public involvement panel (PIP) to analyse real-world public health discourse. Specifically, we analyse public feedback elicited via responses to open text survey questions alongside responses to closed multiple choice survey questions.

Public involvement panel
Public involvement in research makes it possible to take into consideration observations from stakeholders that would otherwise have been overlooked ( Osmanlliu et al., 2022 ). Patient and Public Involvement Panels (PPIP) are an established method to promote public engagement in research, guidelines for which are available from researchers and official sources (see Greenhalgh et al., 2019 ;Ekezie et al., 2021 ;NIHCR, 2022 ;Osmanlliu et al., 2022 ). We adapted the guidelines provided by Ekezie et al. (2021 , p. 349) to engage with members of the public (excluding patients), who provided first-hand insights on the reception of public health messaging as the pandemic progressed. Our PIP comprises 12 members from different social backgrounds, no more than half of whom self-identify as White. Members acted as consultants and reviewers for our project from June 2021 until July 2022. We drew on the PIP's expertise and guidance to gain a better understanding of common information sources consulted by individuals about the pandemic, the impact of specific health messages, and the public's perception of effective health communication strategies, as well as to tailor survey questions (see 3.2 and 3.3.1). For example, through a Google Jamboard ( Varghese, 2016 ) activity ( Appendix 1 ), PIP members offered their views on key features of effective public health messaging, which formed the basis for a specific question in our survey. Insights were recorded over six two-hour sessions held over Microsoft Teams. Between four and six members of the research team were present at each meeting to observe, facilitate discussion and respond to PIP feedback. Unlike focus groups or survey respondents, PIP members are not study participants, and thus verbatim reports of PIP contributions have not been included in the results section.

Survey
Surveys enable researchers to gather insights into the behaviours and attitudes of the main group by questioning a representative sample of a particular population ( Dörnyei and Cszér, 2012 , p. 74). In studies of health message reception, surveys allow for direct insights into the public's response to different message types, their appraisal of the as-pects considered (e.g., changes in health guidance), and a better understanding of how the different receptions cohere with social demographics. On behalf of the University of Nottingham, Ipsos UK interviewed a nationally representative sample of 1,089 adults aged 16-75 in Great Britain. Interviews took place on the online Omnibus between 1st and 3rd March 2022. Quotas were set on age, gender, region, social grade, 4 and working status following the Random Iterative Model (RIM) (see 4 Social grade is a socio-economic classification provided by the UK Office for National Statistics (ONS). It distinguishes six categories of professional occupations: A and B (higher and intermediate managerial or administrative), C1 (supervisory, clerical and junior managerial or administrative), C2 (skilled manual occupations), D and E (semi-skilled or un-   Sharot, 1986 ) (see also Appendix 2 for demographic breakdown of the respondents). Data were weighted to the known offline population 5 for age, working status and social grade within gender and region to correct small scale imbalances in the profile achieved. Survey design and analysis was conducted by the University of Nottingham.
In line with Dörnyei and Cszér (2012 , pp. 75-80), our survey design considered: (i) sampling of the content (informed by the PIP), (ii) using multi-items scales (i.e., we examined certain messaging types with both closed and open questions to valence the potential influence of the wording of the survey in participants' responses), (iii) providing a variety of question types (Likert scales, closed and open questions), (iv) the wording of questions about COVID-19 messaging (co-produced with the PIP), (v) format of the survey (i.e., the survey was to be completed online and survey items were randomised). Before running the survey, questions were piloted and fine-tuned with the PIP members. The full survey is available in Appendix 3 . skilled manual occupations, unemployed and lowest grade occupations. See https://www.ukgeographics.co.uk/blog/social-grade-a-b-c1-c2-d-e . 5 According to the Office for National Statistics (ONS) https://www.ons.gov.uk/ In this paper, we report the results from five multiple choice questions (see Table 1 ) and four open text questions designed in complement to measure the reception of, and self-reported compliance towards, various messaging types. At the beginning of the survey, participants were shown the following scenario and framing in two consecutive display screens as shown in Table 1 .
Questions A through D ( Table 2 ) captured feedback on messaging that appealed to personal responsibility (A), messaging referencing social consequences (B), moralising messaging (C), and positive and negative framing of restrictive guidance (D). Participants were not exposed to information about the messaging types during the survey. Presentation of images for groups 1 and 2 were randomised and presented to half the participants (i.e., each participant saw only one variation) to measure whether small changes in grammar or vocabulary affected levels of self-reported compliance. Responses were recorded using a sevenpoint Likert scale ranging from extremely likely to extremely unlikely (presented in a randomised forward-reverse order), along with a 'don't know' response (anchored to the end). Question E, co-designed with the PIP, was shown to all participants and measured perception of message effectiveness. Participants had to select up to three of the features, which were presented in a randomised order. We also provided 'none of the above' and 'don't know' options. Questions F through G were Table 3 Open text survey questions and associated analyses  shown to all participants, whilst question H was asked on a filtered base (all adults who have received information about COVID-19 from one or more sources). Question F measured the degree of audience involvement with COVID-19 (understood as the extent to which the public had experienced significant consequences from the pandemic), and questions G and H considered public trust attribution to health communication sources.
In addition, four open text questions invited respondents to comment on examples of Coronavirus health messaging and current events. The details are provided in section 3.3 .

Corpus linguistic analysis of open text responses
Three open text survey questions invited feedback on different health messaging strategies ( Table 3 ). The first ( Fig. 1 ) is an example of a self-efficacy message taken from the 'Do Your Bit' campaign. It is NHS branded and provides specific harm reducing instructions with a rationale for compliance. It asks people to wear a mask, keep two meters distance, and wash their hands in NHS buildings to keep patients and staff safe. The second messaging ( Fig. 2 ) is an example of threat or fear appeal messaging, which promotes the message that everyone is at risk. This example comes from the 'Stay Home, Protect the NHS, Save Lives' campaign, where the instruction 'stay home' is provided as part of a slogan. The third message ( Fig. 3 ) contains moralising messaging from a series of 'look them in the eyes' posters taken again from the 'Stay Home, Protect the NHS, Save Lives' campaign. The series featured various individuals who had experienced poor outcomes from the virus. The final open text question sought public opinions on the (then) most recent easing of restrictions in England on 24 th February 2022. We compiled a corpus containing 35,257 tokens of public responses to these four open text survey questions, which were analysed verbatim. We observed that spelling errors were very infrequent, though punctuation and grammatical errors were more prevalent. We did not intend to grammatically tag the data for analysis, and we did not apply manual corrections to spelling in the data; however, all responses were read to confirm that the keywords we identified did not also appear in other responses in misspelled format. Responses were saved in individual .txt files, which were named according to question and response number so that individual responses could be traced back to demographic information and other survey responses if required. Table 4 shows the descriptive statistics for each of the (sub-)corpora. We used Sketch Engine (Kilgariff et al., 2014) to carry out a keywords analysis. Keyness is a statistical measure used to identify words that appear significantly more frequently than expected in a target corpus compared with a reference dataset. We used the .uk domain subcorpus from EnTenTen20 6 as a reference corpus for our corpora of responses to each open text survey question in order to understand 'the main concepts, topics or attitudes discussed in a text or corpus' ( Gabrielatos, 2018 , p. 225). Appendix 4 provides the top 25 keywords for each open text question calculated using the 'Simple Maths' method ( Kilgarriff, 2009 ). 7 The keywords were manually categorised according to theme, using the wider context to identify the dominant usage where necessary. We then carried out a micro-linguistic examination of full responses containing at least one of the selected keywords to draw out salient perspectives and responses to the messaging (see 4.1). We were especially interested in exploring the evaluative positioning of the respondents (argumentation strategies) and any evidence of discursive othering (see positive self-and negative other-presentation from van Dijk 2006 , and Yetkiner, 2021 's related discussion of us vs. them, good vs. bad, and superior vs. inferior) and the legitimisation of (non-)compliance by the survey respondents. This included examining how any social actors (e.g., politicians, respondents) were referred to in the responses, and whether they were associated with negative or positive consequences, traits, stereotypes, and evaluations.
The insights on the reception of public health messaging that we gained from this combined approach will be made available to public health message writers and policymakers to improve the uptake of messaging in the case of future disease outbreaks.

The reception of established messaging strategies in the COVID-19 context
The keywords analysis of the four open text survey questions revealed the dominant topics and attitudes in public responses to each question. Table 5 shows the classification of keywords by question. We examined the wider context of keywords denoting compliance, evaluation, and emotional states for the self-efficacy messaging; compliance, evaluation, and health states for the fear appeal messaging; and compliance, evaluation, emotional states, and health states for the moralising messaging. Finally, we examined the wider context of keywords denoting evaluation and feelings or emotional states in responses to the question on easing COVID-19 restrictions.

Self-efficacy messaging
Overall, our survey respondents were positive about the self-efficacy messaging in their open text responses to the 'Do Your Bit' campaign messaging ( Fig. 1 ). Respondents using keywords denoting compliance ('obey', 'comply', or 'adhere') usually testified that they are prepared to follow or have followed the guidance in this messaging. Fifteen of the 28 responses featuring the word 'comply' contained such personal testimony and one of the most prominent patterns in this data is the phrase 'happy to comply', which occurs in six of these responses. Self-reported compliance does not guarantee actual compliance, but we found evaluations of this messaging were generally positive. Respondents interpreted the tasks as easy to achieve, obvious ('I would, of course, comply'), and 'sensible' (3 instances), indicating that the rationale for compliance is clear.
There was limited resistance to the messaging in the extracts we examined: just two respondents used 'comply' to communicate personal resistance to the messaging, with one expressing concern over 'authoritarianism' and another reporting that they 'will not be complying with any of this'. Additionally, we identified some limited evidence of discursive othering. Two people commented on others' willingness to comply, suggesting that it may be difficult to get others to follow the guidance 'unless it is enforced'. It is possible that this resulted from the '#DoYour-Bit' slogan, which, whilst encouraging, presupposes that non-compliant individials are not acting as responsible team players. The slogan high-lights the role of individuals as part of a wider effort, however, which is in line with the recommendation to avoid attributing health outcomes exclusively to personal responsibility (Institute of Medicine, 2015 ). Respondents to question E (see 3.2) identified the characteristics 'memorable' (favoured by 23% of respondents), concise (favoured by 21% of respondents), 'achievable' (favoured by 15% of respondents) and 'encouraging (favoured by 12% of respondents) as the most important features of effective health messaging. This further underlines the overall suitability of the '#Do Your Bit' slogan. This further underlines the overall suitability of the '#Do Your Bit' slogan.
In terms of evaluative feedback on the quality of messaging content, our respondents reported finding it 'clear' (47 in total) and 'informative' (10 in total). For example, one person highlighted the importance of pictures as well as 'factual', 'clear', 'concise' words for people's understanding, whilst another liked the guidance because it was 'informative and not too judgemental', indicating that the lack of a moralising element was considered to be a positive quality. Conversely, seven respondents judged the messaging to be 'boring', which suggests that improvement could be made to the messaging design, though this could also indicate survey fatigue, or indifference towards the messaging content. Single word responses may also be a result of the legacy effect of messaging; comments such as '[t]hese are the same rules we've been following for 2 years and it's nothing new' demonstrate that consistency in self-efficacy messaging is recognised and understood, though not necessarily engaging, which may account for shorter responses. As a baseline, though, responses to this messaging demonstrated that our respondents were generally compliant -or at least were self-reporting as compliant -overall, which was valuable when contrasting with responses to the fear appeal and moralising messaging.

Fear appeals
In stark contrast, we found the fear appeal messaging to be ineffective and even problematic. Open text responses revealed polarised attitudes to this messaging. To illustrate, the keyword 'scare' was used in two very different ways: some respondents used it dismissively (as in 'Scare tactics which i [sic] wouldn't take any notice of', or 'It's just propaganda designed to scare people'), whereas others used 'scare' to report feelings of fear this image brings up for them, saying they felt 'scared [s]tiff' or they found it 'scary and frightening'. These responses show that this messaging either produced fear in people who read it or caused people to reject the messaging on the basis that it is propaganda designed to deceive them. This is also highlighted in responses containing 'scaremongering', which infers that the messaging is untruthful and something to be dismissed and in line with this, half of the people who used this response did not add anything more than this single word (4 instances). We also found evidence of possible messaging fatigue (someone said they were 'sick of scaremongering') and the suggestion that messaging is not landing as intended ('scaremongering to try to get the message across').
In general, respondents appeared to interpret the instructions correctly, but far fewer people self-reported that they would follow the guidance than for the previous messaging. Though employing fear as a strategy to encourage compliance was effective for a minority, this messaging had an emotional cost, which encouraged stigmatisation of non-compliant people. Motivations attributed to others' non-compliance include immaturity, lack of comprehension, belief in conspiracy theories, and ignorance. Three respondents in the extracts that we examined felt it would cause others to feel 'angry' or aggravated and therefore not listen to it. We also found derisory and hyperbolic responses to the moralising messaging, which misrepresent the guidance (e.g., 'Make you feel guilty for breathing fresh [air]'; 'I scoff at this as a cheap [trick]').
This messaging falls short on providing evidence for audience vulnerability as recommended (see 2.1), focussing instead on past indiscretions. It has been reported elsewhere that people sometimes avoided news about the pandemic to alleviate anxiety ( Moss and Konstantinova, 2021 ), which highlights the unsuitability of fear appeals in the Coronavirus context. It is a known challenge that some people will not comply with actions recommended to mitigate public health risks, (particularly if the recommendations relate to self-limiting behaviours) ( Hauser and Schwarz, 2020 ; Semino, 2021 ); however, ensuring messaging does not promote disengagement by generating additional anxiety is one way to increase compliance and uptake. The fear appeal messaging in Fig. 2 emphasises the severity of the threat and features a simple action to counter the threat (i.e., 'stay home'), but it is possible that as part of the wider slogan 'stay home' was not identified as an instruction.

Moralising messaging
Whilst high self-reported compliance was found for both versions of the moralising messaging in C ('I Wear This to Protect You', see 3.2 and Fig. 4 ), 8 the open text survey questions revealed a different picture. Here, a stronger example of moralising messaging ( Fig. 3 ) triggered a range of highly negative emotional responses. Respondents reported feeling 'sad' (28 instances, e.g., 'it makes me sad to see someone in that condition'), 'shocked' (4 instances, e.g., 'shocked by this and scared'), 'scared' (5 instances, e.g., 'I am scared'), 'uncomfortable' (2 instances, e.g., 'very uncomfortable') and 'harrow[ed]'. Occasionally, an emotional response caused people to want to comply, or at least self-report compliance (e.g., 'It is emotional and makes you want to comply'), but other responses were more troubling. It reminded one person of their family ('It's about your family and it's sad makes you upset'), which highlights a need to consider the effect that this kind of messaging has on a population, who may have witnessed illness and death in close proximity. This serves as a reminder that public health messaging has a responsibility to protect the mental, as well as physical, health of its audience.
Several respondents interpreted the message in Fig. 3 as one of blame directed at them (e.g., 'She's very ill because you ave [sic] not stuck to the rules'; 'This woman is ill and it's your fault'). Rule compliant people were upset at the implication -or perceived accusation -that they have not been following the rules (e.g., 'Makes me feel bad even though i [sic] obey the rules'; 'It is horrendous. It is making innocent people feel guilty and affecting their mental health'). One respondent described the moralising messaging as 'unnecessary emotional blackmail', whilst another felt the moral element had been weaponised against the audience ('I HATE IT because I think it is wrong to try to persuade others using guilt as a weapon'). This is further supported by 'guilt tripping', which featured 15 times. The guilt tripping is mostly agentless but occasionally an agent is named by respondents. For example, one identifies 'the government', whose motivation they believe is a blame-shifting strategy for poor handling of the pandemic early on. Interestingly, some of the favourable responses referenced 'guilt' and 'guilt tripping' as though it is a constructive approach to take. These people felt it was a good way of getting people to listen, or they hinted that they felt the guilt was deserved (e.g., 'It is trying to guilt trip people into following the rules […] sometimes this is the most effective way of getting people to listen'; 'it's trying to make people who don't go by the rules feel guilty. I'm good with it'). In fact, these responses contained lots of discursive othering realised through the stig-matisation of non-conforming people. There is a clear sense from rulecompliant respondents that the behaviours and beliefs of others are not aligned with their own, which emerges in the form of insults where noncompliant people are described as 'idiots' (e.g., 'Makes me cross when Covid-idiots won't obey rules meant to protect everyone'; 'She's ill, she needs to be protected, don't be an idiot, follow guidance'), or lacking in intellect ('Not sure that most of the selfish/thoughtless people in the UK have the intellect to understand the message'). The responses containing the keyword 'selfish' (13 instances) demonstrated just how divisive and polarising this material can be for some people (e.g., 'This is clearly aimed at those selfish individuals who think they know better and have no social care for anyone but themselves'; 'don't be a selfish twat'). The general sentiment is that rule breakers did so for their own personal gain and did not consider the effect this might have on others, particularly on the vulnerable. One respondent reported maintaining feelings of anger towards those who broke the rules ('I was and still am angry at the general public, for being so selfish that they choose to not follow the rules, in order to protect the vulnerable'). Importantly, none of the critical respondents identified themselves as vulnerable. Of 28 responses to this question containing the word 'vulnerable', just three specify the vulnerable party: two refer to the elderly in general, and one refers to a known individual ('my vulnerable partner'). Given Täuber's (2018) assertion that '[t]he moralization of health and lifestyle undermines social cohesion by creating a divide between those conforming to and those deviating from the health norm' (p. 12), it is unsurprising that we found polarised responses to this messaging as well as discursive othering as a result of the stigmatisation of non-conforming people.
Such strongly moralising messaging could be considered to be a catalyst for disharmony as it legitimises the othering of non-compliant people. This is clearly detrimental to the kind of social cohesion that is necessary for protecting public health in the context of a pandemic (or indeed responding to any crisis situation). Furthermore, there are key absences in this messaging. First, a lack of evidence was highlighted by respondents' unwillingness to take the messaging at face-value. Three respondents in these results did not accept that the woman pictured was suffering from COVID-19 (e.g., 'Scaremongering. Could be totally unrelated'; 'total nonsense, people get ill with all kinds of conditions and the lady in the picture might have emphysema for all we know'). The second complaint was a lack of instruction and practical advice. People were seeking constructive content and were not getting it from this messaging (e.g., 'Not constructive, no practical advice is offered'). Instead, it focusses on past behaviours and worse still, past errors or indiscretions.
We found that negative emotional reactions to the messaging were often linked to resistance and dismissiveness of its content and occasionally an outright rejection of messaging (e.g., 'Not interested, its [sic] blackmail'; 'big turn off'). Some of the respondents who were incensed by the content also misinterpreted the guidance. For example, one thought it meant a blanket rule that they could not go out ('this expects an emotive reaction of guilt in order to prevent you from going out. I HATE IT because I think it is wrong to try to persuade others using guilt as a weapon'), whilst another said 'Make [sic] you feel guilty for breathing fresh [air]'. It may be the case that these people are simply using hyperbole to make a point, but in some cases, it could be that misinterpretation is the cost of the unconstructive language used in the messaging. Several respondents referenced dominant conspiracy theories in their responses. For example, one person argued that COVID-19 was 'something that we know is not dangerous to virtually everybody' whilst another said they were 'fed up with covid', arguing that 'we dont [sic] behave like this with the flu'. Such rejections are likely linked to an amelioration of the potential severity of COVID-19, since the word 'flu' appears eight times in the context of being similar or worse than COVID-19 in our survey corpus (e.g., 'We have to get on with life and there has always been flu around'). It is particularly important that this is addressed whilst the long-term risks  appeal messaging, we found respondents were disengaging from this messaging because of its presentation, meaning that disengagement is to some extent avoidable here also.

Social factors affecting messaging uptake
Public reception of messaging varied, with some messaging 'landing' better with certain demographics. The following findings highlight the importance of tailored messaging and seeking public feedback to test the efficacy of messaging with relevant demographics.

Age as an indicator of compliance
We found age to be the most important indicator of compliance, with older people self-reporting as more compliant across all messaging types in closed survey questions (see Figs. 5 and 6 ). 9 Effect sizes were calculated for the overall model, which show that age can explain around 9% of the variance in participant's self-reported compliance.
Audience age is reported to determine the effectiveness of fear appeal messaging, since health promotion intentions reportedly increase with fear in older adults, whereas such health behaviours are found to be reduced in young adults (Institute of Medicine, 2015 ;Hale and Dillard, 1995 ). Though we can report qualitative evidence that responses of older respondents were more favourable for the fear appeal messaging ( Fig. 2 ), the costs of such messaging far outweighed this benefit since othering was more prominent in these responses.

Audience involvement
Our multiple-choice survey question results (see Question F, Table 2 ) also revealed that people with family members who had experienced significant consequences from COVID-19 self-reported greater compliance for the messages A, B, and C (see Fig. 7 ). 10 These individuals are said to have 'high involvement', which can increase willingness to comply 9 A Kurskal-Wallis test revealed a statistically significant difference between the age groups on overall compliance with Covid messaging 2 (5, 1013) = 96.9, p < .0001, r 2 = .09. 10 A Mann-Whitney U test shows that self-reported compliance transformed into a factor shows higher compliance overall (U = 116182.5, Z = 2.195, p = .028, r = 0.07) for the respondents who indicated Covid-19 has caused significant con- Fig. 6. Relationship between self-reported compliance and age. with health messaging. Responses to the two variations of messaging in each of these questions were combined as we identified no significant difference in self-reported compliance between the variations, indicating that self-reported compliance was not significantly influenced by sequences (Md = .281, n = 367) than respondents who didn't (Md = .137, n = 584). This is a small but significant difference between the groups. the language chosen (e.g., the modal verbs 'should'/'must' in image A). Indeed, an overall association of involvement with self-reported compliance was observed across all message types (see Fig. 8 ). 11 11 Mann-Whitney U tests were performed to estimate if respondents that self-identified to have had significant impact from Covid-19 also reported higher compliance with health messaging. These tests showed significant differ- Fig. 8. Relationship between audience 'involvement' and overall self-reported compliance.

Trust and credibility of the health communication source
Public trust and credibility in the source of the messaging emerged as a key issue affecting uptake. The effectiveness of the messaging -and subsequently, compliance with the guidance -is sometimes influenced by external factors, such as the popularity of the spokesperson or historical grievances. Survey respondents' second most highly rated characteristic of effective health messaging was 'from a reliable source' (35% of respondents favoured this).
The NHS source was a positive attribute in the self-efficacy messaging in Fig. 1 , as it is a trusted and credible source for most respondents. The hospital context was important to those weighing up whether they wish to abide by the rules and sometimes this was the deciding factor, as illustrated by this respondent, who said 'I would be happy to comply but only because it was a hospital', implying that they would not follow the measures in another context. The importance of the hospital setting of the messaging featured across responses containing the keywords 'adhere', 'obey' and 'sensible' ( Table 6 , extracts 1-10). The NHS logo did not improve reception of the fear appeal messaging, however, which indicates that messaging source cannot offset deeper issues associated with communication strategy.
The source of health communication has a bearing on messaging reception and public feeling about external issues may change the appropriateness and effectiveness of certain messaging strategies both as the situation progresses, and in future as a result of legacy effects. A factor analysis from closed survey question H (see Table 2 ) condensed the sources from which people preferred to obtain information about COVID-19 into five main 'factors', which showed some overlap in preferences: (i) traditional media (TV, radio, Government, mainstream news, ences between the groups in the messages of image A (U = 126902.5, Z = 2.285, p = .022), B (U = 132365.56, Z = 3.414, p < .001) and C (U = 127191, Z = 2.283, p = .022), as shown in the charts presented in Fig. 7 . print media, posters and medical professionals), (ii) interpersonal relationships (friends, family and colleagues), (iii) academic publications and conferences, (iv) social media (WhatsApp, Facebook, and Facebook direct messages), and (v) employers. Table 7 shows the characteristics of effective health messaging that were preferred by respondents in each of these five factors, which may be used for tailoring messaging according to channel of communication in future.
The Partygate political scandal 12 emerged as a major issue in the survey corpora. For example, the self-efficacy guidance was described in the survey as 'virtually unenforceable in the future thanks to Johnson and his corrupt régime' and the fear appeal messaging was evaluated as 'pretty ineffective coming from a government who don't obey the rules'. Moralising messaging prompted the most references to this issue (see Table 8 ): some respondents expressed their views on the moral character of those involved (line 11) and others felt the messaging highlighted hypocrisy, expressing strong criticism (lines 12 and 13). One person commented that the messaging content was laughable in the wake of Partygate (line 14), whilst another respondent felt that the shock value of the messaging was offensive in the same context (line 15).
Our results indicate that these perceived double standards dramatically impacted on the public's willingness to follow the guidance and act in self-limiting ways, and it is possible that the future use of moralising messaging in health communications will be inappropriate and ineffective in the UK context for this reason. You need to wear a mask, wash hands and socially distance as you are entering a clinical setting. I agree with this and would obey. 6 Take sensible precautions when visiting hospitals or other health care settings. Likely to do so, seems very reasonable to do so. 7 Its sensible advice for being in a hospital. 8 It is telling you that the hospital is being sensible and is asking all patients to do the same. I always pay attention to this but that is because I could become seriously ill if I do not, but unless it is enforced, I do not think the majority of people would comply. 9 NHS keeping sensible measures in place to protect the sick and vulnerable 10 Be sensible and take precautions to prevent the spread of the virus in hospital settings  To try to make the selfish who break the rules, like the PM and other members of No.10 feel guilty and my reaction is that this has happened and there are some selfish people out there including Boris Johnson 12 I'm shocked. Because a lot of us were obeying the rules, and members of parliament in power were not 13 Saying how people will try and get around the rules and make you feel guilty. find it quite ironic as the government are guilty of this 14 I would laugh at this after the way Boris Johnson has failed to follow the rules and has had numerous parties instead 15 I don't need to be shocked; I always try to do the right thing by other people. Politicians and civil servants appear to have completely ignored the rules, so I actually find this kind of shock tactic deeply offensive in that context Table 9 Responses to ending of restrictions in England in February 2022, keywords emphasised

Extract No
Response 16 Its too soon. I cant believe that were ending all restrictions its unbelievably stupid. The government has made it abundantly clear that it does not care for disabled and vulnerable people and that they care about the economy more than lives. We all new at the start of the pandemic that boris wanted to let it sweep through and get herd immunity, so he has ended this too quickly and people are going to die because of it. Its disgusting 17 It is ridiculous given the amount of cases and deaths still occurring. Cases are falling but that is because a lot of the tests have been scrapped. The Government once again panders to it's [sic] business buddies. 18 worried, the government don't know what they're doing and they don't care. They should have dealt with the pandemic in a better way in the first place, rather than being obsessed with the economy and living with covid. 19 That this was a stupid idea driven by Johnson to save his job and simple measures should have been kept in place as they made sense and still do 20 Removing the restrictions, particularly with cases in Omicron still high -and also cutting testing -is lunacy. It's begging another variant to develop. It's a political move and a very stupid one, made by a very stupid Prime Minister. 21 premature and irresponsible and PR exercise/gimmick 22 Its irresponsible but the government has zero credibility and people are cynical and resistive as a result. 23 I find it very irresponsible when the number of infected people and the rate of infection is already so high. It seems the Government has completely given up on controlling the spread of the virus, despite very little being known about the possible long-term effects of the current variant.

Public feedback on easing restrictions
The plan to relax restrictions in England in February 2022 was described in many instances as 'stupid', 'irresponsible' and 'ridiculous', but respondents also considered the social actors responsible for removing restrictions to be stupid: namely 'boris johnson ', 'Johnson', 'Prime Minister', 'boris', 'government', and 'tories'. Respondents attributed three main motivations to those responsible for relaxing the measures (see Table 9 ): economic gain, political gain, and a shift in public health strategy. Respondents felt that the Government had prioritised the economy over public safety (line 16) and that this was a direct result of Government relationships with people in business (line 17), or simply that the economy was being prioritised due to incompetence and indifference (line 18). Others believed it was driven by Boris Johnson to 'save his job' (line 19); simply as a 'political move' (line 20); or a 'premature and irresponsible' exercise in public relations (line 21). One person argued that a lack of Government credibility had led the public to become 'cynical' and 'resistive' towards health measures (line 22). In terms of public health strategies, one respondent felt that the Government had 'given up' trying to control COVID-19 (line 31); whilst another reasoned that Boris Johnson wished to revert to his 'herd immunity' strategy (line 23).
Such significant levels of speculation over the reasons for lifting restrictions indicates that the Government's rationale for relaxing the restrictions was not explained to the public in a way that resonated with them or otherwise the reasoning was not credible or understandable to them.
It is hoped that public health agencies can utilise our findings to tailor the features of their messaging for different audiences, taking into account the messaging strategy, audience age, and preferences for health messaging content according to the audiences' preferred channels of communication.

Reflections and implications
This investigation into the reception of public health messaging, which applies established health communications strategies in the context of COVID-19, enables us to make key recommendations for message writers, as we outline in 5.1. Further, our study has implications for applied CL, which we discuss in 5.2.

Reflections and recommendations
Our findings indicate that effective messaging will provide a clear rationale for adhering to measures and a means for the public to take personal responsibility to contribute to managing the virus. Provided instructions are well-defined and achievable, messaging containing selfefficacy guidance is generally well-received by the public. Importantly, though, the Institute of Medicine (2015) has warned against overreliance on self-efficacy messaging, arguing that guidance should avoid presenting the public as being exclusively responsible for the health outcome.
Whilst members of the public do seek evidence and/or a rationale for carrying out health measures, moralising or fear appeal messaging may not effectively satisfy this. Messaging that focuses on (uncertain) past rule breaking rather than constructive actions that people can take in future is not effective and risks non-compliance. When asking people to act in self-limiting ways, public health messengers must address people's concerns, questions, gaps in knowledge, and the emotional responses highlighted in our results.
We found little evidence that instructions contained within slogans (i.e., the 'stay home' instruction as part of 'Stay Home, Protect the NHS, Save Lives') are successfully identified as such by the public. Whilst slogans have their place in public health messaging, familiarity with these slogans possibly led to instructions being overlooked, or worse, people attaching additional meaning to them. Slogans should support or enhance self-efficacy messaging, but they cannot be the only content to offer instructions in a message. Where clear instruction was provided outside of a slogan, our respondents identified this and were generally accepting of it.
Some COVID-19 messaging runs the risk of negatively impacting on the mental, as well as physical, health of its audience. Messaging that provokes negative emotional responses can cause outright resistance or dismissal of guidance, which in turn affects willingness to act in selflimiting ways. This, in turn, provides an opportunity for public discourse that further entrenches the beliefs of those who reject the guidance to emerge. The moralising and fear appeal messaging styles produced strong reactions and very polarised responses, leading to the stigmatisation and othering of non-compliant individuals. Messaging that legitimises othering can ultimately diminish the social cohesion necessary for protecting public health in the context of a pandemic (or indeed responding to any crisis).
Parallels that we identified in open text responses to moralising and fear-appeal messaging may be due to the presence of a threat element in the moralising messaging, though this is less proximal to the audience than in the fear appeal example. The moralising messaging that we examined was effective at helping people to consider consequences, but this is not enough on its own. The negative effects of this messaging outweigh the benefit of encouraging people to think about consequences since moralising messaging can be a catalyst for disharmony. Combined with public dissatisfaction with political scandals, this raises questions over the appropriateness of using even mildly moralising messaging in the UK context in future.
Trust and credibility of the health communication source affected uptake. References to the NHS are positively received, and people are willing to comply with the guidance in contexts where messaging did not generate negative emotional responses. Furthermore, message writers should be aware that people draw on previous experience to contextualise change and in a fast-paced situation like the COVID-19 pandemic, this can become politically charged. Text-external factors such as Partygate and the political leaning of both the messenger and the audience contribute to determining public acceptance of a particular message. We found speculation about potential motivations for the relaxing of measures in England as the Government's rationale for lifting the measures was not explained to the public in a way that resonated with them. There is potential for misinformation to take hold in the absence of reasoned and scientific rationale and our study identified dominant misinformation such as 'COVID is flu', which should be acknowledged and addressed.

Implications for the future
We have shown that corpus linguistics methods are useful when it comes to analysing public health messaging, especially in contexts where they complement other methods. Our closed-ended survey questions measuring self-reported compliance generated a ceiling effect where people generally self-reported as compliant regardless of messaging type. The corpus analysis of the open-ended survey questions surfaced a more nuanced patterning in responses, and at a faster pace of analysis, than would be possible through manual thematic analysis of survey responses.
Although corpus linguistics can be applied to determine public opinion in direct feedback about any topic of interest, including public health, building a corpus of individual language use gathered specifically for feedback purposes is not always fast, or affordable. Gathering feedback from a PIP is faster but not necessarily representative, though it is a valuable tool for exploring qualitative patterns in the discourse with individual end-users. In circumstances where individual language use is readily available to researchers, it is not necessarily ethical to use it. UK legislation allows for its use for 'special purposes' including 'academic purposes' (Data Protection Act 2018, Schedule 174, condition 1b) or if its 'processing is necessary for reasons of public interest in the area of public health' ( Information Commissioner's Office, 2021 ). We gathered current insights, trends, and fast feedback from comments on news wesbites throughout the pandemic but did not report quoted examples in our public outputs. Individuals' online language use is (usually) intended to appear where they post it; people do not explicitly consent to researchers gathering their language data (and associated metadata) for external use.
Other corpus linguistics and discourse analytical studies have attempted to resolve potential privacy-related criticism by (i) quoting social media Terms and Conditions, (ii) gaining specific consent to reproduce text, and/or (iii) asking individuals to manually submit individual language data. Though these may have been the appropriate solutions available at the time of research, they might not go far enough to preserve privacy and can be time consuming and inefficient. This paucity underlines a need for technical and practical solutions for developing corpora of individual language use, and carrying out corpus-based analyses, in a privacy-preserving and privacy-enhanced way, which puts individuals in control of their own data.
To mitigate this challenge, we are developing a novel privacypreserving corpus linguistics browser extension, PriPA ( Clos et al., 2022 ). The tool has potential applications over a wide range of contexts, where language is both received and produced to gather public perspectives in a fast and scalable way. In the meantime, we have demonstrated how corpus linguistics can be used to extract feedback and areas of concern with traditional approaches to public health messaging, as well as barriers to comprehension and uptake when they are applied in fresh contexts. Our community-focussed approach can be applied to language data on any topic to reveal the opinions of social groups, the results of which can be used to provide more personalised feedback surrounding public health messaging.

Funding acknowledgement
This work was supported by the Arts and Humanities Research Council (AHRC) [grant number AH/V015125/1] and the Engineering and Physical Sciences Research Council 'Trusted Data Driven Products: EP/T022493/1.

Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Since March 2020, the British public has been asked to help prevent the spread of COVID-19 by wearing face coverings, washing hands regularly and maintaining social distancing in public areas. Boris Johnson recently announced the end of COVID-19 restrictions in England in February 2022.
Please now imagine that at some point in the future the Government re-introduces measures to minimise the spread of a new COVID-19 variant.

NEW DISPLAY SCREEN TO BE SHOWN TO ALL PARTICIPANTS
We'd like to show you a series of public health messages. For each of them please indicate how likely or unlikely you would be to comply if such measures were re-introduced as a result of a new COVID-19 variant.
So looking at the first public health message ...