Limited effects of exposure to fake news about climate change

The spread of ‘fake news,’ information that mimics credible reporting in format but not in content or intent, poses potential threats to public health and democracy by misinforming citizens. Understanding whether and how fake news influences individuals’ policy-relevant beliefs and decisions is needed to inform policies and practices to address it. In a preregistered experiment, we ask how exposure to fake climate news casting doubt on the existence of climate change influences individuals’ expressed belief in climate change, their estimate of the scientific consensus regarding it, and their overall trust in scientists. We find little effect of exposure to fake climate news on any of our three dependent variables. Effect sizes associated with exposure were very small, and demographics and political ideology were stronger predictors of beliefs. Our findings suggest exposure to fake climate news is unlikely to strongly influence climate skepticism.


Introduction
The spread of 'fake news,' information that mimics credible reporting in format but not in content or intent (Lazer et al 2018), poses potential threats to public health and democracy. Fake news is not a new problem: for example, conflicts over legitimate reporting and the Lügenpresse-or lying press-have their roots in the political upheavals across Europe in the 19th century (Beiler and Kiesler 2018). However, contemporary researchers are studying fake news with renewed urgency because the Internet and social media have enabled it to proliferate much more quickly and broadly (Vosoughi et al 2018, Lutzke et al 2019 and have thus increased its potential to harm both public health and democratic institutions such as elections (Allcott and Gentzkow 2017, Guess et al 2018, Grinberg et al 2019.
One issue for which fake news is particularly prevalent and potent is climate change. Coordinated misinformation campaigns have spread messages casting doubt on the existence of anthropogenic climate change (Oreskes 2011, Farrell et al 2019 and encouraging inaction on a global challenge that will have profound negative impacts on public health and the global economy (Intergovernmental Panel on Climate Change 2018). Understanding how fake climate news might influence individuals' policy-relevant beliefs and decisions is needed to inform policies to address it, and efforts by scientists and science communicators to refute it and spur climate action. Our study examined the effects of exposure to fake climate news on respondents' beliefs about climate change, viewing these beliefs as key antecedents to climate change-relevant decision-making and policy support (e.g. Bord et al 2000, van der Linden et al 2015. We randomly exposed participants to fake climate news and examined how exposure affects overall belief in the existence of climate change and related judgments of perceptions of the scientific consensus on climate change, and overall trust in scientists. Below, we summarize prior literature investigating the effects of exposure to misinformation, before describing the details of our experiment.

Exposure to misinformation
In order to test the effects of exposure to fake news, we draw on the psychological literature on misinformation, viewing fake news as a subset of the broader category of misinformation. In a common paradigm in the misinformation literature, participants read a series of messages describing an unfolding event, such as a fire (Wilkes andLeatherbarrow 1988, Johnson andSeifert 1994). Participants are given a piece of information pertaining to the fire, such as there being paint cans in the closet where the fire started, that is later revealed to be incorrect. These studies find that participants continue to mention the misinformation when asked about the event even after receiving the correction, termed the continued influence effect (Johnson andSeifert 1994, Lewandowsky et al 2012).
Subsequent research has identified factors that strengthen or weaken the continued influence effect. The illusory truth effect suggests that repeating a piece of information will increase its perceived truthfulness (Hasher et al 1977). Repeating misinformation leads it to be judged as more truthful, even if participants have the knowledge to determine its veracity (Fazio et al 2015). Repeated exposure to fake news causes it to be judged as more accurate , and repeating misinformation increases its continued influence (Ecker et al 2011). Limited evidence suggests that misinformation may have a stronger continued influence when it is consistent with one's beliefs (Ecker et al 2014); on the other hand, skepticism or mistrust of information is associated with a reduced continued influence (Lewandowsky et al 2012).
In addition to promoting skepticism, research has tested several other strategies to reduce the continued influence of misinformation (Lewandowsky et al 2012. A common strategy is to correct the misinformation post-exposure, known as debunking the misinformation. Research suggests that such corrections typically reduce (Lewandowsky et al 2012) but rarely eliminate (Wilkes and Leatherbarrow 1988) the continued influence effect. Corrections are equally effective whether they are made immediately after exposure or later (Johnson and Seifert 1994). Corrections are more likely to be effective when they go beyond refuting the misinformation to also provide the correct information (Mullet and Marsh 2016), an alternative causal explanation for the event (Johnson and Seifert 1994), or a causal explanation for why the information was incorrect (Rapp and Kendeou 2007).
Warning individuals that the information they are about to encounter might be inaccurate has also been found to reduce the continued influence effect. Ecker et al (2010) provided participants with either a general warning that they might be exposed to inaccurate information, or a specific warning explaining the continued influence effect in detail. They found that pairing either warning with a debunking was more effective in reducing the continued influence effect than the debunking alone, but that that neither warning-debunking pair succeeded in eliminating the continued influence effect.

Fake news
The internet, and in particular social media, have enabled the rapid spread of misinformation and provided a platform for malicious actors to purposely spread untrue information in what has been termed fake news (Lazer et al 2018). We view fake news as a subset of misinformation in that fake news mimics news published by reputable news media outlets, but is not generated through the same editorial processes used by reputable outlets, including thorough fact-checking and multiple layers of review. Recent research has found that the average American was exposed to roughly one to three news stories from publishers of fake news in the month before the 2016 American Presidential election (Allcott and Gentzkow 2017). Individuals are more inclined to believe fake news that is consistent with their political ideologies (Allcott and Gentzkow 2017), but those with a greater propensity to think analytically find fake news to be less accurate (Bronstein et al 2018), and are better able to differentiate real from fake news, regardless of its political leaning . Individuals with greater levels of delusional ideation and dogmatism, and religious fundamentalists, are more likely to believe fake news (Bronstein et al 2018).

Study design and hypotheses
While prior misinformation research has largely focused on the effects of exposure to misinformation in the context of fictional scenarios (e.g. Wilkes and Leatherbarrow 1988, Johnson and Seifert 1994, Marsh and Fazio 2006, Rapp and Kendeou 2007, Ecker et al 2010, or on the effects of exposure to misinformation on the perceived accuracy of that misinformation (Fazio et al 2015, relatively less research has examined how the effects of exposure to misinformation might spread to influence policy-relevant beliefs and attitudes. We randomly exposed individuals to genuine fake news headlines casting doubt on the existence of anthropogenic climate change, and examined the extent to which exposure impacted belief in anthropogenic climate change, and two less focal judgments: perceptions of the scientific consensus on anthropogenic climate change, and trust in scientists. We predicted that exposure to fake news would reduce belief in climate change, perceptions of the scientific consensus, and trust in scientists. Additionally, some participants were randomly assigned to receive either a pre-exposure warning or a post-exposure debunking. Based on prior literature (Lewandowsky et al 2012), we predicted that these interventions would reduce but not eliminate the effects of exposure. In addition to these preregistered research questions, we also conducted exploratory research testing whether, as predicted by prior research (Allcott and Gentzkow 2017), fake climate news would have greater effects amongst those for whom it is ideologically congruent, Republicans and conservatives (McCright et al 2016).

Method
We preregistered our study on the Open Science Framework. The preregistration and all study materials, data and code are available at: https://osf.io/pvnkj/.

Experimental design
Participants were randomly assigned to one of four experimental conditions. All participants read 6 headlines that had been independently verified as false by third party fact checkers from Snopes.com and Factcheck.org (figure 1). Participants saw one headline at a time; for each headline, participants answered a follow-up question unrelated to our research question, 'If an average American saw this headline, how likely would they be to read the news story associated with the headline?', meant to encourage them to read the headline. These data were not analyzed.

Assignment to condition
Participants in the Control condition read 6 headlines on topics from popular culture and were told before reading them that they were based on false information. Participants in the Warning condition received the same warning and read 6 headlines casting doubt on the existence of climate change. Participants in the Debunking condition read the climate change headlines; after reading the headlines, but before responding to the dependent measures, participants were told the headlines were based on false information. Participants in the Uncorrected condition read the climate change headlines but received neither a warning nor a debunking statement. Full instructions are located in the supplementary information.

Dependent measures
After reading the headlines, participants were thanked for their responses and informed that the next section of the study would seek their beliefs on current social and political issues.

Climate change belief
Participants answered six questions about their beliefs regarding American sociopolitical topics: gun laws, illegal immigration, Russian interference in the 2016 election, sports betting, tariffs, and climate change, administered on a 1-11 slider scale from strongly agree to strongly disagree. The target question on climate change, 'In 2016, over 100 countries signed an international agreement to reduce greenhouse gas emissions. Science suggests that greenhouse gas emissions from human activities are causing average global temperatures to increase, an idea known as human-caused climate change. Do you agree that human-caused climate change is happening?' was always asked fifth in the order; the other questions were administered in randomized order.

Climate change consensus
Next, participants were asked questions regarding their knowledge of current social and political issues. They estimated four proportions using 0%-100% scales: (1) The proportion of Americans that own a gun in America; (2) the proportion of the American national budget that goes to national defense; (3) the proportion of Americans who believe in God; and (4) and the percentage of climate scientists who consider climate change to be anthropogenic. The target question about the scientific consensus on climate change, 'As far as you know, what percentage of climate scientists say that human behavior is mostly responsible for global climate change?', was always asked third; the placement of the other three questions was randomized.

Trust in scientists
Participants next answered a question used on the General Social Survey asking about trust in different groups in America. Participants were asked, 'How much confidence do you personally have in each of the following groups to act in the best interests of the American public?' using a categorical scale with 4 levels: a great deal, a fair amount, not too much, and no confidence. The groups were, in order: the military, elected officials, scientists, business leaders, religious leaders and the news media. Because less than 3% of our sample indicated no confidence in scientists, we collapsed this category with those who indicated not too much confidence in scientists. This aspect of our analysis was not preregistered. An alternate model specification, following the preregistration, is reported in table S2; results were very similar.

Covariates
Participants responded to covariate and demographic items described in the supplementary information.

Participants and exclusion criteria
Participants were recruited using Qualtrics' online panel service. Power analysis and recruitment details are located the in supplementary information. Of the final sample of 1269 participants, 41% were male, and the mean age was 44.6 (SD=14.1). Two percent of participants had less than a high school education, 21% had graduated high school or achieved a GED, 24% had completed some college, 15% had an Associate's, 26% had a Bachelor's, and 13% a graduate or professional degree. Thirty-nine percent of participants were self-reported Republicans while 56% were Democrats. Similarly, 46% identified as liberals, 19% as moderates, and 34% as conservatives. Relative to the American population, participants in our sample were somewhat more likely to be female, liberal, and to hold a Bachelor's degree. Table S1 is available online at stacks.iop.org/ERC/2/081003/ mmedia and contains summary statistics. Table 1 reports mean climate change belief, climate change consensus estimate, and reported trust in scientists, by condition.

Climate change belief
A one-way ANOVA revealed no significant differences by condition on human-caused climate change belief, F (3, 1265)=1.47, p=0.22. Table 2 presents linear regressions predicting belief as a function of condition and covariates. Model 1 in table 2 presents a linear regression predicting belief in climate change as a function of experimental condition, with the control condition as the reference category and the three fake news conditions, Warning, Debunking, and Uncorrected, treated as dummy variables. Those in the Uncorrected condition reported less belief than those in the control condition, B=−0.494, p<0.05, though the overall model was not statistically significant. Model 2 includes covariates, and displays improved model fit, R 2 =0.42, F(14, 1254)=65.6, p<0.001. Accounting for covariates reduced the standard errors associated with the effects of condition. Those in the Warning condition reported lower levels of belief in climate change, B=−0.414, p<0.05, relative to those in the control condition, as did those in the Debunking (B=−0.411, p<0.05) and Uncorrected (B=−0.472, p<0.05) conditions.
Model 3 contains exploratory tests of whether the effects of condition differ amongst Democrat and Republican respondents, by including interaction terms between each of the three conditions and the Republican indicator variable, and examining only those participants who identified as Republican or Democrat in our survey. These interaction terms were not statistically significant, indicating that the effect of condition did not differ by political party. Overall, Republicans indicated less belief compared to Democrats. Model 4 tests whether the effects of condition differ by political ideology by including interaction terms between each of the three conditions and a centered political ideology variable, with higher values indicating more liberal respondents. Again, the interaction terms were not statistically significant, and more liberal participants indicated greater belief.

Climate change consensus
A one-way ANOVA revealed a non-significant difference by condition on estimated climate change consensus, F (3, 1265)=2.41, p=0.07. In Model 1 of table 3, those in the Warning condition reported lower estimates of climate change consensus, B=−4.492, p<0.05, relative to those in the control condition, as did those in the Uncorrected condition, B=−4.001, p<0.05; overall model fit was non-significant. Model 2 displays higher model fit, R 2 =0.21, F(14, 1254)=24.3, p<0.001, and shows that relative to those in the control condition, those in the Warning and Uncorrected conditions reported lower estimates of the consensus on climate change, B=−5.15, p<0.01 and B=−4.09, p<0.05, respectively. Including covariates decreased the standard errors associated with the estimates of condition. Model 3 indicates that the effect of condition on climate change consensus estimates did not differ across Republicans and Democrats; Republicans gave lower estimates compared to Democrats. Model 4 indicates that the effect of condition did not differ by political ideology; more liberal respondents reported greater consensus estimates.

Trust in scientists
A one-way ANOVA revealed no significant differences by condition on trust, F(3, 1265)=1.07, p=0.36. Table 4 presents ordered logistic regressions predicting trust as a function of condition and covariates. Model 1, predicting trust as a function of condition, shows poor model fit, McFadden's pseudo-R 2 =0.001. Model 2 includes covariates and displays improved model fit, McFadden's pseudo-R 2 =0.16, but no reduction in the standard errors associated with each condition. Relative to those in the control condition, those in the Debunking condition displayed lower trust in scientists, B=−0.388, p<0.05. Model 3 shows a significant interaction between the Warning condition and the Republican indicator variable, B=−0.759, p<0.05, suggesting that receiving the warning before reading the fake news reduced trust amongst Republicans, but not amongst Democrats. Republicans overall reported less trust. Model 4 does not find significant interactions between condition and political ideology; political liberals report greater trust.

Discussion
In a preregistered study, we examined the effects of exposure to fake news casting doubt on the existence of climate change. Participants were randomly assigned to see fake news unrelated to climate change (control condition) or fake climate news; of those who saw fake climate news, some saw uncorrected fake climate news, others received a pre-exposure warning of its falsity and others received a post-exposure debunking. We examined three dependent variables: belief in climate change, estimate of the scientific consensus on climate,   Note. Unstandardized coefficients from ordered logistic regressions predicting trust in scientists, assessed using a 3-point scale. Warning, Debunking, and Uncorrected conditions coded as dummy variables; the control condition is the reference category. Male coded as a dummy variable equal to 1 if self-reported male, otherwise 0. Republican coded as a dummy variable equal to 1 if self-reported Republican, otherwise 0. Model 3 includes only self-reported Democrat or Republican participants. In Model 4, ideology is centered; higher values correspond to more liberal ideology. OR=odds ratio. * p<0.05, ** p<0.01, *** p<0.001. and trust in scientists. Overall, means were similar across these four conditions, for all three dependent variables. After controlling for demographic covariates, we find that compared to the control condition, participants exposed to fake climate news reported lower levels of belief in anthropogenic climate change, and lower estimates of the scientific consensus; their trust in scientists was unaffected.
A pre-exposure warning of the falsity of the fake climate news and a post-exposure debunking largely failed to eliminate these effects of exposure, consistent with prior work (Lewandowsky et al 2012), and in some cases appeared to enhance the effects of exposure: receiving a warning decreased consensus estimates even more than receiving uncorrected misinformation (table 3), and receiving a debunking decreased trust in science but exposure to uncorrected misinformation did not (table 4). These findings are consistent with prior work documenting a backfire effect in the correction of misinformation (Nyhan and Reifler 2010, Nyhan et al 2013, but see also Wood and Porter 2019). While these findings may be spurious, they may also indicate that warnings and debunkings changed the way participants perceived the misinformation and the dependent measures in ways we did not predict, suggesting the need for further research identifying the conditions under which corrections backfire.
Though the effects of exposure were statistically significant after including covariates, these effects were very small: exposure was associated with a partial eta squared of 0.004 to 0.005 for climate belief and 0.001 to 0.007 for consensus estimates, and odds ratios of 0.68 to 0.84 for impacts on trust (Model 2 in tables 2-4). Such small effect sizes may stem from participants' limited exposure to fake news: they read a total of 6 short headlines. However, these effect sizes were associated with regression coefficients with reasonably large magnitude in terms of the scale of the dependent measure, suggesting substantial variation in judgments. Additionally, our experimental exposure to fake news was consistent in magnitude with estimates of real-world exposure to fake news: Allcott and Gentzkow (2017) found that the average American was exposed to only one to three fake news stories in the month before the 2016 American Presidential election. Future work might examine the effects of prolonged exposure. Given the small political margins that seem to be increasingly pervasive in American politics (Smidt 2017), these small effect sizes may be practically meaningful.
However, the overall weak effects of condition suggest that while exposure to fake news may once have been a driver of climate skepticism (Oreskes 2011), here its effects were overshadowed by those of political party and ideology. Republicans/conservatives were less likely than Democrats/liberals to indicate belief in climate change Prior research has found that that individuals are more inclined to believe misinformation that is consistent with prior beliefs (Allcott and Gentzkow 2017), suggesting that any effects of exposure should be stronger amongst Republicans and conservatives. However, we found that exposure to fake climate news had similar effects amongst Democrats and Republicans, and conservatives and liberals, suggesting that misinformation may influence one's beliefs even if the information is not aligned with those beliefs.
Our research is subject to several limitations. We examined the influence of fake news on a single topic, climate change, in terms of its effect on a limited set of beliefs. While prior research suggests that holding a correct understanding of climate change and the scientific consensus on climate change is associated with greater support for climate action (Bord et al 2000, van der Linden et al 2015, research on the determinants of pro-environmental behaviors also suggests the existence of a 'value-action' gap through which environmental concern does not always translate into personal or policy action (Frederiks et al 2015). Future work should expand our research to examine the effects of exposure to fake news on behaviors and attitudes toward specific climate policies, separating the effects of exposure on climate skepticism (studied here) and climate inaction (not studied here).
Additionally, prior research has found that warnings can succeed in reducing the effects of exposure to misinformation when they 'inoculate' participants against the misinformation by exposing participants to weak examples of the misinformation and then refuting them (van der Linden et al 2017, Cook et al 2017). Our warning may have been unsuccessful because it did not contain such an inoculation. Our manipulations also did not directly refute the fake news headlines to which participants were exposed. However, as a result of the limited effects of exposure to fake climate news observed, our study was not ideally positioned to test these manipulations. Future research might expose participants to articles rather than headlines, and adopt experimental designs that enable a fuller accounting of participants' prior beliefs, for example eliciting beliefs pre-and post-exposure.
Finally, the extent to which we were able to observe potential effects of exposure to misinformation was limited by the wording of our dependent measures. Our measure of climate change belief described climate change in relatively mild terms; participants may be more likely to indicate belief in climate change when it is described as less threatening and thus our measure may have inflated belief in climate change and reduced the extent to which we were able to observe an effect of exposure. Similarly, our trust measure asked about perceptions of the intentions of American scientists in general, and did not ask specifically about climate scientists or the accusations that have been leveled against them, such as falsifying data (see the headlines used in our study in figure 1). We may have been more likely to observe effects of exposure had we used more targeted measures of trust in climate scientists. Future research could employ dependent measures capturing additional elements of participants' climate beliefs shown to be important for their actions and reception of climate information, such as their certainty that climate change is occurring (Maibach et al 2009).
Overall, we find that the effects of exposure to fake climate news, where they exist, are small, and affect more focal judgments (belief in climate change, estimates of the scientific consensus on climate change) more strongly than less focal judgments (trust in scientists). Ideology doesn't seem to enhance these effects, despite the political polarization surrounding climate change. Preemptive warnings and after-the-fact debunkings are largely ineffective at reducing these effects of fake news. However, interventions designed to facilitate critical thinking about the content of fake news, and whether to share it, have shown promise in terms of limiting its spread (Lutzke et al 2019, Roozenbeek and van der Linden 2019). These findings underscore the importance of implementing countermeasures that promote greater scrutiny of online news, along with policies aimed at reducing the spread of misinformation, for slowing the spread of misinformation.
Our findings suggest exposure to fake climate news is unlikely to strongly influence climate skepticism. They are consistent with recent work suggesting that fake news may not be shared because it is thought to be informative; rather, its creation and sharing may serve to signal one's identity and group membership (Mercier 2020). Future efforts to understand and combat climate skepticism might focus on unpacking the relationship between climate beliefs and political ideology, for example by focusing on how Republican/ conservative skepticism of climate change may have to do with an aversion to the policy solutions proposed to address climate change (Campbell and Kay 2014).