Comment on ‘Assessing ExxonMobil’s climate change communications (1977–2014)’ Supran and Oreskes (2017 Environ. Res. Lett. 12 084019)

Supran and Oreskes (Environ. Res. Lett. 12 084019) employ a textual content analysis and comparison of 187 climate change communications from ExxonMobil (and its predecessor companies) to determine whether a discrepancy exists between published opinion pieces (‘advertorials’) and internal technical documents. Based on their analysis, the authors conclude that the company (ExxonMobil) misled the public. That conclusion is premised on at least two methodological flaws. First, the authors largely compared data from two different companies who were direct competitors to determine whether there was a discrepancy between them. Ignoring that before 1999 Exxon Corporation and Mobil Oil Corporation were two separate companies, the authors compare the internal documents of one company to the public statements of another in an effort to find discrepancies in the messages conveyed. Second, the publication assessed only a small subset of available advertorials. The authors note that ‘the company [Mobil] took out an advertorial every Thursday between 1972 and 2001’ or approximately 1560 times. Yet they chose to review only the 36 advertorials (or less than 3%) that were selected by another entity, Greenpeace, which has a well-documented history of animosity toward ExxonMobil. The authors’ reliance on limited data sets and their comparison of two unlike data sets call into question the publication’s conclusions.

In 'Assessing ExxonMobil's climate change communications (1977-2014)' [1], Supran and Oreskes claim that ExxonMobil 1 purposely deceived the public by communicating a position on climate change in a series of published advertorials that is inconsistent with other statements contained in internal documents and published in technical journals. To reach that conclusion, the authors conducted textual content analysis and compared 187 documents generated between 1977 and 2004.
The publication is premised on at least two methodological flaws. First, the publication largely compared data from two different companies to determine whether there was a discrepancy between them. This comparison ignores that before 1999, Exxon Corporation and Mobil Oil Corporation were two separate companies; they were incorporated and headquartered in different states and did not share management or employees.
It would be illogical therefore to compare the internal documents of one company during the premerger period (Exxon) with the contemporaneous public statements of the other company (Mobil). Yet the authors have done just that, obscuring the separateness of the two corporations by 'refer[ring] to ExxonMobil Corporation, Exxon Corporation, and Mobil Oil Corporation as "ExxonMobil"' throughout the publication and regardless of whether the companies were independent at the relevant point in time.
Tables 1-4 show the data set that was used for the publication. The vast majority of the public statements evaluated by the authors were issued solely by Mobil before the merger. Of the public statements, 25 of 36 (approximately 70%) pre-date the merger and were generated entirely by Mobil. Conversely, the overwhelming majority of internal documents from  times. Yet they chose to review only 36 advertorials (or less than 3%) that were cherry-picked by another entity, Greenpeace, an activist group engaged in a long running anti-ExxonMobil campaign. 2 This reliance on limited and non-representative data sets (generated with undisclosed selection criteria) further calls into question the validity of the data used to support the publication's conclusions.
Based on these two methodological flaws, it is clear that valid conclusions cannot be drawn from the data sets analyzed.
As further proof that the article is fundamentally flawed, at ExxonMobil's request, Dr Kimberly Neuendorf, a professor at Cleveland State University who developed the content analysis method the authors relied on and cited in their research, conducted a review of the publication [2], and found the content analysis contained 'numerous fundamental and fatal flaws.' Dr Neuendorf concluded the content analysis used in the publication 'is unreliable, invalid, biased, not generalizable, and not replicable.' Dr Neuendorf said the publication did not provide scientific support for either a discrepancy among ExxonMobil's climate change communications, or a claim that ExxonMobil misled the public.
In addition to data selection deficiencies, Dr Neuendorf identified defects in the coding of documents. According to Dr. Neuendorf, 'To maintain objectivity, content analysis coding ought to be conducted by coders who are at arm's-length with regard to the research.' Dr Neuendorf observed that the authors' 'selection of themselves as coders is inappropriate because they are not blind to the purpose of the research or independent of each other.' Dr Neuendorf 's comments on the process and the coding further reinforce our assertion that several errors were made in the process of developing the argument. The publication does not provide scientific support for either a discrepancy among Exxon-Mobil's climate change communications, or a claim that ExxonMobil misled the public.
In light of the authors' comparison of two unlike data sets, their reliance on limited and targeted data sets, and their questionable coding practices, the conclusions set forth in the publication cannot be credited.
Any data that support the findings of this study are included within the article.