THE DIVERGENCE OF ESG RATINGS: AN ANALYSIS OF ITALIAN LISTED COMPANIES

The increasing attention to sustainability issues in ¯nance has brought a proliferation of en- vironmental, social, and governance (ESG) metrics and rating providers that results in divergences among the ESG ratings. Based on a sample of Italian listed ¯rms, this paper investigates these divergences through a framework that decomposes ESG ratings into a value and a weight component at the pillar (i.e. E, S, and G) and category (i.e. sub-pillar) levels. We ¯nd that weights divergence and social and governance indicators are the main drivers of rating divergences. The research contributes to develop a new tool for analyzing ESG divergences and provides a number of recommendations for researchers and practitioners, stressing the need to understand what is really measured by the ESG rating agencies and the need for standardization and transparency of ESG measurement to favor a more homogeneous set of indicators.


Introduction
Sustainable Responsible Investment (SRI) has grown signi¯cantly in recent years, reaching $35.3 trillion in assets under management at the start of 2020, representing an increase of 15% in the last two years and 55% in the last four years and accounting for 36% of total assets under management (GSIA 2020). The United States and, especially, Europe are the leading markets, representing more than 80% of global sustainable investing (GSIA 2020). Environmental, social, and governance (ESG) issues have been more and more included into investors' portfolio selection strategies, through the use of positive or negative screening criteria or explicitly factoring ESG principles into their¯nancial choices (Cellier et al. 2016). Thus, a key requirement of SRI is the access to high-quality sustainability-related data, ratings, and methodologies (European Commission 2020). Nevertheless, the ESG rating industry is largely characterized by the lack of transparency (Chatterji et al. 2009), clear disclosure rules, and rating standardization (Escrig-Olmedo et al. 2014, Bolognesi & Burchi 2021, Conca et al. 2021, contrary to¯nancial reporting and credit ratings. Following this increasing attention to sustainability and social issues, a proliferation of rating providers has characterized the last decades with the development of di®erent SRI products and services, including raw sustainability data, ratings and rankings, indices and benchmarks, consulting services, and reporting practices. Simultaneously, the ESG industry has seen a trend of consolidation with a few big players that, through a series of mergers and acquisitions, currently dominate the ESG rating market (Avetisyan & Hockerts 2017). MSCI, Re¯nitiv, Bloomberg, Sustainalytics (recently purchased by Morningstar), S&P Global, VigeoEiris, and Inrate are some of the most prominent¯rms, which are based in Europe and in the United States. Given the increasing relevance of sustainable investment, the wide number of players with proprietary methodologies, and the lack of strict rules to be followed, the inevitable consequence is a variety of ESG ratings and measurement qualities that di®er in their dimensionality, reliability, and construct validity (Widyawati 2021). Furthermore, ESG is a concept based on continuously changing indicators and often qualitative information (Paltrinieri et al. 2021).
From the previous considerations, it is evident that we need to better understand whether and how the ESG ratings provided by di®erent rating agencies di®er and, above all, the underlying drivers of this divergence in order to e®ectively advise investors and companies about ESG performance. So far, researchers investigating ESG ratings have found mixed results and have mainly looked at aggregated metrics. The convergent validity of the same environmental indicator has been found to have a degree of convergence by some studies (Semenova & Hassel 2015) and a degree of divergence by others (Hedesstr€ om et al. 2011). Conversely, an assessment of aggregated ESG data reveals a low level of convergence among di®erent ratings (Dor°eitner et al. 2015, Chatterji et al. 2016. Furthermore, most prior studies on ESG measurement provide an in-depth, but incomplete examination of validity and reliability issues, with research on MSCI (previously MSCI KLD) ratings dominating the¯eld (e.g. Mattingly & Berman 2006, Chatterji et al. 2009, Delmas & Blass 2010, Kang 2015, Mattingly 2017, with only a few studies comparing ESG ratings of di®erent providers (Berg et al. 2020, Widyawati 2021. The aim of this paper is to examine the di®erences in ESG ratings across a broader sample of rating agencies and understand the sources of divergences. Only a few papers have established a quantitative framework for assessing those disparities. The study most similar to ours is Berg et al. (2020), which identi¯es three causes of ESG rating divergences (i.e. scope, measurement, and weights).
Using the ESG ratings of Italian companies issued by six leading international ESG rating agencies, i.e. MSCI, Re¯nitiv, S&P Global, Inrate, Arabesque S-Ray (hereafter Arabesque), and Truvalue Labs (FactSet) (hereafter Truvalue), we develop a quantitative framework to study the ESG rating divergences. The framework adopts a top-down approach, starting from the overall ESG scores and examining the contribution of each pillar [Environmental (E), Social (S), and Governance (G)]. Divergences are further investigated by decomposing the score into value and weight components of aggregate ESG ratings and at the pillar and sub-pillar levels (i.e. for di®erent categories).
Our main results can be summarized as follows. First, the weight component, in most of the cases, explains the highest percentages of disagreement across ESG ratings of di®erent agencies. Second, considering the pillar level, the divergences related to the Environmental pillar are the lowest ones. On the other hand, the Social and Governance pillars explain a higher percentage of divergences. Third, going deeper at the category level, the analysis reveals that the level of divergences is mainly due to the weight component and it is more relevant for the Governance pillar categories.
We contribute to the emerging research that has documented the divergence of ESG ratings (Windolph 2011, Chatterji et al. 2016, Brandon et al. 2019, Berg et al. 2020, Widyawati 2021. Our main achievement is to explain why ESG ratings diverge by contrasting the underlying methodologies through a replicable framework and quantifying the main sources of divergence. Our research also provides important empirical foundation for future studies that aim to investigate the relationship between ESG performance, rating disagreement, and stock price performance.
The paper is organized as follows. Section 2 presents the sample used for the analyses and documents some preliminary evidences on the divergence of aggregated ESG rating. Section 3 develops the quantitative framework and illustrates the¯ndings of ESG rating divergences in terms of value and weight components, documenting the discrepancies at the pillar and category levels. Finally, Sec. 4 concludes, highlighting the implications of our research as well as future research directions.

Research design and sample
ESG ratings initially appeared in the 1980s as a means for investors to evaluate¯rms based on factors other than solely¯nancial performance, such as the social and environmental performances. Since then, the increasing focus on ESG investing has led to the rise in the number and in°uence of ESG rating agencies (Lopez et al. 2020). According to Li & Polychronopoulos (2020), there are more than 70 di®erent¯rms around the globe that provide some sort of ESG scoring data. However, each ESG rating agency has developed a proprietary methodology with speci¯c steps followed in the assessment of rated¯rms. As a result of this variety of approaches, ESG ratings typically are con°icting and are often not comparable due to discrepancies in de¯nitions and evaluations of ESG constructs. In recent years, the ESG industry has also seen a consolidation tendency that, however, had less to do with best practices and more to do with the strategy of increasing market shares through mergers and acquisitions (Dimmelmeier 2020).
We use data from six di®erent ESG rating providers: MSCI, Re¯nitiv, S&P Global, Arabesque, Truvalue Labs, and Inrate. a Together, these rating agencies are major players in the ESG rating space (Eccles & Stroehle 2018) and cover a substantial part of the overall market for ESG ratings. Overall ESG scores, single-pillar (Environmental, Social, and Governmental) scores, and category scores (comprehensive of values and weights for each pillar and category) were retrieved from public sources and proprietary databases for a sample of 210 Italian¯rms listed on the stock exchange in the years 2019 (188¯rms) and 2020 (182¯rms). b ESG rating score is the general judgment assigned to a company's ESG performance. Pillar scores are assigned to each pillar namely E, S, and G; for instance, the E score refers only to a company's environmental performance and is the summary of the performance of di®erent categories, such as pollution, energy consumption, GHG emissions, etc.
In order to operationalize our model, we had to introduce some adjustments to the raw data. First, we consider the average of the ratings issued during the year when available (i.e. Arabesque, MSCI, and Truvalue) or the rating scores provided at the end of the year when only these¯gures are available (i.e. Re¯nitiv, Inrate). Second, di®erent rating scores assigned by di®erent agencies are translated into a homogeneous scale ranging from 0 to 100. Then, speci¯c assumptions are applied to the providers. In detail, Inrate does not disclose single-score weights. Thus, we estimate them through multiple linear regression models applied to all¯rms covered worldwide by Inrate analysts, controlling for di®erent industrial sectors. MSCI provides two scores: MSCI Industry Adjusted ratings and MSCI Weighted ratings. The latter computes the weighted average of pillars and respective weights; therefore, it is the only one included in the analyses at the pillar and category levels. Finally, Truvalue does not provide the segmentation in pillar scores, but only scores of the di®erent categories, which were therefore grouped into the three main pillars of E, S, and G. More in detail, Truvalue uses Dynamic Materiality percentages; this methodology consists in tracking company data (i.e. number of news) tagged to a speci¯c category over the last 12 months (Truvalue Labs 2020). We use these a We were not able to obtain access to granular data reported by Sustainalytics, which is also a major player in the arena. We thank Arabesque and Inrate for disclosing relevant information for our work. b Some¯rms are added between 2019 and 2020, while others are no longer evaluated by agencies. percentages as category weights, which are summed up to obtain the overall weights for di®erent pillars. Table 1 provides descriptive statistics of the aggregate ratings and their sample characteristics. The baseline year of our analysis is 2020. We tested whether our results are speci¯c to the year of the study by rerunning the analysis for the year 2019 and obtained similar results. c Panel A of Table 1 shows the full sample of¯rms rated by any of the six agencies: this number ranges from 36 to 147. Panel B of Table 1 limits the sample to 22¯rms that have been rated by all the agencies. The latter are some of the largest publicly traded Italian companies, for which the transparency and the availability of ESG information are expected to be better. The mean and median ESG ratings are, in fact, higher in the common sample for all rating providers. We may observe that the mean ESG rating issued by di®erent agencies is quite di®erent: Re¯nitiv has the largest average score, while S&P has the lowest. If the di®erence can be explained by the change in the sample of the covered companies, this possible explanation cannot be applied to the restricted sample (Panel B of Table 1), where we also see relevant di®erences: the average scores range from 49.62 (Re¯nitiv) to 74.69 (MSCI Industry Adjusted).

Correlations and disagreement analysis
In this sub-section, we illustrate the extent of divergence between di®erent rating agencies through a correlation analysis. Table 2 shows the Pearson's correlations between the aggregate ESG rating scores for the full sample (Panel A) and for the common sample (Panel B). It is evident how correlations are low for the majority of c This analysis is not included in the paper, but available upon authors' request. the pairs of rating providers, ranging from 0.03 to 0.64 for the full sample. The average correlations are 0.32 and 0.41, respectively, for the full and common samples. S&P Global and Re¯nitiv show the highest level of agreement between them with a correlation of 0.64 in the full sample and of 0.68 in the common sample. Table 3 reports the correlations for the three di®erent pillars (E, S, and G) for the full sample, while Table 4 shows the same metrics for the common sample. We underline again that MSCI provides this speci¯cation only for the MSCI Weighted score. Correlations are con¯rmed to be quite low, especially for the Governance pillar that shows the lowest correlations, with an average coe±cient of 0.09 for the full sample and of 0.06 for the common sample. The Social and Environmental pillars show higher correlation values. The Social pillar correlations are even slightly higher than the overall ones (0.35 and 0.43). The average correlations for the overall ESG rating score and for the three pillars are summarized in Table 5. These results are largely consistent with prior¯ndings (Chatterji et al. 2016, Berg et al. 2020.

Quantitative Divergence Framework
We now turn to the development of our quantitative divergence framework. ESG ratings are scores that combine a variety of parameters (or indicators) into a single number that is used to assess a company's ESG performance. Technically, such a rating can be expressed in terms of a measurement (or value) and a weight component. Measurement refers to the indicators that are used to produce a numerical value for each attribute. Weights refer to the function that linearly combines multiple indicators into one rating. Our goal is to create a model that can be used to compare two distinct ratings from two di®erent agencies, with a characterization of component di®erences explaining the sources of rating divergence. In particular, we are interested in di®erences in values and weights between the di®erent E, S, and G pillars (¯rst level) and between di®erent categories within the same pillar (second level). The model, thus, may be used to analyze divergences at the category level, which help to explain where divergences at the pillar level come from.
3.1. Divergence at the pillar level ESG scores are given by a weighted sum of the scores attributed to the three pillars (E, S, and G) by their relative weights (W ) for a company i: The overall ÁESG di®erence between the rating attributed by agency a and the one attributed by agency b to the same company i can be decomposed into di®erences caused by each pillar. The following equations disaggregate ÁESG into the di®erent components: where To compute the statistics ÁValues and ÁWeights of each pillar, we apply the same logic. ÁValues is computed as the average of the weights multiplied by the di®erence in the score values, while ÁWeights is calculated as the average of the values multiplied by the di®erence in the weights. The sum of these two components is equal to the variables introduced in Eqs.
(3)-(5). Hence, the overall ÁESG is divided into the following six components: In Table 6, we compute the statistics above for each combination of two di®erent rating agencies, and we¯nd the average divergence between the ESG scores decomposed into the value and the weight components and the average divergence across the three pillars. The numbers are computed for all listed companies that in 2020 are covered by both the agencies. The average ÁWeights is larger than the average ÁValues (55% versus 45%). The Environmental pillar, in most of the cases, is responsible for the lowest percentage of divergence, with the notable exception of Truvalue (when compared to MSCI Weighted, Re¯nitiv, and Arabesque). Indeed, Truvalue's weights disagree a lot with the others, especially for the Environmental pillar. We should remind that Truvalue weights are based on a di®erent methodology than the weights of other agencies, as they rely on the percentage of public news related to speci¯c ESG factors being evaluated in a given company. Instead, the Social pillar explains the highest percentage of divergence (36%). The Governance pillar explains 33% of divergence on average (lower than the Social pillar), even though it is the major source of divergence for the total variance between two ESG scores in certain cases (mostly involving Arabesque and Inrate paired with other agencies). Table 7 reports the decomposition of divergence into ÁValues and ÁWeights for the three di®erent pillars of E, S, and G. As expected, ÁWeights is larger than ÁValues for the Environmental (18% versus 13%) and Governance (19% versus 14%) pillars, while di®erences are comparable for the Social pillar (both values are close to 18%). With the exception of the pairings comprising S&P Global and most of the couples including MSCI, which account for the bulk of the observed variance in value scores, the Social pillar reveals average ÁWeights greater than the average ÁValues.

Divergence at the category level
In this sub-section, we present the lowest level of breakdown of ratings divergence, which is performed at the category level. This represents the single key indicator of sustainability performance tracked by the rating agencies.

2150006-12
This analysis was constrained by the availability of data. Only for Arabesque, MSCI, Re¯nitiv, and Truvalue we have available information about categories and relative weights. As for the aggregate rating, each pillar score is computed as the weighted sum of di®erent categories. Since each rating agency chooses to break down the concept of ESG performance into di®erent indicators and organizes them into di®erent hierarchies, we aggregate the various categories to obtain a¯nal category list common to each pair of agencies.
There a minimum of two to a maximum of 11 single indicators monitored by the di®erent rating agencies when analyzing each of the three pillars. Moreover, Truvalue provides¯ve high-level categories, divided into 26 sub-categories. In order to perform a meaningful comparison of these di®erent rating systems, we develop our categorization of the data using a top-down approach. Re¯nitiv was chosen as reference. Both Arabesque and MSCI categories and Truvalue sub-categories were assigned to Re¯nitiv categories on the basis of an accurate screening of rating providers' de¯nition of each category, which was provided in their methodology documentation. We require that each category could be assigned only to one¯nal category. The¯nal classi¯cation is shown in Table 8 (for Re¯nitiv, MSCI, and Arabesque categories) and Table 9 (for Truvalue sub-categories). The Environmental pillar is divided into three¯nal categories, the Social pillar into four¯nal categories (three for Truvalue, since \Human Rights and Community Relations" is one single category for this agency), and the Governance pillar into two¯nal categories.
Interestingly, Tables 8 and 9 show that there is some scope divergence between di®erent ESG ratings. For instance, categories can be very broadly de¯ned for one agency (e.g. MSCI \Human Capital") or much more detailed for another, making it di±cult to assess the sources of divergence, merely looking at the aggregate pillar scores.
To compute ÁValues and ÁWeights of each¯nal category, the same breakdown logic used for pillars was applied at the category level [see Eqs. (6)-(11)]. In the case of Truvalue, not all the categories and weights are reported by the analysts, since the rating depends on the number of news available related to the speci¯c category. When there was no data, the category is therefore not considered for the computation of the¯nal category (i.e. its weight is equal to zero). Figures 1-3 report the average scoring di®erences computed for each category of the Environmental, Social, and Governance pillars, respectively. The Environmental pillar (Fig. 1) shows the lowest variations across di®erent categories, con¯rming that moderate agreement can be found about the measurement of the di®erent environmental categories considered. The highest divergences are related to the di®erence between the weights of the Emission and Innovation categories. These results are not surprising considering, for instance, that information about the Emission indicator are Note: Truvalue and MSCI when compared with other agencies, we consider only one category for Community and Human Rights. For Truvalue versus MSCI, we consider only one category for Workforce and Community and Human Rights. quite objective and are generally publicly disclosed by large listed companies, leading to low divergences of category values. However, the di®erences concerning weights testify that di®erent agencies attribute di®erent importance to environmental issues.
The Social pillar (Fig. 2) shows higher variations across the components of different categories. Even in this case, the weight component shows higher average di®erences. The disagreement among the values attributed to di®erent categories is also in this case not so relevant, especially for Re¯nitiv versus Arabesque and MSCI versus Truvalue. This means that, despite the wide number of factors evaluated in this pillar, when categories are aggregated, di®erent agencies tend to assign similar values to one company. Thus, if di®erences exist at a more granular level, they tend to o®set when grouped into the¯nal categories.
Lastly, the Governance pillar (Fig. 3) shows the highest variation across weights and values for both categories. The majority of the variance is again explained by the di®erences in weights, which are particularly high. However, for the Corporate Governance (CG) category, di®erences between values are in many cases even larger than those between weights (i.e. Re¯nitiv versus Truvalue, Arabesque versus Truvalue, and MSCI versus Truvalue). One cause might be the subjectivity involved in determining the relevance of each category in di®erent sectors in terms of governance issues. Indeed, the relevance of a category for a speci¯c industry is more straightforward for the Environmental and Social pillars, and the results demonstrate that a certain consensus among agencies may be reached.
In sum, the category analysis has shown that there is substantial score divergence, especially for the weight component. There is, at least, some level of agreement regarding measurement (i.e. category values) of¯rms' environmental categories, for which the information are more easily obtained from public records. However, other categories, especially for the Governance pillar, show high levels of disagreement both for values and weights (e.g. Corporate Governance). Moreover, disagreement might tend to increase with granularity. This is the case of the Social pillar, where divergences seem to compensate each other to some extent through aggregation.

Concluding Discussion and Implications
The aim of this study was to explain why ESG ratings diverge. We developed a framework for comparing di®erent ESG rating methodologies in a systematic way. This framework allows us to split the di®erences between ESG ratings into two main components: ESG values and ESG weights at the pillar and category (second) levels.
We achieved some important results. First, we con¯rm a low correlation between ESG ratings of di®erent rating agencies according to prior literature (Chatterji et al. 2016, Berg et al. 2020. Since our study refers to the scores issued in 2020, it seems that no particular convergence has been experienced in the market compared to previous¯ndings. We also found that the weight component is more relevant than the value component in explaining ESG rating divergences. Second, the Environmental pillar di®erences are the lowest ones in the majority of comparisons with the only exception of Truvalue. This result can be attributed to the di®erences in the indicator weights assigned by Truvalue, which are computed following a di®erent methodology compared to other agencies, depending on the number of company's news related to a speci¯c category (and therefore, pillar).
Third, the Social and Governance pillars explain the majority of di®erences. The Social categories are the widest in number, although the topics addressed are very similar (e.g. human rights; workforce conditions, health, and safety; product quality; and impact on the society) with the primary di®erences attributable to the weights. For the Governance pillar, both weights and values account for a signi¯cant percentage of divergence for many agency comparisons. This latter¯nding can be explained by a higher level of subjectivity in the governance category evaluations, which can vary greatly between agencies, with some performance indicators included by one agency, but not rated by another.
Our¯ndings demonstrate that ESG rating divergence is driven not merely by the di®erences in analysts' evaluations, but also by disagreement about the underlying methodological issues and metrics. Weights divergence is particularly concerning, because it indicates a con°ict on the relevance of di®erent ESG performances and how pillars and categories are related to one another. As a result, even if a¯rm receives the same score value for its ESG performance, the ESG ratings generated by various rating providers might still di®er signi¯cantly.

Implications for scholars and practitioners
Our results suggest various implications for researchers, investors, companies, and rating agencies. Scholars should take into account that the divergence among different ESG pillar and category scores can a®ect the results and comparability of their studies. Certain results that have been obtained on the basis of one ESG rating might not be replicable with ESG ratings issued by another rating provider. This is particularly relevant for the stream of researches investigating the relation between rms' ESG and¯nancial performances. Our¯rst recommendation is to rely on data from more than one agency when analyzing those issues to improve the generalizability of¯ndings and detect di®erences originated by the use of di®erent scores. Another suggestion is to replicate the same study throughout time, because rating methodology and reporting practices are constantly evolving and ESG performance may be in°uenced. Third, researchers are encouraged to build hypotheses around indicators that are more clearly de¯ned (e.g. at the pillar or category level) than the aggregated ESG rating scores in order to rely on more transparent measurements (such as the Environmental pillar). In this situation, it would be still necessary to evaluate the adoption of a variety of di®erent weighting methodologies, as we saw there is greater disagreement on the weights assigned to indicators.
Considering investors, our framework helps them to understand why a company's ESG performance from di®erent rating agencies may diverge. In this case, the choice of a particular rating provider can a®ect their investing decisions in unpredictable ways. A¯rst recommendation is not to look at just one agency, but to compare di®erent ESG indicators from di®erent providers. This advice is especially important when it comes to the Governance pillar and related categories, because the level of disagreement is larger and the range of issues covered is broader and more generic than the ones considered in the other pillars. Conversely, investors may also rely on a single rating agency after persuading themselves that the metrics are consistent with their investing goals. Hence, the recommendation is to collect information about what is measured by each rating agency to select the ratings that best¯t with their needs.
For companies, our results highlight that there is substantial disagreement about their ESG performance. This divergence not only occurs at the aggregate level, but is actually more pronounced for speci¯c categories of ESG performance. Having that in mind, companies should increase the level of information disclosure in order to facilitate the rating process. The amount of information provided can help them to enhance their ESG scores because rating agencies often penalize companies that do not supply enough information, and more transparent information would also reduce analysts' subjective judgment. This would require more e®orts in terms of resources and time spent for sustainability disclosure, but given the increasing importance of assets invested in SRI (GSIA 2020) and the growing awareness about these concerns, it is critical for businesses to take steps in this direction.
Finally, for rating agencies a higher level of transparency with respect to their ESG scores and methodologies used is necessary to better understand what stands behind the ESG ratings. Recently, there has been an increase in the level of transparency; however, while some agencies have begun to publish comprehensive methodology and even some insights into their ratings, many agencies continue to keep detailed descriptions and data on scores and weights con¯dential. The standardization process of ESG ratings should be fueled by the introduction of new requirements and standards from policymakers. For instance, the European Commission is increasingly addressing the problem of corporate sustainability disclosure and could promote the standardization of ESG indicators and measurement. This would be especially important for SMEs, for which requirements are currently lacking and are, therefore, frequently excluded from evaluations due to a lack of information. Indeed, the establishment of transparent criteria would reduce the possible greenwashing phenomenon (Mrkajic et al. 2019) and limit analysts' subjective evaluation.

Limitations and future research directions
Our study has some limitations which could be the basis for scholars seeking to advance knowledge on the topic. First, as our sample consists of Italian¯rms and is therefore context-speci¯c, it would be interesting to validate our¯ndings using a wider sample. Indeed, the research may be expanded to include a European or worldwide sample of¯rms in order to look into regional di®erences. Moreover, other rating agencies can be included to corroborate the di®erences in the value and weight components found, as well as their relative importance. Second, another limitation is given by the lack of a complete overview on the indicators used for the de¯nition of category scores, which has determined a classi¯cation of the categories based on the descriptions provided by the di®erent agencies. Future studies could apply di®erent taxonomies, using more granular information on the indicators considered by agencies to generate category scores, together with more advanced techniques to classify categories. The development of rating methodologies based on Arti¯cial Intelligence and big data analytics also calls for studies on this topic using more sophisticated approaches (e.g. Lanza et al. 2020). Furthermore, research considering the relationship between ESG divergence and stock market performance is surely welcome. Recent evidence has shown that stock returns are positively related to ESG rating disagreement (Brandon et al. 2019), however, more research is needed to support these¯ndings and e®ectively inform investors'¯nancial decisions.