There’s been much talk of late about problems with the ESG rating ecosystem.
First of all, there is no consensus still on what constitutes ESG risk. That leaves a company’s ESG risk profile open to interpretation by ESG rating agencies, analysts, and investors.
Secondly, there’s a potential conflict of interest in some cases. Companies end up using the products and services of the ESG rating agencies themselves and that raises questions about the transparency of the ESG rating process.
We’ve talked about these issues in a recent blog post.
Another problem afflicting the ESG rating ecosystem is that of ‘divergence’. A recent study published in the European Finance Association’s Review of Finance journal points out that ESG ratings issued by six different rating agencies are not in agreement with each other. The study is called Aggregate Confusion: The Divergence of ESG Ratings.
What are the factors causing the divergence? Let’s find out.
What’s this Study All About?
A group of three researchers – Florian Berg, Julian F Kölbel, and Roberto Rigobon – have tried to pinpoint areas of divergence in ESG ratings. They have done so using data from six prominent ESG rating agencies: Kinder, Lydenberg, and Domini (KLD), Sustainalytics, Moody’s ESG, S&P Global, Refinitiv, and MSCI.
The researchers mapped the data to a common taxonomy to find out areas where the ESG rating divergence occurs. They were able to identify three areas: Scope, Measurement, and Weight. We explain these areas in detail below.
Further, the researchers were able to find a phenomenon called the “rater effect” – where companies whose perception is generally positive receive better indicator scores than companies whose perception is generally negative.
Let’s take a deeper look at what all this means.
What was the Methodology Used?
The study we’re talking about builds upon a previous study called Do Ratings of Firms Converge? Implications for Strategy Research. In that former study, researchers framed two questions to identify causes of ESG rating divergence: What are ESG rating agencies choosing to measure? Are they measuring it correctly? The problems the two questions tried to deal with were respectively called Theorization and Commensurability.
However, as successful as the former study was in identifying divergence, the researchers could not pinpoint to what extent differences in theorization and low commensurability drove the divergence in ratings. The reason why the research fell short was that the researchers relied on a data set containing “only a small subset of the underlying indicators that make up the different ESG ratings”.
The present paper we’re examining solves that problem by relying on data from six ESG ratings agencies along with the complete set of 709 underlying indicators. The researchers used the data to provide a quantitative decomposition of ESG rating divergence.
To do that, the researchers first categorized the 709 indicators from different ESG rating agencies into a common taxonomy of 64 categories. The categorization allowed them to observe “the scope of categories covered by each rating and to contrast measurements by different ESG rating agencies within the same category”.
In the words of the researchers: “We create(d) a category whenever at least two indicators from different rating agencies pertain(ed) to the same attribute. Based on the taxonomy, we calculate(d) rater-specific category scores by averaging indicators that were assigned to the same category. Second, we regress(ed) the original rating on those category scores. The regression models yield(ed) fitted versions of the original ratings and we…compare(d) these fitted ratings to each other. Third, we decompose(ed) the divergence into the contributions of scope, measurement, and weight.”
Scope, Measurement, and Weight Divergence
According to the researchers, Scope divergence occurs when ratings are based on different sets of attributes. For instance, one ESG rating agency may choose to include lobbying activities in its assessment while another might choose not to – leading to a scope divergence.
Measurement divergence occurs when ESG rating agencies use different indicators to measure the same attribute. For instance, a company’s labor practices may be evaluated based on workforce turnover or the number of labor-related cases against it.
Weight divergence occurs when rating agencies “take different views on the relative importance of attributes”. For instance, an indicator on labor practices may have a greater weight in the final rating than an indicator on lobbying.
However, the contributions of scope, measurement, and weight divergence to final ESG ratings are intertwined – which makes the difference between any two ESG ratings difficult to interpret.
What are the Main Findings of this Study?
The researchers found that of the three areas of divergence, Measurement Divergence was the main driver. It contributed to 56% of the divergence in ESG ratings. Scope divergence was also key, with a contribution of 38%. Weight divergence contributed a mere 6% of divergence.
The researchers also found that measurement divergence was influenced by a phenomenon called the “rater effect”. This means a company receiving a high score in one category is more likely to receive high scores in all the other categories considered by an ESG rating agency. The rater effect is seen playing a substantial role. “Controlling for which firm is rated and in which category the firm is rated, the rater effect explains 15% of the variation of category scores,” according to the researchers.
The Rater Effect
The Rater Effect potentially takes place because ESG rating agencies have analysts specializing in firms rather than indicators, the researchers say. Therefore, a company with a generally positive perception ends up being seen through a positive lens and receives better indicator scores than a company with a generally negative perception.
“In discussions with S&P Global, we learned about another potential cause for such a rater effect. Some raters make it impossible for firms to receive a good indicator score if they do not give an answer to the corresponding question in the questionnaire. This happens regardless of the actual indicator performance. The extent to which the firms answer specific questions may be correlated across indicators. Hence, the rater effect could also be due to rater-specific assumptions that systematically affect assessments,” the researchers say.
The researchers also believe there may be economic incentives affecting assessment. For instance, a group of researchers referred to as Cornaggia, Cornaggia, and Hund suggest that credit raters may have incentives to inflate certain ratings. “An interesting avenue for future research is whether ESG raters have similar incentives to adjust their ratings.”
In Conclusion
In conclusion, the researchers say that ESG rating divergence does not mean that it is a futile exercise to measure ESG performance. However, it is implied that measuring ESG performance is a challenge. The underlying data and the use of ESG ratings and metrics need careful consideration in each specific case.
“Investors can use our methodology to reconcile divergent ratings and focus their research on those categories where ratings disagree. For regulators, our study points to the potential benefits of harmonizing ESG disclosure and establishing a taxonomy of ESG categories,” according to the researchers.
The researchers further say that harmonizing ESG disclosure would help provide a foundation of reliable data. A taxonomy of ESG categories would make it easier to contrast and compare ratings, they say.
Our Word
All activities related to ESG — whether it be investing, reporting the metrics, or providing a rating – are at an evolutionary stage. On the reporting side of things, various global bodies are at work to create an acceptable set of sustainability standards. The effort is to create a global baseline framework that helps harmonize all ESG reporting and provides transparent and comparable data to all the stakeholders concerned: regulators, analysts, investors, and rating agencies.
The International Sustainability Standards Board (ISSB) is leading a concerted effort to create such a baseline ESG framework. Moreover, there have been calls for introducing digital ESG reporting right at the outset. Digital reporting involves mapping all ESG disclosures to taxonomy elements that are machine-readable. Such digital disclosures are more accessible, easy to analyze, and comparable. Financial reporting in various jurisdictions such as the US, the EU, and the UK has greatly benefitted from digital disclosure.
