Benchmark study of approaches to estimate probability of default in the context of climate risk

Tue 27 Jul 2021

Recently, initiatives to tackle climate-related and environmental risks in the financial services industry have begun across the world. These initiatives followed the adoption of the United Nations Paris Agreement on climate change, the 2030 agenda for Sustainable Development and the European Green Deal. 

Stress testing and scenario analysis are a common framework proposed by different regulatory and supervisory bodies across various countries to assess the impact of climate-related risks on the financial system. Countries like the United Kingdom and France, having started working on pilot climate stress test exercises, are leading by example. However, no consensus regarding the best methodology to use in this context has been reached as of today.

In this article, Mazars presents the results obtained by implementing two methodological approaches to estimate the sectoral probability of default (PD) parameters in the context of climate risk stress testing:

• The first methodology studied was developed as part of the UN Environmental Programme (UNEP) Finance Initiative when piloting the implementation of recommendations outlined by Task Force on Climate-related Financial Disclosures (TCFD)[1]; and
• the second corresponds to the approach followed by the Autorité de Contrôle Prudentiel et de Résolution (ACPR) in its 2020 climate risk stress test pilot exercise[2].

Common basis of the two approaches

Both approaches are built upon a well-established PD methodology, known as the Merton Framework, where the default dynamics are captured via macroeconomic and financial risk drivers. This allows one to project the expected default rate according to the anticipated movements of these risk drivers. The Merton model has been widely used in the industry to derive the IFRS 9 forward-looking and point in time PDs and perform stress testing.

With this foundation, the two approaches differ in the way the framework has been adapted for climate stress testing. In particular, the approaches differ in the mode of incorporation of climate risk factors into the PD calculations. The results obtained by Mazars show that these two distinct approaches can yield similar results.

Sectoral sensitivity approach

In this approach, climate variables and their respective sensitivity coefficients -estimated using a scorecard – are added into the sectoral PD Merton models. As previously mentioned, the method was developed by the UNEP, and we refer to it as the “sensitivity approach” throughout this article.

This approach has two clear advantages:

• it requires few additional inputs; and,
• one can leverage the work performed by the UNEP when determining the hierarchy of the sensitivities (i.e. which sectors have a high climate risk impact vs low).

However, a key drawback is the lack of guidance or references for the level of the sensitivities, so the determination of these inputs is based on expert judgement.

Sectoral approach

The French regulator, the ACPR, produced a set of sectoral Gross Value Added (GVA) forecasts and existing leveraged models from banks when building its climate risk stress testing framework. Climate PD forecasts were produced using the sectoral forecasts given by the ACPR along with either the bank’s pre-existing models or with newly developed models. It is worth noting that when the bank’s PD models did not include GDP as a default risk driver, new models were developed to add it. This variable was necessary to produce forecasts with the sectoral granularity required for the climate risk stress testing assessment. Similarly, the BoE produced a set of sectoral GVA forecasts for its 2021 Climate Biennial Exploratory Scenario (CBES), accounting for different economic sectors’ varying degrees of exposure to climate risks.

This approach has three advantages:

• the work performed by the ACPR can be used as a reference;
• generally, results can be generated with slight modifications to existing models; and
• the model does not introduce additional parameters.

On the other hand, the disadvantages include:

• a strong modelling assumption: “models are still pertinent with sectoral data”; and,
• the requirement of sectoral macro-economic forecasts.

Our study

Mazars implemented both methodologies in an attempt to understand whether the methodologies yield consistent results. In practice, our study consisted of the following steps:

Step 1. Calibrating a PD model to the historical default rate series of the global corporates segment, which, by design, included a GDP variable.

Step 2. Applying the PD model (obtained in step 1) to the ACPR sectoral forecasts to derive results from the sectoral approach.

Figure 1 illustrates the 12-month PD forecast at each reporting date for two ACPR scenarios: orderly and sudden. To provide some context, the ACPR orderly scenario reflects the French roadmap designed to fulfil the commitments made under the Paris Agreement. On the other hand, the sudden and delayed scenarios consist of a sharp increase in the carbon price, the latter starting only in 2030.

As expected, one can see that the risk parameter (i.e. PD) increases with the “severity” of the scenario. A similar fact can be highlighted for the segments: a higher PD is observed for those segments expected to have stronger negative impacts from climate change.

Step 3. Calibrating the sensitivities (in the sensitivity approach) of each sector to climate variables. The sensitivity for each sector was estimated such that the difference in 12-month PD at each calculation date between the results from the sensitivity approach and the sectoral approach is minimised.

Mazars selected the annual difference in carbon price as the climate variable. Other transformations of carbon price were analysed but won’t be considered in this article because the results obtained were consistent with those discussed here.

Figure 2 illustrates the 12-month PD by sector at three reporting dates (2035, 2040 and 2045) utilising the two alternative approaches.

The left graph in Figure 2 illustrates the 12-month PD by sector for the sudden scenario as at 2035 and shows that the two approaches yield consistent results for five out of the seven sectors analysed. There is, however, a significant difference when comparing the two most sensitive sectors: coke and refined petroleum and mining and quarrying.  

On the other hand, for the 2040 reporting date, the results show convergence for all the sectors (refer to the graph in the centre in Figure 2). Small discrepancies were observed for the 2045 reporting date (refer to the right graph in Figure 2). These can be explained by differences in the information carried by the selected climate risk variables (GVA forecasts vs difference in carbon price).

In an attempt to reduce the discrepancies between the two approaches, an additional analysis was performed by implying a climate factor from the GVA forecast of the most affected sector (i.e. coke and refined petroleum products) relative to the GDP forecast. We will refer to this variable as the ACPR implied climate factor and, for each year, define it as follows:

where and are the annual differences of each variable respectively.

The results obtained using the ACPR implied climate factor for 2035, 2040 and 2045 are shown in Figure 3.

The sensitivity approach results fit the sectoral GVA approach results almost perfectly, particularly for the coke and refined petroleum products sector. The latter behaviour was expected as the ACPR implied climate factor was built using the GVA forecasts for this sector. We observed slight discrepancies for some sectors in the sudden and delayed scenarios and when looking at all reporting dates.

Step 4. Analysis of the results.

Table 1 summarises the results of the calibrations performed in step 3, along with the ratio of the sectoral GVA forecast from the ACPR divided by the GDP forecast (sector independent). These ratios are used to benchmark our results as they represent the ACPR expected climate impacts for each of the sectors.  

Furthermore, the values in Table 1  have been colour-coded such that the sectors that are highly impacted by the climate risk are highlighted in red, and those that are less impacted are highlighted in green.

The results obtained when calibrating the sensitivities with both variables (difference in carbon price and the ACPR implied climate factor) are consistent. With both methods, the most impacted sector was coke and refined petroleum products, followed by mining and quarrying, and the least impacted sector was support service activities. The same conclusion is reached when comparing the sensitivities with the GVA/GDP ratios[3].

Finally, we compared our results with the scorecard ratings provided in the UNEP/TCFD study “Beyond the Horizon”[4]. Table 2 outlines the mapping applied and the associated UNEP/TCFD ratings by sector.

Overall, the results presented in Table 1 appear to be consistent with the ratings summarised in Table 2 except for the Metals & Mining sector. The UNEP rated this sector as “Moderate”, whereas the GDP ratios from ACPR indicate the segment is treated as “Highly Moderate”. Notice that there is a clear difference between the ratios of this sector (orange rows in Table 1) and the rest of the sectors rated as “Moderate” (yellow rows in Table 2). The discrepancy could be explained by differences in the segmentation approaches of the two sources.

Conclusion

Based on the results obtained, we conclude that both sensitivity and sectoral approaches can yield equivalent results, particularly when both methods use the same climate risk information to project PDs.

[1] Extending Our Horizons by J. Colas, I. Khaykin, A. Pyanet and J. Westheim, April 2018, available here.

[2] Analysis and synthesis no. 122: The main results of the 2020 climate pilot exercise, by ACPR, May 2021, available here.

[3] These ratios are estimated as the average ratio across scenarios for each sector for the 2050 forecast. For example, we divided the basic metals GVA 2050 forecast of the accelerated scenario by the GDP 2050 forecast for the accelerated scenario respectively, and then average them to consider the three scenarios.

[4] Beyond the horizon: New Tools and Frameworks for transition risk assessments from UNEP FI’s TCFD Banking Programme, by D. Carling and R. Fischer, September 2020, available here.