Estimating PD Cyclicality in the presence of noise: Theoretical discussion and experimental results

Introduction

1/1/22 heralded not only the new year, but the compliance date for UK firms' implementation of the "EBA Roadmap" to reduce unwarranted variability across banks' Risk Weighted Assets (RWAs). A key feature of UK firms' implementations of the EBA Roadmap is the move towards Probability of Default (PD) estimates that reflect long-run average default rates within homogeneous pools or grades, in the Retail exposure class. In SS11/13 the PRA articulates an expectation that the level of cyclicality of PDs not exceed 30%, and asks firms to measure the PD cyclicality using the formula c=(PD(t)-PD(t-1))/(DR(t)-DR(t-1)), where c represents the proportion of cyclical variations in default rate that is explained by the rating model. Importantly, c should describe the rating system’s response to economic changes, which is by-definition independent of the kind of obligors that a bank selects and places on its balance sheet.

In this article, we explore the derivation of the cyclicality formula and explore estimating c in the presence of noise in the numerator and denominator. We built a portfolio emulator to investigate estimation of c in the presence of noise and/or even changes in lending criteria, using direct observation and a regression approach.

The theoretical discussion and empirical results suggest that direct observation of c is unstable in all but a very narrow set of conditions due to the interaction of noise in the numerator and denominator. Using direct observation, the estimated values of c never match the true cyclicality value with which we configured the emulator, to the extent that the true value is close to unobservable in the general case and hard to compare between portfolios or banks. However, when we instead specified c in terms of a linear regression in first differences, we were able to recover estimates that were sufficiently close to the true value using a 12-month time lag for the differencing terms, for a variety of risk appetite and rating system configurations.

Background

The objective of both Basel and IFRS PD modelling is to provide obligor-level PDs that are accurate enough to be used in the regulatory and statutory reporting, respectively.

It is universally recognised that there exists a time-varying component of default risk that may or may not be fully explained by the choice of rating model inputs. This is generally referred to as the exogenous component or credit cycle, and describes the general level of vulnerability of an obligor (or group of homogeneous obligors) to external shocks at any given time.

Obligor PDs are said to be Point In Time (PIT) if the selection of risk factors results in PDs that accurately track temporal variability in default risk, and Through The Cycle (TTC) if they do not track temporal variability at all. In practice, outside of publicly traded obligors (where a market-implied Estimated Default Frequency can be determined), internal models require the grouping of obligors into homogeneous (or approximately homogeneous) groups.

The Bank of England’s stylised example in Appendix 2 of PS17/13 suggests using a binning procedure to assign obligors to rating grades, and then calibrate each rating grade’s PD to an appropriate estimate of the Long Run Average Default Rate (LRADR). The “true” LRADR for portfolios where the annual default rates are not a stationary series is somewhat subjective, and beyond the scope of this article.

The Long Run PD (LRPD) for a rating grade is a grade-level quantity that is assumed to be appropriate to the obligors in that grade at the point of assignment. We use the terminology “hybrid PD” to refer to a rating grade LRPD whose cyclicality lies somewhere between 100% PIT and 100% TTC. It is theoretically possible to select static factors to align the rating grade’s LRPD with the constituent obligors’ TTC PDs; or cyclical risk factors to align the rating’s LRPD with the constituent obligors’ PIT PD. However, neither would be a particularly useful model: TTC PDs fail to capture current information, and PIT PDs tend to result in overly-cyclical capital requirements that can hamper business planning as well constrain lending into the real economy, to the extent that the economic cycle can become amplified by the financial system.

In corporate credit, the notion that rating grades have an element of cyclicality (indeed, a varying element of cyclicality that reduces in better grades) has reasonably strong precedent in terms of methodologies for quantifying the cyclicality and converting between LRPD and PIT or TTC PDs. However, in Retail lending there is less precedent or consensus around model frameworks or assumptions to describe cyclicality.

Portfolio-Level Cyclicality Formula

In SS11/13 the PRA defines cyclicality at the rating system level, as c=(PD(t)-PD(t-1))/(DR(t)-DR(t-1)). The formula is a first-differences solution to the expression PD(t)=CT+c*DR(t).

Whilst a portfolio-level metric may not satisfy corporate credit purists, c is a convenient summary descriptor of the degree to which PDs (and hence RWAs) might move across the cycle.

PD(t) is the average hybrid PD, somewhere between 100% PIT and 100% TTC.

The expression introduces the concept of a Central Tendency (CT). The CT is a portfolio-level measure of average one-year default rates. Whilst the concept of a stable CT may seem attractive, it is only a stable quantity under a narrow set of assumptions:

• Constant risk profile over time: The assumption will not hold if risk appetite or lending criteria have changed over time. i.e. even with TTC rating assignment, the average PD in the portfolio can drift over time; and
• Stationarity in observed default rate: The assumption will not hold if the CT level depends on the selection of observation period.

Over a short interval, the CT can be assumed to be sufficiently constant to cancel, overcoming both limitations and leading directly to the formula in SS11/13. However, an implicit assumption remains, that no measurement noise afflicts any of the PD or DR terms, or differences.

Estimating cyclicality

If we cast the cyclicality problem as estimation of a probability model, we are able to call on estimation theory to assign values to the free parameters. SS11/13 does not impose a probability model for c. We have therefore considered two plausible probability models to describe the underlying “signals” of PD and DR as well as measurement noise. (Note that other probability models are available). Sources of noise in PD include discrete binning (quantisation error) in logit models’ points assignment, as well as LRPD grade binning. Sources of noise in DR include random perturbations from a smooth or idealised second-order process. We can account for measurement noise by representing the terms in the formula for c as x=x’+e where x’ is the true unobservable latent value, and e represents measurement noise.

Estimating cyclicality: Direct observation from draw of ratio distribution

In the presence of noise, the formula for c can be generalised as c=(PD(t)-PD(t-1)+e1)/(DR(t)-DR(t-1)+e2) where e1 and e2 describe noise in the PD difference term and DR difference term, respectively.

Ratios of random variables are typically fat-tailed and difficult to work with. In practice this means that an obvious central or most likely value is often undefined and a bootstrapped solution would be meaningless. Even under the seemingly simple assumption of Normality, the resulting ratio distribution is non-trivial to parameterise.

However, in periods where the magnitude of the difference terms significantly exceeds the magnitude of the noise terms, the error terms can be ignored, and a reasonably robust estimate of c observed directly from a single draw of the underlying distribution. In practice this means finding a period with a large movement in default rate.

Thus, a conflict emerges between finding two periods (or a set of pairs of periods) which exhibit a sufficiently large movement in default rate, but are also close enough together that the constant CT assumption holds well enough for the first difference to be dominated by cyclical movements in PD and DR and not polluted by changes in lending criteria.

Estimating Cyclicality: Ordinary Least Squares estimate of regression in first differences

The formula for c can also be cast as a regression problem with a single noise term, PD(t)-PD(t-1)=c*(DR(t)-DR(t-1))+e. In the case that the noise term e is independent and identically distributed (IID) noise, then an ordinary least squares (OLS) solution for c is achievable. However, if the IID assumption is violated then the result should be considered spurious and is unlikely to generalise.

Portfolio and Hybrid PD Emulator

To emulate UK firms’ mortgage book PDs and default histories, we constructed a hypothetical portfolio whose TTC credit quality can be configured to move over time by replacing maturing and default accounts as well as growing the book in better rating grades. Exposures are introduced in a rating that is 100% TTC. A credit cycle adjustment provides PIT PDs, which allow us to generate hybrid PDs at any user-defined cyclicality as well as default rates using a standard Merton-Vasicek adjustment in distance to default space. We also polluted the hybrid PDs and default rates with a configurable amount of IID noise. We estimated portfolio-level cyclicality using the SS11/13 formula across various time intervals preceding peak DR; as well as peak-to-trough DR.

The figures below present the observed default rate and average hybrid PD, for various configurations. The second chart in particular illustrates how 100% TTC ratings can lead to movements in PD, if lending quality improves over time.

We sought to estimate c for the two probability models described above: Direct observation from a single draw of the ratio distribution, and using an OLS regression in first differences.

Results and Conclusions

The complete set of results are summarised in the table below (you may need to scroll sideways to view).

Conclusions: Direct observation

Across the various portfolio and PD cyclicality configurations, our results are consistent with the theoretical discussion. None of the direct observation results match the true cyclicality we specified, or could be relied upon as an accurate descriptor of the likely PD path under a future stress. In further detail:

• If observed across a short time interval, cyclicality values are polluted by noise and deviate significantly from the true cyclicality.
• If observed across a 6 to 12-month time interval, the impact of noise on cyclicality values is reduced but estimate still deviate from the true cyclicality, due to the single sample of noise in the numerator that we are unable to eliminate.
• If observed peak-to-trough, cyclicality values are impacted by improvements in lending quality. Even a very modest (1 notch) improvement in origination quality drives a cyclicality estimate of 20%, despite a true cyclicality value of 0%.
• If changes in lending quality are implemented before the peak default rate occurs, the “constant CT” assumption does not hold across the 12-month differencing interval and cyclicality observations at peak are reduced.

Conclusions: Regression estimate

Across the various portfolio and cyclicality configurations, the regression using 12-month differences returns estimates with the closest alignment to the true cyclicality. Nevertheless, the results are sensitive to even a single notch improvement in lending criteria, and the timing of the improvement. In the majority of cases, the true value lies outside the estimated 95% confidence interval and some degree of autocorrelation is present in the residuals, meaning that some degree of subjective or qualitative interpretation of the analytical result is required.