Estimating PD Cyclicality in the presence of noise: Theoretical discussion and experimental results

In our previous article, we explored how seemingly reasonable and small variations in assumptions can result in a wide range of cyclicality estimates. In this article we explore the potential impact of various back-casting assumptions, for a UK mortgages portfolio.

The Capital Requirements Regulation (CRR) requires that firms identify homogeneous cohorts, and assign a probability of default (PD) equal to the observed long-run average of one-year default rates. Whilst a fixed definition of precisely what constitutes a long-run average, the Prudential Regulation Authority (PRA) in SS11/13 articulates an expectation that a representative mix of good and bad years includes conditions equivalent to those experienced in the early 1990s.

Banks generally do not possess granular credit risk data across a complete economic cycle. Therefore, firms are compelled to make assumptions about how observed default rates (ODRs) would have behaved, in the period before which internal data is available.

Any textbook on statistical modelling would suggest performing a search for, and selecting, the modelled assumptions with the optimal fit. In this article, we explore the impact of a small set of back-cast assumptions on the portfolio-level cyclicality estimate, using knowledge of market arrears stock.

Portfolio and rating system behaviour

We initially configured our system of obligors and default risk with just two grades – customers in arrears and customers not in arrears - whose true long-run average default rate and grade-level cyclicality we can control. We parameterised the grades’ PDs and cyclicality in line with benchmarks.

We measured the true long-run average (LRA) ODR in each grade over the period 1988 to 2020. We selected 1988 as the start-point to align with the time period in the Bank of England’s stylised example. In practice it would be appropriate to condition on a complete economic cycle.

The true long-run average default rates are presented in the table below, along with the short-run average (SRA) ODR in the period 2008 to 2020. These are the values that a back-cast model should be recovering.

Back-cast models

A common approach in the UK is to use an exogenous index of market arrears, to inform the assumptions about how cohorts’ default rates would have evolved in prior periods, including the early 1990s. We investigated using the following back-cast models to recover the LRA ODR:

• Simple scalar
• Regression differences
• Stylised example

Simple Scalar

Simple scalar extrapolation assumes that grades’ SRA ODRs are uplifted by a shared scalar value such that LRA=s1 * SRA, where s1 is determined from the external index as s1=(average index level 1988 to 2020)/(average index level 2008 to 2020).

Regression in differences

The relationship between grades’ ODR and the index level can be expressed in differences, as dODR(grade,t)=beta0+beta1*dIndex(t)+e where e is an IID noise process and dODR and dPD represent the changes in grade-level ODR and index level respectively, measured in reverse time. We parameterised the relationship using ordinary least squares.

Stylised example

The Bank of England’s stylised example estimates the historical average ODR under both 100% TTC and 100% PIT assumptions, and blends between the two. For the 100% TTC back-cast, a two-step approach is used whereby the index level is first used to extrapolate the portfolio-level default rate, and the portfolio-level default rate is then used to extrapolate the grade-level default rate. The approach set out in the stylised example reduces to the formula LRA ODR = (1-c)*s2*SRA + c*[SRA ODR], where c is the assumed cyclicality, s2 is the scalar relationship (average index level 1988 to 2007)/(average index level in 2008 to 2020).

We estimated the grade-level LRA ODR using a range of plausible cyclicality assumptions.

Results

The table below presents the recovered LRA ODR estimates and cyclicality. Cyclicality is measured by regression in one-year first differences over the period 2008 to 2020, to align with the period where most firms are likely to have internal default data. The subsequent rows show the results obtained for various back-casting assumptions.

The chart below presents the portfolio-level observed default rate (ODR) as well as the weighted average PD (WAPD), under the various back-cast assumptions set out above.

Our analysis suggests that seemingly reasonable variations in back-cast assumptions can significantly influence long-run PD estimates as well as the level of cyclicality that is achieved. The case for applying the scientific method remains as compelling as ever.

At the same time, we recognise that it may not always be possible to obtain a robust regression relationship, meaning that some firms will have little choice but to apply simplified assumptions, and carry the associated impacts until such time as they have enough data to update their models.

After-word: Log-odds adjustments

Given the standard approach of modelling retail credit in log-odds space, we have summarised below the magnitude of log-odds adjustment from the true LRA values with which we configured our system.

In all cases, the adjustment differs between grades, suggesting that the equivalence condition that would allow for a single log-odds adjustment does not hold. Of particular note is that only the regression in differences preserves the rank order of log-odds adjustments.