The 1st of January 2022 marked the deadline for UK banks to be compliant with the European Banking Authority (EBA) roadmap to reduce unwanted variability across risk-weighted assets (RWA). Along with the new definition of default (DoD), cyclicality of probability of default (PD) has emerged at the centre of attention. The Prudential Regulation Authority (PRA) has articulated in SS11/13 an expectation that the level of cyclicality of PDs does not exceed 30% (measured at the portfolio level using a formula that we have discussed previously [1] [2]).
Cyclicality is influenced by risk driver selection (but can also be influenced by the calibration curve, if grade-level inaccuracy is acceptable).
Credit modelling traditionally relies heavily on standard heuristics (“rules of thumb”) for risk driver and model selection, to optimise predictive ability. For the joint optimization of cyclicality as well as predictive ability, there exists significant variability in market approaches (and indeed also variability in what is the appropriate level of trade-off between the two competing constraints). Many approaches are computationally expensive and can fail to generalise across the choice of time-period, sample selection and order of calculation. Firms also need to be mindful of the “novel or narrow” requirement specified by the PRA in SS11/13.
In this article we describe a heuristic for model selection that prioritises the selection of less-cyclical risk drivers without introducing new metrics, rules-of-thumb, or subjective trade-offs. We show in real-world data that a significant reduction in cyclicality can be achieved for a relatively small sacrifice in rank ordering ability.
We trained three logistic regression models on data sourced from Fannie Mae for Single-Family Loan Performance:
All three models were selected using k-fold cross validation to avoid over-fit. Cyclicality was tested in a period of rising default rates.
Model B is an intermediate result. Models A and C share a dependent variable, and are the subject of comparison:
The figures below illustrate the PD over time overlaid with the respective 12-month and 24-month default outcomes for models A and C. Model A tracks the observed default rate better than model C. Whilst model C also tracks default rates post-crisis, this is attributable to survivorship bias as well as system-wide improvements in risk appetite and underwriting quality. This is consistent with our previous simulated results in our previous blog on cyclicality, where even a 100% through-the-cycle (TTC) construction can give rise to portfolio average PDs that drift downwards over time.
Model A
Model C
Output |
Model A |
Model C |
Model Specification |
Predicts the 12-month default outcome for performing loans at month 0 |
Predicts the 24-month default outcome for performing loans at month 0 and also at 12 and is calibrated to the standard 12-month default outcome |
Gini |
85% |
83% |
Cyclicality |
18% |
5% |
The results show that a small yet intuitive modification to standard model selection procedures can achieve a significant improvement in cyclicality, for only a modest reduction in Gini. The Gini value remains comfortably above most firms’ minimum thresholds.
The approach avoids the sorts of computationally complex approaches or manual fine-tuning adjustments that many emerging market approaches have introduced. This saves on not only time and costs, but also energy consumption and carbon footprint.