Countdown to compliance with SS1/23 – model risk management

Target Audience: SMFs with responsibility for Model Risk Management; Heads of Board Risk Committees; CEOs, CFOs, CROs, CIOs, Heads of Internal Audit, Model Risk Management practitioners.

At a glance: SS1/23 takes effect on 17 May 2024 (for a view of the essentials of SS1/23, see our blog here). With 12 weeks to go, banks should be able to demonstrate that key elements of their programme are complete or close to complete. We set out below five key areas of banks’ programmes that Boards and Senior Executives should be ensuring are well-progressed, including:

• gap analysis and remediation plan;
• incorporating model risk into risk appetite;
• identification of the SMF assuming accountability for model risk management (MRM);
• model inventory and risk tiering; and
• development of new MRM governance and control processes.

Given that SS1/23 applies only to banks with existing permissions to use internal models for capital purposes, the PRA will expect those banks to demonstrate a high level of compliance from day one. The probability of the PRA commissioning a report by a skilled person (section 166 review, s.166) for banks that fall short of its expectations is, we feel, high.

Where should banks be?

SS1/23 comes into effect on 17 May 2024 for banks (and building societies) operating in the UK with existing approvals to use internal models for capital purposes.

With 12 weeks to go until the implementation of the MRM rules, where should banks be in their programmes? We set out below five key areas where banks should by now have largely completed their work, and the questions Boards and Senior Executives should be asking to ensure this is the case:

1) Gap analysis should be completed and activities to close gaps should be in train:

a) Every effective regulatory programme has at its core a detailed gap analysis against the requirements in the regulation, with clear evidence of areas of compliance, and a plan to close gaps where they exist.

Questions:

Is the gap analysis robust and the closure plan on track? Are there any areas of weakness? If so, what are we doing to fix them?

Would we be happy to provide the PRA with the outputs of our gap analysis and remediation plan if requested?

How will we incorporate the requirements of the supervisory statement into the assessment of regulatory compliance that must be provided when models are submitted for regulatory approval?

2) Model risk should be incorporated into risk appetite as a level one risk:

a) SS1/23 indicates that the PRA is looking for model risk to be identified, monitored, managed, and reported with the same frequency, detail and level of challenge as Credit, Market, Operational, Strategic, Financial and other level one risks.

b) Model risk appetite should be quantified, with metrics covering both model-level and (as far as possible) aggregate risk, to flow into risk appetite monitoring and reporting.

Questions:

Is model risk incorporated into risk appetite such that it is recognised in the risk management framework as a risk in its own right?

How do we quantify model-level and aggregate model risk? How do we achieve consistency in risk appetite measures for different types of models?

3) The individual taking on the SMF accountability should be identified, and their accountability statement updated accordingly. Other SMF accountability statements should also be reviewed to ensure that, where relevant, MRM is reflected (for example SMF24 Chief Operations may need MRM systems recognised as a priority):

a) Most banks are likely to assign the SMF responsibility for model risk to the CRO (SMF4), however that is not mandatory. Banks must ensure they have identified the most appropriate SMF holder for model risk accountability, and that they can explain why the identified individual is the right person to hold the responsibility.

b) The identified individual should be able to explain clearly to supervisors what reasonable steps they will take to ensure model risk is effectively managed, and if future investment is required to achieve that end, that budget discussions are in train if not already agreed.

Questions:

Who will assume the accountability as SMF for model risk? What process was followed to ensure they are the right person?

Has the SMF holder developed a plan for ensuring MRM is effectively managed, and a communication plan to share this with the supervisor?

Have all relevant SMF accountabilities been reviewed and updated to reflect model risk appropriately?

4) The model inventory and risk tiering of models should be complete , based on the PRA’s definition of a model . The model inventory should specifically address:

a) Climate models – recognising current constraints around data and methodologies.

b) Artificial Intelligence (AI) and Machine Learning (ML) models, including any constraints that may be imposed by the EU AI Act if models are applied to EU customers.

c) Generative AI models – ensuring that Board and senior management understand the distinction between “normal” AI and Gen AI models.

d) End User Computing (EUC) solutions – ensuring that EUCs are included in the model inventory where they meet the definition of a model.

e) Material Deterministic Quantitative Methods (DQM), where such DQMs meet the internal policy threshold for inclusion as models, even if they may not strictly meet the PRA definition.

Questions:

How have we ensured consistency of tiering/materiality across different model types? For example, how do we equate a tier one liquidity model with a tier one credit model with a tier one impairment model?

Have we set the threshold for what is captured as a model sufficiently broadly?

Have we updated and communicated our policy framework to reflect how the definition of model, particularly around Climate, AI, EUCs and DMQs, has changed?

5) Enhanced and/or expanded MRM governance processes and controls should be designed and implementation underway:

a) Board and executive training programmes should have been undertaken, and education about the importance of model risk management should be being rolled out to all relevant staff.

b) Banks should be able to set out how the tiered model review or oversight process will operate, how it will identify risks and how those risks will be monitored, remediated, and reported, particularly for models that have not previously been subject to formal oversight.

c) Enhanced Board governance and reporting should be designed and implementation underway. Reporting to Boards needs to reflect an assessment of aggregate model risk across the bank, and evidence of Board challenge will need to be collected.

Questions:

Following training/education, are Board and Senior Executives able to:

• articulate clearly the bank’s model risk appetite;
• communicate their understanding of the core methodology, strengths and weaknesses of the most material models;
  
• describe model risk as it applies to the bank; and
• talk through the key model risk remediation activities in progress?

How do we satisfy ourselves that revised model risk processes are comprehensive and will identify model risks in an accurate and timely manner? What controls have been put in place to ensure this?

Does revised MRM reporting give sufficient information to allow the Board and Senior Executives to understand how model risk in the bank may manifest? For example, does reporting set out how poor model performance translates into balance sheet or profit and loss impacts?

Conclusion

With just 12 weeks to go until SS1/23 takes effect, banks should by now have completed the analysis, design and development elements of their programmes and be well into the implementation phase.

Given SS1/23 applies only to banks with existing model approvals, and considering the PRA’s propensity for imposing s.166 reviews on banks in recent years, it seems reasonable to believe the PRA will commission such reviews in cases where it feels banks do not demonstrate a high level of compliance on day one. There is also a likelihood that thematic or firm-specific reviews in other areas will incorporate assessments of the extent to which MRM issues may exist, for instance a review into market risk is likely to look at whether MRM for the market risk models meets PRA expectations.

Senior Executives and Boards must ensure that banks are in a position to demonstrate that they have robust MRM processes in place. Asking and answering the questions above will help them understand if the bank is on track.

Footnotes

[1] For our view of the wider impact of SS1/23 on Boards, see our blog Countdown to Compliance with SS1/23: Implications for Board members

[2] The model inventory should capture previously implemented models as well as current models and models in development.

[3] SS1/23 appears to expect banks to have a model definition that captures the widest reasonable range of models and, where appropriate, uses risk tiering to classify the lowest tier of models as least material and so apply a “lighter touch” oversight process to them.