To model or not to model your Operational Risk?

This blog may be of interest to financial sector firms that are considering use of a modelled or non-modelled approach to calculate internal estimates of their Operational Risk (“OR”) Capital.

In this article we will look at a typical scenario assessment process, and then consider the quantification approaches that are suitable for different types of business. We will also highlight the strengths and weaknesses of using different techniques to support the business requirements for firms that are developing an OR quantification approach.

Introducing Operational Risk

Operational Risk is the potential for loss resulting from inadequate or failed internal processes, systems, human errors, or external events, and poses a significant threat to the stability and success of businesses across all industries. In this dynamic and interconnected world, both robust measurement of existing risks and the need for a way to capture and quantify emerging risks from threats such as Climate Change, Cyber attacks, and Pandemics, etc, require a forward-looking Operational Risk scenario assessment process. This assessment process is a critical element within a robust Risk Management Framework for both FCA and PRA regulated Financial Institutions, and therefore serves as a core input for Internal Capital estimates of Operational Risk used in firm’s ICAAP and ICARA.

Operational Risk Scenario Process

Before considering the core steps that are involved in the scenario assessment process, we should first look at the typical building blocks for OR scenario analysis. These are:

• Risk control self-assessments (RCSAs);
• OR event databases;
• A risk taxonomy;
• External benchmarks such as the Basel risk categories; and
• Risk strategy, risk appetite and other applicable KPIs/KRIs.

The information contained within the building blocks described above are combined in the OR Scenario process through the application of the following steps:

1. Identification of OR Scenarios - The information in the building-blocks is synthesized to define a reasonable number of Operational Risk Scenarios (with names such as “Treasury Fraud” or “Cyber / GDPR”) with board level input also used in their selection.
2. Data Gathering - For each OR Scenario defined above relevant data is gathered, and comprehensive workshop materials are assembled.
3. Expert Assessments - Relevant experts are identified and brought together to review the materials in the OR Scenario workshop, and consider severe, but plausible outcomes for the firm for each of the identified OR Scenarios
4. Quantitative Outcomes - Through these workshops, the firm attains a series of quantitative outcomes, which are centered around potential high-impact loss events. A post-analysis report is then created, explaining the process followed and the associated outputs.
5. Governance - As a final step, the results of the OR Scenario process are reviewed, challenged and ultimately approved by appropriate governance forums, confirming the outputs can be used to inform internal Operational Risk capital assessments.

Given the low frequency nature of most ORs, the OR Scenario process is typically hugely important in understanding the OR facing a firm and thus the associated OR capital requirement. It allows firms to overcome an otherwise often crippling lack of historical data, by applying a forward looking perspective on the problem and identifying any immediate changes to business process / controls where weaknesses are found.

Operational Risk Modelling Approaches

Progressing from OR inputs to OR capital requirements involves an additional step.

Here, many firms are faced with an important final choice: whether to use modelled approaches, which includes Scenario-Based, Data-driven, or hybrid Loss Distribution Approaches (“LDA”), or to use a simpler non-modelled approach, such as the Simple Scenario-Summation method.

Who can use which approach?

Large, more systemic PRA regulated firms: there will be increased expectation for modelled approaches and heightened focus by operational risk PRA specialists on the approach used.

The largest FCA prudentially-regulated firms: Are not mandated to use a modelled approach, but typically the top 10-30 in terms of size use a modelled approach.

Small and non-complex firms: typically adopt a non-modelled approach, as the perceived cost of employing models is deemed to be disproportionately high in relation to their organisational size and complexity.

Medium sized PRA-regulated and FCA prudentially-regulated firms: Have a genuine choice as to the approach taken. It is these firms that the rest of this article is focused on.

What is Internal OR Capital and why is a model needed at all?

In the world of Operational Risk management, OR capital is determined using the following (typical) stress levels:

• 99.9^th percentile for PRA regulated firms.
• 99.5^th percentile for FCA prudentially-regulated firms.

In layman's terms, both of these represent an extreme-but-possible financial loss a firm could face from Operational Risk events over the next 12 months (and we note that for some FCA prudentially-regulated firms this is how they conceptualise the problem rather than using specific probability quantiles).

Another way of thinking about these is “the highest amount you would expect to lose from OR over any 12-month period if a thousand (the 99.9^th percentile) or two hundred (the 99.5^th percentile) years were to pass”, assuming all other business environment and control factors were to remain the same.

A modelled approach is helpful here because it allows firms capture three key dynamics:

1. Extrapolating from events that experts and firms have experienced to those they have not

99.5 and 99.9 events are extremely improbable, and thus, no firm or individual has experienced such losses in their lifetime (unless they were exceptionally unlucky!). As such, it is very hard for experts to estimate these directly.

BENEFIT using a modelled approach we can extrapolate from things we can imagine, to those we cannot.

2. Combining frequency and severity information

The occurrence of any material OR event is typically characterised as a low frequency event (subject to the effectiveness of each organisation’s controls), but conversely can have very high severities. As such, the nature of these extreme-but-possible events makes it challenging to imagine whether they are more likely to come from one large event, two medium-sized events, or a series of smaller events.

BENEFIT A modelled approach allows us to combine the frequency and severity information about a scenario together in a logical way to calculate an aggregate loss profile.

3. Allowing for diversification benefit

When we think about combining the risk profile of our different OR Scenarios we often start by assuming that all the extreme-but-possible events happen in the same 12 month period, without allowing for the fact that this makes these already very unlikely events even more unlikely in aggregate.

BENEFIT Using a model that can incorporate diversification benefit is thus considerably more accurate and can also lead to a lower capital charge.

Decision Making Process

When deciding which approach to use for the calculation of OR capital the strengths and weaknesses of using a model should be assessed as part of the business case. We have highlighted those which we have seen to be the most material when helping our clients through their planning process.

The main drawbacks of using a model are:

• Model Risk Management - Regulatory expectations around Model Risk Management (“MRM”) are on the rise; a firm wishing to use a modelled approach should be able to build and maintain a robust modelling framework, have sufficient governance arrangements in place including board level oversight, have access to sufficiently statistically-trained individuals, be able to conduct validations at a regular frequency, and have all other aspects of a suitable Model Risk Management Framework (“MRMF”) in place (for more information see some of our other blogs on the topic: Model risk management | Deloitte UK Model risk management | Deloitte UK Model risk management | Deloitte UK or reach out separately to the blog’s authors).
• Training requirements - The greater complexity of a modelled approach also means that more training is required in the model and its outcomes, before their use can be effective. For some firms, who are not yet ready to take these steps, the non-modelled approach is preferable to running a modelling framework improperly and exposing the firm to unacceptable Model Risk.

The main benefits of using a model are:

• Technical Accuracy - As already detailed, when using a modelled approach we can extrapolate from things we can imagine, to those we cannot, it allows us to combine the frequency and severity information about a scenario together in a logical way to calculate an aggregate loss profile, and it can accurately incorporate a diversification benefit to reduce capital charge.
• Enhanced Data Utilisation - Adopting a modelled approach can allow firms to leverage data more intelligently. For instance, certain data points may be relevant for assessing severity, but not for frequency evaluation or vice versa. This principle also extends to the use of external data, ensuring a more refined and accurate analysis.
• Meaningful and Tangible Process - Relying on experts to guesstimate the likelihood of rare events, such as 1-in-200 or 1-in-1000 year occurrences, can sometimes feel like an impossible task, which can both damage engagement with the OR Scenario process and reduce the credibility of its outcomes. Employing a model allows for the assessment of events that happen once in a career or every 1-in-40 years to be less subjective, providing a more tangible and intuitive perspective across various scenarios. This approach helps maintain engagement and buy-in from a firm’s experts, which is crucial for a robust OR Scenario process.
• Comparability of Scenarios - A significant advantage of the model lies in its ability to transform dissimilar scenarios (e.g., low frequency, high severity, and higher frequency, lower severity) into comparable outcomes, expressed as losses at 1-in-200 or 1-in-1000 years (or any other probability that may be required). Without this capability, experts struggle to differentiate between scenarios accurately when assessing relative risk in the OR Scenario process. The model ensures a clearer understanding, preventing potential biases in risk assessment.
• Higher Accuracy supports Credibility - Emphasising higher accuracy, the model enhances the credibility and usability of the OR capital assessments’ outcomes. This becomes essential when a firm wishes to use these outcomes to support or make strategic investment decisions in controls and automation over the medium to long term. If OR Capital figures do not represent the best estimate of risk, rendering them inadequate for such decisions, the entire process loses value, fails regulatory use-test requirements, and the opportunity to minimise future Operational Risk losses is missed.

Conclusion

Based on the above, we believe that for many of the medium-to-larger and more complex firms currently using non-modelled approaches, there is a strong business case to move to a modelled approach.

When choosing which modelling methodology to use we typically advise honing in on methods which maximise accuracy while minimising complexity, the so called “as-simple-as-possible" principle - long story short: less is more when it comes to OR modelling, but there are a few essential ingredients we have to keep. Feel free to reach out directly to find out more or await our blog on the topic in the coming weeks.

Our proprietary solution (Capital Clarity | Deloitte UK) is one such modelled approach that is gaining a lot of interest in the market. It removes the need to build and implement your own OR model, gives you access to a powerful and user friendly tool that is “glass box” rather than “black box” in nature, gives you access to trained OR modelling and OR Scenario experts, and all the associated analytics needed to use such a model effectively.

If you are interested in knowing more about anything you have read in this article or how we might support you in your OR Capital assessment journey, please reach out to the Capital Clarity team.

Capital Clarity webpage: Capital Clarity | Deloitte UK