To model or not to model your Operational Risk?

This blog explores an important question for financial sector firms designing Operational Risk (“OR”) capital frameworks: whether to use a modelled or non-modelled approach when estimating internal OR capital requirements (for example, within ICAAP or ICARA assessments).

In practice, the answer depends on several factors, including the size and complexity of the firm, the potential for operational risk events to cause harm, and the firm’s ability to support an appropriate model risk management framework. For some firms a modelled approach may provide meaningful analytical benefits, while for others a simpler approach may be more proportionate.

In this article we set out a practical way to think about this decision. To do so, we first briefly outline the typical OR scenario assessment process used by many firms. OR  ‘scenarios’ are typically used as units of measure representing a class of related loss events (for example cyber attacks or trade errors). While scenario assessments are not the focus of this article, they form the starting point for most OR capital frameworks and therefore provide important context for understanding any modelling decisions.
 

Operational Risk Scenario Process

The OR scenario assessment process typically draws on several key inputs. These are:

• risk and control self-assessments (RCSAs);
• internal and external OR loss event data (ILD & ELD);
• a defined OR taxonomy;
• external benchmarks and industry data; and
• the firm’s risk strategy, risk appetite and relevant KPIs/KRIs.

The information contained within these key inputs are combined in the OR Scenario process through the following steps:

1. Identification of OR Scenarios - information from the inputs above is combined to define a set of OR scenarios (e.g. “Treasury Fraud” or “Cyber / GDPR breach”) that are relevant to the firm’s activities, supported by input, review and challenge from senior management or the board.
2. Data Gathering - For each OR scenario relevant data is gathered including loss data, control assessments and other supporting evidence, and workshop materials are prepared.
3. Expert Assessments - Subject matter experts review the scenario materials in structured workshops and consider what severe but plausible outcomes (and other necessary effects) would be for the firm under each scenario.
4. Quantitative Outcomes - Through the workshops, experts answer a series of structured quantitative questions supported by the available data and workshop materials. These typically include assessments of scenario frequency (where relevant) and estimates of potential loss outcomes, usually expressed as both a higher-severity and lower-severity loss outcomes. Experts are encouraged to consider the underlying drivers of loss (for examples regulatory fines, customer compensation, remediation costs or other financial impacts) when forming their estimates. In addition, potential reputational impacts are assessed for use in the broader stress testing exercises.
5. Governance - the outputs of the OR scenario process  are reviewed, challenged and taken through appropriate governance forums before being used to inform internal OR capital assessments.

Given the low-frequency nature of many operational risk events, the scenario assessment process plays a critical role in helping firms understand their OR exposure and the potential capital required to absorb extreme losses. While historical loss data can provide valuable evidence about the nature and drivers of operational risk events (look out for our upcoming white paper on “Utilising data to enhance operational risk scenario and capital assessments for investment management firms”), its relative scarcity means it is typically insufficient on its own to estimate extreme loss outcomes. As a result, firms must combine available data with forward-looking expert judgement when assessing severe but plausible loss scenarios.

In addition to informing capital assessments, the scenario process can also highlight weaknesses in processes or controls and support improvements to the firm’s OR management framework.
 

Translating Scenarios into Operational Risk Capital

Translating the outputs of the scenario assessment process into OR capital estimates requires an additional step. At this stage, firms must decide whether to use a modelled approach, such as the scenario-based or data-driven Loss Distribution Approaches (“LDA”), or a simpler non-modelled approach, such as scenario summation.

Internal OR capital is typically defined as the level of loss a firm could experience from OR events over a 12-month period at a specified confidence level.

For many PRA-regulated firms this is assessed at the 99.9^th percentile, while many FCA prudentially-regulated firms use the 99.5^th percentile.

In practical terms, these levels correspond to extreme but plausible losses that could occur over a one-year horizon.  Another way of thinking about these is the highest OR loss a firm might expect to experience over any given year if many years were to pass – roughly once every thousand years for a 99.9^th percentile, or once every two hundred for a 99.5^th percentile year, assuming the underlying business environment remains unchanged.

Two Approaches to Estimating OR Capital

Once a firm has completed its operational risk scenario assessment process, the next step is to translate those scenario outcomes into an estimate of internal operational risk capital. In practice, firms tend to adopt one of two broad approaches to perform this translation: a non-modelled approach, often referred to as scenario summation, or a modelled approach, where scenarios are used as inputs into a quantitative framework.

1. Non-modelled (typically scenario summation):

Under a non-modelled approach, experts assess the potential impact of each operational risk scenario directly. Because operational risk capital frameworks typically focus on very extreme loss levels (for example the 99.5%–99.9% confidence level), scenario workshops often ask experts to estimate the loss associated with a very rare but plausible outcome, such as a 1-in-200 or 1-in-1000 year loss.

These scenario estimates are then aggregated across scenarios to produce an overall capital estimate. In practice, firms may simply sum the scenario lossesor apply additional steps to reflect diversification between risks.

When estimating such extreme outcomes, scenario workshops often anchor discussion around a “worst-case-but-plausible” event. Scenario frameworks typically ask experts to reason about this using considerations such as:

• Single-event simplification – as a practical convention, the 1-in-200 loss is often assumed to arise from a single severe event, even though in reality extreme loss years may reflect multiple events.
• Risk drivers – identifying the key drivers that determine loss size (for example trade size, client exposure, system outage duration or market movements) and considering how these could be stressed.
• Plausibility, frequency and severity volatility – judging how far those drivers could reasonably move given how often the underlying event is expected to occur and recognising that some risk types exhibit greater dispersion in outcomes, making very large losses more plausible.
• Conservatism – applying suitable conservatism where uncertainty is high, which may involve stressing key drivers further while being careful not to push multiple independent drivers simultaneously to unrealistic levels.

This approach can be proportionate and relatively simple to implement, particularly for smaller or less complex firms, as it relies primarily on expert judgement supported by scenario workshops and governance processes.

However, the approach can present challenges. Extreme operational risk events are, by definition, rare and often outside the direct experience of the experts participating in scenario workshops. As a result, estimating losses at very extreme levels can place significant weight on expert judgement and imagination. It can also be difficult to reflect the different ways extreme losses might arise — for example whether the worst year for operational risk is driven by one very large event, several medium-sized events, or a combination of losses across multiple scenarios. In addition, when aggregating scenario outcomes, firms must consider how likely it is that multiple extreme events occur in the same year, as simple summation may implicitly assume they do.

2. Modelled (typically LDA):

An alternative is a modelled approach, most commonly implemented through a Loss Distribution Approach (LDA). While there are different variants of LDA model, we
suggest that for many firms Scenario-based LDA will be the approach which
generates the most additional insight, without introducing unhelpful complexity(see our blog on LDA modelling, here, for more details). Under this framework, information either generated during scenario workshops or from other sources (e.g. ILD or ELD) is translated into quantitative inputs that describe how operational risk losses arise, typically including how often events might occur (frequency) and how large the associated losses might be (severity).

In practice, the severity dimension is often described through two related concepts: the typical scale of losses and the degree of dispersion around that scale. Scenario workshops therefore often estimate both a higher-severity and lower-severity loss outcome for each scenario. The higher-severity estimate provides an indication of severity magnitude (the general scale of losses), while the range between the higher and lower estimates provides insight into severity volatility, reflecting how widely losses might vary. Together with frequency assumptions, these inputs help determine the probability of losses of different sizes within the model.

Statistical distributions are then fitted to represent the range of possible loss severities, while frequency assumptions determine how often events occur. The model simulates many potential years of operational risk losses by combining these inputs, generating a distribution of possible annual outcomes from which extreme loss levels, such as the 99.5th or 99.9th quantiles typically required for capital, can be estimated. Correlations between scenarios can also be incorporated so that diversification between risks is reflected appropriately in the final capital estimate.

By modelling the process through which very high-severity losses occur, rather than asking experts to estimate extreme losses directly, this approach can better reflect the ways operational risk events actually arise in practice. For example, extreme annual losses may result from a single large event, several medium-sized events, or a combination of losses across different scenarios. A modelled framework can capture these possibilities explicitly and estimate the resulting extreme loss levels in a more structured and consistent way.
 

 Who can use which approach?

Large, more systemic PRA regulated firms: For the largest and most systemically important firms, regulators typically expect the use of modelled approaches and place greater scrutiny on the methodologies used.

The largest FCA prudentially-regulated firms: While not required to use a modelled approach, most of the largest firms choose to adopt modelling frameworks in order to better quantify their OR exposures.

Small and non-complex firms, with limited potential for harm (to customers, markets and market participants): Smaller firms typically adopt simpler non-modelled approaches, as the cost and complexity of implementing and maintaining a modelling framework may be disproportionate to the size and risk profile of the firm.

Medium sized firms: For many medium-sized PRA and FCA prudentially-regulated firms, the choice is less clear cut. These firms often have a genuine decision to make between modelled and non-modelled approaches, balancing analytical benefits against governance and implementation costs. In practice, many firms struggle to articulate a clear rationale for the approach they have adopted, or to demonstrate how they have balanced the key decision drivers. It is these firms that the rest of this article is focused on supporting in their decision.
 

How to Decide Whether to Model OR Capital

For many medium-sized firms, the decision to adopt a modelled operational risk capital framework is not straightforward. Firms must balance the analytical benefits of modelling against the additional complexity and governance requirements it introduces.

The following dimensions can help structure that decision.

A) Consider the Benefits

A modelled approach can provide several important advantages when estimating extreme operational risk losses.

B) Consider the Costs

While modelling can provide analytical benefits, it also introduces additional complexity and governance requirements.

C) Consider What Is Proportionate for Your Firm

Regulators and markets generally expect more sophisticated operational risk frameworks from larger and more complex firms, particularly where the potential for harm to customers or markets is greater.

Firms may wish to benchmark themselves across several dimensions:
 

D) Consider Your Readiness and Aspirations

For some firms the relevant question may not simply be “model or no model”, but “model now or later”.

Many firms first focus on strengthening the underlying components of their operational risk framework, including control assessments, scenario processes, stress testing capabilities and internal loss data collection, before introducing modelling at a later stage.

In this sense, modelling can form part of a longer-term maturity journey in operational risk management.

E) Decide, Document and Govern

Ultimately, firms should ensure that their choice of modelling or non-modelling approach is deliberate and well documented.

This should include:

• clear articulation of the factors considered;
• explanation of the rationale for the chosen approach;
• appropriate governance review and approval.

Even where firms decide not to adopt modelling, documenting the reasoning behind the decision can help demonstrate that the approach taken is proportionate and appropriate.

Conclusion

The decision to adopt a modelled operational risk capital framework is not a purely technical one. It is a strategic decision that should reflect the size, complexity and risk profile of the firm, as well as its ability to support the governance and expertise required for a modelling framework.

For smaller or less complex firms, simpler approaches such as scenario summation may be entirely appropriate and proportionate. However, as firms grow in scale, complexity or potential for harm, the limitations of purely expert-based approaches can become more significant. In these situations, modelled approaches can provide meaningful advantages by allowing firms to combine frequency and severity information, better utilise available data, and estimate extreme loss outcomes in a more structured and consistent way.

Ultimately, the most important step is not whether a firm chooses to model or not, but whether that choice has been made deliberately and supportably. Firms should ensure that their approach is clearly justified, appropriately governed, and aligned with the role operational risk capital plays in their wider risk management and decision-making framework.

For firms that conclude that modelling may be appropriate, either now or as part of a future maturity journey, it is often beneficial to focus on frameworks that capture the key drivers of operational risk losses while remaining as simple and transparent as possible.

If you are interested in knowing more about anything you have read in this article or how we might support you in your operational risk data, scenarios and capital assessment journey, please reach out to the Capital Clarity team here or see our webpage: Capital Clarity | Deloitte UK.