Evidencing Consumer Duty compliance: shedding light on the FCA’s data expectations

Who is this blog for?

Board members, senior executives, Consumer Duty champions, and Consumer Duty (the Duty) monitoring and reporting leads.

At a glance:

• The Duty places an ongoing obligation on firms to deliver good customer outcomes. This, together with the clock counting down to the first Duty Board Report, puts the spotlight on what data firms need to collect to evidence compliance and underpin the conclusions in their Board report. Boards need to be fully engaged in this process.
• Over the last 16 months, the FCA has published a range of documents that refer to the quantitative or qualitative data that it sees as relevant to monitor customer outcomes. Some of these are specific to a sector while others are more general in their application.
• We have analysed 36 FCA Duty‑related publications to understand the FCA’s data expectations in more depth and identified 213 data items which we mapped to at least one of the four Duty outcomes and 16 data themes.¹
• Firms can use this analysis as a point of reference to improve their understanding of what data they need, to assess whether they are delivering good customer outcomes or to spot and address data gaps.
• Although this analysis can provide useful insights it will not provide all the answers. Firms will still need to define good customer outcomes, the metrics they are assessing and the baseline/thresholds for action. They will also need to segment the data that enables them to monitor outcomes for different customer groups, while also identifying how customers with vulnerable characteristics are faring in comparison to other groups of customers.
• Complaints data, Customer Feedback and Quality Assurance are the most material data themes linked to all four Duty outcomes. In our view, firms should develop the necessary systems to gather existing MI and data across these three data themes to enable them to map the data against each of the Duty outcomes and to compare outcomes between distinct groups of customers.
• Beyond the granular data needs, firms need to assess the fitness and robustness of their overall conduct data frameworks to ensure they can source, filter, apply and validate the data required for evidencing Duty compliance.

Overview

Building on our previous ECRS perspective: Preparing the Consumer Duty Board report, this article focuses on the role of data and MI in evidencing Duty compliance. The FCA has been clear it expects firms to use a wide range of data and MI to evidence compliance with the Duty². As Sheldon Mills recently said: “Firms also need to get serious about their data and not assume they can just re-package existing information”.

We carried out a detailed analysis of FCA Duty‑related publications to help firms identify key data points for them to assess whether they are meeting the expectations of the Duty.

Methodology

We analysed 36 FCA Duty-related publications - including podcasts, Dear CEO letters, sectoral multi-firm reviews, speeches, and published data sources^3,4 - published during 2023 as well as PS22/9 and FG22/5 - we will refer to this body of content as the corpus.

Our analysis consisted of identifying and extracting every mention or reference to a quantifiable data metric or data source(data item) in the corpus that firms may use to monitor customer outcomes. We then mapped each data item identified to a data theme (aggregation of related data items) and to one or more relevant Duty outcomes.

We identified 213 data items which we sorted into 16 data themes. Diagram 1 illustrates the methodology.

Diagram 1 - Methodology

Practical considerations when using this analysis

Our work is intended to help firms navigate what potential data/MI they could use to assess whether they are meeting the Duty. Firms should consider the inherent risk associated with their products and customer segments and what MI they need.

Overall findings

Diagram 2 - Data items by Duty outcome⁵

Price and Value in the lead but all outcomes require a significant range of metrics

The Price and Value outcome is the most data-focused Duty outcome with 59% of data items identified mapped to it, in part because price and value lend themselves to easier quantification than the other outcomes. Data items mapped to all other outcomes are relatively evenly distributed, highlighting the need for firms to identify metrics and data sources across all outcomes including those where data might be scarcer, such as consumer understanding.

This reinforces the message that firms need to consider a balanced range of data to provide a high-definition picture of their customer outcomes.

Vulnerable customers: is what we know enough?

Ensuring firms monitor the outcomes for customers with characteristics of vulnerability is a longstanding area of regulatory focus dating back to the FCA’s final guidance on the fair treatment of vulnerable customers early in 2021. The Duty can be seen as a regulatory catalyst that will help the FCA ensure vulnerable customers receive good outcomes. To this end, firms need to be able to monitor outcomes for different customer groups, while also identifying how customers with vulnerable characteristics are faring in comparison to other groups of customers. Firms should also consider what additional actions they have taken for their vulnerable customer groups, and the value that customers receive from that. Firms might find delivering good outcomes for vulnerable customers in closed portfolios particularly challenging due to data gaps. The FCA expects firms that cannot fill data gaps to take additional steps such as enhanced outcomes testing for these customers.⁶

The FCA⁷ found that firms that used a wide range of data metrics and sources were better able to monitor outcomes for different groups of customers and customers in various circumstances and scenarios. These firms were able to identify areas for improvement and remediation for customers. This means firms should ask themselves the question: do we know enough about product performance and outcomes for distinct groups of customers? Using a healthy range of metrics and sources is a good starting point, but drilling deeper into metrics with a wide distribution and exploring what and who is in the tail of these distributions is a must to answer the question with confidence. Across our analysis, we chose not to treat vulnerable customers as a separate data theme. Instead, outcomes for vulnerable and distinct groups of customers should be considered central to most of the metrics and data sources the firm uses to monitor its compliance with the Duty.

Data sharing through the distribution chain

To evidence Duty compliance, the FCA expects firms to request and analyse a range of qualitative distributor data, such as distributors’ feedback, records of target market assessment, and copies of written agreements between distributors and manufacturers. This is in addition to quantitative data such as broker commissions or distributor sales information. The FCA has expressed concerns that some firms are not exchanging sufficient information to assess value, or to evidence appropriate distribution of the product or service.⁸ Firms relying on third parties to provide them with data to monitor outcomes might want to use our analysis as a starting point for deciding which metrics they could request to monitor outcomes across the distribution chain.

The big three

We identified three data themes that are the most material to evidencing compliance with the four Duty outcomes. These are Complaints data, Customer Feedback and Quality Assurance / Outcomes testing. Examples of data items in these themes are formal complaints numbers, root cause analysis, informal complaints such those on social media, customer satisfaction scores and survey results, and customer file reviews. Most firms have mature formal complaint recording systems and dashboards but it is unclear to what extent the data they contain can be extracted and mapped to the different outcomes or to compare outcomes between certain groups of customers. In our view, firms should develop the necessary systems to gather existing MI and data across the three big themes to enable them to map the data against each of the Duty outcomes. More details on customer outcome testing can be found in our report Improving Customer Outcome testing: A practical guide for boards.

Dominant data themes for each outcome

We analysed the distribution of data themes for each Duty outcome. To make this analysis more insightful, we excluded Complaints, Customer Feedback and Quality Assurance from the outcome specific analysis. This means that these three themes will not appear in the outcome‑specific charts. This is because they would all occupy the top three spots for each outcome and would reduce the visibility of those themes that are more unique to each outcome. Those who would like a more in-depth view of the findings are welcome to contact us.

A. Products and Services outcome

Diagram 3 - Distribution of Data themes in Products and Services outcome

In the Products and Services outcome, Distribution and Third-Party Data is the most prevalent theme with 10 data items, followed closely by Customer Profile/Target Market with nine data items. Distribution and Third-Party Data refers to information and insights collected by firms from external sources, such as distributors or brokers. These include metrics such as distributor data on early product cancellations and feedback from firms in the distribution chain. The FCA expects a range of third-party data to form part of the evidence of compliance with the Products and Services outcome.

Customer Profile/Target Market refer to data that is used to create a comprehensive profile of customers and to assess whether the product or service is being distributed to the target market. Firms are expected to have a clear definition of their target market that includes appropriate segmentation within the target market to allow testing. Some data item examples include: number of sales outside target market and number of disengaged customers. The FCA has requested that firms provide evidence of how they are serving their target market which involves retrospective testing of whether the product has been sold to the target market and/or met the needs of the target market. Where there are outliers, firms should take action to engage those customers. We understand that recent FCA challenge to the insurance sector for example has focused on compliance with Product Oversight and Governance rules and on tight target market definitions, adding to the pressure for firms to be more rigorous when it comes to target market matters.

B. Price and Value outcome

Diagram 4 – Distribution of Data themes in Price and Value outcome

It is no surprise that Fees and Charges and Pricing are tied for the top spot data theme in the Price and Value outcome with 18 data items each. Fees and Charges include transaction fees and ancillary charges. The Pricing data theme includes the overall price consumers pay over the lifetime of the product, and potential issues arising from cross-subsidies or cross-selling.

In our analysis, we found that the Price and Value outcome has the highest number of data items that are sector‑specific. Examples of sectoral metrics include: fees charged in the second charge mortgage market, E-money redemption fee data and prices charged for premium finance. This is in addition to expected metrics that include distributional analysis and a focus on data to identify subsets of customers paying a disproportionately high margin or fee or premium relative to the benefits received.

Additionally, the data theme of Benchmarking is unique to the Price and Value outcome and refers to the expectation that firms will compare and evaluate the performance of their products or services against the market. Firms need to be able to evidence that their fees and pricing are in line with market norms, or reasonable with respect to the total costs associated with the product or service. That said, the FCA recently indicated that in many value assessments firms are not relying on solid data and other credible evidence to justify their products’ value to retail customers and pointed out that relying solely on benchmarking will not be sufficient to demonstrate the delivery of good value.⁹ Finally, firms need to be able to interpret complaints data or quality assurance results to gain insight on what they mean for price and value considerations. Usually, firms will consider value at the product level rather than individual, but for example an upsurge in price or charges related complaints could be indicative or broader issues in a product class. This type of insight can help firms act more quickly to avoid harm levels deteriorating.

C. Consumer Understanding

Diagram 5 – Distribution of Data themes in Consumer Understanding outcome

Customer Engagement vs Inertia is the dominant data theme in the Consumer Understanding outcome with 16 data items. This includes a range of quantifiable measures that evaluate the effectiveness of business communication as well as metrics that indicate customer loyalty which, when related to inertia, could indicate poor outcomes. Many of the data items in this theme relate to communications testing and include percentage of customers who take expected action after communication, pre-test results of interventions/communications, customer communications response rates, data on impact of communications across customer journeys such as, for example, customers dropping off a journey after viewing a risk warning, retention rates, and data on customers following communications’ instructions, to name but a few.

The Customer profile/Target Market data theme includes both qualitative and quantitative data metrics and sources. Examples include the need for firms to consider data on consumer behaviour driven by change in product or service design, data from consumer organisations and other external sources on consumer outcomes such as the FCA’s own financial lives survey and other consumer research.

D. Consumer Support

Diagram 6 – Distribution of Data themes in Consumer Support outcome

Time Taken for Resolution of customer issues or requests is the dominant data theme in the Consumer Support outcome and is unique to this outcome with 16 data items. Some examples include time taken for customers’ claims processing, queue times in branches, call waiting times, number of times the customer needed to repeat the reasons for their call, first contact resolution rates, call transfer accuracy metrics and speed to answer.

Customer abandonment and Customer Profile/Target Market¹⁰ are mapped to eight data items each. Customer abandonment is when a customer initiates a transaction or interaction, but then terminates or abandons the process before completing it. Examples in this data theme include call abandon rates or abandoned claim rates.

The right data is only the beginning

Our analysis highlights the wide range of data metrics and sources that firms are expected to consider when monitoring customer outcomes. But having the data is only the beginning. It is essential that firms ensure the Duty is embedded in their culture, starting with being able to demonstrate how the Board has been involved in the Duty journey including the preparation of the Duty Board report. At a more detailed level, defining good customer outcomes is critical to evidence Duty compliance and many firms have already established their baseline standards at a high level. However, we believe firms will also need to develop a more granular view of good outcome indicators, such as on remuneration and incentives and effectiveness and scope of Duty training. A good starting point might be to assess data metrics and sources available across the data themes and identify the baseline for good outcomes in the key themes for each outcome.

Conclusion

Looking ahead, firms should consider prioritising taking action on indicators of potential customer harm and developing an early warning system to anticipate harm by joining the dots across different data themes. This could include overlaying data on loss ratios or high charges with levels of customer engagement or consumer understanding metrics to understand the correlation and interdependencies of different metrics that could indicate potential future harm. In the medium to long term, a system that allows firms to anticipate sources of harm should pay for itself with reduced regulatory challenge, need for remediation or redress and better customer engagement.

Stepping back from the detailed data and metrics needed for Duty compliance, firms need to assess the fitness and robustness of their overall conduct data frameworks. The framework should allow firms to source, filter, apply and validate the data required and ultimately help the business to adapt to changing customer needs and market trends. This is a material challenge that will most likely play out in the months and years ahead as firms get to grips with how far they have to go to continue to evidence Duty compliance.

______________________________________________________________

References:

¹See Technical Annex to this paper for more information on the methodology of our analysis.

²Webinar transcript – Consumer Duty: The Next Steps (fca.org.uk)

³Consumer Duty: the art of the possible in a year – Sheldon Mills, FCA Executive Director

⁴ See more detail in Technical Annex

⁵The combined total exceeds 100% as some data items can be mapped to more than one Duty outcome.

⁶Consumer Duty: the art of the possible in a year – Sheldon Mills, FCA Executive Director

⁷Retail banking Consumer Duty multi-firm work | FCA

⁸Webinar transcript – Consumer Duty: The Next Steps (fca.org.uk)

⁹Consumer Duty: the art of the possible in a year – Sheldon Mills, FCA Executive Director

10Please refer to Consumer Understanding outcome section for examples and explanation of this data theme.