It is obvious that IFRS 17 is a new financial reporting standard, but what some underestimate is the requirement for interpretation and decision-making, a new operating model, processes, systems, and data. Since so many aspects of the current reporting process require simultaneous change, validating and confirming the results is extremely difficult since there are no existing IFRS 17 results to compare against.

While vendor solutions may come with a pre-configured test plan, insurers who have selected a vendor solution for the calculation engine and sub- ledger have not removed the need to test. This is because vendors have approached the requirements of the standard differently and hence insurers need to be sure that there is alignment with their approach. Also, each time a new version of the vendor software is released, it is important to retest the results to ensure validity according to expectations.

To validate the output from the new IFRS 17 components (including processes and models), many teams are using an IFRS 17 prototype, often developed in Excel, as a comparison. These prototypes provide the teams with some confidence around the accuracy of their results, without having to derive the numbers from first principles. The challenge is that the prototypes generally lack sophistication and incorporate assumptions and sample data to simplify the time to deployment. Hence the results are not necessarily representative of those that would need to be presented in the Annual Financial Statements. This comparison can create some uncertainty in the results but is a useful method to hasten the testing process.

For those who have attempted to start testing early, the challenge remains that implementations are delayed, aspects of the new finance and actuarial processes have not yet been defined and the correct data is not available. How can you be sure that your results are valid?

As mentioned above, considerable time has been invested in understanding the Standard and in designing a solution. We are seeing that the time remaining for test planning and execution is narrowing as implementations run into delays and additional build tasks are identified, such as the refinement of the expense allocation engine, the process to segregate investment funds and the integration of the reinsurance data. Few organisations have invested adequate time to develop a robust testing strategy and plan containing details of the testing phases, objectives, resources (who is accountable), management plan and timelines. The following diagram illustrates the aspects to be considered in testing.

The readiness of data required to feed the IFRS 17 solution is causing a delay in the testing process. It is ideal to conduct testing with data that is representative of a live environment, but the data is often not ready due to availability, quality, and granularity issues. Considerable time is being spent in remediating poor-quality data. We also see that clients are having issues in finalising the data specifications from the actuarial teams and in accessing the required data from source systems.

For these reasons, many insurers are using manufactured or derived data which takes time to produce and is constructed to be the best version of the data rather than to replicate reality, which comes with its own associated flaws. The use of dummy data can assist in the testing of individual systems but not in the end-to-end testing of the solution.

To cater for the lack of available data, testing approaches are also being amended to leverage restricted volumes and selected ranges of data only. This means that the testing of performance and scalability of the solution to manage substantial volumes of data is being left for much later in the programmes. The question remains, at what point will testing be conducted on real data at the anticipated volumes and complexity and what does this mean for the first reporting deadlines? What impact will the data have on the length of time it takes to run the end-to-end process to ensure that reporting deadlines are met?

One of the requirements of IFRS 17 is to ensure that the finance close and reporting process can be completed within a set timeframe, particularly for listed insurers. To ensure that this is achievable, the end-to-end process must be tested with as close to real-life conditions as possible. These include business as usual demands, representative data volumes, group consolidation tasks and auditor involvement.

If adequate time is not allowed for this level of testing then bottlenecks in the close process relating to the new processes, operating model, and hand-off points will not be identified and resolved prior to the go live date.

It requires a multi-disciplinary team to implement the components of an IFRS 17 solution. Actuaries, accountants, and IT and data professionals approach testing differently; the jargon used by each discipline is unique, the objectives are nuanced, and the execution is dissimilar.

If you are an actuary, you generally test the output of your models by comparing it to a replication model. It is an iterative process comprising variable changes, calibrations, and scenarios as well as scaling from unit to bulk testing.

If you are in data and IT, then you typically test the system against the functional and non-functional requirements, ensure that the components of the system integrate appropriately and that the solution scales for performance and efficiency. Terminology such as system integration and user acceptance testing are commonly used.

If you are in finance, you typically test the output by reconciling to the input from the respective source systems, with a focus on the completeness, accuracy and validity of the results, as well as whether the logic generates the appropriate accounting entries.

Considering this diversity, it is not surprising to witness confusion as to who is responsible for defining and managing a testing strategy and who will drive the integration of the teams.

Never have three such distinct groups had to work together to deliver a complex solution in Insurance. Actuaries, finance, and IT and data professionals have not typically collaborated to bridge their differences in perspective, jargon, and underlying knowledge. These relationships are further complicated by external parties such as technology vendors and implementation partners who are often added to the mix for an IFRS 17 implementation.

Without a concerted effort to break down the silos, there is a considerable risk of wasting valuable time through confusion and misinterpretation in communication, false assumptions around alignment of approach and tasks falling through the gaps due to confusion around responsibility and accountability.

The testing phases, especially the end-to-end solution testing is highly dependent on the successful collaboration of the various disciplines, it is the proof of the pudding!

Testing of IFRS 17 solutions is not simple; new functionality, processes, calculations, systems, and data requires validation. Sufficient time and planning must be factored into IFRS 17 programmes now to ensure that accurate and complete financial results can be delivered in 2023.

Teams cannot leverage typical IT systems testing approaches to ensure correct results; a tailored testing strategy is required where finance, actuarial, data and IT methodologies for confirming functionality and the production of valid, accurate and complete financial results are considered.

Teams must use 2022 to uncover bugs and errors, to expose weaknesses in the reporting process, remediate issues and to demonstrate to business that the numbers produced are fit for sign off.

The requirements to interpret IFRS 17 and to design and implement a suitable solution have forced teams to break down the traditional silos, to effectively communicate requirements and to overcome misunderstandings to find suitable solutions. Effective testing of the new IFRS 17 processes, operating models and systems demand similar effort, planning and execution. It is hard but necessary!