Opening the black box: Managing algorithm risks

When algorithms go wrong

From Silicon Valley to the industrial heartland, the use of data-driven insights powered by algorithms is skyrocketing. Growth in sensor-generated data and advancements in data analytics and cognitive technologies have been the biggest drivers of this change, enabling businesses to produce rich insights to guide strategic, operational, and financial decisions. Business spending on cognitive technologies has been growing rapidly. And it’s expected to continue at a five-year compound annual growth rate of 55 percent to nearly $47 billion by 2020, paving the way for even broader use of machine learning-based algorithms.1 Going forward, these algorithms will be powering many of the IoT-based smart applications across sectors.

What are algorithmic risks?

Algorithmic risks arise from the use of data analytics and cognitive technology-based software algorithms in various automated and semi-automated decision-making environments.

• Input data is vulnerable to risks, such as biases in the data used for training; incomplete, outdated, or irrelevant data; insufficiently large and diverse sample size; inappropriate data collection techniques; and a mismatch between the data used for training the algorithm and the actual input data during operations.
• Algorithm design is vulnerable to risks, such as biased logic, flawed assumptions or judgments, inappropriate modelling techniques, coding errors, and identifying spurious patterns in the training data.
• Output decisions are vulnerable to risks, such as incorrect interpretation of the output, inappropriate use of the output, and disregard of the underlying assumptions.

Why are algorithmic risks gaining prominence today?

While algorithms have been in use for many years, the need to critically evaluate them for biases, lack of technical rigour, usage flaws, and security vulnerabilities has grown significantly in the recent times. This growing prominence of algorithmic risks can be attributed to the following factors:

• Algorithms are becoming pervasive
• Machine learning techniques are evolving
• Algorithms are becoming more powerful

What do algorithmic risks mean for your organisation?

As noted previously, data analytics and cognitive technology-based algorithms are increasingly becoming integral to many business processes, and organisations are investing heavily in them. But if the issues highlighted in this report aren’t adequately managed, the investments may not yield the anticipated benefits. Worse yet, they may subject organisations to unanticipated risks.