For all the enthusiasm, research and development poured into Generative AI, we have not seen commensurate investment in addressing shortcomings. Yes, there have been numerous experiments and hacks to hijack models, but little attention given to managing the risks.

Fortunately, interest in AI Quality and AI Ethics – collectively Trustworthy AI – is maturing and provides a more than ample launchpad from which to consider risk management of Generative AI. At first glance, new dangers accompany the new capabilities of GenAI, yet looking below the surface, these risks are merely new manifestations of familiar themes that have always been factor in development and deployment of AI. The concepts and tools that enable Trustworthy AI still apply, even as many of the emerging risks and problematic scenarios are more nuanced.

To prepare the enterprise for a bold and successful future with Generative AI, it makes sense to first understand the nature and scale of the risks, as well as the governance options that can help mitigate them.

Robustness and reliability

A hot topic of GenAI blogs is the tendency of LLM models to “hallucinate” or, in other words, to generate inaccurate and/or inconsistent content. Generative AI models are designed to create data that closely resembles real data, but that does not necessarily equate to the generated output being factually correct. Sometimes, the model takes a wrong turn. They can be highly sensitive to the wording of queries, or “prompts.” GenAI hallucinations are merely the natural language equivalent of statistical confidence prevalent in any AI model. The trouble with them is the convincing, authentic, even authoritative presentation.

Fairness and impartiality

Limiting bias in AI outputs is a priority for all models, whether machine learning or generative. The root in all cases is latent bias in the training and testing of data. Organizations using proprietary and third-party data are challenged to identify, remedy, and remove this bias so that AI models do not perpetuate it. This is not just a matter of unequal outcomes from AI-derived decisions.

For example, a Generative AI-enabled chatbot that produces coherent, culturally specific language for an audience in one region may not provide the same level of nuance for another, leading to an application that simply performs better for one group. In practice, this could diminish end user trust in the tool, with implications for trust in the business itself.

Transparent and explainable

Given the capacity for some Generative AI models to convincingly masquerade as a human, there may be a need to explicitly inform the end user that they are conversing with a machine. When it comes to Generative AI-derived material or data, transparency and explainability also hinge on whether the output or decisions are marked as having been created by AI.

For example, a Generative AI text response would be more credible if it could be easily fact-checked, citing its sources. Or an image could confess its generated origins through watermark. Generated recommendations – especially in sensitive and highly regulated sectors such as healthcare –may well require notification to the user that the text was indeed machine generated, or even require further explanation into the rationale behind its recommendation by revealing its sources.

The enormity of the foundation models powering Generative AI magnify the “black box” problem of classical AI greatly. Large Language Models, for example, are very deep neural networks with anywhere from hundreds of billions to a trillion parameters, their sheer size making traditional approaches to model explainability impractical. Issues with transparency and explainability are compounded by the challenge of aligning Generative AI outputs with enterprise priorities and values. To help promote model transparency and ongoing improvement, organizations may look to leverage technology platforms that help evaluate and track model performance, and assess, manage, and document each step of the AI lifecycle. This helps the enterprise evaluate whether an AI tool performs as intended and aligns with the relevant dimensions of trust.

Safe and secure

Powerful technologies are often targets for malicious behavior, and Generative AI can be susceptible to harmful manipulation. One threat is known as prompt spoofing, wherein an end user crafts their inputs to trick the model into divulging information it should not, not unlike how traditional AI models are targeted for reverse-engineering attacks to reveal the underlying data. In addition—particularly given Generative AI’s capacity to mimic human speech, likeness, and writing—there is a risk of massive misinformation creation and distribution.

Generative AI can permit near-real-time content personalization and translation at scale. While this is beneficial for targeted customer engagement and report preparation, it also presents the potential for inaccurate, misleading, or even harmful Generative AI-created content to be disseminated at a scale and speed that exceeds the human capacity to stop it. A Generative AI-enabled system could erroneously create products or offerings that do not exist and promote those to a customer base, leading to brand confusion and potentially brand damage.

More troublingly, in the hands of a bad actor, Generative AI content could be used maliciously to create false or misleading content to harm the business, its customers, or even parts of society. To promote Generative AI safety and security, businesses need to weigh and address a myriad of factors around cybersecurity and the careful alignment of Generative AI outputs with business and user interests.

Accountable

With more traditional types of AI, a core ingredient for ethical decision making is the stakeholder’s capacity to understand the model, its function, and its outputs. Because an AI model cannot be meaningfully held accountable for its outputs, accountability is squarely a human domain. In some use cases, Generative AI makes accountability a much thornier and more complicated matter.

Imagine a potential not-so-distant future, in which large organizations deploy multiple AI agents, ranging from customer service all the way to a public “AI spokesperson.” The AI spokesperson has access to the full suite of social and marketing tools, customer profiles, enterprise data, and more. It could be tuned to specific subjects (e.g., home improvement tips from a home goods retailer).  It could be tweaked to take on a persona befitting the brand or target customer. It could also be deployed at scale, as a branded personal assistant. Forget apps on the smartphone, think of bots for each organization – providing highly personalized recommendations and with a persistent memory able to recall past interactions, even across multiple platforms (e.g., mobile phone, social media, company website, support call centers).

How can the business direct the trustworthy behavior of a persistent AI personality that is operating at such an enormous scale that it eclipses the possibility for transparency and keeping a human in the loop? What happens if one AI spokesperson veers off track in an effort to provide encouragement to its user, or is hacked into lying or deliberately encouraging the misuse of a competing product? Ultimately, the organization providing such a tool is accountable for its outputs and the consequences of those outputs. Whether the enterprise uses a model built in-house or purchases access through a vendor, there needs to be a clear link between the Generative AI model and the business deploying it.

Responsible

Just because we can use Generative AI for a given application does not always mean we should. Indeed, the sword of Generative AI cuts both ways, and for all the enormous good it can be used to promote, Generative AI use cases could also lead to significant harms and disruption. Deep Fakes offer a handy illustration: imagine a scenario where a politician is running for office and an opponent group uses Generative AI to simulate a hyper-realistic video of the candidate saying and doing untoward things. Without context, the voting populace may begin to doubt what is true. This injects confusion and political disruption, and more profoundly, it could undermine the government systems that are crucial to a healthy society.

Imagine a similar scenario in the hands of political activists on the global stage. Audio data could mimic a world leader threatening conflict. Translations could be augmented to misrepresent intentions. Videos could be created to show military conflict that is not actually occurring. And all of this can be done at relatively low cost, in real time, personalized to the audience, and delivered at scale. In this confused space, the line between objective truth and Generative AI-enabled deception blurs.

Yet, even when Generative AI outputs are fruitful (or at least benign), there remain questions about responsible development and deployment. For example, consider that training, testing, and using Generative AI models can lead to significant energy consumption, with implications for climate change and environmental sustainability. This consequence of Generative AI deployment may not align with an organization’s goals for reducing their carbon footprint. In this way, the question of whether it is a responsible decision to develop and deploy a model depends on the organization and its priorities. What is judged to be a responsible deployment by one organization may not be judged the same by another. Enterprise leaders need to determine for themselves whether a Generative AI use case is a responsible decision for their organization.

Privacy & Confidentiality

The data used to train and test Generative AI models may contain proprietary, sensitive or personally identifiable information that needs to be obscured and protected. As with other types of AI, the organization needs to develop cohesive processes for protecting its own intellectual property and trade secrets as well as assuming responsible stewardship over the private data of all stakeholders –data providers, vendors, customers, and employees. As a part of this, the enterprise may rely on anonymization, use synthetic data, or blocking the input of personal data into the system.

There are also significant questions around Generative AI-derived intellectual property. Copyright laws are generally concerned with guarding a creator’s economic and moral rights to their protected work. What happens when something is created by Generative AI with minimal or no human involvement? Can that be copyrighted? For enterprises, consider how Generative AI is used to create business-critical data (e.g., for product prototyping) and whether its derivations legally and solely belong to the organization.

Risk Management in a Rapidly Evolving Field

Effective, enterprise-wide model governance is not something that can be dismissed until negative consequences emerge, nor is it sufficient to take a “wait and see” approach as government rulemaking on Generative AI evolves. The ability for AI – and in particular Generative AI – to be deployed at scale, seeping into an increasing number of processes and products, makes sound risk management tantamount to success and to confidently bridling this powerful technology for competitive advantage. Organizations must consider both sides of the equation – the opportunity and the potential consequences of negligence – already today. The field of AI was already complex and fast evolving, now even more so with the arrival of GenAI. To keep the upper hand, organizations must be conscious of and deliberately manage the associated risks today, while keeping eyes open for future risks yet to emerge as the technology matures and use cases become more widespread.

Fortunately, just as the domains of AI trust hold true for Generative AI models, so too does the prescription for governance. At its core, it is a matter of aligning people, processes, and technologies to promote risk mitigation and establish governance. With the workforce, the duty to identify and manage risk is shared throughout the organization among both technical and non-technical stakeholders.

These stakeholders need a clear sense of roles and responsibilities, as well as workforce training opportunities to enhance their AI literacy and skills to better work with and alongside this technology. The enterprise may also create new roles and groups within the organization. This may take the form of focused AI Ethics and AI Quality boards or become embedded in established control structures. In either case, organizations which populate their AI development and governance teams with a diverse set of skills and backgrounds will be best equipped to shape and govern AI with a well-rounded compliment of perspectives and lived experiences.

In addition to readying the workforce, processes may need to be re-invented to both profit from new GenAI capabilities as well as effectively guard against potential issues. Risk assessment and analysis should be woven into the entire (generative) AI lifecycle, with regular waypoints for stakeholder review and decision making. These include considerations about data (prompts) storage, transfer, and later use to enhance or improve the model, with vital input from multiple practices – among them, legal, compliance, and cybersecurity. Compliance will grow in importance as regulatory bodies worldwide establish rules for the use of Generative AI. Already now in its current draft form, the EU AI Act takes direct aim at risks and responsibilities around foundation models and general purpose AI, with measures such as documented impartiality, model explainability, and data privacy taking a more prominent role.

Time to Act

The game-changing opportunities associated with Generative AI are numerous, yet risks are significant and must be taken seriously. As in any field, managing risks effectively requires focused effort. To ignore them would be careless, even negligent, given the versatility of the technology, lending itself to application in all manner of processes and products. Avoiding Generative AI is not an effective risk management strategy – arguably, the consequences of not embracing Generative AI outweigh the downsides of adopting and integrating the technology into the business model. Organizations across industries are exploring how to capitalize on Generative AI capabilities, and as with many transformative technologies, standing still means falling behind.