How can my organization evaluate third-party Generative AI use?

Clarify whether the GenAI is being provided as an off-the-shelf application, service (AIaaS), or hybrid. The nature of GenAI procurement can have significant implications for each party’s responsibilities and liabilities for ongoing opportunities and challenges tied to its functionality. This can have an impact on everything from design to implementation and rollout, which in turn affects adoption, quality, organizational fit, and workflow integration.

Define the required level of autonomy and human oversight of the GenAI and clarify which parties will assume final responsibility for AI-enabled decision-making. Parties should endeavor to provide a clear chain of accountability commensurate with regulatory practices, particularly where GenAI supports highly impactful or specialized decisions.

GenAI systems are typically trained on extensive datasets and may not possess nuanced understanding of local facts, laws, or cultural variances. Identify which party will be responsible for testing, retraining, and fine-tuning the GenAI model to suit particular demographics in order to avoid pitfalls in deployment and application of the GenAI system.

Clarify which parties have responsibility for ongoing maintenance, training, versioning, auditing for quality/bias, change control, and regulatory reports and reapprovals. These duties help ensure that the GenAI systems continue to operate within regulatory and ethical boundaries.

Define the qualifications and training required for end users and assign responsibility for performing and enforcing those standards. Additionally, assign responsibilities for providing this training, which can influence the outcomes provided by the GenAI systems.

Pre-negotiate protection from third-party claims based on assignments of duties. Beyond protections on paper, substantiate claims the provider can back up with technical and institutional resources.

GenAI providers can require varying levels of access to data, from user inputs and prompts to direct access to company information and data stores. When using GenAI in sensitive areas like medical or legal services, special data obligations apply. Much like cloud computing, GenAI can be used appropriately and responsibly with respect to data in line with applicable ethical and industry standards, if proper attention is paid and documented as to the vendor credentials, obligations, and abilities.

Clearly denote who owns the service or tool and licensing capabilities. Clear, transparent, and mutually agreed-upon terms about IP rights not only protect proprietary interests but also delineate the boundaries of usage and modification rights associated with GenAI. Consider inputs, throughputs, and outputs, including inferences, as well as training and validation data, when it comes to assigning rights in advance.