LLM Availability: A Critical Component for System Reliability

Every organisation deploying critical digital applications understands the importance of cybersecurity. High-profile incidents continue to show the reputational, financial, and operational damage that can result when systems fail or are compromised. As Large Language Models (LLMs) and Agentic AI systems become embedded into business workflows, these risks are evolving. Organisations are no longer only asking whether an AI system is accurate, secure, or compliant; they must also ask whether it is reliably available when the business needs it.

Cybersecurity in AI

Cybersecurity is framed around three core principles: confidentiality, integrity, and availability. In the context of AI, confidentiality and integrity tend to receive the most attention. Topics such as prompt injection, data leakage, and excessive agent permissions are now widely discussed. However, availability remains an under-addressed risk. For businesses seeking to move AI from experimentation into production, this is a significant blind spot.

How does LLM availability affect AI?

Availability is typically measured through uptime. A system with 99.9% uptime may sound highly reliable, but this still equates to almost nine hours of downtime per year. At 97% uptime, annual downtime rises to nearly eleven days. For a proof of concept, this may be tolerable. For an AI-enabled claims process, customer support function, financial control, or operational workflow, it could create serious disruption.

The issue becomes more pronounced as AI systems become more agentic. Unlike a simple chatbot, an AI agent may rely on multiple sequential LLM calls to plan, reason, access tools, retrieve information, and execute a task. Each step introduces another point of potential failure. If five sequential model calls each operate at 97% reliability, the combined probability of success drops to approximately 86%. In practical terms, this means the overall system could fail far more often than any individual model metric might suggest.

This creates an important shift in how organisations should think about AI assurance. Traditional validation often focuses on whether the model produces the right answer, however for workflows in production that use AI, this alone is not enough. Organisations must validate whether the system can continue operating under real-world conditions. Including validating performance when the LLM model is updated, financial viability due to pricing model changes, provider outages, latency spikes, degraded model performance, and partial service failures.

Managing the risk of low AI availability

There are several practical ways to reduce availability risk.

Retries

Retries are often the first line of defence, allowing a failed request to be attempted again before the user experience is affected. If repeated attempts fail, there should be a backup to ensure business-critical systems continue to operate even at a limited capacity.

Fallback architectures

Organisations should also consider fallback architectures, where alternative model hosting regions, locally hosted models or other providers can be used if the primary model becomes unavailable. These fallback routes should be designed intentionally, considering cost, performance, geography, security requirements, and the specific role each model plays within the wider system.

Graceful degradation

More advanced approaches can further improve resilience. For example, an LLM classifier can route tasks based on intent or complexity, ensuring simpler requests are handled by lower-cost or backup models when needed. Frequently requested responses can be cached to maintain continuity during periods of disruption. In some cases, systems can also use graceful degradation, where a simpler but reliable process is triggered when the preferred AI-enabled workflow is unavailable.

Robust validation and CI/CD pipelines

Each model update can change how an LLM interprets tokens, reasons through prompts, and produces outputs. As a result, even a routine upgrade can alter the behaviour of an AI-enabled workflow. Model updates should therefore pass through a defined CI/CD pipeline, with regression testing to confirm that performance remains aligned with the business use case and that no material deviations are introduced.

Monitoring

Monitoring is equally important. Rather than waiting for model calls to fail, organisations can monitor provider health, latency, error rates, and response quality in real time. When thresholds are breached, traffic can be automatically rerouted to alternative mechanisms such as backup models or non-AI processes. This moves resilience from a reactive control to a proactive design principle.

Third-party AI providers and SLAs

Many organisations consume AI through third-party platforms and managed services, limiting their ability to directly implement resilience controls. In these cases, service-level agreements (SLAs) should define expectations for availability, incident management, monitoring, and change notifications. Providers should be required to communicate outages, model updates, and security incidents promptly, while contracts should include remediation or compensation where service commitments are not met. Resilience may be outsourced, but accountability should not be.

AI availability cannot be assumed – it must be engineered and validated

The transition from AI pilot programmes to business-critical workflows with AI at the centre mandates a change in mindset: availability is not a given, it must be deliberately designed and implemented into the whole process from the start. Resilience is as critical in responsible AI deployment as other more discussed areas of responsible AI, such as guardrails and bias. This risk is amplified in agentic workflows, where a single point of failure can cascade through complex workflows, disrupting reasoning, decision-making, and execution.

At Deloitte AI Labs, we have developed a validation framework for LLM-enabled applications that assesses not only model accuracy and security, but also operational resilience, fallback behaviour, outage handling, and production readiness. By testing these risks before deployment – and monitoring them continuously in production – organisations can build AI systems that are not only intelligent, but dependable.

To learn more about validating resilient AI systems, reach out to the contacts listed below.