Every few years, the technology world takes a familiar idea, rebrands it and pushes it back into the spotlight. Agentic AI is a perfect example. As adoption accelerates, organisations are paying greater attention to the risks of agentic AI, often without fully understanding the underlying architectures or whether the models are appropriate for the job. Predictive AI belongs to one family of problem-solvers; generative AI to another. Treating them as interchangeable creates systems that behave unpredictably.
This isn’t an argument against agentic AI. Used well, these systems can be incredibly effective. But the enterprise world, and especially regulated sectors, need a more grounded understanding of how these architectures work and where the real risks lie. Because when organisations misjudge the ingredients, the end result becomes much harder to trust, monitor or defend, as Kit Ruparel, Chief Technology Officer, TCC Group – Recordsure, explains…
Agents are not new, but the stakes are higher now
Software agents have been around for decades, carrying out predefined tasks through distributed systems and microservice architectures. They weren’t truly intelligent—they simply followed rules defined by engineers.
Crucially, those agents communicated with one another through tightly defined interfaces. APIs were explicit. Inputs and outputs were structured. If something failed, it failed loudly. Engineers could trace it, interrogate it, and correct it.
Early examples had a familiar flavour. Microsoft’s Clippy attempted to interpret user intent (often badly), while consumer voice assistants later routed requests to different “skills” or agents based on what users asked. The concept isn’t new.
What is new, however, is the sheer scale of LLM (Large Language Model) adoption. What’s changed is how LLMs allow agentic components to communicate with one another. Instead of relying solely on rigid, formally defined APIs, agents can now pass instructions between each other in plain English. That flexibility is powerful, but it also introduces ambiguity. Natural language is expressive, not deterministic. Subtle differences in phrasing can therefore alter intent, and interpretation becomes probabilistic rather than exact.
Generative AI (Gen AI) is now one of the most widely deployed forms of AI in organisations. Research from McKinsey suggests adoption continues to accelerate: 23% of organisations are already scaling an agent-based AI system in at least one business function, while another 39% have begun experimenting with AI agents.
That acceleration combined with agents effectively ‘talking’ to one another in natural language is why the boundary between stable engineering and experimental behaviour has become harder to see, and why clarity now matters more than hype.
What are the risks of agentic AI?
Agentic AI refers to systems that use one or more AI models to plan, make decisions and complete tasks with limited human intervention. While this architecture can improve efficiency, it also introduces new risks around accuracy, governance, operational control, compliance and data privacy. As multiple AI agents communicate and make decisions on behalf of users, small errors or ambiguous instructions can compound, making outcomes harder to predict, audit and explain.
How the risks of agentic AI begin to emerge
The simplest way to understand the problem is through the example of a pub quiz night.
Imagine you’re the captain of a pub quiz team. Mo knows music, Sally knows football and Tabby knows electronics. A football question goes to Sally because she’s the expert. You’ve interpreted the question, decided who should answer and trusted their response. That’s exactly what an agentic system does: parse, reason, orchestrate and respond.
Next comes a question with two plausible answers. One teammate recalls a legal reference, another remembers a famous drum machine. You weigh both responses and choose the one you trust most. Agentic AI works in much the same way—except every decision is made by another AI model.
That’s how agent systems work: parse, reason, orchestrate, respond. Now imagine replacing each step with a large language model:
- Model A interprets the question
- Model B decides who should answer
- Model C writes the request
- Model D generates the answer
- Model E evaluates the response
Even if the model is 90% accurate, the combined reliability falls sharply. Five LLM-driven steps drop the final accuracy to around 59%. Extend to ten agents and statistically, two out of three final answers become wrong – yet still delivered with total confidence.
Like the childhood whisper game, each hand-off introduces more opportunity for distortion. Because LLMs don’t express confidence scores, incorrect answers can still sound authoritative.
This behaviour is already showing up in enterprise data. A recent survey of more than a thousand organisations found that the proportion abandoning most of their AI initiatives before production jumped from 17% to 42% in a single year. The gap between proof-of-concept performance and real-world reliability is widening, often because organisations underestimate the compounding-error risk inside agent chains.
Predictive AI models work differently because they return confidence scores. That allows organisations to set thresholds, manage uncertainty and build systems that can justify their decisions.
Managing the risks of agentic AI in practice
Agentic AI is becoming part of everyday enterprise software. The challenge isn’t adoption—it’s designing systems that remain reliable, explainable and governable.
Three principles can help reduce those risks:
1. Design communication deliberately
If two components are under your control, don’t default to natural language. Clear APIs reduce ambiguity and allow for the audit trails that regulated sectors require. Generative models should sit where they add genuine value, not where they create unnecessary interpretative layers.
2. Ensure every component can say “I don’t know”
Predictive models already offer probability. Use that. Generative models need structured guardrails that allow them to decline gracefully rather than fabricate their way forward. This single act prevents entire cascades of error.
3. Build governance and traceability into the core
Modern cloud ecosystems now provide identity boundaries, content safety controls, audit trails and monitoring tools designed to support governance, regulatory compliance and data privacy. Any AI system developed outside these controls should be held to the same standards to maintain operational control and reduce organisational risk.
Human oversight remains essential, particularly in regulated sectors such as financial services, where experienced judgement is often the final safeguard against unexpected behaviour.
Reducing the risks of agentic AI through responsible design
Agentic AI will reshape plenty of workflows, but not because it thinks like us. The value lies in the architecture wrapped around it: separating predictive and generative logic, limiting compounded error, controlling uncertainty, and designing systems that remain stable long after the novelty fades.
Organisations evaluating AI solutions should look beyond impressive demonstrations and assess how providers address governance, compliance, data privacy and operational control. Responsible AI frameworks should be able to admit uncertainty, maintain traceability and support regulatory expectations from the outset.
Agentic AI isn’t inherently risky because it’s powerful. It’s risky when adopted without the discipline it demands. In the end, it’s responsible AI design—not hype—that keeps these systems safe, reliable and worthy of trust.
Photo by Immo Wegmann on Unsplash
