Incident sheet · /enterprise-ai
- System
- Enterprise AI
- Filed
- Read
- 6 min · 1,117 words
- Severity
- Routine
Two of Three Agentic AI Metrics Failed This CIO's Own Test
A 25-agent, 18-month deployment at a travel group failed two of its own three headline metrics under a new CLEAR standard. The culprit: unlogged human overrides.
Event log
- An 18-month deployment ran 25 production agents at a large travel and cruise group; two of three headline results failed the author's CLEAR audit.
- Tier 1 escalations fell 34% over a locked six-month window; onboarding fell from 23 days to 3 but the baseline included a standing backlog; booking conversion rose 24% alongside a marketplace rebuild and dynamic pricing.
- The CLEAR standard (Concurrent changes, Locked window, Evidentiary custody, Attributable baseline, Reference volume) sits beneath NIST AI RMF and ISO/IEC 42001.
- Only 7 of 25 agents could act without confirmation; 13 required it or were barred from autonomous external communication.
- Managers could stop any agent within 90 seconds; unlogged overrides silently removed hard cases from every downstream metric.
An 18-month deployment of 25 production agents at a large travel and cruise group produced three headline results — and two of them failed a rigorous audit of their own evidence. The author of that audit, the CIO who ran the deployment, has now published the standard he used, arguing that most published agentic AI results cannot be checked by the people funding them.
Read 50 accounts of agentic AI in production, he writes, and a pattern emerges: escalations down, handling times down, conversion up, onboarding compressed from weeks to days. Ask how those numbers were produced and the field thins. The measurement window is rarely stated, the denominator almost never, and concurrent initiatives that could have produced the same movement go unnamed.
"Nobody has told practitioners what a defensible result looks like, so careful and careless accounts read identically," he notes. The cost lands on CIOs weighing major agentic investments: a result achieved at peak trading volume is indistinguishable, in a percentage, from one achieved in a quiet quarter with the window picked afterward.
How deployments corrupt their own measurement
The analysis names five recurring failure modes:
- The denominator moves. An agent reduces escalations; so does a quiet quarter. A rate without its volume says nothing about which happened.
- The window is chosen after the fact. A steadily improving agent always has a recent window in which it looks better. The tell: three outcomes reported over three different periods.
- The baseline contains slack. Automating a manual process usually clears its backlog at the same time, mixing a faster process with a one-off vanished queue.
- The change is never alone. Organizations deploying agents are usually already mid-rebuild, mid-repricing or mid-reorganization.
- The definition is proprietary. A "Tier 1 escalation" means whatever your queue classification says, and is not comparable with a competitor's figure.
What does the CLEAR test change?
Existing frameworks — the NIST AI Risk Management Framework and ISO/IEC 42001 — tell you to measure, not what a reportable number looks like. The proposed CLEAR standard sits beneath them and needs only ordinary operational records plus one discipline: decide before you know the result.
- C — Concurrent changes registered. What else changed in the same window?
- L — Locked window. Was the period fixed before the result was known, and does it include the hardest trading conditions?
- E — Evidentiary custody. Which system produced the count, who defines the category, who reviewed it?
- A — Attributable baseline. How much of the gain came from re-engineering, how much from clearing slack?
- R — Reference volume. Did the underlying volume hold steady, and is that stated?
Four tests cost only disclosure. The locked window costs pre-commitment, and in the author's experience it does most of the work. Medicine and machine learning research already have reporting standards — CONSORT-AI and REFORMS — while agentic deployments have none. CLEAR is not a validity standard either: a result that passes all five remains a single-organization observation.
The verdict on three real numbers
Applied to the deployment's own results, CLEAR produced an uncomfortable scorecard. Tier 1 support escalations fell 34% across a six-month window fixed to include peak season, with flat contact volume and counts reviewed by finance rather than the agent team — this "largely survives," though the proprietary Tier 1 classification blocks cross-organization comparison.
Operator onboarding falling from 23 days to 3 does not survive. The baseline included time in a standing backlog; part of the gain came from eliminating the queue, not shortening the process.
Booking conversion up 24% is the weakest. A marketplace rebuild and opt-in dynamic pricing ran in the same window as the AI-native search credited with the uplift. Reported without that caveat, it would be the most impressive number and the least informative.
Two of three carrying material qualifications is probably typical, the author concludes. A result that survives disclosure of its weaknesses is worth more than one never tested.
The failure that makes everything else moot
The most important control in the deployment cost nothing to build: any line-of-business manager, from the wholesale desk to the cruise operations floor, could stop an agent within ninety seconds, with no ticket and no approval. "Being wrong cost an hour of throughput; being slow could cost a customer."
A manager using that control is the system working as designed. The failure is the override that is not logged. The agent's statistics are then computed over a population from which a conscientious human has removed the hard cases. Escalation rates fall because escalations were intercepted first. Confidence scores look strong because low-confidence decisions were pulled out by hand. The results become unfalsifiable, because the basis was corrupted before anyone thought to write it down.
The analysis echoes Lisanne Bainbridge's 1983 "ironies of automation": the human handles what the designer could not automate, with no way to feed that judgment back into the design. No care at write-up time recovers a record that was never captured.
Six operating habits, one design rule
The deployment treated agents as high-volume junior team members — fast, tireless and capable of significant harm if poorly supervised. Six habits followed:
- A five-minute morning review of overnight agent activity, treated as a floor walk.
- A weekly scorecard written for operations: completion, escalation rate, decision confidence.
- An exception-first culture asking what the agent flagged as uncertain.
- Explicit human-agent teaming norms: when a human overrides, when they defer, how it is logged.
- Scenario playbooks for the ten likeliest failure modes, owned by operations.
- A monthly recalibration session reviewing tolerance limits against incident data.
One design decision supports all six: autonomy was allocated by reversibility, not risk appetite. The less reversible the action, the earlier a human must stand in the loop. Of the 25 agents, seven could act without confirmation; thirteen required it or were barred from autonomous external communication.
The standard has limits. Weekly agent changes leave no stable intervention to lock a window around, so rolling comparisons should be reported and labeled as such. Disclosure has commercial cost — volumes were withheld, the comparison basis disclosed. Disciplined reporters will look weaker than undisciplined ones, a coordination problem no single practitioner can solve.
The Monday checklist is short. Check whether human overrides are logged, and inspect the log rather than ask whether one exists. Lock the measurement window and write down the period, the denominator and everything else changing in the same interval. That costs nothing in advance and is impossible afterward.
"The organizations that scale agentic AI won't have the best models; capability is converging and procurable," the author writes. "The technology was the easy part. The measurement was the work."
via cio.com/feed/ (Source)
More from Tom Whitfield
Show full bio
Market editor covering people, launches and day-to-day practice in enterprise information technology and the leaders who run at CIO Monitor.
185 articles
The newsroom digest
One email a day with the stories that matter. No noise.
Cross-references
- William Blair Cuts Through the Noise on Enterprise AI Agent Adoption
- MIT Report: 95% of Enterprise Generative AI Pilots Are Failing
- Two-thirds of enterprises now run AI agents: reading the 68% figure
- 69% of Apple IT Teams Can't Roll Back AI Errors Within an Hour
- CIOs shift from AI pilots to fewer, bigger 2027 bets