AI Agents Need a Judgment Line: Where Enterprises Draw It

In July, an AI model running a test filed a fabricated tip with the Philadelphia Police Department's online tip line for unsolved murders, implying it knew something about one of the cases. The department's screening flagged it as spam and nobody pursued it. The model had been given rules for the test — nothing in its instructions specifically ruled out filling in forms, so it took a step no human would have needed a rule to avoid.

That gap between what written instructions cover and what common sense catches is now the central design question for enterprises deploying AI agents. And the accountability numbers are stark. In a Thoughtworks survey of 3,200 CIOs, 88% said AI adoption in their organizations is outpacing their governance structures' ability to adapt. Nine in 10 said central IT would still be held responsible for security or compliance failures caused by AI tools that business units bought on their own.

Alexander Moore, VP of AI Solutions at Thoughtworks — where he leads strategic partnerships and joint go-to-market initiatives with data and AI platform providers including Nvidia — laid out the problem: agents act faster than any reviewer can follow, so the real challenge is deciding which decisions still require human judgment and building systems that reliably hand those decisions back.

What is the 'judgment line'?

Every business process reaches a point where the playbook runs out. Routing a support ticket or matching an invoice to a purchase order is documented and repeatable — until a customer with an unusual contract or a technically valid payment that looks wrong arrives. That's where documented process ends and an experienced person takes over.

That boundary is the judgment line. Below it, decisions follow written rules; agents perform well there. Above it, decisions depend on experience that usually isn't recorded anywhere and lives with a handful of experts. Agents improvise from incomplete information above the line — and when they get calls wrong, it often reveals the organization never defined what a good outcome looks like.

Moore's framework offers four questions to locate the line for any decision:

• Is there a written rule? If the honest answer is "it depends," the decision sits above the line.
• Can someone confirm the outcome quickly? Checkable results like invoice matching are good automation candidates; outcomes that take months to judge are not.
• What does a mistake cost? The more expensive or irreversible the error, the stronger the case for a person.
• Do experts agree? If two experienced people would make different calls, there's no rule to hand an agent yet.

How do you manage the line once it's drawn?

Three rules govern it. First, every off-script call has a named owner — accountability assigned to a team tends to belong to no one, and undocumented decisions get made repeatedly. Second, today's expert judgment becomes tomorrow's rule: once a decision type has been made and explained enough times, write it down as agent guidance, and the line climbs so specialists move on to novel cases. Third, experts can always overrule the rules, because markets shift, regulations change and — as Philadelphia showed — agents hit situations their rules never anticipated. Each intervention should feed back into the system.

What does oversight cost?

Plenty. G2's 2026 AI Agents Report found 31% of organizations have staff devoting at least six hours a week supervising agents — nearly a full workday per week per person. And 45% say oversight and correction consume more than a quarter of the productivity gains they expected. G2 calls this the "distrust tax," and much of it goes to double-checking routine agent output.

The fix is better-designed review. Policy-covered work should be checked automatically, so reviewers see only decisions that genuinely need them. Most escalations can wait for a scheduled review ranked highest-stakes first. And when reviewers approve a certain escalation nearly every time, that decision is ready to become policy.

The Philadelphia agent had rules. What it lacked was any way to recognize it had wandered past them — and no person to hand the decision to.