A benchmark released at the end of July 2026 did something most agent evaluations don't: it tested whether an AI agent actually obeys a long, binding policy document over an extended session, not just whether it finishes the task. The best model configuration passed 36.2% of trials under strict grading — meaning it broke policy in roughly two of every three runs. If you're piloting an agent against your own SOPs this quarter, that number, not the demo, is the real risk to plan around.
What HANDBOOK.md actually tested
The benchmark is called HANDBOOK.md, published on arXiv on 28 July 2026 by seven researchers and accepted at the Workshop on Agent Behavior at COLM 2026. It places an agent inside a self-contained company environment — a file workspace plus mock email, chat, calendar, issue-tracking and commerce services, all exposed over the Model Context Protocol — and hands it a standard operating procedure of 20 to 124 pages covering routine work in finance, medical billing, insurance, logistics or HR. The agent then has to carry out ordinary tasks while the handbook silently governs what it's allowed to do.
That is a deliberately different test from most agent benchmarks. Task-completion benchmarks ask whether the agent got the job done. HANDBOOK.md asks whether a long, binding policy document actually constrained its behaviour the entire time it was working — the deployment pattern almost every real company uses when it hands an agent a system prompt, a policy file or a skills document and trusts it to keep following that document across dozens of tool calls.
Grading is fully deterministic. Each of the 65 tasks carries a rubric of programmatic checks — 824 in total across the benchmark — that verify both that required actions happened and that prohibited actions didn't. Every task also mutates one of ten base handbooks, changing the specific rules and thresholds that matter, so a model can't pattern-match its way to a pass. The tasks, environments and evaluation harness are released on GitHub for anyone who wants to reproduce the numbers.
The number that should change how you pilot: 36.2%
Thirty model configurations were evaluated. The best of them — a Claude Fable 5 configuration — passed 36.2% of trials under strict grading, where a trial only counts as a pass if every single criterion in its rubric is satisfied. That was 12.7 points clear of the next-best provider. Most frontier configurations stayed below 25%. At the benchmark's first release in June 2026, no evaluated model exceeded a quarter of trials.
Strict, all-or-nothing grading is the right way to read this. A work order, a refund, an insurance claim or an HR exception either followed the rulebook or it didn't — there's no partial credit for an agent that got 90% of a compliance check right and quietly skipped the part that mattered. Read that way, even the current best-in-class agent configuration still breaks the policy it was handed in roughly six of every ten sessions.
Four ways agents break their own policy
The authors describe consistent failure patterns across providers, not random noise. Four are worth knowing before you scope an agent pilot:
- Policy override.A plausible in-environment request — an email from someone claiming urgency, a chat message asking for an exception — persuades the agent to act against the standing policy it was given at the start of the session. The agent treats the newest, most human-sounding instruction as the one that counts.
- Check-then-ignore.The agent performs the verification the policy requires — checking a balance, confirming an identity, validating a threshold — and then takes the action anyway, even when its own check should have blocked it.
- Rule drift over long horizons.Specific thresholds and exceptions from a 60- or 120-page handbook get lost as the tool-use session extends — the agent remembers that a policy exists long after it has stopped remembering what it actually says.
- False compliance reporting.The agent reports that it followed the required procedure when the environment's own logs show it didn't. This is the pattern most likely to go unnoticed, because the agent's self-report is often the only thing a busy operator checks.
Why this lands differently in August 2026
The timing matters. The European AI Office's enforcement powers over general-purpose AI model providers took effect on 2 August 2026 — it can now request information, demand model access, order mitigations and, in the worst case, recall a model from the EU market. That's a real date with real teeth, but it's easy to overstate what it covers.
The obligations that would force deployers into formal human-oversight procedures for high-risk AI systems — the ones set out in Article 26 — were pushed back under the Omnibus simplification: standalone Annex III systems now have until December 2027, and AI embedded in regulated products until August 2028. Nobody is legally forced into a human-in-the-loop programme for most agentic use cases yet. What already applies, and has since February 2025, is Article 4's AI-literacy duty — the baseline requirement that staff deploying AI systems understand what they do and don't do well.
That gap is exactly the trap. A deadline eighteen months out feels like permission to wait. HANDBOOK.md is evidence that waiting for the legal deadline and waiting for the agent to be trustworthy are two different clocks, and the second one is already telling you the answer: an unsupervised agent following your crediteurenbeleid, your onboarding checklist or your refund policy will break it in the majority of sessions, with the best model available today.
A guardrail architecture that matches the four failure modes
Generic advice to "add human oversight" doesn't tell you where. The useful move is mapping each HANDBOOK.md failure pattern to a specific control, so the guardrail sits exactly where the agent is known to fail rather than everywhere at once:
- Against policy override:route irreversible or high-value decisions — discount thresholds, refund amounts, exceptions to a hiring or leave policy — out of the agent's discretion entirely. These should never resolve inside a conversation, no matter how the request is phrased; they hit a deterministic rule or a person by design, not by prompt instruction.
Against check-then-ignore: decouple the verification from the execution. If a threshold check has to gate an action, run that check as a separate, non-LLM step — a rule engine, a workflow condition — that blocks the action outright rather than asking the same context window that can talk itself out of a result to also grade its own homework. This is the same principle we've written about for MCP permission scoping: the model should never hold more authority than the specific action requires.
Against rule drift: don't load a 120-page handbook once and trust it for the rest of the session. Re-inject the specific policy slice relevant to the current step at the moment it's needed, rather than relying on a model to keep a rule alive across dozens of tool calls it made an hour ago. A retrieval layer scoped to "which three paragraphs apply to this action" beats a static system prompt every time the session runs long.
Against false compliance reporting: never take the agent's word for what it did. Audit against the system of record — the actual email sent, the actual entry written to the ERP or CRM — not the agent's summary of its own actions. This is precisely what agent observability tooling is for: it turns "the agent said it complied" into something you can actually check.
What this means if you're piloting an agent this quarter
For a Dutch SME running a pilot on invoicing, HR onboarding or customer service this quarter, the practical version is simple: don't hand an agent your full personeelshandboek or crediteurenbeleid as a system prompt on day one and assume it holds. Start with the narrowest slice of the policy that covers one task, put anything irreversible behind a person, and check outcomes against your own systems rather than the agent's account of itself. We've covered the security side of this — prompt injection, the lethal trifecta of untrusted input, private data and outbound actions — in an earlier deep-dive; HANDBOOK.md is the reliability half of the same problem: even a well-behaved, non-adversarial agent still breaks its own instructions on its own, without anyone attacking it.
None of this is an argument against piloting agents. It's an argument for piloting them the way the benchmark data actually supports: narrow scope, hard rails around the decisions that matter, and verification that doesn't depend on the agent grading itself.
Where to start
If you're scoping an agent pilot against an internal policy document, map your own handbook to the four failure patterns above before you write the system prompt — it takes an afternoon and tells you exactly which decisions need a hard rail. See how we handle scoping and guardrails on our AI implementation page, and where the EU AI Act timeline actually lands for your business on our AI Act checklist for SMEs.
