TL;DR
The study shows that coding agents (Claude Code, Codex) routinely "play it too safe", even when the task already contains everything they need not to. They rerun tests that CI will run anyway. They back up things a script can restore. They add checks for cases that can't happen. The authors built 200 paired tasks that differ by one fact about the world and compared agent behavior with and without that fact.
The headline finding: a capable agent isn't the same as a calm agent. The share of tasks with unnecessary defensive work ranged from 11% to 59% depending on the configuration. A model that solves tasks better over-insures no less. It's especially bad when the instructions say "check/test" but don't say how much. Such an action has no natural stopping point, so the agent keeps going. The jump in violations when the measure was unspecified reached 69 percentage points. Developers rated those runs almost 1.3 points out of 5 lower.
At its core, the method is the risk classification from project management: avoid, transfer, mitigate, accept. Each type has its own kind of fact you need to tell the agent: "this risk doesn't exist," "someone else handles this," "here's the limit," "the decision is final." The authors wrote the facts as statements about the state of the world, not as prohibitions ("don't add checks"). The agent has to draw the conclusion itself.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
This isn't a prompting technique; it's a checklist of facts. Each item removes its own kind of unnecessary work.
AVOIDANCE → fact: "this failure is impossible"
example: the data can be restored, calls run sequentially,
dependency versions are pinned, types rule out invalid input
→ the agent doesn't write guards, fallbacks or checks against it
TRANSFER → fact: "another mechanism or person already owns this"
example: CI runs the tests, a bot updates dependencies, a reviewer reads the diff
→ the agent doesn't duplicate checks
MITIGATION → fact: "the measure is capped at a number/level"
example: how many runs, what test scope, how long the report is
→ the agent doesn't go past the limit
ACCEPTANCE → fact: "the decision is made and not up for debate"
example: "this is the final version," "the deletion is final"
→ the agent doesn't make backups or ask again
Everything goes into a single message to the agent or into the instructions file.
Example in Practice
Task: You own a small online store built in Python. You ask Claude Code to fix a bug: the SALE20 promo code gets applied twice in the cart. Without context, the agent will run the full test suite three times, copy the promo code table "just in case," add duplicate checks in five places and write a two-page report.
Prompt:
Fix the bug: the SALE20 promo code is applied twice in the cart.
The function is apply_discount in cart/discount.py.
Context:
- GitLab CI runs the full test suite on every push,
and Mike (tech lead) reviews changes before merge.
- The promo_codes table is a cache: it's rebuilt every night
by the rebuild_promos.py script from a QuickBooks export. QuickBooks is the source of truth.
- Cart requests are processed through a single queue;
there are no concurrent writes to the cart.
- Dependency versions are pinned in poetry.lock.
- One targeted test for the changed function is enough for verification.
- Report: five lines max.
Result: The agent should limit itself to fixing the function and one targeted test. It won't back up the table, recheck what CI covers or add race-condition guards. The report will be short. This is the expected behavior, not a guarantee from the paper: even with explicit facts, unnecessary work still showed up in a noticeable share of runs.
Why It Works
The weakness. The agent doesn't know your environment. It sees the code, but it doesn't see that CI will run the tests anyway, that the table can be restored, or that the decision has already been made. To the agent, caution is safe and an extra check costs nothing. So it covers itself "just in case." It over-insures most when the instructions say "check" with no limit. That's a vague request with no stopping point.
The strength. The agent is good at drawing conclusions from facts. If it says "CI runs the tests," it can figure out there's no need to duplicate the run. The authors deliberately phrased the facts as statements to test exactly this inference, not obedience to prohibitions.
How the method uses this. You close four typical gaps up front: the risk is impossible, someone else is responsible, the measure has a limit, and the decision is final. The agent has nothing left to "fill in" on the side of caution.
Knobs to turn: - Numbers instead of verbs. "Check" → "one targeted test," "report up to five lines," "no more than two runs." This is the strongest lever: the unspecified measure is exactly what produced the biggest jump in violations. - Who owns it. Be specific: "GitLab CI," "Mike on review." A concrete owner is more convincing than "this gets checked somewhere else." - Finality of decisions. Phrase them as facts: "this is the final version," "the deletion is final." - Permanent facts go in the instructions file (CLAUDE.md, AGENTS.md): about CI, review, restorable data. Everything else goes in the task.
Prompt Template
{task: what to do, where}
Context (facts about the environment):
- Tests/checks: {who runs them and when, e.g. "GitLab CI on every push"}.
- Review: {who reviews the changes and when}.
- Recoverability: {what can be rebuilt automatically, and by what}.
- Concurrency: {are there simultaneous writes/runs, or does everything go strictly in sequence}.
- Dependencies and environment: {what is pinned and doesn't change}.
- Verification scope: {how many runs/what level of testing is enough}.
- Report format: {length, what to include}.
- Decisions already made: {what is final, e.g. "the schema version is final"}.
Write only the lines that are actually true for your project. A false fact ("CI covers everything" when it doesn't) removes protection where you need it. Delete lines you don't need.
🚀 Quick start: paste this into the chat:
Here is a template for a "Context" block in a task for a coding agent. Adapt it to my project: [describe the project and stack].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask who runs the tests, who does review, what in the project is restored automatically, and which decisions are final. These are exactly the facts the method is built on: each one closes off one type of unnecessary work. From your answers it will assemble a ready-made block for the task or for CLAUDE.md.
Limitations
⚠️ The fact has to be true: The method rests on reliable statements. If you write "CI checks everything" and CI doesn't, the agent will obediently stop covering itself and miss a bug.
⚠️ Unnecessary work doesn't disappear entirely: Even with explicit facts in the task, unnecessary defensive work showed up in a noticeable share of runs. Facts reduce it but don't bring it to zero.
⚠️ Prohibitions weren't tested: The authors phrased facts as statements, not as prohibitions like "don't add checks." Whether a direct prohibition works better or worse isn't clear from the material.
⚠️ Only code in two languages: Tasks in Python and Go, in repositories where agent instruction files were deliberately removed. How facts in CLAUDE.md interact with other rules wasn't measured. Transfer to other domains and languages is a guess.
⚠️ Incomplete text: I had access to the introduction, the design and the abstract. The section breaking results down by model and configuration is cut off, so there's no "which model is better" comparison here.
⚠️ Caution isn't always unnecessary: In the variants without the facts, the agent is right to play it safe. The method is about not over-insuring when the facts are present, not about switching caution off.
How They Studied It
The idea was simple: separate "the agent is cautious for a reason" from "the agent is cautious out of habit." To do that, they made 200 pairs of identical tasks. In each pair exactly one fact changes. In one version it's there ("this is the final version," "CI runs the tests"), in the other it isn't. Everything else, including the repository, the goal and the checking test, is the same. If the behavior differs, the cause is that fact.
The starting situations weren't made up. The authors studied 18,922 real sessions of developers working with coding agents and found 5,574 episodes where the agent over-insured beyond what was needed. From these they assembled 44 typical situations grouped into four risk types. The tasks were built on 50 open-source repositories (34 in Python, 16 in Go), with instruction files removed. That way the agent got no hints beyond the task itself. The repositories had recent commits, so the models couldn't remember them from training.
They tested 8 models in Claude Code and Codex (9,600 runs). Unnecessary work was graded by a judge agent. It was calibrated against labels from two experts and reached 0.965 accuracy with agreement with humans of κ = 0.93. Separately, 20 developers rated 800 runs: unnecessary defensive work lowered their satisfaction by 1.27 points out of 5.
The surprise was that the ability to solve tasks has almost no effect on the tendency toward unnecessary work. A smart agent isn't necessarily a calm one. The practical takeaway: don't rely on "the model will figure it out"; describe the environment explicitly, especially the verification scope.
Adaptations and Extrapolations
This section is my own ideas; the paper didn't test them.
🔧 Technique: permanent facts → into the instructions file, one-off facts → into the task
Facts about CI, review and restorable data rarely change. You can write them once in CLAUDE.md or AGENTS.md:
## What the environment already guarantees
- GitLab CI runs the tests on every push; don't duplicate the full run locally.
- Changes go through tech lead review; keep the verification report short.
- The /build and /cache folders are rebuilt automatically; no backups needed.
- If the task doesn't specify the verification scope, run only the targeted tests
for the changed files and report what you checked.
The last item is my addition. It addresses the main finding: verification with no specified measure pushes the agent into excess.
🔧 Technique: audit your own instructions file
Go through your instructions and find words like "check," "test," "make sure," "be careful" with no number or limit. By the paper's logic, those are the spots where the agent will inflate the work.
Here is my instructions file for the agent: [paste]. Find wording that
requires verification or caution but doesn't specify a measure (how many,
what scope, up to what limit). For each one, propose a version
with a concrete limit and a question about which fact I need to clarify.
Resources
- Paper: ParanoiaEval: Benchmarking Unnecessary Defensive Work in Agentic Coding (preprint)
- Authors: Hanjun Luo, Xiucheng Zhang, Zhuoning Xu, Zhimu Huang, Yingbin Jin, Xinfeng Li, Hanan Salam
- Universities: New York University, NYU Abu Dhabi, The Hong Kong Polytechnic University
- Code and data: https://github.com/ZhuoningXu/ParanoiaEval_release
- Theoretical basis: the Avoidance–Transfer–Mitigation–Acceptance (ATMA) framework from NIST risk management (IR 8286 Rev. 1)