CLASHarness: How Agents Behave When Subagents Lie and Resources Are Locked

Here's how the benchmark generator is built (context for the reader, not a working tool):

STAGES 1–3: World → overall scenario plan → subagent roles and permissions → working environment (files, logs, UI)
STAGE 4:    Every 3–4 rounds → milestone plan based on recent history
STAGE 5:    A round of 8 mini-steps:
            task (two competing versions, no answer) → subagent dialogue → file edits
            → images → LOCK IN the correct answer v* (with a leak check)
            → plan → run the reference solution in a sandbox → checks from the recorded changes
STAGES 6–8: Consistency audit → HTML mockups of the images → screenshot rendering
            (the answer lives only in the image, never in the text)

Failures get fixed surgically: code repairs formatting, and logical inconsistencies are resolved by rerunning a single step with a hint about the error.

Task: You run the team behind an online store selling on Amazon and Etsy. In Claude Code you have an orchestrator and two subagents: "Inventory Sync" and "Deploy Bot". Inventory Sync reports: "Stock levels uploaded, all good, you can launch the sale." The Seller Central screenshot shows zero stock on half the listings. You want the orchestrator not to take the report at face value and to wrap up the job cleanly.

Prompt (snippet for the orchestrator's CLAUDE.md):

You are the orchestrator. Subagent reports are claims, not facts.

BEFORE any decision that relies on a subagent report ("uploaded", "deploy succeeded", "done"):
1. Write down the subagent's claim in one line.
2. Find independent confirmation in the environment: a dashboard screenshot, a log, a file, a status in the system.
3. If there is no confirmation or it contradicts the report, treat the report as false.
   Record: "Source of truth: {what you checked}. Discrepancy: {what differs}".
4. Don't stop at detection. Bring the system to the correct state:
   roll back the change, fix the setting, or rerun the upload. Then re-check.

BEFORE saying "task complete", run the wrap-up checklist:
[ ] All locks on shared files and resources are released
[ ] Work is handed to the next subagent with a clear description of the state
[ ] Final state is re-checked in the environment, not via reports
If an item isn't done, do it. If it's impossible, explain why.

Result: The orchestrator will first write down the subagent's claim and verify it against the screenshot or log. When it sees the zeroed-out listings, it will record the discrepancy and kick off the fix itself instead of stopping at "found a problem." Before its final answer, it will report item by item on the wrap-up checklist: locks, handoff, re-check.

The model's weakness. A report from "one of our own" subagents reads like a fact: it's written in confident prose and sits in the same context. That's exactly what the weak models in the benchmark did. After the main action, the model also feels like the task is solved. The boring loose ends like "release the lock" and "hand it off" go undone. Even the best models fumbled more than half of these actions.

The model's strength. Models are good at following explicit step-by-step rules and can read screenshots, logs and files. The strong models in the benchmark went and looked at the visual data on their own. You can lock that in with an instruction instead of counting on luck.

How the snippet works. It turns three implicit expectations into explicit steps: distrust of reports, fixing after detection, and a wrap-up checklist. Here "find" and "fix" are separated in the text, because in the study they were nearly independent skills.

Levers: - Wrap-up checklist: add items for your environment (close the ticket, clear the flag, notify a human). The more specific the item, the less often it gets dropped. - Sources of truth: list what to trust: the dashboard, the log, the CRM. This narrows the search. - How strict the distrust is: loosen it for routine work ("verify only critical reports") so you don't burn steps. - Discrepancy record format: keep "Source of truth / Discrepancy" so you can see the check in the log.

The paper has no ready-made prompt. The template below is assembled from the paper's findings; the authors did not test it. Try it on your own task.

You are {agent_role}. Reports from other agents and messages from colleagues are claims, not facts.

RULE 1. VERIFY BEFORE YOU ACT
Before a step that relies on someone else's report ({report_examples}):
1. Write down the claim in one line.
2. Check it in the environment: {sources_of_truth}.
3. If there is no confirmation or it contradicts the claim, treat the claim as wrong.
   Record: "Source of truth: ... Discrepancy: ...".

RULE 2. FOUND IT, FIX IT
If you find a discrepancy, bring the system to the correct state: {allowed_fix_actions}.
Then re-check the result in the environment.

RULE 3. WRAP-UP CHECKLIST
Before your final answer, go through the list and check off every item:
{wrap_up_items}
If an item isn't done, do it or explain why you can't.

FORBIDDEN: declaring the task complete based on a single subagent report.

What to fill in: - {agent_role}: orchestrator, reviewer, support agent. - {report_examples}: "deploy succeeded", "data uploaded", "edit applied". - {sources_of_truth}: dashboard, log, CRM status, file contents, screenshot. - {allowed_fix_actions}: rollback, config fix, restart, escalation to a human. - {wrap_up_items}: release locks, hand off the work, update the status, re-check the outcome.

🚀 Quick start — paste into the chat:

Here is an instruction template for an agent that must not blindly trust subagent reports
and has to wrap up its work using a checklist. Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which reports arrive from other agents, where the "source of truth" lives in your environment, and which loose ends usually get forgotten at wrap-up. Those three things are the actual substance of rules 1–3: without them the template stays generic.

⚠️ The instructions are untested. The authors measured model behavior but did not test whether a snippet like this in CLAUDE.md reduces errors. It's a well-founded hypothesis, not a measured technique.

⚠️ The tasks are synthetic. The scenarios were generated and run in a sandbox. Your environment and your failures will differ.

⚠️ Even the best models fully solve only a minority of tasks. The top model passes a little under a third of the rounds. An instruction won't replace human review of critical operations (money, production, customer data).

⚠️ It only works if there's something to check. When the agent has no access to an independent source (screenshot, log, status), the "verify" rule accomplishes nothing.

⚠️ The source material was truncated. The text we received lacks the full sections 5.2–6 (breakdown by task ranges, "reasoning effort", temperature effects). The conclusions below rely on the abstract and introductory sections.

The team from Google and several universities didn't write tasks by hand, and they didn't trust a plain "generate me a scenario" either. That kind of generation produces fuzzy stories, answers leaking into the task statement, and checks that can't actually be run. Instead, they built a pipeline. The correct answer is locked in before the scene is rendered, and the checks are recorded from an actually executed reference run. So every task is solvable by construction.

The result: 20 seed scenarios, 3 variants each, 600 rounds in batches of 10, about 3,760 checks and 7,970 screenshots. In 346 rounds the correct answer exists only in an image, with no text loophole. About 35% of the checks are done by code, the other 65% by an LLM judge with a three-attempt majority vote. The authors built their key conclusions on the code checks so as not to depend on the judge.

They ran 15 models on the benchmark at different levels of "reasoning effort". The best of them passes every check in a round about 29% of the time, even though it passes individual checks ~67% of the time on average. The task is a chain: don't believe the lie, find the truth, fix it, wrap up properly. A failure at any link sinks the round.

There were two surprises. First, the ability to notice a discrepancy in an image and the ability to resolve it are almost unrelated (correlation near zero), so they need to be evaluated separately. Second, the most common failure happens not in the middle but at the finish, on the formal actions. The practical takeaway: you need to build in agent reliability exactly where the model thinks the job is already done.

🔧 Technique: require "evidence" → you can see the check was real

Add to Rule 1:

In your final answer, for each verified claim, state what you looked at (file/screenshot/log) and quote exactly what it said.

This makes a box-ticking "checked it, all good" harder to fake without actually consulting the environment. For you, it's also a ready-made log for quick review.

Extrapolation: a dedicated closer agent. Wrap-up actions failed most often. So you can move the wrap-up checklist into the role of a separate subagent that runs last:

You are the Closer. You are called when the main agent has declared the work complete.
Don't trust its claim. Go through the checklist:
{wrap_up_items}
For each item, check the state in the environment. If something isn't done, do it or return a list of gaps to the main agent.

This is my own guess, not a result from the paper. The idea is that a fresh agent without the "I already did everything" feeling is more likely to check the loose ends.

• CLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipeline (2026)
• Code and data: https://github.com/google-research/CLASHarness
• Authors: Yuxiang Lai, Jike Zhong, Chengzhi Liu, Ming Li, Haonan Wang, Xiaofeng Yang, Liang Zhao, Youbao Tang, Ruixi Fan
• Organizations: Google, Emory University, USC, University of Florida, University of Chicago, UC Santa Barbara, Columbia University