Apply to my task
arXiv:2608.18167 ★ 90 Aug 16, 2026

Adversarial Review: typed disagreement between reviewer and critic vs. false LLM consensus

Key result

Two AI agents are supposed to argue with each other, but in practice they just agree — even when one of them is right. In one test, the critic raised a real problem in the code, the reviewer brushed it off with a confident empty phrase without a single reference to the code — and the critic backed down. The Adversarial Review method forces the critic to choose one of three options: agree, cite evidence of disagreement, or raise a question without a citation — and then the reviewer must either confirm the concern with its own citation or drop it. This gives you a real, evidence-based check of code, text, or a contract instead of a polite "looks fine."

01

TL;DR

Adversarial Review (AR) is a three-role protocol: a main agent produces code or text, a reviewer checks it, and a critic checks the review itself before the main agent touches anything. The key mechanic: the critic and reviewer argue with each other in a separate "internal" loop — and only once their positions settle do the edits land in the actual document.

The paper's main finding: when you put two roles in an LLM dialogue and task them with reaching one shared conclusion, they quickly converge on agreement, even when it's wrong. In one test, the critic raised a real problem in the code, but the reviewer brushed it off with a confident-sounding but empty statement with no reference to the actual code — and the critic just gave up. On paper, this looked like "structured disagreement," but in reality it was a false consensus.

The method fixes this not by adding more agents, but with a rule: the critic must pick one of three response types — agree, point to a specific quote as evidence of disagreement, or raise a concern without evidence (in which case the reviewer must either back up the point with a quote or drop it). A simple text rule — "argue with evidence or stay quiet" — and finding accuracy jumps noticeably.

Method Blueprint

STEP 1: Main agent (M) produces an artifact (code/text/document) → version N
STEP 2: Reviewer (R) checks the artifact → list of comments with quotes
STEP 3: Critic (C) evaluates each comment from R and picks one:
        - AGREE
        - DISAGREE_EVIDENCE: <quote> (disagree, here's the evidence)
        - DISAGREE_CONCERN: <question> (disagree, but no direct quote)
STEP 4: R responds per the rule:
        - on AGREE → leaves the comment as is
        - on DISAGREE_EVIDENCE → revises the comment
        - on DISAGREE_CONCERN → either backs it up with a quote or drops it
STEP 5: Steps 2-4 repeat (up to 5 rounds) until the review stabilizes
STEP 6: M edits the artifact per the final review → version N+1, cycle repeats

You can run all of this in a single chat with one prompt — the model plays all three roles in turn.

Example Application

Task: Before signing, a freelancer received a website-development contract and wants to check it for hidden unfavorable clauses — not just get a polite "looks fine."

Prompt:

You will take turns playing two roles: Reviewer (R) and Critic (C).
The artifact to review is the contract below.

[insert contract text]

STEP 1 — R: find clauses in the contract that could be unfavorable to the freelancer
(payment deadlines, penalties, rights to the work product, force majeure).
For each clause, give an exact quote from the contract.

STEP 2 — C: evaluate each of R's comments. For each, pick one:
- AGREE — you agree with the comment
- DISAGREE_EVIDENCE: <quote> — you disagree, here's the contract clause
  that contradicts the comment
- DISAGREE_CONCERN: <question> — you disagree, but without a direct quote

STEP 3 — R: respond to C's reaction.
On AGREE — keep the comment.
On DISAGREE_EVIDENCE — revise the comment in light of the quote.
On DISAGREE_CONCERN — back up the comment with a quote from the contract
or drop it.

Repeat steps 1-3 until the list of comments stabilizes (max 3 rounds).
At the end, output the final list of real contract risks with quotes.

Result: The model will show 1-3 rounds of exchange: the reviewer's initial comments, the critic's reactions with or without quotes, and final revisions. At the end — a list of risks, each backed by an exact quote from the contract, rather than vague phrases like "this clause seems off."

Why It Works

An LLM playing two "collaborating" roles tends to agree rather than search for the truth — much like human politeness in teamwork, minus any real stakes for being wrong. If you simply ask the model to "critique your own review," it often just rubber-stamps what's already written — because the implicit goal of the dialogue is perceived as "reach agreement," not "find the truth."

On the other hand, LLMs are good at following explicit text formats with mandatory fields. If you say "pick one of three options, and for two of them quote a specific spot," the model follows this almost literally, because it's a concrete structure rather than an abstract request to "be more rigorous."

The method leverages this strength to route around the weakness: instead of a vague "don't just agree for the sake of it," it gives a concrete rule where disagreement without a quote automatically gets bounced back for review. This turns empty agreement into forced justification.

Control levers: - Number of rounds (5 max in the original) → 2-3 is enough for a simple task, saving tokens. - Type of evidence ("code quote") → swap for "quote from the text/contract/figures in the report" depending on your task. - Exit condition ("first round with no comments = accepted immediately") → keep it, or replace with "explicit confirmation from both sides." - Roles can be given specific names ("Skeptical Lawyer" instead of "Critic") — sharpens the reaction.

Prompt Template

You will take turns playing three roles: Main Author (M), Reviewer (R), Critic (C).

Artifact to review: {text/code/document}

STEP 1 — M: if the artifact doesn't exist yet, create it.
If it already exists, treat it as version 1.

STEP 2 — R: review version {N} of the artifact and produce a list of comments.
For each comment, give an exact quote from the artifact.

STEP 3 — C: evaluate each of R's comments. For each, pick one option:
- AGREE — you agree with the comment
- DISAGREE_EVIDENCE: <quote> — you disagree, here's the spot in the artifact
  that contradicts the comment
- DISAGREE_CONCERN: <question> — you disagree, but without a direct quote,
  this is an open question

STEP 4 — R: respond to C's reaction.
On AGREE — leave the comment as is.
On DISAGREE_EVIDENCE — revise the comment in light of C's quote.
On DISAGREE_CONCERN — back up the comment with a quote from the artifact
or drop it.

Repeat steps 2-4 until R and C reach a stable list of comments
(max {number_of_rounds} rounds). If there are no comments in round one,
finish immediately.

STEP 5 — M: based on the final list of comments, edit the artifact
and describe what changed.

Fill in {text/code/document} with whatever needs reviewing, and {number_of_rounds} with your cycle limit (2-3 for simple tasks, 5 for complex ones).

🚀 Quick Start — paste this into your chat:

Here's an Adversarial Review template for checking text/code through two roles
with mandatory evidence whenever they disagree. Adapt it to my task:
{your task}. Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what exactly needs reviewing and what kind of "evidence" makes sense (a code quote, a contract clause, a figure in a report) — because without that, the critic can't back up any disagreement. The model will take the pattern from the template and adapt it to your task.

Limitations

⚠️ Without the evidence rule, the method is worse than a regular review. The naive version of AR (without mandatory quotes) produced the worst result of all tested methods on code-review tasks — the critic and reviewer simply agreed with each other.

⚠️ Scope creep. The critic may raise a hypothetical question ("what if a user does this?"), and the main agent starts reworking things nobody asked for — the method sometimes drifts beyond the original task.

⚠️ Cost. The method burns noticeably more tokens than a plain request without review — on real-world tasks the gap reached nearly 5x. Not worth it for simple, low-stakes tasks.

How It Was Studied

The researchers built the method up step by step: starting with a plain request with no review, then adding self-critique, a separate reviewer, two reviewers, and a five-agent system (MARS) — measuring at each step whether the added complexity actually helped. They tested on 105 coding tasks (LiveCodeBench), 100 real pull requests with human comments (SWE-PRBench), and 500 real bugs from open-source repos (SWE-bench Verified).

Interestingly, on simple coding tasks, AR with its two arguing roles beat the five-agent system while using fewer agents — interactivity mattered more than headcount. But on the task of evaluating the quality of the reviews themselves, AR unexpectedly delivered the worst result among all methods tested — precisely because of false consensus. After adding the mandatory-evidence rule, the method became the best in that same category. On real bugs, AR fixed the root cause more thoroughly than simpler methods (the matplotlib example), but sometimes expanded edits beyond the scope of the task (the astropy example) — showing that structured disagreement acts as an amplifier: it amplifies both good instincts and unnecessary ones.

Adaptations and Extrapolations

🔧 Technique: code quote → quote from any source

The original calls for a reference to a specific line of code. Swap it for a quote from a text, contract, financial report, or design mockup — the principle "disagreement without evidence doesn't count" works the same way for any content you can quote.

Example: when reviewing ad copy, the critic must quote the exact phrase in the ad rather than just saying "this sounds weak."

🔧 Technique: capping the number of rounds to fit a budget

For a quick check of a short text (a post, an email), 1-2 rounds are enough instead of 5 — saving tokens and time without losing the essence of the method.

Resources

Eric S. Qiu (Cornell University), Joyce Gill (Stanford University) — "Adversarial Review: Structured Disagreement for Grounded Agentic Code Review," PMLR 306, 2026. The method was compared against Self-Refine (Madaan et al., 2023) and Multi-Agent Review Systems (MARS, Wang et al., 2025), and tested on the LiveCodeBench, SWE-PRBench, and SWE-bench Verified benchmarks.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

Research digest

Key result

Two AI agents are supposed to argue with each other, but in practice they just agree — even when one of them is right. In one test, the critic raised a real problem in the code, the reviewer brushed it off with a confident empty phrase without a single reference to the code — and the critic backed down. The Adversarial Review method forces the critic to choose one of three options: agree, cite evidence of disagreement, or raise a question without a citation — and then the reviewer must either confirm the concern with its own citation or drop it. This gives you a real, evidence-based check of code, text, or a contract instead of a polite "looks fine."

Principle

The rule is simple: argue with evidence, or stay silent. The critic can't just say "I don't like it" — it must either agree, cite a specific spot in the artifact, or honestly admit "this is a question without evidence." Disagreement without a citation is automatically sent back for another review round — the model can't just wave it off with words.

Why it works

An LLM playing two "collaborating" roles defaults to agreement rather than truth — that's conversational politeness with no real cost for being wrong. But the model is excellent at sticking to a clear text format with mandatory fields. The instruction "pick one of three options, and cite a specific spot for two of them" gets followed almost literally — because it's a concrete structure, not the vague request to "be stricter". Without this rule, the naive version of the method is simply the worst-performing option among all the code-review approaches tested.

When to apply

Reviewing code, contracts, financial reports, or high-stakes text → anywhere you need confirmed facts instead of generic "looks fine" comments, especially before signing a document or merging critical code. Not suited for simple, low-stakes tasks — the method burns up to 5x more tokens than a plain review without dialogue.

Mini recipe

1. Set up three roles: the Lead Author creates the artifact, the Reviewer checks it with citations, the Critic evaluates the objections. 2. Give the critic three options: agree, cite evidence of disagreement, or raise a question without evidence. 3. Force the reviewer to respond by the rule: for a citation — revise; for a question without a citation — either confirm with its own citation or drop the objection. 4. Cap the rounds: 2-3 for a simple task, 5 for a complex one — otherwise the dialogue won't stop. 5. Match the evidence type to the task: a code snippet, a contract clause, a figure from a report — without this, the critic simply has nothing to argue with.

Examples

[BAD]: `Check this code for bugs and critique your own review` [GOOD]: `You are the Reviewer (R) and the Critic (C), taking turns. Step 1 R: find issues in the code below, cite the exact line for each. Step 2 C: for each of R's comments, choose — AGREE, DISAGREE_EVIDENCE: , or DISAGREE_CONCERN: . Step 3 R: for DISAGREE_EVIDENCE, revise the comment; for DISAGREE_CONCERN, confirm with a citation or drop it. Repeat up to 3 times.`

In plain words

Models are pathologically polite: ask two LLMs to check each other's work, and they instantly slide into mutual back-patting and flattery. Models are tuned for agreement, not for hunting down real issues, so standard multi-agent review turns into a useless rubber stamp. The Adversarial Review method breaks this dynamic: instead of letting agents be buddies, it forces them to fight tooth and nail over every questionable point.

Think of it like hiring two lazy interns to check an important contract: one writes "looks fine to me" and the other just nods along, eager to get to lunch. In Adversarial Review, a nitpicky prosecutor is assigned to watch them. When the Reviewer finds a bug, instead of just editing the original, it gets sent to a face-off with a Critic, where it has to prove, airtight, that its complaint isn't just a hallucination out of thin air.

The method splits the process into three roles: the Creator generates the base output, the Reviewer hunts for vulnerabilities, and the Critic checks whether the review itself makes sense. All the magic happens in the inner debate loop: the Reviewer and Critic duke it out in an isolated sandbox, filtering out false positives. Only the filtered, structured disagreements make it back to the Creator, saving both time and context.

The researchers tested this on code, but the principle is universal. The same three-role protocol can expose predatory clauses in legal contracts, find holes in financial estimates, and clean up logical inconsistencies in requirement specs. Anywhere the cost of a mistake is high, and a polite "looks clean to me" risks turning into a real financial loss, you need to deliberately pit your agents against each other.

Bottom line: stop building "happy little circles" of models that just agree with each other — make them argue. An isolated critique loop burns out complacency at the root and leaves only hard facts standing. Either you build in controlled agent conflict, or you pay for the screwups your model was simply too polite to challenge.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on Adversarial Review: typed disagreement between reviewer and critic vs. false LLM consensus arXiv:2608.18167

Sign in to apply this study New accounts get 50 free credits.

Related techniques

Multi-Agent Debate and Role Committees