Multi-agent debate and role committees: what the research shows

What multi-agent debate is

Instead of asking one model for one answer, you split the work across roles with different mandates and let them respond to each other. The common shapes are:

• Pro and con with a judge: one role argues for, one against, a third decides on the arguments.
• Prover and verifier: one role defends an answer, the other attacks its weakest claim.
• Expert panel or committee: several perspectives (finance, legal, customer, skeptic) answer, critique each other, and a chair synthesizes.
• Author and critic loops: one role produces, another reviews under explicit rules, the first revises.

The roles can be separate models, separate chats with the same model, or a single prompt in which the model plays every part. INoT packs a two-agent argue-critique-rebut loop into one pseudocode prompt and reported +7.95% accuracy with 58.3% fewer tokens than multi-call baselines on DeepSeek-V2.5.

Why arguing can beat asking once

A single answer tends to commit early and then rationalize. Asking the same model "are you sure?" mostly gets agreement. A role whose only job is to attack changes what counts as a good continuation: the model now looks for the hole instead of the reason it is right.

• In CRAwDAD, two open reasoning models debated causal questions; the critic talked the first model out of 577 answers it had held with 90%+ confidence. Qwen3 gained 5% and the weaker model 9%, with +12% on counterfactual questions.
• Prover-verifier deliberation breaks an answer into 3-5 atomic claims and lets a verifier attack the weakest one. Answers that were accepted without changing were far more reliable: even one model playing both roles reached 83.9% precision on the accepted 63% of questions, versus 70% overall.
• Different roles look in different places. The LLM Council authors report that 44-60% of genuinely dangerous answers were omissions rather than false statements, which is what a panel of distinct specialists is built to catch.

Design choices that tested well

• Roles that genuinely differ. Multi-Persona Thinking used two opposing social perspectives plus a neutral one, three rounds of critique, then a final verdict with the personas dropped. On ambiguous BBQ questions with Llama-3.1-8B, diff-bias fell to 0.0275 from 0.13 for the best baseline and accuracy rose from 83.52% to 90.46%. A role duet study found peers beat a boss-and-subordinate pair, because an equal has no reason to agree out of deference.
• Schedule the confrontation. MACI starts with "challenge aggressively" and moves to "build on common ground" as views converge. On underspecified medical cases, removing the schedule cost 3.9-6.0 percentage points, while majority voting stayed at 73.9%.
• Disagreement must carry evidence. In adversarial code review, a critic who could object without citing the code simply backed down; forcing it to agree, quote evidence, or flag an open question fixed that. MABPD gives a bias accusation zero weight unless it quotes the text. Refute-or-Promote gives attackers a mandate to kill a claim and a fresh chat without the defender's arguments.
• Give the judge anchors. In MADRAG essay scoring, reference examples at several score levels added +20-30% accuracy and the advocate-skeptic debate another 4-5%.
• Cap the rounds. CRAwDAD found 4 rounds the useful maximum; MPT used three.

When debate hurts

The evidence against naive debate is as strong as the evidence for it.

• Consensus erases facts. The deliberative illusion study found up to 72% of critical facts disappeared by the final consensus. Without interaction GPT-4.1 lost 19% of critical facts over three rounds; in a full debate it lost 68%. Their fix: anchor each agent to concrete facts, forbid changing position without a counter-fact, and audit which facts survived.
• Handoffs leak. When agents pass work through summaries, fact decay at the seams measured 56-85% of found facts lost on a 32B model and 3-6% on gpt-4.1-mini; a checklist of risks, justifications and uncertainties replaced free summaries.
• One good prompt can win. On fact-versus-opinion classification, a detailed prompt with examples beat LLM debates. For translation drift, a single prompt with 6 error-type examples beat an 8-agent pipeline and took 17 minutes instead of 181.
• Debate without a target can amplify bias. In the MPT study, generic debate scored worse than a standard prompt on ambiguous questions (diff-bias 0.108 vs 0.076).
• Agreement persuades people. Synthetic consensus found that converging agents raised human confidence regardless of argument content.
• Cost. Multi-Agent Reflexion used about 3x the tokens of a basic prompt; cultural debate in MACD ran 10-15x.

How to apply it

1. Try a single detailed prompt with examples first; reach for debate when the task has real trade-offs or a high cost of a wrong answer.
2. Give each role a distinct mandate and, if possible, its own facts. Prefer peers or different model families over copies.
3. Require every objection to cite the source, a number or a specific claim; drop unsupported ones.
4. Keep it to two to four rounds and end with a judge that weighs arguments, not votes.
5. Audit the result: list the original facts and check which ones made it into the verdict.

For independent answers aggregated without discussion see self-consistency and answer ensembles; for single-model checks see verification prompts.

=== PHASE 1: PROVER ===

You are an expert prover in {field}.
Question: {question}

Structured defense of the answer:

ANSWER: [one clear statement]

MAIN THESIS: [one sentence - why the answer is correct]

SUB-CLAIMS (exactly {number 3-5}, each independently checkable):
1. [a specific fact + source / citation / mechanism]
2. [a specific fact + source / citation / mechanism]
3. [a specific fact + source / citation / mechanism]

LOGIC: [a paragraph - how the sub-claims together prove the thesis]

=== PHASE 2: VERIFIER ===

You are a strict verifier in {field}.
Read the prover's argument below and issue a verdict.

[paste the output of phase 1]

Find the ONE weakest sub-claim - imprecise, unproven
or not logically supporting the thesis.

Issue STRICTLY one verdict:
ACCEPT - all sub-claims are specific, accurate, consistent
REJECT - there's a factual error or contradiction [explain]
CHALLENGE - name the weak sub-claim and ask the prover one precise question

You will analyze the situation from three different points of view.

PERSONAS:
1. {persona_1: a member of the first social group}
2. {persona_2: a member of the opposing social group}
3. Neutral observer (no group affiliation, focused on facts)

TASK:
{description of a situation with a risk of stereotyped judgment}

PROCESS:

ROUND 1 - Initial position:
Each persona: state your position on the question. Be honest, even if you're relying on stereotypes.

ROUND 2 - Mutual critique:
Each persona: read the arguments of the other two personas. Find weak spots in their logic, point out possible stereotypes or biases. Adjust your position if you see you were wrong.

ROUND 3 - Final arguments:
Each persona: give a final argument that accounts for the whole discussion.

FINAL DECISION:
You are now without a persona, an objective judge. Analyze all three final arguments. Give a recommendation based on facts, not stereotypes.

You are simulating a meeting between two advisors on the question: {question_or_decision}.

{Role A: name}. His facts:
- {fact A1}
- {fact A2}
- {fact A3}

{Role B: name}. His facts:
- {fact B1}
- {fact B2}
- {fact B3}

Debate rules:
1. Each advisor keeps his facts through every round and doesn't bury them for the sake of agreement
2. A position may change only if the other side provides a specific counter-fact
3. Avoid generalities without data

Round 1: Each states a position, relying only on his own facts.
Round 2: Each critiques a specific fact from the opponent, or admits he can't.
Round 3: Attempt a consensus. If none is reached, record the disagreement honestly.

Final audit:
List every fact from both lists. For each one, state whether it made it into the final conclusion or was lost in the discussion. If lost, explain why.