ROUND 1: Agent A → structured answer + confidence (%)
ROUND 2: Agent B → critique of the logic OR agreement
         ↓
         [If agreement] → FINAL
         [If disagreement] → ROUND 3
         
ROUND 3: Agent A → defends position OR revises
ROUND 4: Agent B → final answer (even without consensus)

Each round is a separate request. Context is passed from one to the next.

Task: You're prepping a pitch for an investor. You need to figure out what actually moves user conversion: the landing page redesign or the new offer.

Prompt (simulated debate in a single request):

You are two analysts arguing about a causal relationship. Keep debating until you reach consensus.

CONTEXT:
- After the landing page redesign, conversion rose from 2% to 3.5%
- At the same time as the redesign, we added a 20% discount for new users
- Both changes went live on the same day

QUESTION: What drove the conversion increase — the redesign or the discount?

DEBATE FORMAT:

[ANALYST A — first answer]
1. Formalization: which cause-and-effect relationships are possible
2. Analysis: which data support each hypothesis
3. Conclusion: what is the most likely cause
4. Confidence: X%

[ANALYST B — critique]
1. Weak spots in Analyst A's logic
2. Alternative interpretation
3. Own conclusion + confidence

[ANALYST A — response to critique]
Defend the position OR revise it in light of the arguments

[CONSENSUS]
Final agreed conclusion. If there is no consensus, explain what data would be needed to settle the dispute.

Result: The model will generate 3-4 rounds of debate. Analyst A might argue the redesign is the cause (better UX → more people reach the offer). Analyst B will point out that without an A/B test you can't separate the effects, and the discount may have been the main driver. The consensus acknowledges the uncertainty and recommends: roll back the discount for a week and measure conversion.

The LLM's weakness: a model is often confident in a wrong answer. Especially on causal questions, where you have to separate correlation from causation. The study found 577 cases where the model was 90%+ sure and still wrong. Without outside criticism, it would have stuck to its guns.

The LLM's strength: a model makes an excellent critic when it is explicitly given that role. It finds logical holes, points out alternative explanations, and demands evidence.

How the method works: debate forces a rethink through structured conflict. The first agent has to either defend its position with arguments or admit the mistake. It works even when the opponent is weaker: what matters is the challenge itself, not the strength of the critic.

Control knobs:

• Number of rounds: 4 is the sweet spot per the study. Fewer, and they don't have time to agree. More, and things stall as the models repeat themselves.
• Confidence: demand a numeric estimate (%). Models change their minds more often when the opponent is more confident.
• Roles: you can give the agents different "personalities": optimist vs. skeptic, practitioner vs. theorist.

You are two experts with different positions. Run a structured debate.

TASK: {task description}

DATA:
{facts, figures, context}

QUESTION: {specific cause-and-effect question}

FORMAT:

[EXPERT A]
Analysis:
- Which cause-and-effect relationships are possible
- Which data support my hypothesis
Conclusion: [specific answer]
Confidence: [X]%

[EXPERT B — CRITIQUE]
Weak spots in A's logic:
- [specific errors or gaps]
Alternative interpretation:
- [another explanation of the same data]
My conclusion: [agree/disagree + own answer]
Confidence: [X]%

[EXPERT A — RESPONSE TO CRITIQUE]
[Defend the position with new arguments OR revise it in light of the critique]
Final conclusion: [answer]
Confidence: [X]%

[CONSENSUS]
Agreed answer: [yes/no/more data needed]
Rationale: [why we reached this conclusion]
What remains uncertain: [if anything]

Placeholders:

• {task description} — the context of the situation
• {facts, figures, context} — the specific data to analyze
• {question} — exactly what needs to be determined

🚀 Quick start — paste into the chat:

Here is a two-expert debate template. Adapt it to my task: [your task]. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask about the data, the context, and the specific question, because causal analysis needs facts, not abstractions.

⚠️ Synthetic benchmark: The study used CLadder, a dataset of "clean" questions. Real business problems are messier: incomplete data, ambiguous wording, hidden variables.

⚠️ Counterfactuals are still hard: Even after debate, accuracy on "what would have happened if..." questions is around 80%. Humans do better (~95% by the researchers' estimates). Don't rely on the model as your only source for high-stakes "what if" decisions.

⚠️ Time: Debate = multiple requests. 10,000 questions took 380 hours. Fine for hard problems, overkill for routine work.

⚠️ Short answers hurt: DeepSeek-R1 often gave 1-2 sentence answers with no reasoning. That made it less persuasive even when it was right. If you simulate a debate, demand detailed answers from both agents.

A team from the University of Calgary took 10,112 questions from CLadder, a benchmark where every question is tied to a formal causal model. The questions are split into three difficulty levels: Rung 1 (statistical associations: "is there a correlation?"), Rung 2 (interventions: "what happens if we intervene?"), Rung 3 (counterfactuals: "what would have happened if?").

The debaters were Qwen3-32B and DeepSeek-R1-Distill-Qwen-32B, two open reasoning models. Picking two different models is deliberate: research shows that same-type agents often "collude" in one direction and miss each other's errors.

Key finding: on Rung 1 (simple questions), 93% were settled without debate: the models agreed right away. On Rung 3 (counterfactuals), a quarter of the questions required an argument. Makes sense: the harder the task, the more room for disagreement.

The surprise: the weaker model (DeepSeek-R1) helped the stronger one (Qwen3). Even when DeepSeek was wrong, its critique pushed Qwen to re-examine and reinforce the correct answer. The value of debate isn't in the opponent being right, it's in the challenge itself.

The baseline was GPT-4 + Chain-of-Thought from the original CLadder paper (2023). GPT-4 scored 70.40% overall accuracy and 62% on counterfactuals. After debate, Qwen3 reached 89.41% overall and 80.35% on counterfactuals.

Context: Prompt for the first agent: step-by-step structured causal analysis (CausalCoT).

You will be asked a causal reasoning question. You should structure your final answer as follows:

1. VARIABLE IDENTIFICATION: Identify and list all relevant variables mentioned in the problem. Clearly state which variable is the treatment/intervention and which is the outcome.

2. CAUSAL GRAPH CONSTRUCTION: Based on the given information, describe the causal relationships between variables. Identify any confounders, mediators, or colliders present in the problem.

3. QUERY TYPE CLASSIFICATION: Determine what type of causal question is being asked:
- Associational (Rung 1): Questions about correlations or conditional probabilities
- Interventional (Rung 2): Questions about the effect of doing/intervening
- Counterfactual (Rung 3): Questions about what would have happened under different circumstances

4. FORMULA IDENTIFICATION: Based on the query type, identify the appropriate formula or method needed to answer the question (e.g., conditional probability, do-calculus, counterfactual reasoning).

5. CALCULATION: Show your work step by step, applying the appropriate formulas to the given probabilities and relationships.

6. FINAL ANSWER: State your definitive answer clearly as "yes" or "no" based on your calculations.

7. CONFIDENCE: Provide a confidence score (0-100%) for your answer.

Important: Be confident in your reasoning. The questions may involve scenarios that seem to contradict common sense - this is intentional. Focus on the formal causal relationships provided, not on real-world expectations.

Context: Prompt for the second agent: critique and error hunting.

You are reviewing another model's causal reasoning. Your role is to:

1. SCRUTINIZE: Carefully examine the reasoning provided for logical flaws, calculation errors, or misapplication of causal concepts.

2. IDENTIFY ISSUES: If you find problems, clearly state:
- What the error is
- Why it's wrong
- What the correct approach should be

3. PROVIDE YOUR ANSWER: Give your own answer with reasoning.

4. CONFIDENCE: State your confidence in your answer (0-100%).

If you agree with the original answer and find no flaws, explicitly state your agreement and provide your confidence.

Previous model's response:
{first_agent_response}

💡 Adaptation for business decisions: Instead of formal causal analysis, a debate about risks and opportunities.

You are two advisors to a startup founder. One is an optimist, the other a skeptic.

DECISION: {description of the decision to be made}

[OPTIMIST]
Why this will work:
- [arguments for]
Potential upside: [what we gain]
Confidence in success: [X]%

[SKEPTIC — CRITIQUE]
What could go wrong:
- [specific risks]
Weak spots in the optimist's logic:
- [where they underestimate the risks]
Confidence in failure: [X]%

[OPTIMIST — RESPONSE]
[Defend or adjust the plan in light of the critique]

[CONSENSUS]
Decision: [do it/don't do it/do it with changes]
Key conditions for success: [what has to be true]

🔧 Technique: names instead of roles → sharper critique

Instead of faceless "Expert A" and "Expert B", give them distinctive names or roles:

• "Michael, a CFO with 20 years of experience" vs. "Anna, a venture investor"
• "A conservative banker" vs. "A serial entrepreneur"

The model produces more distinctive arguments when an agent has a "personality".

🔧 Technique: demand numeric confidence → models change their minds more often

The study showed that models take the opponent's confidence into account. If one agent is 95% sure and the other 60%, the second one backs down more often. Always require Confidence: [X]% in every round.

Paper: CRAwDAD: Causal Reasoning Augmentation with Dual-Agent Debate

Code: https://github.com/finnvamosi/CRAwDAD

Benchmark: CLadder — https://github.com/causalNLP/cladder

Authors: Finn G. Vamosi, Nils D. Forkert (University of Calgary, Hotchkiss Brain Institute)

Related work:

• Pearl's Ladder of Causation (The Book of Why)
• CausalCoT prompting (Jin et al., 2024)
• Multi-Agent Debate surveys (Tran et al., 2025; Tillmann, 2025)