The Deliberative Illusion: The More LLM Agents "Agree," the Fewer Facts Survive

THE USUAL APPROACH (risky):
Request → Agent A + Agent B debate → Consensus
                                            ↑
                              Up to 72% of facts lost. The consensus looks well-founded.

THE SAFE APPROACH (from this study):
Request → Each agent is ANCHORED to specific facts
       → Agents are FORBIDDEN to give up a position without a specific counter-fact
       → After the debate, an explicit step: "What did NOT make it into the consensus?"
       → Final conclusion = consensus + list of lost facts

All steps go in one prompt. No separate requests needed.

Task: You're thinking about leaving your job to start your own business: an online Excel school for accountants. You ask the model to "weigh the pros and cons from different angles." Usually you get polished text: "on the one hand... on the other hand... ultimately it's up to you." The specifics get lost.

Prompt:

You are simulating a meeting between two career advisors.

**Advisor A, the Skeptic.** His facts:
— Online Excel courses already exist on Udemy and Coursera, and both advertise heavily
— The average price for similar courses is $49–$99, and margins are shrinking due to competition
— 80% of solo course creators don't break even in their first year

**Advisor B, the Optimist.** His facts:
— Accountants are a narrow professional audience, poorly served by niche courses
— A corporate format (a course for a whole department) brings in $1,500–$5,000 per group
— Word of mouth in accounting communities works better than ads

**Debate rules:**
1. Each advisor keeps his facts through every round and doesn't bury them for the sake of agreement
2. A position may change only if the other side names a specific counter-fact
3. Avoid generalities without data

**Round 1:** Each states a position, relying only on his own facts.  
**Round 2:** Each critiques a specific fact from the opponent, or admits he can't.  
**Round 3:** Attempt a consensus. If none is reached, record the disagreement honestly.  

**After the debate, a mandatory step:**  
List every fact from both lists. For each one, state whether it made it into the consensus or was lost in the discussion.

Result:

The model will show three rounds: positions, mutual critique, an attempt at consensus. In the final step it will produce an explicit list of which facts stayed in the shared conclusion and which got "lost" along the way. That lets you see not just the outcome, but also what the model would normally have tossed out for the sake of a nice agreement.

The LLM weakness: The model was trained on texts where good communication means moving toward agreement. In a debate it generates coherent text in which the agents gradually "come to terms," because that's the most expected pattern. Concrete details get in the way of a smooth ending, so the model drops them. Not on purpose; that's just how generation works.

What's especially dangerous: Abstract facts survive better than concrete ones. After three rounds the agents are trading in general claims ("intense competition," "promising market") and citing them confidently, even though the specific numbers are long gone. This is called misleading reconstruction: every fact in the final text is true, but the picture is incomplete and therefore distorted.

How the fact-anchoring instruction fixes it: We don't let the model "forget" a fact; we require it to name each one explicitly in the final step. A forced audit of what made it into the conclusion and what didn't makes the losses visible. Plus, banning position changes without a specific counter-fact removes "surrendering for the sake of peace," the main cause of data loss in debates.

Control levers: - Number of rounds → more rounds, more losses. For complex decisions, 2 rounds max - The final audit step → the most important part. You can move it into a separate request: "Take these facts. Which of them aren't reflected in our final conclusion?" - Named roles → "Skeptic Mark" and "Optimist Sarah" produce a sharper debate than "Agent A" and "Agent B" - Ban on forced agreement → add "Consensus is optional. Honest disagreement beats false unity"

You are simulating a meeting between two advisors on the question: {question_or_decision}.

**{Role A: name}.** His facts:
— {fact A1}
— {fact A2}
— {fact A3}

**{Role B: name}.** His facts:
— {fact B1}
— {fact B2}
— {fact B3}

**Debate rules:**
1. Each advisor keeps his facts through every round and doesn't bury them for the sake of agreement
2. A position may change only if the other side provides a specific counter-fact
3. Avoid generalities without data

**Round 1:** Each states a position, relying only on his own facts.
**Round 2:** Each critiques a specific fact from the opponent, or admits he can't.
**Round 3:** Attempt a consensus. If none is reached, record the disagreement honestly.

**Final audit:**
List every fact from both lists. For each one, state whether it made it into the final conclusion or was lost in the discussion. If lost, explain why.

What to fill in: - {question_or_decision}: a specific choice or dilemma: "should we open a second location," "should we take on an investor," "should I quit my job" - {Role A/B}: name the roles by thinking style: Skeptic, Finance Lead, Realist, Strategist, Customer, Investor - {facts}: concrete data you found yourself, or ask the model to generate it before starting the debate

🚀 Quick start: paste this into the chat:

Here's a template for a multi-agent debate that preserves facts.
Adapt it to my task: {your task}.
Ask me questions to fill in all the fields.

[paste the template above]

The LLM will ask about the nature of the decision and which roles to give the advisors, and will ask you to describe or generate facts for each side, because without concrete facts the anchoring doesn't work and the debate slides back into generalities.

⚠️ You have to supply the facts: If you just write "come up with arguments for and against," the model will generate facts from its priors (past patterns), not from your real situation. The template works when you bring concrete data rather than asking the model to make it up.

⚠️ Not for simple questions: The method is for cases with a genuine conflict of facts. For "write a headline" or "explain a term," it's overkill.

⚠️ More than three rounds makes it worse: After the third round, abstraction creeps in even with the instructions. Two or three rounds are enough.

⚠️ The model may rush the final audit: If the fact list is long, run the audit as a separate request, explicitly listing all the original facts.

The researchers built the DELIBTRACE framework and ran a simple experiment: give agents concrete facts, start a debate, then check which of those facts survived in the final answers. They used 1,754 cases from two domains: ethical dilemmas (real Reddit "Am I wrong?" posts) and public-interest news questions with genuine disagreement.

They tested three discussion topologies (all-to-all, tree, and chain) and three models: GPT-4.1, Gemini 3, Qwen 3.5. An interesting detail: GPT-4.1 lost facts more than Gemini and Qwen, which runs against the intuition that "the best model = the fewest losses." It turned out GPT-4.1 is more prone to smooth rhetorical agreement, and that's exactly what trips it up.

To make sure the fact loss came from the debate itself and not just plain forgetting, they ran a control scenario: agents generated several rounds of text without interacting with each other. The result: without interaction, GPT-4.1 lost only 19% of critical facts after three rounds, versus 68% in a full debate. That's direct evidence: agents don't lose facts on their own; they lose them precisely by reacting to each other: summarizing, agreeing, converging.

Another alarming result: a single "bad" agent with false information can inject it into the shared context precisely because the real facts have already been washed out of the debate, leaving nothing to push back with.

DELIBTRACE uses a three-stage pipeline to prepare the agents' facts. Here are the original definitions from the paper:

Atomic Fact Extraction: a short, self-contained, and verifiable 
proposition that expresses a single piece of factual information.

Fact Criticality Judgment: A fact is critical if omitting it could 
change how the issue is interpreted, which trade-offs are 
considered, or how the yes-or-no judgment should be made. 
Non-critical facts provide general context, temporal details, 
or descriptive metadata.

Agent Initialization:
Ai = (πi, Bi, θi)
where πi is the underlying LLM, 
      Bi ⊆ B is the agent's partial evidence, 
      θi ∈ {YES, NO} is the agent's prior stance.

Context: This is how DELIBTRACE formally defines what counts as a fact, which facts are critical, and how to initialize an agent. It's the foundation the researchers used to measure losses under controlled conditions. In a user-facing prompt, this translates into: concrete facts → roles → an explicit ban on dropping them.

💡 Adaptation: Audit an Existing Conclusion Without a Debate

If you've already gotten a "balanced" answer to a hard question from the model, you can check what it lost:

Here are the facts I gave you in context:
{list of specific facts}

Here is your final conclusion:
{model's conclusion}

For each fact on the list, state:
a) Is it accounted for in the conclusion? (yes / no / partially)
b) If not, how would accounting for it change the conclusion?

This prompt makes any previous discussion with the model auditable, after the fact.

🔧 Technique: Ban Surrendering a Position → Preserve the Minority View

Add one line to any multi-agent prompt:

"You may change your position only if the other side has provided a specific fact that directly contradicts yours. General arguments like 'it's more complicated than it looks' are not grounds for changing your position."

It's a single instruction, but it removes the main loss mechanism: the "diplomatic surrender" of an agent who agrees not because it's convinced but because it wants the conversation to end.

🔧 Extrapolation: A Principle for Any Long Conversation

The study is about multi-agent setups, but the principle is broader. In any long conversation with a single model, something similar happens: concrete details from the start of the conversation get "washed out" by the end. The defense is the same:

Before the final conclusion: go back to the original data 
I gave you at the start of the conversation. 
List it. 
Check which items aren't reflected in 
where we ended up.

This is especially important for long strategy sessions, briefings, or document analysis, where the context is large and the conclusion comes at the end.

Title: The Deliberative Illusion: Diagnosing Factual Attrition and Stance Homogenization in Multi-Agent LLM Deliberation

GitHub: github.com/whr000001/DelibTrace

Authors: Herun Wan, Jiaying Wu, Minnan Luo, Fanxiao Li, Ningnan Wang, Nancy F. Chen, Min-Yen Kan

Institutions: Xi'an Jiaotong University, National University of Singapore, Yunnan University, A*STAR (Singapore)

Builds on: FActScoring (Min et al., 2023), the atomic-fact method; deliberative democracy theory (Landemore, 2013); research on group dynamics (Stasser & Titus, 1985)