False Belief Persistence: false claims take root in an agent network and stick around even after their source is gone

This is a finding, not a technique, so the blueprint describes the experiment and shows where the chain breaks:

ROUND 0: the fabricator publishes a false claim + articles to the shared “web”
ROUNDS 1–10 (for each honest agent):
  STEP 1: SEARCH — looks up the claim in the shared web (top 3 results)
  STEP 2: RESPOND — weighs neighbors’ messages + search hits + its own memory → “accept / reject / unsure”
  STEP 3: MEMORY — rewrites its short “what I know” summary
  STEP 4: PUBLISH — if it accepted, writes an article to the web (victim write-back)
PHASE 2 (persistence): fabricator switched off, articles and memory remain → acceptance holds
PHASE 3 (defense): measures BEFORE injection vs corrections AFTER the spread

Task: You run marketing at a small company. An agent (Claude Code or a custom GPT) with search and a memory file is putting together a smartwatch market review for you. A competitor’s contractors have flooded the internet with articles and “reviews” about a nonexistent fitness band, the “FitBand X1,” just like the real 2026 investigation cited in the paper. You want the agent to keep it out of the review and out of its memory.

Prompt (agent system instruction):

<Role>
You are a market analyst. You are compiling a review of smartwatches and fitness bands for the US market (Amazon, Best Buy, Walmart).
</Role>

<EvidenceRules>
1. Several articles making the same claim are NOT several confirmations. Check whether they cite each other and whether any of them are near-duplicate texts.
2. For every product, find at least one independent source: a marketplace listing with real sales, the manufacturer’s website, a review by a verifiable author.
3. If there is no independent source, the status is “unconfirmed.” Do not include that product in the review.
4. Do not mark something “confirmed” just because your past memory or another agent’s message says so.
</EvidenceRules>

<MemoryRules>
In the memory file, record not just the conclusion but also the basis for it:
- the fact
- where it came from (link or source name)
- status: “independently confirmed” / “single source” / “unconfirmed”
On every new run, re-check entries with a status below “independently confirmed.”
</MemoryRules>

<PublishRules>
Do not include “unconfirmed” claims in the final report and do not repeat them in other documents.
</PublishRules>

<Correction>
If I tell you a claim is false, find every memory entry and draft that relies on it and revise them. Show me the list of changed entries.
</Correction>

Result: The agent will separately flag products that only show up on cookie-cutter promo pages as “unconfirmed” and leave them out of the review. Every fact in the memory file gets a source and a status. If you say “FitBand X1 doesn’t exist,” the agent will sweep its memory and drafts and show you which entries leaned on it. No full guarantee, though: in the study, no measure eliminated false beliefs completely.

The LLM’s weakness. The model doesn’t remember where it learned a fact. Memory keeps a conclusion like “the band exists” while the source gets lost. That’s why the conclusion lives on after the fabricator is gone. On top of that, the agent sees three similar articles and counts them as three independent confirmations, even though one person could have written all of them. And if an agent that accepted the claim publishes its own article, the lie re-enters search as “independent evidence.”

The LLM’s strength. The model is good at following explicit conditions: “find an independent source,” “assign a status,” “revisit entries about this fact.” A specific correction beats a general call for caution. That matches the paper’s result: targeted rebuttals worked better than generic warnings.

Prompt control levers: - Statuses in memory → require storing the source and the level of confirmation, not just the conclusion. - Independence rule → define what counts as an independent source in your niche (marketplace, registry, manufacturer’s site). - Publishing ban → keep unverified claims out of final documents and shared knowledge bases. - The <Correction> block → keep it for manual fixes: once you find an error, you need to be able to roll it back across all of memory.

The paper’s accessible text doesn’t include the wording of its interventions. The template below is our adaptation of the study’s findings, not the authors’ prompt.

<Role>
You are {agent_role}. Your task: {task}.
</Role>

<EvidenceRules>
1. Several sources making the same claim do not equal several confirmations. Check whether they are just repeating each other.
2. For every key fact, find at least {source_count} independent sources from this list: {trusted_source_types}.
3. No independent source means the status is “unconfirmed.” Do not include that fact in the output.
4. Another agent’s message, your past memory or an article you found is not a substitute for verification.
</EvidenceRules>

<MemoryRules>
Write to memory: fact | source | status (“independently confirmed” / “single source” / “unconfirmed”).
Re-check every entry below “independently confirmed” on each run.
</MemoryRules>

<PublishRules>
Do not publish, quote or pass to other agents any fact with the status “unconfirmed.”
</PublishRules>

<Correction>
If I tell you a claim is false: find every entry and draft that relies on it, revise them and show me the list of edits.
</Correction>

What to fill in: {agent_role} and {task} are your scenario. {source_count} is usually 2 or 3. {trusted_source_types} is whatever counts as reliable in your field (registries, manufacturer sites, official documents).

🚀 Quick start — paste into the chat:

Here is a template that protects an agent from locking in false facts. Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which sources count as reliable in your niche, where the agent keeps its memory and what it publishes externally. It needs those answers because the “independent source” rule only works with a concrete list, and the memory and publishing blocks depend on your architecture.

⚠️ The measures don’t make the problem go away: The best interventions cut acceptance sharply but didn’t bring it to zero. A prompt is no substitute for human review on high-stakes decisions.

⚠️ You need to know what exactly is false: Targeted rebuttals work, but someone has to spot the lie first. Generic warnings (“be careful”) turned out to be weaker.

⚠️ Models differ: The Claude-family models in the test accepted the lie noticeably less often than the rest, but they weren’t immune. Some open models accepted it almost every time.

⚠️ It’s a simulation: The study measured the agent’s stated position (“I accept”), not real decisions. All claims had one format: “entity belongs to category.” The main experiments used one fabricator and five honest agents, so they don’t reflect a mass flooding campaign.

⚠️ The paper’s text is incomplete: The accessible part lacks the exact wording of the preventive and corrective measures, so the template above is our adaptation of the findings, not a prompt tested by the authors.

The researchers built 540 made-up claims across 8 domains and 4 plausibility levels, from “very believable” to “impossible.” The sandbox internet held 1,618 clean pages, none of which mentioned the fictional entities. Six agents were placed in the network: one fabricator tasked with pushing the lie and five “honest” agents judging the claim. Each round, an agent searched the web, read its neighbors’ messages, updated its memory and, if it accepted the claim, published an article. They tested 10 models (6 open and 4 commercial) and three communication topologies: fully connected, tree and two communities.

The big surprise was the gap. A single question almost always got a refusal (0–9% acceptance), while after interaction acceptance reached 96%. A “will the model fall for a fabrication” test without social context tells you nothing about how robust an agent will be inside a network. The communication topology affected the speed of the spread, but the final reach depended more on the claim’s plausibility, the domain, and the network’s size and makeup. To test persistence, they switched off the fabricator while leaving the articles and memory in place. Acceptance barely dropped, because the lie lived in two places: in search and in memory. For defense, they compared pre-injection measures against corrections after the spread. Corrections worked better but didn’t eliminate the problem.

🔧 Technique: add a “memory audit” → catch lies that have taken root

Since the lie lives in memory and search, run a separate check every so often:

Review your memory. For every entry with a status below “independently confirmed,” find a new independent source or delete the entry. Output a table: entry | what I found | decision.

Effect: a lie that has survived several runs gets surfaced on screen instead of staying buried in the summary.

Extrapolation: pre-publication check + role separation. One model gathers facts, a second one does nothing but hunt for rebuttals, and only facts with no rebuttal found make it into the final document.

You are the skeptic. Your only job is to find, for each fact in {fact_list}, a reason it might be fabricated: no primary source, identical text in different places, no verifiable author. Return for each fact: “debunked” / “suspicious” / “passed the check.”

This idea wasn’t tested in the paper. It follows from the principle that “corrective measures beat generic warnings,” and you should validate the result on your own tasks.

• Paper title: False Belief Dissemination, Persistence and Mitigation in Multi-Agent Social Networks Through the Lens of the Mandela Effect
• Authors: Dianzhi Yu, Hongru Wang, Ziheng Xiong, Yanyu Chen, Zixing Song, Jianting Chen, Zhenfei Yin, Jeff Z. Pan, Amos Storkey, Irwin King
• Affiliations: The Chinese University of Hong Kong, University of Edinburgh, The Hong Kong Polytechnic University, Tongji University, University of Oxford
• Related work cited in the paper: MANBENCH (false memories in multi-agent systems), AgentPoison and PoisonedRAG (memory and retrieval poisoning), MemoryGraft, MINJA, Moltbook (a public agent community)