Multiverse Analysis: Get an Honest Verdict from an LLM by Running Several Analysts Instead of One

You can do this in a single long prompt — the model simulates all three analysts one after another.

STEP 1: Three parallel analyses in one prompt
  Analyst A (skeptic) → looks for weak spots, risks, counterarguments
  Analyst B (neutral) → follows the data with no prior assumptions  
  Analyst C (optimist) → looks for opportunities and supporting factors

STEP 2: Each analyst → reaches a conclusion using its own logic
  Separate section: methods, assumptions, conclusion

STEP 3: Synthesis (as a separate request or at the end of the same prompt)
  → Where all three agree = solid conclusion
  → Where they diverge = zone of genuine uncertainty

The task: Sam is a product manager who wants to convince an investor to ship a new feature in a SaaS product. He's asked ChatGPT "why will this feature be a hit" several times — and every time he got an enthusiastic answer. Now he wants an honest check on the idea before the pitch.

Prompt:

I want to launch a new feature in our B2B SaaS (a task tracker for small businesses, 
~500 paying customers, ARPU $29/month): 
an AI assistant that automatically prioritizes tasks 
based on deadlines and team workload.

We'll run this through three analysts — each does an independent analysis:

**Analyst A — Skeptic**
Role: a venture analyst who has seen 200 failed features.
Task: find everything that could go wrong. 
Which data/assumptions are weak? Where does the logic fall apart?
Methods: look at adoption risk, unit economics, competition, 
technical debt. Conclusion: do you support the launch or not?

**Analyst B — Neutral**  
Role: an independent consultant with no prior opinion.
Task: weigh the arguments for and against without leaning either way.
Methods: separate facts from assumptions, figure out 
what we need to know to make the decision. 
Conclusion: do you support the launch or not?

**Analyst C — Optimist**
Role: a product director who believes in AI features.
Task: find the best-case development scenario.
Methods: look for demand signals, market analogs, 
LTV growth potential. Conclusion: do you support the launch or not?

---
After the three analyses, add a SYNTHESIS block:
- Where all three agree → solid conclusion
- Where they diverge → zones of uncertainty
- The one question/metric to verify before launch

Result:

The model produces three separate analysis blocks with different emphases. The skeptic zeroes in on adoption risk and competition from Notion AI. The neutral analyst asks for data on customer feature requests. The optimist finds analogs and room to grow ARPU.

The Synthesis block makes it clear: where all three converge (say, "validate demand before building anything") — that's a solid conclusion. Where they diverge (say, "how technically hard is this") — that's real uncertainty you need to close with data, not with an LLM's opinion.

An LLM has no "objective" analytical stance. When a model is asked to analyze something, it generates text based on patterns in the prompt. The framing decides which patterns fire: "find the risks" launches one chain, "find the opportunities" launches another. Both chains lead to technically correct but systematically different conclusions.

The more specific the analyst role, the more consistent the methods. A skeptical analyst doesn't just "think skeptically" — it picks more conservative moves: accounts for more variables, simplifies less, is slower to accept conclusions. The optimist does the opposite: simplifies the specification, throws out "anomalous" cases. That's not a bug — it's how an analytical stance shapes methodological micro-decisions.

Synthesizing across a distribution is more honest than a single verdict. When three analysts converge, that's a signal of robustness. When they diverge, it's not "someone got it wrong" — it's "there's real uncertainty here." You see the shape of the uncertainty instead of the illusion of one right answer.

Knobs you can turn in the prompt:

• Number of analysts: 2 is enough for a basic contrast (skeptic + optimist). 3 gives a fuller picture.
• Role detail: The more specifically you describe the analyst (experience, past mistakes, methods), the more consistent its analytical logic.
• Synthesis instruction: You can swap "where they all agree" for "which decision survives the skeptic's critique" — that gives a more conservative bottom line.
• Separate requests: If you want a clean experiment, ask each analyst in a separate request. The model won't "remember" the previous answer and won't adjust to it.

I need a multi-stance analysis of {topic/decision/idea}.

Context: {key facts and data}

**Analyst A — Skeptic**
Role: {who — a seasoned critic, venture analyst, editor}
Task: find the weak spots, the risks, where the logic breaks down.
Focus: {what to check — unit economics / assumptions / competition}
Conclusion: {binary decision — support/don't support} + main argument

**Analyst B — Neutral**
Role: an independent consultant with no prior opinion
Task: weigh the arguments without bias, separate facts from assumptions.
Focus: what we need to know to make a well-grounded decision
Conclusion: {binary decision} + main argument

**Analyst C — Optimist**
Role: {who — an experienced practitioner who believes in this category}
Task: find the best realistic scenario.
Focus: demand signals, analogs, potential
Conclusion: {binary decision} + main argument

---
SYNTHESIS:
- Where all three agree → solid conclusion
- Where they diverge → zones of genuine uncertainty  
- One question/check to close before {action}

Placeholders: - {topic} — what you're analyzing: a business idea, a decision, an argument, a strategy - {key facts} — numbers, context, constraints - {who} — a specific role with experience and a point of view, not a faceless "expert" - {what to check} — specific methods or areas for this analyst - {binary decision} — support the launch / recommend / approve

🚀 Quick start — paste into the chat:

Here's a multiverse analysis template. Adapt it to my task: {your task}. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what to analyze, what data you have, and what decision you need to make — because without context it can't properly tune each analyst's focus. It will take the three-role pattern from the template and adapt it to your specific task.

⚠️ Not a substitute for real data: If you don't have numbers, the analysts will argue about assumptions, not facts. The technique reveals the shape of the uncertainty; it doesn't remove it.

⚠️ Confirmation framing in the context: If earlier in the chat you already discussed the idea in a positive light, the model "remembers" that, and all three analysts may drift into optimistic territory. For a clean run, start a new chat.

⚠️ No guarantee of full separation: In a single prompt, the analysts can "bleed" into each other. For more independent results, use three separate requests with clean contexts.

⚠️ Works worse for purely subjective judgments: If the task is "rate how beautiful this is," you'll get a spread across stances, but not much substantive analysis. The sweet spot is decisions with testable assumptions.

The idea was simple: what if you replicated the classic "many-analyst" studies (where 29 independent teams get the same dataset and test the same hypothesis) — but used LLM agents instead of people? Researchers from Amazon AWS and CMU took three datasets — data on referee racial bias in soccer, an RCT on AI assistance in programming, and ANES surveys on ideology — and ran ~30 independent AI analysts with different framings on each. Every agent had access to Python, wrote its own code, and turned in a report with no human in the loop.

To filter out outright junk, they added an AI auditor — a separate model that checked every run: did the code execute, was the estimand specified correctly, were there any hallucinations? Out of nearly 5,000 runs, 67% passed the audit. Interestingly, the confirmation-seeking personas were rejected most often (57%) — the auditor spotted and removed aggressive p-hacking. But even among the runs that passed, the spread in conclusions stayed huge.

The most telling detail: the researchers deliberately flipped the hypothesis in the AI-and-programming dataset (the original study found a slowdown; they asked the analysts to test for a speedup) — precisely to avoid "contamination" from training data. A smart control. The soccer-referee dataset, by contrast, turned out to be "highly contaminated": some agents reproduced the original paper's conclusions before they even looked at the data.

1. The auditor as a separate request

The study showed the value of a separate auditor — a model that doesn't analyze but checks the quality of the analysis. You can use this in a chat:

🔧 Technique: two-step analysis with an auditor

Step 1: ask the LLM to do the analysis Step 2: new request: "You are an auditor. Review this analysis: which assumptions are unjustified? Where doesn't the conclusion follow from the data? What did the analyst ignore?"

In the auditor role, the model will find weak spots it missed itself while playing the analyst.

2. "Persona contamination" — auditing your own prompts

The study found that confirmation framing sneaks into prompts unnoticed. It's useful to flip this into a diagnostic for your go-to prompts:

🔧 Technique: bias check

Paste in your usual prompt (the one you use for analysis) and ask: "Identify any wording in this prompt that nudges toward a particular conclusion. Rewrite it in neutral language."

You'll often discover that "assess the potential of my idea" is confirmation-seeking, while "test the hypothesis about this idea's potential" is neutral framing.

Many AI Analysts, One Dataset: Navigating the Agentic Data Science Multiverse Martin Bertran, Riccardo Fogliato, Zhiwei Steven Wu Amazon AWS / Carnegie Mellon University, February 2026

Related work cited in the study: - Silberzahn et al. [2018] — the original many-analyst study (29 teams, soccer referees) - Breznau et al. [2022] — 73 teams, immigration policy - Gelman & Loken [2013] — "garden of forking paths"