TL;DR
Researchers compared what top forecasters and average ones lean on when they analyze hard questions. The verdict: the difference isn't search quality, it's the questions they ask themselves after the data is in. Top forecasters systematically do three things that ordinary LLMs almost never do unless told: they run a pre-mortem ("if I turn out to be wrong, why?"), look for alternative perspectives ("what am I missing?") and consider black swans ("what unlikely event would change everything?").
The main finding: By default, frontier LLMs spend ~94% of their effort on gathering information (H, "search") and nearly ignore the pre-mortem (P), other perspectives (O) and wildcards (W). Yet P, O and W are exactly what separate a sharp analyst from an average one. Claude Opus 4.6 ran a pre-mortem in only 9.5% of cases. The best forecaster did it in 37.8%. Worse still, LLMs systematically confuse rhetoric with intent: if a leader says "we will do this," the model reads it as a commitment rather than a negotiating move. It never asks, "why would he say that right now?"
The fix: explicitly instruct the LLM to walk through CHAMPS KNOW, Tetlock's 10-dimension framework for accurate thinking. Three underused dimensions matter most: P (what if I'm wrong?), O (which perspectives am I ignoring?), W (which black swan would change everything?). One prompt, one request, zero infrastructure.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
The method fits in a single prompt, one request:
STEP 1: Norms and precedents (N) → What usually happens in similar situations?
STEP 2: Key players and incentives (K) → Who decides? Why right now?
STEP 3: Comparison classes (C) → What does this resemble from the past?
STEP 4: Pre-mortem (P) → If the forecast is wrong, why?
STEP 5: Other perspectives (O) → What am I missing?
STEP 6: Wildcards (W) → What's unlikely but changes everything?
STEP 7: Bottom line → Conclusion with a probability and the key arguments
Research and conclusion happen in one request. The structure swaps the LLM's default mode ("gather info → answer") for the mode of a strategic analyst.
Example
Task: Mark is the founder of an HR-tech startup. His big question: "Will Google launch a competing ATS (applicant tracking system) in the next 12 months, and should I delay my funding round?"
Prompt:
Analyze the following question as a strategic analyst, using the
CHAMPS KNOW structure. Go through each dimension in order:
**Question:** Will Google launch a competing ATS product (hiring
automation system) in the next 12 months?
**Context:** Google already runs Google for Jobs, the HR-tech market is growing,
competitors include LinkedIn, Greenhouse and Lever. I'm a startup with 200 clients
considering a funding round.
---
Go through each block:
**N — Norms and precedents:**
How does Google usually enter adjacent niches? What is typical of its
product expansion: build, buy, or partner?
**K — Key players and their incentives:**
Who inside Google makes this kind of decision? What incentive do they have
to launch an ATS specifically, and specifically now? What is shifting in their priorities
in 2025?
**C — Comparison classes:**
Which previous Google expansions does this resemble? How is this
situation different from those precedents?
**P — Pre-mortem (MANDATORY):**
Assume that 12 months from now Google has released a competing ATS.
Why might my analysis have underestimated this? Which signals am I probably
ignoring right now?
Now assume the opposite: 12 months from now Google still hasn't entered
the ATS market. Why? What held it back?
**O — Other perspectives:**
Which viewpoints on the situation haven't I considered? How does a
major investor see it? How does a Google manager who was asked
to build an ATS see it?
**W — Wildcards:**
What unexpected event could sharply shift the situation one way or
the other? For example: a regulatory change, an M&A deal,
a change in strategy?
---
**Bottom line:**
Based on the analysis above, estimate the probability that Google enters ATS
in the next 12 months. State the main argument for, the main argument
against, and what would change your estimate.
Result: The model will work through every block in order. In block K you'll get reasoning that goes beyond "Google is big" to "what is the product team's concrete incentive right now." In block P the model will argue against itself and may revise its estimate. In W you'll see scenarios like "if Microsoft buys Greenhouse, Google speeds up." The output won't be "probably yes/no" but a structured conclusion with concrete triggers for revisiting it.
Why It Works
An LLM's default mode is linear. The model gathers information, forms a position and defends it. If a text says "we will do this," it takes that as a signal without asking "why are they telling us this?" This isn't a quirk of one model; it's a pattern. The study showed that even the best frontier agent (Opus 4.6) ran a pre-mortem on its own in only 1 out of 10 cases.
LLMs are good at following a role. When you spell out the reasoning structure, the model executes it. It doesn't "think" differently; it generates text along a different pattern. The "P — pre-mortem" block literally changes the next chunk of generated text: the model is forced to look for counterarguments to the position it just formed.
Three levers for tuning the prompt: - Add "MANDATORY" to block P → the model won't skim it as a formality - Drop blocks M and S → if the question needs no math or rewording, the prompt gets shorter - Replace general context with hard data → block K gets sharper when you feed it real facts about the company or person
Prompt Template
Analyze the following question as a strategic analyst.
Go through each block in order:
**Question:** {question to analyze}
**Context:** {known facts, data, situation}
---
**N — Norms and precedents:**
What typical behavior patterns apply to this situation?
What usually happens in similar cases?
**K — Key players and incentives:**
Who makes the decisions? What are their real incentives right now?
Why might they do this (or not do it) at this moment?
**C — Comparable situations:**
What does this resemble from the past? How is this case different?
**P — Pre-mortem (MANDATORY):**
Assume my forecast turns out wrong. Why would that happen?
Which factors am I probably underestimating right now?
Now assume the opposite outcome. What caused it?
**O — Other perspectives:**
Who might disagree with me, and why? Which viewpoints
haven't I considered?
**W — Wildcards:**
What unlikely but realistic event could change everything?
One or two scenarios.
---
**Bottom line:**
Based on the analysis above: {question to analyze}: what is most likely,
the main arguments for and against, and what would change your estimate?
Placeholders:
- {question to analyze} — a concrete question: "will X enter market Y," "should I do Z," "how will a competitor react"
- {known facts} — what you already know: news, data, rumors, history
🚀 Quick start — paste into the chat:
Here is a template for structured strategic analysis (CHAMPS KNOW).
Adapt it to my task: [your task or question].
Ask clarifying questions to fill in the context.
[paste the template above]
The LLM will ask what question you're analyzing and what context you have, because block K (incentives) needs concrete data about the actors or the analysis stays generic.
Limitations
⚠️ Questions without a clear analytical structure: Pre-mortems and wildcards shine on strategic questions with concrete players and incentives. On creative or subjective tasks ("write a piece about nature") the structure adds nothing.
⚠️ Quality depends on the context in the prompt: Block K ("incentives of key players") will be shallow unless you give the model real data about the person or company. Without context it produces generic patterns, not a specific analysis.
⚠️ It doesn't replace domain expertise: The study showed that even with the right reasoning structure, the LLM sometimes lacks the necessary context (for example, on the specifics of Nigerian union negotiations). Structure improves reasoning, but it can't fill a knowledge gap.
⚠️ An ensemble beats a single run: The best results in the study came from averaging several independent runs. In a chat, that means: run the prompt twice in separate sessions and compare the conclusions.
How They Tested It
The FutureSearch team built BTF-2, a set of 1417 forecasting questions covering October–December 2025. The twist: the questions came with a frozen corpus of 15 million documents captured at the moment the questions were posed. That let them rerun agents over and over on the same data without the agents "peeking" into the future via the internet.
Four top frontier agents (Opus 4.6, Gemini 3.1 Pro, GPT-5.4, Grok 4.20) researched the questions on their own, made forecasts and left full traces: every search query, every page read, every intermediate thought. The researchers then built a SOTA forecaster from an ensemble of agents plus extra techniques, and it beat every single agent on accuracy.
The most interesting part: they took Tetlock's CHAMPS KNOW framework (10 dimensions of good forecasting, proven in randomized studies with humans) and asked Gemini to score every final answer on those dimensions. It turned out the SOTA forecaster leaned on the pre-mortem (P) 4 times more often than the best single agent, and on wildcards (W) almost 10 times more often. In parallel, human experts manually reviewed the 130 worst Opus 4.6 forecasts and found one recurring pattern: the agent took leaders' rhetoric as a commitment and never asked "why are they saying this right now?"
Adaptations and Extrapolations
🔧 Adaptation: A quick pre-mortem without the full structure
If you don't need the full CHAMPS KNOW, just add P and O to any request:
[Your usual question to the LLM]
After your answer, be sure to add two blocks:
**If I'm wrong, why?**
Assume your conclusion turns out to be wrong.
Which factors are you probably underestimating?
**What am I missing?**
Which perspectives or data did you not consider in the main answer?
This is the minimal working version: two extra paragraphs change the quality of the analysis without rewriting the whole prompt.
🔧 Adaptation: Modeling incentives before a negotiation
Built on the finding that "LLMs confuse rhetoric with intent." Use it before a negotiation, a pitch, or a conversation with a partner or investor:
I'm preparing for a negotiation with {name/role}.
Their public position: {what they are saying}
Answer three questions:
1. What REAL incentives sit behind this position?
What do they actually need?
2. What would be a face-saving way out of their current position?
3. Under what conditions would they back off their stated position,
and what would that look like from the outside?
Resources
Evaluating Strategic Reasoning in Forecasting Agents — Tom Liptay, Dan Schwarz, Rafael Poyiadzi, Jack Wildman, Nikos I. Bosse. FutureSearch (2026). Contact: [email protected]
CHAMPS KNOW Framework — Tetlock & Gardner, "Superforecasting" (2015); Chang et al. (2016), a randomized study of CHAMPS KNOW's effect on the accuracy of human forecasts (+6–12%).
Related work: ForecastBench (Karger et al., 2025); Bench to the Future v1 (Wildman et al., 2025); Autocast (Zou et al., 2022).