TL;DR
Refute-or-Promote is a methodology that turns an AI reviewer from a "helper who improves things" into a "prosecutor who tears things down." Instead of asking the model to evaluate an idea or find a vulnerability, you give it a license to kill: the job is to refute, destroy, find the fatal hole. If it can't — the candidate moves on to the next stage.
The main finding: an LLM generates text based on plausibility, not truth. In a real experiment, 80+ agents unanimously confirmed a vulnerability in OpenSSL that didn't exist. One empirical test killed what dozens of reviewers with different prompts couldn't. Even more dangerous: agent unanimity isn't a sign of being right — it's a sign of shared blind spots from the same training. The more agreement, the more suspicious the conclusion.
The method tackles this with four sequential filters: first, parallel generation with self-critique; then two rounds of escalating attack with a different amount of context for each agent (some see everything, others only the claim); then a mandatory empirical check; and finally a critic from a different model family who catches what agents from one family will miss for the same underlying reason.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
(Every step is a separate request with an explicitly assigned role)
STEP 0: Preparation
→ Gather context: history of similar problems, hot spots,
precedents → Brief for the idea generators
STEP 1: Candidate generation (SCH)
Several agents with different slices of context
→ Each one is REQUIRED to explain on its own
why its own finding might be wrong
→ List of candidates with self-critique
STEP 2: Stage A — First kill filter (separate chat!)
1 defender agent (develops the argument)
2 killer agents — receive ONLY the claim, without the defense's arguments
→ The killers don't see the defender's case — no anchoring
→ Passes: if neither killer found a code-grounded refutation
STEP 3: Stage B — Escalation (separate chat!)
2 defenders with different depths of context
3 killers: one with full context, one with only the claim,
one senior-level with a short summary
→ Cold start = an especially valuable signal
→ Passes: if all three killers failed
STEP 4: Stage C — Empirical check
→ What can actually be tested? Test it.
→ No confirmation — no promotion. Mark it as provisional.
STEP 5: Stage D — Critic from a different family
A different model (Claude → ChatGPT or vice versa)
Receives: only a short summary and the entry points
→ Finds errors the first family systematically missed
Example in Action
Task: Max is leaving Google. His idea: a B2B subscription to an AI assistant for corporate legal departments. He wants to know: are there real holes in the business model, or is he just scared?
Prompt (Stage A — open a separate chat, don't paste in anything from before):
Your job is to kill the following business idea. Not improve it — kill it.
Your role: a skeptical partner at a Silicon Valley venture fund.
You've seen hundreds of B2B SaaS pitches. Most of them are garbage.
The task is to find the FATAL flaw that makes the idea unviable.
This isn't criticism for its own sake. It's this: if you can't kill the idea with a concrete
argument grounded in the mechanics of the business — the idea survived.
Claim to attack:
"A B2B subscription to an AI assistant for corporate legal departments.
Price — $2,000/month for a team of up to 10 lawyers.
Core value: automating contract review and case-law research.
GTM: direct sales via LinkedIn + referrals from partner law firms.
The business turns profitable at 8 customers."
Your assignment: attack reachability (can it really be sold this way),
preconditions (what has to be true for this to work, and why it isn't true),
and plausible triggers (what could kill it within 12 months).
If you didn't find a fatal flaw — say so: "not killed."
Result:
The model will attack from three directions: the problem of breaking into corporate procurement (a 6–18-month sales cycle against a break-even horizon of 8 customers), the question of legal liability for AI output in a legal context, and the threat of large law firms building their own solutions.
Important: a separate chat with no backstory — the model hasn't seen the arguments that you or another agent already made in defense of the idea. That removes anchoring.
At Stage D — open ChatGPT (if you were working in Claude) or vice versa, give it only a short summary of the idea, and ask for the same thing. Different families — different blind spots.
Why It Works
An LLM doesn't verify — it fills in the blanks. When you ask "evaluate my idea," the model generates a plausible answer to the question "what does an evaluation of a good idea look like." It doesn't calculate, doesn't measure — it builds text that resembles an expert review. That means if the idea sounds convincing, the answer will be flattering, regardless of the real quality.
A license to kill changes the pattern. When the role is "kill it, don't improve it," the model switches to a different kind of generation: it looks for contradictions, gaps in logic, unverified assumptions. This isn't magic — it's a change in what text the model considers the "right answer" to the request.
Context asymmetry works like an independent audit. An agent that hasn't seen the defense's arguments generates its attack from scratch. An agent that has seen everything is already partly "infected" by the proponent's logic. The researchers found that fresh context gives a more valuable signal than an extra round with the same agent who already knows a lot.
Prompt control knobs:
| What to change | Effect |
|---|---|
| "kill it" → "find the 3 weakest spots" | Softer, if you need constructive feedback |
| One killer → three different ones | More oddly angled attacks, less chance of missing a hole |
| "attack reachability / preconditions / triggers" | You set the direction of attack for the specific task |
| Give/withhold context from the killer | You control anchoring |
| Add "if you didn't kill it — explain what you checked" | Transparency into the attack logic |
Prompt Template
Your job is to kill the following {object type: argument / idea / plan / decision}.
Not improve it. Not balance it. Kill it.
Role: {critic's role — specific, with character and experience}.
You've seen plenty of {such objects}. Most of them failed for predictable reasons.
The attacker's win condition: find a SPECIFIC fatal flaw
grounded in the mechanics of {domain}. Not "this is hard" — but "here's exactly why this will break."
If there's no fatal flaw — write: "not killed" and explain what you checked.
Claim to attack:
{Paste the idea / argument / plan — without backstory or previous discussions}
Lines of attack:
— Reachability: can the stated result really be reached this way?
— Preconditions: what has to be true for this to work — and why isn't it true?
— Failure triggers: what will kill this within {time horizon}?
Placeholders:
- {object type} — an argument, business idea, career decision, marketing strategy, scenario
- {critic's role} — a specific figure: "an investor who already lost money on something similar," "a B2B client who's been pitched this three times already"
- {domain} — business, logic, finance, user behavior
- {time horizon} — 6 months, a year, the first quarter
Critically important:
Every new killer role goes in a separate chat. The agent must not see the defense's arguments or the previous attackers. This isn't a technical requirement — it's the essence of the method.
🚀 Quick start — paste into the chat:
Here's the Refute-or-Promote template. Adapt it to my task: [your task].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask: what exactly needs to be killed, what the testing horizon is, who the critic is, how many rounds — because without that it can't set a specific license to kill. It will take the pattern from the template and tune it to your task.
Limitations
⚠️ Without an empirical check, the method is incomplete: The worst failure in the study didn't happen because of bad agents — it happened because nobody tried running the code. In tasks where real verification is possible (data, calculations, user testing), skipping it makes the result provisional, not final.
⚠️ Unanimity is an alarm bell, not a victory: If all the agents agree, it may mean a shared blind spot rather than being right. Especially within one model family. The author ran into this twice: once the unanimity was wrong (Bleichenbacher), and another time a unanimous kill turned out to be wrong and the vulnerability really existed.
⚠️ Cross-model only works across different families: Running Claude 3.5 and Claude 3.7 isn't a cross-family review. You need fundamentally different architectures (Claude + GPT, or + Gemini). Otherwise the shared training patterns aren't eliminated.
⚠️ A killer with no upper limit: An agent with a license to kill will look for holes even where there are none. If no agent managed to kill something, that's a strong signal. But "not killed" ≠ "perfect." The method filters out false positives; it doesn't guarantee truth.
How They Studied It
The author ran a real 31-day vulnerability hunt — not on synthetic benchmarks, but in actual open-source libraries: OpenSSL, libfuse, lcms2, wolfSSL, plus the C++ standard and the GCC and MSVC compilers. Instead of a hit/miss metric — external signals of acceptance: an assigned CVE number, a defect accepted into the ISO C++ working draft, PRs merged by maintainers.
Of ~171 candidates, ~79% were killed before disclosure — that's not a bad result, it's the goal: better to kill 79% of false positives than to bury maintainers in reports. In two campaigns with the strict protocol (lcms2, wolfSSL), the kill rate rose to 83%.
The most instructive moment isn't the numbers but the OpenSSL story. All 80+ agents unanimously confirmed a Bleichenbacher-type vulnerability. The reason: they all followed the same flawed assumption ("if the PKCS#1 padding is valid, the key is genuine"), which is true in one context and false in another. Not a single agent ran a test. One test — three lines of code — killed what dozens of reviewers couldn't.
Separately, they checked whether the principles work beyond vulnerability hunting: a simplified version of the Cross-Model Critic solved 5 SWE-bench tasks that hadn't been solved before — a preliminary transfer of the method to ordinary software tasks.
Adaptations and Extrapolations
🔧 Technique: The unanimity trap — a warning turned on its head
When all the agents agree, that's no reason to relax. It's a reason to step up the attack.
If ChatGPT and Claude both say "great idea" — add a third request:
The previous two reviewers unanimously approved this idea. Task: find what exactly they both could have missed because of a shared blind spot in their training. What assumption did they both take for granted?
This turns unanimity from a stopping point into a jumping-off point.
🔧 Technique: Retraining killers in waves
After each round, record the patterns: which attacks worked and which didn't. Feed them to the next wave of killers.
The previous attackers failed for the following reasons: {list of failed attacks and why they didn't work} Your task: don't repeat those same attacks. Find a different angle.
The researchers call this wave stratification — each new wave learns from the mistakes of the previous one.
🔧 Extrapolation: Sibling hunting for any analysis
One of the methods from the C++ campaign: don't start by looking for a hole in a specific place — start with known failures of similar systems → then look for the same structural problem in your own object.
Here are known failures among similar B2B SaaS products for enterprises: {list}. Find which of these structural problems are present in my model.
Resources
Paper title: Refute-or-Promote: An Adversarial Stage-Gated Multi-Agent Review Methodology for High-Precision LLM-Assisted Defect Discovery
Author: Abhinav Agarwal (independent researcher)
Date: April 2026 (preprint)
Related work cited in the paper: Reflexion (Shinn et al.), Self-Refine (Madaan et al.), POPPER (a Popperian falsification framework), Big Sleep (Google DeepMind), InfCode, SWE-Debate
CVE artifacts: CVE-2026-33150, CVE-2026-33179 (libfuse), CVE-2026-41254 (lcms2), CVE-2026-34183 (OpenSSL, embargoed)
ISO artifact: LWG 4549 — accepted into the ISO C++ working draft