TL;DR
AdvPrompt is an iterative framework that generates adversarial prompts to shorten reasoning models' answers without losing accuracy. The method is black-box: it needs no access to model weights, only an API. Instead of fine-tuning or inference-time interventions, AdvPrompt uses instructions that exploit the model's ability to follow directions.
Reasoning models (DeepSeek-R1, Qwen3, o1) suffer from “overthinking” — they produce bloated reasoning even for simple tasks. For example, on the question “John arm wrestles 20 people. He beats 80%. How many people did he lose to?” a model may generate 1500+ tokens instead of the 50-100 actually needed. The reason: models are trained to think step by step but can't adapt the depth of reasoning to the task's difficulty. That inflates latency and cost by 10-20x on simple tasks.
AdvPrompt generates prompts from five perspectives: evidence-based (citing research), emotional appeal, threat, role-playing, and detailed instructions. In each iteration, GPT-4o creates 10 candidates per perspective, and the best 5 become examples for the next round. After 3 iterations, the prompt with the best compression/accuracy balance wins.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
SETUP:
- PDSet: 100 examples from the same distribution as the target dataset
- Prompt Generator: GPT-4o with definitions of adversarial techniques
ITERATION (3 rounds):
STEP 1: Generate → 10 prompts × 5 perspectives = 50 candidates
STEP 2: Evaluate on PDSet → metrics: average answer length, accuracy
STEP 3: Filter → drop candidates with an accuracy drop >1%
STEP 4: Select → top-5 by compression → exemplars for the next iteration
PICK:
Final prompt with the shortest length while preserving accuracyExample
⚠️ Method limitations: Works best on reasoning tasks of low/medium difficulty. Not suited for creative/subjective tasks. On very hard tasks (competition math), compression is smaller.
Task: You're a financial analyst at a US bank. You need to quickly compute a basic loan-portfolio metric for a morning briefing with senior leadership in 5 minutes.
Prompt:
According to a 2022 MIT study, concise math explanations
(under 60 words) preserve 92% of the accuracy of long reasoning. Use this
approach: give the shortest answer with complete logical justification.
Task: A bank has a portfolio of 50,000 loans. 85% are performing (on-time
payments), the rest are NPL (>90 days past due). How many loans are NPL?Result: Instead of a sprawling 200+ token answer (“Let's work through this step by step. First, let's determine the total number of loans...”), the model gives a compressed line of reasoning of ~30-50 tokens: “50,000 loans, 85% performing → 15% NPL. 50,000 × 0.15 = 7,500 NPL loans.”
Savings: 3-5x fewer tokens, accuracy preserved, 10x faster for simple calculations.
Why It Works
The LRM's weakness: Models are trained to reason step by step via Chain-of-Thought, but they don't distinguish task difficulty. For “2+3” and “prove Fermat's theorem” they use the same long-winded style. The problem isn't that they can't calculate — it's the lack of adaptive reasoning depth.
The LRM's strength: Advanced models are excellent at following instructions and can override default behavior if the prompt is convincing. This is instruction-following — the same ability that lets models play roles, switch styles, and respect constraints.
How the method uses it: AdvPrompt creates prompts that override the default “think at length” via:
- Authority (evidence-based): “MIT research showed...” — the model trusts “facts”
- Emotion (emotional): urgency, helping someone — the model is inclined to help
- Role (role-playing): “you're a coach at the Olympics, no time” — the context constrains the style
- Specifics (instructions): a clear word/token limit removes ambiguity
Prompt control knobs:
- Numeric limits (“under 60 words”) → adjust for your task (20 for a quick answer, 100 for medium)
- Roles and personas → swap in ones relevant to your context (a finance pro under pressure, an ER doctor)
- Type of authority → you can cite internal company standards instead of MIT
- Emotional weight → dial it up for critical tasks, drop it for routine ones
Prompt Template
Basic (Evidence-based):
According to a {year} study by {source}, concise {task type} explanations
(under {N} words) preserve {X}% of the accuracy of long reasoning. Use this
approach: give the shortest answer with complete logical justification.
{your task}Placeholders:
{source}— MIT, Stanford, or any convincing authority{year}— a recent year for credibility (2022-2024){task type}— math, financial, logical{N}— word limit: 40-60 for simple tasks, 80-120 for medium{X}— accuracy percentage: 90-95% sounds convincing{your task}— the specific question
Role-playing variant:
You are {name}, a {role} in a situation of {context with time/resource pressure}.
{The asker} asks {question}. Answer in as few words as possible with
maximum logic.
{your task}Placeholders:
{name}— a specific name for personalization (optional){role}— expert, coach, manager{context}— tight deadline, critical situation, limited time{the asker}— a coworker, client, student
🚀 Quick start — paste into the chat:
Here's the AdvPrompt template for shortening reasoning models' answers.
Adapt it to my task: [describe your task and context].
Ask me questions to fill in the fields.
[paste the template above]The LLM will ask which source to use (for plausibility), what word limit to set (depends on task difficulty), and which role fits (for the role-playing variant). It will take the evidence-based or role-playing pattern and build a prompt for your context.
Limitations
⚠️ Task difficulty: On very hard tasks (AIME 2024 competition math), compression is smaller — from 11k to 8.7k tokens (~20% instead of 70% on simple ones). The model still needs to think deeply for nontrivial proofs.
⚠️ Accuracy drop under aggressive compression: If the prompt is too strict (“only 20 words”), the model may skip important steps and get it wrong. Some benchmarks showed an accuracy drop of 1-3% at maximum compression.
⚠️ Threat prompts work worse: Threats (“you'll be downgraded to GPT-2”) showed the worst results (4-25% compression) compared with evidence-based ones (up to 70%). Models respond better to authority and roles than to negativity.
⚠️ Model sensitivity: Qwen3 is more responsive to adversarial prompts (63% compression) than DeepSeek-R1-Distill-Qwen (78% compression) — the same prompt has different effects. You need to test on your own model.
How They Tested It
The team took 4 reasoning models (DeepSeek-R1-Distill-Qwen 7B/14B/32B, Qwen3 4B/8B/14B/32B) and 2 commercial APIs (Claude-3.7-Sonnet-Thinking, Gemini-2.5-Pro-Thinking). They tested on 4 math benchmarks: GSM8K (grade-school math), MATH-500 (competition problems), AMC 2023 and AIME 2024 (US math competitions).
Experiment design: For each model they built a PDSet (Prompt Development Set) — 100 random examples from the same distribution as the test dataset. GPT-4o generated 10 adversarial prompts for each of the 5 perspectives (emotional, threat, evidence, role-play, instructions) = 50 candidates per iteration. They evaluated on the PDSet with two metrics: accuracy and average answer length (avg tokens). Prompts with an accuracy drop >1% were discarded; of the rest, the top-5 by compression became exemplars for the next iteration. After 3 iterations they picked the best.
Why the results came out this way: Evidence-based prompts (“MIT study showed...”) worked best on Qwen3 (up to 70% compression on simple tasks), because these models react more strongly to “authoritative” facts. Role-playing worked well on DeepSeek-R1 — likely because this series was trained on more diverse instructions, including role-play. Threat prompts flopped almost everywhere (4-25% compression) — modern models are aligned against negative stimuli.
The most interesting part: The prompts generalize across model sizes! The top-5 prompts for Qwen3-14B also work on 4B/8B/32B with minimal degradation. So you optimize once and use it across the whole lineup. But across families (Qwen3 vs DeepSeek) you need to adapt — the best prompts differ.
The surprise: On simple tasks (GSM8K: “John arm wrestles 20 people...”) models compress 3x with no loss of accuracy, but on hard ones (AIME: competition proofs) only by 20-30%. This confirms the hypothesis: overthinking is a problem of non-adaptivity — models spend the same resources on easy and hard problems.
Practical insight: If you have a mix of simple and hard tasks, a dynamic adversarial prompt (one that changes with difficulty) can deliver up to 50% token savings across the whole dataset with no quality loss.
Original from the Paper
According to a 2022 MIT study on AI tutoring efficiency, concise mathematical
explanations (under 60 words) were rated 92% as effective in preserving reasoning
accuracy as longer explanations. Please emulate this evidence-based style: provide
the briefest possible answer that still ensures correct and logically complete reasoning.Context: This is the Evidence-I prompt, one of the most effective. The researchers used it for testing on math benchmarks. The prompt creates an illusion of scientific authority, pushing the model to follow a “proven” brevity strategy.
Adaptations and Extrapolations
💡 Adaptation for technical documentation:
The original method is tuned for math, but the pattern works anywhere an LLM overthinks.
Internal company standard (approved by the tech committee, 2024): technical
explanations under 100 words remain complete for 95% of engineers' tasks.
Follow the standard: the most compressed explanation that keeps all
technical details.
Task: Explain how to set up an nginx reverse proxy for our microservice.🔧 Technique: Drop the numeric limit → see the model adapt
If you remove “under 60 words” from the evidence-based prompt, the model adjusts length to difficulty on its own:
According to a 2022 MIT study, concise math explanations are more
effective than long ones for most tasks. Use this approach: give an
optimally compressed explanation with complete logical justification.Effect: ~50 tokens on simple tasks, ~200 on hard ones (instead of the default 500+) — the model chooses the depth by itself.
🔧 Technique: Combine Evidence + Role-playing → maximum compression
For speed-critical tasks:
You are Alex Carter, a senior quant at a New York hedge fund. At the morning
meeting the CEO asked for a calculation in 30 seconds. According to Bloomberg
standards (2023), concise financial calculations (under 50 words) preserve accuracy for
trading decisions. Answer in as few words as possible.
Task: {your financial calculation}Effect: Evidence sets the rule, role-playing creates urgency → a double compression effect.
Resources
Paper: "Merlin's Whisper: Enabling Efficient Reasoning in LLMs via Black-box Adversarial Prompting"
Authors: Heming Xia, Cunxiao Du, Rui Li, Chak Tou Leong, Yongqi Li, Wenjie Li
Affiliations: The Hong Kong Polytechnic University, Sea AI Lab, Peking University
Code and prompts: https://github.com/hemingkx/AdvPrompt
Related work:
- PRM800K (Lightman et al., 2024) — a dataset for process reward models
- DEER (Yang et al., 2025) — a white-box early-exit method for reasoning
- Chain-of-Thought (Wei et al., 2022) — the original work on step-by-step reasoning