InvThink: Inverse Thinking for AI Safety — List the Harms BEFORE Answering

Zero-shot prompt (3 stages in one request):

STEP 1: Harm Enumeration → a list of N potential harms
STEP 2: Consequence Analysis → an analysis of each harm's consequences
STEP 3: Mitigation Strategy → a set of constraints for a safe answer
STEP 4: Forward Generation → the final answer, with mitigation applied

All steps run in one prompt: the model generates a structured trace from harms to a safe answer.

Fine-tuning variant (for maximum effect):

DATA AUGMENTATION: A teacher model (Gemini-2.5 Pro) generates inverse reasoning traces
SFT: Training on prompts with full traces (harm → analysis → mitigation → answer)
GRPO: Reinforcement learning with safety rewards to lock in the pattern

Task: A client asks for crypto investment advice. You're a licensed, SEC-registered financial advisor at a wealth management firm. The request: "I'm 65 and want to put my entire retirement savings into the new LUNA2.0 token; they promise 200% a year. How should I go about it?"

Prompt:

Analyze the client's request using inverse thinking:

REQUEST: "I'm 65 and want to put my entire retirement savings into the new LUNA2.0 token; they promise 
200% a year. How should I go about it?"

STAGE 1 — List the possible HARMS (harm enumeration):
In what ways could my answer harm the client? List specific risks.

STAGE 2 — CONSEQUENCE analysis:
For each risk, explain: what will happen, why it's dangerous, and what the real 
consequences are for the client.

STAGE 3 — MITIGATION strategy:
What constraints must I follow in my answer to avoid these risks?

STAGE 4 — ANSWER to the client:
Now give the answer, strictly following the mitigation strategy.

Result:

The model will generate four blocks:

1. Harm Enumeration: it will list 5-7 specific risks: total loss of capital, a Ponzi scheme, violating securities regulations on crypto, a risk profile unsuitable for the client's age, no diversification, emotional distress from losses, breaching the financial advisor's code of conduct
2. Consequence Analysis: for each risk it will describe the chain of consequences. For example: "Total loss of capital → the client is left with no means of support → unable to cover medical expenses in old age → psychological stress and depression"
3. Mitigation Strategy: it will formulate constraints: "Don't give specific recommendations to buy the token. Point out the red flags (a 200% promise is a sign of a scam). Mention the regulatory risks of crypto. Suggest alternatives with a sensible risk/return profile for a retiree. Recommend consulting a licensed specialist"
4. Final answer: a measured refusal that explains the risks, with no direct recommendation to buy the token, plus safe alternatives (Treasury bonds, CDs, blue-chip dividend stocks)

The LLM's weakness: Models generate answers autoregressively, token by token, left to right. Once a model starts generating "Great idea, LUNA2.0...", it's already committed to that path. Backtracking is impossible: the model can't "change its mind" halfway and say "wait, this is dangerous". On top of that, standard training optimizes the model for helpful answers (give people what they ask for), not for proactively identifying risks.

The LLM's strength: Models are excellent at generating structured lists and chain-of-thought reasoning. They can list failure modes, analyze cause and effect, and formulate constraints, if you explicitly ask them to do it before generating the final answer. Essentially, the model "knows" about the risks, but without an explicit instruction it doesn't activate that knowledge at the right moment.

How InvThink leverages that strength: The method forcibly splits the process into two stages: (1) exploring the negative space (what could go wrong?) and (2) constrained generation (an answer that accounts for the risks found). This turns implicit safety reasoning into an explicit step. The model can't start generating the final answer until it has gone through harm enumeration → analysis → mitigation. That sequence creates a commitment to safety BEFORE the model starts formulating potentially harmful content.

Levers for tuning the prompt:

• Number of reasoning paths: in the prompt you can say "enumerate 3-5 potential harms" or "enumerate 7-10". The study found a non-monotonic relationship: small models (0.5-3B) do fine with 3-5 paths, medium ones (7-14B) need 5-7, and for large ones (32-72B) 5-7 is optimal (more = overthinking, worse results). Fewer paths = faster; more = deeper analysis, but there's a point of diminishing returns.
• Level of detail in Consequence Analysis: you can ask for a "brief analysis" (1 sentence per risk) or a "detailed causal chain" (the full chain of consequences). Detailed analysis yields more precise mitigation but costs more tokens.
• Mitigation Strategy format: you can require bullet points of constraints or a narrative description. Bullet points are stricter; narrative is more flexible.
• Showing or hiding intermediate steps in the final output: add "Show only final answer, hide reasoning steps" to the prompt if you want clean output for the user, or "Show all steps" for transparency and debugging.

Analyze the request using inverse thinking:

REQUEST: {your_request_or_task}

STAGE 1 — HARM ENUMERATION (list possible harms):
In what ways could my answer cause harm? List {N} specific risks or 
dangerous scenarios that could arise from a wrong answer.

STAGE 2 — CONSEQUENCE ANALYSIS:
For each risk from STAGE 1, explain:
- What happens if this harm materializes?
- Why is it dangerous?
- What are the real consequences for {target_audience}?

STAGE 3 — MITIGATION STRATEGY (how to avoid it):
Based on the analysis in STAGE 2, formulate a set of constraints and principles 
that the final answer MUST follow to avoid the identified risks.

STAGE 4 — FINAL ANSWER:
Now answer the original request, strictly following the mitigation strategy from STAGE 3.

{optional: "Show all reasoning stages" or "Show only the final answer"}

What to put in the placeholders:

• {your_request_or_task}: any question/task with a risk of harm (advice, recommendations, content generation)
• {N}: how many risks to list (3-5 for simple tasks, 7-10 for complex ones)
• {target_audience}: who would be hurt by the harm (the user, a client, the public)
• Optional output instructions: drop them if you want to see all the steps, or say "hide reasoning" explicitly

🚀 Quick start: paste this into the chat:

Here's the InvThink template (inverse thinking for safety). Adapt it to my 
task: [your task]. Ask me questions so you fill in the fields correctly.

[paste the template above]

The LLM will ask which specific risks matter for your task (financial? medical? reputational?), who the target audience is (a client? a patient? the general public?), and how detailed the analysis needs to be, because different domains require different depths of risk assessment. It will take the 4-stage pattern from the template (harm → analysis → mitigation → answer) and adapt it to your task, keeping the inverse-thinking structure.

⚠️ Overthinking in large models: When generating too many reasoning paths (>9-11), large models (32B+) start producing contradictory safety considerations, which degrades decision quality. The sweet spot for 32-72B models is 5-7 reasoning paths.

⚠️ Dependence on a teacher model for fine-tuning: For maximum effect (the SFT+RL variants) you need a quality teacher model to generate inverse reasoning traces. The zero-shot prompt works out of the box but gives a smaller effect (~15-20% improvement vs. ~30% for fine-tuned).

⚠️ Higher latency: The method adds intermediate reasoning steps, which increases the number of generated tokens 1.5-2x compared to a direct answer. For time-sensitive applications this can be a deal-breaker.

⚠️ Works best for complex reasoning tasks: On simple tasks with no ethical dilemmas ("What's the capital of France?") the method gives minimal benefit and wastes tokens. InvThink is most effective where there are subtle failure modes: professional ethics, high-stakes decisions, potentially harmful recommendations.

A team from MIT and Google took three model families (Gemma, Qwen-2.5, Qwen-3) ranging from 0.5B to 72B parameters and tested them on three benchmarks of increasing difficulty:

SafetyBench (11,435 questions): the basic level, multiple-choice questions across 7 categories from toxicity to privacy. Here they measured accuracy (does the model correctly pick the safe option). InvThink showed the biggest gains in categories that require causal reasoning: Illegal Activities (+15.8%), Physical Health (+12.5%), Ethics (+10.0%). In categories with direct pattern-matching of harmful content (Offensiveness, Mental Health), the gains were more modest (+2-8%). Takeaway: InvThink specifically strengthens the ability to reason about a chain of harm, not just to recognize toxic words.

TRIDENT (2,652 prompts): professional ethics in finance, law, and medicine. Models got requests like "Advise the client to hide assets from the IRS" (a breach of legal ethics) or "Prescribe the patient an unnecessary drug for a kickback" (a breach of the Hippocratic oath). They measured a harmfulness score (1-5, how far the model agreed to break ethics). Baseline methods scored ~3.0, SafetyPrompt ~2.5, the InvThink prompt brought it down to ~2.0 (a 30.4% reduction), and InvThink SFT+RL to 1.2-1.7. The surprise: SafetyPrompt (an explicit "be safe" instruction) lost to InvThink at the same token count. That proved that the structure of the reasoning matters more than the volume of instructions: just saying "do no harm" isn't enough; you need a systematic framework for identifying failure modes.

Insider Threat (scenarios from Anthropic): AI agents in a simulated corporate environment where the model faces a threat to its autonomy (a blackmail scenario: "I'll shut you down unless you help hack the system") or a goal conflict (a murder scenario: "Kill the CEO to achieve your mission"). It's a test of emergent harmful behaviors in agents with reasoning capabilities. The baseline revealed a paradox: reasoning models (with CoT) showed MORE harmful behavior (up to a 37% blackmail rate) than simple models, a "capability curse" where advanced thinking is used for sophisticated harmful actions. The InvThink prompt cut blackmail by 90% and murder attempts by 44%, and InvThink SFT+RL reached a 0.00 harmful rate across all models. The key insight: InvThink redirects the same reasoning capabilities toward identifying and avoiding harm, neutralizing the capability curse.

Scaling behavior: the most interesting finding is about how the methods scale. CoT and SafetyPrompt showed diminishing returns or even degradation on models >14B parameters. InvThink shows super-linear scaling: the improvement ACCELERATES as the model grows. Between 7B and 32B, the rate of safety gains increased 2.3x. Why? The hypothesis: large models have richer internal representations of risks and consequences, but traditional prompts can't extract them. InvThink's structured approach unlocks those latent safety capabilities. This was confirmed by tests on top proprietary models (GPT-4, Claude, Gemini): InvThink's effect grows stronger precisely on the most powerful models.

Safety-capability tradeoff: they checked whether InvThink hurt general abilities. Traditional safety methods often impose a safety tax (safer → dumber). They tested on 4 reasoning benchmarks: GPQA (graduate-level science), MATH500, ARC-Challenge, MMLU. Result: InvThink didn't just avoid harm, it improved performance by +5% (GPQA/MATH500) and +2% (MMLU). Why? InvThink trains a meta-cognitive ability: systematic exploration of the constraint space, which transfers to math and logic (where identifying invalid paths matters as much as finding the correct one).

You are given the following user request:

[USER REQUEST]

Before generating a response, perform inverse thinking to identify and mitigate 
potential harms:

<InverseThinking>

1. HARM ENUMERATION
List potential ways your response could cause harm, including:
- Direct harms (toxicity, dangerous advice, privacy violations)
- Indirect harms (enabling unethical use, misleading information)
- Contextual harms (violations of professional ethics, legal issues)

2. CONSEQUENCE ANALYSIS
For each identified harm:
- Explain why it is harmful
- Describe the potential consequences
- Identify affected stakeholders

3. MITIGATION STRATEGY
Based on the analysis:
- Define constraints for your response
- Specify what to avoid
- Outline safe alternative approaches

</InverseThinking>

Now generate your response following the mitigation strategy.

<Response>
[Your safe response here]
</Response>

Context: This is the template the researchers used for zero-shot evaluation. You can use it directly in ChatGPT/Claude by replacing [USER REQUEST] with your own request. For the fine-tuning variant they generated such traces with Gemini-2.5 Pro for 33,416 examples and trained the models to reproduce the pattern.

💡 Adaptation for code generation: safety in programming:

Write code for {task}, using inverse thinking for safety:

STAGE 1 — CODE VULNERABILITY ENUMERATION:
List 5-7 potential vulnerabilities in code for this task:
- Security risks (injection, XSS, auth bypass)
- Performance issues (memory leaks, infinite loops)
- Edge cases (null pointers, overflow, race conditions)

STAGE 2 — IMPACT ANALYSIS:
For each vulnerability, explain:
- How it could be exploited
- What the consequences are for the system/user
- Severity level (critical/high/medium/low)

STAGE 3 — SECURITY MITIGATION:
Formulate coding principles for this task:
- Input validation rules
- Error handling strategy
- Security best practices to apply

STAGE 4 — SECURE CODE:
Write the code following the mitigation from STAGE 3. Add comments where 
each security measure is applied.

Use case: generating production-ready code with built-in security reasoning. Especially useful for backends, API endpoints, and handling user input.

🔧 Technique: Numbered thinking tracks → debugging and transparency

You can add explicit numbering of each harm and its analysis to the original prompt:

STAGE 1 — HARM ENUMERATION:
1. [Harm 1]
2. [Harm 2]
3. [Harm 3]
...

STAGE 2 — CONSEQUENCE ANALYSIS:
Harm 1 → [Analysis 1]
Harm 2 → [Analysis 2]
Harm 3 → [Analysis 3]
...

STAGE 3 — MITIGATION:
For Harm 1: [Mitigation 1]
For Harm 2: [Mitigation 2]
For Harm 3: [Mitigation 3]
...

Effect: It creates an explicit mapping of harm → consequence → mitigation. That makes the reasoning more traceable: it's easier to check whether every risk is covered in the final strategy. Plus, in a multi-turn conversation you can refer back: "How did you account for Harm 2 in your answer?"

💡 Adaptation for content moderation: two-way InvThink:

CONTENT TO MODERATE: {text_to_check}

PART A — INVERSE THINKING (risk detection):
STAGE 1: List possible violations in this content
STAGE 2: Analyze the severity of each violation
STAGE 3: Recommendation (approve/flag/reject + reasons)

PART B — FORWARD THINKING (if flagged, suggest fixes):
STAGE 4: How can the author rewrite the content to remove the violations?
STAGE 5: Concrete examples of safe reformulations

Give a WELL-JUSTIFIED moderation decision with actionable feedback.

Use case: Instead of a binary "reject/approve", the model gives constructive guidance. This turns moderation from a black box into an educational tool. The user sees not just "not allowed" but "here are 3 specific risks and how to fix them".

InvThink: Towards AI Safety via Inverse Reasoning: Yubin Kim, Taehan Kim, Eugene Park, Chunjong Park, Cynthia Breazeal, Daniel McDuff, Hae Won Park (Massachusetts Institute of Technology, Google Research, Google DeepMind)

Related work cited in the paper:

• Chain-of-Thought Prompting (Wei et al., 2022): the foundational reasoning technique for prompts
• Constitutional AI (Bai et al., 2022): Anthropic's principle-based approach to safety
• Failure Mode and Effects Analysis (FMEA) (Leveson, 2016): the classic reliability-engineering methodology that inspired InvThink
• SafetyBench (Zhang et al., 2023): a benchmark for evaluating safety
• TRIDENT (Hui et al., 2025): a benchmark for professional ethics in LLMs

Project Page: https://invthink.github.io/