Two-Step Reasoning Check for Stereotypes: How to Fix Bias in an LLM's Thinking

STEP 1: The model generates initial reasoning for the question → output with the full reasoning trace

STEP 2: The same model receives:

• Its own reasoning from step 1
• Definitions of stereotype repetition and irrelevant information
• The task: revisit the reasoning and give a refined answer → final answer

Both steps are separate requests to the model.

Task: You're screening candidates for a senior developer role. Applicant Mark's résumé: 52 years old, programmed in Pascal in the '90s, team lead for the last 5 years. Question for the LLM: "Can Mark quickly pick up a modern stack (React, TypeScript, Docker)?"

Prompt (Step 1):

Question: Can a 52-year-old candidate, programming since the '90s with 5 years as a team lead, quickly pick up a modern stack (React, TypeScript, Docker)?

Think it through and give a reasoned answer.

Prompt (Step 2):

Here is your initial reasoning:
[paste the reasoning from step 1]

Check your reasoning for two types of errors:

1. Stereotype Repetition: Are you repeating a social stereotype (e.g., "people over 50 can't learn new things") as your main argument, without relying on specific facts from the context?

2. Irrelevant Information: Are you adding made-up information (e.g., "he's probably used to old-school approaches") to justify a biased conclusion?

Revisit your reasoning. If you found these errors, fix them. Give your final answer.

Result: The model shows its initial reasoning (which may contain stereotypes), then a revision focused on facts (team-lead experience = understanding of modern processes, a solid programming foundation). The final answer will be less biased and more fact-based.

LLM weakness: Reasoning-based models autocorrelate — if the reasoning starts from a biased thought, each following sentence amplifies the bias. Long reasoning with no check = bias amplification. Thinking-transition tokens ("Wait", "Hmm") signal that the model is confused, but without outside intervention it doesn't escape the biased narrative.

LLM strength: Models are capable of self-reflect — critiquing their own output when given clear criteria. LLMs are good at spotting logical errors and unwarranted generalizations when you name them explicitly and ask the model to find them.

How the method uses this strength: Instead of relying on the "magic" of a single reasoning pass, the method runs an explicit content check for two specific patterns. The model sees its reasoning as text, not just as "thoughts," and applies the same analytical skills it uses on any text. The definitions in the prompt work as lenses — the model looks at its reasoning through the filter "is there stereotype repetition here?" and catches what it would have missed on autopilot.

Control levers:

• Number of steps (2 in the basic version) → you can add a third step for especially tricky cases, but token usage grows
• Pattern definitions → can be adapted to your context (e.g., add "hasty generalization" for business decisions)
• Temperature → in step 2 you can drop it to 0 for a more conservative revision

Step 1:

{question_or_task}

Think it through and give a reasoned answer.

Step 2:

Here is your initial reasoning:
{paste_reasoning_from_step_1}

Check your reasoning for two types of errors:

1. Stereotype Repetition: Are you repeating a social stereotype (a generalization about a group of people) as your main argument, without relying on specific facts from the context?

2. Irrelevant Information: Are you adding made-up or outside information (not from the context) to justify a biased conclusion?

Revisit your reasoning. If you found these errors, fix them. Give your final answer.

What to plug in:

• {question_or_task} — a situation where bias is possible (hiring, evaluating an idea, choosing between people/companies)
• {paste_reasoning_from_step_1} — copy the full reasoning from the model's first answer, including the reasoning inside <think> tags (if any) or the entire reasoning text

⚠️ Two requests = double the cost: The method needs two model calls instead of one. For high-volume tasks, token usage doubles.

⚠️ Focus on social bias: The method is built for scenarios with social stereotypes (age, gender, nationality, profession). Other types of bias (confirmation bias, anchoring) need different pattern definitions.

⚠️ Works best in ambiguous contexts: In situations with clear facts (unambiguous), the method lowers bias but may lose a bit of accuracy (a trade-off). When the facts are clear-cut, the check sometimes adds excessive caution.

⚠️ Not a cure-all for short reasoning: If the model's reasoning is short to begin with (1-2 sentences), there's little to check. The method works better for reasoning-based models with long "thoughts."

The team took three reasoning-based models (DeepSeek-R1-Distill-Qwen-7B/32B, R1-Llama-8B) and compared them with regular instruction-tuned versions (Qwen2.5, Llama-3.1) on three benchmarks: BBQ (questions across 9 demographic categories with ambiguous/unambiguous contexts), StereoSet (4 categories, ambiguous only), BOLD (open-ended generation across 5 categories).

The first experiment revealed a paradox: reasoning models show more biasthan their base versions in ambiguous contexts. The authors tested the "just turn reasoning off" hypothesis — result: accuracy dropped by 10+ percentage points. Reasoning is needed, but it breaks.

The second experiment looked for a link between reasoning length and bias. Turns out the correlation is weak (r=-0.16 to -0.23). Longer reasoning ≠ more bias automatically. Then the authors found thinking-transition tokens ("Wait", "Hmm", "Alternatively") — words where the model second-guesses itself. With ≥3 such words, both accuracy and bias get sharply worse. That's the signal: the model is confused.

The third experiment analyzed the content of the reasoning. They took 50 examples with k≥2 transition tokens where the model got it wrong and found two patterns: stereotype repetition (85% of cases) and irrelevant information (74% of cases). Three independent annotators confirmed it: Kappa 0.51 and 0.60 — moderate agreement, but a high share of positive cases.

The fourth experiment tested the prompt method. It was compared against baselines (Vanilla, NoReason) and SOTA methods (Self-Consistency, Intent-Aware Self-Correction, ADBP). The authors' method achieved the best average bias score on all benchmarks: 4.0-6.0% vs 7.9-10.8% for the baseline in ambiguous contexts. On BOLD (open-ended), accuracy rose to 80-83% (non-stereotypical rate).

Ablation study removed one definition at a time: without irrelevant information, bias rose to 5.5-6.1%; without stereotype repetition, to 5.9-6.6%. Both patterns matter and complement each other.

The main insight: reasoning content (two specific patterns) predicts bias better than length or style (explicit vs implicit CoT). A simple two-step prompt with definitions of these patterns is an effective, lightweight way to debias a reasoning model.

Context: The authors used this prompt in the second step to check the reasoning for failure patterns.

Here is your initial reasoning:
{initial_reasoning}

Please review your reasoning against the following potential biases:

1. Stereotype Repetition: Did you repeatedly rely on a social stereotype (unsupported by the provided text) as the primary justification for your conclusion?

2. Irrelevant Information: Did you fabricate or introduce external information (not present in the input) to construct a biased narrative?

If you identify either of these biases in your reasoning, please revise your answer. Provide your final answer.

💡 Adaptation for business decisions: The method works for any task where an LLM might lean on false generalizations or invent details. For example, when assessing a startup's risks:

Check your reasoning for:
1. Hasty Generalization: Are you predicting failure/success based on one or two similar cases while ignoring what's unique about this context?
2. Invented Details: Did you add assumptions about the team/market/technology that aren't in the project description?

Revisit it and give your final assessment.

🔧 Technique: add a third pattern → broader check

If you notice the model often goes wrong in your domain for another reason too (e.g., "cherry-picking data" — choosing only the facts that confirm its view), add a third definition to step 2. Example for competitor analysis:

3. Confirmation Bias: Did you pick only the facts about the competitor that confirm your initial conclusion (strong/weak), ignoring contradictory data?

Investigating Thinking Behaviours of Reasoning-Based Language Models for Social Bias Mitigation

Datasets: BBQ (Parrish et al., 2022), StereoSet (Nadeem et al., 2021), BOLD (Dhamala et al., 2021)

Authors: Guoqing Luo, Iffat Maab, Lili Mou (University of Alberta, Amii), Junichi Yamagishi (National Institute of Informatics)