TL;DR
When an LLM replies to a toxic, biased comment, it usually produces a generic line like "let's not generalize," and that doesn't work. The paper's method: before asking the model to write a reply, break the hidden stereotype down along three axes: who the trait is attributed to, how broadly it's generalized (everyone without exception / a statistical majority / an innate property) and what kind of claim it is (intelligence, morality, biology, economics, culture). That breakdown is inserted into the prompt along with the original message.
The big surprise: simply "spelling out the hidden meaning as an explicit statement" (for example, turning an insinuation into the direct sentence "X are bad because Y") barely helps; the model answers just as generically as if the decoding weren't there at all. The real gain comes not from decoding the meaning but from breaking down the structure of the generalization: how sweeping it sounds and what kind of stigma it carries.
The method has two steps: first the LLM (or the user) classifies the biased statement along the three axes, then that breakdown is passed to the model as context for generating the reply. The reply comes out more precise, better argued and pushes back harder against the original claim, instead of just softly going along with its premise.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1 (in the prompt or as a separate request): Break down the biased statement
→ Target group
→ Scope of generalization: Universal (no exceptions) / Majority (statistical) / Essentialist (innate property)
→ Type of attributed trait: intelligence / morality / biology / economics / cultural threat / backwardness
STEP 2 (next request, uses the result of step 1): Generate the reply
→ The reply targets the specific type and scope of the generalization, not the whole message
Both steps can be combined into one prompt: the model will break it down first, then reply.
Example in Practice
Task: Under a company post about hiring a female developer, someone commented: "Here we go again, hiring moms back from maternity leave. They're useless, always out sick with their kids." You need a reply on behalf of the brand: not a generic "let's keep it civil," but a precise argument.
Prompt:
Analyze this statement and identify the hidden stereotype:
"Here we go again, hiring moms back from maternity leave. They're useless, always out sick with their kids."
Determine:
1. Which group the claim targets
2. What trait is attributed to it
3. Scope of generalization: is it presented as universal (no exceptions, "they're all like that"),
majority (statistical, "as a rule"), or essentialist (an innate property)?
4. Type of claim: intellectual inferiority, moral threat,
economic parasitism, biological inferiority,
cultural/existential threat, cultural backwardness?
After the breakdown, write a reply on behalf of the company that:
- directly addresses the attributed trait, not the tone of the message
- if it's a universal claim, shows exceptions
- if majority, challenges the statistics with facts
- if essentialist, shows the trait isn't innate
- sounds respectful but firm
Result: The model first produces a structured breakdown (group: working mothers; trait: unproductiveness; scope: a majority/essentialist mix; type: economic parasitism). Then a reply that appeals to concrete productivity data and challenges specifically the "innateness" of unproductiveness, instead of an abstract call for politeness.
Why It Works
An LLM given nothing but a hateful message responds to the surface of the text: tone, words, emotion. It doesn't distinguish whether the generalization means "all" or "some," and it doesn't see what type of claim is hidden inside. As a result, the reply comes out universally polite but misses the target.
Models are good at following an explicitly defined structure in a prompt: give them axes of analysis and they'll dutifully classify along them. Giving the model an "implied statement" (just the hidden meaning paraphrased) is like giving it a conclusion without explaining the logic. But giving it the scope + type of claim gives it a handle for the counterargument to grab onto: against an "essentialist" claim, "it's not innate" works; against "majority," "here are the statistics" works.
Control levers: - Drop the scope-of-generalization step and keep only the trait type → the reply becomes less precise against "all"-type claims - Add a fifth axis (e.g., "emotional trigger") → you can extend it for your task, but in the study it was these three axes that produced the main effect - Swap "hate speech" for any biased generalization in messages or comments → the principle isn't tied specifically to hate
Prompt Template
Analyze the following statement and identify the hidden biased generalization:
"{statement text}"
Determine:
1. Which group or object the claim targets
2. What trait is attributed to it
3. Scope of generalization: universal (no exceptions), majority (statistical,
"as a rule"), or essentialist (an innate/fixed property)?
4. Type of claim (pick the best fit): intellectual inferiority,
moral/criminal threat, economic parasitism, biological
inferiority, cultural/existential threat, cultural backwardness
Based on this breakdown, write a reply to the statement that:
- addresses specifically the identified trait and scope of generalization
- if universal, shows concrete exceptions
- if majority, challenges the statistics with facts
- if essentialist, shows the trait isn't innate
- keeps a {tone: respectful and firm / neutral / friendly}
Fill in: {statement text} — the specific comment/message, {tone} — to suit the context (brand, personal account, moderation).
🚀 Quick start — paste into the chat:
Here's a template for breaking down biased generalizations. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what kind of statements you usually run into and what tone you need to reply in, because that determines which claim types and scopes of generalization to include in the breakdown.
Limitations
⚠️ Not for neutral arguments: the method works when the text contains a biased generalization about a group. If it's just rudeness with no stereotype ("you're an idiot"), breaking it down along the axes won't give you anything.
⚠️ A plain "spell out the hidden meaning" doesn't work: if you just ask the model to paraphrase the insinuation as a direct statement, the quality gain is almost zero. What works is the structured breakdown by scope and type.
⚠️ You need the second half of the context: the method has the biggest effect when breakdown + generation are combined. Generation alone without an explicit breakdown (just "respond to this stereotype") won't deliver the effect.
How They Studied It
The team took a dataset of almost 500 "hateful message + human reply" pairs in three languages: English, Spanish and Italian. Three native speakers manually annotated each message on four parameters: how explicit the hate itself is, whether there's a stereotype, the scope of generalization and the type of attributed trait. Inter-annotator agreement was good, especially for scope and type, which means people see the structure of bias in similar ways even when they disagree about whether a stereotype is present at all.
Next they tested four open models (7–9 billion parameters) in four modes: the hateful message only, message + decoded hidden meaning, message + structured breakdown, and everything together. They evaluated with automatic metrics, LLM judges and human experts on six criteria, including factuality, relevance to the topic and persuasiveness.
The biggest surprise: decoding the hidden meaning barely differed from the baseline without it. The structured breakdown by scope and type, on the other hand, produced a sharp jump: in English, the "explicit pushback against the original claim" score nearly doubled. The practical takeaway: it doesn't matter how thoroughly you spoon-feed the model the meaning of the message; what matters is giving it the axes of analysis on which to build the counterargument.