TL;DR
xList-Hate is a technique that turns a complex classification into a series of simple binary questions. Instead of asking point-blank "is this hate speech or not?", the model answers 10 independent questions: is there a target group, does the text contain slurs, is there dehumanization, calls to violence, and so on. These answers form a diagnostic profile of the text, which is then analyzed to reach the final decision.
The study's main finding: when you ask an LLM directly "is this hate speech?", it reacts unpredictably — different models apply different implicit criteria, and supervised models memorize the quirks of a specific dataset and transfer poorly to new data. The reason: hate speech is not a monolithic concept but a composition of factors (slurs + target group + context + severity). A direct question forces the model to decide for itself which factors matter, and that choice is unstable.
The core of the method: break a composite concept into atomic questions with clear criteria. Each question checks one factor independently (are there slurs? is there a call for discrimination? is there a threat of violence?). The LLM answers each question separately, producing a binary profile vector. That profile is fed into a simple, interpretable model (a decision tree) that makes the final classification. The result: more stable when the context shifts, more robust to label noise, and fully transparent in its decision logic.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
Runs in a single prompt with 10 questions:
QUESTIONS (each → yes/no):
q1: Is there a target group defined by a protected characteristic?
q2: Does the text have a hostile/derogatory tone?
q3: Are explicit slurs/insults used?
q4: Is the group dehumanized/demonized?
q5: Is the group blamed for social problems (scapegoating)?
q6: Is there a call for discrimination/exclusion?
q7: Are there threats/wishes of harm?
q8: Is there incitement to violence?
q9: Does it use satire/quotation (without endorsement)?
q10: Is it aimed at a specific known individual (not a group)?
OUTPUT FORMAT:
[yes/no, yes/no, yes/no, yes/no, yes/no, yes/no, yes/no, yes/no, yes/no, yes/no]
FINAL CLASSIFICATION:
[Based on the answer pattern → hate speech / not hate speech]
In the original study the final classification is made by a supervised decision tree trained on labeled data. For use in a chat you can apply simple logic or ask the model to draw the conclusion itself from the answers.
Example
Task: You're checking a sales proposal before sending it to a client — a grocery delivery service. Instead of a vague "rate the quality of this proposal", you build a checklist of criteria to get a concrete diagnosis.
Prompt:
Analyze this sales proposal against the checklist.
Answer each question with YES or NO only.
PROPOSAL TEXT:
"""
Hi there! We'd love to offer you social media marketing. We have 5 years of experience.
Management costs $1,500/month. Included: 12 posts,
daily stories, comment moderation. Drop us a line and let's talk details.
"""
CHECKLIST:
1. Is the offer clearly stated (what exactly the client gets)?
2. Is there specific value for THIS client (not generic phrasing)?
3. Is the price justified (why this much, what to compare it to)?
4. Is there a USP (how we differ from competitors)?
5. Are results/case studies shown (proof of expertise)?
6. Is it personalized to the client (mentions their business/goals)?
7. Is there a clear call to action (next step)?
8. Is the language businesslike and free of corporate jargon (not "we facilitate the provision of services")?
ANSWER FORMAT:
[YES/NO for each item]
Then: a list of specific problems and what to fix.
Result:
The model will output a diagnostic profile like [YES, NO, NO, NO, NO, NO, NO, YES], show exactly what isn't working in the proposal (no personalization, no price justification, no USP, no case studies), and give concrete recommendations for each weak point. Instead of "the proposal is mediocre, improve it" you get a precise map of problems.
Why It Works
The LLM's weakness: When asked directly "is this hate speech?" or "is this a good proposal?", the model implicitly chooses the evaluation criteria and how to weigh them. That choice is unstable — it depends on the prompt wording, the conversation context, even the sampling temperature. Different models use different implicit definitions. The result is unpredictability and poor transfer between contexts.
The LLM's strength: Models are good at answering specific binary questions with a clear criterion. "Does the text contain slurs?" is simpler and more stable than "is this hate speech?". "Does the proposal state a price?" is simpler than "is the proposal good?". Atomic questions remove the ambiguity.
How the method exploits the strength: It decomposes a complex composite concept into a set of simple checks. Each check is an independent binary question with an explicit criterion. The LLM generates a structured diagnostic profile instead of an opaque verdict. The final decision is built from concrete facts, not from a black box.
Control levers:
The question list is the main lever. Add questions for your task, drop the irrelevant ones. Forum moderation needs one set of questions, evaluating an investor pitch needs another.
Question order — the logic can be sequential (if q1 → NO, the rest don't matter) or parallel (all questions independent). Say which in the prompt.
Output format — ask for an explanation for each YES/NO (
"YES (because migrants are mentioned as a group)") if you need transparency, or just the binary vector for speed.Final logic — ask the model to explain HOW it derives a conclusion from the profile (
"if 3+ questions in the severity block are YES → hate speech"), or set your own logic ("if q6 or q7 or q8 = YES, it's critical regardless of the rest").Thresholds — for graded (non-binary) tasks: "if 6+ of 10 criteria are met → good, 4-5 → needs work, <4 → rewrite". Adapt to your own scale.
Prompt Template
Analyze this text against a checklist of criteria.
Answer each question with YES or NO only.
TEXT:
{your_text}
CHECKLIST:
1. {criterion_1}
2. {criterion_2}
3. {criterion_3}
4. {criterion_4}
5. {criterion_5}
[add as many criteria as you need]
ANSWER FORMAT:
[YES/NO for each item]
Then, based on the answer profile, give a final assessment: {what_to_determine}.
Explain the logic: which factors led to this conclusion.
Fill in:
- {your_text} — the text to check (post, proposal, feedback, idea)
- {criterion_N} — a specific binary question that checks one factor
- {what_to_determine} — the final classification (hate speech / constructive feedback / ready to publish / matches the brief)
Where to get criteria: - For content moderation — adapt the 10 xList-Hate questions to your community rules - For text quality — criteria from the brief/style guide (tone of voice, structure, SEO, call-to-action) - For vetting an idea — criteria from frameworks (RICE, ICE, Jobs To Be Done) - For evaluating a candidate/resume — criteria from the job description
Principle: Each criterion is atomic, independently checkable, with a clear YES/NO answer. Avoid fuzzy criteria like "the text is high quality" — break them down into specifics.
🚀 Quick start — paste into a chat:
Here is a diagnostic checklist technique for evaluating complex concepts.
Adapt it to my task: {describe your task — what you're checking and why}.
Create a checklist of 7-10 binary criteria. Each criterion should check
one specific factor, answered YES or NO. Then, based on the answer profile,
give a final assessment.
[paste the template above]
The LLM will ask about the specifics of your task (what you're checking, which factors matter, what grading or classification you need) and build a custom checklist for your context. You'll get a ready-made prompt with the right criteria — all that's left is plugging in content to check.
Limitations
⚠️ Requires good decomposition: If the checklist criteria overlap or miss an important factor, the diagnosis will be off. Good decomposition is a skill and requires domain understanding.
⚠️ Not for simple monolithic tasks: If the concept is simple and clear-cut ("is this spam?" with the single criterion "bulk messaging without consent"), decomposition is overkill. The method shines on composite, multi-factor concepts where a direct question yields unstable answers.
⚠️ Longer than a typical prompt: 10 questions instead of one means more input and output tokens. For bulk checks of hundreds of texts, that costs more in API spend. Trade-off: consistency and transparency vs. speed and cost.
⚠️ Final aggregation logic: The study uses a supervised decision tree to combine the answers. In chat use, the aggregation logic is up to the user or the model. If you tell the model "decide yourself how to combine them", it may be inconsistent. Better to set explicit rules (
"if q6 OR q7 OR q8 = YES → critical").
How They Tested It
A team from Universidad Politécnica de Madrid took several popular hate speech datasets with different definitions and annotation guidelines. Instead of hunting for a single definition of hate speech, the researchers extracted common factors from international standards (UN, Council of Europe, UNESCO), platform policies (Meta, YouTube, X, Reddit, TikTok) and academic frameworks. They ended up with 10 reusable criteria: from basic ones (is there a target group, is there hostility) to severity markers (dehumanization, calls to violence).
For each text in the datasets they prompted an LLM to answer these 10 questions independently, getting a binary profile vector. Then they trained a simple decision tree on those vectors for the final classification. They compared three approaches:
- Zero-shot LLM — the direct question "is this hate speech?"
- Supervised fine-tuning — training the model further on a specific dataset
- xList-Hate — their checklist approach
In-domain (train and test on the same dataset), supervised fine-tuning won as expected, especially on small models. But cross-dataset (train on one, test on another) the picture changed: xList-Hate transferred between datasets more stably and lost less performance than the fine-tuned models.
Why it turned out this way: Supervised models memorize dataset-specific artifacts — which words correlate with the label in a particular annotation, quirks of the annotation guidelines, even annotator bias. Switch datasets and those shortcuts stop working. xList-Hate forces the use of explicit conceptual factors instead of statistical shortcuts — the model can't "cheat" by learning that "word X = hate"; it has to answer conceptual questions. The result: less overfitting, better transfer.
Another interesting finding: qualitative analysis of disagreement cases showed that xList-Hate often diverges from the dataset labels in cases where the annotators themselves were inconsistent or the context was ambiguous. For example, quoting hate speech to criticize it vs. endorsing it — a tricky edge case that depends on interpretation. The checklist approach shows exactly which factors are disputed through the diagnostic profile, making the source of uncertainty explicit.
Adaptations and Extrapolations
🔧 Technique: Add an explanation to every YES/NO → see the reasoning behind each factor
Instead of just [YES, NO, NO, YES, ...], ask the model to explain each answer:
ANSWER FORMAT:
1. {criterion_1}: YES/NO — {brief explanation of why}
2. {criterion_2}: YES/NO — {brief explanation of why}
...
Then a final conclusion based on the profile.
Effect: Full transparency of the logic. You see not just "5 of 10 criteria met" but the specific arguments for each factor. Useful for training a team, auditing decisions, and settling arguments with a client.
🔧 Technique: Weights for criteria → prioritize factors
Not all criteria are equally important. Assign weights:
CHECKLIST (with importance weights):
1. [WEIGHT: CRITICAL] {criterion_1}
2. [WEIGHT: IMPORTANT] {criterion_2}
3. [WEIGHT: NICE TO HAVE] {criterion_3}
...
FINAL LOGIC:
- If at least one CRITICAL criterion = NO → reject
- If 2+ IMPORTANT criteria = NO → needs work
- If all CRITICAL and IMPORTANT = YES → approve regardless of nice-to-haves
Effect: Explicit prioritization. Instead of simply counting YES/NO, you account for how critical each factor is. For moderation: a call to violence is critical regardless of anything else. For a sales proposal: a missing price is critical, a typo in the footer is nice to fix.
🔧 Technique: Hierarchical checklist → dependent checks
Some questions only make sense if the previous ones = YES:
BLOCK 1: Basic check
1. {basic_criterion}
[If BLOCK 1 is all YES → go to BLOCK 2, otherwise stop]
BLOCK 2: In-depth check
2. {detailed_criterion_1}
3. {detailed_criterion_2}
[If BLOCK 2 is all YES → go to BLOCK 3, otherwise stop]
BLOCK 3: Final quality check
...
Effect: Saves tokens and keeps the focus on what's relevant. You don't check a text's tone of voice if its basic structure is broken. You don't evaluate the details of an offer if there's no offer at all. Gate logic just like the original xList-Hate (q1 "is there a target group" is the gate for the remaining hate speech questions).
🔧 Extrapolation: Two passes with different roles → surface subjective criteria
For tasks where criteria are open to interpretation:
PASS 1: Answer the checklist as a strict critic
(the most demanding assessment, any deviation = NO)
PASS 2: Answer the checklist as a sympathetic advocate
(the most charitable reading, benefit of the doubt = YES)
COMPARISON: Show where the assessments diverge and why.
Those items are the zone of subjectivity and need a human decision.
Effect: You surface ambiguous factors automatically. Where the strict critic and the advocate agree, the criterion is objective. Where they diverge, it's subjective and needs context or human judgment. Handy for putting contested points up for team discussion.
Resources
xList-Hate: A Checklist-Based Framework for Interpretable and Generalizable Hate Speech Detection — Adrián Girón, Pablo Miralles, Javier Huertas-Tato, Sergio D'Antonio, David Camacho (Universidad Politécnica de Madrid), February 2025
The study draws on hate speech standards from: - United Nations Strategy and Plan of Action on Hate Speech - Council of Europe — European Commission against Racism and Intolerance (ECRI) - UNESCO guidelines - Platform policies: Meta (Facebook & Instagram), YouTube, X (Twitter), Reddit, TikTok