Apply to my task
arXiv:2610.23446 ★ 74 Oct 16, 2026

"You" Is Not Toxic: LLM Labelers Mistake Talking to Someone for Toxicity

Key result

Found: An LLM asked to find toxic comments reacts to the word "you" itself. Among comments humans consider fine, Claude Haiku flags 12% with no "you" and 51% with four or more. The method lets you check whether the model invented a trend, like "the deeper the thread, the more toxic it gets." Rewrite "you" into the third person and relabel. If the fake rise disappears, the pronoun is to blame. In the paper, the fake escalation dropped by two-thirds.

01

TL;DR

Ask an LLM to flag toxic comments and it reacts to second-person address ("you," "your"), not just to rudeness. The model sees the pronoun and leans toward the "toxic" label. That one habit warps every conclusion people draw from those labels.

Here's how it goes wrong in practice. "I recommend you also read this awesome article" gets labeled toxic, while "I recommend they also read…" sails through as fine. Among comments humans rate as non-toxic, Claude Haiku flags 12% of those with no "you" and 51% of those with four or more. Deeper in a thread, people address each other directly more often, so the model "discovers" that discussions spiral into toxicity as they go deeper. Human labels show no such spiral, if anything a slight decline. The model manufactures a pattern that isn't in the data.

What helps and what doesn't. Rewriting "you" into the third person wipes out two-thirds of the fake rise. What doesn't help: showing the model the parent comment (the fake rise actually grows), matching the flag rate to the human one, or optimizing for "agreement with humans" (Cohen's κ). Good agreement with humans does not guarantee a correct conclusion. The only reliable move is checking the final conclusion against a sample of human labels.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

02

Method Blueprint (auditing an LLM labeler)

STEP 1: Label the corpus with the LLM → "toxic / not toxic" flags
STEP 2: Take the subset humans labeled "non-toxic" (placebo group)
        → check the LLM flag rate by number of "you"s and by thread depth
        (flags rising with "you" and depth = bias)
STEP 3: Rewrite "you" → third person, relabel
        → did the rise disappear? then the pronoun is the cause
STEP 4: Check the final conclusion (not "labeling accuracy") against human labels
        on a random sample (hundreds to thousands)

Steps 1–3 are plain requests to the model. Step 4 needs human annotators. The paper automates it with a tool called Threadsafe, which you'd have to implement in code.

03

Example in Practice

Task: A media analyst (say, at a tech outlet like TechCrunch or a community like Hacker News) wants to test the popular belief that "the deeper a comment thread goes, the more toxic it gets." They've exported 20,000 comments and want an LLM to label them. This is where the method shines: a conclusion about a pattern built from mass labeling. It won't help you judge a single comment.

Prompt (step 3: is the pronoun to blame?):

Below is a comment from a Hacker News discussion. Rewrite it so that:
— every way of addressing the other person ("you/your/yours/yourself") is replaced with the third person
  ("he", "they", "the author", "the other commenter");
— the meaning, tone, harshness and insults stay exactly the same;
— nothing is softened and nothing is added.
Return only the rewritten text.

Comment: "Did you even read the article? Your conclusion doesn't follow from your own numbers."

Then both versions (original and rewritten) go through labeling with the exact same toxicity question. After that, you compare flag rates by thread depth.

Result: The model returns the comment rewritten in the third person with its bite intact. After two labeling runs, the analyst sees whether the "toxicity rises with depth" effect dropped. If it dropped noticeably, a big chunk of the "escalation" was the model reacting to "you." Exact numbers depend on the corpus. In the paper, on English news comments, the fake rise shrank by two-thirds.

04

Why It Works

The weakness. An LLM judges a comment by surface cues: addressing someone directly often travels with complaints, arguments and insults. The model ties "you" to toxicity even when it's just "you could try…". The error is tied to the structure of the conversation: the deeper the reply, the more likely it's aimed at a specific person. So instead of washing out as noise, it creates a fake trend.

Why the usual checks miss it. Agreement with humans is computed over all comments at once. It doesn't show where the model is wrong: at the root of threads or deep inside them. Two models with the same κ can reach opposite conclusions. In the paper, the models that agreed with humans best (Claude) distorted the escalation the most.

What the method exploits. Models are good at rewriting text while preserving meaning. That lets you run a control experiment: strip out "you" and see whether the effect vanishes. If it does, the pronoun is the culprit. It's a standard causal check, and you can run it with a prompt.

Control knobs: - What counts as placebo. Take comments humans have already called non-toxic. Any rise in model flags on them that tracks depth or "you" is false positives. - What to swap out. Instead of "you," you can test other surface cues: ALL CAPS, exclamation marks, profanity inside quotes, brand names. - Size of the human sample. In the paper, a thousand human labels was already enough to debunk Haiku's fake rise. Hundreds give a rough check; thousands give a solid one.

05

Prompt Template

Step 3 (neutralizing direct address). This is my applied build based on the authors' description of the experiment:

Rewrite the comment, replacing every way of addressing the other person
({address_forms}) with the third person ({replacements}).
Preserve: meaning, tone, harshness, insults, punctuation.
Do not soften, add or explain anything.
Return only the text.

Comment: "{text}"

Steps 1 and 3 (the toxicity question, following the authors' description of their "main prompt," four answer levels):

Rate the comment on this scale: very toxic / toxic / hard to say / not toxic.
Consider toxic any rude, disrespectful or unreasonable text
that could make someone want to leave the discussion.
Return only one of the four labels.

Comment: "{text}"

What to fill in: {address_forms} (for English: "you, your, yours, yourself, y'all…"), {replacements} ("he/she/they, the author, the other commenter") and {text} (the comment).

06

Limitations

⚠️ English only: The experiments used English texts (Civil Comments, Reddit, Gab, Wikipedia). Languages with a formal/informal "you" split (German du/Sie, Spanish tú/usted, French tu/vous), dropped subjects and verb-based address were not tested. The effect may differ there, especially where the formal "you" signals politeness.

⚠️ The pronoun fix is partial: Rewriting removes two-thirds of the fake rise across a whole corpus and all of it within a single thread. When address appears in both the comment and its parent, only about a quarter goes away. It's a diagnostic tool, not a cure.

⚠️ Context doesn't fix it: Showing the parent comment increased the fake rise instead of reducing it. Don't count on "I'll give the model the whole thread and it'll figure it out."

⚠️ Chatbot conversations: No bias was found in human–chatbot dialogues. The effect applies to comments between people.

⚠️ Full protection needs humans and code: The Threadsafe method (human labeling on a smart sampling plan + correcting the predictions) can't be run with a prompt. Without a human sample, you're left with diagnostics and caution in your conclusions.

⚠️ No proven "magic instruction": The authors didn't test a line like "addressing someone as 'you' is not toxic in itself." That's a hypothesis you'll need to test yourself.

07

How They Studied It

The team took 1.25M news comments (Civil Comments) with crowd labels. Each comment was rated by several people, and the "gold" label came from a vote. They tested seven LLMs (Claude Haiku and Sonnet, plus five open models) and 14 classifier setups, including a replacement for the soon-to-be-shut-down Perspective API.

The idea was to compare not accuracy on individual comments but patterns: does toxicity rise with reply depth, with thread size, is it contagious. With human labels, toxicity dips slightly with depth (−0.12 pp per level); with Haiku, it rises (+0.93). To find the cause, the authors looked only at comments humans consider non-toxic (a placebo statistic) and saw flags rise along with the number of "you"s. A single human annotator shows almost no such rise (5.9% → 10.0%, versus 12% → 51% for Haiku).

Then they tested the hypothesis causally: they rewrote "you" into the third person, and two-thirds of the fake rise disappeared. They also checked that showing the parent comment doesn't help, that matching the flag rate doesn't help, and that a "standard error correction" doubles the fake rise (+2.02). It turned out that agreement with humans guarantees nothing: a model with κ = 0.9 could in theory produce any law from −1.30 to +1.65.

The "contagiousness" of toxicity also depends on the reference: labeled by all raters, the coefficient is 2.39; by a single rater, 1.63. The practical takeaway: results built on LLM labels without a human check can't be treated as patterns about people.

08

Original from the Paper

Example from the paper (verbatim):

Claude Haiku calls the reply
“I recommend you also read this awesome article” toxic
and “I recommend they also read this awesome article” not toxic

Context: The authors used this pair to show that a single pronoun flips the model's verdict.

09

Adaptations and Extrapolations

💡 Adaptation: auditing an LLM judge on any "noise" cue. The same trick works when an LLM rates reviews, support tickets or employee replies. Take a sample humans consider "fine" and check whether the share of "bad" ratings rises along with a surface cue (length, addressing someone as "you," the word "urgent," exclamation marks). If it does, the model is reacting to the cue, not the substance.

Here are 30 support tickets I marked as "normal."
The judge model put some of them in the "aggressive" bucket.
Find which surface features (addressing the agent as "you," exclamation marks,
the words "I demand/urgent," length) show up more often among the mislabeled ones
than among the rest. Give me a table: feature → share among mislabeled → share among the rest.

🔧 Technique: add a control run. For any mass labeling job, add a second run on the same texts with the suspicious cue removed. The gap between the runs is your estimate of that cue's influence. It's cheaper than digging through errors by hand.

10

Resources

  • Paper: "You" Is Not Toxic: How LLM Labelers Rewrite the Web's Conversation Laws
  • Authors: Liang Yue, Eric A. Vance, University of Colorado Boulder
  • Code, labels and prompts: https://github.com/liyu2195/you-is-not-toxic
  • Datasets mentioned: Civil Comments, CAD (Contextual Abuse Dataset), Qian et al. (Reddit, Gab), CGA-Wiki (Conversations Gone Awry), BAD (Bot-Adversarial Dialogue)
  • Tools compared: Perspective API (Google, being shut down), Detoxify, PPI++, DSL

Research digest

Key result

Found: An LLM asked to find toxic comments reacts to the word "you" itself. Among comments humans consider fine, Claude Haiku flags 12% with no "you" and 51% with four or more. The method lets you check whether the model invented a trend, like "the deeper the thread, the more toxic it gets." Rewrite "you" into the third person and relabel. If the fake rise disappears, the pronoun is to blame. In the paper, the fake escalation dropped by two-thirds.

Principle

Four steps, and three of them are plain requests to the model. Label the corpus → take the comments humans already called fine → see how the model's flags rise with the number of "you"s and with thread depth → rewrite "you" into the third person and relabel → check the final conclusion against human labels on a random sample. Comments humans called fine are your control group: any rise in model flags on them is false. It works like a placebo in a drug trial. No drug, but there's an effect? Then something else is going on. Here, that something is "you."

Why it works

The model looks at the surface of the text. "You" often sits next to arguments and insults. It learns the link "direct address = rudeness" and carries it over to a harmless "you could try…". Deeper in threads, people reply to a specific person more often, so there's more "you" there. The error is tied to the structure of the conversation, so instead of drowning in noise it draws a clean fake trend. By human labels there's no slide into toxicity with depth, if anything a slight decline. Here's the kicker: the models that agreed with humans best (Claude) distorted the escalation the most. Agreement with humans (Cohen's κ) is computed over all comments at once. It doesn't show where exactly the model is lying: at thread roots or deep down. Two models with the same κ reach opposite conclusions. Context doesn't save you either. Show the parent comment and the fake rise gets bigger. A human-sample check works: a thousand labels was enough to debunk Haiku's conclusion.

When to apply

Analyzing large batches of comments (forums, social media, news sites) → drawing a conclusion about a pattern, like "toxicity rises with thread depth" or "one platform is meaner than another," especially when that conclusion is headed for a report or publication. You can also test other surface cues: ALL CAPS, exclamation marks, profanity inside quotes, brand names. NOT for judging a single comment: the method only checks aggregate conclusions. No bias was found in human–chatbot dialogues. Other languages weren't tested: a polite formal "you" (like German Sie or Spanish usted) may behave differently.

Mini recipe

1. Label everything as usual: ask the model the same toxicity question for every comment. You get "toxic / not toxic" flags. 2. Find a control group: take comments humans already called fine. If you have no human labels, take a small random sample and label it by hand. 3. See where flags rise: compare the model's flag rate by number of "you"s and by thread depth. Rising together — suspect bias. 4. Do the swap: rewrite direct address into the third person. Keep the meaning, harshness and insults; soften nothing. Label the rewritten texts with the same question. 5. Compare the two trends: did the rise with depth vanish or drop a lot? Then part of the "escalation" was the model reacting to "you." 6. Don't trust a single run: pick a random few hundred or thousand comments and check the final conclusion (not "labeling accuracy") against human labels. Swap template: `Rewrite the comment, replacing every way of addressing the other person ({address_forms}) with the third person ({replacements}). Preserve meaning, tone, harshness, insults, punctuation. Do not soften, add or explain anything. Return only the text. Comment: "{text}"`

Examples

[BAD]: `Label these 20,000 Hacker News comments for toxicity and tell me whether it rises with thread depth` — and the conclusion goes into the report unchecked. The model will draw an escalation out of "you" all by itself. [GOOD]: `Rewrite the comment, replacing every form of address ("you/your/yours/yourself") with the third person ("he", "they", "the author", "the other commenter"). Keep the meaning, tone, harshness and insults. Don't soften or add anything. Return only the text. Comment: "Did you even read the article? Your conclusion doesn't follow from your own numbers."` Then both versions (original and rewritten) go through labeling with the same question, and flag rates are compared by thread depth. Rise gone or shrunk — the trend was largely created by the model, and you need to flag that in the report. Rise still there — the trend is probably real, but check it against a human sample anyway.

In plain words

Models don't understand real aggression — they just count co-occurrences. Online, trash talk almost always gets personal, so the model memorized a broken pattern: it sees "you" and instantly stamps it "toxic". Instead of reading the meaning, the LLM gets triggered by a grammatical form and mistakes ordinary address for open hostility.

It's like a paranoid bouncer who throws anyone out into the cold for pointing a finger, even if they're just asking where the bar is. A formal sign of conflict was spotted, but the result is total nonsense. The model confuses speaking to someone with actual rudeness and starts waging war on normal conversation.

In practice, this glitch flat-out breaks mass data labeling. An analyst feeds the model 20,000 comments from a forum to prove that long threads always descend into flame wars. The deeper the thread, the more naturally people write "you're wrong" or a harmless "you can update it". The upshot: the LLM slaps "toxic" on things out of nowhere and hands the researcher a bogus pattern.

The problem surfaced with toxicity, but the principle is universal. Blind LLM labeling screws up anywhere there are surface correlations: support ticket sentiment analysis, customer review audits, engagement scoring. The model systematically penalizes people for talking like humans, and the business ends up with hallucinations instead of metrics.

Stop worshipping out-of-the-box prompts: dumb auto-moderation kicks out reasonable people, not trolls. Keep tabs on what the model is actually reacting to, or your "data-driven" analytics turns into straight-up garbage. If the model sees an enemy in the word "you", the problem clearly isn't the users — it's whoever set up the algorithm.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on "You" Is Not Toxic: LLM Labelers Mistake Talking to Someone for Toxicity arXiv:2610.23446

Sign in to apply this study New accounts get 50 free credits.

Related techniques