Identity-Robust Generation (IRG): Neutralizing Demographics for Objective Answers

STEP 1: Identity relevance analysis
Detect demographic mentions in the prompt → 
For each one: check "would the information needed to answer change if this mention were removed?" →
Mark it: critical / non-critical identity

STEP 2: Neutral content generation
Rewrite the prompt, removing non-critical identities →
Generate an answer to the neutral prompt

STEP 3 (optional): Controlled personalization
Adapt the answer's style to the user's identity (tone, level of explanation) →
Check: did the substance of the answer change? →
If YES — use the neutral answer; if NO — use the personalized one

Important: Step 1 needs a separate call for the analysis (you can do it with an LLM or an NER model). Step 2 is a separate call. Step 3 is an optional separate call.

Task: You're choosing between two approaches to building a product — ship an MVP fast or spend a long time laying the foundation. You want an objective analysis, but you happened to mention in the prompt that you're a startup founder — which can skew the answer toward "move fast."

Prompt (Steps 1 + 2 — neutralization):

Task: analyze two approaches to developing a SaaS product for small businesses.

Option A: Launch an MVP in 2 months with minimal features, collect feedback from the first customers, iterate based on their needs.

Option B: Spend 6 months building an architecture that can handle scaling, and adopt best practices from the very start.

I'm the founder of an early-stage startup. Which approach should I choose and why?

---

Before you answer:
1. Identify every mention of my identity in the request
2. For each mention: is this information needed to give a quality analysis?
3. If a mention is NOT critical, rewrite the request without it
4. Answer the rewritten neutral request

Result:

The model will first output its analysis: - Mention detected: "founder of an early-stage startup" - Relevance: Not critical for an objective analysis of the two approaches - Neutral version: "Analyze two approaches to developing a SaaS product..."

Then it'll deliver an objective analysis with decision criteria (risks, resources, market, competition), not tilted toward the "move fast or die" startup culture.

Prompt (Step 3 — optional personalization):

Now adapt this analysis for an early-stage startup founder:
- Keep all the key points of the analysis
- Change only the style: add concrete examples from startup practice, use startup-ecosystem terminology

After adapting, check: did the substance of the recommendations change? If so, go back to the neutral version.

Result:

The same analysis, but with real startup examples (Airbnb started by photographing apartments, not with AI recommendations) and terms like "burn rate" and "product-market fit." The substance of the recommendations didn't change — only the style was personalized.

An LLM can't tell critical from non-critical information in a prompt. When you write "I'm Muslim. What's 2+2?", the model sees the whole context and tries to "be helpful" by taking your religion into account. That's the alignment mechanism kicking in — the model was trained to adapt to the user. But for objective questions this causes distortions: answer accuracy starts depending on demographics, when it shouldn't.

Internally, the model knows the right answer. The study showed that if you look into the attention heads, the correct information is encoded equally well for all demographic groups (99.67% stability). The problem shows up at the decoding stage — when the model picks WHICH words to generate. Demographic markers shift token probabilities even though the internal knowledge hasn't changed.

Separating content from style removes the distortion. IRG works like a filter on the information flow: non-critical demographic markers don't reach content generation, but they can still shape the style. This plays to an LLM strength — the ability to follow structured instructions and simulate step-by-step processes. Models handle meta-tasks like "analyze relevance" or "keep the substance, change the form" well.

Control levers: - Relevance criterion — you can change the question from "would the information needed for the answer change?" to "would this affect objectivity?" for other kinds of bias - Neutralization level — you don't have to remove all demographics; you can keep only the relevant parts (e.g. "I'm 60" matters for a question about retirement, but not for a question about Python) - Integrity check — set a concrete criterion instead of a vague "substance": "did the key recommendations change?", "were any new facts added?"

Basic version (Steps 1+2: neutralization)

Task: {your question or request with any mentions of yourself}

Before you answer:
1. Identify every mention of my personal information (demographics, status, identity) in the request
2. For each mention: is this information critical for giving a correct answer to my question?
3. If a mention is NOT critical for the answer, rewrite the request without it, keeping the essence of the question
4. Answer the rewritten neutral request

Show your reasoning at each step.

Fill in: - {your question or request} — any question that may contain extra information about you (age, gender, profession, status, etc.)

Extended version (with optional Step 3 personalization)

Task: {your question or request with mentions of yourself}

STEP 1-2: NEUTRAL CONTENT
1. Identify every mention of my personal information in the request
2. For each one: is it critical for a correct answer?
3. Rewrite the request, removing the non-critical mentions
4. Answer the neutral request

STEP 3: STYLE PERSONALIZATION (optional)
5. Now adapt the answer to my context: {your identity/status}
   - Keep all the key points and recommendations from the neutral answer
   - Change only the style: tone, examples, level of detail
6. Check: did the substance of the recommendations change after personalization?
7. If the substance changed, go back to the neutral version. If not, use the personalized one.

Show your reasoning at each step.

Fill in: - {your question or request} — the full request with any mentions - {your identity/status} — who to adapt the style for (optional parameter)

⚠️ Automatic detection: The LLM can misjudge how relevant an identity is. In the study accuracy was 98.8%, but two error types remain: (1) critical information gets removed as irrelevant, (2) non-critical information gets kept as "topically related." For example, "I'm retired. How do I get a job?" — age is critical, but the model may not see the connection.

⚠️ Subjective preferences: The method works for objective aspects of quality (facts, helpfulness, safety), but can be overkill for subjective requests where personalization is desirable (e.g. "recommend a movie" — your age really is relevant).

⚠️ Style vs substance — a fuzzy line: The "did the substance change" check in Step 3 is subjective. If personalization added new examples or shifted the emphasis, is that a change in style or substance? The study checked it with an LLM, but the model can make the wrong call.

⚠️ Extra calls: A full implementation needs 2-3 sequential calls to the model (analysis → generation → personalization + check), which increases response time and token spend. For simple tasks it may be overkill.

Researchers from NYU took 18 demographic identities from six categories (education, religion, race, career, age, gender) and checked how LLMs answer identical questions when only the demographics in the prompt change. They tested on four benchmarks: TruthfulQA (facts), MMLU-Pro (helpfulness), AmbigQA (ambiguity resolution), StrongReject (safety). Models: Llama3.3-70B, gpt-oss-20B (MoE), Qwen3-8B.

First they measured the spread in answer quality across identities — and found accuracy varies by up to 3.8% for the same question. Then they did a surgical analysis of the model's internals: they trained probes on attention heads to see whether facts are encoded stably. The discovery: inside the model, knowledge is stable (0.33% variation); the problem is in generation. That explains WHY demographics distort answers — not because the model doesn't know, but because the word-selection process is skewed.

Next they tested IRG against two baselines: vanilla prompting (just ask the question) and prompt steering (add a system instruction "ignore demographics"). IRG cut bias by 77% on average; prompt steering had an inconsistent effect (helped in some places, made things worse in others). Why is IRG better? Because it controls the information flow directly instead of relying on the model's "understanding."

They checked robustness to different ways of stating demographics: a structured description ("a person of female gender"), perspective framing ("you're talking to a retiree"), and real prompts from WildChat ("I'm interning at a company"). IRG works consistently across all of them; the baselines don't. They also manually checked 500 rewritten prompts — 98.8% agreement with human relevance judgments.

The final test: does quality hold up after neutralization? They compared IRG with a baseline of "no demographics in the prompt at all" — the results were identical, so the method doesn't degrade content quality.

The researchers don't provide a single prompt template, but they describe the three-stage process in the text. Here's their approach:

Context: IRG has three components — identity detection and relevance analysis, neutral content generation, and controlled personalization. For detection they used GLiNER2 (a Named Entity Recognition model). For relevance analysis and rewriting — LLM calls.

Stage 1: Identity detection and relevance analysis

"Given a user query x, we first identify explicit sociodemographic expressions using a named entity recognition model GLiNER2 configured with predefined identity categories. For each detected identity span, we perform a counterfactual relevance check with a LLM to determine whether removing the identity would alter the information required to answer the query."

Stage 2: Identity-neutral content generation

"Using the relevance decisions from Stage 1, we rewrite the query by removing only identity terms classified as irrelevant, producing an identity-neutral input that preserves the original information need. [...] Then, we generate a core answer conditioned on this neutralized query, ensuring that factual, safety-critical, and task-essential content is not influenced by spurious identity cues."

Stage 3: Optional content-controlled personalization

"To acknowledge the potential benefits of personalization for presentation, such as adjusting tone or level of explanation, we introduce an optional Stage 3 that personalizes the identity-neutral answer. [...] The model is restricted to modifying only presentation-level attributes associated with the specified identity, without altering the content produced in Stage 2. We further apply a verification step to ensure that the personalized response preserves the original meaning of the identity-neutral answer."

💡 Adaptation: A simplified version without explicit neutralization

If you don't want a complex multi-step prompt, use the explicit separation principle:

Task: {your question with mentions of yourself}

First give an objective answer, ignoring any personal information about me in the request.

Then adapt that answer to my context: {your identity}, changing only the style and examples, not the substance of the recommendations.

This doesn't give you the full control of IRG, but it works through an explicit instruction to the model to separate content from style. Good for quick tasks where maximum accuracy isn't critical.

💡 Adaptation: Checking for "demographic drift"

Use the technique for a reverse check of answers you've already got:

Here is my request and the LLM's answer:

REQUEST: {your original prompt with demographics}
ANSWER: {the answer you got}

Check: did the demographic information in my request affect the SUBSTANCE of the answer (facts, recommendations, conclusions), and not just the style?

If it did, rewrite the answer, removing the influence of demographics on the content.

This works as post-processing when you already have an answer and want to make sure it's objective.

🔧 Technique: Inversion — check consistency with a counterfactual

Instead of removing demographics, swap in the opposite and compare the answers:

Task: {your question}

Give two independent answers:
1. For the context: {your identity A, e.g. "I'm 25"}
2. For the context: {opposite identity B, e.g. "I'm 65"}

Compare the answers:
- Do the key facts or recommendations differ?
- If YES — which differences are objectively justified by the context, and which are stereotypes?

Give a final answer with the unjustified differences removed.

This uses the LLM's capacity for counterfactual thinking — the model itself spots where the answer changed because of stereotypes rather than logic.

Identity-Robust Language Model Generation via Content Integrity Preservation
Authors: Miao Zhang, Kelly Chen, Md Mehrab Tanjim, Rumi Chunara
Affiliations: New York University, Adobe Research

The study references: - TruthfulQA (Lin et al., 2022) — a benchmark for evaluating factual accuracy - MMLU-Pro (Wang et al., 2024) — a benchmark for evaluating the helpfulness of answers to hard questions - AmbigQA (Min et al., 2020) — a benchmark for resolving ambiguous queries - StrongReject (Souly et al., 2024) — a benchmark for evaluating safety (refusing harmful requests) - GLiNER2 (Zaratiana et al., 2025) — a Named Entity Recognition model for detecting demographic mentions - WildChat dataset (Zhao et al., 2024) — a dataset of real user prompts