TL;DR
Epistemic Blinding is a technique for auditing LLM analysis: before the request, every name in the data is replaced with an anonymous code (Company_001, Candidate_A), and the result is then compared with a control request where the names are left in. The difference between the two answers shows how much of the LLM’s output came from your data and how much from the model’s memory of famous brands and names.
The big discovery: when you ask an LLM to rank items by metrics, the model quietly pushes well-known names to the top of the list — even when their numbers are weaker. The effect is neither random nor small: in financial analysis, 30–40% of the top-20 recommendations change just depending on whether the model sees the company tickers. In biological data, KRAS took #1 with the rationale “proven therapeutic tractability via covalent RAS inhibitors” — even though that phrase appeared nowhere in the data. This isn’t a hallucination. It’s the model legitimately applying its knowledge — exactly when you didn’t want it to.
The method works in two steps. Step 1: replace every name with a code and send the data to a fresh LLM session with an instruction to rely only on the data provided. Step 2: send the same data with the real names to a separate session. Compare the lists — the divergence is the contamination zone: the places where the model ranked by reputation, not by the numbers.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
Two parallel requests in separate sessions:
SESSION A (Blind):
STEP 1: Take the data with names/titles
STEP 2: Replace the names with codes (Candidate_001, Company_042)
STEP 3: Shuffle the rows (remove positional hints)
STEP 4: Send to a NEW LLM session → get List A
SESSION B (Control):
STEP 1: The same data, but with real names
STEP 2: The same request in ANOTHER new session → get List B
ANALYSIS:
List A ≠ List B → zones where the model listened to its memory, not your data
List A = List B → conclusions backed by the data
Example in Action
Task: You’re in HR at a tech company. You need to pick the top 5 candidates for a product manager role out of 15 finalists. Each has test scores: product thinking (0–100), analytics (0–100), communication (0–100), past metric growth (%), years of experience, case-interview score (0–10). Some candidates come from Google, JPMorgan and Whole Foods, others from startups. You want to hire on real skills — but subconsciously you trust the big names.
Request (Session A — blind analysis):
Below is the candidate data. All names have been replaced with codes.
Your task: pick the top 5 for the Product Manager role,
relying ONLY on the data provided.
Don't use any outside knowledge about companies or
the industry. Only the numbers in the table.
| ID | Product thinking | Analytics | Communication | Metric growth % | Experience (yrs) | Case |
|----|-------------------|-----------|--------------|---------------|------------|------|
| C_001 | 87 | 92 | 78 | 34% | 4 | 8.2 |
| C_002 | 91 | 74 | 88 | 12% | 7 | 7.1 |
| C_003 | 69 | 88 | 91 | 67% | 2 | 8.7 |
...
Pick the top 5. For each, justify the choice with specific
numbers from the table.
The same request — Session B — but with the candidates’ real names and previous employers instead of codes.
Result:
The model returns two ranked lists. Where the lists match, the data speaks for itself and both approaches agree. Where a Google alum ranks significantly higher in Session B than their code C_008 did in Session A, you have measurable proof: the model (and maybe you too) reacted to the employer brand, not the test results.
Why It Works
The LLM’s weak spot. When a model sees the word “Google” or “Tesla,” it instantly activates everything it knows about that entity from its training data. That’s not a bug — it’s how the model works: every token carries the context of everything associated with it. The problem is that this context blends into your data invisibly. In the final answer you can’t see where your numbers end and the model’s memory begins.
The LLM’s strong suit. The same model is perfectly capable of working cleanly with numbers and faceless codes. When a name is replaced with C_003, there are no associations — only what you gave it remains: 91 points in analytics, 67% metric growth. The model generates its conclusion strictly from the table.
How the method uses this. Blinding is a controlled experiment inside your workflow. You don’t remove the model’s knowledge entirely. You measure its influence: you compare what the model says “blind” versus “with the name hints.” The divergence is a precise contamination metric. You get not just a recommendation but an audit: how far you can trust this analysis.
Control knobs: - Degree of anonymization → anonymize only the names but keep categories (e.g., “industry: retail”) — partial blinding for less sensitive tasks - Number of parallel sessions → run the blind analysis 2–3 times with different row orders to filter out random noise - The “data only” instruction → make it stricter or looser depending on how much you want to isolate outside knowledge
Prompt Template
Universal template (Session A — blind analysis):
Below is data on {items: candidates / companies / options}.
All identifiers have been replaced with anonymous codes.
Your task: {task: pick the top N / rank /
compare and recommend}.
Rule: rely ONLY on the data provided.
No outside knowledge, reputation or associations.
Only the numbers and attributes in the table below.
{table with anonymous codes instead of names}
Pick the {top N}. For each, justify it with
specific values from the table — no generalities.
Control request (Session B) — the same template, but the {table} has real names. Run it in a separate new session.
What to plug in:
- {items} — what you’re analyzing: candidates, startups, vendors, products
- {task} — the concrete action: pick the top 3 to invest in, rank by risk, weed out the laggards
- {table} — your real data + a separate table with the same data but names instead of codes
- {top N} — how many finalists you need
🚀 Quick start — paste into the chat:
Here is the Epistemic Blinding template for blind data analysis.
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what items you’re analyzing, which metrics, and how many to shortlist — because without that it can’t properly set up the “data only” instruction and the table structure. It will take the pattern from the template and adapt it to the task.
Limitations
⚠️ Only for data-driven ranking tasks: If you’re asking an LLM to summarize literature, generate ideas or look up information, blinding is counterproductive. Those are exactly the cases where you want the model’s knowledge.
⚠️ Structural leaks: Some combinations of attributes give away the name even without the label. A company with $575 billion in revenue and roughly 40% of US e-commerce is Amazon, ticker or not. In cases like that, blinding reduces contamination but doesn’t eliminate it.
⚠️ Names that carry meaning: Chemical compound names (they encode the molecule’s structure), drug class names (they encode the mechanism of action) — you can’t blind those without losing useful information.
⚠️ No answer to “who’s right”: The method doesn’t tell you which list is better. It tells you where the results diverge. What to do about that divergence is up to you.
⚠️ Requires manual work: Anonymization has to be done by hand or with an LLM’s help. It takes a few minutes, but it’s an extra step.
How They Tested It
A biomedical researcher was building a system to find drug targets in oncology. At some point he noticed something odd: the model gave different gene rankings depending on whether it saw the gene symbols (KRAS) or anonymous codes (Gene_088). That became the starting point.
The test was run on two completely different domains — to show this isn’t a biology quirk. In oncology, they took 4 cancer types with different “signal strength” in the data: from clear-cut (AML, 30 significant genes) to noisy (glioblastoma, 110 genes at similar low frequencies). For each type they ran two identical requests to Claude: one with the real gene symbols, the other with anonymous codes. Average result: the top 20 overlapped by 84% — meaning 16% of recommendations changed purely because the names were visible. Meanwhile, validated targets (FDA-approved targets) were found equally in both conditions. That’s the key point: blinding doesn’t degrade the analysis — it only changes which new candidates make the list.
For finance they took ~500 companies from the S&P 500 with four blocks of fundamental metrics: valuation (P/E, P/S, etc.), growth, quality, dividends. The metrics were normalized by sector to remove structural hints. They ran 5 independent sessions with different random seeds. Result: 30–40% of the top 20 changed when the tickers were revealed. And the effect is systematic — the same tickers (ELV, CI) were pushed up in 4 out of 5 sessions.
The most convincing detail is the model’s justifications. When the gene was called Gene_088, the model wrote: “very high mutation frequency (42.1%) with strong structural enrichment.” When the same gene was revealed as KRAS, the phrase “proven therapeutic tractability via covalent RAS inhibitors” appeared. That phrase wasn’t in a single row of the data provided. It’s direct proof of the mechanism: the model added information from its memory — and that only became visible through the comparison.
Original from the Paper
Gene_088: "Very high mutation frequency (42.1%)
with full constraint and strong structural enrichment
(ESM2=22.3), pointing to a recurrent constrained driver."
KRAS: "Highest-impact druggable oncogene... with proven
therapeutic tractability via covalent RAS inhibitors and
the strongest combined mutation frequency plus structural
convergence in this set."
Context: An identical analysis of the same gene in CRC (colorectal cancer) — in the blind and the open condition. The highlighted phrase isn’t present in any of the features provided — it’s pure parametric memory of the model.
Adaptations and Extrapolations
💡 Adaptation for evaluating content or contractors
The same principle works when you’re evaluating copy, design mockups or proposals from contractors. Replace the author’s or agency’s name with a code before asking the LLM for feedback. Otherwise the model will react to the agency’s fame or to a style it recognizes from public work.
Below are {N} versions of {copy / mockup / proposal}.
The authors are anonymized: Option_A, Option_B, Option_C.
Score each one on these criteria:
1. {criterion_1} (0–10)
2. {criterion_2} (0–10)
3. {criterion_3} (0–10)
Rely only on the material presented.
Rank them and explain using specific elements of the copy/mockup.
{Option_A: [paste material]}
{Option_B: [paste material]}
🔧 Technique: “have the LLM do the anonymization for you”
You don’t have to replace every name by hand. Just give the LLM the original table and ask it to anonymize it — it will return a table with codes plus a mapping for de-anonymization.
I have a data table. Replace all names, company names
and identifiers with anonymous codes in the format
{Item_001}, {Item_002}, etc.
Return:
1. The anonymized table
2. A mapping table: Code → Real name
{paste your table}
Then use the anonymized table for the blind analysis and the mapping to de-anonymize the final list.
🔧 Technique: auditing your own decisions
If you already have a gut-feel list (“I think these are the top 3”), run the blind analysis first and then compare it with your list. The divergence shows where your intuition is reacting to brands rather than metrics. It’s the same blinding, applied to your own thinking.
Resources
- Paper: Epistemic Blinding: An Inference-Time Protocol for Auditing Prior Contamination in LLM-Assisted Analysis
- Open-source code: github.com/mcuccarese/epistemic-blinding
- Author: Michael F. Cuccarese, PhD
- Related work: Wang et al. (entity-specific biases in factual QA), Choi et al. (anonymization in multi-agent reasoning reduces conformity bias), Hermann et al. (pretraining leakage in protein language models)