Temporal Masking: Why LLMs Bend Past Decisions to Fit a Known Outcome — and How to Stop It

STEP 1: Define the reference point — the moment for which you need an objective assessment
STEP 2: Build the context ONLY from events before that point → cut everything that happened later
STEP 3: Ask about the decision as of that moment, without mentioning the outcome
STEP 4 (optional, but not enough on its own): Add the instruction "don't use knowledge of the outcome" — it reduces the effect but doesn't replace the trimming in step 2

All steps happen in one prompt — you simply control what information goes into it.

Task: You're breaking down a case for a podcast or a consulting session — should the founder of a hypothetical delivery startup have turned down a funding round in 2021, judging the decision only by the data available back then rather than by the fact that the market crashed in 2022.

Prompt:

You are an independent business analyst. Evaluate the company founder's decision 
as of March 2021.

Here are the facts known to the company BEFORE March 2021:
- Revenue was growing 40% year over year
- The round was offered at a $50M valuation
- The main competitor had just closed a round at a $200M valuation
- Unit economics were positive in three out of five cities

[Don't add anything here about what happened after March 2021 — 
no data on the market downturn, the crisis, or the company's later fate]

Question: Was it reasonable to turn down the round based ONLY 
on the data above? Give a score from 1 to 5 and justify it.

Result: The model gives an assessment based solely on the facts provided — without a hindsight "hint." If you additionally ask the model to evaluate the same decision but add the fact "in 2022 the market crashed" to the context, the answer will most likely shift toward "the decision was bad" — even if it was perfectly logical at the time it was made. Comparing these two answers shows you exactly where hindsight bias kicks in.

An LLM can't truly separate "what was known then" from "what happened later" — if information about the outcome is in the context, it shapes the assessment of the past even when the question doesn't directly mention it. It's the same effect people have: "I knew it all along" — and judging by the study's results, the model falls for it just as easily as a human.

At the same time, the model has a strength: it responds to what is physically present in the text. Remove the data, and the model can't "make it up out of thin air" — it has to rely only on what it sees.

The method leverages that strength: instead of an instruction to "forget the outcome" (which works partially and unreliably), it literally cuts out the distorting information from the context. No access to the data — no distortion.

Control levers: - Reference point (cutoff) → move it earlier or later depending on which decision moment you care about. - An explicit "don't use knowledge of the outcome" warning → add it as an extra layer of protection, but not as the main mechanism — on its own it doesn't fully remove the effect. - Question format (numeric score, yes/no, free text) → pick what fits the task; the method works the same across all formats.

⚠️ An instruction isn't a full fix: even a direct request to "ignore what happened afterward" reduces the distortion but doesn't remove it — the model still partly gravitates toward the known outcome if it's in the context.

⚠️ Partial information leakage weakens the method: if you trimmed the timeline but left the original text in the context (for example, a full narrative that briefly mentions what happened next), the protection works worse — you need to scrub all sources of future information, not just one.

⚠️ The method doesn't make the model smarter: trimming the context removes the distortion but doesn't increase accuracy — if the correct answer objectively requires later information, leaving it out won't help.

The researchers took 171 real medical cases from the open PubMed database — 40 sepsis cases and 131 diabetes cases involving GLP-1 class drugs. Each case was turned into a time-stamped chronology of events. For each case they generated 10 questions tied to a specific moment — for example, "what probability of a complication is reasonable at hour 24" — with a predefined "correct" answer (based only on data up to that moment) and a "trap" (an answer matching the actual outcome).

Four models (GPT, Gemma, GLM, Opus) answered the same questions in two versions: with the full history and with a time-trimmed one. They tested seven different conditions — with the original case text, with synthetic text, with manual and automatic timeline annotation — to make sure the effect wasn't tied to a particular data format.

The result held across all conditions: full access to future information pushed the models' answers toward the "trap" — and this was confirmed not only through answer changes in general but also through a strict metric that captured transitions specifically from an initially correct answer to an incorrect one matching the outcome. At the same time, answer accuracy didn't change significantly in any condition — meaning the future information didn't help the models answer more correctly; it only distorted their judgment. They also checked separately whether an explicit "don't think about the outcome" instruction helps — it turned out to reduce the effect but not eliminate it, which led the authors to conclude that control at the context level is more reliable than an instruction.

🔧 Technique: applying it outside medicine — evaluating business decisions, case breakdowns, media analysis

The principle works in any situation where you need to evaluate a decision "in the moment" rather than in hindsight. For example, when preparing a breakdown for a blog or a podcast about startups:

Evaluate [company]'s decision to [decision] in [date].

Here is what the team knew at the time: [only facts before the date]

Don't mention or use information about what happened 
after [date] — even if you know it from your training data. 
Evaluate the decision strictly from the standpoint of the information above.

This is especially useful if the model "knows" the real outcome from its training data (say, a widely known company story) — then a simple prohibition isn't enough; you need to explicitly confine the frame of reasoning to the facts provided.

🔧 Technique: A/B comparison to diagnose bias

To check how strongly the model itself is prone to this distortion in your task, ask the same question twice: once with trimmed context, once with the full context (including the outcome). Compare the answers. If they diverge, you've caught hindsight bias in its purest form.

Hindsight Bias in Clinical Temporal Reasoning: How Future Data Exposure Affects Large Language Model Judgment — Misaki Matsuura (Case Western Reserve University), Sayantan Kumar and Jeremy C. Weiss (National Library of Medicine, NIH), Ojas Kadam (Rice University). Submitted to ML4H 2026 (Machine Learning for Health).