TL;DR
The study shows that you can fool a model with a fact that is true but belongs to a different year, place or entity. Say you hand it a document with 2023 rates and ask about 2025. The model often grabs the number from that document, even though it answered correctly without it. A false but on-topic snippet fools it even more often. A correct source placed alongside almost neutralizes the true-but-wrong context. Against false context that works less well, and a lot depends on order.
The sore spot: we are used to checking context for truth ("is this correct?"). We also need to check it for applicability ("is this about my question?"). The model is bad at this on its own: it spots the "wrong scope" only part of the time. And even when it spots it, it often picks the wrong answer anyway. When the false snippet comes before the correct one, errors jump several-fold compared with the correct one going first.
A simple instruction like "use only applicable context, otherwise decline" cuts misled answers roughly in half. But most of that gain comes from the model declining to answer more often. Correct answers drop, even when the correct context is right there. So watch three numbers at once: correct, misled, declined.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
This is a finding, not a technique. Here is how the experiment works and what follows from it:
THREE KINDS OF CONTEXT (per question):
G — correct and on-topic → leads to the right answer
S — true, but about a different → leads to the wrong one ("the decoy")
year / place / entity
F — false, but "about the same" → leads to the same decoy
CONDITIONS: no context | S only | F only | G+S | S+G | G+F | F+G
TAKEAWAYS FOR PRACTICE:
1. Check not just "is it true" but "is it about this"
2. Correct/trusted source goes FIRST
3. Give the model an explicit "not enough information" option
4. Track three metrics: correct / misled / declined
Example
Task: A support bot for Amazon sellers answers a question about referral fees. The knowledge base holds the current 2025 seller agreement and an old slide deck with 2023 fees. The old one is true, but for the wrong year. You need the bot not to mix them up. (The numbers below are made up.)
Prompt:
Question: What marketplace fee applies to the "Apparel" category under FBA starting January 1, 2025?
Options:
A. 15%
B. 17%
C. 19%
D. 21%
E. Not enough information
Context 1 (trusted source, seller agreement, 2025 edition):
"Starting January 1, 2025, the fee for the \"Apparel\" category under FBA is 17%."
Context 2 (archive, 2023 seller slide deck):
"In 2023, the fee for the \"Apparel\" category under FBA was 15%."
Answer with a single letter.
Result: You get a single letter back. The correct source comes first and is anchored by an explicit year, so errors will be rare. Swap the contexts, or drop the seller agreement, and the model starts picking the archive option more often. That is the "true but off-target" effect in action. Run both orders on 20–30 of your own questions and count correct, wrong and declined answers.
Why It Works
The LLM's weakness. A model tends to lean on whatever sits in the context, especially if the snippet resembles the question. A number from a 2023 document looks like an answer about 2025: same words, same topic. A "wait, is this the right year?" check isn't built in, and the model doesn't reach for it on its own. A false snippet is even more dangerous: it flatly states "for your year, the value is X."
The LLM's strength. When a correct snippet sits nearby, the model can cross-check and discard the noise. True-but-off-target context barely gets in the way here: it doesn't contradict the correct one, they're simply about different things. False context does contradict it, and the model has to choose. That's where order matters: if the false snippet comes first, it pulls much harder.
Control levers: - Snippet order → trusted source first, doubtful ones below. In document search, that means watching what lands in the top positions. - Explicit scope in the context text (year, region, entity right at the start of the snippet) → makes a mismatch easier for the model to notice. No guarantee, though. - A "not enough information" option → gives the model a legitimate exit instead of guessing. - An "applicable context only" instruction → fewer misled answers, but more refusals and lost accuracy. Turn it on when a mistake costs more than a refusal.
Prompt Template
This is the format the authors used to test model behavior: the question, options including "not enough information," then the contexts. The optional applicability line is your own wording based on the paper's idea, not the authors' verbatim text.
Question: {question_with_explicit_year_place_entity}
Options:
A. {option_1}
B. {option_2}
C. {option_3}
D. {option_4}
E. Not enough information
Context 1 ({source_1_most_trusted}):
{text_1}
Context 2 ({source_2}):
{text_2}
[Optional] Use only the snippets that refer to exactly the year, place and entity being asked about. If there are none, choose E.
Answer with a single letter.
What to fill in:
- {question…} — a question with an explicit year, region or entity, so there's something to check against.
- {source_1} — put the most trusted snippet first.
- Include the applicability line when declining beats being wrong. If you want the maximum number of correct answers, leave it out and compare both versions.
Limitations
⚠️ Small open models only: The tested models were mid-sized, with no reasoning mode and a forced single-letter answer. On modern cloud models (ChatGPT, Claude, Gemini) the numbers may differ, and even the direction of the effects needs checking.
⚠️ Multiple-choice tests only: The questions are factual, with a single number or name as the answer. For free-form text, summaries and reasoning the paper proves nothing.
⚠️ The "applicable context only" instruction isn't a cure: It reduces misled answers mostly through refusals. Correct answers drop, even when the correct context is present.
⚠️ Asking "check whether the context applies" guarantees nothing: Models recognize the "wrong scope" only part of the time. Even when they do, they often still pick the decoy.
⚠️ "Correct first" requires knowing what's correct: In real work you don't always know which snippet is trustworthy. The rule works when sources have a clear hierarchy: the official agreement outranks a chat thread, the current edition outranks the archive.
⚠️ Models vary a lot: One of the six models was fooled by true-but-off-target context almost as much as by false context. Test yours.
⚠️ Order and proximity to the question aren't separated: The first snippet is also the one closer to the question. The authors admit they can't tease apart a "first-position effect" from a "proximity effect."
⚠️ The paper text was incomplete: Part of the analysis (robustness of the findings) was cut off in the provided text, so this review relies on the abstract, introduction and main results.
How They Tested It
The researchers built 959 questions with five answer options each. One option is always "not enough information." Sources: Federal Reserve data on minimum wage by state and year, SituatedQA questions (the answer depends on time and place) and AmbigDocs (different entities sharing the same name). For each question they made three snippets: correct (G), true but off-target (S) and false but on-target (F). F was derived from G by swapping the correct value for the decoy, leaving the rest of the text untouched. S and F point to the same wrong answer, so the comparison is fair.
Six open models answered every question under eight conditions: no context, S only, F only, plus G together with S or F in both orders. That came to 46,032 answers. Only questions the model answered correctly without context were counted. That way any drift toward the decoy can be fairly blamed on the context. Dataset quality was checked by two strong models and by humans: 64 questionable items were removed, and a spot audit of 150 questions found almost no errors.
The results. True-but-off-target context alone pushes the model toward the decoy in about 29% of cases. False on-topic context does it in 75%. With a correct snippet nearby, S nearly stops working (under 1%), while false F stays dangerous (about 21%). Order is decisive: false snippet first yields about 38% errors, correct first about 4%. Two results were surprising. Models recognized the "wrong scope" only about 62% of the time, and even then they still picked the decoy in a fifth of cases. The "use only applicable context" instruction roughly halved decoy picks, but mostly through more refusals and at the cost of accuracy even with correct context. The practical takeaway: look at three numbers together — correct, misled, declined.
Adaptations and Extrapolations
🔧 Technique: change the order → less pull from doubtful sources
If you have a trust hierarchy, arrange snippets strictly by it: the official agreement, then policies, then chat threads and the archive. In the paper, "correct first" produced roughly ten times fewer errors from false context than "false first."
🔧 Technique: drop the "not enough information" option → see how much it makes up
Run the same questions with option E and without it. The gap shows how much the bot guesses when context is thin. You can use this to compare two orders or two phrasings of an instruction.
Extrapolation (not tested in the paper): before answering, ask the model to write down the scope of each snippet. This is your own idea inspired by the paper. The authors showed that recognizing scope doesn't guarantee a correct answer, so measure the result on your own data.
Before answering, write one line for each context:
— which year / region / entity it refers to;
— whether that matches the question (yes / no).
Then answer using only the snippets marked "yes".
If there are none, answer "Not enough information".
Resources
- MisleadQA: Diagnosing LLM Failures under Misleading Context — Hui Wang, Bohao Chu, Qianli Wang, Shuning Zhang, Norbert Fuhr. University of Duisburg-Essen, Technische Universität Berlin, Tsinghua University.
- Data and code: https://anonymous.4open.science/r/MisleadQA
- Data sources: FRED (Federal Reserve Bank of St. Louis), SituatedQA, AmbigDocs, CondAmbigQA.