Lost in the middle: how to make an LLM use the whole long document

Symptom: the answer ignores part of what you pasted

You paste a long contract, transcript or thread, and the answer is fluent but built from the first pages. A clause near the end is missing, a fact from the middle is "not in the document", or the model answers a question from a nearby example instead of yours. It feels like the model did not read to the end, and in a measurable sense it did not.

In a needle-in-a-haystack test on Balzac novels, the anatomy-of-long-context study saw ChatGPT-5-mini fall from 100% to 80% at exactly half depth and Claude-4.5-haiku drop to 50% on logical inference in the 20-60% stretch, while Gemini-2.5-flash and Deepseek-v3.2 held steady. Position of the question matters even more: moving the task from the end of a long context to the middle took one model from 96% to 8% accuracy (task-position study, 9 models).

Why it happens

• Attention is skewed to the edges. Models learned from texts where the important part is the introduction or the conclusion. In long reasoning the middle can get 3-5x less attention than the start and end (SELF-ANCHOR), and Lost-in-the-Later documents a separate blind spot for the end of the context.
• The last tokens drive generation. Whatever sits right before the answer gets priority. A question in the middle, surrounded by similar solved examples, gets confused with them: 76% of mid-position errors were numbers lifted from neighboring examples (task-position study).
• Causal reading. Each token only sees what came before it, so options or data placed before the question are read without knowing what is being asked (Prompt Repetition).
• Length itself costs reasoning. Even with perfect retrieval, models lost 13.9-85% accuracy as context grew; Llama-3.1-8B quoted the conditions for 970 of 1000 problems yet solved 24.2% fewer. The drop persisted when filler was whitespace and when distractions were masked (context length alone hurts).
• Scattered facts. Sparse facts across a long text are missed far more than the same facts grouped in one block (fact-distribution tests).

Fixes ranked by measured effect

1. Put the task last, or repeat it at the end. When the question has to sit mid-context, a copy at the very end restored accuracy to within ±4 points of the best position for all 9 models at 8K tokens (task-position study). Repeating it twice in the middle did not reliably help; the copy has to be last.
2. Repeat the whole prompt (non-reasoning models). Sending the prompt twice gave 47 wins and 0 losses across 70 model-benchmark tests; on finding a name in a 50-item list, accuracy rose from 21% to 97% with triple repetition (Prompt Repetition, Gemini, GPT, Claude, Deepseek). For long inputs, repeat only the question and instructions.
3. Give an explicit structure map. State how sections relate ("A leads to B and C under condition X") before the source. HELIOS reports accuracy rising from 45-71% to 94-96% on structured code tasks.
4. Split one broad request into narrow passes. Asking for four named categories instead of "all requirements" raised coverage from 61% to 92% (PEGS). For long checklists, sharding found a single pass truly checks only 15-30% of items; groups of 4-32 items, each with the full document, fix it.
5. Extract first, then reason on a short context. Ask for exact quotes of the relevant passages, then answer from those alone. On RULER QA with GPT-4o this added a modest +4% (68.4% to 72.4%); the authors expect larger gains on weaker baselines (Retrieve-then-Reason). A dense summary built by self-questioning did better than the full 11,000-token document, 0.80 vs 0.76 F1 (Chain of Summaries).
6. Tell the model to cover all parts evenly. The CK prompt pairs "use ONLY the provided text" with "draw on ALL parts evenly, especially the end" (Lost-in-the-Later). A cheap addition; the paper reports fewer hallucinations and better coverage.
7. Anchor long reasoning to the question. Plan-then-reason steps that restate the original question at each step keep multi-step analysis on track (SELF-ANCHOR).
8. Let capable models navigate instead of read. Frontier models got +2.7% more accurate searching split files than reading everything from the prompt (file-native context).

A combined prompt stack

Sources disagree on one point: the CK-prompt paper says main instructions go first, while the task-position study says the task goes last. They are compatible as a sandwich, which the long-context anatomy study recommends: instructions and output format before the document, separators around it, and the key requirement repeated after it.

1. Instruction, format and "use every part of the text, do not skip the middle".
2. Optional structure map or category list (one category per pass for large documents).
3. The document between --- separators, with relevant facts grouped where you control the layout.
4. The task or question again, last.
5. For very long inputs, run an extraction pass first and feed only the quotes into the answering prompt.

What does not work

• Trusting the advertised window. ChatGPT-5-mini accepted 272k tokens but was reliable to about ~100k (effective-length tests).
• A blanket "don't make it up". At maximum length it cut ChatGPT-5-mini fact extraction from 89% to 72% through over-refusal, while Deepseek and Gemini lost only 1-2% (same study). Use it when faithfulness matters more than recall.
• Solved examples as separators. At 64K, one model lost 88 points of accuracy with solved problems around the task and none with neutral Wikipedia text (task-position study).
• Padding instead of repetition. Filling the same length with dots had no effect; only repeating meaningful tokens helped, and repetition adds little for reasoning models that already reread the prompt (Prompt Repetition).
• Files for weak tool users. Some open-source models lost up to 22% when asked to search files instead of reading the prompt (file-native study).

{context_part_1}

{intermediate_instruction}

{context_part_2}

────────────────────────────────────────

Reminder, your task is: {question_or_instruction}

{a clear instruction — what to extract / analyze}

{critical constraints and requirements — if any}

{output format — structured}

IMPORTANT: {if completeness matters — say explicitly "read the whole document, don't skip the middle"}

{if faithfulness is critical — add: "If the information isn't there, say so. Don't make it up."}

---

{DOCUMENT / CONTEXT}

---

{repeat the key instructions — to exploit recency bias}

STEP 1 (first request — extraction):

{long_context}

Find and quote EXACTLY (word for word) every passage that mentions:
{search_criteria_list}

Copy each relevant passage in full. Don't make anything up, don't shorten anything.

---

STEP 2 (second request — solving):

Here is the relevant information:

{passages_extracted_in_step_1}

{your_question}