SAKE: Duplicating Information to Shield It from Distortion by Reasoning

SAKE only changes how the information is laid out — all in one prompt:

BLOCK 1: Knowledge stack (reverse chronological)
  [Most recently retrieved document]
  [Second most recent document]
  ...
  [First document]

BLOCK 2: Question
  [The user's original query]

BLOCK 3: Reasoning + documents step by step
  [Reasoning step 1] → [document 1]
  [Reasoning step 2] → [document 2]
  ...
  [Current reasoning step] → [latest document]

The final answer is generated after the whole structure.

Task: You're comparing the business models of two e-commerce companies for an investor pitch — you need a precise analysis grounded in public data.

Prompt:

RETRIEVED INFORMATION (read objectively):

DOCUMENT 2 (latest search):
Etsy in 2024: GMV $12 billion, fees 8-12%, 
monetized through seller advertising. Model — marketplace.

DOCUMENT 1 (first search):
Amazon in 2024: GMV $700 billion, its own fulfillment network, 
fees 5-7% + logistics. Model — hybrid retail.

---

QUESTION:
Which business model is more resilient to a demand slump: 
Amazon or Etsy?

---

REASONING:

STEP 1: To assess resilience, I need to understand the revenue 
structure and operational risks of each model. I need 
data on monetization and costs.

→ [DOCUMENT 1: Amazon in 2024...]

STEP 2: Amazon earns from fees + logistics and 
has invested in infrastructure. If demand falls — 
there's a risk of underused warehouses. Now let me compare with the competitor.

→ [DOCUMENT 2: Etsy in 2024...]

FINAL CONCLUSION:

Result: The model will produce a comparative analysis anchored in the numbers from the documents. Thanks to the duplication at the start, the GMV and fee data are not distorted by the hypotheses from Step 1 — the model first "read the facts clean", then applied them to the reasoning. Without SAKE, the model might bend the numbers to fit the "warehouse risk" logic while ignoring the upside of Etsy's ad-based monetization.

LLM weakness: When a model processes new information after a long chain of reasoning, the attention mechanism focuses heavily on the earlier tokens — on what the model has already "thought". The new data gets encoded through the lens of the accumulated context and loses its objectivity. The study showed that the longer the reasoning before the search, the higher the "interference strength" — the document is read not as a fact but as a continuation of the model's own thoughts.

LLM strength: Models handle duplicated information in different positions very well. If the same text appears twice, attention on the second occurrence can "anchor" to the first one — and use the clean, undistorted representation.

How SAKE exploits this: By placing the documents at the very start first (with no reasoning context), the model encodes them "as is" — semantic integrity is preserved. Then the same document after a reasoning step provides context for applying it — the model understands "why we searched" and "how it relates to the task". But attention automatically latches onto the first, clean representation — the facts don't get rewritten to fit the hypotheses.

Control levers in SAKE:

• Order in the knowledge stack (reverse chronological): the freshest document goes first, to minimize the influence of even the other documents. You can order by importance instead of chronology.

• Duplicating all documents vs. only the key ones: if you're short on tokens, duplicate only the critical information (numbers, facts) and leave the rest only after the steps.

• Explicit block separators (---, headings): reinforce the structure and help the model "switch" between reading modes (objective → contextual).

• An instruction at the top of the stack: add "read objectively, without interpretation" before the documents — it strengthens the semantic anchor.

REFERENCE INFORMATION (facts without interpretation):

{document_N — most recently retrieved}

{document_N-1}

...

{document_1 — first retrieved}

---

TASK:
{question or goal}

---

REASONING AND APPLICATION:

STEP 1: {your thinking, logic, hypothesis}

Source to check against: {document_1}

STEP 2: {develop the thought taking document_1 into account}

Source to check against: {document_2}

...

STEP N: {final logic}

Source to check against: {document_N}

---

CONCLUSION:
{final answer grounded in the facts}

How to fill it in: - {document_1...N} — any external data: articles, numbers, quotes, search results - Document order at the top: reverse (the most recently retrieved goes first) - After each reasoning step, insert the document in forward order (as you used it)

🚀 Quick start — paste into the chat:

Here is the SAKE template for protecting information from being distorted by reasoning. 
Adapt it to my task: [describe the task — analysis, comparison, testing a hypothesis]. 
Ask me which documents/data I have and what order to put them in.

[paste the template above]

The LLM will ask: "Which documents or data do you want to analyze? In what order did you get them?" — because SAKE is built on placing the same information twice: a clean read at the start + contextual application step by step. It will take the duplication pattern and structure your content around it.

⚠️ Token cost: Duplicating documents doubles the context size — long texts can hit the window limit. SAKE pays off on hard tasks with long reasoning, but for a short request the overhead isn't worth it.

⚠️ The effect shows up in multi-step reasoning: If the task is solved in one step without iterative search, SAKE won't give you an edge. It's most valuable where there are several "reasoning → search → reasoning" cycles.

⚠️ Built for search scenarios: The technique is designed for working with retrieved information (documents, web search, knowledge bases). It doesn't apply to creative tasks with no external facts (brainstorming, storytelling).

The team took three reasoning models (Qwen3-4B-Thinking, Qwen3-30B-A3B-Thinking, QwQ-32B) and tested them on multi-hop QA benchmarks: HotpotQA, 2WikiMultiHopQA, MuSiQue, FRAMES, GAIA. The experiment was simple: they varied the length of reasoning before the search (pre-search reasoning) and measured how well the model used the retrieved documents.

The key finding: with oracle search (where the documents are guaranteed to contain the answer), performance dropped as pre-search reasoning grew. On 2WikiMultiHopQA the F1 score fell by 10-20 points with long reasoning — even though the right answer was sitting in the documents! That proved KID: a model can't integrate knowledge if it has been reasoning for a long time beforehand.

Then they ran a mechanistic analysis: they measured attention interference strength — the share of attention that document tokens give to the earlier reasoning instead of the document itself. The chart showed a sharp rise in interference as pre-search reasoning increased: the model "reads" the new information through its old thoughts and distorts it.

SAKE was tested on the same benchmarks — up to +37.6% improvement in F1 on hard bridge-comparison questions. The attention analysis showed that interference strength goes down — the duplication at the start works as an anchor, and the model latches onto the clean representation.

The surprise: the Search-o1 baseline (the state-of-the-art "reason → search → reason" approach) sometimes lost even to plain RAG, precisely because of KID — long reasoning hurt instead of helping. SAKE didn't just improve Search-o1, it also showed that "more reasoning" on its own, without protection against KID, can do harm.

Input sequence at reasoning step t+1:

X_{t+1} = [I; [d_t; ...; d_1]; q_0; r_1; d_1; ...; r_t; d_t]
          └──────┬──────┘              └────────┬────────┘
          Knowledge Stack St      Reasoning-aligned traces

Where:
- I: task instruction
- St = [d_t, d_{t-1}, ..., d_1]: cumulative knowledge stack (reverse chronological)
- q_0: original query
- r_i: reasoning at step i
- d_i: search result at step i

Context: The researchers formalized SAKE as an explicit structure for the input sequence. The knowledge stack St is placed right after the instruction and contains all retrieved documents in reverse order (the freshest first). Then comes the original question, followed by "reasoning → document" pairs in forward order. This dual positioning lets the model encode the knowledge twice: unbiased (in the stack) and contextualized (after the steps).

💡 Adaptation for fact-checking a text:

When you're writing something with facts in it (an article, a report, a pitch) and want an LLM to check the accuracy, use SAKE in reverse: first the clean draft as the semantic anchor, then the sources to check against, then iterative edits.

ORIGINAL TEXT (objective version, no edits):
{your full draft}

---

SOURCES TO CHECK AGAINST:
{statistics, links, data}

---

STEP-BY-STEP CHECK:

Claim 1 from the text: "{quote}"
Checking against: {relevant source}
Verdict: [correct/inaccurate/fix]

Claim 2 from the text: "{quote}"
Checking against: {relevant source}
Verdict: [correct/inaccurate/fix]

---

FINAL TEXT:
{corrected version, keeping the original style}

Same logic: the original text is encoded "clean" and isn't distorted by the checking process. When the model delivers its verdicts, it holds onto the original wording instead of rewriting the meaning to fit the sources.

🔧 Technique: an explicit instruction for each block → control over the reading mode

[MODE: OBJECTIVE ENCODING — read the facts without connecting them to the task]

DOCUMENT 1: {data}
DOCUMENT 2: {data}

---

[MODE: TASK-ORIENTED REASONING — apply to the task]

TASK: {question}

STEP 1: {reasoning}
→ Linking to: [DOCUMENT 1]

STEP 2: {reasoning}
→ Linking to: [DOCUMENT 2]

Effect: the model explicitly switches between "read as is" and "apply in context". It sharpens the split between the semantic anchor and reasoning contextualization.

🔧 Technique: remove the duplication → measure SAKE's contribution

If you're not sure SAKE makes a difference on your task, run an A/B test: - Version A: full SAKE (with duplication) - Version B: reasoning-aligned only (documents after the steps, no stack at the start)

Compare the results. If there's no difference, your task is either too short (no long reasoning) or doesn't demand factual accuracy (creative work, opinions).

💡 Extrapolation: SAKE + Chain-of-Thought for debugging reasoning

Combine SAKE with explicit CoT tags so you can see where the model "lost" a fact:

FACTS (anchor):
{document_1}
{document_2}

---

TASK: {question}

---

<think>
STEP 1: {hypothesis}
Checking against: {document_1}
Extraction: {what I took from the document}
Confidence: {high/medium/low}
</think>

<think>
STEP 2: {development}
Checking against: {document_2}
Extraction: {what I took}
Confidence: {rating}
</think>

<answer>
{final answer}
</answer>

The <think> tags expose the intermediate steps, and Extraction shows explicitly what the model took from each document. If a fact comes out distorted at the end, you can see at which step things went sideways.

Knowledge Integration Decay in Search-Augmented Reasoning of Large Language Models
Sangwon Yu, Ik-hwan Kim, Donghun Kang, Bongkyu Hwang, Junhwa Choi, Suk-hoon Jung, Seungki Hong, Taehee Lee, Sungroh Yoon
Seoul National University, Samsung SDS
Preprint, February 2026