arXiv:2510.03223 ★ 77 Oct 3, 2025

SELF-ANCHOR: Steering an LLM's Attention Automatically Through Planning

Key result

Finding: LLMs badly lose focus in the middle of long reasoning: the original question and intermediate conclusions get many times less attention than the beginning and end of the context. The SELF-ANCHOR method keeps the model focused on the key parts of the task across the entire reasoning chain, even when the context balloons to thousands of tokens. The trick: the model breaks the task into plans itself, and then at each reasoning step it explicitly "anchors" its attention back to the original question and the current plan via a structured JSON format. Result: the model doesn't "forget" what it's supposed to solve, even at step 10 of the logic chain.

TL;DR

SELF-ANCHOR is a prompting method that automatically steers the model's attention while it reasons. The gist: the model first breaks the task into plans (plan steps), and then, as it generates each reasoning step, it explicitly "anchors" its attention back to the original question and the current plan. This prevents the classic problem where the model "forgets" key parts as the context grows.

The problem is that an LLM has a limited attention budget. In long reasoning chains (Chain-of-Thought, planning, iterative generation), intermediate steps and the original prompt get lost in the middle of the context. The model starts paying more attention to the latest tokens than to what actually matters for the task. Result: reasoning errors, drifting off topic, losing sight of the original goal. This is called attention misalignment: attention shifts to the wrong place.

The method works in two stages on every iteration: (1) generate the next plan with attention on the question, (2) generate the reasoning for that plan with attention on the question + the current plan. The process repeats until the final answer. On top of that, the strength of the attention "anchoring" is tuned dynamically: if the model is confident (high token probabilities), anchoring is relaxed; if it's unsure, anchoring is strengthened.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

Method Blueprint

ITERATION i:

STEP 1 (Planning):
Inputs: system prompt + question + all previous plans
Attention anchor: question
→ Generate: plan_i (the next solution step)

STEP 2 (Reasoning):
Inputs: system prompt + question + all previous plans + plan_i
Attention anchor: question + plan_i
→ Generate: reasoning_i (solving the current plan)

Repeat until the final answer

Both steps run in a single prompt (via a structured JSON format).

The full version (requires API access) also modifies the model's logits via the SPA (Selective Prompt Anchoring) mechanism to strengthen attention steering.

The simplified version (works in a chat) uses only the structured prompt, with no logit modification.

Example in Practice

Task: Decide whether a small software company should launch its new SaaS product for small businesses on the Shopify App Store

Prompt:

Break this task into a step-by-step plan and solve it, constantly returning to the original question and the current plan step. Format: JSON with "plan step" → "reasoning for that step" pairs.

Task: Should our company launch on the Shopify App Store with a SaaS product for small businesses (a CRM for retail, $29/month subscription)? Current base: 200 customers via direct sales. Competitors on the App Store: 5-7 similar apps priced at $25-49/month.

Return the answer in this format:
{
  "Step 1: [name]": "[reasoning]",
  "Step 2: [name]": "[reasoning]",
  ...
  "Final answer": "[conclusion]"
}

Result: The model will return structured JSON with 5-7 analysis steps: sizing up the App Store's target audience, analyzing competitors, working out unit economics, assessing risks, and checking that the product meets the marketplace's requirements. At every step the model will explicitly tie its reasoning to the original question and the current plan. At the end: a well-grounded conclusion with concrete conditions (e.g., "Yes, but only if...", "No, because...").

Why It Works

LLMs suffer from attention dilution in long contexts. The self-attention mechanism is physically limited: the more tokens, the less attention "weight" each one gets. By default, models look hardest at the beginning (attention sink) and the end of the context (recency bias), while the middle, which is exactly where the intermediate reasoning and the details of the original question live, gets too little attention. Result: the model can "forget" what exactly it's supposed to solve or lose an important intermediate conclusion.

SELF-ANCHOR plays to an LLM's strength: its ability to follow structured instructions and work with explicit links. Planning creates a clear hierarchy (question → plan → reasoning), and explicitly reusing key parts of the context (by repeating the question and the current plan) compensates for the weakness of the attention mechanism. The model doesn't have to "keep everything in its head" at once; it follows a structure that constantly reminds it of what matters.

Control levers for the prompt:

Output format (JSON vs. free text) → JSON reinforces the structure; free text is more flexible for unusual tasks
Anchoring frequency (every step vs. once every N steps) → less often for simple tasks, every step for complex ones
What to anchor (just the question vs. the question + all plans) → question + current plan only = the sweet spot; all plans = diluted attention
Anchoring strength (in the full API version) → dynamic tuning via the harmonic mean of token probabilities

Prompt Template

Break the task into a step-by-step plan and solve it, constantly returning to the original question and the current plan step. Format: JSON with "plan" → "reasoning" pairs.

Task: {your_task}

Return the answer in this format:
{
  "Step 1: [plan_name]": "[detailed_reasoning_for_this_step]",
  "Step 2: [plan_name]": "[detailed_reasoning_for_this_step]",
  ...
  "Final answer": "[final_conclusion]"
}

At every reasoning step, explicitly connect it to the original task and the current plan.

What to plug in:

{your_task}: the specific problem you need solved (a business decision, a situation analysis, a logic check, multi-stage calculations)

🚀 Quick start: paste this into the chat:

Here's a SELF-ANCHOR template for structured reasoning with automatic focus on the key parts. Adapt it to my task: [your task]. Ask me questions to clarify the details.

[paste the template above]

The LLM will ask about the specifics of your task (which factors to consider, how deep the analysis should go, what output format works for you); it needs this to break the task into plans correctly. It will take the "plan → anchored reasoning" pattern from the template and adapt it to your case.

Limitations

⚠️ Doesn't fix logic errors: The method keeps the focus on the right context but doesn't make up for a model's weak reasoning skills. 42% of the errors in the study were wrong conclusions even with attention focused correctly.

⚠️ Doesn't help with misunderstanding: If the model misread the task from the start (36.5% of errors), structuring won't save it. You need a clearer initial prompt.

⚠️ Overhead on simple tasks: Planning + a structured format add tokens. For tasks like "capital of France" it's overkill; use plain CoT or no special prompting at all.

⚠️ Calculation errors remain: 21.5% of errors were arithmetic slips. SELF-ANCHOR doesn't improve math skills, only how the process is organized.

How They Tested It

The researchers took 6 models (from 3B to 30B parameters: Llama-3.1/3.2, Phi-4/Phi-4-mini, Qwen3-4B/30B) and tested them on 6 benchmarks of varying difficulty: grade-school math (GSM8K), GMAT-level problems (AQuA), competition math (MATH), commonsense (StrategyQA), social reasoning (T4D), and a multi-task suite (BIG-Bench Hard). They compared against three baseline prompting methods (Chain-of-Thought, Plan-and-Solve+, Re-Reading) and five specialized reasoning models (DeepSeek-R1, Phi-4-reasoning, and others).

The key experiment: an ablation study (Table 4) showed that even without modifying attention via the API (i.e., just the structured prompt), the method delivers gains. For example, on Llama-3.1-8B: GSM8K +11%, T4D +8.5%, BBH +3.3% vs. plain CoT. That proves structured planning works on its own, and the logit modification is an extra boost.

The surprising finding: SELF-ANCHOR with non-reasoning models nearly catches up with reasoning models. For example, Phi-4-mini (4B, a regular model) + SELF-ANCHOR = 88% on GSM8K vs. 81% for Phi-4-mini-reasoning (a specially trained model). A gap of 5-7%, without the huge cost of RL training. That shows attention misalignment is a critical problem, and fixing it with prompting has an effect comparable to expensive post-training.

Dependence on difficulty: the method gives the biggest boost on medium- and high-difficulty tasks (0.5-1.0 on the difficulty scale). For easy tasks (<0.33 difficulty) the gain is smaller: the overhead of the structure outweighs the benefit. They also noticed that the harder the task, the longer the reasoning chain SELF-ANCHOR generates (consistent with findings from Wu et al., 2025: hard problems need more steps).

Original Prompt from the Paper

Context: This is how the model was prompted in the full version of SELF-ANCHOR (a simplified reader-friendly version is in "Prompt Template" above)

Decompose the given problem into step-by-step plan, reasoning the plan and solve the problem step by step in JSON format. For each plan step, provide a key-value pair: the key is the plan step, the value is the detailed reasoning for that step.

Sample output:

json

{
  "Step 1: Identify the pronoun": "The pronoun in the sentence is 'her'.",
  "Step 2: Determine the antecedent of the pronoun": "To find the antecedent, we need to look for the noun that 'her' refers to. In this case, 'her' is a possessive pronoun that refers to the analyst.",
  "Step 3: Analyze the sentence structure": "The subject of the sentence is 'The secretary', and the object of the verb 'helped' is 'her'",
  "Step 4: Identify the antecedent based on the sentence structure": "'her' refers to the analyst, not the secretary.",
  "Step 5: Choose the correct option": "Since 'her' refers to the analyst, the correct option is (B) Helped the analyst book a flight."
}
```

## Adaptations and Extrapolations

### 💡 Adaptation for reviewing legal documents

Same plan → reasoning structure, but focused on **finding contradictions**:
```
Check this contract for internal contradictions. Break the review into steps and analyze each step, constantly returning to the goal: finding inconsistencies.

[contract text]

Return in this format:
{
  "Step 1: Check the subject of the contract and the price": "[contradiction analysis]",
  "Step 2: Check deadlines and termination terms": "[contradiction analysis]",
  ...
  "Contradictions found": "[list with clause references]"
}
```

### 🔧 Technique: drop the JSON → get natural language while keeping the structure

If JSON feels too rigid, you can loosen it to "think out loud, but in a structured way":
```
Break the task into plans. At each step:
1. Write the name of the plan
2. Justify the solution in detail
3. Remind yourself of the original question before moving to the next step

Task: {your_task}

This gives more natural-sounding text but keeps the principle of "anchoring through explicit reuse of context".

Resources

Paper: "SELF-ANCHOR: Large Language Model Reasoning via Step-by-Step Attention Alignment" (Preprint, 2025)

Authors: Hongxiang Zhang, Yuan Tian, Tianyi Zhang (Department of Computer Science, Purdue University)

Full API version: Requires implementing the SPA (Selective Prompt Anchoring) mechanism via logit modification; see the original paper and code (to be released upon acceptance)

Simplified chat version: Uses the prompt pattern from this summary

Research digest

Key result

Principle

The process is iterative, and each iteration has two parts. First: the model generates the next plan (the next solution step) focused on the original question. Second: the model generates reasoning for that plan, keeping attention on both the question AND the current plan. The neat part: the model doesn't keep everything in its head like in regular CoT; it follows a rigid "plan → reasoning with an explicit link" structure. Both steps are packed into one JSON: `{"Step 1: [plan]": "[reasoning]", "Step 2: ...": "..."}`. Repeat until the final answer. Anchoring works through explicit reuse of key parts of the context at every iteration: the model literally sees the question and plan before generating each chunk of reasoning.

Why it works

LLMs suffer from attention dilution in long contexts; it's a physical limitation of the self-attention mechanism. The more tokens, the less "weight" each one gets. By default the model looks hardest at the beginning (the pull of the first tokens) and the end (bias toward the latest ones), while the middle of the context gets 3-5x less attention. That's exactly where the intermediate conclusions and details of the original question sit. The key insight: a structured format that explicitly repeats the key parts compensates for the weakness of the attention mechanism. The model doesn't rely on "memory"; it follows a clear hierarchy (question → plan → reasoning) that constantly reminds it of what matters. This uses an LLM's strength: following structured instructions.

When to apply

Multi-step analytical tasks → specifically business decisions, checking complex logic, and multi-stage calculations, especially when the reasoning chain runs longer than 5-7 steps and there's a risk the model will "lose the thread". Ideal for tasks like "decide whether to launch a product" (which requires analyzing the audience, competitors, economics, and risks). NOT suited for simple tasks: questions like "capital of France" or tasks solvable in 1-2 steps; planning just adds tokens with no payoff.

Mini recipe

1. Set the structure: in the prompt, write `Break the task into a step-by-step plan and solve it, constantly returning to the original question and the current plan step` 2. Output format: ask for JSON with `"Step N: [plan_name]": "[reasoning_for_this_step]"` pairs + a final block `"Final answer": "[conclusion]"` 3. Explicit link: add the instruction `At every reasoning step, explicitly connect it to the original task and the current plan` 4. Plug in the task: insert your specific problem in place of {your_task} 5. Run it: the model will create the plans itself and anchor its attention at every step through the structure

Examples

[BAD]: `Analyze whether to launch on the Shopify App Store with a SaaS product for small businesses (CRM, $29/month subscription). Competitors: 5-7 similar apps at $25-49` (The model will produce a 2-3 paragraph stream of consciousness, forget the specific numbers and conditions by the end, and give a vague conclusion) [GOOD]: `Break the task into a step-by-step plan and solve it, constantly returning to the original question and the current plan step. Format: JSON with "plan" → "reasoning" pairs. Task: Should we launch a retail CRM ($29/month subscription) on the Shopify App Store? Base: 200 customers via direct sales. Competitors on the App Store: 5-7 similar apps, $25-49/month. Return: {"Step 1: [plan]": "[reasoning]", ..., "Final answer": "[conclusion]"}. At every step, connect the reasoning to the original task and the current plan.` (The model will return structured JSON with 5-7 steps: App Store audience analysis, competitor assessment, economics, risks, compliance with requirements. Each piece of reasoning is explicitly tied to the question. The finale: a concrete conclusion with conditions)

In plain words

The trouble with every large language model is that it has the memory of a goldfish. As soon as a reasoning chain gets longer than a couple of paragraphs, the model starts to drift: it remembers perfectly what it wrote a sentence ago but completely forgets what the original question even was. That's just how the machinery works: the attention mechanism gets smeared across the context, and important details drown in intermediate rambling. The Self-Anchor method fixes this by forcing the model to keep "glancing back" at home base, so it doesn't turn into the grandpa who starts telling a joke and ends with a complaint about his knees.

Think of it as a trip to a giant big-box store with a shopping list. A regular model walks in, sees a sale on chips, gets distracted by the smell of fresh pastries, and ends up leaving with a cart full of junk and no bread. Self-Anchor is when you tape the shopping list right onto the cart handle and check it every two minutes. You don't just keep walking; you constantly "anchor" your moves to the original plan so you don't wander deep into the marketing traps. Technically you already knew why you came, but without the list in front of you you'd have blown it anyway.

Technically, the magic happens in stages: first the model sketches a plan of steps, and then, as it generates each chunk of the answer, it has to refer back to the original question and the current plan item. This is called Step-by-step Attention Alignment. Instead of hoping for the best, the prompt makes the model reread the core of the task before every new conclusion. If we're evaluating a SaaS launch on the Shopify App Store, the model doesn't just ramble about fees; at every step it checks: "OK, I'm on fees right now, but how does this answer the question about whether this company should launch?" That kills attention dilution, the nasty effect where the middle of the context turns into gray noise for the model.

The principle is universal and applies anywhere you need real thinking, not just text generation. It was tested on math and logic, but in practice it rescues any complex analysis, from legal audits to writing code. Anywhere the model risks "losing the thread" and starting to hallucinate, Self-Anchor keeps it on a leash. SEO is dying, GEO is being born, and with them comes the need for rigid structures like this, because an AI consultant that forgets the point of your request three minutes into the conversation is an epic fail.

Bottom line: if you want a sensible result from a model on long tasks, don't give it free rein. Use Self-Anchor: make it write out a plan and anchor its attention at every step. Either you control the model's focus, or you get confident nonsense in beautiful prose. 6 control elements and constant checking against home base are the only way to make an LLM actually reason instead of imitating a stream of consciousness. Anyone who doesn't learn to "train" attention like this will keep complaining that "the AI is dumb on hard tasks".

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on SELF-ANCHOR: Steering an LLM's Attention Automatically Through Planning arXiv:2510.03223

77/100

Good

Lab score for practical value

Who it’s for

Business analystProduct ManagerLawyerData ScientistMarketer

Apply this study to your task

Related techniques

Get the week’s best papers by email