State Design: Three Principles for Multi-Step Tasks in LLMs

Three independent principles: use them separately or together:

PRINCIPLE 1 — Running summary (Rolling Summary)
  Every step: compress the history into 1-3 sentences → pass it to the next request
  Instead of: [step1] + [step2] + [step3] + ...
  Pass: [compressed summary] + [current step]

PRINCIPLE 2 — Description format
  For most tasks → plain language
  For structured data (lists, tables) → JSON/lists
  BUT: structured formats only work in strong models (GPT-4o, Claude 3.5+)

PRINCIPLE 3 — Text map (Visualization of Thought)
  For spatial tasks → ask it to draw an ASCII map BEFORE answering
  The act of drawing the map = the act of spatial reasoning
  Important: it must BUILD the map, not just receive a ready-made one

Principles 1 and 2 work within one prompt. Principle 1 requires a separate request to update the summary at each step.

Task: You're launching a new product, an online course on personal finance. You're having a long conversation with Claude: discussing the audience, structure, pricing, name, and sales channels. After 20 messages the model starts getting confused: it suggests things you've already rejected and forgets key constraints.

Prompt (Principle 1 — running summary):

We're working on launching an online course. 
After each of my replies, update the block below — 
it must stay no longer than 5 lines 
and contain only what determines the next step.

=== CURRENT PROJECT STATE ===
[Paste the summary from the previous step here, or write "starting out"]
=============================

My question/task right now: {current question}

First update the STATE block — remove what's outdated, add what's new.
Then answer the question.

Prompt (Principle 3 — text map for a spatial task):

I need to arrange tables for a networking event in a 10×8 meter room.
What we have: entrance (left wall), stage (right wall), 
6 round tables (1.5 m diameter), a catering area (2×2 m in a corner).

First draw a text map of the room — 
use these symbols: E=entrance, S=stage, O=table, C=catering, .=empty.
Then, based on the map, propose a layout and explain the logic.

Result: With Principle 1, the model starts every answer with an updated state block: what's decided, what's open, what's rejected. It gets the next question with up-to-date context instead of the baggage of 20 messages.

With Principle 3, the model first draws a grid of the room with symbols, then gives a layout based on that map. The answer will be spatially grounded rather than a plain text list of "put this here, put that there".

The LLM's weakness on long tasks. The model has no memory of its own between requests; every time, it rereads the entire prompt. When the history is long, important details end up "in the middle" and the model literally loses sight of them. This is a well-documented effect: models do worst with information that's neither at the beginning nor at the end of the prompt.

Why a summary beats the full history. The full history is noise plus signal. A summary is just signal. At each step the model gets a compact, current snapshot of what matters right now. That lowers the chance it reacts to stale context. One important nuance: the model itself should write the summary, because it knows what matters for the task.

Why drawing a map beats a ready-made image. A paradoxical result from the study: the process of building a text map is more effective than receiving a ready-made image. When the model builds the map itself, it's forced to work through the space step by step: what's where, what the distances are, what's next to what. That is spatial reasoning. A ready-made picture doesn't require any of this; the model just "looks" and can get it wrong.

Control levers: - Summary length → set a limit in words or lines. Shorter = less noise, but a risk of losing details - Update frequency → you don't have to update every step; once every 5 messages works too - Map detail → do you need a rough sketch or a detailed grid? Specify the grid size - Summary format → a list of key decisions or a narrative? Match it to the task

Principle 1: Running summary

We're working on {task}. 
After each of my messages, update the STATE block — 
no longer than {number} lines, only what matters for the next step.

=== STATE ===
{initial context or "starting out"}
=============

Now: {current question or assignment}

Response order:
1. Update STATE
2. Answer the question

What to fill in: - {task}: "launching a course", "developing a strategy", "writing a book" - {number}: 3-7 lines, depending on how complex the task is - {initial context}: what's already known, or "starting out"

Principle 3: Text map

{Task description with spatial elements}.

Step 1: Draw a text map/diagram — 
use these symbols: {list of symbols}.
Add a legend.

Step 2: Based on the map, solve the task and explain the logic.

What to fill in: - A task involving the placement of objects: a floor plan, a route, a sequence, a structure - {list of symbols}: make them up yourself or ask the model to suggest symbols

🚀 Quick start: paste this into the chat:

Here is a template for handling multi-step tasks with a running summary. 
Adapt it to my task: {your task}. 
Ask me clarifying questions.

[paste the template above]

The LLM will ask how long the task will run, what needs to be tracked, and what format to keep the state in, because the summary has to contain exactly what's needed for your specific task.

⚠️ A bad summary is worse than the full history: If the model writes the summary sloppily and loses critical details, it starts making wrong decisions. That's worse than a long history. Check what goes into the summary in the early steps.

⚠️ Structured formats (JSON, matrices) are for strong models only: In GPT-3.5 or weak local models, a structured state format can make results worse. The safe default is plain text.

⚠️ Long horizons remain a weak spot: Even with the best state representations, almost all models degrade on tasks with 20+ steps and multiple subgoals. The method helps but doesn't fully solve the problem.

⚠️ VoT requires a capable enough model: Weak models draw maps with errors, and then a wrong map leads to wrong conclusions. Use it with GPT-4o, Claude 3.5+, and similar models.

A team from Leiden University ran a rigorous experiment: same tasks, same models, and they changed only the way the state was represented, nothing else. They used seven environments: Tower of Hanoi, Messenger (grid navigation), and five BabyAI levels of varying difficulty. They tested eight models, from small ones (LLaVA-3.8B) to large ones (Llama3.3-70B, Qwen3-VL-32B).

An interesting design detail: in the Messenger task, objects are named with different synonym words on each run (robot/scientist, sword/wand) to rule out memorizing the right answer. That made the test more honest.

The most unexpected result involved visualization. The expectation was that providing a real image would help the model with spatial tasks. It turned out the other way around: a text map the model built itself worked better than a ready-made picture. This suggests that spatial reasoning in LLMs isn't "look and answer" but step-by-step processing. Building the map is itself the reasoning.

Another insight: the summary helped almost every model in Tower of Hanoi (where the state is compactly described by the current disk configuration), but gave mixed results in Messenger (where tracking the relative distance to objects is critical, and some models lost that during compression).

Past trajectory:
Step 1: You took action stay. You (agent) don't have the message.
    You see:
    - bird 2 steps away
    - sword 1 step away
    - fish 3 steps away
Step 2: You took action east. You (agent) already have the message.
    You see:
    - bird 2 steps away
    - fish 4 steps away

After applying the summary principle (≤25 tokens), the same history looks like this:

Summary of past actions:
Moved east, collected message from sword; bird 2 steps away, fish 4 steps away.

Context: An agent solves a navigation task in a grid world: it needs to pick up a message and reach the goal while avoiding an enemy. At each step the prompt receives either the full history (top) or a compressed summary (bottom).

💡 Adaptation: a two-tier summary for complex projects

If the task has several independent tracks (for example, a product launch: marketing + product + operations), keep two levels of summary:

=== STRATEGIC CONTEXT (rarely changes) ===
Goal: {overall goal}
Key decisions: {what's been settled for good}
Constraints: {budget, deadline, the key NO}
==========================================

=== OPERATIONAL STATE (updated every step) ===
Currently working on: {current track}
Latest decision: {what we decided in the previous step}
Open questions: {what's still unresolved}
==============================================

My question: {question}

The strategic block is like a constitution. The operational block is like a desk. Splitting them lowers the risk that the model "loses" a constraint set at the start of the conversation.

🔧 Technique: Mandatory VoT (Visualization of Thought) before any answer

Applies to structuring tasks where there are dependencies between elements:

Before answering:
1. Draw a diagram/map of the dependencies as a text diagram
2. Mark on the diagram {what matters — the bottleneck / the critical path / the conflict}
3. Then give your answer

Task: {task}

The logic: a model that builds the dependency diagram itself sees conflicts and connections that stay implicit in text.

🔧 Technique: Summary quality check

If you're using the running summary principle for long work, add a checkpoint every 10 messages:

Only one task right now: review the STATE block.
What in it is no longer relevant? What important item is missing?
Propose a corrected version.

This "calibrates" the summary and protects against errors piling up during compression.

State Design Matters: How Representations Shape Dynamic Reasoning in Large Language Models

Authors: Annie Wong, Aske Plaat, Thomas Bäck, Niki van Stein, Anna V. Kononova

University: Leiden Institute of Advanced Computer Science, Leiden University

Code: https://tinyurl.com/state-representations-dyn

Related work cited in the paper: Visualization-of-Thought (Wu et al., 2024a), SmartPlay (Wu et al., 2024b), BALROG (Paglieri et al., 2025), Chain-of-Symbol (Hu et al., 2024), Reflexion (Shinn et al., 2023)