Task decomposition and prompt chaining: what the research shows

What task decomposition is

Task decomposition splits one hard request into narrow steps the model handles one at a time, either as numbered steps inside a single prompt or as a chain of separate prompts where each output feeds the next. Prompt chaining, least-to-most and plan-and-solve are versions of the same idea. The shapes that recur in the research:

• A fixed plan: named steps, each with one action and an explicit handoff.
• Collect, structure, analyze: gather broadly, put the data in a table, filter last.
• Extract, then apply rules: pull facts into JSON first, run the decision logic on the clean facts.
• Atomic checks: a set of yes/no questions instead of one holistic judgment.

It differs from chain-of-thought in who designs the route. In chain-of-thought the model writes its own reasoning; in decomposition you, or a separate planning step, fix the steps and the model executes them.

Why it works: one load per step

The explanation the papers converge on is interference between jobs. Reading raw text, holding a rulebook, filtering and computing compete for the same attention in one pass. In the O-RADS ultrasound study, giving eight LLMs a report plus all the classification rules in one prompt produced accuracy that swung from 65% to 96% depending on the model; extracting the facts into JSON first and applying the rules in a separate step put every model at 95-99%. The WEBDART web-agent study traces the same overload: agents fell from 46.6% on simple tasks to 8% on complex ones, and a collect, structure, analyze split raised GPT-4.5 from 21.6% to 35.3% on the hard set while matching the baselines on simple tasks.

Checklists show the effect most plainly. Given a 100-item checklist in one request, a model in the sharding study actually checked 15-30% of the items and marked the rest "compliant" by default. A larger reasoning budget did not fix this; splitting the checklist into groups of 4-32 items, each with the full document and a required quote, did. A useful side effect of narrow steps is that errors become locatable: you can see whether extraction or the rule went wrong.

Variants that tested well

1. A plan written into the prompt. In Routine, laying out the task as named steps, each with an action and an explicit "use the result of step N", lifted GPT-4o from 41.1% to 96.3% on enterprise tool-calling scenarios. The model stopped planning and executing at the same time.
2. Translate, then decide. TRACE has the model map free text onto canonical terms from a vocabulary, then applies explicit if-then rules in a second step; recall grew almost 1.5-fold, and each error points to either the vocabulary or a rule.
3. Atomic yes/no checks. Atomic Instructions replaced "pick 1 of 232 fallacy types" with 3-5 binary questions per option plus an "often confused with" pass. Claude Sonnet 4 went from 42.2% to 62.9%; ChatGPT-4o gained only 3.4% because it skipped the step-by-step checks, and a hierarchical category-then-type version fell to 25%.
4. A chain of single-purpose prompts. In an incident-response study, one comprehensive prompt gave generic advice in 98.3% of runs; a diagnosis, plan, risk chain on the same small model produced recommendations 80 times more specific.
5. Category passes for extraction. Splitting "extract all the requirements" into four PEGS categories raised coverage from 61% to 92% in the PEGS study.

When it helps and when it hurts: the seam tax

Decomposition earns its cost on multi-step tasks with several sources, filters or rules. On simple tasks it only adds latency and tokens, and the retrieve-then-reason study notes that it does not fit tasks needing the whole context at once, such as translating or summarizing a full document.

The main counter-evidence is the loss at the seams. The decomposition-tax study ran 21 open models through 3-4 step chains and found that steps seeing only the previous step's output lost up to 40 accuracy points, in one case from 55% to 15%. Re-inserting the original task right after the step that compressed the data recovered a median of about 63% of the loss, not all of it, and a single comprehensive prompt usually stayed more accurate. The fact-decay study found the same at handoffs between agents: a 32B model dropped 56-85% of the facts it had found when writing a free-form summary for the next stage, a stronger model 3-6%; a mandatory list of risk, justification and uncertainty items nearly eliminated the loss.

Structure also needs a model that follows it. In the AHP prompting study, the full multi-step instruction in one prompt made the weaker open models Mistral and Llama3 decide worse than with no structure, while GPT-4o-mini handled it. Rules of thumb from these papers: split only where one prompt demonstrably fails, pass the original input along with each intermediate result, make handoffs itemized lists rather than prose, and keep the chain short.

STEP 1 — Extraction:
You are an analyst who extracts facts from text. Read {text} 
and extract only the listed features into JSON. Do not draw conclusions 
and do not classify.

Features:
{feature_list_with_descriptions}

Text:
{text}

Output only JSON. If a feature is not found, null.

---

STEP 2 — Applying the rules:
You have structured data and classification rules. 
Apply the rules strictly in order and find the first one that fires.

Rules:
{rules_in_if_then_format}

Data:
{json_from_step_1}

Output: the final category and the rule that assigned it.

Here is the full context/document:
{document_text}

Check ONLY these items (the rest will be checked separately, don't worry about them):
{criteria_group}

For each item in this group:
1. Provide an exact quote from the document confirming or refuting the condition
2. Issue a verdict: passed / failed / no data in the text

--- (repeat for each group of criteria) ---

Final request:
Collect the verdicts from all groups into one table. Separately highlight items marked "failed" and "no data."

--- (optional, for reviewing material from someone with an incentive to persuade you) ---

For each item marked "passed": find a counter-argument — why the condition might actually NOT be satisfied. Compare the "for" and "against" arguments against the document text and issue the final verdict.

[Message 1]
Here is the task: {task_text}

{stage_1_instruction}. Be sure to preserve all relationships 
stated between the elements of the task — don't just list them separately.

[Message 2 — after the stage where data was compressed]
Original task (for reference): {task_text}

Result of the previous step: {stage_1_output}

{stage_2_instruction}

[Message 3 — final]
{final step instruction: formulate the answer based on everything above}