Parallel-Distill-Refine: Short Parallel Drafts and a Compact Summary Instead of One Long Chain of Thought

ROUND 1 (Parallel): generate several drafts at once (e.g., 8 or 16)
    |
DISTILL: compress them into a small, bounded workspace
   - a global summary of all drafts, or
   - the top-k drafts (k = 2-4)
    |
ROUND 2 (Refine): new drafts, each conditioned on the workspace
   "Treat the summary as unverified; use it as context"
    |
... repeat for a few rounds, fewer drafts each round ...
    |
FINAL ROUND: a single answer

Sequential Refinement (SR) = the same loop with one draft:
answer -> refine -> refine -> ...

In the paper, this loop runs in code: the model is called many times, and a fixed operator builds the workspace. In a regular chat you can approximate it by hand (our adaptation, see below): run the same request in several chats, ask for a summary, then ask for a refined answer.

Task (our illustration, not from the paper): you need a correct answer to a counting problem and don't fully trust a single run of the model.

Step 1: Drafts in parallel. Send the same prompt in 4 separate chats (or regenerate 4 times):

How many four-digit numbers have digits that add up to 9?
Reason step by step and put your final answer in a box: [answer].

Step 2: Distill. In a new chat, paste the 4 solutions and ask for a short summary. The paper does not publish its summary prompt; this wording is ours:

Below are 4 independent solutions to the same problem.
Write a short summary (under 200 words) for another solver:
- the answers each solution reached
- the approaches used, and where they agree or differ
- any step that looks wrong or unchecked
Do not choose a final answer yourself.

Problem: {problem}

<solution_1> ... </solution_1>
<solution_2> ... </solution_2>
<solution_3> ... </solution_3>
<solution_4> ... </solution_4>

Step 3: Refine. In another new chat, give the problem plus the summary. This prompt follows the paper's refinement prompt closely:

Solve the following problem efficiently and clearly.
Reason step by step and put your final answer in a box.

Problem: {problem}

Here is a summary of the reasoning traces by a few other solvers:
<summary>
{summary from the previous step}
</summary>

Treat the summary as unverified; and use the summary as context
to come up with an answer.

Result:

The refined answer should be built on what the drafts agreed on and should resolve where they disagreed, instead of starting from zero. If the drafts disagree (say, 165 vs 220), the summary makes the disagreement visible, and the refinement step has to check which counting step went wrong. This is our expectation of how it plays out; the paper measured the method on AIME problems with checkable answers, averaging 16 runs per question.

Short calls, accumulated evidence. A long chain of thought keeps everything in one ever-growing context. PDR replaces that with a bounded workspace: each round only sees the problem plus a compact summary. The paper's conclusion: "evidence accumulation via bounded summaries can substitute for long reasoning traces while holding latency fixed."

Parallel drafts give diversity. Several independent attempts explore different approaches. The distillation step keeps what is useful across them (agreement, promising partial results, likely mistakes) and drops the rest, so the next round starts from a better position than any single draft.

"Treat it as unverified" forces checking. The refinement prompt tells the model not to trust the summary blindly. This matters: the paper found that when only incorrect candidates were admitted into the workspace, accuracy dropped sharply, more for o3-mini than for gemini-2.5-flash, which the authors read as weaker self-verification. The method is only as good as the model's ability to verify, refine and produce diverse drafts.

Control knobs (from the paper's experiments):

• Drafts per round: the paper tried 8 drafts then 1; 16, 8, then 1; and 16, 8, 4, then 1, carrying k = 2-4 candidates forward.
• How to distill: a per-draft top-k selection and a global summary consistently beat a shared top-k selection and a random pick, and the gap grew with larger thinking budgets (exception: for o3-mini on AIME 2025, the global summary was best).
• Error analysis in SR: asking the model to analyze its previous answer's errors before refining helped o3-mini (AIME 2025: 73.1 to 77.9) but not gemini-2.5-flash (AIME 2024: 88.8 to 87.7).
• Which budget you care about: if latency matters and you can run calls in parallel, PDR; if total token cost matters, SR can be as good or better (it was on the total-compute frontier for gemini-2.5-flash).

Sequential Refinement (one chat, cheapest)

{task}

Here is a candidate response from a previous attempt:
<model_response>
{previous answer}
</model_response>

Treat the response as unverified; and come up with a better answer
without starting from scratch.

The last sentence is the paper's own wording. Repeat a few times, pasting the newest answer each time.

Parallel-Distill-Refine (by hand, our adaptation)

STEP A - in {N} separate chats (or N regenerations), send:
{task}

STEP B - in a new chat, send:
Below are {N} independent attempts at the same task.
Write a summary of at most {length} for another solver:
the answers reached, the approaches, where they agree or
disagree, and any step that looks wrong or unchecked.
Do not choose a final answer.
Task: {task}
<attempt_1> ... </attempt_1>
...
<attempt_N> ... </attempt_N>

STEP C - in another new chat, send:
{task}
Here is a summary of the reasoning by a few other solvers:
<summary>{summary}</summary>
Treat the summary as unverified; and use the summary as
context to come up with an answer.

What to fill in:

• {task} — the full problem, the same in every chat
• {N} — number of parallel attempts (the paper used 8-16 per round; 3-5 is a practical number by hand)
• {length} — a hard limit on the summary (e.g., 200 words), so the workspace stays bounded
• {summary} — the summary from Step B

🚀 Quick start: paste into the chat:

Here's a Parallel-Distill-Refine template (draft in parallel -> summarize -> refine).
Adapt it to my task: [describe the task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what the task is, whether it has a checkable answer, and how many attempts you can afford; it needs that to set the number of drafts and the summary length. It will take the draft-summarize-refine structure from the template and adapt it to your case.

⚠️ Tested only on competition math: all results are on AIME 2024 and AIME 2025, problems with a single checkable answer, using two API models (o3-mini, gemini-2.5-flash) and one 8B model. The paper lists coding, planning and other domains as future work; using it for estimates, writing or analysis is our extrapolation.

⚠️ Wrong drafts poison the workspace: admitting only incorrect candidates into the summary caused large drops in accuracy for all models. If most of your drafts are wrong in the same way, the summary can push the final answer toward the shared mistake.

⚠️ More total compute: PDR wins at the same sequential budget (latency). It spends extra tokens on drafts that are later discarded. At the same total budget, SR was the better option for gemini-2.5-flash, while PDR stayed ahead for o3-mini.

⚠️ By hand it is noisier: the paper ran the loop in code with fixed operators and averaged 16 runs per question. A manual version in chat with a few attempts will vary from run to run, and the summary step is your own prompt, not the paper's.

The team (Meta Superintelligence Labs with UCL, Mila and Princeton) tested two models: o3-mini (medium reasoning effort) and gemini-2.5-flash, on AIME 2024 and AIME 2025. Each score is the average over 16 independent runs per question. They compared long chain of thought, SR and PDR under two budgets: sequential (tokens along the accepted path, a proxy for latency) and total (all tokens, including discarded drafts).

Main result: at a matched sequential budget, PDR formed the best accuracy-per-latency frontier. For o3-mini at about 49k sequential tokens: AIME 2024 76.9 to 81.5 to 86.7 (long chain of thought, SR, PDR); AIME 2025 73.5 to 77.1 to 82.9. In one comparison at the same total budget of 442k tokens, SR scored 90.4 with all 442k on its sequential path, while PDR scored 90.6 with only 172k, a 2.57 times smaller sequential budget.

What goes into the workspace matters. They compared four ways to distill: a global summary, top-k per draft, a shared top-k, and a random k. Per-draft top-k and global summary won, and the gap grew with bigger budgets. In "oracle" tests, feeding only correct candidates into the summary improved results, while feeding only incorrect ones caused a pronounced drop, worse for o3-mini.

Training for the loop. Finally, they trained an 8B model (Llama-3-style) with reinforcement learning on a 1:1 mix of ordinary long-reasoning rollouts and one-round PDR rollouts. At a 65,536-token sequential budget, running PDR on top, the continually trained model reached 80.8 on AIME 2024 and 70.4 on AIME 2025, vs 75.8 and 65.8 for ordinary RL: +5.0 and +4.6 points. Its plain long-reasoning scores did not drop (70.0 vs 67.5 on AIME 2024).

Context: These are the paper's refinement prompts (Appendix B). The first is used in Sequential Refinement, the second in PDR when the workspace is a summary. The paper does not print the prompt used to write the summary itself.

SR refinement prompt:

Solve the following math problem efficiently and clearly. Please reason step by
step, and put your final answer within $\boxed{answer}$. Where [answer] is just
the final number or expression that solves the problem.

Problem: {{ problem }}

Here is an example candidate response wrapped in angle brackets:

<model_response>
# Solution response from previous turn
</model_response>

Treat the response as unverified; and come up with a better answer without
starting from scratch.

PDR refinement prompt (summary workspace):

Solve the following math problem efficiently and clearly. Please reason step by
step, and put your final answer within $\boxed{answer}$. Where [answer] is just
the final number or expression that solves the problem.

Problem: {{ problem }}

Here is a summary of the reasoning traces by a few other solvers:

<summary>
# Summary of all solutions from the previous iteration
</summary>

Treat the summary as unverified; and use the summary as context to come up with
an answer.

💡 Adaptation: three chats for a numeric estimate

Outside math (our extrapolation, not tested in the paper), the same pattern works for any question where independent attempts can disagree:

I need a reliable estimate: {question, e.g., "how many support tickets
should we expect in December?"}.

Data: {your numbers}

I will ask this in 3 separate chats and paste the answers back here.
Now: summarize the 3 answers below in at most 150 words - the
estimates, the assumptions behind each, where they disagree.
Then, treating the summary as unverified, give your best estimate
with the reasoning.

<answer_1> ... </answer_1>
<answer_2> ... </answer_2>
<answer_3> ... </answer_3>

🔧 Technique: SR with error analysis

The paper's error-analysis variant of SR helped o3-mini but not gemini-2.5-flash, so try it and compare:

Here is your previous answer:
<model_response>
{answer}
</model_response>

Treat the response as unverified. First list the specific errors or
weak steps you can find in it. Then come up with a better answer
without starting from scratch.

🔧 Technique: a verification gate before distilling

Because incorrect drafts in the workspace were the biggest failure mode in the paper, add a quick check before summarizing (our addition):

Below are {N} drafts. Before summarizing, check each one:
- Does its final answer follow from its own steps?
- Can any step be verified quickly (plug the answer back in,
  estimate the order of magnitude, recount a small case)?
Keep only the drafts that pass, then summarize them for the next round.
If none pass, say so instead of summarizing.

Pro: fewer wrong drafts reach the summary. Con: the model's own checks can be wrong too, so keep checks concrete (plug the answer back in, recount a small case).

• Rethinking Thinking Tokens: LLMs as Improvement Operators (arXiv 2510.01123, v1, October 2025)
• Authors: Lovish Madaan, Aniket Didolkar, Suchin Gururangan, John Quan, Ruan Silva, Ruslan Salakhutdinov, Manzil Zaheer, Sanjeev Arora, Anirudh Goyal
• Meta Superintelligence Labs; University College London; Mila, Université de Montréal; Princeton University
• Benchmarks: AIME 2024, AIME 2025 (mean over 16 runs)
• Models: o3-mini, gemini-2.5-flash; an 8B model trained with operator-consistent RL