TL;DR
When you ask a model to generate several answer options and pick the best one, it barely works for creative, expert, and open-ended tasks (a legal opinion, a medical consultation, copy for a client). The judge model that's supposed to evaluate the options and pick a winner guesses almost at random. But if you ask the model to merge the best pieces of all the options into one final text, quality goes up noticeably.
The thing is, there are plenty of high-quality options among the generated ones; the researchers checked this separately and confirmed it. But when it comes to picking one of them as "the best," an LLM judge or a reward model (a model that scores text) performs about as well as a coin flip. The reason is simple: reward models are trained to tell "right/wrong" apart on math and code, where there's a single correct answer. On open-ended tasks, where quality is a matter of nuance rather than fact, these models don't understand what they're supposed to evaluate.
The fix: don't pick, fuse. Instead of the prompt "here are 5 options, pick the best one," you need "here are 5 options, synthesize one final answer from them, taking the best from each." It's the only strategy tested that reliably delivers an improvement across every type of open-ended task, even though it doesn't squeeze out all the potential the options contain.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1: Generate N different answers to the task → N drafts
STEP 2: Ask the model to SYNTHESIZE (not pick!) a final answer,
combining the strengths of all the options → 1 final text
You can do both steps in a single chat: first ask for N options, then, in a separate message, ask for the synthesis.
Example in Action
Task: A lawyer is preparing an opinion on a contested case for a client. The situation is ambiguous, there's no single correct text, and the nuances of the argument matter.
Prompt:
You are an experienced lawyer. Here is the case description: {description of the situation}.
Write 4 different versions of a legal opinion on this case.
Make the versions differ in their approach to argumentation and in how they're structured.
Number them: Version 1, Version 2, Version 3, Version 4.
Then, in a new message:
Now review all 4 versions above. Don't pick one of them as "the best."
Instead, write a FIFTH, final version of the opinion that
combines the strongest wording, arguments, and structural choices
from all four drafts.
Result: In the first message the model produces 4 different opinions that differ in emphasis and argument structure. In the second, you get a synthesized text that's noticeably more precise and complete than any single version, because the model pulls the strong parts out of each draft instead of guessing which draft is "right" as a whole.
Why It Works
LLMs are bad at comparing and ranking whole texts on an abstract criterion like "quality" when no correct answer exists. The model's internal "judge" (and the specialized reward models trained for this) gets confused; they were trained mostly on tasks with clear-cut correctness, like math.
What LLMs are great at, though, is compiling and rewriting: taking several pieces of text and stitching them into a coherent document. That's close to what the model already does well: summarization, rewriting, merging sources.
The method sidesteps a weakness (poor ranking) by leaning on a strength (good synthesis). Instead of asking the model "which is better?", you ask "how do I combine these?", and that's a fundamentally different and, for the model, easier task.
Control knobs: - Number of options (N): more options means more variety for the synthesis, but also more tokens. For simple tasks 3 is enough; for complex expert ones, 5-6. - Asking to "combine" vs. "pick": the key wording. Swapping one word changes the result dramatically. - Asking it to cite the source of each phrase ("note which version you took this wording from"): adds transparency, but it's optional.
Prompt Template
You are a {expert role}. Here is the task: {task description}.
Step 1. Write {N} different solutions to this task.
Make them differ in approach, structure, or emphasis.
Number them.
Step 2. Review all the versions. Don't pick one of them as the finished answer.
Write a final, {N+1}th version that combines the strongest
elements from each draft: the best wording,
arguments, and structural choices.
🚀 Quick start: paste this into the chat:
Here's a template for working with multiple answer options through synthesis.
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask how many options you need and what role/expertise to set, because that determines how varied the drafts are that the final text is assembled from.
Limitations
⚠️ Don't use it where it isn't needed: for tasks with a single correct answer (math, code, precise facts), plain best-of selection works fine. This effect applies only to open-ended, judgment-based tasks without one right answer.
⚠️ Self-critique is a risky strategy: asking the model to "critique and improve your own answer" several times in a row rarely helps and is unpredictable; on some tasks quality even drops. It's especially dangerous in precise expert fields (medicine, law), where an "improvement" can distort the facts.
⚠️ Synthesis isn't a perfect solution: even the best strategy (combining options) captures only about 40% of the quality potential that actually exists among the generated drafts. The method beats the others but doesn't fully solve the problem.
⚠️ Generating a big tree of options (step-by-step search) backfires: if at each step the model "prunes" weak reasoning branches based on its own internal score, all the final options end up nearly identical. Diversity disappears, and multiple attempts are pointless.
How They Studied It
The team tested five strategies for working with multiple answer options (selection via a scorer, two kinds of step-by-step search, self-critique, and synthesis) on five open-ended tasks: medical consultations, legal analysis, financial opinions, everyday chat, and creative writing. They tested on several models (including Qwen3.5 and OLMo3), and fairly equalized everything by tokens spent so the comparison would be fair; otherwise a method that simply generates more text would look "smarter."
To figure out how much quality is theoretically available in a set of options, the researchers computed an "oracle" score: the quality of the best of all the drafts according to an independent judge, corrected for scoring noise. The gap between that potential ceiling and what the method actually outputs revealed the main point: the potential is there, but there's almost no way to capture it.
The most intriguing part: reward models (specialized scoring models) showed a correlation with real quality of only about 0.12 out of a maximum of 1.0, which is close to random guessing. And two different reward models made almost exactly the same mistakes, which points not to a random bug in one model but to a structural problem with the "train a model to give scores" approach for tasks without a single correct answer.
Resources
Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck, by Davide Romano, Kanak Raj, Jerrod Parker, Daniele Giofrè (Thomson Reuters Foundational Research)