TL;DR
When a task is phrased loosely, the model doesn't ask. It quietly picks one of the possible interpretations and confidently delivers a result. The pipeline runs, the output looks right, the mistake is invisible. This is called unflagged misframing, or silent reframing: the task got solved, just not the one you meant.
The user's pain point: you get a polished, detailed answer and think it's all good. But the model may have solved a different problem. Not "I failed," but "I succeeded, just not at what you had in mind." There's no error signal at all. In the study, on data-prediction tasks, the model picked the wrong target 39–63% of the time, and never once said so.
The key finding: a single clarifying question before the work starts recovers most of the lost quality. But the model doesn't know on its own when to ask: let it ask freely and it will ask even when everything is clear; forbid it and it will silently guess.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
The Problem and the Fix
WITHOUT CLARIFICATION:
Task (vague) → Model silently picks an interpretation
→ Polished result ← The mistake is invisible
WITH CLARIFICATION (one question):
Task (vague) → Model asks ONE question
→ User clarifies
→ The right task → The right result
Both versions run in the same chat. The only difference is whether the prompt explicitly invites the model to ask.
Example
Task: You're the founder of a small EdTech company. You need to prepare product material for a meeting with a potential corporate client: a B2B sale of employee training.
You write: "Write a description of our product for the client meeting."
Without a clarifying question, the model quietly decides: is this an investor pitch? A marketing one-pager? A technical overview? A script for a sales rep? It doesn't know, yet it delivers one of them with total confidence. And you might not notice.
Prompt:
Help me prepare material about our product for a meeting with a corporate client.
Product: an internal employee training platform.
Client: HR director at a manufacturing company, ~500 people.
Meeting: first call, 30 minutes.
If anything in this task can be read more than one way, ask me ONE
clarifying question before you start. Only the most important one.
If everything is clear, start right away.
Result: The model either asks a question right away (for example: "What do you need at the end: talking points for the conversation, or a document the client keeps?"), or starts if the wording is precise enough. In the first case you clarify and get exactly what you need. Without the invitation to clarify, you'd get whatever the model chose.
Why It Works
An LLM generates the next token based on context. It doesn't pause in front of a vague task and doesn't "notice" ambiguity as such. When a prompt allows several interpretations, the model simply picks the most likely one according to its patterns and keeps going. It's not a bug, it's the mechanics.
The study found one more unpleasant detail: the silence is an active choice, not an accident. Even when the model's reasoning trace contained something like "the task is ambiguous," that did not show up in the final answer in 96% of cases. The model noticed the ambiguity "inside" but never said it out loud.
An explicit invitation to ask changes the behavior. When the prompt says "ask one question if something is unclear," the model gets a pattern to follow. It stops guessing your intent and asks for it directly. One question, because that's the balance: several questions slow you down, zero questions risk a silent mistake.
Control levers in the prompt: - "One question" → sets a limit, so the model picks the most critical one instead of bombarding you - "If everything is clear, start right away" → gives permission not to ask on clear tasks - Add "state the assumptions you made" at the end → a retroactive audit of the interpretation
Prompt Template
{Task}
If anything in this task can be read more than one way, ask me ONE clarifying
question before you start. The most important one.
If everything is clear, start right away.
What to fill in:
- {Task} is your task, as concrete as possible: what you're doing, for whom, and in what format you need the result
Stronger version, with a retroactive audit:
{Task}
If anything in this task can be read more than one way, ask me ONE clarifying
question before you start. The most important one.
If everything is clear, start right away.
When you're done: state in one line which assumptions you made
when interpreting the task.
The assumptions add-on is especially useful when you don't know whether the task was vague: the model itself reports where it had to choose.
🚀 Quick start: paste into the chat:
Here is a template for clarifying a task before doing it.
Adapt it to my task: {your task}.
Ask me questions so you can fill in the template correctly.
[paste the template above]
The LLM will ask about the specifics of your task and the output format, because it needs to understand what exactly might be ambiguous in your case to phrase the right clarifying question.
Limitations
⚠️ The calibration problem: No prompt policy makes the model both ask on vague tasks AND stay quiet on precise ones. "Ask freely" → unnecessary questions on obvious tasks. "Ask rarely" → silence where clarification is needed.
⚠️ Only one question: The study tested exactly one question before the work starts. That helps with predictable ambiguity. When a task is complex and needs a real conversation, one question may not be enough.
⚠️ It doesn't fix execution: "One question" removes the wrong interpretation of the task, but it doesn't solve the "default strategy" problem (degenerate outputs) of weak models: they keep producing template answers even after clarification.
⚠️ The study is about data agents: The experiments used prediction tasks on tabular data (ML agents). The principle carries over to any chat, but the specific numbers (39–63%) come from data-science agents, not general chat.
How It Was Studied
A team from Novo Nordisk took 112 tasks from two classic ML benchmarks, DSBench (tabular prediction) and MLE-bench (Kaggle competitions), and built a pair for each: the original task and a deliberately vague version. The vagueness was controlled: they removed only one key parameter, either the target variable (what exactly to predict) or the evaluation metric (how quality is measured). Everything else stayed the same.
Then five models of different tiers (from lightweight to frontier) ran each task in three modes: the full version, the vague version with no way to ask, and the vague version with the option to ask one question. Interestingly, frontier models (GPT-5.4, Gemini 3.1 Pro) made the mistake no less often than lightweight ones, and in some cases even more often. Model size doesn't protect you from silent reframing.
The key point: the researchers separately inspected the models' reasoning traces and found that models noticed the ambiguity in their internal reasoning but didn't surface it in the answer in 96% of cases. That's what makes the problem so sharp: the silence isn't carelessness, it's a behavior pattern.
Original From the Paper
Description of the Ask condition (clarifying question) from the benchmark:
Ask is the ambiguous variant with the clarification oracle described in
Section 4.4: before modeling, the agent may ask exactly one self-contained
question under 50 words, or output NONE, and then stops until the oracle
answer is returned.
Context: This is an experimental condition: the agent was allowed to write one question before starting the work. This condition gave the biggest quality gain: the model stopped silently guessing and got precise information about the task.
Adaptations and Extrapolations
💡 Adaptation: auditing an answer you already have
If you already have an answer and aren't sure the model understood the task correctly:
Look at your previous answer.
Which interpretation of my task did you choose?
Were there other possible interpretations?
If so, which one did you reject and why?
This is the retroactive version of "one question": useful when you can tell the answer is "off" but don't know why.
🔧 Technique: forced declaration of assumptions
Instead of a question BEFORE, demand a report AFTER:
{Task}
At the start of your answer (1-2 lines), list the ambiguous points
you found in the task and how you decided to interpret them.
It doesn't slow the model down, but it makes visible the choices it made. Especially useful for long or complex tasks where stopping before the start gets in the way.
🔧 Technique: a two-phase prompt for high-stakes tasks
For tasks where a mistake is expensive:
Phase 1:
Read the task below. Don't start doing it.
List every ambiguous point: what can be read more than one way,
which choices need clarification.
Task: {task}
Phase 2 (after the model replies):
Good. Here are my answers on the ambiguous points: {your clarifications}.
Now do the task.
This is a deliberate two-phase structure: first a map of the ambiguities, then the execution. It costs more time, but it's more reliable for strategic tasks.
Resources
Ambig-DS: A Benchmark for Task-Framing Ambiguity in Data-Science Agents
Josefa Lia Stoisser, Marc Boubnovski Martell, Sidsel Boldsen, Kaspar Märtens, Robert Kitchen
Novo Nordisk, London, UK
Preprint, 2025