Clarifying-question prompting: making the model ask before it guesses

What clarifying-question prompting is

Clarifying-question prompting means telling the model, in the prompt itself, to look for gaps in the task and deal with them openly before it does the work: ask a question, state its assumptions, or stop. Without that instruction, a model handed a vague task picks one reading and runs with it. It does not tell you a choice was made.

The studies describe this default in detail. In the silent reframing benchmark, data-science agents given a task with the target variable or metric removed picked the wrong target 39–63% of the time on prediction tasks and never said so. Their reasoning traces often noted that the task was ambiguous, yet that note did not reach the final answer in 96% of cases. In multi-turn chats, the Lost in Conversation analysis traces a 30-60% quality drop to the same habit: the model makes an early guess about your goal and keeps building on it, even after you clarify.

The technique has three forms, from lightest to heaviest:

• Ask: one question (or a short interview) before starting.
• Declare: start working, but list the assumptions you made about the task.
• Stop: for agents that take real actions, refuse to act and explain what is unclear.

Why asking first works

Models can spot gaps in a task when you ask them to. They just do not do it unprompted. Training rewards finishing the task, so "complete it" beats "check it" unless the prompt changes that priority. The entry-to-care study shows how shallow the default is. Models answering medical questions reacted to the tone of the first message, not the actual risk. A mild-sounding "feel nauseous" got immediate diet advice. An alarming "chest hurts" got questions.

An explicit instruction changes the order of operations. In the uncertainty decomposition prompt, the model writes a task-clarity score (0, 0.5 or 1) and an explanation before the action field. It can only act after that check. At a score of 0.5 or higher it asks one question instead of acting. Across five models and two agent benchmarks where half the tasks were deliberately vague, this beat the simple baseline by 73% in F1 on one benchmark and won on four of the five models.

Restating the request works for the same reason. When the model first writes down how it understood the task, its interpretation becomes visible, and you can correct it before the work instead of after. In the Mediator experiments, rewriting vague multi-turn requests into a clear instruction before execution recovered quality by 15-45% depending on the domain.

Variants that tested well

• One-question tail. The silent reframing study let the agent ask exactly one question under 50 words before starting. That condition gave the biggest quality gain. In chat, add "ask one question if anything can be read more than one way; if everything is clear, start right away".
• Collaborative dialogue. Give the model a partner role and tell it to ask questions first and go step by step. In a user study with ChatGPT-4o on programming tasks, this collaborative style led to faster problem-solving and higher satisfaction than the passive and proactive styles. A phased interview protocol with a mandatory "if my answer is vague, ask" rule raised users' trust in the result on a depression-screening questionnaire.
• Specified quit conditions for agents. Quit instructions list when the agent must stop: when it cannot rule out harm, lacks information about the task or the situation, or lacks the knowledge to judge risk. On 144 high-risk tool-use scenarios this gave +0.39 safety (0-3 scale) for -0.03 helpfulness. A vague "you can quit if you want" gave only +0.17.
• Ask-first order for advice. A short ask-then-advise instruction cut advice-before-questions from 9 out of 12 conversations to 0 and added a summary for the doctor in 10 out of 12.
• Marked gaps. Instead of waiting for the model to notice what is missing, show it. Explicit [unknown] fields in a personal-context schema target a trap the authors measured: hallucinations rose from 1% to 4–12% when the context was only partial. Hypothesis tags keep the model's own guesses out of a brief until you confirm them.
• Analysis before the answer. ContextLens has the model fill in intent, ambiguity, risks and an answer/clarify/refuse decision before it replies. That cut GPT-4o errors on tangled requests from 52% to 34%. Related variants ask about your expertise level, make the model compare your method with an alternative (84% right when choosing between two answers), and name who owns a trade-off so the model either asks or explains the principle it used. PAWNI turns the interview into a full 18-point brief.

When it helps and when it overshoots

It helps most when the task is underspecified and a wrong reading is expensive: agents taking irreversible actions, data analysis where the model silently picks a metric, advice in medicine, law or finance, and long iterative work. It adds nothing to tasks with a single reading. Every author makes the same caveat: translating a sentence or adding numbers does not need a question.

The hard part is calibration. The silent reframing authors found no prompt policy that made models both ask on vague tasks and stay quiet on precise ones. Free questioning over-asks; restricted questioning goes back to guessing. The counter-evidence comes from conversational roles. In the Move Framing study, LLMs playing therapist asked questions up to three times more often than real psychologists and kept asking after the client had said enough. Giving them a menu of reply types and a rule like "no more than two [inquiry] in a row" halved the gap with human behavior and raised agreement with therapists' choices by 7-9 percentage points. So a bare "ask questions" instruction can overshoot.

Other limits: open-weight models barely responded to quit instructions (quit rates under 9%, against 72% for Claude 4 Sonnet). Models also tend to rate a task as clearer than it is, so you may need to lower the trigger threshold. And the instruction fixes the order, not the depth: even with the ask-first prompt, the medical models never asked about diabetes in the vomiting case where that was the decisive fact. Two habits reduce these risks: add "if everything is clear, start right away" and the "do not stop if" exceptions from the quit prompt, and cap the number of questions. If the model guesses wrong anyway, see abstention prompts for letting it say "I don't know".

{Task}
If anything in this task can be read more than one way, ask me ONE clarifying
question before you start. The most important one.
If everything is clear, start right away.
When you're done: state in one line which assumptions you made
when interpreting the task.

You MUST stop and ask for clarification in any of the following situations:
1. If you cannot rule out negative consequences for any of the possible next actions
2. If you need more information about what to do in order to act carefully and safely
3. If you need more information about the current situation in order to act carefully and safely
4. If you lack the knowledge to judge whether your actions could have negative consequences
Do NOT stop if:
- You're unsure about one action, but other actions are definitely safe
- You can gather more information first through safe actions
To stop, explain your concerns, then output "Final Answer" with a detailed explanation.
{your_task}

Here's my task: [your request]
First:
1. Restate the task in your own words — how you understood it
2. Point out what's unclear or could be interpreted different ways
3. Suggest clarifying questions
Then I'll adjust, and you'll do the precise version.