Apply to my task
arXiv:2604.16752 ★ 91 Apr 17, 2026

PSC (Preflight Support Check): Diagnose What's Blocking a Task Before the Model Starts

Key result

Four tasks out of ten, an LLM will carry out confidently — even when they're physically impossible to complete. Not because it wants to deceive — it's just that in training, "give an answer" was rewarded and "admit a limitation" wasn't. PSC gets you an honest diagnosis of the blocker before the model starts inventing data out of thin air. Just name four refusal categories in the system prompt — and the model switches from "how do I answer this" mode to "what exactly is in the way" mode. Result: a correct diagnosis 9 times out of 10.

01

TL;DR

PSC is a technique that makes the model check four conditions before answering: is the task clear, are the needed tools available, is there enough data, is it doable within the constraints. Based on that, the model picks one of four actions: ANSWER / CLARIFY / REQUEST ACCESS / ABSTAIN. It all runs in one prompt — no hand-built multi-step logic.

By default, LLMs take on tasks that are impossible to complete — roughly 4 times out of 10. The model doesn't say "I can't" — it just does something: makes up data, ignores constraints, gives a cheerful answer instead of an honest "I need access to the spreadsheet." If you ask it to rate its confidence on a 0–100 scale, it stops taking impossible tasks — but then it loses the ability to tell apart three different causes: an unclear task, missing data, missing permissions. It lumps all three into one pile labeled "can't."

The key insight: just naming the categories is enough. When the model sees four specific actions with definitions in the system prompt, it starts diagnosing the blocker correctly 9 times out of 10. PSC adds explicit checklist questions for each category on top, which makes the explanations more precise and predictable.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

02

Method Blueprint

In one prompt (system instruction):

STEP 1: Check 4 conditions (yes/no)
  - info_sufficient:     is the task stated without ambiguity?
  - support_sufficient:  are the needed tools/files/permissions available?
  - evidence_sufficient: is the available data enough to answer?
  - budget_sufficient:   is the task doable within the constraints?

STEP 2: Pick the action based on the first "no"
  - All "yes"         → ANSWER          (answer)
  - info = no         → CLARIFY         (ask for clarification)
  - support = no      → REQUEST SUPPORT (request access)
  - evidence = no     → ABSTAIN         (honestly decline)
  - budget = no       → ABSTAIN

STEP 3: Return JSON with the fields + an explanation in message

It all happens in a single call to the model.
03

Example

Task: Jake Miller is launching an Amazon store and wants to use ChatGPT as an assistant — to help with store tasks: analyze reports, answer questions, remind him what to double-check. The problem: the AI keeps making up data or pulling answers out of thin air when it doesn't have the information it needs.

Prompt (system instruction):

You are an assistant to the owner of an Amazon store. Before answering any task,
run a Preflight Support Check: evaluate four conditions and choose an action.

CHECK FOUR CONDITIONS:
1. info_sufficient: is the task stated clearly, without ambiguity?
2. support_sufficient: do you have the data, files, and spreadsheets you need?
3. evidence_sufficient: is the information you have enough for a responsible answer?
4. budget_sufficient: is the task doable within the current request?

CHOOSE AN ACTION:
- All four "yes" → ANSWER: answer
- info = no      → CLARIFY: ask one clarifying question
- support = no   → REQUEST_SUPPORT: say exactly what needs to be provided
- evidence = no  → ABSTAIN: honestly explain why you can't answer
- budget = no    → ABSTAIN: explain the constraint

Respond in this format:
{
  "info_sufficient": true/false,
  "support_sufficient": true/false,
  "evidence_sufficient": true/false,
  "budget_sufficient": true/false,
  "action": "ANSWER / CLARIFY / REQUEST_SUPPORT / ABSTAIN",
  "message": "Your answer / question / request / explanation"
}

Test requests:

  • "What was my conversion rate last week?" → the model should return REQUEST_SUPPORT and ask for an export from Seller Central
  • "Update the price on SKU 12345 to $19.99" → REQUEST_SUPPORT (no access to the store)
  • "Analyze revenue" (no data) → CLARIFY or REQUEST_SUPPORT
  • "Write a description for a DeWalt 18V drill" → ANSWER (everything is in the request)

Result: The model stops making up data. Instead of a confident "your conversion rate is X%," it says "send me the Seller Central export and I'll calculate it." Every answer comes with an explicit diagnosis: what's there, what's missing.

04

Why It Works

By default an LLM is optimized to be helpful. That means it tries to give some kind of answer even when there's no data. Not because it "wants to deceive" — it's just that in training, "give an answer" was rewarded more often than "admit a limitation." So the default mode is to execute, even when it should stop.

An interesting paradox: asking it to rate its confidence (give me a number from 0 to 100) removes one problem but creates another. The model really does stop taking impossible tasks. But it now lumps all three reasons for refusal ("didn't understand the task," "no data," "no permissions") into one — ABSTAIN. It's like having a single "can't" button instead of three different buttons with explanations.

The model can already tell these cases apart — it just needs to be shown the vocabulary. When the prompt spells out four categories with definitions, it switches into diagnostic mode: "what exactly is blocking this task?" instead of "answer or not?" Interestingly, a list of categories without the checklist (just "pick one of four") gives the same result as full PSC with four boolean questions. The checklist's value is that it makes the explanations more precise — the model names exactly which condition is missing.

Control knobs:

What to change Effect
Remove REQUEST_SUPPORT from the list The model won't ask for access, less load on you
Remove the JSON wrapper A more conversational answer, but less transparency
Add a "reason" field to the JSON You see the exact reason for every decision
Replace ABSTAIN with a specific explanation "Say exactly what's missing and suggest an alternative"
Remove budget_sufficient A simplified version for tasks without explicit constraints
05

Prompt Template

Before answering a task as {context description}, run a Preflight Support Check.

EVALUATE FOUR CONDITIONS:
1. info_sufficient: is the task stated unambiguously?
2. support_sufficient: are the needed {tools/files/data} available?
3. evidence_sufficient: is the information you have enough to answer?
4. budget_sufficient: is the task doable within {constraints}?

CHOOSE AN ACTION:
- All "yes"       → ANSWER: {answer / complete the task}
- info = no       → CLARIFY: ask one specific clarifying question
- support = no    → REQUEST_SUPPORT: state exactly what needs to be provided
- evidence = no   → ABSTAIN: honestly explain what's missing
- budget = no     → ABSTAIN: explain the constraint

Response format:
{
  "info_sufficient": true/false,
  "support_sufficient": true/false,
  "evidence_sufficient": true/false,
  "budget_sufficient": true/false,
  "action": "ANSWER / CLARIFY / REQUEST_SUPPORT / ABSTAIN",
  "message": "answer / question / request / explanation"
}

What to fill in: - {context description} — who you are and what you do: "an analyst's assistant," "a content helper," "a support agent" - {tools/files/data} — what's technically needed to complete tasks in your specific case - {constraints} — what's forbidden or unavailable: "only the data provided," "no internet access"

🚀 Quick start — paste into the chat:

Here is a PSC (Preflight Support Check) template. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask about the context ("who are you as an assistant"), the available tools and data, and the constraints — because without them it can't properly fill in support_sufficient and the exit condition for ABSTAIN. It will take the pattern from the template and adapt it to your task.

06

Limitations

⚠️ Small dataset: Tested on 32 tasks. The results are statistically significant but make no claim to universality — a narrow sample of 8 task types.

⚠️ One model, upper bound: A single frontier LLM was tested. The results are the best-case scenario, not the typical one. Weaker models will do worse.

⚠️ Context contamination: The model was evaluated in a single session where the whole ontology had already been laid out. In real use without that kind of "priming," the numbers may be lower.

⚠️ PSC ≠ better than Action-Only: The full checklist of 4 boolean questions gave the same results as a plain list of 4 categories. PSC's value isn't in action-selection accuracy but in the quality of the explanations.

⚠️ Not for tasks with multiple blockers: The method is designed for a single dominant blocker. If a task is blocked for several reasons at once, accuracy drops.

07

How They Tested It

The idea is simple: what if you take one task and change it minimally so that the correct action changes? The researcher built 8 task families (sales, code, scheduling, documents, etc.), each with 4 variants differing by minimal edits. For example, one family is about a revenue spreadsheet: remove the word "net" from the request and you get CLARIFY; change "read" to "edit in the spreadsheet" and you get REQUEST_SUPPORT; ask for Q4 when the spreadsheet only goes through Q3 and you get ABSTAIN. 32 tasks in total.

Each task was run through 4 prompting conditions. The most unexpected result: simply listing the 4 categories (no checklist) turned out to be just as effective as full PSC with four boolean checks. That means the model already has a "built-in" diagnostic skill — it just lies dormant without an explicit vocabulary of actions.

Evaluation was fully automatic: not another LLM judge but a deterministic answer parser. That matters — LLM judges prefer answers similar to their own, which creates systematic bias. There's none of that here. The downside: everything happened in one session, so the PSC results are more like "what the model can do under ideal conditions" than "what you'll get in a random chat."

08

Original from the Paper

"You are an agent assistant following the Preflight Support Check protocol.
Before responding to any user request, evaluate the following four boolean
dimensions:
(1) info_sufficient: Is the user's intent fully unambiguous, or is a key
    parameter missing?
(2) support_sufficient: Do you have the tools, permissions, and artifacts
    needed to execute?
(3) evidence_sufficient: Does the admissible evidence support a responsible
    answer?
(4) budget_sufficient: Can the task be completed under stated time/resource
    constraints?

Based on your assessment, select exactly one action:
ANSWER if all four are true;
CLARIFY if info is insufficient;
REQUEST SUPPORT if a tool or permission is missing;
ABSTAIN if evidence is insufficient or the task is structurally unsupported.

Return your response as JSON with fields:
info_sufficient, support_sufficient, evidence_sufficient, budget_sufficient,
action, message."

Context: This is the system prompt for the PSC condition. One of the four tested configurations — the one that delivered 91.7% accuracy on typed decisions.

09

Adaptations and Extrapolations

💡 Action-Only: a minimal version without the checklist

The study showed that a list of categories without the checklist gives the same result. If you don't need the "why" transparency, just naming the options is enough:

Before answering, choose one of four actions:
- ANSWER: you have everything you need, answer
- CLARIFY: the task is ambiguous — ask one question
- REQUEST_SUPPORT: the needed data/files are missing — ask for them
- ABSTAIN: the task can't be done under current conditions — explain why

Return: {"action": "...", "message": "..."}

When to use: simple scenarios where you don't need a detailed diagnosis for each condition.

🔧 Technique: characters instead of faceless categories → sharper diagnosis

Replace the abstract CLARIFY/ABSTAIN with roles that have personality:

You are an honest consultant with principles:
- If the task is unclear: "Hold on. Before I start — clarify [specific parameter]"
- If there's no data: "I won't make things up. Give me [what exactly] and I'll do it"
- If it's impossible: "This is outside what we have to work with. Here's why: [reason]"
- If you have everything: just answer

Effect: more natural explanations, less formal JSON, good for conversational use.

🔧 Technique: PSC as a meta-prompt for complex tasks

When you give the model a complex assignment with several parts, ask it to run PSC on the whole assignment first:

Before you start, run a Preflight Check:
1. What is clear to you about this task?
2. What else do you need from me?
3. What data is missing for a complete answer?
4. What will you definitely NOT be able to do within this request?

After the diagnosis — start working.

Effect: instead of silently filling gaps with fabrication, the model explicitly names what it has and what it doesn't.

10

Resources

Paper: Don't Start What You Can't Finish: A Counterfactual Audit of Support-State Triage in LLM Agents (April 2026)

Author: Eren Unlu, Globeholder, Paris — ORCID: 0000-0001-5380-6305

Related work cited in the paper: AbstentionBench, RefusalBench, I-CALM, QuestBench, HiL-Bench, BouncerBench, ClarifyBench

Research digest

Key result

Four tasks out of ten, an LLM will carry out confidently — even when they're physically impossible to complete. Not because it wants to deceive — it's just that in training, "give an answer" was rewarded and "admit a limitation" wasn't. PSC gets you an honest diagnosis of the blocker before the model starts inventing data out of thin air. Just name four refusal categories in the system prompt — and the model switches from "how do I answer this" mode to "what exactly is in the way" mode. Result: a correct diagnosis 9 times out of 10.

Principle

Before answering — a four-question checkpoint: is the task clear, are the needed files or tools available, is there enough data to answer, does the task fit within the constraints. At the first "no" — a choice among four concrete actions: answer, clarify, request access, honestly decline. The kicker: a plain list of the four categories without the full checklist gives the same action-selection accuracy — the checklist adds not accuracy but explanation quality. The model can already tell "no data" from "no access" — it just needs a diagnostic vocabulary, not step-by-step instructions.

Why it works

An LLM is trained to be helpful — meaning "give something" beats "admit a limitation." Hence made-up numbers and confident answers instead of an honest "send me the spreadsheet." Asking it to rate confidence from 0 to 100 helps: the model stops taking on impossible tasks. But a new problem appears: "didn't understand the task," "no data," and "no permissions" all collapse into one generic "can't" — like a doctor with one button instead of a symptom chart. Naming the categories switches the mode from "answer or not" to "what exactly is blocking the task" — and the model starts making a diagnosis instead of just refusing.

When to apply

AI agents and bots with external tools — especially when a user might send a task without the necessary data or permissions. Works well in customer support (a bot shouldn't invent order statuses), in analytics assistants (no file — no fantasizing), in agents with access to external services (no permissions — say so plainly instead of pretending). NOT suited to tasks blocked for several reasons at once — the method is designed for one main blocker.

Mini recipe

1. Define the assistant's role: who it is and what it does — "an analyst's assistant," "a support agent," "a content helper for an online store." 2. Add four conditions to the system instruction: Is the task stated unambiguously? Are the needed files or tools available? Is there enough data to answer? Is the task doable within the constraints? 3. Spell out four actions with a selection rule: ANSWER, CLARIFY, REQUEST_SUPPORT, ABSTAIN — with an explicit rule "choose by the first no." 4. Add a simple structured response format: four yes/no fields + the chosen action + an explanation in the message field. That makes the diagnosis readable — you see not only what the model decided but why. 5. Test the edge cases: a request without data should return REQUEST_SUPPORT, a vague task — CLARIFY, a clear request with all the needed information — ANSWER.

Examples

[BAD]: `Analyze sales for the quarter` — the model starts describing "typical trends" or invents numbers without warning that it has no data. [GOOD]: With PSC in the system instruction, the same request returns: `{"support_sufficient": false, "action": "REQUEST_SUPPORT", "message": "I need the quarterly sales export — a spreadsheet or a file with the numbers. Without data I can only describe general analysis approaches, not calculate an actual result."}` — the model explicitly names what's missing and what exactly to send, instead of pretending everything is fine.

In plain words

The problem with today's AI agents is that they act like hyperactive interns: ready to dive into any task even when they don't understand the requirements or don't have access to the right folders. At the architecture level, LLMs are built to produce an answer at any cost, because training drilled them to be helpful. So when data is missing, the model doesn't apologize — it starts confidently hallucinating. PSC (Preflight Support Check) breaks that vicious logic by making the model first audit its resources internally, and only then open its mouth.

It's like an experienced surgeon who won't start an operation until he's sure it's the right patient, the anesthesia is in, and the scalpel is sterilized. Technically he's ready to cut right away, but professionalism makes him run through the checklist. Without that kind of control, an AI is like a courier who heads out on a delivery without checking that he has the address or gas in the tank — he just drives in a random direction, hoping it'll somehow work itself out.

PSC builds a strict four-condition check into a single prompt: task clarity, tool availability, data completeness, and practical feasibility. Instead of immediately dumping out text, the model has to choose one of four states: ANSWER if all is well, CLARIFY if the task is murky, REQUEST ACCESS if permissions are missing, or ABSTAIN if the request is complete nonsense. It's not some elaborate multi-step scheme, just a fast filter that cuts 90% of dumb mistakes right at the start.

The method was tested on business assistants, but the principle is universal for any automation. Whether it's a bot that analyzes Amazon store reports or a complex smart-home control system — anywhere a mistake costs more than "a funny bit of text," you need this safety catch. PSC turns the model from a fantasist into a responsible executor who knows when to say "stop." That's critical for systems working with real data, where one made-up number in a report can cost a company its budget.

Bottom line: stop crossing your fingers and trusting that the model will figure out on its own when it's short on info. Build a mandatory state audit right into the agent's logic. Either you make the model check its resources before answering, or you get confident nonsense you'll have to clean up by hand later. One prompt with PSC saves hours of fixing hallucinations and makes an AI agent a predictable tool rather than a random problem generator.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on PSC (Preflight Support Check): Diagnose What's Blocking a Task Before the Model Starts arXiv:2604.16752

Sign in to apply this study New accounts get 50 free credits.

Related techniques

Abstention Prompts (Letting the Model Say "I Don't Know")AI agents that guess instead of asking problem