Select-then-Solve: Your Choice of Reasoning Strategy Shapes LLM Answer Quality — Sometimes More Than the Model Itself

STEP 1: Classify the task → type (code / search / analysis / knowledge)
         ↓
STEP 2: Pick a strategy from the matching table
         ↓
STEP 3: Open/create a prompt in that mode → ask your question
         ↓
STEP 4: Get the answer

All in one chat. The key choice is STEP 2, before you send the task.

Strategy selection map (extractable):

⚠️ Where the method shines: tasks where people habitually add CoT (“think step by step”) even though the task doesn’t need it or actively suffers from it.

Task: You run a Discord bot for your online course. You ask Claude to write a Python script that pulls the list of participants from Google Sheets and sends each of them a welcome message via the API. Out of habit you add to the start of the prompt: “Think step by step before writing the code.”

What to do instead (according to the study):

Write a Python script that:
— Reads the participant list from Google Sheets (gspread)
— Sends each participant a welcome message via the Discord Bot API
— Handles errors: what if a user has blocked the bot

Use async if it speeds things up.
Return finished, working code.

DON’T add: “Think step by step,” “Explain your solution,” “Break the task into parts.”

Result: The model returns clean, working code with no extra reasoning. According to the study, adding CoT to coding tasks lowers code quality — the model starts “explaining” instead of “writing,” which wrecks the program’s syntax and structure.

Task 2 (Plan-Execute): You need a competitive analysis of three food delivery services in Chicago: DoorDash, Uber Eats, Grubhub. Prices, speed, coverage areas, reviews.

Prompt:

Task: competitive analysis of food delivery services in Chicago 
(DoorDash, Uber Eats, Grubhub).

Step 1 — Draw up an analysis plan: which criteria, in what order, 
what needs to be found for each service.

Step 2 — Execute each item of the plan in sequence.

Step 3 — Pull the results together into a comparison table.

Don't move on to step 2 until the plan is ready.

Result: The model first lays out the structure of the analysis (criteria, logic), then fills it in for each player, and finishes with a table. Plan-Execute wins here because the task is multi-step and requires accumulating information rather than a free-form answer.

An LLM isn’t a universal thinker — it’s a pattern matcher. When you add “think step by step,” the model literally starts generating text in a “step-by-step reasoning” format. For math that helps — the structure pulls out the right patterns. For code it gets in the way: step-by-step explanations break the program’s syntactic integrity.

Strong models already reason internally. GPT-4o and Claude Sonnet apply internal mechanisms to a simple question like “capital of France?” — without your command. By adding “think step by step,” you’re not improving the process, you’re adding noise to the output. CoT helps weak models: +46 percentage points on knowledge tasks. For strong ones it’s neutral or harmful.

Strategy control knobs in your prompt:

• 🔧 Remove “think step by step” from coding tasks → cleaner code, less “explaining instead of solving”
• 🔧 Add “draw up a plan first” → activates Plan-Execute, helps on multi-step research tasks
• 🔧 Add “write an answer → find the weak spots → improve” → activates Reflection, good for texts and analysis
• 🔧 Add nothing → Direct — often wins on factual questions even with strong models
• 🔧 Don’t ask the model to pick a strategy → self-routing only works for GPT-5; for the rest it lowers the result (see Limitations)

A universal “strategy selection” template to use before kicking off a task:

Task: {task description}

Solving mode: {pick one}
— [DIRECT] just give the answer
— [COT] think step by step before answering  
— [PLAN-EXECUTE] first break it down into steps, then execute each one
— [REFLECTION] write a draft → find the weak spots → improve
— [RECODE] solve it with code and run it

Chosen mode: {your choice from the table above}

{the task itself}

Placeholders: - {task description} — one sentence about what you need - {pick one} — delete the modes you don’t need, keep one - {your choice} — the mode name from the table

🚀 Quick start — paste into the chat:

Here is the Select-then-Solve template. 
Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what type of task it is, what result you expect and whether there are format constraints — because picking the right mode requires understanding the nature of the task (code/search/analysis/text). It will take the right mode from the template and build a working prompt.

⚠️ Self-selection doesn’t work: Don’t ask the model to pick a strategy on its own — that only works reliably for the very strongest models. Mid-tier models doing self-routing drop below the baseline (worse than just asking directly). Pick the strategy yourself from the table.

⚠️ ReAct needs tools: The “find information — reason — act” strategy only works if the model has access to search or tools. In basic ChatGPT/Claude without plugins it can’t be used at full strength.

⚠️ The table isn’t gospel: These are task-level statistics. Your specific task may be an exception. If CoT gives a better result on code, trust the result, not the table.

⚠️ Ceiling effect: Some tasks (complex agentic scenarios like τ-bench) stay hard under any strategy. Switching paradigms doesn’t substitute for the model’s capabilities.

⚠️ Model-dependent: The weaker the model, the more a complex strategy (CoT, Reflection) helps. The stronger the model, the more often plain Direct wins. Tune your choice to the specific model.

The team set out to ask a simple but awkward question: if you take the same model and wrap it in different strategies — which one wins? And on which tasks?

They built a unified framework, PARADIGM, where all six strategies work with the same tools, the same interface and the same evaluation code — only the reasoning logic changes. Then they ran four top models (GPT-5, Gemini-3-Flash, Qwen3-Max, Qwen3-30B) through ten benchmarks — code, math, search, knowledge, agentic tasks. Total: ~18,000 runs.

The big surprise — how stable the pattern is. Even though GPT-5 solves AIME at 90% and Qwen3-30B at 11%, the ranking of strategies stays the same across all models. ReAct wins on GAIA for all four models. ReCode is best on HumanEval for all of them. Direct or CoT win on MMLU for all of them. That means: the task, not the model, determines the best strategy. The results aren’t specific to GPT-5 — they transfer.

Another non-trivial result — the self-routing test. The researchers asked: what if you just ask the model to pick its own strategy? It turns out this works only for GPT-5 (67.1% vs 62.4% for Direct). All the other models drop below the baseline with self-routing. That’s unexpected: it takes a separate “meta-ability” to understand how you think — and most models don’t have it.

🔧 Technique: a personal strategy table for your recurring tasks

If you have recurring task types, build your own map once:

My recurring tasks and strategies:

1. [Write a blog post] → Reflection (draft → critique → final)
2. [Fix a bug in the code] → Direct (no CoT!)  
3. [Put together a competitive analysis] → Plan-Execute
4. [Answer a tough client question] → CoT (if I don't know the answer right away)
5. [Learn a new topic] → CoT + Direct, alternating

Save this in your system prompt or as a note next to the chat.

🔧 Technique: Reflection mode for improving any text

The study showed that Reflection (critique + revision) works consistently on tasks with an open-ended quality bar. In a chat:

DRAFT: {your text, or ask it to write one}

Now:
1. Find 3 weak spots in this text
2. Suggest specific improvements for each
3. Write the final version incorporating the edits

Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents Heng Zhou, Zelin Tan, Zhemeng Zhang et al. (Preprint, under review)

• 🌐 Project page: hengzzzhou.github.io/STS
• 💻 Code: github.com/hengzzzhou/STS
• 📬 Contact: [email protected]