TL;DR
Thinking more ≠ answering better. When a model has to pick one option from a fixed set, brief reasoning (literally 2–3 lines) produces an accurate choice. Long reasoning drags the result below the "just answer, no reasoning" baseline.
The key finding: it's not a straight line, it's a hump. Think a little — accuracy jumps +45%. Think longer — the model slides below where it would have been with no reasoning at all. Why: during long reasoning the model "talks itself into" the wrong option or invents options that weren't on the list.
FR-CoT is a template that locks in this effect: the model first names its choice, then briefly states the key parameters, then answers. The rigid order kills hallucinations and keeps accuracy at the peak — no long reasoning required.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
Everything happens in a single prompt:
STEP 1: Name the choice → "Approach: [name]"
STEP 2: State the key parameters → "Key parameters: [2–3 points]"
STEP 3: Give the answer → based on steps 1–2
📌 Critical: step 1 comes first. The model names its choice before any explanation. This isn't style — it's the mechanism that blocks drifting into the wrong option.
Example
Task: A Coursera sales rep is reviewing a call recording with a prospect. He needs to decide which objection-handling technique to use: reframing, clarifying questions or social proof.
Prompt:
The customer says: "That's too expensive, I've seen it cheaper elsewhere."
From this set of objection-handling techniques, pick one and justify it:
— Reframing (shift the focus from price to value)
— Clarifying questions (find out what exactly they're comparing)
— Social proof (examples from other customers)
Answer strictly in this structure:
Technique: [name]
Key parameters: [2–3 reasons why this one]
Rep's reply: [a concrete line to say]
Result:
The model first commits to a technique (Technique: Clarifying questions), briefly explains why (we don't know what exactly the customer is comparing, so we need to narrow it down), and immediately gives a ready-to-use line for the rep. No long breakdown of all three options, no "on the one hand... on the other hand." One short block — a clear choice — ready-to-use text.
Why It Works
The LLM weakness when choosing between options. When it has to pick one of several given options, a model has no "memory of the choice" — it generates token by token. Long reasoning builds up context that gradually shifts the probabilities. Once it starts analyzing every option in detail, by the end of its reasoning the model "talks itself into" a different option — or invents one that was never there.
What the model does well. If you put a choice marker right at the start (Technique: X), the following tokens are generated within the logic of that choice. Models are good at holding on to a "frame" once it's set and filling it in consistently.
How FR-CoT exploits this. The template forces the choice into the first position — before any reasoning. That stops the model from "sliding" into another option along the way. The reasoning becomes short (it only needs to justify a choice already made, not weigh them all), and the final answer comes out accurate.
Control knobs: - The option list → the more precisely the options are worded in the prompt, the better the choice. Fuzzy descriptions give fuzzy results - Number of "Key parameters" → 2–3 points is optimal. More, and you're back to long reasoning that breaks the effect - Instruction strictness → the phrase "Answer strictly in this structure" matters. Without it the model may reshuffle the blocks
Prompt Template
{task_context}
From this set of options, pick one:
— {option_1}
— {option_2}
— {option_3}
Answer strictly in this structure:
Approach: [option name]
Key parameters: [2–3 main arguments]
{output_type}: [finished result]
What to fill in:
- {task_context} — a description of the situation and the input data
- {option_1/2/3} — the concrete options from your set (3–5 of them, each with a short explanation)
- {output_type} — what you need at the end: "Email text", "Rep's reply", "Task statement"
🚀 Quick start — paste into the chat:
Here is the FR-CoT template. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask which options to put on the list and what output you need — because the template doesn't work without concrete options: FR-CoT is built for choosing from a given set, not for generating from scratch.
Original from the Paper
Step 1 -- Identify:
Function: [function_name]
Key args: [arg=value, ...]
[Based on the above, the JSON function call is:]
JSON: ...
Context: The researcher tested agents that call API functions — the model received a list of 2–4 allowed functions and had to pick one and fill in its arguments correctly. FR-CoT was built into the prompt so that the model had to write the function name as the very first words of its reasoning.
Limitations
⚠️ Tasks without a clear set of options: FR-CoT is built for choosing from a defined list. For open-ended tasks ("write a post") the structure loses its point — there's nothing to name in step 1.
⚠️ It redistributes, it doesn't boost: FR-CoT doesn't make the model smarter. It removes one specific type of error — picking the wrong item from the set. Errors in the answer itself (inaccurate arguments, weak text) remain.
⚠️ Long reasoning on demand = risk: If you're used to adding "think carefully, consider every option" — for choose-from-a-list tasks this can make results worse. The effect is non-linear: thinking a little is good, thinking a lot is worse than not thinking at all.
⚠️ Tested on small models: The experiments used Qwen2.5 1.5B and 7B. For GPT-4, Claude 3.5 and other large models, the degradation from long reasoning may be less pronounced — they hold focus better.
Adaptations and Extrapolations
💡 Adaptation: choosing the tone of an email
When you need to write to a customer and pick the right tone:
Situation: a customer left an unhappy review about delivery; the tone is irritated
but not aggressive. Keeping the relationship matters to us.
Pick the tone of the reply from this set:
— Formal (official apology, reference to company policy)
— Warm (personal involvement, acknowledging the customer's feelings)
— Businesslike (facts, concrete actions, no emotion)
Answer strictly in this structure:
Tone: [name]
Key parameters: [2–3 arguments]
Email text: [ready reply to the customer]
🔧 Technique: drop the "Key parameters" block → an ultra-short answer
If you only need the choice, with no justification:
Approach: [name]
{output_type}: [result]
This is even shorter — closer to the optimal 8–16 tokens from the study. Good for simple tasks where no justification is needed.
🔧 Technique: add a second level of refinement
For complex tasks where one choice leads to a sub-choice:
Approach: [general method]
Variant: [specific version of the method]
Key parameters: [2–3 arguments]
Result: [answer]
For example: Approach → negotiation strategy; Variant → the "conditional agreement" technique.
How They Tested It
A researcher from Tsinghua took 200 tasks from a standard function-calling benchmark (BFCL v3) — in each, a user makes a request and the model must pick one of 2–4 API functions and correctly fill in its parameters as JSON. He then ran a systematic sweep: every task went through 6 different reasoning budgets, from 0 to 512 tokens. That revealed the full accuracy curve, not just "CoT helps / doesn't help."
The most interesting part is the breakdown of errors into three types: the wrong function from the list, a made-up function (not on the list), wrong arguments. It turned out that at d=0 the main problem is picking the wrong function (30.5% of tasks). At d=32 that error almost disappears (1.5%). But at d=256 it comes back (28%), joined by a second one — the model invents functions that aren't in the set (18%). This isn't "the model got tired of formatting JSON" — it's the model actively "talking itself out of" the right answer during long reasoning.
Especially telling is the comparison of FR-CoT with constrained decoding (forcibly restricting the choice through token probabilities — an infrastructure-level trick). On the 7B model: FR-CoT reaches 83%, constrained decoding 63.5%. That 19.5-percentage-point gap shows the point isn't the technical restriction — it's that structured reasoning by itself steers the model the right way. A prompt structure beats a technical crutch.
Resources
Paper: Brief Is Better: Non-Monotonic Chain-of-Thought Budget Effects in Function-Calling Language Agents (2026)
Author: Xuan Qi, IIIS, Tsinghua University — [email protected]
Benchmark: Berkeley Function Calling Leaderboard v3 (BFCL) — gorilla-llm/Berkeley-Function-Calling-Leaderboard on HuggingFace
Related work: - Chain-of-Thought prompting [Wei et al., 2022] - Test-time compute scaling [Snell et al., 2024] - "Overthinking" in mathematical reasoning — a parallel phenomenon in another domain