TL;DR
Researchers at Tsinghua University found a paradox: the more explicit constraints a prompt has, the worse the model solves the actual task, even when those constraints are completely achievable. They tested an unusual hypothesis: take models' successful answers, extract their characteristics (style, structure, length), turn them into explicit requirements and add them back into the prompt. The logic: the model was ALREADY doing this on its own, so it should handle an explicit requirement just fine. The result was the opposite.
The main finding: adding 5 constraints cuts task success by 15-40%, even for top models. Claude-Sonnet-4.5 loses 15% of its performance on multi-hop questions, and mid-size models (30-70B) keep only 65-85% of their original accuracy. Code is the worst: seven models dropped below 60%. Meanwhile, the models satisfy the constraints themselves 94%+ of the time. The problem isn't instruction following; it's that an excess of explicit requirements wrecks the logic of the solution. A requirement like "use 15 words" or "start with a question" pulls the model's attention away from the substance of the task.
The mechanism is simple: the model focuses on form and loses the content. An analysis of attention scores showed that in failed cases the model pays significantly more attention to constraint tokens, especially toward the end of generation. Instead of reasoning through the task, it "remembers" the formatting requirements and slips up either in the logic (reasoning error) or in the final output (output specification error). The practical takeaway: fewer explicit requirements = better results.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint: the problem
The standard approach:
Prompt = Task + Constraint_1 + Constraint_2 + ... + Constraint_5
Result = The model focuses on the constraints → loses focus on the task
What happens: - The model WITHOUT constraints → solves it correctly - The same model WITH constraints (extracted from its own answer) → fails the task - It still satisfies the constraints (format is correct), but the answer is wrong
Constraint types (worst to best): - Length → SUSTAINSCORE 77.2% - Keyword → 77.4% - Style → 82.3% - Method (solution approach) → 84.1% - Structure → 85.0%
Example
Task: Write a Python function to scrape eBay listings: extract the price, title and link from the HTML.
❌ A prompt overloaded with constraints:
Write a Python function to scrape listings from eBay.
Requirements:
- The function must be exactly 25 lines of code
- Use the BeautifulSoup library
- Start with a docstring
- Add exception handling
- Style: detailed comments on every step
- Return the result as a list of dictionaries
✅ A prompt without unnecessary constraints:
Write a Python function to scrape listings from eBay.
Extract the price, title and link from a listing's HTML block.
Use BeautifulSoup.
Result:
In the first case the model will: - Count lines of code instead of focusing on the logic - Pad out comments to hit the "detailed style" - Possibly miss edge cases (no price, broken HTML) - Maybe break the scraping logic to stay within the line limit
In the second case the model will: - Focus on correct data-extraction logic - Pick a sensible length and style on its own - Most likely add the necessary checks without being told
The paradox: if the first prompt produces working code, and you extract requirements from it (25 lines, docstring, comment style) and add them explicitly, the model may fail the task on the next run.
Why it works (or rather, doesn't)
The LLM's weakness: Models are autoregressive generators: they predict the next token from the previous ones. For every token, the attention mechanism spreads "attention" across all tokens in the context. The more explicit constraints in the prompt, the more requirement tokens in the context, and the more the model gets "distracted" checking "am I following the constraint?" instead of "am I solving the task correctly?".
The researchers measured a Constraint Attention Score: the share of attention the model gives to constraint tokens during generation. They found that in failed cases the model pays significantly more attention to the constraints, especially in the late stages of generation. Instead of finishing the chain of logic ("so the answer = X"), the model checks "did I stay under 200 words?", "did I use the right word?".
Two kinds of errors:
Reasoning Error: the model veers off the right line of reasoning. Example: on a math problem it starts bending the solution to fit the length requirement and skips steps.
Output Specification Error: the model reaches the right answer but mangles it while "packaging" it into the required format. Example: the algorithm in the code is correct, but an edge case is missing because the model was thinking about the line limit.
The LLM's strength: Models are excellent at solving tasks when they focus on substance, not form. Without explicit constraints, the model picks a sensible length, style and structure on its own, and it works better.
Practical takeaway: The more minimal the prompt, the better. Specify only critical constraints (for example, an output format that has to be parsed). Everything else (style, length, structure) the model will handle on its own, and most likely do better.
How to apply it
What NOT to do (too many constraints):
{task}
Requirements:
- Length: {number} words/lines/paragraphs
- Style: {style description}
- Structure: {structure requirements}
- Include the words: {list of keywords}
- Method: {specific approach to solving it}
What TO do (minimal constraints):
{task}
{1-2 critical requirements, only if truly necessary}
A critical requirement is one without which the result is useless: - An output format for automated processing (JSON, CSV) - The tech stack (a specific library, if compatibility matters) - A hard size limit (a 280-character tweet, a 60-character title tag)
A NON-critical requirement is something the model will do on its own: - "Write in a professional tone": the model will pick an appropriate style by itself - "Use 3 paragraphs": the model will structure it by itself - "Add examples": if they're needed for completeness, the model will add them
Control knobs
1. Number of constraints → tokens and focus
The biggest quality drop comes from the first 5 constraints added. After that it plateaus.
Knob: Keep explicit requirements to ≤3. Every additional constraint risks losing focus.
2. Constraint type → different levels of toxicity
Hard constraints (length, keywords) hit harder (77.2-77.4% SUSTAINSCORE).
Soft constraints (structure, method) hit softer (84.1-85.0%).
Knob: If you need a constraint, choose a soft (semantic) one over a hard (syntactic) one.
3. Instruction order → prioritization
The researchers tested different structures: - Constraint → Task (constraint first) - Task → Constraint (task first) - "Solve it correctly first, then format it"
Result: Every variant showed a drop. Reordering doesn't save you.
Knob: Instead of shuffling, cut what's unnecessary.
4. Explicit prioritization → an illusion of help
The template "First solve the task correctly, then format it to the requirements" didn't help.
Knob: Don't rely on "magic phrases." If a constraint is unnecessary, delete it.
A meta-prompt for optimization
🚀 Quick start: paste this into the chat:
Here is my prompt:
[paste your prompt]
Analyze it and remove the UNNECESSARY constraints: the ones the model would
follow on its own without being told. Keep only the CRITICAL ones, without which
the result would be useless.
Rule: if a requirement is about length, style, structure or "exactly how to do it,"
it's a candidate for removal. If a requirement is about a data format for processing
or the tech stack, it's critical.
Give me the optimized prompt and explain what you removed and why.
The model will analyze your prompt, strip the extra constraints and explain its reasoning. You'll end up with a minimal prompt that gets better results.
Limitations
⚠️ Not for critical formats: If the output has to be parsed by code (JSON, XML, CSV with an exact structure), the constraint is necessary. The study is about an excess of NON-critical requirements.
⚠️ Not for hard limits: A 280-character tweet or a 60-character title tag are objective limits of the medium. You can't drop them.
⚠️ Math is sensitive to method: "Solve it with a system of equations" can be critical if another method would give a wrong answer. But "explain it in 3 paragraphs" is unnecessary.
⚠️ Code is the most sensitive: Quality on code drops by as much as 30-50% with 5 constraints. Keyword constraints are especially dangerous: requiring specific words/functions can break the logic.
⚠️ CoT models are prone to Output Specification Error: Models with long reasoning chains solve the task correctly but botch the final answer when trying to "package" it into a format. If you use o1/QwQ, give them freedom in the final output.
How the research was done
The Tsinghua University team tested a paradoxical idea: what if a model fails a task NOT because the constraints are too hard, but simply because they're explicit?
Experiment design:
They took 1000 math problems (GSM8K, SVAMP, OlympiadBench, MATH500), 1200 multi-hop questions (HotpotQA, 2WikiMultiHop, Musique) and 319 coding tasks (HumanEval, MCEval).
They ran the models WITHOUT constraints and kept only the successful answers.
From each successful answer they extracted self-evident constraints:
- Hard constraints (length, keywords): exact parameters extracted with Python scripts
- Soft constraints (method, style, structure): characteristics described by GPT-5
They added those constraints back into the prompt and ran it again.
The key logic: The model ALREADY did this in its successful answer → so it CAN → if it fails with an explicit requirement, that's instruction-following interference, not inability.
The results were surprising:
- Claude-Sonnet-4.5 (top of the market) lost 15% on multi-hop QA
- 30-70B models kept only 65-85% of their performance
- GPT-4.1-MINI: 90.9% on instruction following, 77.1% accuracy on code, but only 50.8% SUSTAINSCORE on code
- Seven models dropped below 60% on code
Mechanistic insight: They measured attention patterns: in failed cases the model pays significantly more attention to constraint tokens, especially in the late stages of generation. The chart shows both curves (success/failure) rising toward the end of generation, but the failure curve rises much more steeply: the model refocuses on checking constraints instead of finishing the logic.
An interesting finding: They checked the effect of post-training. RL (reinforcement learning) improves BOTH task performance AND robustness at the same time. Supervised fine-tuning on long CoT data, on the other hand, boosts performance but significantly lowers robustness: models become more fragile under constraints.
Validation: They confirmed the quality drop does NOT come from: - Prompt length (paraphrased without constraints, quality was fine) - Instruction order (changing the structure didn't help) - Constraint type (all types cause a drop, with little spread)
Practical takeaway: This isn't an artifact of the experiment; it's a fundamental property of current LLMs: too many explicit requirements break focus on the task.
Resources
On the Paradoxical Interference between Instruction-Following and Task Solving
Yunjia Qi, Hao Peng, Xintong Shi, Amy Xin, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li
Department of Computer Science and Technology, BNRist; Shenzhen International Graduate School, Tsinghua University
GitHub: https://github.com/kijlk/IF-Interference