Key Aspects of the Study:
The researchers tested how many simultaneous instructions modern LLMs can handle before their performance starts to drop. The test: write a business report that includes 10 to 500 specific keywords, one instruction per keyword. Every model degrades as the list grows, but in different ways: the best reasoning models (o3, gemini-2.5-pro) stay near-perfect up to 150 or more instructions, others decline steadily, and small models drop sharply after a few dozen. Even the best model reached only 68% at 500 instructions. Under heavy load, models also follow instructions near the start of the list better than those near the end (the "primacy effect"); this bias was almost absent with few instructions, peaked around 150-200, and faded again at 300+.
Key Result: When a prompt carries a long list of instructions, putting the most important ones near the top may help, because later items are dropped more often under load. With only a few instructions the study saw almost no order effect, and at extreme counts failures became uniform.
The Core of the Method, Explained:
The method that falls out of this study can be summed up as "managing the LLM's attention and cognitive load". Picture handing tasks to an intern. Give them 3 clear tasks and they will probably nail all of them. Give them a list of hundreds and even a strong intern starts dropping items, more often from the end of the list. LLMs behave in a similar way, though the breaking point depends heavily on the model.
Main takeaways and a playbook for users (the study tested only keyword-inclusion instructions; applying it to roles, goals and formats is our extrapolation):
The "Important stuff first" rule (Primacy Effect): The instruction that matters most to you should sit at the very top of the prompt. It can be a role ("You are a seasoned marketer"), the core goal ("The main thing is to convince the reader to buy the product") or a format requirement ("Return the answer as a JSON object"). In long instruction lists, items that come later are more likely to be dropped.
Know when to stop (Cognitive Load): Don't try to cram dozens of unrelated requirements into a single prompt. The more instructions you add, the higher the odds the model starts skipping them (under moderate load mostly the ones near the end; at extreme load more evenly). If your task is truly complex, consider splitting it into several sequential requests (our suggestion; the paper did not test splitting).
Expect the right kind of mistake: As the instruction count grows, models mostly skip (omission) instructions rather than half-following them (in the test, using a variant of the required word). At low counts the two error types were more balanced, and reasoning models like o3 kept more near-misses. So when a long list is only partly done, check for missing items first.
Watch the core task too: The study checked whether overload also hurts the main task (report coherence). For most models it did not, or only slightly; only o3 and o4-mini became markedly less coherent, partly because they wrote short reports packed with keywords. Trimming instructions you don't need is still sensible, but the paper does not show that fewer directions improve the text itself.
Practical Applicability Analysis:
Direct applicability: Extremely high. Anyone can improve their prompts right away just by reordering instructions. Instead of writing "Write a piece about the environment, be positive, use a bulleted list, and add a call to action at the end", the user writes: "Act as an environmental expert. Your main task is to write an upbeat piece with a call to action at the end. Use a bulleted list for the facts". With a short prompt like this the study found almost no order effect, so treat the reshuffle mainly as a habit that starts to matter once instruction lists get long.
Conceptual value: It gives users a simple, powerful mental model: "An LLM's attention is a finite resource that gets spent top to bottom through the prompt". That helps people stop treating the LLM as a magic box and start working with it as a system with clear rules and limits.
Adaptation potential: The paper tested only keyword-inclusion instructions in English business reports and notes the results may not generalize to other tasks or instruction types. Applying the idea to writing, code generation, data analysis or planning is our extrapolation. Adapting it is easy: before writing a complex prompt, ask yourself, "Which single instruction is absolutely critical to success? What comes second?" The answers define the structure of your prompt.
A Practical Example:
Our illustration, not from the paper.
Say a user wants an LLM to write a script for a short promo video (Reels/Shorts) about a new fitness app.
# TASK
You are an experienced screenwriter and creative director who specializes in short viral videos for social media (Reels, Shorts, TikTok).
Your main goal is to write a script for a 15-second video that makes viewers really want to download our new fitness app "FitGo".
---
# CONTEXT
* **Product:** The "FitGo" mobile app.
* **Key feature:** An AI coach that builds personalized workouts based on the user's goals and current fitness level.
* **Target audience:** People aged 20-30 who want to start working out but don't know how, or find it boring.
---
# SCRIPT STRUCTURE AND REQUIREMENTS
1. **Response format:** Present the script as a table with three columns: "Shot (visual description)", "Copy (voiceover or on-screen text)", "Timing (seconds)".
2. **Tone:** Energetic and motivating, but not pushy. Avoid clichés like "change your life in one day".
3. **Key message:** "FitGo is your personal trainer in your pocket that makes fitness simple and fun".
4. **Must-have element:** The video must end with a clear call to action (CTA): "Download FitGo for free and start your first workout!".
5. **Restriction:** Do not mention competitors.
Why It Works:
This prompt applies the ordering idea from the study (with this few instructions the study measured little order effect, so here the gain is mostly clarity):
- Primacy effect: The most important instructions sit at the top.
- Role (
You are an experienced screenwriter...): Sets the context and the quality bar. This is the first and most important instruction. - Main goal (
...makes viewers really want to download...): Defines the end result the model should aim for. This is the second most important instruction.
- Role (
- Managing cognitive load: Instead of one long list of requirements, they are grouped into logical blocks (
CONTEXT,REQUIREMENTS). Key requirements are numbered, which also helps the model process them, and the most important ones (format, tone, CTA) lead the list. - Clarity and prioritization: The prompt explicitly calls out the "main goal" and the "key message", helping the model see where to point its "attention" first. Less important details, like "do not mention competitors", sit at the end of the list, so even if the model ignores them, the core result won't suffer.
Another Practical Example
Our illustration, not from the paper.
Task: a user asks an LLM to help draft a polite but firm letter to a neighbor asking them to keep it down after 11 p.m.
# TASK
Act as an experienced conflict-resolution specialist and mediator.
Your main task is to help me write a letter to my neighbor. The main goal of the letter is to ask them, politely but very firmly and unambiguously, to stop making noise (loud music, shouting) after 11 p.m.
---
# KEY LETTER PARAMETERS
1. **Tone:** Balance is everything. The letter must be **non-confrontational and respectful**, yet **confident and firm**. No passive aggression and no apologizing.
2. **Structure:**
* Open with a friendly greeting.
* Describe the problem without blaming (e.g., "I've noticed that lately...").
* State the request clearly (stop the noise after 11 p.m.).
* Offer a solution or a conversation.
* End on a positive note.
3. **What to avoid:** Do not use threats (calling the police, complaining to the landlord) in this first letter. Focus on a good-neighborly solution.
Why This Example Works, Mechanically.
This prompt works because it also applies the principles of managing LLM attention:
- Role and Goal come first: The prompt opens by assigning a role (
conflict-resolution specialist) and stating the main goal (politely but firmly ask). That tunes the model in right away and sets the direction of the output. - Focus on the hardest part: The trickiest part of the task is getting the tone balance right. That is why the tone instruction (
non-confrontational, yet confident and firm) is first in the requirements list. Putting it first makes it less likely to be lost if the list grows long. - Structure for clarity: Rather than just saying "write a polite letter...", the prompt breaks the task into clear structural pieces (greeting, problem description, request). That lowers the model's cognitive load and helps it work through every part of the assignment in order.
- Negative constraints at the end: The instruction about what not to do (
Do not use threats) is placed last. It is a secondary constraint. If the model "forgets" it, that is far less damaging than forgetting the right tone. In other words, we deliberately gamble on a less important instruction to reliably land the more important one.
Usefulness Score: 92
Main Evaluation Criteria
- A. Relevance to prompting techniques: Yes, the study directly analyzes how the number and position of instructions affect the result.
- B. Better conversational answers: Yes, the findings help make answers more accurate and predictable when working with complex, multi-part requests.
- C. Direct practical applicability: Yes, the key findings (especially the "primacy effect") can be used immediately in any chatbot without special tools or knowledge.
- D. Conceptual value: Very high. The study gives users an intuitive grasp of an LLM's "cognitive load" and "limited attention", explaining why models "forget" or ignore some instructions.
- E. New useful practice: The paper fits several clusters at once:
- Cluster 2 (LLM behavioral patterns): The key finding is the "primacy effect", the model's tendency to follow instructions at the top of the prompt more reliably.
- Cluster 3 (Prompt structure optimization): Shows directly that instruction order is critical.
- Cluster 7 (Reliability and stability): Explains why reliability drops as the number of instructions grows and how to anticipate it.
- Practicality checklist (+15 points): Yes, the study explains where to put important information in a prompt, how to structure complex requests, and reveals non-obvious quirks of LLM behavior.
Numeric Usefulness Score
The study earns a high score because its findings are fundamental and directly usable by anyone to improve their prompts. It doesn't hand you a mere "trick"; it gives you a conceptual understanding of LLM limits, which is far more valuable in the long run.
Arguments for a high score: 1. A direct call to action: The "primacy effect" finding is a golden rule of prompt engineering. The study puts science behind it and shows a systematic pattern across 20 models, strongest at around 150-200 instructions. Users can start putting their most important instructions first right away and get better results. 2. Explains "why it doesn't work": The paper explains why an LLM sometimes "ignores" part of your request. It's not the model being "dumb"; it's "cognitive overload". Understanding that helps users stop fighting the model and instead adapt their requests by splitting or simplifying them. 3. Broad relevance: The test used a specific task (including keywords), and the authors caution it may not generalize, but the ideas about attention and cognitive load are worth trying in other tasks, from essay writing to code generation.
Counterarguments (why the score could be lower): * An artificial test: The test task (fitting up to 500 keywords into a report) is pretty niche and rarely comes up in everyday use. People usually give instructions of different kinds (style, format, content limits), not just a list of words. So extrapolating the numbers directly (e.g., "the model breaks after 150 instructions") may not hold for other tasks. * No ready-made "templates": The study gives principles, not copy-paste prompt templates. Users still have to figure out for themselves which of their instructions matters most.