Apply to my task
arXiv:2507.11538 ★ 92 Jul 15, 2025

The primacy effect: LLMs do what comes first, and the rest falls off the end

Key result

An LLM doesn't forget instructions at random; it systematically ignores the ones at the end of the prompt. It's not a bug and not a fluke. It's a stable pattern: the model's attention flows top to bottom and runs out. Controlling instruction order lets you build prompts where critical requirements always get met, even when there are lots of instructions. The trick: put the most important things first: the role, the main goal, the key format requirement. Anything at the bottom the model may skip: not mix up, but outright skip. Those are fundamentally different errors.

How Many Instructions CanLLMsFollow at Once?

Key Aspects of the Study:

The researchers tested how many simultaneous instructions modern LLMs can handle before their performance starts to drop. They found that every model has a "cognitive load" ceiling, and once it is hit, the model starts ignoring instructions wholesale. The most important discovery: models remember and follow instructions given at the start of the prompt far better than the rest (the "primacy effect").

Key Result: Always put your most important instructions at the very top of the prompt, because that is where the LLM spends most of its "attention".


The Core of the Method, Explained:

The method that falls out of this study can be summed up as "managing the LLM's attention and cognitive load". Picture handing tasks to an intern. Give them 3 clear tasks and they will probably nail all of them. Give them a list of 50 and they will likely remember the first few and forget or mix up the rest. LLMs behave in much the same way.

Main takeaways and a playbook for users:

  1. The "Important stuff first" rule (Primacy Effect): The instruction that matters most to you should sit at the very top of the prompt. It can be a role ("You are a seasoned marketer"), the core goal ("The main thing is to convince the reader to buy the product") or a format requirement ("Return the answer as a JSON object"). Anything that comes after is more likely to be ignored.

  2. Know when to stop (Cognitive Load): Don't try to cram dozens of unrelated requirements into a single prompt. The more instructions you add, the higher the odds the model starts skipping them (and not at random: mostly the ones near the end). If your task is truly complex, split it into several sequential requests instead of building one giant, unreliable prompt.

  3. Expect the right kind of mistake: When a model "gets tired", it usually just skips (omission) instructions rather than executing them wrong. If you asked for 10 facts and got only 6, the model most likely "forgot" the last 4 rather than botching them.

  4. Don't trade quality for quantity: The study showed that instruction overload hurts not only compliance but also the quality of the core task (the report text, for example, became less coherent). Better to give 3-5 key directions and get a solid result than 20 directions and get a rambling text that only half-follows them.


Practical Applicability Analysis:

  • Direct applicability: Extremely high. Anyone can improve their prompts right away just by reordering instructions. Instead of writing "Write a piece about the environment, be positive, use a bulleted list, and add a call to action at the end", the user writes: "Act as an environmental expert. Your main task is to write an upbeat piece with a call to action at the end. Use a bulleted list for the facts". This simple reshuffle dramatically improves the odds of success.

  • Conceptual value: It gives users a simple, powerful mental model: "An LLM's attention is a finite resource that gets spent top to bottom through the prompt". That helps people stop treating the LLM as a magic box and start working with it as a system with clear rules and limits.

  • Adaptation potential: The principle is universal. It works for writing, code generation, data analysis, planning and so on. Adapting it is easy: before writing a complex prompt, ask yourself, "Which single instruction is absolutely critical to success? What comes second?" The answers define the structure of your prompt.


A Practical Example:

Say a user wants an LLM to write a script for a short promo video (Reels/Shorts) about a new fitness app.

# TASK

You are an experienced screenwriter and creative director who specializes in short viral videos for social media (Reels, Shorts, TikTok).

Your main goal is to write a script for a 15-second video that makes viewers really want to download our new fitness app "FitGo".

---
# CONTEXT

*   **Product:** The "FitGo" mobile app.
*   **Key feature:** An AI coach that builds personalized workouts based on the user's goals and current fitness level.
*   **Target audience:** People aged 20-30 who want to start working out but don't know how, or find it boring.

---
# SCRIPT STRUCTURE AND REQUIREMENTS

1.  **Response format:** Present the script as a table with three columns: "Shot (visual description)", "Copy (voiceover or on-screen text)", "Timing (seconds)".
2.  **Tone:** Energetic and motivating, but not pushy. Avoid clichés like "change your life in one day".
3.  **Key message:** "FitGo is your personal trainer in your pocket that makes fitness simple and fun".
4.  **Must-have element:** The video must end with a clear call to action (CTA): "Download FitGo for free and start your first workout!".
5.  **Restriction:** Do not mention competitors.

Why It Works:

This prompt follows the study's findings to the letter:

  1. Primacy effect: The most important instructions sit at the top.
    • Role (You are an experienced screenwriter...): Sets the context and the quality bar. This is the first and most important instruction.
    • Main goal (...makes viewers really want to download...): Defines the end result the model should aim for. This is the second most important instruction.
  2. Managing cognitive load: Instead of one long list of requirements, they are grouped into logical blocks (CONTEXT, REQUIREMENTS). Key requirements are numbered, which also helps the model process them, and the most important ones (format, tone, CTA) lead the list.
  3. Clarity and prioritization: The prompt explicitly calls out the "main goal" and the "key message", helping the model see where to point its "attention" first. Less important details, like "do not mention competitors", sit at the end of the list, so even if the model ignores them, the core result won't suffer.

Another Practical Example

Task: a user asks an LLM to help draft a polite but firm letter to a neighbor asking them to keep it down after 11 p.m.

# TASK

Act as an experienced conflict-resolution specialist and mediator.

Your main task is to help me write a letter to my neighbor. The main goal of the letter is to ask them, politely but very firmly and unambiguously, to stop making noise (loud music, shouting) after 11 p.m.

---
# KEY LETTER PARAMETERS

1.  **Tone:** Balance is everything. The letter must be **non-confrontational and respectful**, yet **confident and firm**. No passive aggression and no apologizing.
2.  **Structure:**
    *   Open with a friendly greeting.
    *   Describe the problem without blaming (e.g., "I've noticed that lately...").
    *   State the request clearly (stop the noise after 11 p.m.).
    *   Offer a solution or a conversation.
    *   End on a positive note.
3.  **What to avoid:** Do not use threats (calling the police, complaining to the landlord) in this first letter. Focus on a good-neighborly solution.

Why This Example Works, Mechanically.

This prompt works because it also applies the principles of managing LLM attention:

  1. Role and Goal come first: The prompt opens by assigning a role (conflict-resolution specialist) and stating the main goal (politely but firmly ask). That tunes the model in right away and sets the direction of the output.
  2. Focus on the hardest part: The trickiest part of the task is getting the tone balance right. That is why the tone instruction (non-confrontational, yet confident and firm) is first in the requirements list. The model will put maximum "attention" on exactly this critical aspect.
  3. Structure for clarity: Rather than just saying "write a polite letter...", the prompt breaks the task into clear structural pieces (greeting, problem description, request). That lowers the model's cognitive load and helps it work through every part of the assignment in order.
  4. Negative constraints at the end: The instruction about what not to do (Do not use threats) is placed last. It is a secondary constraint. If the model "forgets" it, that is far less damaging than forgetting the right tone. In other words, we deliberately gamble on a less important instruction to reliably land the more important one.

Usefulness Score: 92

02

Main Evaluation Criteria

  • A. Relevance to prompting techniques: Yes, the study directly analyzes how the number and position of instructions affect the result.
  • B. Better conversational answers: Yes, the findings help make answers more accurate and predictable when working with complex, multi-part requests.
  • C. Direct practical applicability: Yes, the key findings (especially the "primacy effect") can be used immediately in any chatbot without special tools or knowledge.
  • D. Conceptual value: Very high. The study gives users an intuitive grasp of an LLM's "cognitive load" and "limited attention", explaining why models "forget" or ignore some instructions.
  • E. New useful practice: The paper fits several clusters at once:
    • Cluster 2 (LLM behavioral patterns): The key finding is the "primacy effect", the model's tendency to follow instructions at the top of the prompt more reliably.
    • Cluster 3 (Prompt structure optimization): Shows directly that instruction order is critical.
    • Cluster 7 (Reliability and stability): Explains why reliability drops as the number of instructions grows and how to anticipate it.
  • Practicality checklist (+15 points): Yes, the study explains where to put important information in a prompt, how to structure complex requests, and reveals non-obvious quirks of LLM behavior.
03

Numeric Usefulness Score

The study earns a high score because its findings are fundamental and directly usable by anyone to improve their prompts. It doesn't hand you a mere "trick"; it gives you a conceptual understanding of LLM limits, which is far more valuable in the long run.

Arguments for a high score: 1. A direct call to action: The "primacy effect" finding is a golden rule of prompt engineering. The study puts science behind it and shows it is not a fluke but a systematic pattern. Users can start putting their most important instructions first right away and get better results. 2. Explains "why it doesn't work": The paper explains why an LLM sometimes "ignores" part of your request. It's not the model being "dumb"; it's "cognitive overload". Understanding that helps users stop fighting the model and instead adapt their requests by splitting or simplifying them. 3. Universality: Although the test used a specific task (including keywords), its conclusions about attention and cognitive load are universal and apply to any task, from essay writing to code generation.

Counterarguments (why the score could be lower): * An artificial test: The test task (fitting up to 500 keywords into a report) is pretty niche and rarely comes up in everyday use. People usually give instructions of different kinds (style, format, content limits), not just a list of words. So extrapolating the numbers directly (e.g., "the model breaks after 150 instructions") may not hold for other tasks. * No ready-made "templates": The study gives principles, not copy-paste prompt templates. Users still have to figure out for themselves which of their instructions matters most.


Research digest

Key result

An LLM doesn't forget instructions at random; it systematically ignores the ones at the end of the prompt. It's not a bug and not a fluke. It's a stable pattern: the model's attention flows top to bottom and runs out. Controlling instruction order lets you build prompts where critical requirements always get met, even when there are lots of instructions. The trick: put the most important things first: the role, the main goal, the key format requirement. Anything at the bottom the model may skip: not mix up, but outright skip. Those are fundamentally different errors.

Principle

Don't toss instructions in any old order; rank them by importance. The A→B→C approach: A: Role and main goal: whatever defines the whole result goes first. The model "sees" it before everything else. B: Key requirements: response format, tone, must-have elements. Second priority. C: Constraints and nice-to-haves: "don't mention competitors", "avoid clichés". If the model ignores these, it's not the end of the world. The rule is simple: the more critical the instruction, the higher it goes in the prompt. Knowingly risk the less important stuff to reliably land the important stuff.

Why it works

The study documented a stable "primacy effect": models pay significantly more attention to instructions at the start of the prompt than at the end. When overloaded, models don't get confused, they skip. And the skips come specifically from the tail of the prompt, not at random. An LLM's attention is a finite resource that gets spent top to bottom. Beyond the omissions, the output itself suffers too: with an overly long list of requirements, the text becomes less coherent even in the parts where instructions were followed. Fewer instructions, better quality for each one.

When to apply

Anywhere you have more than two or three requirements for the answer: writing, code generation, analysis, planning, drafting emails. Especially critical for: — Complex tasks with format requirements (table + tone + structure + CTA) — Tasks where one missed element breaks the whole result (JSON format, legal wording) — System prompts with a long list of rules NOT a "fix" if the task itself needs 20+ simultaneous constraints; in that case, split it into several sequential requests.

Mini recipe

1. Rank by criticality: before writing the prompt, ask yourself: "which single instruction matters most? what's second?". The answers define the structure. 2. Put role and goal first: not 'write a piece about X, and you're an expert in Y', but the other way round: first you're an expert in Y, then write a piece about X. 3. Group by importance: key format and tone requirements come right after the goal. Minor constraints go at the very end. 4. Trim the fat: if you have 10+ instructions, think about which ones actually affect the result. Often 3–5 key ones beat 15 added "just to be safe". 5. Check the tail: look at the last 2–3 instructions. If the model ignores them, is that a problem? If yes, move them up.

Examples

[BAD]: `Write a fitness post, be motivating, use a bulleted list, add a call to download the app, use plain language, no clichés, keep it under 150 words` [GOOD]: `You're a copywriter for short motivational social media posts. Main goal: convince the reader to download the FitGo fitness app. Tone: energetic, no "change your life" clichés. Format: bulleted list of facts + a call to action at the end. Length: up to 150 words. Avoid jargon.` What changed: the role and goal moved to the front, so the model knows what it's aiming for before reading the rest. The "no clichés" constraint moved above the throwaway word-count line. The most important thing, convincing people to download, won't get lost even if the model "tires out" by the end of the list.

In plain words

Today's AI models act like know-it-alls, but in reality they're like a waiter at the dinner rush: order a coffee, a dessert and the check at once and he'll manage, but dump 20 conditions on him at once and he'll start mixing up orders or just bring you an empty cup. The study clearly shows that models have a cognitive load ceiling. As soon as the number of instructions in a single prompt crosses a critical mark, answer quality falls off a cliff. These aren't just random "glitches"; it's a fundamental architectural limit: the model simply can't keep its focus on a dozen different constraints without sacrificing meaning.

It's like juggling chainsaws while singing the national anthem and riding a unicycle. Technically you're doing it all at once, but in practice either the song comes out mumbled or a chainsaw lands on your head. In LLM land it looks like this: you ask the model for a text "with no adjectives, in the style of Hemingway, mentioning five brands, and in Latin, please", and you get back mush where half the conditions are simply ignored. The model takes the path of least resistance and drops the "extra" details just to produce something coherent.

What actually works: hierarchy and prioritization. The researchers found that if you pack your requirements into a structured block and call out the main task, the model holds up longer. But even top-tier models like GPT-4 or Claude 3 start fumbling once the instruction count passes 10-15. The most effective approach is atomicity: instead of stuffing style, structure and a banned-words list into one prompt, break the job into stages. First ask for the substance, then a style pass, and only at the end a check against the hard constraints.

The principle is universal: it works for code generation, scriptwriting and heavy analytics alike. Whether you're asking for a workout plan or a smart contract, the law of diminishing returns applies everywhere. If your prompt has more than five "don'ts" and seven "musts", you're already playing the lottery. Tests across different domains confirmed it: the more specific and numerous the requirements, the higher the chance the model will hallucinate or just ignore the context.

Bottom line: stop playing criminal mastermind and stuffing the model with endless instructions in a single message. Fewer conditions, higher accuracy. If you want a perfect result, use a step-by-step approach and don't make the model juggle too many balls. Otherwise you won't get a masterpiece, you'll get a mediocre brush-off that technically has everything but is useless in practice.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on The primacy effect: LLMs do what comes first, and the rest falls off the end arXiv:2507.11538

Sign in to apply this study New accounts get 50 free credits.

Get the week’s best papers by email

One email on Sundays with the highest-rated new techniques. Unsubscribe with one click.