TL;DR
Long conversations with an LLM have 9 types of characteristic failures — Interaction Smells. The study cataloged them by analyzing thousands of real conversations. The gist: with every new message, the model gets worse at following requirements from earlier messages — not because it "gets dumber", but because early context literally loses weight against the fresh stuff.
The most common failure is Must-Do Omission (38% of cases): the model "forgets" mandatory rules you set earlier. You wrote at the start "always give examples", "use only a professional tone", "don't suggest option X" — and 6-8 messages later the model has stopped doing it. You only notice the mistake at the end, when it's time to redo everything.
InCE (Invariant-aware Constraint Evolution) is the principle that fixes this: explicitly extract every constraint from the conversation and check against them before generating a response. Two steps: first the model writes out a list of all the "rules of the game", then it checks each response against that list. It works manually in a regular chat.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
COMPONENT 1 — Invariant extraction (IEM)
→ The model lists ALL active constraints from the conversation history
→ Mandatory requirements + prohibitions + decisions already made
→ Format: an explicit numbered list
COMPONENT 2 — Pre-response audit (PSD)
→ The model checks: does the planned response violate even one constraint?
→ If it does, it says so BEFORE generating
→ Only if everything is clean does it give the answer
Both components run in ONE prompt.
Insert it as a checkpoint every 5-7 messages or whenever you add a new requirement.
A Taxonomy of Interaction Smells
It helps to know exactly what breaks. Here are the 9 types:
Category 1 — The quality of your request - Ambiguous Instruction — the request can be read different ways, so the model guesses - Incomplete Instruction — key details are missing, so the model fills them in itself
Category 2 — The model breaks your rules (the most painful) - Must-Do Omission ⚠️ 38% — the most frequent — forgets mandatory requirements from early messages - Must-Not Violation — does what you explicitly prohibited
Category 3 — The model breaks what already worked - Signature Mismatch — changes the structure/format without warning - Cross-Turn Inconsistency — contradicts itself between messages - Partial Functionality Breakdown — a new response breaks something agreed on earlier - Code Rollback — brings back bugs that were already fixed - Repetitive Response — repeats the same thing instead of moving forward
Example
Task: You're working with Claude on a content strategy for a Substack newsletter. The session is already 10+ messages long. At the start you agreed: "B2B only", "no consumer-brand case studies", "every point with a metric". A few messages later the model starts citing Netflix and adding sections without numbers.
Prompt:
Before continuing, do two steps:
**STEP 1 — Conversation invariants:**
List all active constraints from our conversation:
- Mandatory requirements: what must be present in every response
- Prohibitions: what must not be done or used
- Decisions made: what we've already agreed to use
- Formats and styles: the established structure
If there are no constraints, write "No constraints recorded."
**STEP 2 — Audit:**
Check: does your planned next response violate
even one of the listed constraints?
If so, say so and suggest how to respect the constraint
while still completing the task.
After these two steps, do this: write the section on post formats.
Result: The model first shows a full list of everything it's "keeping in mind" — valuable in itself: you immediately see what it has "lost". Then it explicitly checks whether the next section breaks any rule. Only after that does it write the section on post formats. If it "forgot" the metrics requirement, you'll see it before the answer, not after.
Why It Works
The LLM's weakness: context decay. In long conversations the model pays more attention to recent messages. Early requirements literally "sink" — they don't vanish from memory entirely, but they influence the answer less and less. That's exactly why Must-Do Omission leads at 38%: the rule isn't forgotten, it's "crowded out" of active influence by new content.
The LLM's strength: working with what's explicit. The model is great at following requirements written right now in the current message. If a constraint is visible and explicit, it will follow it. The problem arises precisely when the requirement was "a while ago, somewhere back there".
How InCE uses this: The method carries "faded" constraints from early messages into the current one — making them explicit and visible again right at the moment of generation. There's no architectural magic here: what was buried in an old message is now right in front of the model's "eyes".
Control levers: - Audit frequency — every request or only when the topic changes: for critical sessions, do it more often - Listing depth — "briefly" or "break it down by category", depending on task complexity - Reaction to a violation — add "if a violation is unavoidable, ask me before answering" for especially important requirements - When to apply — insert the template at the start of a session as a "vaccine" or in the middle as a "diagnostic"
Prompt Template
Before answering, do two steps:
**STEP 1 — Conversation invariants:**
List all active constraints from our conversation:
- Mandatory requirements: [what must be present in every response]
- Prohibitions: [what must not be done / used]
- Decisions made: [what we've already agreed to use]
- Formats and styles: [the established structure and tone]
If there are no constraints, write "No constraints recorded."
**STEP 2 — Audit:**
Check: does your planned response violate even one
of the listed constraints?
If so, say so and propose a solution before answering.
**Task:** {your next request}
What to fill in: In {your next request} — any step within the current session. Insert this template once a session passes 5–7 messages, when you add a new requirement, or when it feels like "something went wrong".
🚀 Quick start — paste into the chat:
Here's a template for keeping constraints in check during long conversations.
Adapt it to my task: [your task — for example,
"I'm writing an investor pitch deck together with you"].
Ask me questions to figure out which requirements I need to track.
[paste the template above]
The LLM will ask which requirements are critical for you (tone, format, content constraints, decisions made) — because without that the invariants list will be empty and useless. It will take the template's structure and tailor it to the context of your task.
Limitations
⚠️ Token weight: In long sessions the invariants list grows. Listing every constraint each time adds volume. On very long sessions it's noticeable. Fix: run the audit every 5–7 messages rather than every time.
⚠️ The model can "forget" during extraction too: If a requirement was at the very start of a very long conversation, the model may fail to recall it during the IEM step — the same problem we're trying to solve. Insurance: keep your own short list of constraints and add it explicitly.
⚠️ Doesn't cure the first category of smells: Ambiguous and Incomplete Instruction are about the quality of your requests, not the model's memory. InCE doesn't help with them. The only fix is to sharpen your requests yourself.
⚠️ Overkill for short sessions: If the conversation is 2–3 messages long, the template only adds complexity with no benefit. It works where rules have actually piled up.
How They Studied It
The team took two huge datasets of real conversations — WildChat (1 million conversations with ChatGPT/GPT-4) and LMSYS-Chat-1M (1 million conversations with 25 models) — and filtered down to the ones about writing code. That gave ~66,000 conversations. They then randomly picked 378 multi-turn ones and handed them to a team of 4 PhDs and 4 grad students for manual analysis using "open card sorting": you look at each conversation and come up with failure categories yourself, with no predefined framework.
What's interesting in the numbers: Must-Do Omission took the lead by a huge margin — 38%. Meanwhile user errors (Ambiguous + Incomplete Instructions combined) scored only ~8%. That flips expectations: we're used to thinking a bad result = a bad prompt. But the study says the main problem isn't how you phrase things — it's that the model stops following rules that were already set.
They then tested six current models — GPT-4o, DeepSeek-Chat, Gemini 2.5, Qwen2.5-32B, Qwen2.5-72B, Qwen3-235B — and confirmed that Must-Do Omission and Partial Functionality Breakdown reproduce on every single one. A more powerful model doesn't guarantee protection from these failures. Finally, the InCE framework cut the frequency of critical smells by about 13.5% and raised task success by 6.67% — modest but reproducible numbers.
Adaptations and Extrapolations
🔧 "Constraint crystal" — a vaccine before the session starts
Instead of extracting invariants mid-session, lock them in with the very first message:
We're starting work on {project}.
Set the following invariants for our entire session:
REQUIRED in every response:
- {requirement 1}
- {requirement 2}
PROHIBITED:
- {constraint 1}
- {constraint 2}
DECISIONS MADE:
- {decision 1}
Confirm you've understood all the constraints.
Repeat them at the start of each of your responses
until I say "drop the invariants".
This prevents Must-Do Omission from the outset instead of treating it mid-session. Visibly repeating them at the start of every response is an easy way to keep the invariants "in focus" for the whole session.
🔧 "Smell diagnostics" — auditing a conversation already in progress
If it feels like "something went wrong" but you can't tell what exactly:
Check our current conversation against three criteria:
1. Must-Do Omission — are there requirements from early messages
that you've stopped following?
2. Cross-Turn Inconsistency — are you contradicting yourself
between different responses?
3. Partial Breakdown — have you broken anything
that was agreed on earlier?
For each item: either "clean"
or "found: [what exactly and in which message]"
Resources
An Empirical Study of Interaction Smells in Multi-Turn Human-LLM Collaborative Code Generation
Binquan Zhang, Li Zhang, Lin Shi, Song Wang, Yuwei Qian, Linhui Zhao, Fang Liu, An Fu, Yida Ye
Datasets: WildChat, LMSYS-Chat-1M, WildBench v2