Reply Requirements: How to Make an Agent Answer Clearly, in One Message at the End of Its Work

OPTION 1 (one chat, cheap):
  Agent finishes the work → you add a message with the requirements
  → agent writes the reply following the 6 points

OPTION 2 (two passes, stronger):
  STEP 1: Agent writes its reply as usual → draft
  STEP 2: New chat: your conversation + work log + draft + requirements
         → rewritten reply
  STEP 3: Check against the log: nothing lost, nothing invented

The 6 requirements: answer to the latest message → what was done → result → next step → where to check → definitions of every unfamiliar name.

Task: The owner of an Amazon store asks Claude Code to dig through the quarter's order export: why did conversion drop on the electronics product pages? The agent spent an hour crawling through files, running scripts and taking notes. It comes back with something like "for segment seg_B2 the effect is in variant v3_cut, see the table from run_14." You have no idea what seg_B2, v3_cut or run_14 are. So you add the requirements message.

Prompt (sent after the agent has finished working, before its final reply):

Write your final reply to me. Keep in mind: I have only seen my own messages
and your earlier replies. I have not seen your files, notes, script output,
or any names you made up while working.

The reply must contain, in this order:
1. A direct answer to my latest question: why conversion dropped
   on the electronics product pages.
2. What you did (2–4 bullets, no internal names).
3. The result: the key numbers and the conclusion.
4. The next step: what I need to decide or do.
5. Where to check: which file, sheet or report holds the supporting
   data. Give the path and what to look for.
6. A definition of every name, label, number or abbreviation
   I haven't seen. Better yet, replace it with a plain-words description.

Write in English.

Result: The agent opens with a direct answer to the question instead of a play-by-play of its work. Instead of "seg_B2" you get a description like "shoppers who came in from ads." Internal identifiers are either replaced with words or spelled out. At the end it tells you where the data lives so you can check it. In the study, this trick cut the share of unusable replies from about 52% to 43%, and replies had fewer errors and omissions.

The model's weakness. The agent writes its reply inside its own working context. That's where its notes, run names, source numbers and script outputs live. To the agent, all of that is "common knowledge." The model can't tell "I know this" apart from "the user knows this." So it keeps talking in its own working dialect. More than half of the invented labels were written into the agent's files and memory before it ever replied. Every new session drags those old labels back in.

The model's strength. It's good at following a concrete checklist. "Define every name I haven't seen" is a checkable action: go through the text and spell things out. "Keep the reader in mind" doesn't define any action like that. That's why the notice "the user has only seen their own messages" had almost zero effect, while the same notice plus six requirements made a noticeable difference.

How the method exploits this. The requirements go at the end, right before the reply, and each point closes off a specific type of failure. "Answer the latest question first" fixes replies that miss the point. "Definitions" fix invented labels and opaque jargon. "Where to check" swaps a "see R07" reference for a path you can actually open yourself. A second pass in a fresh chat works even better: the rewriter doesn't drag in working jargon because it sees only what you saw.

Control knobs: - The checklist → add your own items for the task ("risks," "what's still unknown"). Drop extras if the reply needs to be short. - Where the message goes → put it at the end, right before the reply. In the paper, what worked was a message at the point of reply. - The second pass → turn it on for long tasks with lots of tool calls, that's where failures pile up. - The work log → always give it to the rewriter, otherwise it may invent details. Then check the text against the log.

Option 1. Requirements message (in the same agent chat/session, before the final reply):

Write your final reply to me. Keep in mind: I have only seen my own messages
and your earlier replies. I have not seen your files, notes, project memory,
tool output, or any names you introduced yourself while working.

The reply must contain, in this order:
1. A direct answer to my latest message: {my_latest_question}.
2. What you did.
3. The result.
4. The next step.
5. Where to check: the file path, sheet, section or command where
   the supporting data lives.
6. A definition of every name, label, identifier or abbreviation
   I haven't seen. Better yet, replace it with a plain-words description.

Write in {language}.

Option 2. The rewriter (new chat, a second model or the same one):

You are rewriting an agent's final reply for the client.

The client has seen only:
<conversation>
{my_messages_and_earlier_agent_replies}
</conversation>

The client has NOT seen the agent's work log:
<work_log>
{tool_calls_outputs_notes_files}
</work_log>

The agent's draft reply:
<draft>
{draft_reply}
</draft>

Rewrite the reply so the client can understand it from the conversation
and the text itself. The reply must contain, in this order:
1. A direct answer to the client's latest message.
2. What was done.
3. The result.
4. The next step.
5. Where to check the supporting data.
6. A definition of every name, number or abbreviation the client
   hasn't seen. Or replace them with a description.

Rules: do not add any facts that are not in the log. Do not drop key
results from the log. Reply language: {language}.

What to fill in: {my_latest_question} is your last message. {language} is the language you read in. For option 2, paste in the conversation, the log (an export of the agent's steps or its notes) and the draft.

🚀 Quick start — paste into the chat:

Here is a template of requirements for an agent's reply. Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what you asked the agent to do, what language you read in, and what you need to verify in the result. It needs that to make the six points match the failures you actually run into. It'll keep the template's structure and tailor it to your task.

⚠️ The effect is moderate: Requirements cut the share of unusable replies from 0.52 to 0.43. That's better, but about forty percent of replies are still hard to understand. The method doesn't fully solve the problem.

⚠️ A "reader warning" doesn't work: If you write "the user can't see your work" in the system prompt, replies barely change. You need a concrete list of contents, and it has to land at the moment of the reply.

⚠️ Rewriting can distort facts: The researchers checked every rewritten reply against the work log. A rewriter without the log risks losing results or adding extras. Rewrite with the log, then verify.

⚠️ You need access to the log: The second pass requires pulling the work log or notes out of the agent. In a chat without tools that's easy; in an off-the-shelf agent you'll have to copy it by hand.

⚠️ Lab conditions: The client was played by a script with short directives. The reader was a reader model, validated against humans at the level of agreement of an individual annotator. A real client with different context may see failures differently.

⚠️ Incomplete details: The available text doesn't include the verbatim prompts from the appendix, so the templates are reconstructed from the paper's description.

The idea was simple: an agent reports to a person who hasn't seen its work, and that person has to understand the report. The researchers took 8 models from 7 families. Each one worked through four projects: two research projects and two business ones, such as changing a subscription price and redesigning onboarding. That produced 32 long sessions and 480 replies. The client was played by a script: 15 short messages across three sessions, and between sessions the agent saved notes into "project memory."

Reading the replies was handed to a reader model. It steps into the client's shoes, asks questions about anything unclear, and looks for answers in the reply text and the visible conversation. No answer means an information gap. The reader was validated on 300 replies: five humans labeled them independently, and the reader's agreement with the majority (κ = 0.60) matched the humans' agreement with each other (0.63).

Result: 54% of replies were unusable; the best model failed 30% of the time, the worst 82%. The surprise: Claude Opus 5 did worst and GPT-5.6 Sol did best, with GPT producing no invented labels at all. After three or more tool-using turns, failures hit 70%, versus 48% with no tool calls. Over 53% of invented labels had been written by the agent into its files or memory before it replied. The takeaway: the reply speaks the agent's working dialect because that dialect is already sitting in its context.

Next, on 256 reply points, they compared six ways of rewriting the reply. A plain "the user has only seen their own messages" gave 0.50 versus 0.52 (within the noise). The requirements message gave 0.43, and a fresh chat with requirements gave 0.42. A fresh chat without requirements gave only 0.47. A second model rewriting the reply against the requirements brought the share down to 0.36, and a blind human comparison confirmed it. The practical takeaway: what matters is the concrete list of requirements, not the separate pass by itself.

The full prompts are in the appendix (not in the available text). Here is how the main text describes the requirements:

The requirements condition appends a message at the point of the reply
that repeats this statement and specifies what the reply must contain:
an answer to the latest message first, what was done, the result,
the next step, where the evidence can be checked, and a definition
of every name the user has not seen.

Notice condition: adds to the system prompt a statement that the user
has seen only their own messages and the agent's earlier replies.

Context: This is how the researchers phrased the two instructions. The second one ("requirements" at the end) worked; the first one ("notice" in the system prompt) had no significant effect.

💡 Adaptation for CLAUDE.md / AGENTS.md: move the six points into a standing instruction for the agent. The link to the paper's findings is indirect: in the study, requirements were added at the point of reply, not in an instructions file, so check that the agent still follows them at the end of long tasks.

## Final reply to the user
The user cannot see your files, notes, tool output,
or names you made up yourself. Every final reply contains,
in order: a direct answer to the latest question; what was done; the result;
the next step; where to check (file path or command); a definition
of every name the user hasn't seen.
Do not use internal identifiers (run_14, R07, seg_B2) without spelling them out.

🔧 Technique: rename labels at write time → less jargon in memory. The paper's data suggests that some labels come from files and memory. You can add to the instruction: "When you write notes to a file, put a plain-words explanation next to every label." This is my extrapolation; the paper did not test this intervention.

Paper title: Measuring and Reducing Unusable Replies of LLM Agents Authors: Yilin Jiang, Fei Tan, Xuanyu Yin, Guanchu Wang, Xia Hu, Yutao Yue Affiliations: East China Normal University; The Hong Kong University of Science and Technology (Guangzhou); Shanghai Artificial Intelligence Laboratory; Institute of Deep Perception Technology, JITRI Related ideas from the paper: decontextualization (Choi et al., 2021; Newman et al., 2023), common ground in dialogue (Clark & Brennan, 1991; Shaikh et al., 2024).