Episodic Memory Through Reflections: How an LLM Learns From Its Mistakes

LEARNING LOOP (on the first batch of questions):
STEP 1: Answer → get the top 10 relevant conversation snippets, answer the question
STEP 2: Evaluate → compare the answer with the correct one, compute the F1-score
STEP 3: Reflect → analyze the mistake, describe its cause and how to avoid it
STEP 4: Store → save {question, context, answer, correct answer, reflection}

APPLICATION (on new questions):
STEP 1: Search → find the top 3 similar cases in memory
STEP 2: Context → add them to the prompt as examples
STEP 3: Answer → use the examples and reflections to answer

The learning loop and the application are separate requests to the model.

The task: You've been emailing a client through your CRM for months: discussing the project, revisions, payments, deadlines. 300+ messages over six months. The client asks: "Did we ever discuss the video testimonials option?" You need an accurate answer from the history, not a hallucination.

Prompt for building episodic memory (first iteration):

You're helping me answer a client's questions based on our correspondence. 

Here are 5 questions and snippets of the correspondence:

[Question 1]: When did the client first mention the budget?
[Context]: {top 10 relevant messages}
[Your answer]: "In early March"
[Correct answer]: "March 12, in the message about approving the spec"

[Question 2]: Did we discuss a discount for prepayment?
[Context]: {top 10 relevant messages}
[Your answer]: "Yes, 10%"
[Correct answer]: "No information"

...3 more questions...

For each question, analyze:
- Did you answer correctly, and why?
- If the question was about a date, did you get the time reference right?
- If the information wasn't there, did you correctly say "no information"?
- What mistake did you make, and how can you avoid it in the future?

Write a short reflection (2-3 sentences) for each.

Prompt for applying the memory:

Answer the client's question using the correspondence snippets and past examples.

EXAMPLES FROM THE PAST:

[Example 1]
Question: When did the client first mention the budget?
Context: {messages}
Correct answer: "March 12, in the message about approving the spec"
My answer: "In early March"
Reflection: "I gave an approximate answer instead of the exact date. I should use the specific date from the message if there is one."

[Example 2]
Question: Did we discuss a discount for prepayment?
Correct answer: "No information"
My answer: "Yes, 10%"
Reflection: "I made up information that wasn't in the correspondence. If the topic never came up, I should honestly say 'no information' and not fill in the gaps."

[Example 3]
Question: What revisions did the client request in the second phase?
Correct answer: "Change the button colors and remove the extra blocks"
My answer: "Change the design"
Reflection: "The answer was too general. I need to name the specific revisions from the messages."

CURRENT QUESTION:
Question: {client's question}
Correspondence snippets: {top 10 relevant messages}

Use the past reflections. If there's no information, say "no information". If there's a date, give the exact one. The answer should quote the correspondence precisely.

Answer:

Result: The model will see the patterns from past mistakes: don't fill in gaps, use exact dates, quote specifics. The answer will be either an exact phrase from the correspondence or an honest "no information", with no hallucinations.

An LLM generates text from patterns; it doesn't "recall facts". Long context = more patterns, more noise, more confusion. Especially in conversations with a time structure ("when did we discuss X") or where you need to say "I don't know" instead of guessing.

The LLM's strength is learning from examples (few-shot learning). Show the model "here's what a correct answer looks like" and "here's a typical mistake", and it will adapt to the pattern. Reflection adds a meta level: the model sees not just "right/wrong" but why it was wrong.

EpMem exploits this: instead of 20,000 tokens of full history, we give 3 cases with reflections (~1,000 tokens). The model sees the context of the mistake ("made up a fact"), the reflection ("don't make things up") and the correct answer ("no information"). That trains the pattern: when a semantically similar question comes up, apply a similar answering strategy.

The quirk of adversarial questions (questions whose answer isn't in the context): they're semantically similar to real conversation topics. For example, "Did we discuss a discount for prepayment?" might look like "Did we discuss payment terms?", but the first never happened. Plain RAG will pull "payment terms" as relevant → the model says "yes". The reflection teaches: "if the exact topic isn't there, say no; don't extrapolate".

Control levers:

• Number of examples (currently 3) → raise to 5 for harder tasks, lower to 1 to save tokens
• First batch of questions → pick the most typical mistakes so the reflections cover different failure types
• Reflection structure → set a specific focus: "analyze only time-related errors" or "focus on hallucinations"
• Similarity metric → use semantic search or categorical matching (question type: date/fact/no answer)

Building episodic memory (the learning loop):

You answer questions about {context type: a conversation/document/history}. 

Here are {number} examples of your answers:

{question}: {question}
{context}: {relevant snippets}
{your_answer}: {your answer}
{correct_answer}: {reference answer}

...more examples...

For each one, analyze:
- Did you answer correctly? Why?
- If the question required {specifics: an exact date/facts/recognizing missing info}, how did you do?
- What mistake did you make?
- How can you avoid it in the future?

Write a short reflection for each (2-3 sentences).

Store each set of {question, context, your_answer, correct_answer, reflection} in memory.

Applying the memory:

Answer the question using {context type} and past examples.

EXAMPLES FROM THE PAST:
{top 3 similar cases with reflections in the format:
Question: ...
Context: ...
Correct answer: ...
My answer: ...
Reflection: ...}

CURRENT QUESTION:
Question: {new question}
Context: {relevant snippets}

Use the reflections from the examples. {Answer rules: exact quotes/dates/no information}.

Answer:

Placeholders:

• {context type} — a conversation, a document, an email history
• {number} — 5-10 examples for the first loop
• {specifics} — what matters for the task (dates, facts, missing information)
• {answer rules} — what the answer should look like (quotes, own words, format)
• {relevant snippets} — context from RAG or manual selection

🚀 Quick start: paste this into the chat:

Here's an episodic memory template for LLMs. Adapt it to my task: [describe your task, e.g.: "answer a client's questions about the project history" or "analyze six months of team meetings"]. 

Ask me questions to fill in the fields: what the context is, what the typical questions are, what specific requirements the answers have.

[paste the template above]

The LLM will ask about the context source (conversation/documents), the question types (facts/dates/opinions) and the answer format (quotes/brevity/detail), because EpMem works better when the reflections are tuned to the specifics of the task. It'll take the "question → answer → reflection" structure and adapt it to your case.

⚠️ Synthetic dataset: The study used synthetic conversations, not real ones. Real conversations with noise, typos and ambiguity may show different results.

⚠️ Multi-request workflow: You first have to build the memory (the learning loop), then apply it. That's several requests to the model, not a one-shot.

⚠️ Requires reference answers for learning: To create reflections you need correct answers. Without them, the method doesn't work. It fits tasks where you can check the first N answers by hand.

⚠️ Better for strong models: On small base models (Llama 3B base, Mistral 7B base), EpMem brings no improvement. It works on instruct models (GPT-4o mini, Llama/Qwen Instruct) that understand structured output and reflection.

⚠️ Not for simple tasks: If the questions are simple one-hop ("capital of France"), the overhead of reflections isn't worth it. The method is for complex multi-hop, temporal and adversarial questions.

⚠️ Can overfit on "no information": If the first batch has a lot of adversarial questions, the model may start saying "no information" more often even where the answer exists (example: Qwen with PromptOpt + EpMem).

The researchers used the LoCoMo benchmark: 10 synthetic conversations of 300+ turns each (~9,000 tokens, 35 sessions, spanning months). Questions are labeled into 5 categories: single-hop (simple facts), multi-hop (connecting information from different parts), temporal (about dates and time), open-domain (general knowledge) and adversarial (the answer isn't in the conversation).

They tested on small open-weight models (Llama 3.2-3B, Mistral 7B, Qwen 2.5-7B) + their instruct versions + GPT-4o mini. They compared 6 approaches:

1. Full Context — the entire conversation in the prompt (baseline, 23,000 tokens)
2. RAG — top 10 snippets via semantic search (650 tokens)
3. A-Mem — agentic memory with tags and links (3,300 tokens)
4. RAG + PromptOpt — RAG + prompt optimization via feedback
5. RAG + EpMem — RAG + episodic memory with reflections
6. RAG + PromptOpt + EpMem — everything combined

The first conversation (199 questions) was used to build the memory (generating reflections, optimizing prompts), and the other 9 conversations (1,787 questions) for evaluation. The metric was F1-score (answer accuracy), plus the average number of tokens per request.

What was surprising:

• Full Context isn't the best: you'd think the whole context = maximum information, but models get lost in 20k+ tokens, especially on multi-hop and temporal questions (the "lost in the middle" effect).
• RAG + EpMem beats Full Context on strong models while saving 90% of tokens, so it's not about the volume of context but its quality and structure.
• Adversarial questions (no answer in the conversation): Full Context hallucinates (26-52% F1), while RAG + EpMem says "no information" (68-95% F1). This shows that episodic memory teaches the model metacognition: knowing what it doesn't know.

Practical takeaway: If your task demands honesty ("I don't know" instead of guessing), adversarial examples in the first learning batch are critical. The model will pick up the pattern "looks like the topic, but there's no exact fact → say no information" and carry it over to new cases.

Why they reached these conclusions: Reflection gives the model meta-context: not just "what's right" but "why I got it wrong". That's more powerful than plain few-shot examples. The model generates reflections like: "I gave an approximate answer instead of an exact date; I need to use the specific date from the text" or "I made up information that wasn't there; if the topic never came up, I should honestly say 'no information'". When these reflections land in the context of a new question, the model adapts to the pattern.

Context: The prompt for creating a reflection (Reflection Step) from the paper.

Reflect on your performance in the following QA example. Focus specifically on:
- Whether your answer was correct or not, and why.
- If the question required temporal reasoning, how well you handled it.
- If the question had no answer in the context, whether you correctly identified that.
- What reasoning errors (if any) occurred, and how to avoid them in future similar cases.

Provide a short reflection in a few sentences.

Question: {question}
Context: {top-10 retrieved utterances}
Correct Answer: {label}
Predicted Answer: {prediction}

Reflection:

Context: The prompt for applying episodic memory (QA Step).

You are an AI assistant that answers questions based on a given conversation. Use the current context and optionally refer to past examples and reflections to help you reason, but focus on the current question.

Based on the given conversation snippets, write a short answer for the following question in a few words. Do not write complete and lengthy sentences. Answer with exact words from the conversations whenever possible. Use the following examples to answer correctly.

Examples: {top-3 most similar episodic memories with reflections}

Below are retrieved snippets from a conversation between two people.
{top-10 retrieved utterances}

Question: {query}

Now, answer the current question using the conversation context above. Refer to the past examples and reflections only if they help clarify your reasoning. If the answer to the question requires you to do temporal reasoning, use DATE of CONVERSATION to answer with an approximate date. If the question cannot be answered, write 'No information available'.

Answer:

💡 Adaptation for analyzing customer success correspondence:

Say you're a customer success manager working with a client for months: discussing the integration, bugs, features, upsells. You need to find fast: "When did the client complain about slow loading?" or "Did we discuss a data-export API?"

Build an episodic memory for the correspondence with client {company name}.

The first 10 questions with correct answers (verify them yourself):
1. When did the client first mention the Salesforce integration?
   Correct answer: {date, message}
2. Did we discuss a discount for an annual plan?
   Correct answer: {yes/no, details}
...

For each one:
- Provide context from the correspondence (top 10 relevant messages)
- Answer
- Compare with the correct answer
- Write a reflection: where the mistake is, why, how to avoid it

Save the reflections. For new client questions, use the 3 most similar examples as context.

Effect: honest "no information" answers instead of made-up ones, exact dates instead of "sometime in March", specifics instead of generic phrases.

🔧 Technique: Remove the reflection structure → see the model's free-form thinking

The original prompt imposes a clear reflection structure ("Focus specifically on: ..."). If you remove the structure:

Reflect on your performance in this QA example. Why did you answer this way? What went wrong or right?

Question: {question}
Context: {context}
Correct Answer: {label}
Predicted Answer: {prediction}

Reflection:

Effect: The model writes reflections in its own words and may spot patterns that don't fit the predefined structure. Useful for exploring error types at the start.

🔧 Technique: Different reflection types for different question categories

Instead of one reflection for everything, build specialized memory sets:

SET 1: Temporal Questions (questions about dates)
Reflections focus on: date accuracy, the difference between "roughly" and "exactly", using DATE of CONVERSATION.

SET 2: Adversarial Questions (questions with no answer)
Reflections focus on: when to say "no information", how to tell a "similar topic" from an "exact fact", not making things up.

SET 3: Multi-hop Questions (connecting facts)
Reflections focus on: how to link information from different parts, not losing intermediate steps.

For a new question, determine the category (you can ask the LLM) → pull examples from the matching set.

Effect: More precise reflections for each task type, fewer generic tips like "be more accurate".

💡 Extrapolation: EpMem + Self-Consistency to reduce hallucinations

Combine episodic memory with self-consistency (generating several answers and picking the consensus):

You answer the question using {context} and past reflections.

EXAMPLES: {top 3 reflections}
CONTEXT: {relevant snippets}
QUESTION: {question}

Generate 3 different answers:

Answer 1:
Answer 2:
Answer 3:

Now analyze:
- Which answers agree?
- Which contradict each other?
- If all 3 say "no information" → most likely there really is none
- If 2 of 3 are the same → probably correct
- If all 3 differ → not confident, say "no information"

Final answer:

Effect: Fewer hallucinations through internal cross-checking. If the model makes up something different in each of 3 variants, that's a sign of hallucination. If it says the same thing 3 times, there's more confidence.

Evaluating Long-Term Memory for Long-Context Question Answering (arXiv:2504.XXXXX) — Alessandra Terranova, Björn Ross, Alexandra Birch, School of Informatics, University of Edinburgh

LoCoMo benchmark — Maharana et al., 2024. A benchmark of long conversations for QA.

Reflexion (Shinn et al., 2023) — Language agents with verbal reinforcement learning. The foundational idea of reflections for LLMs.

ExpeL (Zhao et al., 2024) — Experiential Learning framework. LLM agents learn from experience without gradient updates.

A-Mem (Xu et al., 2025) — Agentic semantic memory system with contextual tags and dynamic links.