STAGE I: Experience Exploration (building the library)
├─ The exploration agent explores the environment → generates trajectories
├─ The critic model scores quality (1-5) → filter ≥4
├─ High-quality trajectories → permanent store D_mem
└─ In-progress trajectories → temporary store D_proc (for real-time guidance)

STAGE II: Memory Injection (finding relevant experience)
├─ New task I → hybrid search (FAISS + BM25)
├─ Top-K trajectories from D_mem → relevant examples M_t
└─ Formatting into a structured guide

STAGE III: Memory-Augmented Inference (execution with memory)
├─ Prompt = {task I, examples M_t, history H_t, screenshot s_t}
├─ The agent generates an action: a_t ~ π_agent(P_t)
└─ Loop until done (the agent decides via a 'finish' action)

All three stages can be adapted for manual work in a chat (see Why It Works).

Task: You write sales-driven product descriptions for marketplaces (Amazon, Etsy). You've built up a folder of winning descriptions: products with high CTR and conversion. A new client asks you to describe wireless earbuds.

Prompt:

My task: write a sales-driven description for TWS wireless earbuds with active noise cancellation, mid-range price ($50-80), target audience: students and young professionals.

Here are three of my past successful descriptions of similar products:

[EXAMPLE 1: Bluetooth speaker description]
- Format: short paragraphs, emojis, focus on lifestyle benefits
- Structure: hook → 3 key benefits → social proof → call to action
- Result: CTR 8.2%, conversion 12%

[EXAMPLE 2: Sports earbuds description]
- Format: bullet points, tech specs + everyday scenarios
- Structure: problem → solution → specs → warranty
- Result: CTR 7.9%, conversion 15%

[EXAMPLE 3: Smartwatch for students description]
- Format: storytelling, second person ("imagine"), emphasis on price
- Structure: a day in the life → features in context → price as a benefit
- Result: CTR 9.1%, conversion 18%

Analyze these examples, identify the patterns that work for my audience, and write the earbuds description using the best elements.

Result:

The model will identify the patterns behind the successful descriptions (short form, lifestyle focus, concrete usage scenarios), explain why they worked for your audience, and write a new description combining the best elements. You'll get the text + a rationale for the chosen structure based on analysis of your past examples.

The LLM's weakness: Models don't remember past sessions. Every new chat is a tabula rasa. Even if you solved a task perfectly yesterday, today the model starts from scratch. This hurts most with recurring tasks with variations: writing copy, analyzing data, solving standard problems. Every time you re-explain the context and lose the approaches you'd worked out.

The LLM's strength: Powerful in-context learning: the model learns very well from examples within a single prompt. Give it 2-3 example solutions → it picks up the pattern and applies it to a new case. This works better than abstract instructions: not "write concisely and clearly" but "here are three texts where I wrote concisely and clearly; do it like that".

How the method uses this strength: EchoTrail-GUI formalizes what power users do intuitively: collecting successful prompts and reusing them. The study revealed three key mechanisms:

1. Critic filtering: not all examples are useful. Low-quality trajectories are more harmful than having none at all (the ablation showed that without the filter the result is worse than baseline). In chat: don't save everything, save only what actually worked.

2. Hybrid search (semantic + keyword) finds relevant examples more precisely. In chat: don't grab the first example you find; look for the one closest in task + context.

3. Structured formatting: examples aren't raw logs but abstracted patterns _{what_was_there → what_it_thought → what_it_did}_. In chat: don't copy-paste a whole conversation; extract the essence: problem + approach + result.

Control levers for adaptation:

• K (number of examples): The study found K=2 optimal. In chat: 1-3 examples; more dilutes the focus, fewer gives too little context.
• Filter quality threshold: A stricter filter → a smaller library but higher quality. In chat: save only examples with a proven result (happy client, metrics went up, task solved).
• Memory format: The study abstracts screenshots into text (device-agnostic). In chat: don't store raw conversations, store templates: strip the specifics, keep the structure.
• Real-time guidance: The temporary store _D_proc_ helps avoid repeating recent mistakes. In chat: keep a "what DOESN'T work" log for the current project, and the model will take it into account in the next iteration.

Basic template (adapting Stage III)

My task: {new_task_description}

Context: {key_details — audience, constraints, goal}

Here are my past successful solutions to similar tasks:

[EXAMPLE 1: {task_name}]
Situation: {what_it_was}
Approach: {how_I_solved_it, key_steps}
Result: {what_came_out, metrics_if_any}

[EXAMPLE 2: {task_name}]
Situation: {what_it_was}
Approach: {how_I_solved_it}
Result: {what_came_out}

[EXAMPLE 3 if needed]

Analyze these examples:
1. Which patterns work for this type of task?
2. What can be applied to my new task?
3. Create a solution by combining the best elements from the examples.

What to plug in: - {new_task_description} — the specific task at hand right now - {key_details} — context that affects the solution - Examples — 1-3 of your past successful cases, as close as possible in task type - Example structure: minimum words, maximum substance: situation/approach/result

Template with critic evaluation (adapting Stage I)

I solved a task: {what_I_did}

Here is my solution:
{the_prompt_or_approach_I_used}

Result:
{what_came_out}

Rate the quality of this solution on these criteria:
1. **Effectiveness**: was the goal achieved? (1-5)
2. **Reusability**: can it be applied to similar tasks? (1-5)
3. **Efficiency**: are there obviously redundant steps? (1-5)
4. **Clarity**: will the logic make sense a month from now? (1-5)

Overall score (1-5) and rationale: is this example worth saving to my knowledge base?
Threshold for saving: ≥4.

Use it as a filter: save only examples scoring ≥4. This keeps noise from piling up.

🚀 Quick start for complex workflows:

If you want to automate building the library:

Help me build a system for collecting and reusing successful experience. Here's what I do regularly: {your_recurring_work}.

Tasks:
1. Create a template for saving successful examples (which fields are needed?)
2. Suggest filtering criteria (when is an example worth saving?)
3. Give me a prompt for finding a relevant example before a new task

[paste the basic template above as a structural reference]

The model will ask about the specifics of your work, task types and success criteria, and build a personalized experience-collection system.

⚠️ Cold start: The method needs an accumulated library of successful examples. You solve the first 10-20 tasks "from scratch" while saving the best ones. The full benefit shows up once you reach critical mass (the study used 4000+ trajectories, but for manual work 20-50 quality examples in your field are enough).

⚠️ Low quality hurts: The study found a paradoxical effect: with the critic filter switched off, the result became worse than baseline (an agent with no memory at all). Bad examples mislead more than no examples. In chat: be strict about filtering, don't keep "seems okay" stuff, only what's proven.

⚠️ Manual search is labor-intensive: The study uses automatic retrieval (FAISS + BM25). In chat you look for examples by hand, which takes discipline and an organized library. Topic folders or Projects in ChatGPT / Projects in Claude partly make up for it.

⚠️ Context limits: If your examples are bulky (long conversations, long texts), you'll hit the context limit fast. Solution: abstract the examples: not the whole conversation but a digest {problem → approach → result}.

The team built an autonomous system for an Android environment. An exploration agent based on Gemini 2.5 Flash explored apps (up to 30 steps per trajectory), generating diverse sequences of actions. Each trajectory was scored by a critic (Gemini Flash Lite) on a 1-5 scale, and only trajectories scoring ≥4 went into the permanent memory store. That's how they assembled EchoTrail-4K, a dataset of 4000+ high-quality trajectories.

An interesting design choice: Progressive Intent Focus: the agent starts in a curiosity-driven mode (explore the interface) and after a few steps switches to target-focused mode (formulate a goal and achieve it). This gives a balance of breadth + depth: the trajectories are diverse but also meaningful and complete.

They tested on two benchmarks: AndroidWorld (a variety of real apps) and AndroidLab (a controlled environment with detailed metrics). Baselines: GPT-4o, Gemini-1.5-Pro, open models like Qwen2.5-VL-72B, and specialized agents UI-TARS and UGround. They compared not just Success Rate but also Sub-Goal Success Rate (completion of subtasks), RRR (reversed redundancy ratio: the higher, the fewer wasted actions) and ROR (the share of reasonable operations).

The results are convincing: On AndroidWorld with GPT-4o, EchoTrail-GUI reached 51.7% SR versus 34.5% for baseline GPT-4o, and even beat specialized fine-tuned systems (URST 46.6%, UI-TARS 46.6%). With Qwen2.5-VL, the open model caught up with fine-tuned agents (46.6% SR) without any retraining. On AndroidLab the effect is even stronger: Qwen2.5-VL's Success Rate rose from 23.9% to 37.5% (+57% relative), and GPT-4o's from 31.2% to 48.1% (+54%).

Why such a big effect? Ablation studies showed the most critical component is critic-based filtering. When it was removed, the result dropped below baseline (31.0% versus 34.1% for the model with no memory at all). This confirmed the hypothesis that bad examples actively hurt. Hybrid retrieval also matters (without it, -6% SR), but less dramatically. Real-time guidance during exploration adds another +4% SR.

A curious visualization: a UMAP projection of task embeddings showed that the generated trajectories (1) semantically overlap with the real test tasks (the agent isn't generating nonsense), but (2) extend coverage, filling gaps that the test set doesn't have. In other words, autonomous exploration really does find diverse experience rather than just rediscovering what's already known.

🔧 Technique: A critic for self-checking → on-the-fly reflection

Instead of scoring after execution (as in the study), build the critic into the process:

My task: {task}

Here's how I plan to approach it:
{your_draft_plan}

Before executing, evaluate this plan as a strict critic:
1. Where might there be errors?
2. Which steps are redundant?
3. What have I missed from the context?
4. Score 1-5: execute as is, or refine first?

If the score is ≥4, execute the plan.
If <4, propose an improved version and re-score it.

This adapts the _R_critic_ mechanism but applies it proactively rather than retrospectively.

🔧 Technique: Dual memory for different task types

The study uses a single store _D_mem_. You can keep several libraries by domain:

I have three types of tasks:
1. E-commerce copywriting (library of ~30 examples)
2. Data analytics and insights (library of ~20 examples)
3. Project planning and breakdown (library of ~15 examples)

My new task belongs to type: {type}

[paste examples from the matching library]

Analyze the patterns specifically for this type of task and apply them to the new one.

A clear split by domain improves retrieval precision: you don't mix contexts.

🔧 Technique: Temperature control for exploration vs exploitation

In the study, the exploration agent uses Progressive Intent Focus. You can steer the balance via temperature:

# Exploration mode (looking for new approaches)
Task: {task}

Here's the standard way to solve it: {your_usual_approach}

Ignore it. Come up with 3 radically different approaches I've never tried. Be creative.

[temperature: 1.2-1.5 for diversity]

---

# Exploitation mode (using what's proven)
Task: {task}

Here are the proven approaches:
[examples from the library]

Pick the most suitable one and adapt it. Minimal deviation from what's proven.

[temperature: 0.3-0.5 for stability]

Use exploration when the library is small or the task is new. Use exploitation when you need reliability and have relevant examples.

🔧 Technique: A meta-prompt for maintaining the library

Automate the routine of collecting experience:

You are my knowledge management assistant. After every completed task, ask:

1. "Was the task solved successfully?" (Yes/No)
2. If Yes: "Briefly describe the solution: situation → approach → result"
3. Rate it on these criteria:
   - Effectiveness (1-5)
   - Reusability (1-5)  
   - Efficiency (1-5)
   - Clarity (1-5)
4. If the average score is ≥4: "Save to the library with tags: {category}, {difficulty}, {context}"
5. If <4: "What could be improved so it's worth saving next time?"

Let's start. I just solved a task: {description}

The model takes on the role of critic + archivist. You just answer the questions, and the curation process becomes automatic.

EchoTrail-GUI: Building Actionable Memory for GUI Agents via Critic-Guided Self-Exploration
The EchoTrail-4K dataset (4000+ trajectories for an Android environment)
Benchmarks: AndroidWorld, AndroidLab
Related concepts: Retrieval-Augmented Generation (RAG), Reward Model, In-Context Learning

Runze Li, Yuwen Zhai, Bo Xu, Liwu Xu, Nian Shi, Wei Zhang, Ran Lin, Liang Wang
East China Normal University, Alibaba Group, Shanghai Innovation Institute