TL;DR
CDSE is a way to automatically improve an agent's "skill" (a file of instructions and scripts, like the ones in Claude Code) using the logs of its past runs. The knowledge pulled from those logs doesn't get dumped into a single document. Instead it's sorted into four roles: Meta (how to plan, check your work, and know when to stop), Experience (tricks and workarounds for specific situations), Tool (how to use the tools), and Script (ready-made scripts with built-in checks). Each role has its own rules for updating and verification.
Skills usually grow as one document: "here's another lesson, tack it on at the end." But the lessons from a single run are all over the map. One says "look at the structure of the Excel workbook first," another says "use this particular library so you don't lose the formulas," a third says "the sorting step belongs in a script." Each needs to be stored, checked and loaded differently. The authors examined 500 popular skills: 96.8% mix knowledge from at least two roles. And if you merge the roles back into one document, quality takes the biggest hit of any change they tested.
The method runs in five stages: extract lessons from successful and failed runs (with evidence from the log) → filter out the vague and the one-off → assign each lesson to a role → update each role its own way and resolve conflicts → rewrite the "signposts" (descriptions of when to load which part of the skill). On average it beats an agent with no skill by +7.6 pp and the best competitor by +2.6 pp.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1. Analyze runs → lessons + log lines as evidence
(successful: what to keep, where the success is fragile;
failed: the decisive mistake and the minimal fix;
cause not proven → stays "unclear")
STEP 2. Filter → keep only what is proven, applicable, reusable
(drop generic advice, one-off cases, unproven claims, duplicates)
STEP 3. Assign → each lesson gets 1 primary role + at most 1 secondary
(Meta / Experience / Tool / Script)
STEP 4. Update per role → rules for Meta, Experience, Tool, Script
(Script accepted only after self-check and independent verification)
STEP 5. Reconcile → remove contradictions between roles with minimal edits
STEP 6. Descriptions → rewrite the "signposts": when to load which part
In the paper, stages 1–6 are an automated pipeline. You can reproduce them by hand with a few LLM prompts.
Example in Action
Task: An Amazon and Etsy seller set up a Claude Code skill for the monthly reconciliation of the marketplace sales report against actual payouts. In six months the skill has ballooned into an 80-line wall of mismatched notes. The agent sometimes skips the currency check, sometimes breaks the Excel formulas, and every single time rewrites the same sorting logic from scratch. There are logs from 12 sessions: 8 successful, 4 with errors.
Prompt:
<Role>
You are a skill editor for an agent. Your job: turn session logs into a
clean skill with four parts: Meta, Experience, Tool, Script.
</Role>
<Context>
Skill: "Reconcile marketplace report with payouts".
Agent's task: reconcile the sales report (xlsx) against the bank statement
and find discrepancies.
Current skill: [80 lines of text pasted here]
Successful sessions: [8 excerpts pasted here]
Failed sessions: [4 excerpts pasted here]
</Context>
<Pipeline>
Step 1. For each session, extract lessons.
- Successful: what is worth keeping; where the success is fragile.
- Failed: where the decisive mistake was and what minimal fix
the log supports.
- Attach a quote from the log to every lesson. If the cause can't be
proven from the log, mark it "cause not established" and don't turn it into a rule.
Step 2. Filter. Keep only lessons that are backed by the log,
specific, and useful in other sessions. Throw out generic advice,
cases specific to a single session, unproven conclusions, and duplicates.
Step 3. Assign to parts. Each lesson gets one primary part and
at most one secondary:
- Meta: planning, self-monitoring, final check, stopping conditions.
- Experience: "if condition X, do Y" tricks, ways to recover from failures.
- Tool: how to work with a specific tool.
- Script: a repeated operation worth packaging as a script
(input, output, how to verify).
Step 4. Update each part by its own rules:
- Meta and Experience: short atomic rules; Experience with an explicit trigger condition.
- Tool: grouped by tool.
- Script: propose code only if the operation repeats, is deterministic,
and there is a way to verify it; otherwise don't propose it.
Step 5. Reconcile the parts: find contradictions and duplicates between them
and fix them with minimal edits.
Step 6. For each part, write a short "signpost" description: which tasks
it applies to, when to load it, what to expect from it.
</Pipeline>
<Output>
Return four sections (Meta, Experience, Tool, Script), then their descriptions.
Separately: a list of discarded lessons with the reason.
</Output>
Result: Instead of a wall of text, the model returns a skill in four sections. Inside you'll find short rules, and the Experience part will have explicit "if…, then…" conditions. Script may include a proposed sorting script with its input, output and test described. Separately you'll get a list of discarded lessons with reasons, plus "signposts" for each part. Quality depends on your logs, so double-check the evidence quotes.
Why It Works
The weakness: when all the lessons live in one document, the agent doesn't know which part to read right now. A rule about stopping (should I re-check?) is tangled up with a rule about a specific library. Updating a document like that is hard: it's unclear how to verify each edit. Piling up notes doesn't make the agent better on its own, and sometimes it actively gets in the way.
The LLM's strength: it's good at parsing logs, finding the decisive step and sorting text into given categories. If you demand a quote as evidence, the model is less likely to make up "lessons." If you give it a role and a trigger condition, it phrases rules more precisely.
How the method exploits this: roles separate knowledge by purpose. Meta transfers to other tasks best. Experience and Tool are tied to the environment and transfer worse. The script is rarely called but can matter a lot: on spreadsheets, removing it caused the biggest drop (−8.0), even though it was invoked in only 6% of cases. The "signposts" work because the agent sees them before loading a part and decides what to open.
Levers: - Number of roles: if the task is simple, keep Meta and Experience and drop the rest. - The condition in Experience ("if…, then…") is the single most important phrasing. - The filter in step 2: tighten it if the skill keeps bloating; loosen it if you have few logs. - Step 6: descriptions matter even when the part isn't loaded. Make them specific: which tasks it applies to, when it's needed. - Rounds: the first one delivers most of the gain; after that the effect weakens and bounces around.
Prompt Template
<Role>
You are a skill editor for an agent. You turn session logs into a skill with four
parts: Meta, Experience, Tool, Script.
</Role>
<Context>
Skill name: {skill_name}
Agent's task: {agent_task}
Current skill: {current_skill}
Successful sessions: {successful_sessions}
Failed sessions: {failed_sessions}
</Context>
<Pipeline>
Step 1. Extract lessons.
- Successful sessions: what is worth keeping; where the success is fragile.
- Failed sessions: the decisive mistake or missed step and the minimal
fix confirmed by the log.
- Every lesson gets a quote from the log. No evidence for the cause:
mark it "cause not established" and don't make a rule out of it.
Step 2. Filter. Keep only what is proven, specific and reusable.
Throw out: generic advice, single-session cases, unproven claims, duplicates.
Step 3. Assign. One primary part and at most one secondary:
- Meta: planning, self-monitoring, final check, stopping.
- Experience: "if condition, then action", ways to recover from failures.
- Tool: how to work with a specific tool.
- Script: a repeated operation as a script (input, output, check).
Step 4. Update the parts:
- Meta, Experience: atomic rules; Experience gets an explicit condition.
- Tool: grouped by tool, covering every tool.
- Script: only what is repeatable, deterministic and verifiable.
Describe a test for the script; no test, no script.
Step 5. Reconcile. Remove contradictions and duplicates between parts with minimal
edits. Don't change what has been verified.
Step 6. Write a "signpost" for each part: scope, conditions,
expected benefit.
</Pipeline>
<Output>
Four sections + descriptions. Separately: discarded lessons with the reason.
</Output>
What to fill in: the skill's name and goal; the current skill text (can be empty); excerpts from successful and failed sessions (key steps, errors, outcome). Keep successful and failed sessions separate: the method relies on asking different questions of each.
🚀 Quick start — paste into the chat:
Here is the CDSE template. Adapt it to my task: [your task and skill].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask which skill you're improving, what the agent is supposed to do, whether there's existing skill text, and which sessions succeeded or failed. That matters because the method is built on comparing wins and losses: without the session outcome, the model can't tell "keep" from "fix." It'll take the rest from the template's pattern.
Limitations
⚠️ You need graded runs: the method relies on logs with a known outcome (success or failure). If you don't know which sessions went well, it won't work.
⚠️ The full method is a pipeline: in the paper it's an automated system with analyzers, filters, script self-checks, and publishing only after verification. The manual chat version is simpler and its verification is less reliable. This goes double for Script: never trust a script until its test has actually been run.
⚠️ The edge over competitors is modest: compared with an agent that has no skill, the gain is noticeable, but over the best existing methods it averages around two to three percentage points.
⚠️ The first round does nearly all the work: later rounds give less, the results bounce around and sometimes drop. Don't keep "training" a skill forever.
⚠️ Tool and Script transfer poorly: Meta almost always helps on a different task, Experience only selectively, and Tool and Script are hit-or-miss. Their rules are tied to the environment.
⚠️ Partial validation: they tested open models (GLM, DeepSeek, MiniMax) inside Claude Code and Codex on six benchmarks. On your tasks and models, results may differ. The method's actual prompts are in the appendices, not the main text.
How They Studied It
First, the authors dissected 500 popular skill packages and labeled what kind of knowledge each contained. It turned out that 96.8% mix at least two roles: meta-instructions, empirical tricks, tool tips, and scripts. That's where the idea of separating roles came from.
Then they tested the method on six benchmarks: office-document Q&A, spreadsheets, search, math, programming, and business automation. They used three models and two runtime environments (Claude Code and Codex). The data was split 2:7 into training and test sets, and the skill was "trained" over three rounds. There were three competitors: Trace2Skill, SkillOpt-lite and EvoSkill. CDSE won in all 36 combinations.
The most interesting part is the ablations (switching parts off). Merging the roles into one document causes the biggest drop: for example, on SWE-Pro from 68.4 to 62.6, and on business automation from 32.9 to 24.3. Removing the "signposts" hurts too (spreadsheets 30.7 → 26.4). The surprise: Script was invoked in just 6% of spreadsheet runs, yet removing it caused the largest drop (−8.0). So a part's importance doesn't equal how often it's called: the descriptions are visible to the agent up front and shape its behavior even when the part is never opened.
Transferability is telling as well. A skill trained on one model helps on another (11 of 12 cases, +4.2 pp). Meta from one task improves 9 of 10 other pairs (+3.6 pp on average). General working principles transfer better than specific tricks and tools.
Adaptations and Extrapolations
💡 Adaptation for CLAUDE.md / AGENTS.md: the same "split by role" principle can be applied directly to an instructions file, no logs needed. Instead of one flat list of rules:
# CLAUDE.md
## Meta (how to work in general)
- Before editing: draft a 3–5 step plan, then act.
- At the end: run the tests and reread the diff; if tests are red, you're not done.
## Experience (if condition, do this)
- If the project has migrations, look at the latest one first; don't invent the schema.
## Tool (how to use the tools)
- Run tests with {command}; the linter with {command}.
## Script (repeatable)
- To generate the report, use scripts/report.py; don't rewrite it.
This is my own adaptation, not an approach the paper tested for CLAUDE.md. The paper measured the effect on skills that evolved automatically.
🔧 Technique: add a "signpost" → the agent opens irrelevant stuff less often. For each skill or section, write a short description: "When to load: …; what for: …." In the paper, visible descriptions influenced behavior even when the parts weren't loaded.
Resources
- Paper title: From Trajectories to Differentiated Skills: Cognitive Differentiation for Self-Improving Agents (CDSE).
- Authors: Yewei Zhao, Derong Xu, Wei Chen, Yin Xu, Peilun Zhou, Xiaohan Wang, Guojun Yin, Wei Lin, Mingjun Xiao.
- Affiliations: University of Science and Technology of China, Meituan, City University of Hong Kong.
- Competitors and related work: Trace2Skill, SkillOpt-lite, EvoSkill, SkillsBench, the Agent Skills format.
- Appendices F (skill labeling) and G (update details) are missing from the provided text.