TL;DR
UTT is a technique where a weak model solves tasks, a strong model shows the correct solution wherever the weak one fails, and a third role distills the lessons into a single text — a “textbook” (the Primer). Every new addition gets tested: if the weak model does worse with it on the same tasks, the addition is thrown out. The finished Primer simply goes into the weak model’s context. No weights are touched, no fine-tuning required.
The usual move is to hand a weak model a generic prompt: “be careful, think step by step.” That barely helps, because the prompt has no idea where this particular model goes wrong. A strong model asked to “write instructions for the weak one” is guessing too. In UTT the textbook is built only from real failures: you take tasks the strong model solved and the weak one didn’t. So the text ends up full of concrete techniques and pitfalls the weak model was actually missing. The surprising part: a text written for one weak model also worked on a different model that played no part in creating it.
The loop has four roles: the Student (weak model) solves a task. If it gets it wrong, the Prompter uses the task and the verifier’s feedback to write a teaching instruction. The Teacher (strong model) follows it to produce a target solution, which must pass verification. The Synthesizer merges the verified records into the shared Primer. A candidate is accepted only if it does no worse than the previous version on the same batch of tasks.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
SETUP: run Teacher and Student on a set of tasks with verifiable answers
→ keep the tasks where Teacher beats Student (training split)
→ set aside separate tasks for the final test
LOOP over task batches (batch = 16 in the paper):
STEP 1 Student: solves the task with the current Primer → answer
correct → next task
STEP 2 Prompter: task + verifier feedback → teaching instruction (where the gap is)
STEP 3 Teacher: task + instruction → demonstration solution
solution fails verification → record is discarded
STEP 4 Synthesizer: current Primer + verified records → candidate Primer
STEP 5 GATE: does Student with the candidate do no worse on the same batch than with the old Primer?
yes → candidate becomes the Primer; no → keep the old one
RESULT: the Primer goes into the context of the Student (or another model) as is
In the paper, code drives the loop. You can also play the roles by hand in separate chats — more on that below.
Example
Task: You run a store on Amazon and Etsy. Your bookkeeper sends scanned supplier invoices, and a cheap model pulls out the supplier’s VAT number, the VAT rate, the net amount and the date. It keeps confusing “VAT 20/120” with “VAT 20%”, grabs the intermediary’s VAT number instead of the supplier’s, and gets lost on invoices with several rates. The expensive model handles these invoices correctly. You have 40 invoices with hand-checked correct answers. Comparing an answer against that reference is the verification step in the method.
Prompt (step 2 — Prompter, in a chat with the expensive model):
You are the Prompter. Here is a document, the cheap model’s answer, and the result of checking it against the reference.
Document: Invoice #412 dated 03/14, supplier Daisy Supply Ltd (VAT no. GB123456789),
intermediary J. Miller Trading (VAT no. GB987654321), two line items: VAT 20% and VAT 10%.
Model’s answer: VAT no. GB987654321, VAT 20%, net amount $1,184.00.
Check result: VAT number wrong (intermediary taken), VAT rate incomplete
(second rate missing), net amount wrong.
Write a teaching instruction for the cheap model: what exactly it did
wrong and which rule it should follow. Do not process the document yourself.
Result: The model returns a short teaching instruction with a diagnosis: which field got mixed up and which feature of the document gives it away. Next, you hand this instruction and the document to the expensive model (the Teacher role), check its answer against the reference, and stack up the verified records for the Synthesizer. After 10–15 such records you have a Primer: a list of invoice-parsing rules with examples. You paste it into the cheap model’s system prompt and test it on invoices that weren’t used in training.
Why it works
The weakness. Generic hints (“check the VAT number, be careful”) don’t know where this particular model slips. If you simply ask a strong model to write instructions, it will cook up advice “in general.” A model that has only been shown the correct answers doesn’t understand how its own reasoning went off course.
The strength. LLMs follow textual rules and techniques well when they’re concrete and backed by a worked breakdown. A strong model can explain how a wrong path differs from the right one. Checking against the reference filters out explanations that sound great but are wrong.
How the method exploits this. The textbook grows only from real failures: tasks where the strong model is right and the weak one isn’t. A lesson makes it into the text only if the demonstration passed verification. A new version is accepted only if it didn’t hurt results on the same tasks. That way the Primer keeps only what actually closes the gap.
Knobs: - Batch size (16 in the paper) → smaller batch: more frequent updates and cheaper checks, but a noisier accept/reject decision. - Gate criterion (“no worse”) → tighten to “strictly better” if the Primer starts to bloat. - Roles → the Prompter and Synthesizer can be the same strong model. That’s exactly what the paper does: Teacher, Prompter and Synthesizer are one model. - Task set → the more varied the error types, the broader the Primer. But it gets longer, which means more expensive in tokens. - Test tasks → keep them separate, or you’ll be measuring rote memorization instead of learning.
Prompt Template
In the paper, each role has its own generic system prompt, not tied to any task. The actual texts are in Appendix A.3, which isn’t in the excerpt provided. Below is a reconstruction based on the role descriptions, preserving their logic and structure.
<Role_Student>
You solve tasks of this type: {task_type}.
Before solving, carefully read the textbook below and follow its rules.
<Primer>
{current_Primer}
</Primer>
Task: {task}
Give your answer in this format: {answer_format}
</Role_Student>
<Role_Prompter>
You are the Prompter. You are given a task, the student’s answer, and the verification result.
1. Identify exactly where the student went wrong.
2. Write a teaching instruction: which rule or technique the student was missing.
3. Do not solve the task yourself. Diagnosis and instruction only.
Task: {task}
Student’s answer: {student_answer}
Verification result: {verifier_feedback}
</Role_Prompter>
<Role_Teacher>
You are the Teacher. Solve the task following the teaching instruction.
Show your reasoning and the final answer in the format {answer_format}.
Task: {task}
Instruction: {teaching_instruction}
</Role_Teacher>
(Check the Teacher’s answer against the reference. If it fails, don’t use the record.)
<Role_Synthesizer>
You are the Synthesizer. You have the current textbook and new verified records.
Each record: task, student’s error, instruction, correct solution.
1. Extract general rules and techniques from the records, not task-specific answers.
2. Merge them into the textbook: combine duplicates, remove contradictions, keep what already worked.
3. Return the full updated textbook.
<CurrentPrimer>{current_Primer}</CurrentPrimer>
<Records>{verified_records}</Records>
</Role_Synthesizer>
<ValidationGate>
Run the Student on the same batch of tasks twice: with the current textbook and with the candidate.
Count the correct answers.
If N_correct(candidate) >= N_correct(current) → accept the candidate.
Otherwise → keep the current one.
</ValidationGate>
What to fill in: {task_type} — your real-world task type with a verifiable answer. {task} — a single example. {verifier_feedback} — what exactly didn’t match the reference. {current_Primer} — empty at the start, or a short intro.
🚀 Quick start — paste into a chat:
Here is the UTT (Universal Textual Teaching) template. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what type of tasks you have, what a correct answer looks like and how you verify it. It will also ask which model is the weak one, which is the strong one, and how many labeled examples you have. That matters because without a verifiable answer, the gate and record filtering don’t work. From your answers it will assemble four ready-to-use prompts and a run order tailored to your task.
Limitations
⚠️ You need verification with an unambiguous answer. The method rests on comparison with a reference: it decides whether the Teacher’s demonstration is correct and whether to accept a candidate. In the paper that means math with known answers and GPU code that gets run and measured. For texts like “a good post” or “a persuasive email,” no such check exists.
⚠️ You need a task set. The paper’s budget is 100 and 200 examples per task, plus separate test tasks. Without labeled cases you can’t run the loop, and a manual pass over 10–15 examples yields a weak, unstable Primer.
⚠️ The Teacher has to outperform the Student. Training tasks are only those where the strong model is right and the weak one isn’t. If the Teacher can’t handle the task, there’s nowhere to get the lesson from.
⚠️ Side effects are possible. On some tasks the weak model already solved, accuracy dropped after adding the Primer. In math, two of three configurations showed a noticeable drop — the paper cites 9.2 and 14 points; the rest of the text is cut off. Code showed no such effect. Always test the Primer on tasks the model solved without it.
⚠️ Transfer is uneven. A Primer built for one model usually worked on another, but not equally well. In math, the Primer written for Qwen gave Flash a noticeably smaller boost than Flash’s own Primer. On code speed, transfer from Opus to Qwen was weaker. For anything important, test the Primer on your target model.
⚠️ Narrow coverage, low baseline. The paper tested only math and GPU code. Starting accuracy there was very low (about 9% on code), and gains always look bigger from such a starting point. Don’t count on +40 points if your model already solves 80% of tasks.
⚠️ Context and cost. The Primer is a long text attached to every request. It eats tokens and context space.
How the research was done
The idea was simple: instead of fine-tuning the weak model, write it a textbook. The authors took two hard tasks: olympiad math (Omni-MATH-2) and GPU kernel generation (KernelBench), where the code is executed and measured for correctness and speed. They assembled three “teacher → student” pairs: DeepSeek Pro → Flash, Pro → Qwen3.6-27B and Claude Opus → Flash. All tasks were run on both models and split into subsets: where the teacher is stronger (source of training and test tasks), where the weak model already succeeded, and where both failed. The textbook was built only on the training split; the rest was kept for clean evaluation.
Results from the table: on KernelBench, Flash’s accuracy with the Primer rose from 9.4% to 48.6%, fast solutions (Fast1) from 9% to 35%, and on math from 27.6% to 51.7%. The comparison included one-shot, few-shot, a teacher-written summary, automatic prompt optimizers (APE, MIPROv2, GEPA) and fine-tuning methods (SeqKD, Fine-tune-CoT, RSR, LUFFY). According to the authors, UTT beats them, but the numbers are cut off in the available excerpt.
The most surprising part: the Primer created for Flash was dropped into Qwen, which had no role in creating it, and accuracy there rose by 32 points. The Primer created by Opus worked about as well on Qwen (51.0% on code and 65.8% on math). The key practical takeaway: textual lessons mined from one model’s mistakes often generalize to another. And on tasks the weak model already solved, accuracy on code didn’t dip — it went up.
Adaptations and extrapolations
These are my own extensions of the idea. The authors did not test them.
💡 Adaptation for an agent’s instruction file: instead of a Primer for a chat model, we build rules for CLAUDE.md or AGENTS.md. The agent runs 10–20 typical tasks; on failures, a strong model (or you) works out which rule it was missing, and the Synthesizer compiles them into the file. Gate: before accepting the rules, rerun the agent on the same tasks.
You are the Synthesizer. Here is the current CLAUDE.md and a breakdown of {N} agent failures
(task, what the agent did wrong, what it should have done).
Derive general rules, merge duplicates, remove contradictions.
Return the updated CLAUDE.md, no longer than {limit} lines.
🔧 Technique: the gate as a standalone trick → protection against instruction bloat. Accept any edit to a system prompt only if results on a fixed set of 10–20 cases are no worse than before. This works even without the other roles.
Here are two versions of a system prompt: A (current) and B (with new rules). Run both on these 15 cases and count how many answers match the reference. If B is no worse than A, keep B; otherwise keep A.
Resources
- Universal Textual Teaching for LLMs — Zhanyi Lu, Huan Wang, Westlake University
- Project page: https://alexlu99.github.io/UTT/
- Benchmarks: KernelBench (Ouyang et al., 2025), Omni-MATH-2 (Ballon et al., 2026)
- Related methods: APE, ProTeGi, OPRO, MIPROv2, TextGrad, GEPA, Dynamic Cheatsheet, ExpeL, AutoManual, SkillX