TL;DR
Spec Learning is a technique that turns 20 "good answer / bad answer" pairs into a ready-to-use system prompt. Not by hand, but through a chain of LLM requests: first the model extracts principles from the examples, then assembles them into coherent text that works as an instruction right away.
LLM users are stuck in a vicious cycle: write a prompt → get a bad answer → tweak it → still bad. The problem isn't laziness but mechanics: people are bad at articulating rules in their heads. You can see that "something's off" but don't know how to explain it to the model. The result: a three-line prompt that half works.
Spec Learning breaks the cycle. You don't have to articulate the rules; showing examples is enough. The model extracts the principles itself from the difference between good and bad answers, then assembles them into a system prompt. The whole process is 2 chat requests.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
Input: ~20 pairs of [task + good answer + bad answer]
STEP 1: PROPOSE+COMPRESS — one request
The LLM reads all the pairs → extracts principles → clusters similar ones
STEP 2: SYNTHESIZE — second request
The LLM turns the principles into a coherent instruction paragraph
(NOT a numbered list: a paragraph written as a character description)
Result: a system prompt → paste it at the start of future conversations
Both steps happen in a regular chat. No code, no API.
Example in Action
Task: You run a Substack newsletter about investing in the style of Morgan Housel: no hype, plenty of numbers, a dash of irony. Claude writes either in slick corporate-speak or like a generic Wall Street hype blog. You want to teach it your voice once and for all.
Prompt (Step 1: extracting principles):
You are a style analyst. I'll give you 20 pairs: each pair is [task] + [good post]
+ [bad post]. Your job is to find the rules that explain why one post
is better than the other.
Pairs:
---
Pair 1:
Task: {write about the market drop}
Good: {text_1_good}
Bad: {text_1_bad}
---
Pair 2:
Task: {write about the Fed rate decision}
Good: {text_2_good}
Bad: {text_2_bad}
---
[...the other 18 pairs...]
---
Find 5–8 specific principles. For each one:
- State the rule in one sentence
- Give the number of the pair that illustrates it best
- Flag it if several pairs say the same thing: that's an important signal
Prompt (Step 2: assembling the system prompt):
Now take these principles and assemble them into a system prompt for running
an investing newsletter.
Format requirements:
- Write in the second person ("you are..."), as a description of a character with personality
- Do NOT make a numbered list: a coherent paragraph only
- Include: voice and style, typical mistakes to avoid,
priorities when choosing wording
- Length: 150–250 words
Principles:
{principles_from_step_1}
Result:
After step 1 the model will show 5–8 specific principles such as "use numbers instead of evaluative words," "irony works when the facts speak for themselves," "no 'undoubtedly.'" After step 2 you get a coherent, character-style system prompt you can paste into Claude's Custom Instructions or at the start of every conversation. The result is noticeably more on-target right away than if you'd explained it in your own words.
Why It Works
LLMs are bad at guessing preferences out of thin air. When you write "make it livelier," "no corporate-speak," "in my style," those are words without an anchor. The model interprets them from its own training, not from your head. It systematically misses because there's no signal.
But LLMs are great at spotting patterns through contrast. Two texts side by side, one good and one bad, are a ready-made classification task. The model sees the difference and puts it into words. Twenty such pairs create enough signal for the principles to be specific and stable.
A paragraph beats a list. This is the paper's counterintuitive finding: coherent character-style text (JANUS) consistently outperforms a numbered list of the same rules (BULLETS). Probably because a paragraph creates a behavioral context rather than a set of instructions. LLMs follow a role better than they follow rules.
Control knobs: - Number of pairs → 20 is the sweet spot. More doesn't help. Fewer than 10 works worse. - Example selection → pick at random. The study showed that trying to hand-pick the "most contrasting" or "best" pairs hurts the result. The logic: a random sample shows typical patterns, not edge cases. - Final prompt format → ask for a paragraph, not a list. Add the instruction "write it as a character description." - Number of principles → ask for 5–8. Too few and you lose nuance; too many and the prompt bloats and loses focus.
Prompt Template
Step 1: extracting principles
You are a preference analyst. I'll give you {count} pairs of examples for the task: {task_description}.
Each pair: [task prompt] + [good version] + [bad version].
{example_pairs}
Find {5-8} principles that explain the difference between the good and bad versions.
For each principle:
- One sentence: what to do (or not do)
- The number of the pair that illustrates it best
If several pairs point to the same rule, merge them: that's an important signal.
Step 2: assembling the system prompt
Take these principles and assemble them into a system prompt for {task_description}.
Format: a coherent second-person paragraph ("you are an expert in..."),
written as a description of a character with personality and priorities.
NOT a numbered list.
Include: voice and tone, typical mistakes to avoid, top priorities.
Length: 150–250 words.
Principles:
{principles_from_step_1}
Placeholders:
- {count}: how many pairs you provide (20 recommended)
- {task_description}: what the LLM writes/does (newsletter posts, customer emails, code reviews)
- {example_pairs}: your examples in the format: task / good answer / bad answer
- {5-8}: the number of principles you want
🚀 Quick start: paste into the chat:
Here is the Spec Learning template, a method that turns examples into a system prompt.
Adapt it to my task: {your task}.
Ask me questions to fill in all the fields.
[paste the template above]
The LLM will ask about your task and request good/bad examples, because without that contrast there are no principles to extract. It'll take the structure from the template and adapt it to your context.
Limitations
⚠️ Heterogeneous tasks: If you want a "general-purpose assistant for everything," the method gives very weak results. On the general-helpfulness dataset HH-Helpful it barely beat random guessing. Spec Learning only works when the task has concrete, articulable rules.
⚠️ Narrow sample = baked-in bias: A system prompt built from 20 examples literally encodes whatever was in those 20 pairs. If your examples are unintentionally biased, the prompt will amplify that bias.
⚠️ Weak at holding boundaries: A text-based system prompt is easier to "break" with a direct instruction in the conversation than behavior built into the model's weights. For high-stakes tasks (medicine, law), don't rely on this alone.
⚠️ You need examples: The method requires some upfront work: collecting 20 pairs. If you have no examples, only an idea of "how it should be," Spec Learning won't help.
How They Tested It
Researchers from Carnegie Mellon put the question bluntly: can a system prompt built from 20 examples compete with fine-tuning on 1,000 examples? They tested on seven datasets, from math problems and code to psychotherapy dialogues and Stack Overflow answers. The range was deliberate: from tasks with an objectively correct answer to tasks where preferences are subjective.
They compared three versions of the same model: the base version, a version with the compiled system prompt, and a DPO version (fine-tuned on 1,000 pairs with weight updates). Scoring was done by a third-party LLM judge, deliberately chosen from a different model family to avoid "home-team" bias.
The result was surprising: on six of the seven datasets the system prompt beat DPO, using 50x less data and without touching the weights. The most striking case was Stack Exchange (Q&A): a win in 83% of comparisons versus 50% for the base model. The only flop was HH-Helpful, a "general helpfulness" dataset with a grab bag of requests. There the method barely managed 58%. The logic is clear: you can't compress "help with anything" into 8 rules.
The counterintuitive finding about example selection, that random beats every clever strategy, has a simple explanation: you want the typical signal, not the edge cases. Twenty random pairs show what works on average, not what works in extreme situations.
Adaptations and Extrapolations
🔧 Iterative refinement: testing new content against the compiled principles
After step 1 (extracting principles), don't throw the list away. Keep it as a checklist for evaluating new texts. Write a post → ask the LLM: "Here are the principles. Score this text against each one. What's being violated?" You get concrete feedback instead of "add more life to it."
🔧 Adaptation: swap the numbered list → paragraph in any prompt
The study shows that when you write system instructions, a coherent paragraph works better than a numbered list. This applies well beyond Spec Learning. If you have Custom Instructions or a project system prompt, try rewriting your list of rules as a second-person paragraph with personality. Odds are the results will improve.
🔧 Extrapolation: a team standard built from shared examples
Several people on a team write in different ways. Ask each of them to pick 5 of their "favorite" texts from the last six months. Build 20 pairs from that pool. Run Spec Learning. You get a system prompt that reflects the team's collective taste: not averaged out, but with real preferences.
Resources
Paper: Towards Spec Learning: Inference-Time Alignment from Preference Pairs
Authors: Dhriti Krishnan, Tejas Goyal, Jaromir Savelka, Carnegie Mellon University
Venue: NeurIPS 2026, Position Paper Track
Related methods: Inverse Constitutional AI (ICAI, Anthropic), Direct Preference Optimization (DPO), JANUS synthesizer