TL;DR
When you're teaching ChatGPT a new task — a custom format, unusual logic, made-up rules — it works better if you explicitly spell out the rule rather than just showing examples and hoping the model figures out the pattern on its own. Researchers compared both approaches on five novel tasks and found: a clear rule almost always beats a set of unexplained examples in accuracy.
The "show examples and the model will get it" intuition doesn't always hold. Models are bad at inferring a rule from examples when the task requires combining conditions — say, "urgent if A OR B, AND also C." That kind of logic is hard to reconstruct from a handful of demonstrations: examples give you data points, not the formula itself. The model ends up guessing a rule that's either too narrow or too broad.
A rule solves this problem simply: it names the condition directly, instead of making the model reverse-engineer it from scattered data. Stacking examples on top of an already-clear rule barely improves results — if the rule is written fully and unambiguously, demonstrations become redundant.
Method Blueprint
APPROACH 1 (rules): Describe the logic and format in words → the model applies it to a new case
APPROACH 2 (examples): Show 5-10 input-output pairs with no explanation → the model guesses the rule itself
APPROACH 3 (combo): Rule + examples together → barely better than the rule alone
All of this happens in a single prompt — no iterations, no separate follow-up requests.
Example in Practice
Task: You want to teach ChatGPT to triage support tickets using a custom priority logic — not the standard "complaint = urgent" — but based on your own business criteria.
Prompt (the version that works — using a rule):
Classify the customer ticket as URGENT or NORMAL using this rule:
The ticket is URGENT if:
(the customer mentions "financial loss" OR "legal risk")
AND
the tone of the message is aggressive or distressed.
In all other cases — NORMAL.
Response format: only the word URGENT or NORMAL.
Customer ticket: {ticket text}
The less reliable version (using examples):
Here are 10 example tickets with urgency labels:
[10 "text — label" pairs with no explanation of the logic]
Classify this ticket: {text}
Result: The rule-based version consistently handles edge cases correctly — for example, an aggressive tone with no mention of money or legal risk. The example-based version stumbles more often on exactly these "compound" conditions (AND/OR), because the model isn't seeing the logic directly — it's reconstructing it from a limited set of data points.
Why This Works
When a model learns from examples, it has to guess the hidden rule — like a game of "spot the pattern." That works fine when the rule is simple (one variable, a familiar category). But when the task requires combining several conditions — like a multi-step arithmetic operator or AND/OR logic — guessing the formula from a few examples gets hard: the examples simply don't cover the whole space of possibilities.
One of the strengths of modern LLMs is that they're good at following explicitly stated instructions. This is specifically reinforced through instruction tuning: models are trained to obey verbal rules.
The method simply removes the need to guess: instead of making the model reverse-engineer the logic from data, you hand it the ready-made formula. That's where the effect comes from — a rule gets to the target more directly than a pile of examples.
Control levers: - Complexity of the condition's logic (how many AND/OR combinations) → the more complex the condition, the bigger the payoff from an explicit rule - Reliance on the model's "general knowledge" (jargon, cultural categories, language patterns) → here a rule barely helps; examples do just as well - Number of examples stacked on top of the rule → feel free to cut these if the rule is written clearly — it saves prompt space with no quality loss
Prompt Template
Perform the task according to the following rule. Don't rely on guessing — the rule describes the logic completely.
RULE: {describe the exact condition, including any AND/OR combinations, if applicable}
RESPONSE FORMAT: {describe exactly what the output should look like}
Input data: {your data}
🚀 Quick start — paste into the chat:
Here's a template for teaching the model via an explicit rule. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.
[insert the template above]
The LLM will ask what logic underlies the classification/task (conditions, formats, edge cases) — because the method only works if the rule is described fully and unambiguously.
Limitations
⚠️ "Gut feel" tasks don't benefit from rules: if the task relies on categories the model already knows (language patterns, established associations, "industry jargon") — a rule gives almost no advantage. Examples work just as well, because the model already "knows" these categories from training.
⚠️ A bad rule is worse than none: if the rule is written vaguely or incompletely, it won't save your results. The rule needs to be tested and unambiguous — otherwise there's no gain.
⚠️ Stacking examples on top barely helps: if the rule is already clear, extra demonstrations in the prompt just waste space without adding accuracy.
How It Was Studied
The team invented five novel tasks across different domains — from a card game about finding a "set," to a 3×3-grid board game, to a made-up mathematical operator and an artificial language with grammatical agreement. Each task was given to the model three ways: rule only, examples only, and rule plus examples together — on identical test cases.
They tested several model families (Gemma 3, Qwen 2.5/3, OLMo 3) at different sizes, using GPT-5.4 as a reference point for comparison. Result: rules beat examples on almost every task, especially where algebraic logic is needed — step-by-step calculations, combining conditions with AND/OR. The gap shrinks or disappears where the task relies on "gut feel" — grammar or familiar semantic categories.
Surprisingly, adding examples on top of an already-clear rule barely improved accuracy — and neither did simply piling on more examples. Instruction-tuned models amplified the advantage of rules, but didn't lose their ability to learn from examples.