TL;DR
Atomic Instructions is a technique that breaks a complex classification or selection task into a chain of simple binary (yes/no) questions. Each question checks one specific feature. If all the answers match the reference pattern, the option fits. The researchers applied this to classifying logical fallacies: instead of “identify which of 232 fallacies this is,” they gave the LLM a series of questions like “Does the text emphasize a particular word? Does the meaning change if the emphasis shifts?”
Models struggle with complex choices among many options — too many factors to weigh at once. The LLM starts guessing from surface cues instead of analyzing systematically. It gets especially bad when the options are similar: the model confuses “manipulation through context” with “manipulation through emphasis,” even though they're different things. Meanwhile, LLMs are great at answering simple yes/no questions and good at comparing patterns.
The method uses a two-stage process: (1) break each option's complex description into 3-5 atomic questions with reference answers, (2) before the final choice, check the similar options that often get confused with the first one. That forces the model into a detailed comparison instead of a quick gut call. Claude Sonnet 4 improved accuracy from 42% to 63% (+20.7%), Gemini 2.5 Flash from 43% to 52% (+8.7%).
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
SETUP (done once):
1. For each option → write 3-5 binary questions
2. Record the reference answer pattern (e.g.: Yes-Yes-No-Yes)
3. Build a “often confused with” list for similar options
USE (every time):
STEP 1: Initial check → answer the questions for each option → find matches with the reference
STEP 2: Check look-alikes → for the matches, take “often confused with” → answer their questions too
STEP 3: Final comparison → pick the option with the best pattern match
All three steps in one prompt.Example
Task: Choose a content format for a blog post — pick between “case study,” “how-to guide,” “expert opinion,” “news,” and “market overview.”
Prompt:
Help me choose the best content format for this topic: “How Gopuff beats competitors on delivery speed.”
Use the atomic questions method. For each format, answer the questions:
CASE STUDY:
1. Is a specific company/project the main subject? (reference: Yes)
2. Does it describe the process of solving a problem? (reference: Yes)
3. Are there specific result metrics? (reference: Yes)
HOW-TO GUIDE:
1. Are step-by-step instructions given? (reference: Yes)
2. Can the described action be repeated? (reference: Yes)
3. Are there concrete examples of application? (reference: Yes)
EXPERT OPINION:
1. Is there a subjective assessment of the situation? (reference: Yes)
2. Does it rely on the author's personal experience? (reference: Yes)
3. Does it contain forecasts or recommendations? (reference: Yes)
Often confused: Case study ↔ Expert opinion
Step 1: Check all three formats
Step 2: For matching formats, check “often confused with”
Step 3: Pick the format with the best pattern match
Response format:
[Format]: Yes-Yes-Yes (100% match)
[Another format]: Yes-No-Yes (67% match)
Choice: [format] — because [brief explanation]Result: The model produces a step-by-step check of each format with a match count. It shows where the pattern matched 100% and where 67%. If “case study” matched perfectly but there's overlap with “expert opinion,” it shows a detailed comparison of those two. At the end — a clear choice explained by patterns, not gut feeling.
Why It Works
LLMs handle complex choices among many options poorly — the model starts leaning on surface cues and patterns from its training data instead of systematically analyzing every criterion. It's especially bad when the options are conceptually close.
LLMs are excellent with simple binary questions — that's a core capability of a language model. Plus, models are good at matching sequences (pattern matching) — “my answers Yes-Yes-No match the reference Yes-Yes-No.”
The method turns a complex choice into a series of simple checks. Instead of “pick 1 of 232,” it becomes “answer 4 questions, compare with the reference.” The two-stage check (all options first, then the look-alikes) pushes the model past its first impression and into a detailed comparison.
Control knobs:
- Number of questions (3-7 is optimal) → more questions = more accurate, but slower
- Question wording → avoid subjective ones (“high-quality”), use objective ones (“contains metrics”)
- “Often confused” list → add the 2-3 closest options for critical decisions
- Reference patterns → you can allow several valid patterns (Yes-Yes-No OR Yes-No-No)
Prompt Template
Task: {describe what needs to be chosen/classified}
Use the atomic questions method to choose.
OPTIONS AND QUESTIONS:
{Option 1}:
1. {Binary question 1}? (reference: {Yes/No})
2. {Binary question 2}? (reference: {Yes/No})
3. {Binary question 3}? (reference: {Yes/No})
{Option 2}:
1. {Binary question 1}? (reference: {Yes/No})
2. {Binary question 2}? (reference: {Yes/No})
3. {Binary question 3}? (reference: {Yes/No})
[repeat for all options]
SIMILAR OPTIONS:
{Option 1} ↔ {Option 3}
{Option 2} ↔ {Option 5}
PROCESS:
Step 1: Check all options, answer the questions
Step 2: For matching options, check the “similar options”
Step 3: Pick the option with the highest pattern match
RESPONSE FORMAT:
[Option]: Yes-No-Yes (67% match)
[Another option]: Yes-Yes-Yes (100% match)
Choice: [option] — [brief explanation]Filling in the placeholders:
{describe what needs to be chosen/classified}— the specific selection task (content format, type of customer issue, solution method){Option N}— the name of the option{Binary question N}— a simple yes/no question about one specific feature{Yes/No}— the reference answer when this option fits
How to write atomic questions:
- Take the option's description
- Pick out 3-5 key features
- Phrase each feature as a yes/no question
- Avoid subjectivity (“good,” “high-quality”) — only verifiable facts
- One question = one feature (not “does it have A and B?” but two separate questions)
🚀 Quick start — paste into the chat:
I need to choose between several options. Help me build a system of atomic questions for an accurate choice.
My task: [describe the task]
Options: [list the options]
For each option, write 3-5 binary questions with reference answers. Add an “often confused with” list for similar options.
[paste the template above]The LLM will ask about the details of your task — which criteria matter and how the options differ. That's needed to write precise atomic questions that separate similar options. It will take the structure from the template and fill it with concrete questions for your task.
Limitations
⚠️ Requires upfront prep: You have to write the atomic questions and reference patterns once. For a one-off task, that may be overkill.
⚠️ Not for subjective criteria: The method works when criteria are objectively verifiable. “Is the text creative?” has no clear yes/no. “Does the text contain metaphors?” or “Does it ask the reader questions?” does.
⚠️ Complexity grows with many options: With 50+ options the model may skip steps or get confused. 5-20 options is the sweet spot.
⚠️ Look-alikes still get confused: Even with detailed questions, models sometimes miss subtle nuances between very close options. The method reduces errors but doesn't eliminate them.
How They Tested It
The researchers took a dataset with 232 types of logical fallacies, each with a description and examples. Baseline LLM classification accuracy was 42-44% (worse than a coin flip for some categories). The team manually decomposed each fallacy into 3-5 binary questions with reference answers. The process ran in three rounds: generation by an LLM → manual review by the authors → cross-review between authors.
They tested on Claude Sonnet 4, ChatGPT-4o, and Gemini 2.5 Flash. Baseline approach: give the model all 232 descriptions and ask it to choose. Stepwise approach: give step-by-step questions and ask the model to answer them and compare with the reference. Enhanced approach: add a Prolog graph of “this fallacy is often confused with that one” links — the model must check similar options before the final choice.
The most interesting part: Claude Sonnet 4 improved by 20.7% (42.2% → 62.9%), but ChatGPT-4o barely moved (+3.4%). Why? The researchers found that ChatGPT-4o often ignored the instruction to “check ALL options step by step” and chose based on first impressions. Claude followed instructions more precisely. This shows that instruction-following capability is critical for the method — a good structure isn't enough; the model has to actually use it.
They also tested hierarchical classification (first pick the category, then the subcategory, then the specific type) — accuracy dropped to 25%. Models made a mistake at the second level and rarely corrected it at the third, even though they were allowed to revise.
Adaptations and Extrapolations
💡 Adaptation for Diagnosing Customer Issues
A support team receives a ticket and needs to quickly determine the issue type to route it to the right specialist.
Prompt:
Determine the customer's issue type using atomic questions.
TICKET: “Paid for a subscription 3 days ago, but the features still aren't available”
ISSUE TYPES:
TECHNICAL OUTAGE:
1. Did the problem appear suddenly? (reference: Yes)
2. Is the functionality not working at all? (reference: Yes)
3. Is the problem limited to one user? (reference: No - widespread outage)
ACCOUNT ISSUE:
1. Is it related to access/permissions? (reference: Yes)
2. Did it follow an action on the account? (reference: Yes)
3. Does it require checking settings? (reference: Yes)
BILLING:
1. Is it related to payment? (reference: Yes)
2. Was the customer charged but the service isn't active? (reference: Yes)
3. Has it been >24 hours? (reference: Yes)
Often confused: Technical outage ↔ Account issue
Step 1: Check all types
Step 2: For matches, check the look-alikes
Step 3: Determine the type with the best pattern
Format: [Type]: Yes-No-Yes (67%) → Route to: [department]🔧 Technique: Weighted Questions → Prioritizing Critical Questions
Not all questions matter equally. You can add weights.
Template modification:
{Option 1}:
1. {Question 1}? (reference: Yes, weight: 3) ← critical feature
2. {Question 2}? (reference: Yes, weight: 2)
3. {Question 3}? (reference: No, weight: 1) ← secondary feature
Scoring: a match on a critical question = 3 points
Pick the option with the highest point total, not the highest match percentageUse it when one feature is more critical than the others. For example, when choosing a content format: having metrics matters more than writing style.
🔧 Technique: Negative Patterns → “Definitely NOT This” References
Add patterns that rule out an option.
Template modification:
{Option 1}:
Positive pattern: Yes-Yes-No (must match)
Negative pattern: No-No-Yes (if it matches — definitely NOT this option)
Logic: First eliminate options by negative patterns, then choose by positive onesUse it when it's easier to tell that an option definitely doesn't fit than that it does. For example, when classifying review sentiment: if it contains “terrible” or “disgusting,” it's definitely not positive, even if the word “good” appears.
Resources
Follow My Lead: Logical Fallacy Classification with Knowledge-Augmented LLMs
Wang, O. P., Bansal, T., Bai, R., Chui, E. M., & Gilpin, L. H.
UC Santa Cruz, 2025
AID-LF dataset: https://github.com/olivianxai/AID-LF
Base FALLACIES dataset: Hong et al., 2024