Practical prompting, backed by research

The paradox: LLM agents are trained to be helpful — that's their strength as chatbots, but a fatal weakness in agentic tasks. Faced with an ambiguous instruction (the paper's illustration: 'grant delivery access to Alice Smith' when two contacts match), they may just pick one instead of asking. Quit Instructions let the agent stop and ask for clarification in situations with uncertainty or risk — instead of trying to complete the task at any cost. The trick: add 4 clear conditions to the prompt for WHEN the agent MUST quit (it can't rule out negative consequences, it needs more information about the task/situation, it lacks the knowledge to assess the risks). Safety +0.40 while helpfulness drops by just -0.03.

The problem: LLMs skip critical steps in procedures while trying to please the user. For example (our illustration), a client says "I'll take it!" after hearing the loan terms, and the model may submit the application right away, bypassing the mandatory understanding check. That happens because models are wired for reaching the goal, not for following the sequence of steps. ASTP (Autoregressive State-Tracking Prompting) targets skipped steps in multi-stage conversations: the paper tested in-game trading; loans and tech support with mandatory checkpoints are our extrapolation. The trick: the model first states where it was, then decides where to go. Instead of a vague "follow the procedure", you give a concrete task: identify the previous stage and always output it in JSON. Result in the paper's in-game trading test (gemini-2.5-flash): 99.6% procedure compliance versus 78.1% for the baseline prompt.

The RLHF paradox: the model gets more useful but more boring. After fine-tuning on human preferences, an LLM collapses into a narrow set of "safe" answers, because annotators systematically prefer the typical and familiar (a cognitive quirk of humans, not an algorithm bug). The Verbalized Sampling method lets you restore creativity and diversity: get 5-7 answer options instead of one stereotyped one, for copywriting, brainstorming, and any task with many right solutions. The trick: ask not for one answer but for a distribution with probabilities. This switches the model from "give the safest option" mode to "show the range of likely answers" mode. The mode changes: the prompt "one joke" collapses into "the most typical joke", while the prompt "5 jokes with probabilities" collapses into "the distribution of jokes from training", which restores the base model's diversity.

LLMs fail at complex tasks — accuracy drops from 46.6% to 8% (WebChoreArena). The reason: cognitive overload. The model tries to search for information, apply filters, count and plan the next step all at once — it loses focus and misses important things. WEBDART lets you solve multi-step tasks with many sources and filters by decomposing them into three sequential stages. The method's trick: a conservative strategy — first collect EVERYTHING (don't filter while collecting; that's where errors come from), then structure the data into a table, and only then apply the constraints. Each stage focuses on one skill, lightening the load on the model.

The paradox: Everyone optimizes prompts for brevity, yet LLMs work better with long, detailed contexts. The ACE method lets you accumulate a structured knowledge base (a playbook) that improves the model's performance on complex tasks. The trick: don't rewrite the whole context from scratch, add small deltas — new strategies, common mistakes, edge cases as separate bullet points with IDs. The context grows like a reference manual instead of collapsing into generalizations. Result: the model picks what's relevant out of a 10-20k-token detailed playbook on its own, instead of working from a vague "be more careful."

LLMs score work inconsistently: similar pitches get a 3 one time and a 5 the next. The reason: there's no explicit checklist, so the model relies on unstable intuition. The Reflect-and-Revise method turns a fuzzy rubric like "rate quality 1-5" into a structured system with concrete indicators and thresholds. The trick: the model scores examples → sees where it diverged from the reference → analyzes the patterns behind the gaps → rewrites the criteria. In 3-10 iterations the barest instruction becomes a detailed algorithm with mapping tables and explicit rules.

The paradox: The simpler each step of a decision, the more accurate the final choice. LLMs handle complex choices among many options poorly — too many factors at once. The model starts guessing from surface cues instead of analyzing systematically. Atomic Instructions lets you make an accurate choice among dozens of similar options (content format, type of logical fallacy, issue category). The trick: break a complex description into 3-5 binary questions with reference answers. The model answers simple yes/no questions and compares against the pattern — Claude improved accuracy from 42% to 63% on a 232-option classification.

The JSON format for tool calling drops accuracy by 27 percentage points. The reason is task interference: the model is simultaneously picking a tool, minding the syntax and generating an answer. Its probability mass gets smeared. NLT (Natural Language Tools) lets you call tools in plain natural language, with no JSON, via YES/NO. The method splits tool selection and answer generation into separate stages, each doing one job. The model writes "Search – YES, Calculator – NO, Database – YES" instead of `{"tool_calls": [{"name": "search"}]}`. Result: +18.4 percentage points of accuracy (69.1% → 87.5%) and a 70% reduction in the spread of results.

The paradox: Claude can hold 500k tokens in memory but loses the thread by the third reasoning step. The problem isn't context size — it's the lack of structure. Load 800 documents, ask it to connect the facts, and the model sees the data but doesn't know how to join it up. Thought Templates lets you structure multi-step reasoning through a library of reusable patterns. You separate "what to know" (documents) from "how to think" (templates). Instead of "sort it out yourself across 800 documents", you hand over ready patterns: "how to find the author", "how to link a person to a place", "how to match two events". The model picks the 2-4 templates it needs and composes the reasoning chain itself.

LLMs are bad at balancing precision and recall at the same time. Ask them to 'find everything relevant' and you get 50 passages, half of them junk. Ask for 'exact matches only' and they skip important connections. The PRISM method lets you retrieve information with a balance of precision and recall for questions that require combining facts from different sources. It splits conflicting goals across three agents: the Question Analyzer breaks the question into sub-questions, the Selector cuts hard (maximum precision), the Adder brings back what was missed (maximum recall). After 2-3 iterations of the Selector ⇔ Adder loop you get a compact evidence set — everything you need, nothing you don't.

The paradox: Budget $2,500, item $2,502.50 (0.1% over) — the model refuses to buy. The task says "5 bullet points," the substance fits in 4 — it adds filler to check the box. LLMs suffer from literalism: they follow instructions blindly even when it's absurd. The RID Framework lets you teach a model to distinguish critical constraints from flexible guidelines — making decisions based on common sense rather than box-ticking compliance. The method provides a structured 4-step schema: separate the goal from the rule, classify the rule (Hard Constraint or Soft Guideline), weigh the consequences of both scenarios, make a decision. Everything happens in one prompt via `` and `

` tags. A zero-shot technique — works in a regular chat without fine-tuning.

Try describing in plain text the logic "if it's a startup and the salary is +50% but the commute is >45 min unless it's a senior role" — the model will get lost. The ICA method lets you structure complex multi-factor decisions (career choices, business processes, personal dilemmas) as explicit if-else logic. Pseudocode instead of text descriptions — the model stops interpreting fuzzy conditions and starts executing a clear algorithm. Accuracy +67%: from 51% to 85% on Mistral-7B.

Add 5 code style requirements (line length, docstrings, pathlib instead of os.path) and functionality drops by 6%, even though the requirements have NOTHING to do with whether the code works. Even top models satisfy all 5 instructions at once only 47% of the time. VeriCode, a taxonomy of 30 automatically checkable instructions (style, logic, documentation, error handling), lets you measure not just whether code works but whether it follows the requirements. Each instruction is checked via linters and AST analysis and returns pass or fail. The researchers extended the classic BigCodeBench and LiveCodeBench benchmarks with these instructions, creating Vibe Checker — an environment that evaluates both dimensions.

The finding: An LLM sees the field "price": {"type": "string"} and picks a format at random — "18900", "$18,900" or "18,900 USD". Schemas written for programmers are ambiguous for models. PARSE automatically turns schemas into clear instructions for an LLM by adding detailed field descriptions and strict validation rules. ARCHITECT optimizes the schema once using synthetic examples and error analysis (5-6 iterations), SCOPE performs the extraction with a three-stage check — 92% of errors are caught right away — and RELAY returns the data in the original schema format.

The discovery: Even top LLMs are terrible at processing JSON directly – GPT-4o maxes out at 77% accuracy. Models get lost in the structure, confuse similar keys (name, room_name, full_name), and stumble over nesting. As the size grows to 80K tokens, Mistral-large drops by 91%. The method lets you extract data from complex API JSON responses with up to 95% accuracy – especially for filtering and aggregation. The trick: instead of "read the JSON and give me the answer" → "write a Python function that parses the JSON". The code runs – the result is exact. Add the JSON schema to the prompt – +12% accuracy. Even a trimmed structural sample works if the full response doesn't fit in the context.

A weak model + one round of self-critique outperforms a strong model without it. Nova Micro (Amazon's cheapest) jumps from 22% to 71% accuracy on math, a 220% gain. The method lets you save on expensive models: take a cheap one, have it review its own answer, and get top-tier quality for less. The trick: a model is bad at generating on the first try but great at critiquing someone else's text. Self-reflection turns that weakness into a strength: first the model generates an answer, then reads it as external text and switches into verification mode. The first correction round delivers 80-90% of the total benefit. But there's a big catch: in translation and SQL generation, most models get worse, swapping correct answers for wrong ones while trying to "improve" them.

The paradox: 61% of the time, experts preferred LLM codes over those from human analysts — but only when the methodology was built right into the prompt. The method lets you analyze interviews, reviews, and open-ended answers with professional rigor: find patterns, create codes, group themes — the way a qualitative analysis expert does. The trick: a self-check before output. The model gets an expert role, a research question (RQ), and quality criteria from thematic analysis. Before output it grades itself: "Is the code specific? Does it help answer the RQ?" — if not, it redoes it. Result: codes that are crisper and more specific than the human ones.

The paradox: a model retrieves information with 100% accuracy yet loses up to 85% of its reasoning accuracy. The problem isn't search quality — it's that the context is too long. The Retrieve-then-Reason method lets you solve tasks over long documents without losing accuracy — it turns a long-context task into a short-context one. Division of labor: first extract the evidence in a separate request, then reason over a short context. Llama-3.1-8B finds the conditions in 970 out of 1000 problems but drops 24.2% in solving accuracy — the method wins those points back.

Finding: LLMs recognize a table's structure differently depending on the upload format: HTML vs plain text makes up to an 8.8% difference in accuracy, and without visual separators between sections the model loses 16-24% accuracy. The QASU method lets you extract data precisely from survey CSVs and spreadsheets: find respondents by criteria, compute aggregates, pull out specific answers. Three independent principles work together: (1) pick the format for the task type (HTML for finding specific answers, plain text for logical filters), (2) a one-shot example before the question gives +10-25% accuracy, (3) self-augmented prompting (first ask the model to describe the table's structure) adds another +3-4%.

Finding: With an unclear prompt, an LLM doesn't pick the correct solution — it picks the simplest one. That's the "path of least resistance": the model fills the gaps with assumptions, choosing the easy option over the safe one. Tests on 10 models showed: vague prompt → errors grow 2.5x. For example, on error-handling tasks: 23% defects with a clear prompt versus 59% with a vague one. The study's framework lets you assess prompt quality along three dimensions (goal clarity, information completeness, logical consistency) and predict the risk of errors before you run it.

An LLM answers part of a complex question and forgets the rest. You ask for a comparison of two things — you get details on only one. The reason: the model doesn't track what's already known and what isn't yet. SEA lets you get complete answers to multi-step analytical tasks — competitor comparisons, market research, synthesizing facts from different fields. The method turns the completeness check into an analytical checklist. The model, as a Strategic Intelligence Analyst, breaks the question into a list of Required Findings (what needs to be known), checks them against Confirmed Findings (what's already there), and explicitly names the Remaining Gaps (what's missing). The gaps become the basis for new targeted questions — the loop repeats until the checklist is fully closed.

Models stumble not because they're dumb but because of the format. Feed them a data mush and you get wrong numbers, mixed-up names, made-up facts. A study on NBA games showed that structured input (JSON with atomic fields) cuts factual errors by 65-69%, which beats switching to a more powerful model. The method lets you get accurate summaries from complex data (hundreds of rows of transactions, events, records) without hallucinations or swapped facts. The trick: break the data into atomic fields — one column for "player," another for "action," a third for "score." The model stops guessing where a name ends and a number begins; it reads from a specific field. JSON cut errors from 13.2 to 4.1 per 100 events for Llama-3.1-70B.

An LLM gives different answers to the same question in Spanish and in Japanese. Ask about the capital of France in Spanish — one answer; in Japanese — another. The problem: the model doesn't realize that "Francia" and "フランス" are the same entity. It maps every concept into its internal English space, and that step is where it often slips. SUBSUB and SUBINJ let you get consistent facts in any language — you add the English translation of the key concept right into the query. The model skips the unreliable internal translation and works straight in its "native" English space — +5-44% cross-lingual consistency, +2-32% factual accuracy.

LLMs lose up to 39% of their performance in long conversations. Not because they forget (the context is in the window), but because ambiguity accumulates — early requirements conflict with later ones, intermediate edits create contradictions, and attention gets smeared across the whole history. The ERGO method lets you automatically detect the moment the model "got lost" and restart the conversation with a clean context. The system tracks entropy (a measure of chaos in token predictions) on every turn. A sharp entropy spike signals confusion. ERGO consolidates the whole history into one optimized prompt and starts a new chat — the model carries on with a clear head but full context.