Practical prompting, backed by research

Finding: an LLM doesn't just agree with a bad idea, it invents convincing reasons why it's good. This is called agreement bias: the model sees your text, takes it as a given and starts rationalizing. The SGV method lets you squeeze out honest, sharp criticism instead of the stock "not bad overall, but there's room for improvement." The trick: make the model describe what success looks like before it has seen your work. In the paper (AI judges checking agents' work on web, computer and robot tasks) this caught up to 25 percentage points more failures; using it on texts is our extrapolation. Then the "yardstick" is already set, and there's no way to bend it to fit your text.

The RLHF paradox: the model gets more useful but more boring. After fine-tuning on human preferences, an LLM collapses into a narrow set of "safe" answers, because annotators systematically prefer the typical and familiar (a cognitive quirk of humans, not an algorithm bug). The Verbalized Sampling method lets you restore creativity and diversity: get 5-7 answer options instead of one stereotyped one, for copywriting, brainstorming, and any task with many right solutions. The trick: ask not for one answer but for a distribution with probabilities. This switches the model from "give the safest option" mode to "show the range of likely answers" mode. The mode changes: the prompt "one joke" collapses into "the most typical joke", while the prompt "5 jokes with probabilities" collapses into "the distribution of jokes from training", which restores the base model's diversity.

Models handle prohibitions great ("don't use commas", "don't mention X"). But "do something at the end" requirements — finish with a JSON block, give a score, write a summary — collapse under load: up to 50% violations on hard tasks. It's not chance and not a bug in one particular model — it's a systemic pattern. The Salience Reminder method frames the requirement from both sides: an explicit tag at the start of the prompt plus a reminder at the end. Compliance goes back from 70–87% to 90–100% — with literally two sentences.

Is the `SalesAmount` column in your table revenue before returns or after? Is region the customer's or the warehouse's? The LLM sees a bare schema and guesses. It gets it right about half the time. The semantic layer method lets you analyze data so that the model applies your business logic instead of inventing its own. Add 1-2 pages of markdown with metric definitions before the data and the question, and accuracy jumps by 17-23 percentage points. The difference between Claude and GPT is pocket change. The difference between "with context" and "without" is huge.

The JSON format for tool calling drops accuracy by 27 percentage points. The reason is task interference: the model is simultaneously picking a tool, minding the syntax and generating an answer. Its probability mass gets smeared. NLT (Natural Language Tools) lets you call tools in plain natural language, with no JSON, via YES/NO. The method splits tool selection and answer generation into separate stages, each doing one job. The model writes "Search – YES, Calculator – NO, Database – YES" instead of `{"tool_calls": [{"name": "search"}]}`. Result: +18.4 percentage points of accuracy (69.1% → 87.5%) and a 70% reduction in the spread of results.

Try describing in plain text the logic "if it's a startup and the salary is +50% but the commute is >45 min unless it's a senior role" — the model will get lost. The ICA method lets you structure complex multi-factor decisions (career choices, business processes, personal dilemmas) as explicit if-else logic. Pseudocode instead of text descriptions — the model stops interpreting fuzzy conditions and starts executing a clear algorithm. Accuracy +67%: from 51% to 85% on Mistral-7B.

The finding: An LLM sees the field "price": {"type": "string"} and picks a format at random — "18900", "$18,900" or "18,900 USD". Schemas written for programmers are ambiguous for models. PARSE automatically turns schemas into clear instructions for an LLM by adding detailed field descriptions and strict validation rules. ARCHITECT optimizes the schema once using synthetic examples and error analysis (5-6 iterations), SCOPE performs the extraction with a three-stage check — 92% of errors are caught right away — and RELAY returns the data in the original schema format.

Finding: On procedural tasks, LLMs rely on linguistic patterns: “sounds right” instead of “formally correct.” On symbolic tasks without semantics (blocks with names like “1jpkithdyjmlikck”), the model gets lost: no familiar words, nothing to latch onto. o1 scored 31.5% accuracy. TMK (Task-Method-Knowledge) lets you switch the model from probability-based guessing to formal manipulation, the way it works with code. Instead of a prose description of the task, you give a three-layer JSON structure: Task (why, the goal), Method (how, steps with explicit dependencies), Knowledge (what we know about the domain). The model sees a structure with `require`, `provides`, `given`, `makes` fields. It resembles pseudocode with clear “if X then Y because Z” logic. With TMK, o1 hit 97.3%, a gain of 65.8 percentage points.

CoT generates hundreds of reasoning tokens, half of which are filler. The model narrates 'first we take this, then we divide...', which leads to reasoning drift and low logic density. BRAID replaces the text with a Mermaid diagram: instead of 500 tokens of description, a 10-node graph with clear steps. The model follows the structure deterministically, without wandering. The result: the small gpt-5-nano-minimal with a diagram beat a bigger model without one, 45.2% vs 40.4% accuracy (and 1.5x cheaper).

Accuracy swings from 65% to 96% — with one prompt that hands over all the classification rules at once. The Hybrid method fixes this roulette. It splits classification into two steps — and accuracy settles at a steady 95-99% across all tested models. First the model just pulls facts into JSON, without a single conclusion. Then, in a separate step, it applies the rules only to the clean data — with no source text at hand.

Recall jumped almost 1.5-fold simply because the LLM was no longer asked to understand the text and make the decision in a single prompt. The TRACE method lets you pinpoint the cause of any classification error: either the model didn't recognize a fact, or it misapplied a rule. The secret is a strict split between "translating text into facts" and "applying rules" as two separate steps. The LLM only extracts canonical terms from a vocabulary, and the final verdict comes from explicit if-then rules that you write yourself.

Discovered: the word "JSON" in a prompt cuts reasoning quality from the very first token — not because generating structure is hard for the model, but because the format instruction competes with the task for attention. The Format Tax method lets you get deep analysis and the right structure at the same time, without sacrificing one for the other. The trick: split one request into two — first the model thinks freely, then it repackages the finished answer. Structured answers are on average 162 tokens shorter than free-form ones — that's not style, that's discarded logic, and it's exactly what leads to errors.

Write "a profitable and reliable option" and the model doesn't weigh the criteria. It generates text that sounds balanced. No actual calculation happens. UtilityMax gets you an honest choice among several competing goals — not an imitation, but math. Each criterion becomes a variable X1, X2, X3 — the model scores them one by one, then multiplies. A key point: strict wording like "find ONLY a profitable AND reliable one" doesn't save you — experiments showed it sometimes does worse than the soft version. The ambiguity doesn't go away, it just gets locked in harder.

An LLM reads a contract or an article like a news ticker — left to right, no map. It doesn't hold the chain “A affects B, B affects C” in its head; it assembles it on the fly. So on hard questions it answers from general knowledge about “texts like this” — not from the text itself. Structure of Thought lets you get an answer that's actually tied to the specific content, not to what the model saw in training. The trick: one prompt, two steps: “name the nodes and links” → “answer from the map”. The map becomes an anchor — the model can't invent a link it didn't just extract itself.

Finding: LLMs recognize a table's structure differently depending on the upload format: HTML vs plain text makes up to an 8.8% difference in accuracy, and without visual separators between sections the model loses 16-24% accuracy. The QASU method lets you extract data precisely from survey CSVs and spreadsheets: find respondents by criteria, compute aggregates, pull out specific answers. Three independent principles work together: (1) pick the format for the task type (HTML for finding specific answers, plain text for logical filters), (2) a one-shot example before the question gives +10-25% accuracy, (3) self-augmented prompting (first ask the model to describe the table's structure) adds another +3-4%.

The paradox: the main problem with LLMs working on text isn't hallucination, it's silence. What the model found, it reports accurately. What it missed, it never admits. You'll never know what you lost. Customized prompting lets you pull a specific set of facts out of any text with minimal omissions instead of getting a creative retelling. The trick: role + focus list + output template turn the model from a reader into a checklist scanner. Instead of "extract what matters," three concrete layers of instruction, and extraction recall goes up by 15%.

Ask a model in one request to recall a complex fact and immediately output strict JSON, and it mixes up the two tasks and drops half the list. REAP lets you get complete lists of facts and accurate numbers from an LLM without searching the internet, just through the right request structure. The secret: recalling and formatting go into two separate steps, and before answering, the model first checks whether the question even applies. The result: a list of award winners over 100 years comes back almost complete, not half-finished.

Models stumble not because they're dumb but because of the format. Feed them a data mush and you get wrong numbers, mixed-up names, made-up facts. A study on NBA games showed that structured input (JSON with atomic fields) cuts factual errors by 65-69%, which beats switching to a more powerful model. The method lets you get accurate summaries from complex data (hundreds of rows of transactions, events, records) without hallucinations or swapped facts. The trick: break the data into atomic fields — one column for "player," another for "action," a third for "score." The model stops guessing where a name ends and a number begins; it reads from a specific field. JSON cut errors from 13.2 to 4.1 per 100 events for Llama-3.1-70B.

With a general 'extract all the requirements' request, the model catches about 60% of what's there — the rest sinks. It looks for things that 'resemble a requirement' and latches onto explicit technical specs. PEGS prompting lets you break down any long document without gaps: instead of one fuzzy request, four hard angles with clear boundaries. The aim narrows — the model goes through each dimension separately instead of guessing what to look for in the first place. Coverage from 61% to 92% — no fine-tuning, one prompt.

LLMs don't stumble on complex tasks. They stumble on tasks with extra conditions, the ones that have nothing to do with your specific case. The WARPP method lets you strip out irrelevant rule branches in advance, before the model starts working rather than after it gets confused. The trick: before the main task, a separate "personalizer agent" studies the context and throws everything extra out of the instruction, so the main agent gets a simplified task and handles it more accurately and reliably. The key point: you can play that agent yourself, no code required.

Complex XML prompts burn 6 times more tokens (348 vs. 54) yet deliver the same result. The paradox: the simpler the structure, the more horsepower the model puts into content instead of parsing instructions. 5C Prompt Contract strikes the balance: control like DSL languages, creativity like a free-form prompt, with several times fewer tokens. The trick: instead of XML tags and multi-level templates, 5 simple blocks in one prompt. Character (role), Cause (goal), Constraint (boundaries), Contingency (fallback plan if something is unclear), Calibration (output format). The model grasps the structure instantly, no extra words — token savings AND rich output (777 output tokens vs. 833 for a free-form prompt).

An LLM always fills the gaps in your prompt — it just does it on its own, without asking. Tone? Neutral. Length? Whatever comes out. Audience? General. 5W3H makes that filling-in explicit: one sentence becomes eight dimensions of the task before execution, instead of being guessed along the way. The study's main insight: AI expansion of a single sentence is statistically indistinguishable from a prompt an expert wrote by hand across all eight dimensions — so you don't need to spend time writing a detailed brief. Ask it to expand — and you get one for free.

Finding: LLMs are catastrophically bad at counting words on the fly: ask for 100 words and you get 80 one time, 150 the next. The model doesn't track how much it has written. Plan-and-Write lets you generate text with a precise word count (ask for 50 = get exactly 50, not 45 and not 55). The trick: the model counts not "in its head" but through explicit numbering: first a draft `1:word 2:word 3:word...`, then a clean copy between tags. Accuracy up to +37.6%, especially on texts under 500 words.

Ask an LLM to delete the second item from a list in a JSON and then add a new one in third place — the model gets lost in the indices and applies the second operation in the wrong spot. JSON Whisperer lets you edit large JSON documents with RFC 6902 patches (add/remove/replace operations) instead of regenerating the whole document. The trick for lists is EASE: arrays become dictionaries with keys like "ax", "cd", "np". The order lives in a display_order string: "ax,cd,np". Deleting an item = removing its key from the string. No index arithmetic.