TL;DR
Spec-Driven Test Generation is a two-step technique that forces the model to first describe in words how the code is supposed to behave, and only then write tests. The model goes through each function and pins down: what's expected on input (preconditions), what's guaranteed on output (postconditions), which cases are already covered by tests, and which aren't. Only after that does it write tests to close the gaps.
When you ask an LLM to "write tests for this code" directly, it latches onto what it sees in the code rather than what the code is supposed to do. The result: tests that mirror the code's logic (bug and all), skip edge cases, and feel shallow — the model has nothing to anchor on besides the function's literal text.
The method fixes this by turning implicit expectations into an explicit document — essentially a behavior checklist. The model can't just write a test "because why not"; it has to first explain why the test exists and which condition it's checking.
Method Blueprint
PHASE 1 (extraction, one or more prompts):
STEP 1: The model reads the code + docstrings + comments + existing tests → understands the function's business logic
STEP 2: For each function, the model spells out:
- D (description): what the function does
- Pre (preconditions): what must hold true BEFORE the call
- Post (postconditions): what's guaranteed AFTER the call (success, errors, side effects)
STEP 3: The model checks each condition against existing tests → marks ✅ Tested or ❌ Untested
STEP 4: For every ❌ Untested — proposes a concrete test (Sug)
→ Output: a spec document (.spec.md)
[optional: a human edits the spec by hand before phase 2]
PHASE 2 (test generation, separate prompt):
STEP 5: The model writes tests covering all ❌ Untested conditions from the spec
→ Output: a ready-to-run test suite
Example
Task: A developer is building the backend for an e-commerce store and wrote a function called apply_promo_code that calculates a discount from a promo code. The function is already partially covered by tests, but the developer isn't sure all the edge cases are handled (expired promo code, negative discount, double application).
Prompt (step 1):
I want to test this function. Before writing any tests, extract the contract of its behavior.
Code:
{paste the full apply_promo_code function}
Existing tests (if any):
{paste the current test file}
For the function, produce:
- Description: what it does
- Preconditions: what must be true before the call
- Postconditions: what's guaranteed after the call — including error handling, edge cases, side effects (bonus deduction, logging, etc.)
- For each condition, mark ✅ Tested (already covered by an existing test) or ❌ Untested (not covered), and propose a concrete test for each untested condition.
Don't write tests yet. Contract only.
Result: The model produces a structured list of conditions like "promo code expired → returns 'expired' error" or "discount cannot exceed order total" — each flagged as covered or not. Some conditions will come back flagged ❌ — those are the blind spots a developer usually misses on a quick review.
Prompt (step 2):
Here's the contract from the previous step:
{paste the spec}
Write tests that cover every condition marked ❌ Untested. Use Arrange-Act-Assert format, with test names like "what_is_tested_under_which_condition".
Result: The model produces a ready-made set of tests, each tied to a specific missing condition — not generic "just in case" tests, but targeted checks for boundary scenarios.
Why It Works
Models are great at continuing a pattern that's been explicitly laid out, but poor at independently figuring out what's worth checking when simply asked to "write tests." Without an explicit anchor, the model copies the code's structure rather than its intent — hence missed edge cases and tests that pass even against broken code.
What LLMs are genuinely good at is organizing information that's already implicitly present in the text — docstrings, comments, calls to the function elsewhere in the codebase. By spelling out pre- and postconditions, the model surfaces this implicit information explicitly.
The method exploits exactly this: instead of asking the model to guess at tests right away, it first makes the model "say the contract out loud." It's like asking a lawyer to first list out every clause in a contract, and only then check whether the parties fulfilled them — instead of reading the contract and the compliance check at the same time.
Control knobs: - Drop the "mark Tested/Untested" step → the model just lists the contract without tying it to coverage — faster, but less targeted. - Add "include private/helper methods" → fuller coverage, but a longer spec and slower turnaround. - In step 1, add "pay special attention to error handling and boundary values" → sharpens the focus on edge cases, if that's your priority. - Add a human-in-the-loop step (you edit the spec yourself before phase 2) → more control, less model autonomy.
Prompt Template
I want to test this code. Before writing any tests, extract the behavior contract for each function.
Code:
{code}
Existing tests (if any):
{existing_tests}
For EVERY function, produce a block:
### Function: {name}
- Description: what the function does and what it's for
- Preconditions: what must be true BEFORE the call
- Postconditions: what's guaranteed AFTER the call — error handling, side effects, return values, edge cases
- For each pre- and postcondition, mark:
✅ Tested — if the condition is already checked by an existing test
❌ Untested — if the condition isn't covered, and propose a concrete test
Don't write tests yet. Contract only.
---
(After receiving the contract — second prompt)
Here's the contract above. Write tests that cover every ❌ Untested condition. Use Arrange-Act-Assert format, with clear test names.
Drop your code into {code}, and existing tests (if any) into {existing_tests}.
🚀 Quick Start — paste this into the chat:
Here's the Spec-Driven Test Generation template. Adapt it to my task: {your task — e.g., "test the discount calculation function in an e-commerce store"}.
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask about the code, programming language, and whether tests already exist — because without that it can't figure out what's covered and what isn't. It'll take the pattern from the template and adapt the contract structure to your specific code.
Limitations
⚠️ Scope of applicability: the method has been validated on code with clear inputs/outputs (functions, methods, APIs). For UI code, one-off scripts, or code without a clear behavioral contract, the payoff likely won't be as strong.
⚠️ Cost: two steps instead of one means more tokens and more generation time. For small, trivial functions, the overhead might not be worth it.
⚠️ Dependent on context quality: if the code has no docstrings, comments, or existing tests, the model has nothing to anchor on when deriving the contract — spec quality can suffer.
⚠️ Not a silver bullet against misses: even with the method, some contracts remain incomplete (per the study, the contract catches the relevant bug roughly 60-80% of the time across several attempts — not 100%).
How It Was Studied
The researchers took 90 real bugs from Google's internal issue tracker (C++, Java, Python, Go) — not synthetic ones, but actual production bugs with confirmed fixes. They compared two agents running on the same model (Gemini 3 Flash) with the same toolset: a plain agent that's simply asked to "write tests," and a spec-driven agent that's required to write a contract first.
Tests were run in a "clean slate" setting — with no pre-existing tests, so the agent couldn't cheat by copying an existing structure and had to rely purely on understanding the code. Each configuration was run 5 times, with statistics computed via bootstrap and McNemar's test.
Result: the spec-driven agent caught 9.8 percentage points more historical bugs and achieved 2.5 points higher branch coverage — both differences statistically significant. A separate judge model (stronger than the one that wrote the tests) rated the spec-driven tests as higher quality than the baseline in 77.8% of cases, and even higher quality than human-written tests in 56.7% of cases. An interesting detail: the more precisely the contract "caught" the actual root cause of the bug, the higher the chance the resulting test would find that bug (54.9% vs. 19.4% when the contract didn't match the bug) — meaning the quality of this intermediate step directly predicts the final outcome.
Adaptations and Extrapolations
🔧 Technique: carrying the principle over from tests to any code review
The same trick works beyond tests. Before asking a model to review or rewrite code, have it first write out the contract: what the function is supposed to guarantee. The model will start spotting mismatches between what the code promises and what it actually does — and that's exactly where bugs live.
Before reviewing this code, write out for each function: what it should guarantee on input and output (based on docstrings, naming, and how it's used elsewhere). Then check the code against this contract and point out where the implementation doesn't match the expected behavior.
🔧 Technique: contract as documentation
The spec itself (Phase 1) is a valuable byproduct. Even if you don't need tests, ask the model to generate this kind of contract for undocumented legacy code — you'll get a readable description of the system's behavior that you can hand to a coworker instead of spending an hour explaining "how this thing works."
Resources
Tufano M., McClure J., Cambronero J., Cheng R., Shi S.Y., Wei R., Chen D., Ivancic F., Dalloro L., Rondon P. Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation. Google, 2026.