AssumptionMiner: Surfacing the Hidden Decisions an LLM Makes Behind Your Back

When an LLM writes code from a short request, it quietly makes dozens of small decisions: how to store a password, what to return on an error, which data format to use. You never asked for any of that — the model just picked an option and didn't tell you. AssumptionMiner is a method that makes the model list every one of those hidden decisions before you look at the code.

The core pain: code can pass every test and look like it works, while the decisions inside are wrong. Say you asked for a "login function," and the model silently hashed passwords with MD5 (outdated and vulnerable) instead of bcrypt. Tests are green, security is a disaster. On average every task hides 3-4 of these silent decisions, and without a dedicated check you simply can't see them.

The method fixes this with two steps in a single request: first the model reasons out loud — what it decided on its own, what wasn't in the request, and what the alternative was. Then it turns that reasoning into a structured list with categories (input validation, response format, error handling, data storage, performance, security). You read the list, change what you don't like, and ask it to rewrite only the piece of code that depends on that decision.

Method Blueprint

STEP 1 (inside one prompt): The model gets the original request + the finished code
       → Reasons: "which decisions in this code were NOT required by the request?"
       → Names a realistic alternative for each one

STEP 2 (same prompt, continued): Turns the reasoning into a list
       → Format: category + description + alternative + severity (low/medium/high)

STEP 3 (separate request, optional): You change one decision from the list
       → Ask it to rewrite ONLY the piece of code that depends on that decision
       → The rest of the code stays untouched

Example in Action

Task: You asked Claude/ChatGPT to write a script that parses a CSV of customer reviews and saves them to a database — a classic chore for a freelancer or analyst automating the boring stuff.

Prompt:

You are a senior developer reviewing code for hidden decisions.

Here is the original request:
"Write a function that reads a CSV of customer reviews 
(columns: name, rating, text) and saves them to a database"

Here is the generated code:
[paste the code the model gave you for this request]

Task:
1. REASONING: list every decision in the code that was NOT 
   explicitly stated in the request. For each, name a realistic alternative.
2. FORMAT: lay out the list like this:
   - Category (input validation / data format / 
     error handling / data storage / performance / security)
   - What the model decided
   - Why it chose that
   - What the alternative was
   - How critical it is (low/medium/high)

Result: The model returns a list of 3-5 hidden decisions: for example, that it picked SQLite instead of PostgreSQL, that it never checks the rating is within 1-5, that a duplicate review silently overwrites the old one, that the file encoding is hard-coded as UTF-8. You see exactly what you'd otherwise only catch by reading the code line by line yourself.

Why It Works

An LLM doesn't "think" before writing code — it generates it as the most likely sequence of tokens, and every intermediate decision (which algorithm, what error format) is made implicitly, with no explanation. Asking the model to "write bug-free code" doesn't work, because the model doesn't know what counts as a bug for you and what's expected behavior.

But the model has a real strength: it's great at explaining after the fact when asked. It can look at finished code and coherently say why it chose that option — it's the same skill as explaining someone else's code.

The method exploits this with a simple trick: it splits generation and explanation into two explicit steps within one prompt. First "write the code," then "now look at this code and find what you decided on your own." That turns silent decisions into text you can read and push back on.

Control knobs: - Decision categories — trim the list (for a throwaway script, drop "performance" and "persistence" and keep only "error handling" and "data format") - Severity level — ask it to show only "medium" and "high" so you don't drown in trivia - Targeted regeneration — instead of "rewrite all the code," ask "rewrite only function X, where the decision about Y changes" — saves time and doesn't break what already worked

Prompt Template

You are a senior developer auditing code for hidden decisions.

Original request: 
{your_original_request}

Generated code:
{code}

Task:
1. REASONING: list every decision in the code that was explicitly 
   NOT required by the request. For each, name a realistic alternative.
2. FORMAT: return the list as:
   - Category: {one of: input validation / data format / 
     error handling / data storage / performance / security}
   - What was decided:
   - Why (in the model's view):
   - Alternative:
   - Severity: {low/medium/high}

After the list, wait for my edits. When I point to a decision to change, 
rewrite ONLY the part of the code that depends on that decision 
and leave the rest of the code unchanged.

🚀 Quick start — paste into the chat:

Here is the AssumptionMiner template for checking code for hidden decisions. 
Adapt it to my task: {your task}. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what your code is and what the original request was — because without those two things it can't compare "what was asked" with "what was done," and that comparison is what the whole method rests on.

Limitations

⚠️ Accuracy drops on complex decisions: the method is good at finding what the model decided (e.g., "MD5 was used"), but worse at pinning down which design decision exactly is behind it — it can confuse similar categories or miss the link to a specific line of code.

⚠️ Edits can "drift apart": if you change one decision and several parts of the code depended on it (a cascade effect), the method doesn't always catch every affected spot — some code may stay inconsistent with the edit.

⚠️ Rare categories are a weak spot: security and long-term data storage decisions show up less often in everyday tasks, and the model is worse at finding them — yet those are exactly the ones with the most serious consequences.

Resources

Jie JW Wu, Michigan Technological University. A benchmark of 180 tasks with 676 annotated hidden decisions, code and data: zenodo.org/doi/10.5281/zenodo.21535058