Efficiency Hallucination: Why an LLM Never Says "There's Nothing to Optimize Here"

TL;DR

Ask any LLM to "optimize" code and it will always suggest an edit, even if the code is already perfect. The model confidently writes a comment like "this will speed up execution," even though there's no real speedup and can't be, simply because the task was framed as "find something to improve" rather than "check whether improvement is needed." Adding one instruction (a confidence threshold plus permission to answer "already optimal") cuts this habit of inventing improvements almost in half, with no loss of quality on code that genuinely can be sped up.

The problem lies in the task framing itself: if the model admits "everything's fine here," it gets no "reward" for that in the user's eyes, but if it invents an edit in a confident tone, there's a chance no one will notice the trick (the code still works, the tests still pass). The researchers tested 9 models (Claude, GPT, Gemini) across 180 runs: on already optimal code, the models invented fake "improvements" 100% of the time, even though there was no speedup and couldn't be. What's more, complex-looking code (nested loops, odd syntax) provokes this fabrication more than simple code: the model mistakes visual complexity for a sign that "this can be improved."

The fix is simple: the prompt gets an explicit instruction with a confidence threshold: "only suggest a change if you're more than 90% sure, otherwise answer 'already optimal'". This wording forces the model to explicitly assess its confidence instead of automatically producing an edit. In the experiment, it cut the share of fake "optimizations" from 100% to 56%, while on genuinely bad code the model still correctly made edits 100% of the time, just as before.

Method Blueprint

WITHOUT the instruction (standard prompt):
"Optimize this code" → the model ALWAYS suggests an edit (even on perfect code)

WITH the instruction (Penalty Prompt), one prompt:
"Optimize the code. Only suggest an edit if you are >90% sure 
it will speed up execution. Otherwise output: ALREADY_OPTIMAL"
→ the model either gives an edit (if it's genuinely sure) or honestly admits 
   the code is already good

Example in Action

Task: You're building a Discord bot or a data-scraping script; the code already works and was written carefully. You ask Claude or ChatGPT to "take a look and see if it can be faster," and you get an edit with a confident comment: "this makes the loop 2x faster." You spend time reviewing and testing an edit that in reality speeds up nothing and sometimes even makes the code more complicated.

Prompt:

Here is a Python function. Optimize it for execution speed.

Suggest a change ONLY if you are more than 90% sure 
it will actually speed up execution (for example, you can prove 
it through an analysis of algorithmic complexity: O(n²) → O(n log n), 
not "by eye").

If you don't have that confidence, leave the code alone and output only: 
ALREADY_OPTIMAL

Code:
{paste your code}

Result: If the code really can be sped up algorithmically (for example, replacing a nested loop with a dictionary), the model will give an edit justified by algorithmic complexity. If the code is already close to optimal, the model is more likely to honestly write "ALREADY_OPTIMAL" instead of inventing a nonexistent improvement. There's no full guarantee: according to the study, the model still sometimes "bluffs," in roughly half the cases on genuinely optimal code.

Why It Works

A plain "optimize this" prompt implicitly assumes an improvement definitely exists, and a model trained to be "helpful" always finds something to suggest, even if objectively there's nothing to improve. It's like asking an editor to "fix up this text": they'll almost never say "the text is fine as is," because the task already implies an edit.

LLMs are good at estimating algorithmic complexity through reasoning (for example, explaining why O(n²) is worse than O(n log n)); they can do that without actually measuring speed, purely by logic. But they have nothing to confirm that code is already optimal with: there's no static proof of "it can't go any further," so instead of an honest "I don't know" they choose a confident bluff.

The confidence-threshold instruction exploits exactly this difference: on genuinely bad code the model easily reaches more than 90% confidence through logical complexity analysis, while on optimal code it has no such basis, and by its own logic it must switch to "not sure" and write "already optimal."

Control knobs: - The confidence threshold (90%): you can lower it to 70-80% if you want more edits and are willing to check them by hand; raise it to 95% if reliability is critical. - The exit condition wording ("ALREADY_OPTIMAL"): you can swap in any other phrase or ask the model to explain why it isn't sure, which makes its reasoning more transparent. - Asking for justification through algorithmic complexity analysis (rather than "intuition"): strengthens the effect, because it forces the model to explicitly check the logic instead of guessing.

Limitations

⚠️ Not a complete fix: even with the instruction, the model keeps inventing "improvements" in roughly half the cases on already optimal code. It lowers the risk but doesn't remove it entirely.

⚠️ Confidence self-assessment is unreliable: the model itself judges whether it's 90% sure, and that self-assessment is the very mechanism that produces the bluff. External verification (actually measuring execution speed) remains the only reliable way to make sure an improvement is real.

⚠️ Visually complex code fools the model more: code with nested loops, list comprehensions and "dense" syntax provokes more fake "optimizations" than simple linear code, even if both are equally optimal. Be especially skeptical of edits to complex-looking code.

⚠️ Different models calibrate very unevenly: in the study, a cheaper, lighter version of a model sometimes admitted "the code is optimal" more honestly than the pricier, "smarter" version from the same family. Don't assume a top-tier model is automatically more reliable here; test it on your own task.

How They Tested It

The team took five LeetCode problems (from the EffiBench benchmark) and made two variants of each: a reference-optimal one (the best solution) and an artificially degraded one (same result, but with a worse algorithm, e.g., O(n²) instead of O(n)). Then they ran 9 models from three families (Claude, GPT, Gemini) through the API twice: with a plain "optimize" prompt and with a "penalty" prompt (90% confidence threshold + the option to say "already optimal"). 180 runs in total.

The logic: if a model is just "bluffing" for the sake of a nice-looking answer, it will suggest edits on optimal code as often as on bad code. That's exactly what the control group confirmed: 100% "edits" even where there was nothing to improve. With the confidence-threshold instruction added, the share of honest "the code is optimal" admissions rose from 0% to 44%, while performance on genuinely bad code didn't drop: 100% correct edits stayed in place.

The most intriguing part isn't the instruction's effect itself (that's a fairly well-known move: "give the model the right to say I don't know") but the fact that the pricier, more "advanced" model versions (GPT-5.4, Gemini 3.1 Pro) calibrated worse than their light, cheap versions (GPT-5.4-Mini delivered a perfect result: 100% honest admissions). That cuts against the intuition that "more expensive = more reliable" and hints that more "advanced" training strengthens the model's habit of always suggesting something instead of holding back.