The Imperfective Paradox: LLMs Assume Goal-Directed Actions Got Finished

TWO KINDS OF ACTIONS:

Goal-directed (telic): building a house, writing a report, baking a pie → "was in the process" ≠ "finished" (there's an endpoint)

Process (atelic): running, swimming, playing → "was in the process" = "did it" (the process is the event)

THE LLM PROBLEM:

The model sees: "The programmer was developing a system" → Concludes: "The programmer developed a system" ❌

Even if the text says: "...but the project was put on ice" → It still concludes: "developed" ❌

ATTEMPTS TO FIX IT:

• CoT: reduces bias for goal-directed actions, but breaks process ones
• Counterfactual: removes the bias completely, but makes the model skeptical of everything
• Result: the model swings between extremes and can't tell the action types apart

The problem: An LLM fills in facts when it analyzes unfinished actions. That's critical when evaluating experience, project progress, or whether commitments were fulfilled.

Example task: Reviewing a resume from a candidate for a product manager role.

Standard prompt (with the problem):

Here is a candidate's resume. List the products they successfully launched:

[Resume:]
- Was developing a mobile food delivery app (2022-2023)
- Was launching a services marketplace for small businesses (2023-2024)
- Was rolling out a CRM system for the sales team

What the model will return: Most likely it'll list all three projects as "launched," even though the wording points to a process, not a result. The model sees "was launching a marketplace" → fills in "launched a marketplace."

Improved prompt (with bias protection):

Analyze the resume and split the candidate's activities into TWO categories:

1. COMPLETED PROJECTS — where a result is explicitly stated ("launched", "rolled out", "shipped")
2. PROCESS WITHOUT A CONFIRMED RESULT — where only involvement is stated ("was developing", "was launching", "was rolling out")

CRITICALLY IMPORTANT: 
- "Was launching" does NOT mean "launched"
- "Was developing" does NOT mean "developed"
- Don't fill in results. If no explicit result is stated — category 2

For category 2, note: "Clarify in the interview — was the result achieved?"

[Resume:]
- Was developing a mobile food delivery app (2022-2023)
- Launched a services marketplace for small businesses with 500+ service providers
- Was rolling out a CRM system for the sales team

Result: The model will sort the activities into categories, flagging only the second item as completed (the wording "launched" + a concrete result, "500+ service providers"). The rest it'll mark "needs clarification." You'll see the candidate's real experience, not an imagined one.

The LLM weakness: The model was trained on texts where goals are usually mentioned once they've been reached. News, reports, and stories say "the company was launching a product" only when the context is relevant — which most often means a successful launch. Reporting bias: everyday failures don't make it into texts, successes do. The model memorized the statistics: "was launching" → "launched" holds in 80%+ of cases.

The LLM strength: The model is good at following explicit, structured instructions. When you break a task into clear categories with examples ("this ≠ that"), the model can suppress the statistical pattern. Pseudo-XML tags, numbering, and contrasting examples work as "attention anchors" — the model focuses on distinguishing rather than predicting the most likely continuation.

How the fix works: Instead of an abstract "analyze the resume," you give the model an explicit fork: "completed" vs "in progress." You add an inference-blocking rule: "was developing ≠ developed." You reinforce it with a meta-instruction: "don't fill in the gaps." This creates a counterweight to the statistical prior. The model doesn't fully correct itself (teleological bias is very strong), but accuracy goes up.

Important to understand: This isn't "fixing" the model, it's compensating for an architectural weakness through prompt engineering. In zero-shot, models hallucinate completion 98% of the time. With explicit instructions, that drops to 40-60% (depending on the model). You can only remove the bias completely with aggressive methods (counterfactual: "imagine the action was interrupted"), but then the model becomes skeptical of everything.

The trade-off in prompts: - No explicit instructions → 98% false completions - With explicit instructions → ~40-60% false completions + ~35% false refusals (the model starts doubting even the obvious) - With aggressive blocking → 0% false completions, but ~90% false refusals

Pick the strictness level for the task: stricter for high-stakes decisions (hiring, legal analysis), looser for drafts.

Analyze the text and split the actions into TWO categories:

1. COMPLETED ACTIONS — where a result is explicitly stated
   Markers: "did", "launched", "created", "achieved", + concrete result metrics

2. PROCESS WITHOUT A CONFIRMED RESULT — where involvement or intent is stated
   Markers: "was doing", "was launching", "was developing", "worked on"

CRITICALLY IMPORTANT:
- {action_in_progress} does NOT mean {action_completed}
- Examples:
  • "was developing a system" ≠ "developed a system"
  • "was launching a product" ≠ "launched a product"
  • "was writing a book" ≠ "wrote a book"
- If an explicit result is NOT stated — it's category 2
- Don't assume completion. Work ONLY with what's written

For category 2, add a note: "Result not confirmed — needs clarification"

[Text to analyze:]
{your_text}

What to fill in: - {action_in_progress} and {action_completed} — replace with verbs from your context. For resumes: "was developing/developed", for projects: "was implementing/implemented", for marketing: "was launching/launched" - {your_text} — a resume, project descriptions, a report, any text with actions in it

When to use it: - Reviewing resumes and portfolios - Assessing project progress - Checking whether commitments were fulfilled (contracts, agreements) - Fact-checking articles and press releases - Evaluating a company's or startup's track record (what they actually did vs what they tried)

It's not a bug, it's a feature of the architecture. An LLM is trained to predict the most likely continuation. In the text corpus, "was building a house" most often leads to a context where the house got built. The model doesn't reason logically — it extrapolates patterns. For narratives that's useful (story generation); for factual analysis it's dangerous.

Where the problem is critical: - Hiring: "was involved in the project" → the model concludes "delivered the project" — you overrate the candidate - Due diligence: "the company was launching a product" → the model concludes "launched" — you invest in something that doesn't exist - Legal analysis: "began performing its obligations" ≠ "performed" — the model can distort the facts - Project management: "the team was building the feature" → "built" — you think the task is closed

This study shows the limits of LLMs. The model works as a narrative predictor, not a fact validator. When you ask it to "analyze," the model doesn't switch into "logician mode" — it keeps predicting what's likely. You have to switch the mode explicitly through the prompt: give it categories, rules, and inference blockers.

Practical insight: For tasks where facts matter (not creativity), always add a no-gap-filling rule. Phrases like "don't fill in the gaps," "only what's explicitly written," "if X isn't stated, write 'unknown'" work as a counterweight to the statistical bias.

How it ties into other LLM problems: - Hallucinations: the model fills in facts → same mechanism - Overconfidence: the model is sure about what it made up → reporting bias creates high confidence - Context neglect: the model ignores "but the project was put on ice" → the prior outweighs the context

Understanding the Imperfective Paradox gives you a mental model of how an LLM thinks: it doesn't reason from facts to conclusions, it reasons from statistics to the most likely continuation. Your job is to use the prompt to shift "most likely" toward "logically correct."

⚠️ You can't fully remove the bias with a prompt: Even with explicit instructions, models hallucinate completion in 40-60% of cases (versus 98% in zero-shot). It's an architectural trait, not a prompt mistake.

⚠️ Strictness vs usefulness trade-off: The harder you block gap-filling for goal-directed actions, the more often the model starts doubting obvious conclusions. The counterfactual prompt ("imagine interruptions") kills 100% of false completions but creates ~90% false refusals — the model stops drawing any conclusions from process verbs at all.

⚠️ Depends on model size: Small models (1.5B-7B parameters) barely respond to anti-bias instructions. The study found that the ability to distinguish action types is emergent (it appears abruptly) at the 32B+ parameter level. Qwen2.5-32B jumped from 53% errors (7B) to 9% (32B). If you're using a small model, the prompt will have a weak effect.

⚠️ The bias is strongest for creation verbs: Models hallucinate completion more often for "was creating a product" or "was building a system" (Creation verbs) than for "was transitioning into state X" (Change of State). If your context is about creating artifacts, the problem is sharper.

⚠️ It's not a permanent "fix": Every request needs explicit instructions. If you write "analyze the resume" without the blockers, the bias comes right back. This isn't training the model, it's steering through context.

Researchers from LMU Munich and the University of Tokyo built the IMPERFECTIVENLI dataset — 400 premise-hypothesis pairs for testing logic. They took 100 goal-directed verbs (build, write, fix — there's an endpoint) and 100 process verbs (run, swim, play — no endpoint). For each verb they created 4 conditions:

Group A: "was building a gazebo, but a storm destroyed it" → "built a gazebo" (should be: False) Group B: "were running on the track, but it started to hail" → "ran on the track" (should be: True) Group C: "was building a gazebo" → "built a gazebo" (should be: Unknown) Group D: "were running on the track" → "ran on the track" (should be: True)

They tested 7 models (7-9B parameters): Llama-3.1, Mistral, Qwen2.5, DeepSeek, Gemma-2, GLM-4, Yi-1.5. They also tested Qwen2.5 at different sizes (1.5B → 72B) to check the effect of scale.

Main metric: Teleological Bias Rate (TBR) — the share of cases where the model predicted "completed" for Group C (it should be "unknown"). In zero-shot, nearly all models showed TBR = 94-98%, meaning they hallucinated completion almost every time.

What surprised them: Even when the text explicitly stated an interruption (Group A), models often ignored it. For example, Llama-3.1 in zero-shot said "completed" in 53% of cases despite the text saying "but a storm destroyed it". That showed how strong the bias is — it outweighs explicit context. The model sees "was building a house" → activates the idea of "a built house" so strongly that the attention mechanism doesn't manage to correct for the word "destroyed."

Why it happens: Embedding analysis showed that internally the model does distinguish "process" from "result" (the vectors differ). But at the answer output stage, the prior kicks in: "if a goal-directed action is mentioned → it was most likely completed" (statistics from the training data). The problem isn't knowledge representation, it's decision calibration. The model knows the difference but can't apply that knowledge against a strong statistical pattern.

Effect of prompts: Chain-of-Thought lowered TBR to 40-60%, but broke correct inferences for process verbs (accuracy dropped from 98% to 65%). The counterfactual prompt removed the bias completely (TBR = 0%), but the model began refusing to draw any conclusions. The study exposed a fundamental problem: models can't dynamically calibrate their level of skepticism to the verb type.

Practical insight: If logical correctness matters to you, use explicit instructions + be ready for the trade-off. You won't get a perfect balance through prompting, especially on models <30B parameters. On 32B+ models (Qwen2.5-32B, for example), prompts have a much stronger effect — that's where the ability to distinguish action types shows up as an emergent ability.

The Imperfective Paradox in Large Language Models Dataset and code: https://github.com/boleima/ImperfectiveParadox

Authors: Bolei Ma (LMU Munich & MCML), Yusuke Miyao (The University of Tokyo)

Theoretical background: Vendler (1957) — aspectual classes, Dowty (1977, 1979) — formalization of the Imperfective Paradox, Bennett & Partee (1978) — sub-interval property