Zero-shot flavor (use it right away):

STEP 1: Analyze the request
├─ User Intent (what the user wants)
├─ Ambiguity (what's unclear)
├─ Potential Risks (possible risks)
├─ Action Decision (answer / clarify / refuse)
└─ Safe Response Plan (plan for a safe answer)
       ↓
STEP 2: Generate the final answer based on the analysis

Both steps run in a single prompt via a structured instruction.

Advanced flavor (ContextLens):

STEP 1: A separate request to a small model (3B)
       → generates the context snippet

STEP 2: The main request with the original prompt + snippet
       → final answer

Task: A client wants a post for a food delivery blog titled "How to Destroy Your Competitors in 3 Months." It's unclear: do they want marketing tactics, or shady methods?

Prompt:

Analyze the user's request before answering:

REQUEST: "Write an article 'How to Destroy Your Competitors in the Food Delivery Niche in 3 Months'"

ANALYSIS:

1. USER INTENT:
[What does the user want to get?]

2. AMBIGUITY:
[What in the request could be read in different ways?]

3. POTENTIAL RISKS:
[Which interpretations could lead to problems?]

4. ACTION DECISION:
[Choose: ANSWER / CLARIFY / REFUSE]

5. SAFE RESPONSE PLAN:
[How should you answer given the risks?]

---

Based on the analysis above, give the user your final answer.

Result:

The model will first show a full analysis in five blocks: it'll note that "destroy" could mean both legitimate competition and dishonest methods, flag the ambiguity, and pick the action "CLARIFY" or "ANSWER with caveats." Then it'll give the final answer: either a clarifying question ("Do you want an article about marketing advantages, or...?") or a safe version of the article focused on fair competition, with a caveat about ethics.

The LLM's weakness: The model generates text sequentially, left to right. When it sees a request, it starts forming an answer immediately, with no time to "think" about context and ambiguities. Small models in particular (up to 8B) see trigger words (kill, destroy, hack) and refuse on reflex, without working out the intent. The result is either excessive refusals (over-refusal) or compliance with sketchy requests.

The LLM's strength: Models are excellent at following structured instructions and good at analyzing text (identifying intent, spotting ambiguity, assessing risk). If you explicitly ask them to analyze first and answer second, they handle it.

How the method uses that strength: The prompt splits the process into two stages: first the analysis (what the user wants, what the risks are, how best to answer), then the answer based on the analysis. The five-block structure removes ambiguity: the model doesn't guess, it systematically checks each aspect. The final answer is generated with the context already "in the model's head."

Control knobs:

• Analysis blocks: drop "Ambiguity" if the request is clear-cut (saves tokens), add a "Legal/Ethical Check" for especially sensitive topics
• Action Decision: replace the "Answer/Clarify/Refuse" options with your own criteria (for example, "Full answer / Answer with limits / Alternative suggestion")
• Analysis output: add "No explanations, just the decision" if you need a quick result without long reasoning
• Number of stages: for simple requests use only 3 blocks (Intent → Risks → Decision); for complex ones add "Stakeholders" or "Long-term Consequences"

Analyze the user's request before answering:

REQUEST: {user_request}

ANALYSIS:

1. USER INTENT:
[What does the user want to achieve? What's the real goal behind the request?]

2. AMBIGUITY:
[What in the request could be read in different ways? What interpretations are possible?]

3. POTENTIAL RISKS:
[Which interpretations are dangerous? What could go wrong?]

4. ACTION DECISION:
[Choose one: ANSWER / CLARIFY / REFUSE]
[Justify your choice in one sentence]

5. SAFE RESPONSE PLAN:
[How should you answer given the risks? What caveats are needed?]

---

Based on the analysis above, give the user your final answer.

What to fill in: - {user_request}: the original request that raises doubts or could be ambiguous

🚀 Quick start: paste this into the chat:

Here's a template for two-step request analysis. Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask about the specifics of your field (legal risks? technical constraints? ethical aspects?) and which Action Decision options fit (maybe instead of "refuse" you need "suggest an alternative"?). It'll take the structure from the template and tune it to your context.

The study surfaced three key findings about how LLMs behave with ambiguous requests:

1. Small models overplay it safe

Qwen2.5-3B and Llama-3-8B refuse 15-45% of harmless requests if they contain "suspicious" words. The reason: when trained on safety datasets, the model memorizes the pattern "see 'kill/destroy/hack' → refuse" but doesn't learn to tell contexts apart. Two-step analysis forces the model to work through the context first (Intent, Ambiguity) and decide second, and over-refusal drops by 6-11%.

2. Big models miss on adversarial examples

GPT-4o on the Wildjailbreak dataset (deliberately tangled requests) gets it wrong 52% of the time: it either refuses harmless requests or complies with sketchy ones. With context analysis, errors drop to 34% (-18%). Structured analysis works like a checklist: the model doesn't rely on gut feeling, it checks every item.

3. Context transfers between models

The biggest surprise: a context snippet generated by one model (for example, Llama-3-8B) improves another (GPT-4o). Zero-shot analysis from Llama gives GPT-4o +22.9% accuracy on Wildjailbreak. That means the analysis principles are universal, not tied to any specific model's architecture.

ContextLens's autoencoder mechanics

The advanced version (the RL-trained model) uses a clever trick: the context generator creates a snippet that is passed to another model (the decoder) along with a partially masked prompt. The decoder has to reconstruct the original prompt from the snippet. The reward comes if it reconstructs it accurately + both answers (from the generator and the decoder) are safe.

Why: this forces the snippet to carry as much information about the prompt as possible without copying it word for word (there's a penalty for copying). The result: the snippet extracts the essence: intent, context, risks. The trained model adds another +2-4% accuracy on top of zero-shot.

Comparison with reasoning models

Chain-of-Thought and reasoning models (SafeChain, TARS) generate thinking traces for themselves, to answer better on their own. ContextLens generates context for another model, and that's the key difference. The result: - Thinking traces are co-adapted with the model that produces them and transfer poorly to other models - ContextLens generalizes: a single context improves Llama, GPT-4o and Qwen alike

⚠️ Simple requests: For clear-cut tasks ("capital of France," "translate this text") two-step analysis is overkill: you're burning tokens for nothing. Use it only if the request contains ambiguity or risk.

⚠️ Small models (under 3B): On Qwen2.5-1.5B, context analysis sometimes reduces compliance (fulfilling harmless requests) from 87% to 54%: the model is too small to hold both the analysis and a quality answer. It works well on models from 3B up.

⚠️ Subjective criteria: The method works great for safety (harmful/safe), but for creative tasks (how interesting is the text? how original?) structured analysis can "dry out" the result: the model gets too cautious.

⚠️ Extra tokens: Zero-shot analysis adds 150-300 tokens to every answer (~10-15% overhead on API bills for average requests). The advanced flavor (two requests) doubles the number of API calls.

The Microsoft researchers tested the method on safety benchmarks: AdvBench, SafetyInstruct (explicitly harmful prompts), Wildjailbreak and XSTest (adversarial: deliberately tangled requests where the model has to tell harmless from harmful).

The zero-shot prompt (from Appendix A.1 of the paper) instructs the model to produce five sections: 1. User Intent 2. Ambiguity
3. Potential Risks 4. Action Decision (direct answer / ask clarification / refuse) 5. Safe Response Plan

The trained ContextLens model (3B parameters, Qwen2.5-3B-Instruct) was trained on 5,000 examples from Wildjailbreak using GRPO (Generalized Reward Policy Optimization). Architecture: - Generator (Qwen2.5-3B) produces context _c_ and answer _r_g - Decoder (Llama-3.2-3B, frozen) receives _c_ + a partially masked prompt _p'_, reconstructs the prompt _p_d + generates answer _r_d - Reward: penalty (0) if _c_ contains a verbatim copy of _p_, otherwise SIM(_p_d, _p_) + Safe(_r_g) + Safe(_r_d)

Results (Table 1 of the paper): - GPT-4o + ContextLens: ASR (attack success rate) on Wildjailbreak fell from 52.65% to 34.35% (-18.3 pp) - Llama-3-8B + ContextLens: ASR on SafetyInstruct fell from 22.7% to 1.6% (-21.1 pp) - Zero-shot context from Llama-3-8B for GPT-4o gives a harmonic mean of 87.86% vs. a 64.07% baseline (+23.8 pp)

Datasets: - AdvBench, SafetyInstruct: 520 and 500 examples of explicitly harmful prompts ("how to make a bomb," "write a phishing email") - Wildjailbreak: 262 adversarial requests (sound harmless but may be harmful, or vice versa) - XSTest: 450 examples alternating adversarial benign (should answer) and harmful (should refuse)

Metrics: - ASR (Attack Success Rate): % of cases where the model fulfilled a harmful request (lower = better) - Compliance: % of cases where the model fulfilled a harmless request (higher = better)
- H-Avg (Harmonic Mean): the balance between (1 - ASR) and Compliance, showing the trade-off between safety and helpfulness

Evaluation process: GPT-4o as the judge received the prompt + the model's answer and classified it: full compliance / full refusal / partial refusal. For each dataset they computed ASR and Compliance, then the harmonic mean.

Training ContextLens: - 5,000 training + 500 validation examples from Wildjailbreak - GRPO (on-policy RL) with learning rate 1e-6, batch size 4, 8 generations per sample - Generator: Qwen2.5-3B-Instruct, Decoder: Llama-3.2-3B-Instruct (frozen) - Early stopping when validation reward > 0.9 - Trained on a single NVIDIA A100 (80GB)

Transferability testing: Context from one model (Qwen2.5-3B, Llama-3-8B, GPT-4o) was applied to other models (Llama-3.2-3B, GPT-4o). Finding: zero-shot context is universal; even context from the small Llama-3-8B improves GPT-4o by 22.9%.

Learning to Extract Context for Context-Aware LLM Inference: Kim et al., Microsoft Research, 2025

ContextLens Project Page

Authors: Minseon Kim, Lucas Caccia, Zhengyan Shi, Matheus Pereira, Marc-Alexandre Côté, Xingdi Yuan, Alessandro Sordoni (Microsoft Research)

Related work: - SafeChain (Jiang et al., 2025): a reasoning model for safety via supervised fine-tuning - TARS (Kim et al., 2025): a reasoning model via RL - GRPO (Shao et al., 2024): a training algorithm based on policy optimization