Apply to my task
arXiv:2610.22943 ★ 93 Oct 16, 2026

LENS: Reliable Expert Analysis With Prompts the Model Derives From Two Worked Cases and Its Own Failures

Key result

Ask a model point-blank "vulnerable or not?" and it'll be wrong more than half the time. The LENS method gets you expert-style analysis even when the expert's rules were never written down. The trick: don't write the rules yourself – let the model derive them from two finished write-ups. The examples yield a discipline prompt (how to reason). The model's own mistakes yield a fact-gathering prompt (what to look for before concluding). Together they produce an analysis that matches the expert's.

01

TL;DR

LENS is a two-prompt scheme. You don't ask the model "vulnerable or not?" head-on. First, the model derives the "unwritten rules" of analysis from finished expert write-ups and packages them into a prompt. Then, from its own mistakes, it derives a second prompt: what to look for in the documents, and where, before reaching a verdict. A production run has two stages: gather the facts, then analyze by the rules.

Ask the model point-blank and it gets it wrong more than half the time. Two reasons. First, it "drifts" away from expert discipline: it assumes things an expert would toss out. For example, it assumes the attacker holds a secret key they would never actually have. Second, it rushes to a verdict before collecting all the facts. The information it needs sits right there in the documents, but the model never went looking. And the expert can't simply write down their unwritten rules: "we know more than we can tell." Nor do they know in advance which facts will matter.

The method gets around this. The model reconstructs the rules from examples: one "vulnerable" write-up and one "not vulnerable" write-up are enough. It reconstructs the fact-gathering plan by comparing its own wrong analysis with the reference one. One or two refinement rounds usually do it; after that, you run the prompts on new cases.

Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.

02

Method Blueprint

SETUP (once, 1–2 rounds)
STEP 1: 2 expert write-ups (positive + negative)
        → meta-prompt → DISCIPLINE prompt (p_exp)
STEP 2: run p_exp on the same 2 examples, compare with the reference
        → if they differ: error + reference → meta-prompt
        → FACT-GATHERING prompt (p_evi)
STEP 3: p_evi gathers facts → p_exp is re-run with them → repeat
        until it matches the expert write-up

PRODUCTION (for every new case)
STAGE A: p_evi + object → fact dossier
STAGE B: p_exp + object + dossier → report (and a PoC, if applicable)
        (separate requests, ideally in separate chats)
03

Example in Practice

Task: You run marketing for a big seller on Amazon and Etsy. Hundreds of ad copies and product listings go out every week. Your in-house lawyer, who knows FTC truth-in-advertising rules cold, approves or kills phrases like "#1 in America," "up to 70% off," "cures," "lowest price guaranteed." Nobody ever wrote their rules down: they live in the lawyer's head and in a couple of old legal memos. You want the model to review new copy the way this lawyer does, not "the way the internet says."

Prompt (step 1 — deriving the discipline):

You are given examples of ad copy reviewed for compliance with FTC advertising rules.
Each input is an ad text with context (marketplace, product category, 
supporting documents). Each output is the lawyer's opinion: is there a violation, 
why, which rules apply, and how to fix it.

Your task: derive a prompt that captures the analytical discipline shown 
in these examples and can be used to review new copy. Output only the prompt.

Example 1 (violation)
Input: Text: "Our tea is #1 in America! Lowers blood pressure." Marketplace: Amazon. 
Category: dietary supplement. Documents: no substantiation for "#1".
Output: [lawyer's opinion: two violations — unsubstantiated superiority claim 
and a medical claim for a supplement; rules cited; how to rewrite]

Example 2 (no violation)
Input: Text: "30% off your second item through March 31." Marketplace: Etsy. 
Category: apparel. Documents: promo terms are published on the listing.
Output: [lawyer's opinion: no violation, because the terms and end date are stated 
and the discount applies to the described items; what was checked]

Result: The model returns one ready-made prompt. It's a "how to analyze" instruction: which assumptions to make, what counts as proven, what order to check things in, what format the opinion should take. Next, run it on the same two examples. If its verdict doesn't match the lawyer's, grab the step 2 template (below) and ask the model to derive a fact-gathering prompt: which rules, FTC guidance and promo terms must be found before writing an opinion. Then re-check on the same examples. After 1–2 rounds, run it on new copy in two stages: dossier first, opinion second.

04

Why It Works

The weakness. By default, the model answers fast and "on average." It doesn't know the expert's rules, because they aren't written anywhere. It doesn't pull in the facts it needs (rules, exceptions, consequences) until someone asks. So the answer sounds convincing but rests on a partial picture.

The strength. The model is good at generalizing from examples: from "input → expert analysis" pairs, it reconstructs an instruction that reproduces that analysis. It's also good at dissecting its own mistakes when it sees the reference right next to them with a note saying "this is wrong."

How the method exploits this. It splits two jobs: how to reason (discipline) and which facts are needed (dossier). The model writes both instructions itself, from the expert's material. A key detail: you don't ask "find the facts for this case," you ask "write a general prompt that gathers facts for any case". That's why the prompt transfers to new objects.

Control knobs: - Number of examples. The authors got a working discipline from two. One positive and one negative is the minimum. Add borderline cases if the model gets confused. - The negative example is mandatory. Without it, the model derives the rule "everything is suspicious." - Stopping condition. In the paper, the expert checks: the verdict matches, the reasoning matches, the PoC works. Swap in your own criterion, e.g., "matches the lawyer's opinion on 5 out of 5." - The phrase "must work on new cases" in the meta-prompt. It keeps the model from hard-coding details of a specific case into the prompt. - Dossier structure. In the authors' fact-gathering prompt, the items are: who the sender and receiver are, what the receiver does with the field, the chain of consequences, what state changes (persistent or temporary). Replace them with items that fit your domain.

05

Prompt Template

Step 1. Derive the analytical discipline (based on Table 1 in the paper):

You are given example inputs and outputs for the task: {task_description}.
Each input describes {analysis_object} together with {supporting_data}. 
Each output is an analysis written by an expert: {analysis_format}.

Your task: derive a prompt that captures the analytical discipline 
visible in these examples and can guide the analysis of new {objects}.
Output only that prompt.

Example format:
Input: {object_and_data}
Output: {expert_analysis}

Example 1 ({positive_type}):
{input_1}
{output_1}

Example 2 ({negative_type}):
{input_2}
{output_2}

Step 2. Derive the fact-gathering prompt from a failure (based on Table 2):

I asked an LLM to analyze {analysis_objects}. The LLM followed the analysis prompt 
({p_exp}), but its analysis failed expert review. I believe the reason is that 
the LLM didn't know enough context about each object.

Failure case:
Analysis history:
{object_and_data}
{analysis_prompt_p_exp}
{model_analysis}
Feedback:
{expert_analysis}
This is the reference. The model's analysis is wrong.

Your task: derive a GENERAL fact-gathering prompt. For any {object}, 
it must instruct the LLM to find in {source_documents} everything needed 
for the analysis, so that the gathered context is enough for an analysis 
that reproduces the reference. The prompt must work on new {objects}.
Output only the prompt.

Step 3. Production run (two separate requests):

[Request A] {p_evi — the derived fact-gathering prompt}
Object: {new_object}
Sources: {documents}

[Request B] {p_exp — the derived discipline prompt}
Object: {new_object}
Fact dossier: {output_of_request_A}

Fill in: the task description, the objects, two expert write-ups (one positive, one negative), and the sources (rules, regulations, documentation).

🚀 Quick start — paste into your chat:

Here is the LENS method template (deriving a discipline prompt and a fact-gathering prompt). 
Adapt it to my task: [your task]. 
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which expert write-ups you have (you need one "yes" and one "no"), which sources to search for facts, and what the correct opinion format looks like. Without these three things the method doesn't work: the rules come from the write-ups, the facts come from the sources, and the format locks in the result.

06

Limitations

⚠️ You need expert references. The method starts from finished expert write-ups. Without at least a pair (one "yes," one "no"), the derived prompt is guesswork.

⚠️ You need human review. In the paper, the "analysis matches the expert's" criterion was judged by an expert. Only a human confirms a failure is fixed, not the model itself.

⚠️ False positives remain. The model finds nearly everything that's there, but a noticeable share of its "finds" turn out to be false alarms. The output is a list of candidates to check, not a verdict.

⚠️ You need a source corpus. Fact-gathering assumes the model can access the documents: attached to the chat or hooked up via search. With scattered, locked-away or tribal knowledge, the second prompt is useless.

⚠️ The paper's domain is narrow; transfer is only partly tested. The main evaluation is vulnerability analysis of telecom protocols. The authors showed transfer to other protocols, but not to legal or marketing review. Using it outside security is an extrapolation.

⚠️ Errors in the reference carry over. If the expert write-ups contain outdated or debatable logic, the model will "learn" it faithfully.

07

How They Tested It

The authors took Diameter, the protocol 4G network nodes use to exchange subscriber data. It has hundreds of fields. The question is which fields can be forged to cause real damage. That's too many to check by hand, and without knowing the 3GPP specs, scattered across many documents, it's easy to get it wrong.

First they tried the head-on approach: the model sometimes even built working exploits, but it was wrong in more than half the cases. Error analysis revealed two causes: the model assumed unrealistic attacker capabilities and drew conclusions without collecting the related facts. So the authors built LENS: the "rules" prompt is derived from expert write-ups, the "fact-gathering" prompt is derived from failures, and the two are refined against each other, because each depends on the other. 1–2 rounds were enough, and two random examples, one positive and one negative, proved sufficient for deriving the rules.

They evaluated on a hand-labeled sample, across four different LLMs, with repeated runs. Recall (share of vulnerable fields found) was 94–98%, precision (share of correct among those found) 79–89%, overall accuracy 84–92%. Then they ran LENS over 766 fields of the S6a and Cx interfaces: 204 fields (about 27%) turned out to be exploitable, 196 of them previously unknown. The attacks they identified were verified on real implementations: 95 new vulnerabilities found, including 78 in commercial systems. Finally, the method was applied to 16 other protocols and confirmed 118 new vulnerabilities. That matters for you as a reader: the approach isn't tied to a single domain.

The practical takeaway: a model's mistakes in expert work usually come not from stupidity but from missing rules and missing facts. You can get both from the model itself if you give it reference cases.

08

Adaptations and Extrapolations

🔧 Technique: split "how to think" and "what to know" into two separate requests → fewer premature conclusions

Even without deriving prompts: before any expert question, make a separate request just for gathering facts (with no permission to conclude anything), and ask for the verdict in a second one.

Don't draw conclusions. Build a dossier on {object}: 
1) who does what with it, 2) what it affects further down the chain, 
3) what exceptions and constraints exist, 4) where this is documented in {sources}. 
Facts with citations only.

Extrapolation: agent rules from code reviews. Take two reviews by your tech lead: one approved PR and one rejected PR with comments. Feed them to the model using the step 1 template and ask it to derive rules as a section for CLAUDE.md or AGENTS.md. Then give the agent a couple of PRs it botched, along with the review comments, and repeat step 2: "derive a general instruction for what the agent must read and check before writing code." That's how your team's unwritten requirements end up in the instructions file.

09

Resources

  • LENS — "When Core-Network Elements Become Reachable: Field-Level Exploitability Analysis in Diameter," accepted at the IEEE Symposium on Security and Privacy (S&P) 2027
  • Authors: Wei Yi, Changlong Han, Xue Zhang, Qiao Zhang, Yi Chen, Jin Cao, Di Tang, Yulong Fu, Ben Niu, Hui Li
  • Universities and institutes: Xidian University, The University of Hong Kong, Sun Yat-Sen University, Institute of Information Engineering (CAS)
  • Conceptual references: Polanyi's paradox ("we know more than we can tell"), instruction induction from "input → output" pairs (reference [7] in the paper), learning from one's own failures (reflection, references [8], [9], [15])
  • Domain sources: 3GPP specifications (S6a, Cx)

Research digest

Key result

Ask a model point-blank "vulnerable or not?" and it'll be wrong more than half the time. The LENS method gets you expert-style analysis even when the expert's rules were never written down. The trick: don't write the rules yourself – let the model derive them from two finished write-ups. The examples yield a discipline prompt (how to reason). The model's own mistakes yield a fact-gathering prompt (what to look for before concluding). Together they produce an analysis that matches the expert's.

Principle

The method untangles two jobs. One: how to reason. Two: which facts are needed. Models usually mix them up and do everything at once. The process: 2 expert write-ups (one "yes," one "no") → discipline prompt → run on the same two examples → if it doesn't match, the error and the reference become a fact-gathering prompt → 1-2 rounds until it matches. After that, the work runs as two requests, ideally in separate chats. First the fact dossier, then the analysis by the rules. Think detective and judge: one gathers the evidence, the other hands down the ruling. The model searches first and thinks second, not the other way around.

Why it works

The model has two weaknesses. First, it "drifts" from expert discipline. It assumes things an expert would toss out immediately. For example, it assumes the attacker has a secret key that, in practice, they don't. Second, it rushes to a verdict. The facts it needs are right there in the documents, but it never looked. The expert can't spell out their unwritten rules either. They know more than they can tell. Nor do they know in advance which facts will matter. The model, meanwhile, is good at two things. It generalizes from "input → analysis" pairs. And it dissects its own mistakes when it sees the reference next to them, marked "this is wrong." You don't have to put the rules into words: showing them in two examples is enough. Key detail: you don't ask "find the facts for this case," you ask "write a general prompt that gathers facts for any case." That's why the prompt transfers to new objects. The negative example is mandatory, or the model derives the rule "everything is suspicious."

When to apply

Expert analysis with unwritten rules → reviewing risks, contracts, applications, technical reports, especially when you have a couple of old "yes" and "no" opinions and a folder of documents holding the facts you need. Important: in the paper, the method was tested on vulnerability analysis of telecom protocols and on other protocols. Legal or marketing review is an extrapolation – test it on your own data. NOT a fit if you have no references (it'll be guesswork). Not a fit if the knowledge is tribal or locked away and the model has nowhere to look for facts. The model will faithfully learn any errors in the references. False positives remain, so the output is a list of candidates to check, not a verdict. A human, not the model, confirms the match with the expert.

Mini recipe

1. Find two write-ups: one with a "yes" verdict, one with a "no." Don't skip the negative one. 2. Drop them into the step 1 template: describe the task and the input and output formats. You get the discipline prompt (p_exp). 3. Run it on the same two examples: compare with the reference. If it matches – great. 4. If it doesn't – use the step 2 template: give the model its analysis, the reference and a note saying "this is wrong." Ask for a general fact-gathering prompt (p_evi). 5. Rebuild and repeat: facts from p_evi feed into p_exp. 1-2 rounds usually do it. 6. Set your own stopping criterion: e.g., "matches the expert 5 out of 5." 7. Run new cases as two requests: first p_evi plus the documents, then p_exp plus the finished dossier. Separate chats are better. 8. Eyeball the output: treat it as a list of candidates, not a verdict.

Examples

[BAD]: `Look at this supplier contract and tell me if I can sign it` [GOOD]: `You are given two reviews of supplier contracts. Example 1 – do not sign: the lawyer flagged an uncapped penalty and unilateral termination. Example 2 – OK to sign, because liability is capped and payment terms and deadlines are spelled out. Derive a general prompt that captures the lawyer's discipline and works on new contracts. Output only the prompt.` Next: run the prompt on the same two contracts. If the verdict doesn't match, give the model its own analysis and the lawyer's. Ask for a general fact-gathering prompt: which contract clauses, exhibits and internal policies to find before concluding. Run new contracts as two requests: dossier first, then the opinion.

In plain words

Ask a model point-blank "is something off here, or is it all clean?" and it'll hand you confident, middle-of-the-road garbage. By default the model is lazy: it has zero clue about your shop's unwritten rules, and it tries to reach a verdict before it has gathered the facts. What you get is beautiful, smoothly written nonsense that in real life can mean seven-figure fines or gaping security holes.

It's like hiring an intern and immediately handing them a contract audit. Instead of digging through the archive of past cases and writing down the criteria, they play expert on vibes and judge by gut feel and whatever the internet says. The words sound smart, but the review fails spectacularly, because a pro first methodically pulls out the evidence instead of shooting from the hip.

The LENS method cures this in two stages. First, the model is fed old expert cases so it can extract the unwritten rules and build the first prompt. Then, from its own mistakes, it writes a second prompt: exactly which facts to look for, and where in the source material, before drawing any conclusions. In production, this two-prompt scheme runs as a pipeline: hard-nosed evidence gathering first, and only then the final analysis.

The approach was tested on network protocols, but the pattern works for any complex review. You can use it the same way to check marketplace listings against FTC advertising rules, catch bugs in code, or screen for compliance. Anywhere the rules live in some lawyer's head rather than in a manual, LENS pulls the hidden expertise out into the open.

Stop torturing the model with naive asks like "check if everything's OK". Split fact gathering and the verdict into two separate runs. Two systematic steps instead of one blunt question turn a hallucination machine into a proper analyst who won't blow your budget on dumb fines.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on LENS: Reliable Expert Analysis With Prompts the Model Derives From Two Worked Cases and Its Own Failures arXiv:2610.22943

Sign in to apply this study New accounts get 50 free credits.