TRACE: Splitting the LLM's Job Into "Extracting Facts" and "Making the Decision" With Explicit Rules

STEP 1 (LLM prompt): Extract canonical terms from the request text
                     using a fixed vocabulary → list of facts/symptoms + context

STEP 2 (rules, not the LLM): Apply explicit if-then rules to the extracted
                          facts → final decision

Step 1 needs an LLM (understanding conversational speech, synonyms, dialects). Step 2 is fixed logic that a human can apply directly or hand to an LLM as a separate simple prompt: "apply these rules to these facts."

Task: A sales team receives inquiries through a chatbot and needs to quickly tell whether this is a hot lead that should go straight to a rep, or an inquiry that can wait in the general queue.

Prompt (Step 1 — extraction):

Here's a message from a customer: "{inquiry_text}"

Extract only the facts that are explicitly mentioned in the text,
using this list of terms (if something similar is mentioned,
map it to the canonical form from the list):

- Budget: specified / not specified / "no budget"
- Decision timeline: "urgent/today/this week" / "within a month" / not specified
- Writer's role: owner/director / employee without decision rights / unclear
- Competitor mentioned: yes/no
- Ready to meet: today-tomorrow / this week / not ready

Don't conclude "hot lead or not." Just list the facts.

Prompt (Step 2 — rules, as a separate message or in the same chat):

Here are the extracted facts: {step 1 result}

Apply the rules:
- If "Role: owner/director" AND "Timeline: urgent" → HOT
- If "Budget: not specified" AND "Role: employee without decision rights" → NOT hot
- If "Ready to meet: today-tomorrow" → HOT, regardless of everything else
- Otherwise → queue for follow-up

Output the final verdict and state which rule fired.

Result: At the first step, the model produces a short list of facts with no interpretation, just recording what's explicitly written in the inquiry. At the second step, you get a verdict that names the specific rule that fired. If the result turns out wrong, you'll know exactly whether the error was in fact extraction (the model missed an important detail) or in the rules themselves (the rule is too narrow or outdated), and you can fix that specific piece without touching the rest.

LLMs struggle with tasks that require them to simultaneously understand conversational text and precisely apply a complex, multi-step decision criterion: they blend the two operations and start relying on their general intuition instead of your specific rules. Hence the instability: today the model reacts to one detail, tomorrow it misses the same detail in a similar case.

On the other hand, LLMs are excellent at translating conversational text into structured terms: recognizing synonyms, jargon, mixed languages, typos. That's exactly where the model's strength fully shines.

The method takes that strength (translating into terms) and separates it from the weakness (applying complex logic). You write the logic yourself as explicit rules: they don't "drift" from run to run, and you can read, edit and fix them surgically.

Control levers: - Vocabulary of terms — the more detailed and specific the list of synonyms, the more accurate the extraction. Expand it as you run into new phrasings that mean the same thing. - Rules — you can add or change them one at a time, without rewriting the whole extraction prompt. That's the method's main payoff: edits don't require re-validating the entire system. - Two separate requests instead of one — if you want to see the intermediate result (what exactly got extracted) for debugging, keep the steps separate. If the task is simple and you trust the model, you can ask it to do both steps in one prompt, but with an explicit requirement to list the facts first and then apply the rules.

STEP 1 — Fact extraction:
Here's the text: "{text}"

Extract facts from the text using this vocabulary of terms (map to the canonical form,
even if the text uses a different phrasing/synonym/jargon):
{list_of_terms_and_their_synonyms}

Don't draw a final conclusion. Only list which terms from the list appear in the text,
and what context is given (if applicable: {additional_context_e_g_age_timeline_role}).

---

STEP 2 — Rule application:
Here are the extracted facts: {step_1_result}

Apply these rules in order and state which one fired first:
{list_of_if_then_rules}

Output the final verdict: {category_1} or {category_2}.

🚀 Quick start — paste into the chat:

Here's the TRACE template — a method that separates "fact extraction" from "rule application"
for precise decisions based on clear criteria. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask which terms/synonyms matter for your task and exactly which rules you want to apply, because the method won't work without them: the vocabulary and the rules are your expertise, and nobody can make them up for you.

⚠️ You need criteria that can be formalized: the method only works where the decision can be spelled out as explicit if-then rules. For highly subjective tasks (rating the creativity of a text, the tone of an email), decomposition won't help, because there's no clear logic to pull out of the model.

⚠️ Extra work up front: you have to build the vocabulary and the rule list by hand once. Without that prep work, the method won't get off the ground; it's not a "one-line prompt."

⚠️ Costlier and a bit slower: two steps instead of one increase request cost and response time. If the task is simple and errors aren't critical, the overhead may not pay off.

⚠️ Extracting context doesn't always improve accuracy: in the study, separately extracting context (e.g., age, gestational term) didn't improve the result by itself, but it remained necessary to be able to verify whether a context-dependent rule fired correctly.

Auditable Emergency Triage for Maternal and Newborn Care in India. Shobhit Jagga, Aman Dalmia, Niharika Priyadarshini et al. (Noora Health, The Agency Fund). AAAI 2027.