Thought Templates: A Library of Reusable Reasoning Patterns for LLMs

Stage 1: Build the template library (once)

• Take 10-50 sample questions with answers from your domain
• Ask an LLM to create reasoning templates: name + description + steps
• Save them into a "template library"

Stage 2: Apply (every request)

• Put into the prompt: the question + ALL templates + the documents
• The model picks the 2-4 templates it needs
• Applies them in sequence, linking the facts
• Returns an answer that names the templates it used

Stage 3: Improve the templates (periodically, optional)

• Collect the model's mistakes
• Ask another LLM to analyze what's wrong with the template
• Get feedback in natural language
• Update the weak templates

Task: Help a lawyer untangle a tricky housing case in Chicago. The client bought a condo in 2020 that turned out to be under shared ownership — and now a neighbor complains that the renovation violates his rights. You need to connect the state condominium statute + case law + Chicago's local ordinances.

Prompt:

Question: Does a neighbor have the right to block a renovation in a shared-ownership property if the work doesn't affect common areas but creates noise during the daytime?

Library of reasoning templates:

Template 1: Find the applicable legal provision
Description: Find the section of law that governs this specific situation
Steps: 
1. Identify the area of law (civil, housing, administrative)
2. Find the relevant statute or code
3. Pull the key terms out of the situation
4. Match them against the wording of the sections

Template 2: Link the provision to how courts apply it  
Description: Understand how the provision works in practice through court decisions
Steps:
1. Take the section of law you found
2. Find relevant court cases
3. Identify how courts interpret this provision
4. Determine whether there are exceptions or special conditions

Template 3: Apply local refinements
Description: Account for local rules that supplement state law
Steps:
1. Identify the jurisdiction (city, county)
2. Find local ordinances on the topic
3. Check that they don't conflict with state law
4. Add any extra requirements

Documents:
[Illinois Condominium Property Act]
[3 court decisions on shared ownership]
[City of Chicago ordinance on renovation work]

Task: Apply the relevant templates in sequence. For each step, state:
- Which template you're using (number + name)
- What you're doing at this step
- What the intermediate result is

Final answer: a short conclusion with justification.

Result: The model will show 3 steps with explicit references to the templates:

1. Template 1 → finds the section of the Condominium Property Act on the use of a residential unit
2. Template 2 → cites case No. 33-2451/2019, where the court separated "common property" from "private space"
3. Template 3 → applies Chicago's ordinance on "quiet hours"

The final answer comes out structured: "The neighbor CANNOT block the work but can demand that you stick to the permitted schedule. Basis: state statute + case law + Chicago ordinance."

LLM weakness: Models handle implicit multi-step logic poorly. When they need to chain fact A → inference B → fact C → final answer, they often "lose the thread" or skip intermediate steps. Long context makes it worse — the more documents, the harder it is to figure out HOW to connect them.

LLM strength: Models are excellent at following explicit, structured instructions. Give them a clear pattern like "first do X, then Y using the result of X" and they'll execute it precisely. The problem is that we usually DON'T give them such patterns — we just say "here are the documents, figure it out".

How the method leverages that strength: Thought Templates turn implicit reasoning into explicit reasoning. Instead of "sort it out yourself across 800 documents", you hand over ready patterns: "to find an author — template 1; to link to a place — template 2". The model still composes the chain itself, but follows a clear structure.

Prompt control levers:

1. Number of templates: Start with 5-10 core ones → add more as tasks get harder. The study showed that even 25% of the templates deliver 80% of the effect.
2. Composability: Break complex patterns into simple sub-templates. Instead of one "Full competitor analysis", make 3: "Find the product" + "Find the price" + "Compare with ours". That gives you flexibility.
3. Instruction explicitness: If you want to see the reasoning process, add "For each step, explain what you're doing". If you only need the answer — "Use the templates silently, give the final result".
4. Feedback loop: After using them, collect the errors → ask a separate LLM session: "Here's the template, here's the error, how do I improve it?" → update the template in the library.

Task: {description of a task that requires multi-step reasoning}

Reusable reasoning templates:

TEMPLATE_{number}: {Template name}
Description: {What this reasoning pattern does}
Use when: {Which situations it applies to}
Steps:
1. {First step}
2. {Second step}
3. {Third step}
Example: Question "{example}" → "{how to solve}"

[Repeat for every template in the library]

Documents/context:
{all relevant documents, data, facts}

Instructions:
1. Determine which templates from the library apply to the task
2. Apply them sequentially or in combination
3. For each template you use, state:
   - TEMPLATE_{number} | {Name}
   - What you're doing at this step
   - The intermediate result
4. Give the final answer with justification

Final answer: {answer format}

What to fill in:

• {description of a task} — your specific question
• {number} — the template's sequence number (1, 2, 3...)
• {Name}, {Description}, {Steps} — filled in from your template library
• {documents} — all the context needed to answer
• {answer format} — what you expect back (short conclusion, detailed analysis, list)

🚀 Quick start — paste into the chat:

I need to build a library of reasoning templates for tasks like: {describe your domain}.

Here are 3-5 examples of typical questions I deal with:
1. {example question 1}
2. {example question 2}
3. {example question 3}

Create 5-10 reusable reasoning templates for me in this format:
TEMPLATE_N: Name
Description: what it does
Use when: which situations
Steps: [numbered list]
Example: question → how to solve

Make the templates composable — 10 simple ones beat 3 complex ones.

The LLM will analyze your examples, pull out the recurring reasoning patterns (lookup by category, comparing two objects, checking a condition, etc.) and build a ready-to-use library. You get a structured set of templates you can put to work right away.

⚠️ Overhead on simple tasks: If a question takes one step ("Capital of France?"), loading 15 templates into the prompt is a waste of tokens. The method pays off on multi-hop reasoning, where you need to connect 3+ facts.

⚠️ You have to build the library: Before using it, you need to invest time in creating 10-50 templates. For a new domain you'll either start from scratch or adapt existing ones. You can delegate this to an LLM (give examples → ask for templates), but the upfront prep is unavoidable.

⚠️ Tokens spent on templates: Each template takes 50-150 tokens. A library of 30 templates = ~3000 tokens in every request. On short contexts that matters; on long-context (128k+) it's pocket change.

⚠️ Quality depends on the templates: If the templates are poorly designed (too abstract or too narrow), the effect is weak. You need to iterate — create → test → improve.

A team from KAIST, Amazon and the University of Minnesota tested the method on four multi-hop QA benchmarks: MuSiQue (general knowledge), CRAG (complex, unusual queries), FanOutQA (long Wikipedia articles) and Housing QA (legal questions about real estate). They ran it on top models: Claude Sonnet 4, Gemini 2.5 Flash, GPT-4.1 and the open-source DeepSeek-R1.

They compared against four baselines: (1) just the question with no context, (2) Chain-of-Thought ("think step by step"), (3) Corpus-in-Context (load all documents into the prompt), (4) CIC + CoT (the combo). Result: Thought Templates beat every option by 5-10 F1 points depending on the dataset. On MuSiQue the score went from 63.87 to 73.30; on CRAG from 17.32 to 30.08.

Surprising finding: Templates created for one model (GPT) work on others (Claude, Gemini, DeepSeek) with almost no loss in quality. That makes a template library a reusable asset, not tied to any particular model. What's more, even the top 25% of templates deliver competitive results — you don't need hundreds; 10-15 good ones are enough.

The logic behind the results: models already know how to reason step by step (CoT proves it), but they don't know WHICH steps to apply in multi-hop tasks. The templates supply the missing structure. And the harder the task (more steps, more heterogeneous documents), the bigger the effect — on Housing QA (legal queries with 3-5 steps) the gain was the largest.

Practical takeaway: If you work with long-context models and complex questions, investing 2-3 hours in building a template library pays off with a steady +10-15% in answer quality. The library carries over between models and tasks — it's not a one-off time sink.

💡 Adaptation for single-step tasks (simplified version):

For simpler tasks where multi-hop is overkill, you can use a mini version with 3-5 templates just for your narrow domain:

Task: {a question from a narrow domain — e.g., an SEO audit}

Mini template library (SEO only):
1. Technical analysis → check speed/mobile-friendliness/indexing
2. Content analysis → assess quality/uniqueness/structure
3. Backlink analysis → link profile/quality/anchor text

Site: {URL or data}

Apply ALL three templates in sequence and give a summary for each.

Here there's no template selection — the model applies all 3 like a checklist. It's simpler than full TOTAL but keeps the structure.

🔧 Technique: turn templates into "agents" → sharper specialization

In the original, templates are faceless patterns. But you can give them roles to amplify the effect:

Task: {startup analysis}

Team of template experts:

AGENT "Financial Analyst" (TEMPLATE_1):
Description: Evaluate unit economics and burn rate
Steps: ...

AGENT "Product Strategist" (TEMPLATE_2):
Description: Check product-market fit
Steps: ...

AGENT "Competitive Analyst" (TEMPLATE_3):
Description: Compare against similar products on the market
Steps: ...

Data: {the startup's pitch deck}

Task: Each agent gives its own conclusion. Then combine them into an overall verdict.

By calling templates "agents" with roles, you trigger the LLM's role-play mechanism — the model keeps each template more tightly focused on its own area.

💡 Extrapolation: combine with Tree-of-Thoughts to "branch" solutions

If a task allows several solution paths, you can combine Thought Templates with Tree-of-Thoughts:

Task: {a tough dilemma — e.g., where to put the budget: marketing vs product}

Step 1: Identify 2-3 applicable templates from the library
Step 2: For EACH template, generate 2 candidate solutions
Step 3: Score each candidate against the criteria {list}
Step 4: Pick the best path

Template library: [paste]
Data: [paste]

This builds a tree of options (template A → solutions A1, A2; template B → solutions B1, B2) and picks the best one. Useful when a single template doesn't give a clear-cut answer.

• "When Thoughts Meet Facts: Reusable Reasoning for Long-Context LMs"
• Soyeong Jeong, Taehee Jung, Sung Ju Hwang (KAIST), Joo-Kyung Kim (Amazon), Dongyeop Kang (University of Minnesota)
• Code: https://github.com/starsuzi/ToTAL
• The LOFT benchmark for long-context evaluation