NLT (Natural Language Tools): Calling Tools in Plain Language Instead of JSON

The task: Customer support for a Shopify store. A customer writes: "I want to return my sneakers and find out where my order with the phone is. Can I get on a call with an agent?"

Prompt (Selector):

You are the Shop Support assistant, helping with customer requests.

Your job is to determine whether the following topics are mentioned or whether action is needed.

Available tools:

Product return (the customer wants to return a purchase, ask about returns or an exchange)
Order status (where the package is, when it will arrive, tracking number)
Contact an agent (the customer explicitly asks to talk to a human, call or message someone)

First, think about which tools are relevant. Then output the name of each tool and YES or NO. At the end, write "Assessment complete."

Format:

Thinking: {your reasoning}

Product return – YES/NO
Order status – YES/NO
Contact an agent – YES/NO

Assessment complete.

Result:

The model will output a structured list: "Thinking: the customer wants to return sneakers, so that's a return. They're asking about the phone, so that's order status. They want a call, so that's contacting an agent. Product return – YES, Order status – YES, Contact an agent – YES, Assessment complete." The parser will run all three tools in parallel. Then, at the Response step, the model will receive the return details, the order status and the agent's number, and give the customer a complete answer.

JSON hurts because of task interference. When the model has to simultaneously (1) understand the request, (2) choose tools from a list, (3) stick to JSON syntax and (4) generate an answer, every task suffers. The model's probability mass gets spread thin. The Tam et al. study showed -27.3 pp of accuracy just from requiring JSON instead of natural language. The stricter the format, the bigger the drop.

Models are strong at natural language; that's what they were trained on. Most RLHF and SFT goes into ordinary speech, not JSON. Open-weight models are especially weak at structured formats (too little data) but good at text. NLT plays to that strength: YES/NO are just words, and the model generates them easily and accurately.

Splitting the tasks eliminates interference. The Selector does one thing: picks tools. The Response does one thing: answers the user. Each model focuses on its role instead of juggling. Plus, the list of all tools lands at the end of the output (the most recency-favored spot) rather than at the start of the context as with JSON, so positional bias works in your favor.

Control knobs:

• Thinking: Drop it for simple tasks → save tokens. Add detail for debugging → see the model's logic.
• Output format: Swap YES/NO for a 1-5 scale → get a confidence score. Or "YES (high confidence)" / "MAYBE" / "NO".
• "Assessment complete": A marker for the parser. You can swap it for a different one or drop it if your parser works differently.
• Number of tools: Fewer tools = shorter output, faster parsing. More tools = broader coverage, but the model thinks longer.

You are the {agent_name} assistant, helping with {work_context}.

Your job is to determine whether the following topics are mentioned or whether action is needed.

Available tools:

{Tool_1} ({when to use})
{Tool_2} ({when to use})
{Tool_3} ({when to use})

First, think about which tools are relevant. Then output the name of each tool and YES or NO. At the end, write "Assessment complete."

Format:

Thinking: {your reasoning}

{Tool_1} – YES/NO
{Tool_2} – YES/NO
{Tool_3} – YES/NO

Assessment complete.

Fill in:

• {agent_name}: the AI assistant's name (Shop Support, Career Bot, Research Assistant)
• {work_context}: customer support, career consulting, information search
• {Tool_N}: specific tools with a description of when to call them

🚀 Quick start: paste into the chat:

Here's an NLT template for calling tools. Adapt it to my task: [your task].
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask what tools you have and when to call them, because the method needs a clear description of each one. It'll take the "Thinking → YES/NO list → Assessment complete" pattern and plug in your details.

⚠️ Latency overhead: There are always at least two calls to the model (Selector + Response), even when no tools are needed. If tools aren't called 80% of the time, you're paying in extra wait time. JSON can answer right away; NLT can't.

⚠️ Selection only, not parameters: The study is about choosing tools (which ones to call), not passing arguments (e.g., "find tickets for {date}", where the date is a parameter). NLT can be extended to handle parameters, but that's a separate problem.

⚠️ Architectural complexity: You have to manage two prompts, a parser, and context handoff between steps. Tools are described right in the prompt, so changing one tool = rewriting the prompt. With JSON, tools live in the API and are easier to change.

⚠️ Single-turn: The study used single requests. In multi-turn conversations you need to pass context between steps, which makes the system more complex.

The team took 10 models (GPT-5, Claude Sonnet 4, Gemini 2.5 Pro, DeepSeek-V3, Llama 4, Qwen, Kimi) and ran them through 6,400 trials. Two scenarios: customer support for a music club ("Alex", 7 tools) and a mental health bot ("Sage", 8 tools). Each scenario had 16 requests: some needing no tools, some needing one, and some requiring several in parallel.

They compared two approaches: 1) JSON tool calling via the models' native APIs, and 2) NLT with natural language. The tool descriptions were identical; only the output format differed. To rule out a "good prompt" effect, they built a second set by having GPT-5-nano rewrite all the prompts (the perturbed version). Each request was run 5 times to account for variance.

The result is unambiguous: NLT wins across the board. But the breakdown is interesting: open-weight models gain +26.1 pp (58.7% → 84.8%), while flagship closed-weight models gain only +10.6 pp (79.6% → 90.2%). Why? OpenAI, Anthropic and Google pour resources into RLHF specifically for JSON tool calling. Open-weight models get less of that training, but they're good at natural language, and NLT takes advantage of that.

The most dramatic example: DeepSeek V3 jumped from 78.4% accuracy with high variance (SD = 15.2%) to 94.7% accuracy with minimal variance (SD = 4.0%). From middle of the pack to the top. Prompt engineering matters too: non-perturbed prompts give +21.2 pp, perturbed ones +15.4 pp, but the NLT effect holds in both cases.

Tokens: NLT uses -31.4% tokens overall (1319 → 905). Input shrinks by 47.4% (no JSON boilerplate), output grows by 37.9% (every answer lists all the tools, even the NOs). A net saving.

Context: The authors built a prompt for customer support at the music club "Yes! Music" with an agent named Alex. 7 tools: Website Information, Talk to a Human, Past Purchases, and so on.

You are an assistant to Messenger, a chatbot designed to help with customer service requests.

Your mission is to identify if any of the following topics have been brought up or if an action is relevant.

Your list of possible options include the following:

Website Information (questions about your website: where to purchase tickets or merch, order tracking, policies, or how to use the site)

Talk to a Human (explicit requests to reach a live agent by chat/phone/email)

Past Purchases (questions about previous orders, receipts, warranties, returns, or order lookups)

Your output should begin by thinking whether or not any of these are brought up or if they are relevant. Then, include the name of every topic, followed by either yes or no. You should end with "Assessment finished."

It should always be in the following format:

Thinking: (insert thinking)

Website Information – YES/NO

Talk to a Human – YES/NO

Past Purchases – YES/NO

Assessment finished.

💡 Adaptation: Confidence scoring instead of a binary choice

Swap YES/NO for a 1-5 scale or LOW/MEDIUM/HIGH to capture the model's confidence:

Website Information – CONFIDENCE: HIGH
Talk to a Human – CONFIDENCE: MEDIUM
Past Purchases – CONFIDENCE: LOW

The parser can apply a threshold (e.g., only MEDIUM and above) or pass the confidence downstream for prioritization. Handy when a request is ambiguous: you can see where the model is unsure.

🔧 Technique: Debug mode via explicit output at each step

Add an instruction to print the intermediate steps for debugging:

For each tool, explain in one sentence why it's YES or NO:

Website Information – YES (the customer asked where to buy merch)
Talk to a Human – NO (didn't ask to reach an agent)
Past Purchases – YES (mentioned a previous order)

It's like print() in code: you can see the model's logic. Remove it in production (extra tokens), use it while tuning the system.

💡 Combo: NLT + Chain-of-Thought in the Thinking section

Beef up the Thinking block with explicit step-by-step reasoning:

Thinking:
Step 1: The customer mentioned "return my sneakers", so that's a product return.
Step 2: They asked "where my order with the phone is", so order status is needed.
Step 3: They wrote "can I get on a call", an explicit request to contact an agent.
Conclusion: all three tools are needed.

Product return – YES
Order status – YES
Contact an agent – YES

Assessment complete.

Chain-of-Thought in Thinking + NLT's structured output = a power combo for tricky cases.

Natural Language Tools: A Natural Language Approach to Tool Calling In Large Language Agents by Reid T. Johnson, Michelle D. Pain, Jordan D. West (PokketCoach, 2025)