Apply to my task
arXiv:2507.14447 ★ 91 Jul 19, 2025

Routine: how a step-by-step plan lifts GPT-4o tool-calling accuracy from 41% to 96%

Key result

41% → 96% tool-calling accuracy (GPT-4o, one corporate HR agent scenario): the difference between "figure out what to do yourself" and "here's your plan." Routine helps execute complex multi-step tasks with fewer skipped steps, less freelancing and less lost context. The trick: you move the planning outside the model. You lay out the task step by step right in the prompt and specify a concrete action for each step. The model no longer spends resources on "what do I do next?" and focuses purely on execution. From 41.1% to 96.3% on GPT-4o with an expert-written plan in the prompt.

Routine: A Structural Planning Framework forLLMAgent System in Enterprise

Key Aspects of the Study:

The study presents "Routine," a framework for giving an LLM a clear, structured, step-by-step plan for complex tasks, especially ones that require several tools (functions). Instead of leaving the model to guess what to do, you give it detailed "assembly instructions" where every step is spelled out. This sharply cuts errors and makes the LLM's behavior more predictable.

Key Result: Giving the LLM a clear, step-by-step plan (a "Routine", drafted by experts) raised GPT-4o's tool-calling accuracy in a corporate HR agent scenario from 41.1% to 96.3% (each step's tool call checked against a reference; Qwen3-14B went from 32.6% to 83.3%). The test covered one enterprise scenario with 25 tools.


The Core Idea of the Method:

For an everyday user, the "Routine" method means you stop handing the LLM complex tasks in one broad sentence and start acting as a "project manager" who writes a detailed spec for the model.

Imagine you need the LLM to do multi-stage work, for example, analyze competitors and build a content plan from that analysis.

  • Bad approach (no Routine): "Analyze my competitor X's social media and suggest a content plan for my company Y for next week." The model may miss something, misread the priorities or deliver the result in an awkward format.

  • Good approach (with Routine principles): You break the big task into logical steps yourself and lay them out as a clear plan right in the prompt. For each step, you give its number, name, a short description of the goal and, most importantly, which "tool" (imaginary or real) the model should use.

The method for the user (our chat adaptation; the paper itself tested agents calling real tools through MCP servers, not chat prompts): 1. Decomposition: Break your complex task into 3-7 logical, sequential steps. 2. Structuring: Lay these steps out as a numbered list. Use Markdown for clarity (### Step 1:). 3. Explicit instructions: For each step, specify: * Name: Short and clear (for example, "Competitor tone analysis"). * Description: What exactly needs to be done at this step. * "Tool": Specify which action to perform (for example, tool: analyze_text, tool: generate_ideas, tool: format_as_table). In the paper each step names a real tool; in chat, an imaginary label is our adaptation and may help the model focus on a specific operation (not tested). 4. Passing context: State explicitly that the results of the previous step should be used in the next one.

This approach turns the LLM from a "creative genius" into an "executive assistant" that follows your plan. In the paper this made multi-step tool calling far more reliable; for ordinary chat tasks we'd expect a similar but untested benefit.


Practical Applicability Analysis:

  • Direct applicability: Users can immediately start using an adapted version of "Routine" in their prompts in any chatbot (ChatGPT, Claude, etc.). All it takes is structuring your request as a step-by-step plan with Markdown, as shown in the examples below (our adaptation; the paper's numbers come from an agent calling real tools). No special tools required.

  • Conceptual value: The study gives the user an invaluable insight: an LLM works best when the "cognitive load" of planning is taken off its shoulders. Instead of hoping the model will figure out the right sequence of actions on its own, the user learns to create that sequence themselves. This shifts prompting from "making a request" to "managing a process."

  • Adaptation potential: The method adapts easily. Instead of the complex system with separate modules described in the paper, the user builds a "system in one prompt." Their brain plays the "Planner," and the LLM plays the "Executor." A prompt written using the "Routine" method becomes the link that makes the collaboration reliable.


Practical Example:

Our illustration, not an example from the paper.

You are an experienced marketing analyst. Your job is to help me develop a content strategy for my travel blog based on an analysis of a popular competitor.

**Context:**
*   **My blog:** "Travel Light," focused on budget trips and travel hacks.
*   **Competitor:** The popular travel blog "World in Your Hand," known for its colorful photos and luxury tours.
*   **Goal:** Find my own niche and set myself apart from the competitor by offering unique content.

**Carry out the following instructions (Routine) step by step:**

### **Routine: Content Strategy Development**

**Step 1: Competitor content analysis**
*   **Description:** Analyze the hypothetical last 10 posts on the "World in Your Hand" blog. Identify the 3-4 main topics they cover, their tone of voice and their main content formats (for example, photo stories, longreads, videos).
*   **Tool:** `analyze_competitor_content`

**Step 2: Identifying weaknesses and opportunities**
*   **Description:** Based on the analysis from Step 1, determine which topics or formats the competitor does NOT use. Find the "blind spots" my "Travel Light" blog can claim. For example, they don't write about budget routes or saving money on trips.
*   **Tool:** `find_gaps_and_opportunities`

**Step 3: Generating content ideas**
*   **Description:** Using the opportunities from Step 2 and my blog's focus, generate 5 specific post ideas. Each idea should be unique and highlight my blog's strengths (affordability, travel hacks).
*   **Tool:** `generate_content_ideas`

**Step 4: Building the content plan**
*   **Description:** Present the 3 best ideas from Step 3 as a weekly content plan table (Mon, Wed, Fri). For each idea, give a headline, a short description and a format (article, Stories series, short video).
*   **Tool:** `create_content_plan_table`

**Step 5: Final summary**
*   **Description:** Write a short summary (2-3 sentences) wrapping up the proposed strategy and its key advantage.
*   **Tool:** `summarize_strategy`

Why It Works:

This prompt is effective because it directly applies the principles from the "Routine" study:

  1. Task decomposition: The complex task "develop a strategy" is broken into 5 clear, sequential steps. This lowers the chance the model misses something or lumps everything together.
  2. Clear structure: Headings (###), numbering and lists (*) create a rigid structure that the LLM easily parses and follows. The model understands this isn't just text but an action plan.
  3. Focus through "Tools": Specifying Tool: ... for each step, even if the tool is imaginary, nudges the model to concentrate on one specific micro-task (analysis, generation, formatting). This mimics the per-step tool names in the paper's agent setup; whether imaginary labels help in chat was not tested.
  4. Controlled data flow: Each step implicitly builds on the result of the previous one, creating a logical chain. The model doesn't try to do everything at once but processes the information in sequence, which should reduce (not eliminate) errors and "hallucinations".

Another Practical Example

Also our illustration.

You are a personal event-planning assistant. Help me organize a birthday celebration for a friend.

**Context:**
*   **Friend's name:** Alex.
*   **Number of guests:** 10-12 people.
*   **Budget:** $600.
*   **Alex's interests:** Board games, craft beer, science fiction.
*   **Date:** Next Saturday.

**Please carry out the following plan (Routine) to organize the event:**

### **Routine: Birthday Plan**

**Step 1: Brainstorm the event format**
*   **Description:** Given Alex's interests and the number of guests, suggest 3 different birthday formats (for example, an evening at a board game café, a trip to a themed bar, a house party). Briefly describe the pros and cons of each.
*   **Tool:** `brainstorm_formats`

**Step 2: Choose and detail the venue**
*   **Description:** Based on the formats suggested in Step 1, pick the best one for the budget and interests. Find 2-3 hypothetical places in the city that fit this format and compare them on price and atmosphere.
*   **Tool:** `select_and_detail_location`

**Step 3: Plan the menu and shopping**
*   **Description:** For the chosen venue, suggest menu options (snacks, drinks) that fit within the remaining budget. Make a list of the food and items to buy.
*   **Tool:** `plan_menu_and_shopping`

**Step 4: Entertainment plan**
*   **Description:** Put together an hour-by-hour entertainment plan for 4 hours, based on Alex's interests. Include specific board games, sci-fi music and maybe a short trivia quiz.
*   **Tool:** `create_entertainment_schedule`

**Step 5: Final checklist**
*   **Description:** Pull all the key items (venue, time, shopping list, plan) into one final checklist for a readiness check the day before the event.
*   **Tool:** `generate_final_checklist`

Why This Example Works.

This example works for the same reasons as the previous one, but the following mechanics matter especially here:

  1. Managing constraints: The budget, the number of guests and the interests are hard constraints. Splitting the work into steps lets the model apply these constraints one by one at each stage instead of trying to hold them all "in its head" at once. For example, the budget first shapes the choice of format (Step 1), then the venue (Step 2) and finally the menu (Step 3).
  2. Preventing premature conclusions: Without the "Routine" structure, the model might immediately suggest a specific café without considering alternatives. The step-by-step plan makes it brainstorm first (Step 1) and only then choose (Step 2), which leads to a more thoughtful, higher-quality result.
  3. Creating artifacts: Each step produces a concrete "artifact" (a list of formats, a list of venues, a menu). The final step (generate_final_checklist) uses all the previous artifacts to create a useful final document. This mimics a workflow where the output of one stage becomes the input for the next, ensuring the answer is consistent and complete.

Usefulness Score: 90

02

Main Evaluation Criteria

  • A. Relevance to prompting techniques: Very high. The study proposes "Routine," essentially an advanced, strictly structured prompt template for complex tasks.
  • B. Better conversational answers: Very high. It demonstrates a massive accuracy gain (from 41% to 96% for GPT-4o on tool calling in one enterprise HR scenario), which points to more reliable answers in agent scenarios.
  • C. Direct practical applicability: Medium-high. Although the full framework with separate modules and fine-tuning isn't available to everyday users, the core principle, building a step-by-step, structured plan (a "Routine") inside the prompt, is completely applicable and extremely useful.
  • D. Conceptual value: Very high. The study brilliantly illustrates the "mental model" of an LLM as an executor rather than a thinker. It suggests that reducing the model's planning load through explicit, step-by-step instructions is a major lever for reliability. It reveals important limitations: branches hurt smaller models (Qwen2.5-7B: 56.3% without vs 49.7% with branches; GPT-4o barely affected), and an extra, irrelevant Routine in the prompt dropped GPT-4o from 96.3% to 76.6%.
  • E. New useful practice (clustering): The paper falls into several key clusters at once:
    • #1 (Phrasing techniques): "Routine" is an advanced form of decomposition and step-by-step thinking.
    • #3 (Structure optimization): The whole point of the paper is creating the optimal prompt structure for complex tasks.
    • #7 (Reliability and stability): The main goal and outcome is a sharp increase in reliability and fewer errors.
  • Practicality checklist (+15 points): Yes, the paper provides ready-made constructions, shows how to structure requests and reveals non-obvious LLM behavior, which leads directly to higher accuracy.
03

Numeric Usefulness Score

Arguments for a high score: The study offers not just a tip but a whole methodology, "Routine," that can be adapted to everyday tasks. It gives the user an extremely powerful tool for steering an LLM in complex scenarios: turning one big, fuzzy request into a series of small, clear steps. The results (tool-calling accuracy rising from 41% to 96% in the paper's HR scenario) show that this approach works in practice, at least for structured agent workflows. The conceptual value is huge: it teaches the user to think like a "manager" for an LLM "executor."

Counterarguments (why not 100): 1. Focus on systems, not chat: A large part of the paper is devoted to agent system architecture, model fine-tuning and data distillation. These aspects have no direct practical use for an everyday user working in a chatbot window and may confuse them. 2. Adaptation required: The user has to figure out how to adapt the "Routine" idea to a single prompt, dropping the system details (MCP servers, memory, etc.). That takes some abstract thinking, unlike techniques such as "add the phrase 'think step by step'."

The final score of 90/100 reflects the huge practical and conceptual value of the core idea for advanced users who are ready to adapt a systems approach to their prompts.


Research digest

Key result

41% → 96% tool-calling accuracy (GPT-4o, one corporate HR agent scenario): the difference between "figure out what to do yourself" and "here's your plan." Routine helps execute complex multi-step tasks with fewer skipped steps, less freelancing and less lost context. The trick: you move the planning outside the model. You lay out the task step by step right in the prompt and specify a concrete action for each step. The model no longer spends resources on "what do I do next?" and focuses purely on execution. From 41.1% to 96.3% on GPT-4o with an expert-written plan in the prompt.

Principle

A standard prompt puts the model in a "plan and execute at the same time" position. It guesses the sequence, keeps the constraints in mind, watches the format, and somewhere along the way loses the thread. Routine splits the roles: you're the planner, the model is the executor. Each step gets a name, a description and an action type: "analyze," "find gaps," "generate ideas," "format as a table." The output of step 1 becomes the input for step 2. The model rides the rails instead of laying the track.

Why it works

An LLM struggles to hold "what to do," "in what order" and "what data is already available" all at once. It's like asking a cook to invent a recipe, chop vegetables and watch the stove at the same time: something's going to burn. Routine takes the "planning load" off the model: it gets a ready-made route and spends all its resources just moving along it. The study tested this on real function-calling tasks in one corporate HR scenario. Without a plan, GPT-4o got 41% of the tool-calling steps right. With Routine, 96%. Not because the model got smarter, but because it stopped doing two things at once.

When to apply

Tasks with three or more steps, especially when you need to collect data, process it and format the result in sequence. Tested in the paper only on agent tool calling; in chat we'd expect it to help with: competitor analysis, project planning, multi-stage research, automating repetitive work routines, preparing reports and strategies. Keep only the relevant plan in the prompt: an extra, irrelevant one cut GPT-4o's accuracy from 96% to 77%. NOT a fit for simple one-step requests, where structure just clutters the prompt and slows down the answer.

Mini recipe

1. Break the task into 3-7 steps. Each step is one specific action. Not "analyze and suggest" but first "analyze," then "suggest." 2. Lay it out as a numbered plan with headings, for example `### Step 1: Competitor analysis`. Markdown helps the model read it as an instruction rather than plain text. 3. For each step, specify three things: what exactly to do, which concrete action to apply ("find gaps," "generate 5 ideas," "format as a table"), and what counts as the step's output. 4. Pass data explicitly: write "using the results of step 1". Don't count on the model figuring out where to get the information on its own. 5. End with a final wrap-up step: a 2-3 sentence summary or a checklist. This makes the model pull everything together instead of cutting off mid-thought.

Examples

[BAD]: `Analyze a competitor's social media and make a one-week content plan for my travel blog` [GOOD]: `You are a marketing analyst. Carry out the plan step by step: ### Step 1: Competitor analysis Analyze the last 10 posts on the "World in Your Hand" blog. Identify 3-4 main topics and the tone. ### Step 2: Finding gaps Using the results of step 1, find topics the competitor ignores, especially around budget travel. ### Step 3: Content ideas From the gaps in step 2, generate 5 specific ideas for the "Travel Light" blog. ### Step 4: Content plan Take the best 3 ideas from step 3 and lay them out in a Mon/Wed/Fri table: headline, description, post format. ### Step 5: Wrap-up Write 2-3 sentences on the key way this strategy differs from the competitor's.`

In plain words

Corporate AI agents today are mostly chaotic chatbots that try to solve the task on the fly, relying on the model's raw logic. The problem is that in business, "we'll see how it goes" logic doesn't fly: you need strict procedures and predictability. The Routine study proposes moving the structure of classic software into the models' brains. Instead of making the LLM reinvent the wheel every time, you give it a structural framework where every step isn't a random thought but a clearly defined block of business logic.

It's the difference between a self-taught intern and a seasoned employee with a thick binder of job instructions. The intern may be a genius, but they'll inevitably forget to check stock in the warehouse or botch a return because they "just didn't think of it." Routine turns the model into that methodical employee: it has an action algorithm to follow, yet inside each step it keeps its flexibility and smarts. Technically it's still AI, but it runs far more predictably.

The method rests on an expert-written plan: numbered steps, each with a name, a description and the tool to call, plus explicit passing of results from one step to the next. The agent works through the plan step by step instead of improvising its own route. That cuts errors a lot but doesn't remove them: smaller models still made mistakes, and an extra, irrelevant plan in the prompt pulled GPT-4o's accuracy down from 96% to 77%.

The authors tested this on one corporate HR agent scenario, but the idea is worth trying more widely: from writing code to planning complex trips. Anywhere there's a repeating process and the cost of a mistake is above zero, structural planning is likely to help. It's the shift from "talk to me" to "run the process," where the AI becomes not just a conversation partner but a full-fledged execution engine.

Bottom line: the era of "just write a good prompt" is ending, and the era of agent architecture is beginning. If you want AI to actually work in business rather than just entertain your coworkers, don't rely on free-flowing thought. Spell out the structure with something like Routine and keep the model within the boundaries of the process. Whoever is first to pack the chaos of models onto structural rails gets working automation, while everyone else keeps complaining that the AI mixed everything up again.

AI lab assistant

Apply this study to your task

Describe what you're working on. AI takes the method and the measured results from this paper and adapts them to your case, or writes a ready prompt built on them.

Based on Routine: how a step-by-step plan lifts GPT-4o tool-calling accuracy from 41% to 96% arXiv:2507.14447

Sign in to apply this study New accounts get 50 free credits.

Get the week’s best papers by email

One email on Sundays with the highest-rated new techniques. Unsubscribe with one click.