Pseudocode and program-aided prompting: when code beats prose

What code prompting is

Code prompting covers two related moves. The first is writing the prompt as code: the instructions become pseudocode, if-else rules, a formula or a diagram instead of prose. The second is reasoning through code: the model writes a program, the program computes or searches, and the model reports the result. This is often called program-of-thought or program-aided prompting.

Both rest on the same observation. Models have seen huge amounts of code, and code has one reading where prose has several. "Analyze the market" can mean dozens of things, while competitors = analyze_competitors(product, region) names the input, the output and what the next step receives. A program that filters a JSON list or solves a set of equations also does not make arithmetic slips or lose track of a nested field the way token-by-token text can.

Why code works better than prose for some tasks

Three mechanisms appear across the studies.

• Explicit structure removes guessing. In prose, conditions blur: what matters most, where a branch ends, what "not" applies to. The ICA study at Airbnb rewrote real customer-support workflows from rich text into Intent-Context-Action pseudocode. On Mistral-7B, accuracy rose from 16% (rich text without chain-of-thought) to 70% (ICA with chain-of-thought) without fine-tuning, and to 85% with fine-tuning. Small models gained the most: +54% for Mistral-7B against +13% for a large model.
• Code does the part text does badly. Direct reading of long API responses topped out at 77% for GPT-4o in the tool-output study, and accuracy fell by up to 91% for Mistral-large as responses grew from 10K to 80K tokens. Asking for a Python function instead beat direct answers in 13 of 15 models, and adding the JSON schema gave +12%.
• Solvers can backtrack; text cannot. A model writing step by step cannot undo a wrong early choice. In the symbolic solver study, GPT-4o with chain-of-thought solved 11-12% of hard Zebra logic puzzles, while having it translate the conditions into code for a constraint solver solved 62-72%.

Variants that tested well

• Pseudocode prompts. CodeAgents frames a multi-step task as a function with named intermediate variables and a # comment before each step explaining why it is there. The comments act as built-in reasoning, and the variables keep step 3 tied to the output of step 1. The paper reports accuracy gains of up to 36 percentage points and large token savings in its multi-agent setting.
• If-else decision rules. Intent, Context, Action: state the goal, list the factors, order the checks by priority, and give every branch a concrete action.
• Write code instead of reading data. For filtering or aggregating JSON, ask for a function that takes the data and returns the answer, with the schema included (tool-output study). If the response is too long, a trimmed sample with one object of each type is enough.
• Declarative code for solvers. Show one toy example in which each sentence of the problem is copied into a comment followed by its code, then give the real problem (symbolic solvers). Use SymPy for equations and a constraint library for logic.
• Cross-check words against code. CoT-PoT solves the problem once in prose and once in code. The two fail in different ways, so agreement is strong evidence: when they agreed, the answer was correct with probability of at least 0.991 for all six models tested, and 78.6% of problems were settled with exactly two requests.
• Other formal notations. A Mermaid diagram of the solution let a small model beat a bigger one, 45.2% against 40.4%. A utility formula that scores each criterion separately and multiplies replaces "profitable and reliable". Math symbols such as ∈ and ¬ cut 62-81% of tokens on filtering tasks. Pseudocode for an internal debate between two agents gave +7.95% accuracy with 58% fewer tokens than repeated calls.

When to use it and when prose wins

Code prompting pays off on multi-step workflows with branching rules, on structured data, on calculations with many interdependent values, and on search problems with a large space of combinations. It also tends to save tokens because pseudocode is denser than prose.

The limits are as clearly documented:

• Simple problems. On GSM8K arithmetic, chain-of-thought beat code, 94.6% against 92.6% in the solver study. When a rule is hidden in the meaning of words, it is easy to drop during translation into code, and the solver cannot recover it.
• Single lookups. For extracting one value from JSON, a direct answer was sometimes more accurate. 10 out of 15 models showed this in the tool-output tests.
• Model size. Symbolic notation is not universal. In the MetaGlyph tests, mid-size models of 7-12B parameters showed 0% symbol understanding, and the intersection operator worked poorly even on a top model (21.4%). UtilityMax needs top-tier models that give usable numeric estimates.
• Subjective tasks. Rigid if-else trees and formulas do not suit creative writing or judgments without clear criteria.
• Execution. Program-of-thought is most reliable when the code actually runs, in a code interpreter or locally. Code the model only writes but never runs is a weaker check.

A workable rule: use prose chain-of-thought for short or meaning-heavy problems, and switch to code when the task has many conditions, large data or heavy computation. For high-stakes numbers, run both and compare. Related pages: chain-of-thought prompting, structured output and self-consistency.

def create_marketing_strategy(product='meditation app', audience='city dwellers aged 25-40'):
    # Step 1: Find competitors and understand where our product is stronger
    competitors = analyze_competitors(product)
    # Step 2: Find the audience's unmet needs based on the competitor analysis
    pain_points = find_gaps(audience, competitors)
    # Step 3: Create 3 key brand messages for the needs we found
    brand_messages = create_messages(pain_points, count=3)
    # Step 4: Put it into a table with promotion channels and key points
    final_answer(format_as_table(brand_messages))

Here is a JSON response from {source} and its schema.
Write a Python function that:
1. Takes the entire JSON as a dictionary (parameter data)
2. {task description: filters/extracts/counts}
3. Returns the result as a string
Code requirements:
- Starts with "def "
- Checks that keys exist and values aren't None
- Compares strings in lowercase
- Cleans numeric strings (strips "$", spaces) before converting
- Returns only the result, with no extra text
JSON schema:
{schema}
JSON:
{data}
Python function:

Solve the problem two ways.
{problem_or_data}
**Method 1 — Reasoning in words:**
Go step by step, explaining each one. Arrive at a specific answer.
**Method 2 — Python code:**
Write and run code that solves the same problem. Show the result.
After both methods:
- If the answers match → "Result confirmed: [answer]"
- If they don't match → find the discrepancy and pinpoint the error