TL;DR
FlashScript is a set of instructions a coding agent follows to turn one successful run of a recurring task into a neuro-symbolic program (NSP). That’s a script where actions and calculations are written as code, the values specific to a given run are pulled out into parameters, and the model is called only where judgment is needed. Each such call is typed: llm<T> returns a string, a yes/no, or a choice from a list. It sits at a fixed point in the code.
A regular agent reinvents the path every time. It solves the same task once with an extra chart, once with a different column name. Even a “skill” (a text instruction distilled from a past run) reduces the variance but doesn’t eliminate it: the model still interprets the instruction and does the math “in its head.” When up to ten subtasks were glued into one request, the agent with a skill mixed them up. Accuracy dropped to 23%, and it added charts nobody asked for.
The method works in three steps. First, the code is written from a plan: identify the template variables and decide what can be written as a rule. Then every spot that needs the model is assigned to one of three layers: input, computation, or output. Finally, model calls are cut back with three techniques: cache, turn judgments into rules, and combine questions into one call. The test result: 100% accuracy, identical tool calls on every run, roughly 70x fewer input tokens, and three times faster.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 0: Work through the task with an agent once, until you’re happy with the result (regular chat/agent)
STEP 1: PLAN → which values change from run to run (parameters),
which decisions can be written as rules, where AI is needed
STEP 2: TAG THE CALLS BY LAYER:
• INPUT (text → typed value: “LA” → the “Los Angeles” sheet)
• COMPUTATION (a semantic decision: “is this a scammer?”)
• OUTPUT (nice wording for something the code already computed)
STEP 3: CUT BACK model calls:
• cache with a check (guard) → call only if the check fails
• codification → a judgment becomes a rule (regex, thresholds)
• clubbing → independent questions in a single call
STEP 4: The code is written by a coding agent in one request: request template + trajectory + tool descriptions
STEP 5: Verify separately: code with regular tests, LLM calls with repeated runs and contracts
Example in Practice
Task: Every Monday, a marketplace manager asks an agent to go through the export of customer reviews for their Amazon store. It needs to compute the average rating for each SKU, flag reviews about defects and returns, and then write a summary for the owner. Once, they worked through this with the agent until the result was right: fixed the table format, explained what counts as a “defect.” Now they want it to come out exactly the same every week, without paying thousands of tokens for a rerun of the same reasoning.
Prompt (given to Claude Code along with the trajectory file):
You are an engineer who turns a successful agent run into a repeatable program.
I need a Python program where actions and calculations are code, and the model is called
only for judgments, via llm<T>(question, data) -> T.
<Inputs>
<RequestTemplate>
Go through the review export {csv_file} for the week {start_date}–{end_date}.
Compute the average rating for each SKU, flag reviews about defects or returns,
and write a summary for the owner of the store {store_name}.
</RequestTemplate>
<ReferenceRun>
The file reviews_run.md is the full log of a successful run: reasoning, actions, my corrections.
</ReferenceRun>
<Tools>
Read CSV with pandas; write Excel with openpyxl; call the model through the llm function.
</Tools>
</Inputs>
<Plan>
Write a plan first, code after it:
1. List what changes from run to run → parameters.
2. List the decisions from the run. For each one, decide: rule, cache, or model call.
3. Find shortcuts and accidental assumptions in the run. Don’t carry them into the program
unless they follow from the request template.
</Plan>
<Layers>
Assign the llm<T> calls to layers:
- Input: turning free text into a typed value
(which column is the “rating,” even if it’s named “Score”).
- Computation: semantic decisions that can’t be written as a rule (is this review about a defect?).
- Output: a written summary of numbers the code has already computed.
</Layers>
<Reduction>
- Cache: store values from the run in a dictionary; call llm only if the check fails.
- Codification: turn anything repetitive into a rule (keywords, thresholds).
The model must not do counting or summing.
- Clubbing: independent questions about one review go into a single call.
</Reduction>
<Output>
The finished program with the parameters from step 1, a list of all llm calls
(layer, type T, input), and 3 tests on new data.
</Output>
Result: The agent will first produce a plan: the parameters, the run’s decisions, and what happens to each (rule, cache, call). Then it writes a script where computing ratings and formatting the table are done in code. The model is called at a few points: matching column names, deciding borderline reviews that the rules didn’t catch, and writing the summary from numbers already computed. You also get a table of all LLM calls by layer and a set of tests. Rerunning it the next week gives the same format and the same steps; only the wording of the text summary changes.
Why It Works
The LLM’s weakness. When an agent solves a task from scratch every time, it picks a new path each time, and at a temperature above zero, the path drifts. The model is bad at doing math “in its head”: in the paper, it added up daily values in its reasoning and got them wrong. A “skill” from a past run doesn’t fully help. The model still decides on its own how to execute it. It can also inherit an accidental shortcut from the reference run. For example, sums were compared instead of percentages, both approaches gave the right answer, and the skill locked in the wrong one.
The LLM’s strength. A model is good at narrow semantic tasks: understanding that “LA” means Los Angeles, deciding whether a listing looks suspicious, presenting facts nicely. A coding agent is also pretty good at rewriting a run as code. So the method gives the model exactly those tasks and leaves everything else to code.
How the method exploits this. All actions are fixed in code, and variance remains only inside the typed calls. The answer type (yes/no, a choice) limits the model’s freedom. The three techniques (cache, rule, clubbing) cut the number of calls to it, and with them the randomness.
Control levers: - What you give the agent as input. The request template alone already generalizes well. Template plus trajectory gives fewer calls and lets you check the run’s decisions. A trajectory alone without the template is too specific. - What to push into code. The more decisions are written as rules, the more stable and cheaper it is, but the fewer input variations the program can handle. The more decisions stay with the model, the more flexible but scattered it is. - User-facing parameters. A criterion (“what counts as a defect”) can be pulled out into a text field and plugged into the LLM call. - Planning. Without the “plan first, then code” requirement, accuracy in the tests was 0.88–0.96, and the programs broke on rephrased requests. With it: 1.00.
Prompt Template
The paper’s text doesn’t include the instructions themselves; they’re in the authors’ appendix. Below is a template assembled from the paper’s description: plan, rules, layers, reduction techniques, the llm<T> type.
<Role>
You are an engineer who turns a successful agent run into a repeatable
neuro-symbolic program: actions and calculations are code, and the model is called
only for judgments via llm<T>(question, data) -> T, where T is the answer type
(string, Boolean, choice from a list).
</Role>
<Inputs>
<RequestTemplate>{request_template_with_variables}</RequestTemplate>
<ReferenceRun>{successful_run_trajectory}</ReferenceRun>
<Tools>{tools_and_API_description}</Tools>
</Inputs>
<Plan>
Write the plan first, code after:
1. Which values change from run to run → program parameters.
2. Which decisions were made in the run. For each: rule / cache / llm call.
3. Which steps of the run are accidental assumptions or shortcuts.
Don’t carry them over unless they follow from the request template.
4. What input variations are possible (other phrasings, edge cases).
</Plan>
<Layers>
Assign each llm<T> call to one layer:
- INPUT: free text → typed value.
- COMPUTATION: a semantic decision that’s hard to write as a rule.
- OUTPUT: a written presentation of a result computed by code.
Insert the call at a fixed point in the code. Check that the answer matches type T.
</Layers>
<Reduction>
- CACHING: store a value observed in the run in a dictionary. Before using it,
check the condition. Call the model only if the check fails.
- CODIFYING: write a recurring judgment as a rule (regex, threshold, dictionary).
The model must not make a decision by doing calculations in its reasoning.
- CLUBBING: combine independent questions into a single call.
</Reduction>
<Output>
1. The plan.
2. The program with parameters.
3. A table of all llm calls: layer, type T, input.
4. Tests: the deterministic part and 3 rephrased requests.
</Output>
Fill in: {request_template_with_variables}: your typical request with blanks in place of the changing values (“for {week},” “for the city of {city}”). {successful_run_trajectory}: an export or chat log of the successful run. {tools_and_API_description}: what the agent uses to read data and write results.
🚀 Quick start: paste this into a chat (ideally Claude Code or another coding agent):
Here is the FlashScript template. Adapt it to my task: [describe the recurring task].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask which values in the request change from time to time, which decisions you made by hand in the successful run, and which tools are available. It needs this to separate the parameters from the fixed steps and figure out what can be written as a rule. It will take the structure from the template and build a plan for your task.
Limitations
⚠️ Preliminary results: This is a short paper with “first results.” One small model, one benchmark, 12 request templates. Don’t carry the 70x savings and 100% accuracy figures over to just any task.
⚠️ Requires code and somewhere to run it: The output is a Python script with model calls. It has to be run, stored, and maintained somewhere. The method won’t work in a regular chat; you need a coding agent and API access to the model.
⚠️ The authors’ full instructions aren’t in the text: They’ve been moved to the appendix. The template above is a reconstruction from the described principles, not a copy of the original.
⚠️ A narrow type of task: It was tested on analytics with clear correct answers: read data, compute, compare to a threshold, build a chart. On creative, fuzzy, or exploratory tasks where the path differs every time, the method is untested.
⚠️ It doesn’t remove randomness: The method only locks it inside typed calls. The wording of answers and borderline judgments can still drift.
⚠️ Mistakes from the reference run move into the program: If the successful run contained a shortcut that happened to produce the right answer, it ends up in the code. The plan requirement helps, but you still need to review the code.
⚠️ A trajectory alone hurts: If you give the agent a single run without the request template, the program is too tied to that specific case. On new instances of the task, accuracy dropped, and some runs crashed with an error.
How They Tested It
The team used the WorkBench benchmark: 12 templates of analytics tasks, ten variants of each, 120 tasks in total. An example: “if the number of active users is more than 10% lower than on November 24, build a chart.” They worked with a small model, Claude Haiku 4.5, at temperature 1. Everything was tried three times to see the variance, not just correctness. For each template, the best run of a regular agent was turned into a “skill” and into an NSP. A coding agent wrote both.
The results were compared with a regular agent and an agent with a skill. The regular agent reached the same state as on its first try in 82% of repeats. The skill raised that figure to 99%, but not to 100%. Then the authors glued up to ten independent requests into one. And that’s where the big surprise showed up: the agent with skills, given all the skills it needed, fell to 23% accuracy, added extra charts, and forgot the needed ones. Under the same conditions, the NSP stayed correct and completely identical. On 108 new task variants, the programs also solved everything, while the skill managed 96%.
The most practical result is the experiment with generation instructions. A plain “write a program from the template” gave 0.96 from the template and 0.88 from the trajectory. But the programs broke on rephrasings. Adding the requirement “plan first, write decisions as rules, assign calls to layers” raised accuracy to 1.00. The template / run / both comparison shows that correctness comes from the template and the plan, while the run adds savings (fewer calls, 1.5 versus 1.8 per run) and a way to check the run’s decisions.
Adaptations and Extrapolations
🔧 Technique: pull the judgment criterion out into a parameter → the user controls the decision without touching code
The authors suggest making judgments configurable: a menu choice or a text field. In the reviews scenario, it looks like this:
In the program, add a parameter {defect_criterion} (a text field, default value:
“a complaint about a defect, breakage, or mismatch with the description”).
Plug it into the llm<bool> call on the COMPUTATION layer.
Now the manager changes the definition of “defect” with a single line and immediately sees how the result changes.
Extrapolation: a “when to compile” rule in the agent’s instructions file (my idea, not tested in the paper):
If the user asks for the third time to run a task with the same request template,
offer to turn the successful run into a program: write a plan, find the parameters,
decide what becomes a rule and where an llm<T> call is needed. Don’t run it without confirmation.
A rule like this in CLAUDE.md or AGENTS.md turns a “one-time lucky run” into an asset that keeps compounding.
Resources
- FlashScript: Compiling Recurring Agent Workflows into Reusable Neuro-Symbolic Programs: Sadra Sabouri, Shreya Sinha, Souti Chattopadhyay (University of Southern California); Hosein Hasanbeig, Adam Sage, Cale Darling, Ashish Tiwari, Katie Garms, Sumit Gulwani (Microsoft).
- The authors’ instructions for generating NSPs: https://figshare.com/s/c812d719d2d43bae034c
- The WorkBench benchmark (Styles et al., 2024), arXiv:2405.00823.
- Related work on compiling trajectories: SKILL-DISCO, TraceCompiler, Agent Workflow Memory.