TL;DR
Game Completion Boost is a set of instructions for an agent (a skill) that makes it test its own work the way a real user would. It breaks the requirements into checkpoints, cross-checks the product's internal state against what actually shows up on screen, and re-walks every affected path after each fix. If a fix breaks something, it rolls back.
The study's headline finding: “it builds and it runs” tells you almost nothing about whether the product works. Agents nearly always nail the build: the project opens, compiles, and the main scene loads. But that's not a real game. In most cases they score less than half the points for game logic, and only 16% of runs make it to the end of the game. The agent confidently reports “done,” while in reality part of the game was never built, the rules break, and the save vanishes after a restart.
GCB closes this gap in three steps. First, the agent derives the player's path from the requirements. Then it plays, looks at the result, and checks it against the state. Finally, it fixes things with a safety net: save the project, edit, re-check, and roll back if anything broke. On hard games this added 14–24 points on a score computed purely from execution.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1: Requirements → checkpoints (preconditions, files, required evidence)
→ verify every state is reachable through normal player actions
STEP 2: Player action → “before/after” snapshot + state/logs from the same session
→ hunt for mismatches (“the trap fired on a safe floor tile”)
+ static checks: camera framing, UI overlap
STEP 3: Fix → reset checks for the touched files and their dependencies
→ save the project + protect checkpoints already passed
→ edit → re-check the target and every protected checkpoint
→ regression? roll back : accept
→ log a diagnosis for the next attempt
FINAL: any fix voids acceptance → full playthrough + restart with save verification
All of this runs as fixed, task-agnostic instructions for the agent, plus a few small helper scripts that schedule the order of checks.
Example in Practice
Task: You ask Claude Code to build a WhatsApp booking bot for a barbershop: pick a barber, pick a time, pay through Stripe, get a reminder two hours before. Twenty minutes later the agent writes: “Done, the bot starts, all modules are wired up.” You open the bot, and after you pick a barber it goes silent. And if you restart the server, the bookings are gone.
Prompt (paste it into CLAUDE.md or at the top of the task):
You are building a WhatsApp booking bot for the barbershop “The Beard Room.” Requirements:
1) the client picks a barber; 2) sees open slots for the next 7 days;
3) picks a slot and pays $30 via Stripe (test mode);
4) the booking is saved; 5) a reminder arrives 2 hours before;
6) canceling a booking frees up the slot; 7) after a server restart, all bookings are still there.
“It runs” and “it builds” are NOT the definition of done.
Follow this protocol:
1. CHECKPOINTS.
Break the requirements into checkpoints. For each one, write to checkpoints.md:
the precondition (what must already be passed), the files it touches,
and the evidence required (bot reply, database record, log entry).
Verify that every state is reachable through normal client actions
(via real messages to the bot, not by writing directly to the database).
2. STATE VS. VISIBLE RESULT.
After every client action, compare what the bot replied with what was written to the database/logs
in that same session. Look for mismatches: the bot said “booking confirmed” but the database is empty;
a slot is shown as open but it's actually taken.
3. SAFE FIXES.
Commit before every fix. Protect the checkpoints already passed.
After the fix, reset the checks for the changed files and everything that depends on them.
If the fix touched a shared or unclear file, reset all checks.
Accept the fix only if the target checkpoint AND all protected ones pass.
If anything broke, roll back to the commit. Write a diagnosis to checkpoints.md
(what broke and why) before trying again.
4. ACCEPTANCE.
Any fix voids acceptance. Final run: walk the full client path from the first message to the reminder,
then restart the server and confirm the bookings are still there.
Only then write “done” — and show a table: checkpoint → passed/not → evidence.
Result: The agent will first create the checkpoints file, then run through the client's scenario, logging evidence as it goes. Expect it to find and fix some bugs on its own: say, a slot that isn't freed on cancellation, or a booking lost after a restart. Fixes will come with commits and rollbacks. At the end you get a “checkpoint → status → evidence” table instead of a breezy “everything works.”
Why It Works
The model's weakness. An agent grades its own work by the easiest signal available: “the code compiles, no errors.” That's a tempting definition of done. The study shows that agents are weakest exactly where behavior has to be verified. Weak models leave half the product unbuilt. Mid-tier models build everything, but the rules break in real use. Strong models break the transitions: restarting, reopening, showing the result to the player. A single pass/fail score hides all of this.
The model's strength. The model is good at following a clear protocol, keeping notes in a file, and cross-checking two data sources. It can compare “what was recorded” with “what was shown” and spot the mismatch. It can also read its own diagnosis and avoid repeating a mistake.
How the method uses this. It turns a vague “check that everything works” into concrete checkpoints with mandatory evidence. It guards against the biggest risk of fixing things, where you fix one thing and break another: re-checking what already passed, plus rollback. And it forbids calling the task finished until the full path has been walked again after the last fix.
Control knobs: - How aggressively to reset checks: resetting only what depends on the changed files saves time; resetting everything is safer. For small projects, reset everything. - Type of evidence: a database record, a log, a screenshot. Pick whatever the user actually sees. - Final acceptance: add your own task-specific “restart”: a cold start, logging back in, a fresh browser. - Diagnosis file: drop it if the task is short. Keep it if the agent is going in circles.
Prompt Template
You are building {what_we_build}. Requirements:
{numbered_requirements_list}
“It builds” and “it runs” are NOT the definition of done. Follow this protocol:
1. CHECKPOINTS.
Break the requirements into checkpoints. For each one, write to {notes_file}:
the precondition (what must already be passed), the files it touches,
the evidence required ({evidence_type}).
Verify that every state is reachable through normal user actions,
without bypassing the interface.
2. STATE VS. VISIBLE RESULT.
After every user action, compare what the user sees
({what_user_sees}) with the internal state ({where_to_check_state}).
Look for mismatches. Separately check static things: {static_checks}.
3. SAFE FIXES.
Before each fix: {save_version}. Protect the checkpoints already passed.
After the fix, reset the checks for the changed files and everything that depends on them.
Changed a shared or unclear file? Reset all checks.
Accept the fix only if the target checkpoint AND all protected ones pass.
Something broke? Roll back. Write a diagnosis (what broke, why), then try again.
4. ACCEPTANCE.
Any fix voids acceptance. Final run: walk the full user path
from {path_start} to {path_end}, then {post_restart_check}.
Only then write “done” and show a table: checkpoint → status → evidence.
What to fill in: {what_we_build} — the product; {notes_file} — the checkpoint file (e.g., checkpoints.md); {evidence_type} — a database record, a log, a screenshot; {what_user_sees} — a bot reply, a page, an email; {where_to_check_state} — the database, a log, a file; {static_checks} — layout, overlapping elements, truncated text; {save_version} — a commit; {post_restart_check} — restart the service and confirm the data is still there.
🚀 Quick start — paste this into the chat:
Here is a self-verification protocol template for an agent (the Game Completion Boost method). Adapt it to my task: [your task].
Ask me questions to fill in the fields.
[paste the template above]
The LLM will ask what exactly you're building, what the user sees and where the internal state lives, how to save a version, and what counts as a “restart.” That's what makes the checkpoints and evidence concrete: without them, the protocol collapses into a generic “check everything.”
Limitations
⚠️ Tested on games only: GCB was tested on complex games built in game engines. The paper doesn't show what it does for websites, bots, or internal services. Carrying it over is a hypothesis you'll need to test.
⚠️ Not all of the method is text: the original is instructions plus helper Python scripts that schedule the queue of checks. The “prompt-only” version is simpler and possibly weaker.
⚠️ Tested on three models in one environment: the gains are shown for three models in one tool (Codex CLI) and on the hardest games. What happens on simple tasks is unknown.
⚠️ The agent must be able to “play”: if the agent can't launch the product and feed it real input, the protocol is pointless. You need launch, input, and access to state.
⚠️ Run-to-run noise: the same agent on the same task differs by 8 points on average between runs. One lucky or unlucky run proves nothing.
⚠️ Effort ≠ results: the number of steps and tokens is barely related to the final score. Asking it to “work longer” is no substitute for a verification protocol.
⚠️ Rankings don't transfer: model rankings depend on the engine. A model that came second on one platform finished last on another. “Model X is better” conclusions hold only for your environment.
How They Studied It
The authors started by assembling 50 agent tasks: ten for each of five game engines (Defold, Godot, Roblox Studio, Unity, Unreal Engine). The agent gets an empty project and a design document written like a contract: every rule is verifiable, and every rule maps to a hidden check. The hidden grader plays the game with real input, like a QA tester: it clears levels, restarts, closes and reopens the game. That's how it confirms the save is still alive. Each task is broken down into 47 checks on average. Reference games built by engineers together with coding agents scored about 96 out of 100. In other words, the tasks are genuinely doable.
Next they tested six models across four harnesses (1050 runs, three per “game–configuration” pair). The picture: the build almost always succeeds (about 9 out of 10 points for integration), while the gameplay falls apart. Only 16% of runs reach the end, and just 4% are fully solved. The best autonomous configuration scored 65 out of 90; the rest scored no more than 41. The engineers' reference builds scored 87. The authors' conclusion: the gap lies in the agents themselves, not in the difficulty of the tasks.
Some curious details. When agents were grouped by skill level, weak ones most often left the game unfinished, mid-tier ones broke the rules, and strong ones broke transitions and restarts. In other words, the better the agent, the later it stumbles, and a binary pass/fail score hides that. Another finding: rankings depend on the engine (rank correlation between engines ranges from 0.14 to 0.62), and the same game scored 10–21 points higher on heavyweight engines. The reason: those engines run more checks before launch.
Hence the practical takeaway. If the failures are unfinished pieces, broken rules, and lost state, the cure is scenario-based verification with a replay after every fix. GCB was assembled from three well-known ideas: tests derived from requirements, regression test selection, and reflection after failure. On hard games, it lifted the execution score of three models by 14–24 points.
Adaptations and Extrapolations
🔧 Technique: drop the scripts → keep the checklist by hand
Replace the helper scripts with a
checkpoints.mdfile containing a table: checkpoint | depends on | files | status | evidence | date checked. Add a rule: “after a fix, mark every row with touched files asstaleand re-check them.” The agent acts as its own scheduler, and you can see the state of verification at a glance.
🔧 Technique: add a strict “don't report done” rule
Add at the end: “If even one checkpoint is not passed or not verified, start your reply with the words ‘NOT DONE’ and list those checkpoints.” This guards against the agent's habit of signing off with a shiny “done.”
Extrapolation (an idea from the benchmark's authors, carried over to your tasks). In the benchmark, the product must expose its state to the outside through a small interface so the checker can read it. The grader was written from the contract, without peeking at the code. Bring this into everyday development:
Product requirement: add a service command /debug_state that
shows the key values ({key_list}) and computes them from real data,
not from constants. Then write checks based ONLY on the requirements list —
without reading the implementation code — and run them through real user actions.
That way the checks don't bend to fit code that's already written.
Resources
- Paper title: From Code to Playable Systems: CEGame-Bench for Long-Horizon Game Creation Agents (preprint, NJU-LINK)
- Authors: Zixuan Dong, Yanghai Wang, Hanyan Bian, Jiaxi Tao, Junze Li, Jiadong Wang, Xinyu Che, Lin Liu, Baoyun Peng, Xiaodong Wang, Jiaheng Liu
- Affiliations: National University of Defense Technology, Nanjing University, University of Tokyo, Xi'an Jiaotong University, Academy of Military Sciences
- Contacts: [email protected], [email protected]
- Cited foundations of GCB: specification-derived tests (Jia et al., 2026); reflection after failure (Shinn et al., 2023, Reflexion); regression test selection (Gligoric et al., 2015)
- Note: the available text does not include Appendix H with the full GCB instructions; the template above was assembled from the description of the three mechanisms in the main text.