Coding Agents as Illustrators: SVG and Canvas Instead of an Image Generator for Diagrams, Design and Text in Images

STEP 1: Identify the task type → structure/text/diagram OR photo/people/lighting
STEP 2a: Structure → coding agent (SVG or Canvas) → file + render
STEP 2b: Photorealism → a regular image generator
STEP 3: Check → a checklist of "what must be in the image" (yes/no per item)
STEP 4: 1–2 revision rounds, no more → final result
(For style: add a written drawing procedure for the chosen technique to the context)

All of this happens in an ordinary agent session. Steps 1–4 run in a single session.

Task: An Amazon seller is building listing images for an insulated travel mug. They need a 1200×1200 infographic: the mug in the center, four benefit badges ("Hot for 12 hours", "304 Steel", "16 oz", "Leak-proof"), and arrows from the badges to the mug. Regular generators garble text and mess up the number of badges. This is exactly the zone where code wins.

Prompt:

You are working as a layout designer. Draw an infographic for an Amazon
product listing as a single SVG file, 1200×1200 px.

Content:
- Center: a simplified vector travel mug (body, lid, handle), navy blue
- Background: light gray, #F2F4F7
- Four rounded-corner badges, two to the left and two to the right of the mug:
  1. "Hot for 12 hours"
  2. "304 Steel"
  3. "16 oz"
  4. "Leak-proof"
- A thin arrow from each badge to the matching part of the mug
- Sans-serif font, English text, nothing cut off at the edges

Work like this:
1. First, list the coordinates and sizes of all blocks.
2. Write the SVG.
3. Render it to PNG and open the result.
4. Check it against the checklist, answering "yes/no" for each item:
   - Exactly 4 badges?
   - Does the text on each badge match the spec word for word?
   - Does the text stay inside the badge?
   - Does every arrow point at the mug?
   - Size is 1200×1200?
5. Fix only the items marked "no". No more than two revision rounds.
Save the result as card.svg and card.png.

Result: The agent will first output a table of block coordinates, then the SVG code and a PNG render. Then it walks through the checklist and tells you what it fixed. The mug will be stylized and flat, not photographic. The badge text will be readable and verbatim. The block count will match the spec. For a lifelike photo of the mug with glossy highlights, you need a different tool.

The weakness: a regular image generator paints the picture "all at once," and details like the exact number of objects, long text or block placement aren't written down anywhere explicitly. So they drift.

The strength: the model is excellent at writing code. In SVG and Canvas, every element has coordinates, a color and text. That's a structure you can check by eye and item by item. Counts, labels and ordering can be verified right in the markup.

How the method gets around the weakness: the task moves from "paint it in pixels" to "describe it as a program." The agent stops guessing the whole image and assembles it from explicit parts. That's why design, diagrams and symbols come out better, while lighting, materials and anatomy stay weak: describing them in code takes far too many subtle decisions.

Control levers: - Harness (the drawing interface): SVG and Canvas 2D gave the best results. Drawing pixel by pixel or with simple primitives did noticeably worse. - Number of revision rounds: fewer than you'd think. In the study, a 4x budget barely improved the result. - Checklist: the more concrete the items ("exactly 4 badges"), the more precise the fixes. - Written style procedure: add the construction order and a description of the technique to get a medium-specific result.

The paper contains no ready-made prompts: the authors describe protocols rather than publishing prompt texts. The template below is assembled from their findings (code instead of pixels, a checklist, few revision rounds).

You are working as a {executor_role}. Draw {what_to_draw} as a single
{code_format} file, size {size}.

Content:
{list_of_elements_with_exact_text_and_numbers}

Style: {style_colors_fonts}

Work like this:
1. List the blocks with their coordinates and sizes.
2. Write the code.
3. Render the result and open it.
4. Check it against the checklist, answering "yes/no" for each item:
   {checklist_of_3-6_verifiable_items}
5. Fix only the items marked "no". No more than {number_of_rounds} revision rounds.
Save the result as {file_name}.

• {code_format} — SVG for static diagrams and infographics, HTML Canvas for complex graphics.
• {checklist} — only items you can answer "yes/no" (counts, verbatim text, boundaries, size).
• {number_of_rounds} — 1–2. More barely helps.

🚀 Quick start — paste into a chat or an agent:

Here is a "drawing with code plus a checklist" template. Adapt it to my task: {your task}.
Ask me questions to fill in the fields.

[paste the template above]

The LLM will ask about the exact text, numbers and sizes, the format (SVG or Canvas) and what counts as an error. It needs this to build the checklist, which is what the whole quality check rests on.

⚠️ Photorealism: People, landscapes, lighting, materials and perspective are worse with a coding agent than with a regular generator. In the "scenes" and "people" categories it wins only a small fraction of comparisons. For these tasks the method actually hurts.

⚠️ Aesthetics: Structural accuracy is not the same as beauty. The agent produces a clean, flat image. Anatomy and natural forms stay below native-generator quality in every configuration.

⚠️ A strong modern generator: Against gpt-image-2, the agent wins in design and symbols only by a slim margin. Most of the headline gains were measured against the older gpt-image-1.

⚠️ Extra iterations: Quadrupling the refinement gave only a small structural gain and didn't make the image noticeably better overall.

⚠️ Part of the paper is a hypothesis: The idea of agents having an "internal visual language of thought" rests on indirect observations (the same layout and tones across different runs). It yields no practical takeaways.

A team from UCLA and Tsinghua University built the AgentDrawBench benchmark: 200 prompts of seven types, from photos to diagrams, drawn from four published test sets. They then had agents (GPT-6-Astra, Kimi K3, DeepSeek V4.1 Flash) draw them in different ways: pixel by pixel, with shapes, with Canvas 2D, with SVG, through the JS Paint graphics editor and through Blender. Results were compared with the native generators gpt-image-1 and gpt-image-2. They measured three things. Structure was checked with a yes/no checklist for every element of the prompt. Perception was scored on ten criteria such as anatomy, lighting and material. Overall preference was decided by a judge model in pairwise comparisons, with the image order swapped to remove position bias.

The results were sharply lopsided. Structural accuracy for Canvas 2D was 85.8% versus 68.8% for gpt-image-1. In design and symbols the win rate reached 92%. In scenes and people there were almost no wins (13% and 6%). Two things were surprising. Blind drawing scored 83.0% versus 85.6% with image feedback. And a fourfold budget improved the result by just 1.2 pp. The practical takeaway: the bottleneck is the drawing interface, not the number of attempts.

A separate experiment showed that the SVG code an agent converts an image into helps answer questions about that image. Accuracy when answering from the code was 69.0%, from the rendered version 65.0%, from the original 74.0%. In the "academic materials" and "digital world" domains, answers from the code were more accurate than answers from the image itself. The reason is simple: the code explicitly holds the text, coordinates and object counts, so they don't get lost in the background. In a button-counting example, the agent counted four from the code but miscounted three from the photo.

🔧 Technique: "Convert to code first" → precise answers about diagrams and screenshots

The paper shows this on image question answering: code reconstruction helped with documents, diagrams and interfaces. Example:

Copy

Before answering, convert this dashboard screenshot into SVG: every block, label and number
must be a separate element with coordinates. Then answer the question
using only the SVG: {question}

This is an extrapolation: on your own data, compare answers from the direct path and the code path. In the paper the effect depends on image type and is weaker for photos, scenes and people.

🔧 Technique: add a written style procedure → get a medium-specific result

In the paper, procedures (construction order, tone grouping, edge control) were extracted from video demonstrations and fed to the agent in context. You can package such a procedure as an instructions file or a skill:

Copy

"Watercolor" style: 1) start with transparent washes in broad patches; 2) leave
the light areas of the paper untouched; 3) layer from light to dark;
4) finish by adding accents with a dry brush.

The paper's results here are qualitative, shown on examples. There are no quantitative measurements of the benefit.

• Paper: How Far Can Coding Agents Go in Visual Generation?
• Authors: Hengguang Zhou, Yuanhao Ban, Jiaqi Feng, Xiaohuan Pei, Justin Cui, Cho-Jui Hsieh
• Universities: University of California, Los Angeles; Tsinghua University
• Datasets and tools mentioned: AgentDrawBench (the authors' benchmark), WorldBench, MMRB2, JS Paint, Blender Cycles, SketchAgent