TL;DR
The study shows how to slice a large image into tiles the right way so a model with “eyes” (a VLM, vision-language model) finds small objects without losing anything along the way. The rule is simple. Show every tile at no smaller a scale than the object appears in the full image. Overlap neighboring tiles by at least the size of one object. Do that, and tiling on average doesn't hurt the result and more often improves it noticeably.
The main pain: you send a blueprint, a screenshot or a photo of a store shelf as a whole, and the model finds 15 objects out of 23 one time and 9 the next. The cause is how the input works. The model compresses the image down to a fixed budget of “visual tokens” (tile-like patches). An object on a large sheet gets less than one patch per side and turns into a smudge. Doubling the budget barely helps; per-side resolution only grows by its square root. Cram lots of objects into one call on top of that, and accuracy also drops with the count. Worse still, on some models “high-resolution mode” performed worse than the default.
The method has three moves. First, slice the image into overlapping tiles. Then look at each tile in a separate request. Finally, merge the answers and remove duplicates at the seams. “Safe” here means: no tile shows an object smaller than the full image does, and no object fails to sit entirely inside at least one tile.
Want this for your own case? Apply this study to your task: AI adapts its method to what you are working on and writes the prompt.
Method Blueprint
STEP 1: Estimate the object size (in pixels) and the model's per-image limit
STEP 2: Pick a tile size ≈ the model's limit WITHOUT downscaling (so the model doesn't shrink it itself)
STEP 3: Slice with stride = tile size − object size (overlap ≥ 1 object)
STEP 4: Each tile → a separate request → a list of objects with coordinates within the tile
STEP 5: Convert coordinates to global ones, remove duplicates at the seams → final list
Steps 1–3 are prep (by hand or with a script). Step 4 is separate requests. Step 5 is one request or a spreadsheet.
Example in Practice
Task: A cost estimator at a construction company has received the electrical drawing for one floor of an apartment complex. The sheet is about 11000×7700 pixels. They need to count outlets and switches against the legend for a cost estimate in dollars. Send the sheet whole, and each symbol turns into a dot, and the models miscount.
Prompt (for each tile; the tile is cut with an overlap of one symbol):
This is tile {N} of {total} from a large electrical drawing. The tile was cut from the sheet without downscaling.
The overlap with neighboring tiles is about 60 pixels.
The reference symbol for “duplex outlet” is in the attached legend image.
Task: find ALL duplex outlets in this tile.
For each one, give the center of the symbol in tile pixels (x, y).
If a symbol is cut off by the tile edge, still list it and mark it “on edge.”
Don't make anything up: if you're not sure, mark it “uncertain.”
Format: table | # | x | y | status (clear / on edge / uncertain) |
At the end: the total number found in the tile.
Result: for each tile you get a table with coordinates and statuses, plus a total. There will be about two dozen tiles, each with a few symbols. In the last step you (or a separate request) convert the coordinates to global ones and remove duplicates at the seams: points closer than one symbol's size count as one object. The sum gives you the number for the estimate. You check the “on edge” and “uncertain” lists by eye.
Why It Works
The weakness: the model has an image budget: it cuts the picture into patches, and there's a limited number of them. A large sheet gets compressed, and the object gets a fraction of a patch. Doubling the budget only raises “visibility” by 41%. On top of that, the more content in a call, the worse the model is at listing everything. So a large image has two problems at once: the objects are small and there are too many of them at once.
The strength: the model is good at finding objects when they're large and few. Tiling gives you both at once: the scale goes up, the content per call goes down. An overlap of one object guarantees that no object gets cut along a seam without a “whole” copy somewhere else.
The levers: - Tile size → smaller tile = larger object, but more calls. The maximum is whatever the model won't shrink on its own. - Overlap → at least one object. Less, and you'll lose objects at the seams. More means more duplicates and tokens. - Scale relative to the whole → don't shrink. A typical mistake: “I'll normalize the objects to the same pixel size and slice.” If the object is already large, the tiles end up smaller than the whole, and the result is worse. - High-resolution mode → don't turn it on by default. On some models it was a disaster. Test on 2–3 examples. - Drawing grids, labels, arrows on top → adds no information. Cropping helps, overlays don't.
Prompt Template
The paper has no ready-made prompts. The research gives you a tiling rule. The template below is the wrapper around it for querying the model on one tile.
This is tile {number} of {total} from the image “{image_name}.”
The tile was cut without downscaling. Neighboring tiles overlap by {overlap_px} pixels.
Task: find ALL objects of type “{object_type}” in this tile.
What the object looks like: {description_or_reference}.
For each object, give its center (x, y) in tile pixels.
If an object is cut off by the edge, include it and mark it “on edge.”
If you're not sure, mark it “uncertain.” Don't make up objects.
Format: table | # | x | y | status |
At the end, on one line: “Found in tile: {count}.”
What to fill in:
- {number} and {total} — the tile's sequence number and the total count.
- {overlap_px} — the size of your object in pixels (no less).
- {object_type} and {description_or_reference} — what you're looking for; attach a reference image if you can.
For the merge step (a separate request):
Here are {total} tables of objects found per tile, plus the offset of each tile:
{tables_and_offsets}
Convert the coordinates to global ones (add the tile offset).
Merge points closer than {threshold_px} pixels into a single object.
Output the final table and the total count. Separately list objects with the status “uncertain” and “on edge.”
🚀 Quick Start — paste into the chat:
Here's a template for safely tiling an image to find small objects. Adapt it to my task: {your task}.
Ask me questions to fill in the fields. Also write a Python script that slices the image with overlap
and saves the offset of each tile.
[paste the template above]
The LLM will ask for the image size in pixels, the size of the target object and the chosen model's per-image limit. Those three numbers are needed to compute the tile size and stride according to the paper's rule. It takes everything else from the template.
Limitations
⚠️ The theory rests on two assumptions: on average, more tokens per object and less content per call don't hurt. The authors tested this, but it doesn't hold equally everywhere.
⚠️ High-resolution mode can backfire: on OpenAI models, feeding more pixels than the default made the result worse in a number of cases. Don't assume “more pixels” is automatically better.
⚠️ Rescaling “to the object size” hurts: if the objects are already large, the tiles after normalization end up smaller than the whole, and quality drops in most of the tested cases.
⚠️ Tiling doesn't create information: if the object isn't resolved in the pixels of the original image, zooming won't help. That's a limit of the data, not of tokens.
⚠️ Duplicates and seams: overlap produces repeat detections. Without a merge-and-check step, the final count will be inflated.
⚠️ Lots of manual work: a sheet with 20 tiles needs a script (an LLM can write it) or patience. Token usage is several times higher than a single call.
⚠️ Scope of testing: the main testing ground was blueprints and floor plans, plus natural and synthetic images. For screenshots, shelf photos and slides the conclusion is logical but was tested less.
How They Studied It
The idea was to describe, instead of yet another “zoom trick,” two quantities at the input: how many patches fall on each side of the object (S) and how much content the model has to cover per call (L). Any method (tiling, high-resolution mode, a zoom agent) simply moves these two levers. The authors derived a theory from this, then tested it on roughly 177 thousand queries across 797 images and 11 models from five families. Data: a new set of 40 construction drawings, 10 more held-out drawings, FloorPlanCAD plans, natural images and synthetic ones. They pre-registered their hypotheses before running anything.
On the controlled images, not one of the 40 predicted relationships was violated. In the post-hoc check on other data, 61 out of 89 held up, and four of the failures came from OpenAI models given more pixels than the default. A telling failure: one of the high-resolution modes scored an F1 of 0.33 on full sheets versus 0.89 with the standard one. The popular recipe “rescale to a fixed object size and slice” (with 44 px patches) lowered recall in 19 out of 21 cases. The overlap-and-no-downscaling rule on blueprints never made the result significantly worse and improved it by up to +0.28. On a sample sheet of 11236×7698 pixels, the symbol got S=0.40 when sent whole and S=1.5 when tiled into 20 pieces. That cost 49 thousand tokens versus 2.5 thousand, but reliably found all 23 symbols. One more result: as the number of blocks in a single call grew, the F1 for the first block dropped by 0.22. In other words, the sheer volume of content degrades quality on its own. The takeaway for practitioners: don't slice “however,” slice by the rule, and check on a couple of examples whether the scale did any harm.
Adaptations and Extrapolations
🔧 Technique: drop the grid and labels on the image → stop wasting time
The study shows that drawing grids, labels and dots on top of an image you've already sent adds no information: it's the same picture with different coloring. Instead of “draw a grid and search the cells,” it's better to crop the cells as separate tiles and run them through the rule above.
Extrapolation (not tested in the paper, but consistent with its reasoning about zoom agents): the tiling rule as an instruction for an agent, for example in a Claude Code instructions file.
## Working with large images
If you need to find or count small objects in an image larger than {limit_px}:
1. Don't send the image whole, and don't turn on high-resolution mode without testing.
2. Slice it into tiles no larger than {limit_px} (no downscaling), with an overlap of at least the object size.
3. Don't reduce the scale relative to the original.
4. For each tile, build a list with coordinates, convert to global ones, remove duplicates.
5. Show objects on edges and uncertain ones as a separate list for human review.
Resources
- Paper: Why VLMs Miss Small Objects, and When Zooming In Is Safe.
- Authors: Junzhe Shi, Shida Jiang (Systems Engineering, University of California, Berkeley), Yuan Gan (Quotr AI).
- Code, the REDP-X40 benchmark, a safe-tiling planning tool and raw model responses: https://github.com/shijunzhe/vlm-small-objects