Vol. XXIX · Multimodal Models Plate 6 of 7image tokens, resolution, cost, context window, screenshots, downscaling, vision pricingCE

What Pictures Cost.

Count the patches: that is the price of an image in tokens, context and money.

A served multimodal model bills images as input tokens, and the count follows from patching (plate XXIX.1). Anthropic documents its rule exactly: an image of width W and height H costs ⌈W/28⌉ × ⌈H/28⌉ visual tokens; images whose long edge exceeds 1,568 pixels, or that would cost more than 1,568 tokens, are downscaled to fit on the standard tier, and a high-resolution tier for newer models raises those limits to 2,576 pixels and 4,784 tokens.

Images therefore compete with text for the context window and the budget, and the costs add up fastest where images repeat: an agent that takes a screenshot every step (plate XXV.6), a document of many pages, a video sampled at a few frames per second. The workshop computes the tokens for an image of the chosen size under both tiers and the total for a sequence of such images.

Below: visual tokens against image size for a square image under both tiers; readouts for the chosen size and the number of images.

What you will learn
Workshop
Square images of the chosen side. Standard tier: long edge at most 1 568 px and at most 1 568 tokens; high-resolution tier: 2 576 px and 4 784 tokens; downscaled preserving aspect ratio until both limits hold (computed here as the largest side s with ⌈s/28⌉² within the token limit). Total: tokens per image times the number of images.

Resolution is a budget decision

A 200-pixel thumbnail is 64 tokens; a 1 000-pixel square, 1,296. Beyond the cap the price stops rising — a 4 000-pixel square is downscaled to 1,521 tokens on the standard tier and 4,761 on the high-resolution tier — but so does the detail. An agent taking fifty 1 092-pixel screenshots spends 76,050 tokens on images alone, and a 200 000-token context holds only about 131 of them, which is why computer-use agents prune old screenshots from their context (plate XXV.4).

The choices follow: send images at the lowest resolution that preserves what the task needs — text legibility usually sets the floor — crop to the relevant region instead of sending a full screen, and prefer text extracted from documents when the layout does not matter. The token count is also the compute count: every visual token is processed by every layer, like a word.

Video multiplies the problem by time: sampled at one frame per second, ten minutes of video is 600 images before any compression across frames.

Terms

Visual token
One patch of an image as the model processes it.
Downscaling cap
The size limit beyond which images are shrunk before tokenising.
Resolution tier
A model’s limits on image size and tokens.
Cropping
Sending only the relevant region of an image.
An image costs tokens in proportion to its patches, up to a cap; resolution and cropping are cost decisions.

Sources

  1. Anthropic, “Vision” documentation (accessed October 2026).
  2. Liu, Li, Li and Lee, “Improved Baselines with Visual Instruction Tuning”, CVPR 2024.