YouTube Summaries

← All summaries

Images vs Text: Token Efficiency for LLMs

2026-07-13 Mon ⏱ 13 min prime

Prime does a victory lap over a claim his friends once told him was wrong: that feeding models images can be more efficient than feeding them text. A new DeepSeek paper reports you can recover ~10 text tokens from a single image token at near-100% accuracy, meaning a model's internal representation of an image is roughly 10x denser than its representation of text. Prime explains the intuition (via a Sean Godecke / "Shawn Geodec" blog post), then runs his own tower-defense-game experiment to test whether image context actually beats string context in practice — and finds the answer is a very senior-engineer "it depends."

The claim and the numbers

DeepSeek's paper: images can encode text ~10x more efficiently with near-100% recovery accuracy. Related research claims 59–70% lower end-to-end bills from using images. Prime spent ~2 billion tokens producing the video. He nods at a related trick from George Mandez: run audio through FFmpeg sped up 2–3x to save tokens with similar results.

Why images can beat text

The blog's explanation, as Prime relays it: text tokens are discrete while image tokens are continuous. A model has a finite vocabulary (say ~50,000 tokens), each mapping to an embedding of ~1,000 floating-point numbers. Text tokens occupy only 50,000 fixed points scattered in an otherwise infinite continuous space — very sparse. Image token embeddings can be any point in that continuous space, so a single image token can be far more expressive than a run of text tokens. Another framing: you send an LLM a stream of integer token IDs, which the model expands internally into an inefficient representation that encodes meaning and usage — so beating text is unsurprising. The author also draws a brain analogy (humans read scrambled words as images), which Prime dismisses as weak since we do not really know how the brain works. He does buy a concrete example: a Rust compiler error shown as an image, with big arrows pointing at the problem and the fix, conveys far more usable context at a glance than reading every text token.

The experiment

Prime shoved his game's entire context file into a single image using "PX Pipe" (Pixelpipe), the tool cited for the 59–70% cost reduction with Fable 5. His tower-defense game can run in a JSON mode (no rendering) where an AI drives it via what behaves like an MCP interface — mouse operations sent in, flowing through the real program. He ran five models concurrently for hours, comparing context-as-string against context-as-image.

Results

Context-as-string: 50 successful games, averaging ~9 minutes each, with solid damage and smart tower/card placement that maximized output. Context-as-image: only 23 successful games; time ballooned to ~20 minutes average, some games taking 30 minutes. Damage was broadly similar (lower at the max, higher across the middle), hits were about the same, but more enemies reached the end with higher health, dying the player faster. Token counts from cursor: string context ~306 million total tokens over 324 events; image context only 213 million — but for well under half the games played. Getting to a verifiable experiment at all cost him ~1.5 billion tokens.

Takeaway

Prime's read: the image strategy was actually harder and more confusing to use well; the model seemed to guess its way through ("yes, this is a tower defense game") without playing as sharply, and the wildly longer per-game time is the tell. His conclusion is the classic senior answer — it depends — so do not rush to convert everything to images just because Twitter is excited. Experiment and do your own homework. He also notes today's stacks (caching, tooling) are built around text as the primary input, so image pipelines may simply be under-optimized for now; even a current win could be dwarfed by gains once the tech matures. It is close to an apples-to-oranges comparison.