Open weight is great, local models are overrated – Theo
- https://www.youtube.com/watch?v=wnfxSxP8pGs
- Original title: I need to rant about local models
Theo delivers a long-overdue "crash out" drawing a hard line between two ideas that get conflated: open weight models (which he loves) and local models (which he thinks are overrated and misleading). His core claim is that the impressive open weight drops everyone celebrates — GLM-52 in particular — cannot actually be run on consumer hardware, and that the real value of open weight isn't running it yourself but the competition it creates among cloud hosts. He's pro open weight and anti the delusion that you'll escape Anthropic and OpenAI by putting GPUs in your closet.
Open weight vs local: the distinction that matters
Theo opens by clarifying scope so the comment section doesn't misread him: open weight models are awesome, essential to a healthy AI ecosystem, and worth defending — especially as governments start restricting certain models. GLM-52 is, in his words, unbelievably good, nearly catching up to closed frontier models like Opus 4.6/4.7 while being downloadable. But "download yourself" is not "run yourself." That gap is the whole video.
You cannot run the good models at home
GLM-52 is conservatively ~400 GB; full-precision BF16 is ~1.5 TB. Even quantized/pruned versions sit around 200 GB — not running on almost any consumer hardware. What people actually run locally is stuff like a quantized Gemma 4 that "barely functions." The models that are open-weight-and-incredible are a completely different category from the models you can actually run at home.
VRAM is the bottleneck, not system RAM
A key technical point: a gaming PC with 128 GB of RAM and a 5080 (16 GB VRAM) or 5090 (32 GB VRAM) still can't fit big models, because inference is bound by VRAM, not system RAM. Macs work because they use unified memory (RAM doubles as VRAM). The three real consumer options for lots of unified memory are: a high-RAM MacBook (now ~$3,000 extra for 128 GB after price hikes), a DGX Spark (slow, ~12 TPS, "just sits there"), or a Strix Halo box / Framework desktop. None of these are real high-end Nvidia GPUs, so they run these models slowly. The trade-off is brutal: enough RAM but weak compute, or strong compute (5090) but not enough VRAM.
Hardware costs are insane
The 5090 resells around $4,300 just for the GPU. The RTX 6000 Pro Blackwell (96 GB VRAM, near-identical chip) is ~$13,000 for the same performance and just more VRAM. A "tiny box" with four RTX Pro 6000s runs ~$75,000 and could almost run GLM-52 with awkward VRAM-splitting. An AMD alternative is ~$12,000 with 128 GB VRAM but still only fits Flash-tier models. Theo suggests these GPUs are a better buy as resale assets than as things to actually use.
Benchmarks are gamed and open models burn tokens
He's dismissive of the constant stream of unheard-of "locally runnable-ish" models (e.g. "Ornith") that claim to match Opus on saturated, abused benchmarks like SWE-bench Verified. On genuinely good benchmarks like Deep SWE, open models like Kimi score far lower (30% vs 52 for a frontier model), and Opus-low was both better and cheaper to run because open weight models burn far more tokens.
Even if local models caught up: parallelism kills it
Granting a hypothetical where a 30B-ish model matches frontier quality, the next wall is parallelism. Theo's real workflow fluctuates between 1 and 40 agents doing independent inference (multiple Codex/T3 threads, sub-agents, Claude and Codex simultaneously). A rig that can run GLM-52 once cannot run it 10 times. Idle GPUs are wasted money; needing a sixth lane when you built for five blocks you entirely. Caching, vLLM, and batching don't meaningfully solve this.
State of the art is more than benchmark scores
Frontier means orchestrating multi-stage workflows, computer use, and vision — GLM-52 doesn't even have vision, so it can't take a screenshot or see the UI it just built. The best models are also getting bigger (he cites Fable as somewhere between 2 and 10 trillion parameters), moving further out of consumer reach.
Electricity and phones
Running a single 5090 24/7 in San Francisco is roughly $5/day — ~$2,000/year in electricity, and that's one GPU. On-device phone inference (Gemini 3 on tensor chips for message/weather summaries) is genuinely valuable for privacy and not hitting the cloud, but it destroys battery and phones aren't improving fast enough — budget Android CPUs have barely improved since 2022, with slight regressions in the cheap tier.
The real win: competition among cloud hosts
The punchline, echoing a message from a viewer named Fry: the benefit of open weight isn't local execution — it's competition on the provider and hardware side. No one can undercut Opus because of Anthropic's licensing, but many providers host GLM-52 at different price/speed points (OpenRouter shows Wafer Fast, Fireworks, Friendly, Deep Infra, etc.). Renting solves nearly every problem on his list: the runnable-vs-good gap, hardware cost, parallelism, and electricity (data centers go where power is cheap). You only lose privacy, which Secure Compute may fix later.
The pricing catch: tokens, not price-per-token
Even in the cloud, headline prices mislead. GLM-52 at $3/M-out vs Opus 4.8 at $25/M-out looks ~10x cheaper, but per actual Deep SWE run it's Opus ~$8 vs GLM-52 ~$4 — only ~2x — because open weight models burn far more tokens. They're also effectively slower since generating 3x the tokens negates a 20% speed edge (the same problem that kills 3.5 Flash). On the artificial-analysis intelligence-vs-cost chart, GLM-52 Max is a genuinely competitive dot, but GPT-5.5 medium/low sits right next to it and cheaper despite higher per-token pricing.
Bottom line
Open weight is awesome and necessary — a competitive marketplace keeps the whole industry improving (he credits Deep Seek R1 for shifting the field, and built T3 Chat because he loves the Deep Seek line). His only complaint is the delusion that you'll run good frontier-quality open weight models on your own GPUs at reasonable performance. If you do it for fun, great — but stop pretending it's a realistic path out of the pockets of Anthropic and OpenAI, because it hurts the open weight movement by tricking people.