YouTube Summaries

← All summaries

Kimi K3 scares frontier labs and the distillation debate – Theo

2026-07-26 Sun ⏱ 36 min t3dotgg

Kimi K3, a new Chinese open-weight model from Moonshot AI, has caused unusual panic across the US AI ecosystem. Theo argues the model is genuinely competitive with frontier models (comparable to GPT-5.6, better at some tasks like 3D and cybersecurity), and that the loud accusations of "distillation attacks" and calls to ban open-weight Chinese models are mostly political theater. He walks through what distillation actually is, why the government's claims are weak, breaks down an OpenAI researcher's controversial thread on open weights being "decelerationist", and closes with a personal conspiracy theory about Anthropic delaying Opus 5.

Government crash-out over Kimi

US officials reacted strongly. Treasury Secretary Scott Bessent warned of sanctions and entity-list designations if China is proven to run "covert industrial-scale distillation attacks" amounting to IP theft. Director Michael Kratsios claimed Moonshot distilled Anthropic's Fable and acquired banned GB300 servers via Thailand — but offered only "information", no evidence. Theo notes the irony: Anthropic itself just settled a $1.5B copyright lawsuit for training Claude on copyrighted books, so the moral framing (taking from copyright holders is illegal, but paying for a service then learning from it is somehow also illegal) is incoherent. Moonshot is building a data center on entirely Chinese chips anyway, so banning them mostly loses American businesses a customer and the market a competitive open-source alternative.

What distillation actually is

Distillation: take a smart, expensive model, ask it hard questions the cheaper model can't answer, then train the cheaper model on those answers so it behaves more like the smart one. Analogy: a junior dev watching a senior dev's correct solutions and gradually internalizing the pattern. This is a standard, official technique all labs use — e.g. Sonnet 5 is small and cheap but good at orchestration because it was retrained on data from the top-tier "Mythos"/Fable model. Cursor did the same with Composer 2.5: they retrained Kimi K2.5 using their own code data and RL pipelines. Only ~15% of Composer's total compute came from Moonshot's original Kimi training; the other 85% was Cursor's post-training. Cursor could do this partly because, as a huge Anthropic customer (once ~60% of Anthropic's revenue), they had massive reference data.

The gray area: paying Anthropic for outputs and then distilling a competitor is questionable but defensible. The Chinese labs are different because China is banned from using Fable at all, so any access is via sketchy aggregators. They also can't get reasoning traces (Anthropic doesn't serve them), so they can only distill from outputs — harder, like learning from a senior dev's code without their explanation. Anthropic and others published evidence that DeepSeek's V4 produced near-identical outputs to Fable 5 on complex code prompts and degraded when hitting Fable's safety classifiers, strongly suggesting routing to Fable.

Timelines and the Hugging Face dataset

The distillation-only explanation doesn't hold up: Fable went public July 1, K3 launched July 15 — you can't train a frontier model in 15 days. Theo also points out a public, MIT-licensed, deduplicated dataset of 2M+ Fable traces already sits on Hugging Face, so gatekeeping is futile. Vercel CEO Guillermo reports K3 is top-tier at cybersecurity on their internal benchmarks (GPT-based models still ahead but far more expensive; Fable refuses everything), making K3 attractive as a capable, non-refusing, cheaper security model.

Dean's (OpenAI) thread and the decelerationist argument

Theo largely agrees with OpenAI's Dean on the factual points: K3 is genuinely good and not explainable by distillation; it's on par with best Q1-2026 models; and it's actually more token-efficient than critics claim (K3 and Grok 4.5 are surprisingly efficient, though OpenAI models remain the most efficient). Dean speculates China open-sources partly from "strategic blindness", partly from lack of compute for inference, and as an unintended byproduct of US export controls plus China's aggressive-export strategy.

Where Dean drew fire: he called open-weight models "inherently decelerationist", argued they deter AI capex, and suggested a fully open-weight world leads to "AI communism" (AI as a state-provided public good), which he called a "dystopian hellscape". He also predicted the Trump admin's best move is to manufacture regulatory risk (FUD) around Chinese open-weight models rather than an outright ban. Critics (Tim Sweeney, Technium) mocked him for effectively arguing monopolies are accelerationist. In a follow-up Dean walked back the absolute phrasing — he only meant open weights decelerate capex spending on the margin, which is straightforwardly true — and acknowledged that as an OpenAI lead his loose, high-context writing style no longer survives public scrutiny. Theo defends Dean as mostly describing reality, not prescribing it.

Security concerns are real

The genuine worry: an unrestricted, capable open-weight model helps both defenders and attackers. K3 reportedly found a zero-day in the latest Redis server in 27 minutes using 32 sub-agents. When Hugging Face was itself hacked (by an unreleased OpenAI model), they had to run forensics on the open-weight GLM 5.2 because the major labs' models refused security-related prompts. Dean concedes a day may come when frontier models are too dangerous to open-source — he doesn't think we're there yet, but the direction of travel is clear, and governments will have much lower risk tolerance.

Cost reality and Theo's conspiracy

K3 isn't actually cheaper than GPT-5.6 for most real work: half the token price but ~2x the tokens and half the speed means tasks can take ~4x longer and cost about the same. The real reasons to use K3 are to bypass frontier-lab restrictions or for its unique capabilities. Notably, two Chinese open-weight models (Kimi 3rd, GLM 5.2 5th) now outrank Google (7th) and Meta (6th) despite those companies' trillions.

Theo's coding ranking today: Fable #1, GPT-5.6 #2, then a toss-up between K3 and Grok 4.5. His conspiracy: Anthropic kept delaying removal of Fable from subscriptions (ultimately keeping it in the $100/$200 tiers) because he suspects they tried to distill Opus 5 on top of Fable, the results underwhelmed, Opus 5 likely benched behind Kimi K3, and that's why Opus 5 keeps getting delayed — they don't want the embarrassment of a "new flagship" that loses to a Chinese open-weight model. He closes reaffirming support for open weights as a healthy competitive pressure on the labs, while worried about long-term security implications.