Local LLM coding: Qwen vs frontier on real codebases
- https://www.youtube.com/watch?v=zPqcS5AvQvQ
- Original title: Local AI Coding is Finally Good Enough
An AMD-sponsored head-to-head of local Qwen models against Opus 4.7 on real production codebases (Excalidraw in TypeScript, Warp in Rust), each given one easy and one hard feature task pulled from actual repo issues. Verdict: frontier models remain clearly better, but local models have crossed the "genuinely useful" threshold — provided you prompt them with detailed specs, break work into small tasks, and accept roughly 5x slower completion. The target audience is developers whose code cannot leave the building (ITAR, HIPAA, finance/IP policies).
Why local at all
Cloud frontier models are cheaper and better thanks to heavy inference subsidies — if you can use them, do. But many devs legally or contractually can't: defense (ITAR), healthcare (HIPAA), hedge funds with "no code leaves the building" policies. Compliance paths (BAA, FedRAMP, GovCloud) exist but still require the company to approve provider, model, region and data flow — often a non-starter. For them the options are hand-writing everything or local models.
Setup
- Models: Qwen-3-72B-class MoE coder (80B params, ~3B active, quantized, CPU-offloaded) and Qwen-3.6-27B dense (quantized GGUF, fits fully in VRAM). Frontier baseline: Opus 4.7 — explicitly a reference point, not a fair same-hardware comparison.
- Hardware: AMD Ryzen Threadripper 9980X, Radeon AI Pro R9 700 (32 GB VRAM), 128 GB DDR5, Ubuntu 26.04, running llama.cpp (found faster than LM Studio etc.), ROCm acceleration, MoE CPU offload for the big model.
Excalidraw (TypeScript)
- Easy: highlighter mode for the free-draw tool. Both passed typecheck and both worked. Opus modeled
highlighteras a real property on the element data model (intent survives save/reload/export); Qwen 3.6 just created a normal free-draw element with big stroke width and low opacity — visually identical, but the "highlighterness" is lost after creation. Works, but weaker architecture. - Hard: five-pointed star shape (touches toolbar, element types, rendering, hit testing, collision, restore logic). Both got it working. Opus produced deliberate star-specific geometry and kept star collision separate from diamond math, but hijacked the "5" keybind from an existing tool. Qwen 3 Coder Next didn't steal a keybind (better UX call) but generalized diamond+star collision into a helper that always uses star points — so diamond collision can now run through star geometry. A real bug that the type checker can't catch: compiles, passes checks, looks perfect in the UI. Exactly the kind of subtle defect that compounds if nobody reads the code.
Warp (Rust)
- Easy: /clear-history slash command. Both passed cargo check. Opus produced the architecturally cleanest version — wired through existing workspace actions and confirmation dialogs — but it deletes the whole conversation instead of clearing history, i.e. failed the actual requirement. Qwen 3.6 did (roughly) the requested thing with clunkier UX (double-enter confirm, truncates the visible history), though the history reappeared after restarting Warp.
- Hard: command bookmarks (context menu, SQLite schema, persistence, side panel, re-run on click). Opus built all the modules including a feature flag, and bookmark/unbookmark worked, but it invented its own panel type instead of integrating with the existing left-panel state model — so the panel icon never appears and the panel is unreachable; also clicking a bookmark inserts the command instead of executing it. Qwen 3 Coder Next touched all the right areas but never compiled: 47 errors (wrong Warp UI APIs, type mismatches, missing enum variants, non-exhaustive matches, moved values), and eventually gave up, saying someone familiar with the codebase should finish it. A clear ceiling for the local model.
Conclusions
- Frontier models win — unsurprisingly — but the local Qwen models genuinely got useful work done, unthinkable a year earlier.
- Treat local models like frontier models of one-two years ago: very specific prompts, rigorous spec sheets, tasks broken into small sequential chunks. They do notably better on easy/pattern-following tasks than on architecture-spanning ones.
- Local runs took at least 5x longer than Opus. The workable pattern: run the model on a mundane task in the background while you work on the interesting one, then review and hand it the next task.
- If your code can't leave the building, local AI coding is finally a net positive for real work.