The open-weight coding gap closed — so why are your agents still worse?
The open-weight coding models caught up this summer. GLM-5 landed in February — MIT-licensed, and posting SWE-bench numbers a hair behind Claude Opus. Kimi K2.6, DeepSeek V4, Qwen3.6 Max all clustered within a few points of the proprietary frontier. So I did the obvious experiment: pointed my agent at a top open-weight model instead of my usual one, and waited to get better. I didn't. And that non-result is the actual story: for most of us, the model was never the bottleneck. The harness is.
Where things actually stand
First, a reality check on the hype, because the internet is drowning in AI-generated "model news" with invented version numbers. Sticking to what's genuinely confirmed: GLM-5 (Z.ai, Feb 2026, MIT license, a 744B/40B-active MoE) hit around 77.8% on SWE-bench Verified — top open-weight on the public indexes at launch, within striking distance of the best closed models. Kimi K2.6 (Moonshot, ~April 2026, a 1T-param MoE) and DeepSeek V4 and Qwen3.6 Max all sit "just behind top closed models" on the credible leaderboards. Parity-ish, not identical — and I'd treat any specific ranking as a weekly snapshot, because these boards reshuffle constantly and half the "GLM-5.3 / Kimi K3" chatter you'll find is content-farm fiction.
But the point of this post isn't the leaderboard. It's what happened when I swapped.
The swap that changed nothing
I run Claude Code as my daily driver and OpenCode for overflow, so trying a different model underneath is a five-minute change. I expected the frontier-vs-open gap to show up as worse diffs. Instead the agent was worse in the same ways it's always occasionally worse: it lost the thread on a multi-file change, re-read files it already had, retried a malformed tool call, missed context that was one directory over. None of that is the model failing to reason. It's the scaffolding around the model — how tools are defined, how context is assembled and pruned, how retries and verification work — doing a mediocre job of setting the model up to succeed.
A decent model with a great harness beats a great model with a bad harness. If upgrading the engine didn't help, the engine wasn't the problem — the chassis was.
This isn't just my anecdote. Anthropic's own engineering writing on context engineering makes the case plainly: agent performance hinges on curating the smallest set of high-signal tokens, on tool design, on just-in-time retrieval — not on raw model size. Sourcegraph frames it the same way: "the model is no longer the only bottleneck; the pipeline that feeds it matters just as much." The practitioner consensus that's formed in 2026 is that harness quality is where the delta lives. (With a nice twist: minimal, low-ceremony harnesses often beat feature-heavy ones — it's harness quality, not harness complexity.)
The self-hosting asterisk
"Open weights" also doesn't mean what people hope. These frontier open models are enormous mixture-of-experts networks — Kimi K2.6 at a trillion parameters wants hundreds of gigabytes of VRAM even quantized; DeepSeek V4 is bigger. That's a multi-GPU cluster or a rented inference endpoint, not your laptop and not a single 24GB card. What "open weights" actually buys you is real but different: portability (pin a version, no silent model swaps under you), price competition (multiple providers hosting the same weights), privacy and control (run it inside your boundary if you have the hardware, or a private endpoint if you don't), and an MIT license you can build on. It does not buy you a free frontier model on hardware you own.
What I actually changed
Not the model. After the swap that did nothing, I put the time where it pays: better tool definitions, tighter context assembly (feed the agent the three files that matter, not the twenty that don't — my own "context rot" post is the receipt for why), a verification step that re-checks the diff, and a retry that doesn't just resend the same broken call. That moved the needle in a way no model swap did. So the practical advice is unglamorous: stop chasing the leaderboard's current #1, benchmark whatever you use against your own repo, and spend your energy on the scaffolding. The open-weight models are good enough now that they're no longer your excuse. The harness is where your agent is still losing.