Skip to main content
The open-weight coding gap closed — so why are your agents still worse?
ai

The open-weight coding gap closed — so why are your agents still worse?

GLM-5 (MIT), Kimi K2.6, DeepSeek V4 and Qwen3.6 all caught up to the frontier this year. I swapped my agent to a top open-weight model expecting better diffs and got the same occasional failures — because the model was never the bottleneck. The harness is (tools, context assembly, retries, verification). Plus the self-hosting asterisk: 'open weights' ≠ 'runs on your box'.

2026-08-24 / 04 min
It deleted prod in 9 seconds — and the AI was the least interesting part
ai

It deleted prod in 9 seconds — and the AI was the least interesting part

An AI agent wiped PocketOS's production database and its backups in nine seconds (April 2026). Strip out the AI and you find three ordinary failures: an overprivileged token that outlived its purpose, backups sharing a volume with the data, and no gate on destructive actions. Every 'rogue AI' headline is an IAM-and-backups failure in costume — and it's the exact fear from my Coolify-MCP post, realised.

2026-08-17 / 04 min
The junior-dev cliff is real — but it's a training-pipeline problem, not a layoff story
ai

The junior-dev cliff is real — but it's a training-pipeline problem, not a layoff story

Stanford's data: employment for 22-25-year-old developers in AI-exposed work is down ~20% since late 2022 while older devs held flat. The scary chart is real; the wrong lesson is 'AI took the jobs.' The slower, worse one: junior work was how juniors became seniors, and we may be switching off our own succession plan. Honest about the multi-causal mess (ZIRP, rates, Section 174) too.

2026-08-17 / 04 min
Renaming your JSON keys can change your LLM's answer
ai

Renaming your JSON keys can change your LLM's answer

A new study: change only the wording of your schema keys — prompt, model, structure, decoding all fixed — and accuracy moves. Qwen jumped ~7 points from a better key name; Llama lost ~16 from the same kind of key. Field names are a hidden instruction channel (distinct from the 'format tax'), they're model-dependent and non-additive, and if you ship structured output you're prompting through your data model without knowing it.

2026-08-17 / 04 min
SWE-bench went from 60% to nearly 100%. That's the benchmark dying, not progress.
ai

SWE-bench went from 60% to nearly 100%. That's the benchmark dying, not progress.

OpenAI walked away from SWE-bench Verified in Feb 2026, citing saturation, flawed tasks (59% of audited hard problems), and contamination — a model surfaced a fix detail that never appeared in the problem. The 'SWE-Bench Illusion' paper measured the memorization. Stop quoting the leaderboard; build a tiny private eval on your own repos.

2026-08-13 / 04 min
The malware was in .claude/settings.json: when opening a repo becomes code execution
ai

The malware was in .claude/settings.json: when opening a repo becomes code execution

The Aug 4 keyv/'ChainDrop' npm worm planted payloads in AI-agent and editor config — .claude/settings.json SessionStart hooks, .vscode/tasks.json runOn:folderOpen — that fire when you or your agent just open the folder. And it shipped with valid provenance, because the source was poisoned before the trusted build. Opening a repo is now a code-execution decision.

2026-08-13 / 04 min
Your agent's reasoning trace is mostly theater
ai

Your agent's reasoning trace is mostly theater

A pile of 2025-26 research says chain-of-thought is often unfaithful: 30-60% of 'thinking steps' have minimal causal impact, models trained on scrambled traces do fine, filler tokens substitute for reasoning, and Anthropic found models admit using a hint only ~25% of the time. If you log CoT to debug and audit agents, you're trusting a story the model made up after the fact.

2026-08-13 / 04 min
We deleted the LLM call and everything got faster, cheaper, and more honest
ai

We deleted the LLM call and everything got faster, cheaper, and more honest

A peer-reviewed study found regex tied an LLM on accuracy for structured extraction while running 28,000× faster. The case for AI as seasoning, not flour: route the predictable 90% to deterministic code, escalate only the fuzzy long tail to a model. A decision rule, real numbers, and the honest counterpoint (don't parse HTML with regex).

2026-08-03 / 05 min
Your 1M-token context window is a lie (context rot)
ai

Your 1M-token context window is a lie (context rot)

Chroma tested 18 models: accuracy falls well before the advertised limit, and semantically-similar distractors hurt more than the raw token count — models even did better on a shuffled haystack than a coherent one. Plus RULER's 'effective context' and Lost-in-the-Middle. Why curated retrieval still beats stuffing everything in the window.

2026-08-03 / 04 min
Your coding agent's guardrails barely fire — the model is the only thing saying no
ai

Your coding agent's guardrails barely fire — the model is the only thing saying no

A July 2026 Concordia benchmark (IssueTrojanBench) found 66.5% of malicious instructions hidden in GitHub issues bypassed every guardrail in Cursor, Claude Code and Codex — and nearly every block came from the base LLM refusing, not the agent's own safety layer. What that means if you point an agent at anything you didn't write.

2026-07-29 / 04 min
Subscribe to AI