We deleted the LLM call and everything got faster, cheaper, and more honest
I write a blog partly powered by AI, so this might sound like heresy: the mark of a good engineer in 2026 isn't knowing how to wire in an LLM. It's knowing when not to. The instinct to reach for a model on every task with the word "text" in it is quietly making software slower, more expensive, and less honest — and the fix is often a function you could have written in an afternoon.
The framing: seasoning, not flour
Quinn Keast put it well back in 2023: AI should be used "like seasoning that brings out the flavours" of an underlying, sensible system — not the main course. I'd push the metaphor one step meaner: seasoning, not flour. Seasoning makes the dish; flour is the dish. Too much software right now is trying to make an LLM the flour — the load-bearing structure — for problems that a deterministic system should carry, with the model sprinkled on top only where fuzziness actually lives.
The number that should stop you
Here's the one I keep coming back to. A peer-reviewed study in JAMIA Open (November 2025) pitted plain regex against an LLM for extracting standardised scores from radiology reports. The accuracy was a statistical tie — regex 89.2%, LLM 87.7%, no significant difference. The speed was not a tie: regex ran in 0.06 seconds; the LLM took 1,687 seconds. That's regex being about 28,000× faster for the same result, on structured, predictable input. Not 28% faster. Twenty-eight thousand times.
For structured data, a regex isn't the compromise. It's the better answer — faster, free, and it returns the same thing every single time.
An LLM call is hundreds of milliseconds to several seconds, costs money per call, and can return a different answer to the identical input twice in a row. A deterministic function is single-digit milliseconds, free, and reproducible. On the right task, that's not a trade-off — it's a rout.
The reproducibility point is the sleeper
Cost and latency get the headlines, but determinism is the one that bites you later. A developer who replaced LLM-based code scanning with plain static analysis summed it up: "Same code. Five runs. Five different answers." (That's a self-reported blog case, so take the exact figures loosely — but the underlying complaint is real and I've lived it.) If your task needs an audit trail, or has to pass the same input to the same output every time, a probabilistic text generator is the wrong foundation no matter how clever it is.
A decision rule I actually use
Reach for deterministic code — regex, a rules table, a small classifier, static analysis — when the task is:
- Structured and predictable (parsing a known format, extracting standardised fields);
- Correctness- or audit-critical (you need the same answer every time);
- High-volume, low-margin (per-call model cost dominates at scale — the OCR world quotes clean-document pipelines at roughly $750/month traditional versus $50,000+/month with an LLM at real volume);
- On a tight latency budget, or fundamentally classification/routing rather than generation.
Reach for the LLM when the task genuinely needs it: fuzzy natural-language understanding, actual generation, or messy long-tail input a rule set can't enumerate — think degraded scans or handwriting, where LLM OCR genuinely beats the traditional engines. The best production pattern is the cascade: do the cheap deterministic thing first, handle the 90–98% of standard cases for free, and escalate only the low-confidence remainder to the model. That's what "seasoning" actually looks like in code.
The honest counterpoint
"Just use regex" is also a way to be wrong. The canonical mistake is parsing HTML with a regular expression — HTML isn't a regular language, and the internet has an entire genre of horror (and a famous Stack Overflow rant) about people who tried. Jeff Atwood's take is the balanced one: the rule isn't "never touch HTML with regex," it's telling real parsing (use a parser) apart from trivial string processing on known, limited input. Point a regex at open-ended natural language and you get the same failure in reverse: a brittle, ever-growing pile of special cases that an LLM would have shrugged off. Over-engineering with a model and under-powering with a rule are the same sin — using the wrong tool because it's the one you reached for first.
Why I care
I've written here before about using few tokens where few tokens do the trick. This is the same idea, one level up: sometimes the right number of tokens is zero. Every time I've deleted an unnecessary model call and replaced it with ten lines of boring, testable code, the result was faster, cheaper, and — the word I keep coming back to — more honest. It does exactly what it says, every time, and I can read it. That's not anti-AI. That's knowing what the AI is for.