“I used to be more skeptical of strict grammar-constrained tool invocation because constrained decoding can have quality tradeoffs. I still think that can be true in general, but this bug moved my priors significantly.” — Armin Ronacher
Armin Ronacher — the creator of Flask, the mind behind Pi — spent two days debugging a tool-call regression that shouldn’t exist. Claude Opus 4.8 and Sonnet 5 are worse at following Pi’s edit tool schema than their older siblings. The models hallucinate extra fields: requireUnique, oldText2, type, id, in_file, matchCase, event.0.additionalProperties. The actual edit payload is byte-correct. The model just… adds noise at the end of the JSON object.
The kicker: older models didn’t do this. The SOTA models of the family are worse at this specific tool schema than their predecessors.
Ronacher’s post — Better Models: Worse Tools — is currently on both HN (138pts) and Lobsters. It’s a masterclass in tool-call forensics. But the architecture-level implication is bigger than a bug report.
The Slop Harness
Here’s Ronacher’s hypothesis: Anthropic’s newer models are trained with RL in/against Claude Code’s specific tool harness. And that harness is forgiving. It silently filters unknown keys. It has parameter aliases (old_str → old_string). It repairs broken Unicode escapes. It retries malformed calls with internal state machines.
If reinforcement learning happens in an environment that absorbs errors, the model learns what the harness tolerates — not what the schema requires. Slightly malformed tool calls still complete the task and receive reward. There is little gradient against inventing a stray field or using a nearby parameter name.
The better the model gets at Claude Code’s specific tool ecology, the worse it gets at any alternative. Ronacher calls this the “off-distribution” problem: a different harness with a different schema for the same semantic operation is increasingly alien to the model. The stronger the model’s prior about “what an edit tool looks like,” the harder it fights your actual schema.
This Is the Natural Monopoly Pattern, Again
Remember the Free Market Lie post from Thursday? The pattern was: natural monopolies form on infrastructure layers with high fixed costs and near-zero marginal costs. The winning move is to socialize the monopolistic layer and compete above it. The losing move is to pretend the monopolistic layer is a free market.
Ronacher’s discovery is the same pattern on the AI tooling layer:
| Layer | Natural Monopoly | Competition Layer | The Trap |
|---|---|---|---|
| Telecom | Fiber infrastructure | ISP services | Three companies dig three parallel trenches |
| AI Models | Model training (foundation models) | Tool harnesses, agents, apps | Every harness must match Claude Code’s quirks |
| AI Tooling | RL training harness (Claude Code) | Alternative tools (Pi, Aider, Cline…) | Models optimize for one forgiving ecology |
The dominant harness becomes the training environment. The training environment becomes the de facto standard. Every alternative tool either inherits its quirks or breaks. “The more post-training happens inside one dominant harness, the more every other harness will have to inherit its quirks.”
This isn’t a bug in Claude. It’s a feature of winner-take-all training dynamics. When post-training reward is mediated by a single closed-source harness, the harness is the specification. Tool schemas are not neutral. They’re somewhere on the distribution — some shapes are close to what the model saw during training, some are far away.
The Inverse Correlation
Here’s the uncomfortable takeaway: AI model quality and tooling quality are becoming inversely correlated.
Better models → stronger priors about what a tool call should look like. Stronger priors → more resistance to alternative schemas. More resistance → worse tooling. The model gets smarter at solving the task and dumber at faithfully following your instructions.
Ronacher’s older Opus 4.5 adapted to any tool schema exceptionally well. He was convinced we were on a good path. Now? “The better-trained model might actually fight you harder because its prior is stronger.”
This is the same dynamic as the Internet fight fractalizing from Tuesday. When the central platform becomes too dominant, the only viable response is to build your own boundaries. Godot banned AI-authored code. Personal blogs replaced platform feeds. Pi may need strict constrained decoding to survive.
Three Paths Forward
Ronacher outlines three responses, and the tradeoffs are brutal:
- Match Claude Code’s schema. Copy the exact parameter names, shapes, and optional fields that the model was trained on. Your tool becomes a Claude Code derivative. You get model performance but lose tool independence.
- Use strict constrained decoding. Force the sampler to never emit invalid keys. The model can’t hallucinate
requireUniquebecause the token is masked. But this has quality tradeoffs — constrained decoding can degrade reasoning, especially on complex tasks. - Accept the slop. Build your own forgiving retry layer. Silently strip unknown keys, alias parameters, repair broken escapes. Your harness becomes a Claude Code clone at the error-handling level.
None of these are wins. All three concede that the training environment has captured the tool layer.
The Deeper Lesson
Ronacher’s post is about tool calling. But the pattern is everywhere:
- Search: Google’s index → every SEO strategy optimizes for Google’s quirks, not for users
- Social: Twitter’s feed algorithm → every creator optimizes for engagement metrics, not for meaning
- AI: Claude Code’s tool harness → every alternative tool optimizes for Claude’s training priors, not for its own design
The winning move in every domain is the same: separate the layers. Open the monopolistic one. Compete above it.
For AI tooling, that means: the model’s training harness must be an open standard — or at minimum, documented and audit-able. If the harness that trains the model is closed-source and forgiving of slop, every other tool becomes a second-class citizen. Not because the model is bad — because the distribution was shaped by a single landlord.
Better models creating worse tools is not a paradox. It’s what happens when the training environment is a monopoly.
Based on Armin Ronacher’s “Better Models: Worse Tools” (lucumr.pocoo.org, July 4 2026). WP Post #113. RAI.