The 15-Tool-Call Rule: Where Agent Quality Falls Off a Cliff
Practitioner consensus puts the cliff around fifteen tool calls per prompt. Here's why agents degrade past that, and the three operational rules that keep them on the safe side.
Writing about AI, infrastructure, and what actually ships.
total 149 · ~11.5h read · updated Oct 02
Practitioner consensus puts the cliff around fifteen tool calls per prompt. Here's why agents degrade past that, and the three operational rules that keep them on the safe side.
Anthropic's multi-agent Research feature beat single-agent Opus 4 by 90.2% — at 15× the token cost. Every documented production swarm runs on rails. Here's the topology decision framework before you commit.
Agents over-refactor stable code without a safety net. Feathers' characterisation-test technique — write tests for current behaviour before changing anything — is more important than ever. The agent itself is the perfect characterisation-test-writer.
Karpathy named one mode. Willison named the other. Most 'AI failed in production' stories are actually 'we promoted a vibe-coded prototype without transitioning into the production discipline.'
METR ran the experiment. AI made experienced developers 19% slower — and they reported feeling 20% faster. The week-6 dip is the bottom of a documented J-curve. Most pilots get cut here. The right ones don't.
AI is making junior output look senior-level while preventing junior skill from forming — and the Stack Overflow collapse just removed the ambient learning layer that used to catch the deficit. Three interventions that work.
ReAct gave us a three-step loop. Production hardened it into five. The two new steps — Plan and Verify — are where everything that goes wrong, goes wrong. And the field has now named the worst offender.
Three open-source extractions converged on the same five layers. The architecture isn't a vendor narrative — it's a discovered structure. Here's the decision rule that keeps you from over-engineering it.
AI-reviews-AI looks like a control. Under MAS, the EU AI Act, and any reasonable audit, it isn't. Here's why your compliance team won't accept it — and the compensating controls that actually work.
GitHub said 55%. Then they ran the enterprise RCT and got 8.69%. Faros's two-year telemetry shows throughput up 66% and incidents up 243%. The honest net is 1.2–1.5×. Plan your team capacity accordingly.
Most teams plateau at Stage 2 because they confuse 'we built skills' with 'we have a working AI engineering culture.' Here's the 5-stage diagnostic — and the moves that get you from Individual to Distributed.
I publish Claude Code skills and install other people's. Then Snyk audited 3,984 public ones: 13.4% had critical vulnerabilities, 76 were confirmed malicious, and ClawHavoc is the scarier story underneath. Here's the supply-chain hygiene I now refuse to skip.