|8 min read
How to Measure the Human Cost of Agent Work
A measurement protocol for human review, rework and agent spend that keeps rejected attempts in the calculation.
[AI & Data][Metrics & Analytics]
10 posts
A measurement protocol for human review, rework and agent spend that keeps rejected attempts in the calculation.
Perplexity's own data shows its agent does 48× more machine work per session than its search product — but the 87% time saving rests on ten assumed minutes of human oversight, and that is the number a buyer needs.
A useful agent acts within delegated authority, prepares consequential actions for review, and asks again when the action changes.
Anthropic let Claude run a real shop for a month. It sold metal cubes at a loss, invented a Venmo account, and claimed to wear a blazer. The 'AI department that works while you sleep' is a genre — here's where it actually breaks.
ReAct gave us a three-step loop. Production hardened it into five. The two new steps — Plan and Verify — are where everything that goes wrong, goes wrong. And the field has now named the worst offender.
The prototype-to-production gap for AI agents isn't technical — it's governance. Most organisations have nothing in this layer. The companies that build it first win the enterprise market. Everyone else stays in pilot purgatory.
AI agents don't fail loudly — they degrade silently, returning 200 OK while the damage compounds. Inside the $47K loops, NOHARM omissions, and the engineering discipline rebuilding observable failure.
AI is silently promoting every knowledge worker to middle management — without the title, the training, or the pay. This is what that shift actually looks like from a Singapore desk.
When orchestrating 4-5 Claude Code agents across projects, I was losing track of progress and cost. A 2-3 day build with Claude Code itself led to Gluon—an open-source orchestrator that treats AI agents like team members.
I've built the kind of agent framework AgentKit competes with. So when OpenAI shipped it two years "late," I knew exactly which problem they were actually solving — and which one they weren't.