26 Minutes vs 33 Seconds: What Production Data Tells Us About Agentic AI

The paper Perplexity co-authored on its own agent product opens with a clean pair of numbers. Across 10,000 matched query pairs from people who use both products, "Computer performs 26 minutes of autonomous work per user session, versus 33 seconds for Search" — a 48× ratio. From there the authors model a matched task falling from 269 minutes to 36: 87% of the time saved, 94% of the cost. The paper lists three of its four authors at Perplexity, both products compared are Perplexity's, and the preprint is not peer-reviewed. That is a reason to read the tables rather than the abstract, not a reason to discard the numbers.

The 48× is a measurement; the 87% is a model that fixes human oversight at ten minutes per task, and nothing in the paper measures whether that is true.

What the paper measured

The design is sound for what it measures. The authors took users of both products, embedded their opening queries, and kept pairs with cosine similarity above 0.99 — the same task, typed into both products by the same person. The Computer side had to invoke at least one "do" tool (code execution, browser actions, file creation, external API calls), so a Computer session that only searched was excluded. The window ran from 27 February to 27 May 2026, starting two days after Computer launched.

The 26 minutes and 33 seconds are means of wall-clock execution; the medians are 9 minutes and 14 seconds. Computer's wall-clock is capped at three hours per session, and footnote 4 says parallel execution means it underestimates total machine time. A Computer session averaged 5.3 queries to Search's 2.8 (full sample, opening query included).

Quality is a next-turn proxy. Each response is scored from what the user does next — re-asks, corrections, retries — in multi-turn sessions only. Mid-plus-high dissatisfaction ran 1.3% on Computer against 2.9% on Search; any level, 10.8% against 16.6%. That is not an accuracy rate. On my reading of the definition, a flawed artifact accepted without a corrective turn scores as satisfied; PPC Land raised the same point in June. The paper's own first limitation is that the three-month window is an early-adoption period skewed to power users and paying subscribers.

What the human did next

The follow-up taxonomy is the more useful table. On a 1,000-pair subsample where both sessions ran to multiple turns, the authors classified all 15,507 follow-ups (7,093 Search, 8,414 Computer) with an LLM into ten categories. The rows the argument turns on:

No single category moves two percentage points. The paper's reading: "Search turns reflect shorter digest-and-execute loops, whereas Computer turns reflect longer review-and-extend loops," and, in the discussion, "the user's role shifts from operator to supervisor, reallocating time toward higher-order work such as direction, verification, and task extension."

Fair, but look at what the table does and does not show. The mix of what the human asks for barely changed. The object of each turn did. A verification follow-up on Search interrogates a 33-second answer. The same category of follow-up on Computer interrogates an artifact assembled from bash, write, edit and browser calls over 26 minutes. The paper knows what those steps cost a human to do: Table 3 prices a bash call at 5 minutes, a write at 15, an edit at 10, a browser task at 10. What they cost a human to check, it prices at a flat constant — which is the next section. Checking that artifact needs a done-check the agent cannot fake, and a count of turns cannot tell you how long the check took.

(One precision note, my arithmetic: this subsample averages 8.4 Computer follow-ups per session to 7.1 for Search. Those are follow-ups only, in pairs where both sides went multi-turn; they are not comparable to the 5.3 and 2.8 above, which come from the full sample and count the opening query.)

The ten minutes

Section 7.1 builds the 269-minute baseline by summing each Computer session's "do" calls at Table 3's rates — the manual work a Search user would still have faced after getting the answer. The Computer side gets one sentence: "Computer's human time is fixed at 10 minutes of oversight per task (e.g., writing the prompt and reviewing the output)." The same ten minutes whether the domain's average Computer session runs 15 minutes (Consumer Goods) or 38 (Programming). Ten minutes that also has to cover writing the prompt. Add it to the 26 minutes of machine time and, by my arithmetic from the paper's components, you have the paper's 36.

The authors are not hiding this. Footnote 10, attached to a separate fixed-cost proxy (query characters per session), reads: "This approach does not capture the verification cost for Computer outputs so its fixed cost might be underestimated." Their fourth limitation lists human supervision time among the assumed inputs and says the absolute magnitudes should be read as approximate. The LLM-based cross-check (227 minutes against 269) re-estimates the Search-plus-human side from query text; it is not a measurement of the ten.

They also ran the sensitivity analysis. The cost advantage survives the oversight assumption inflated 26×, to 260 minutes; the time advantage survives 24×; in the tightest domain, 12× and 11×. Read from the paper's end, that is robustness. Read from the other end — again my arithmetic from the paper's components — 26 minutes of machine time plus 240 minutes of oversight is about 266 minutes against a 269-minute baseline. The time advantage "survives" at the point where it is roughly zero. Both readings are the same fact. On the paper's averages, the purchase is safe across the whole range; the saving is anywhere between 87% and nothing, and the deciding variable is the one nobody measured.

Buy on the speed number, plan on your own

The decision I'd make, and one you can disagree with: don't plan team capacity on the 87% until you have timed a competent reviewer verifying a handful of completed agent artifacts in your own domain. Buy the tool on the speed number if you like; budget the review on your number, not the paper's ten minutes.

The obvious objection: the sensitivity analysis already covers oversight up to 240 minutes, so why measure? Because it covers the purchase, not the capacity plan, and not the question of who does the reviewing. At 240 minutes of oversight the tool still pays for itself on average, and a human still spends four hours on the task — checking instead of doing. Planning on review time rather than the vendor's saving is a position I already hold from coding telemetry; this is the same position with a vendor's own knowledge-work tables behind it.

Three moves, none of which needs a number the paper doesn't have:

• Time the review per artifact for the task types you intend to delegate — a drafted report, a file edit, a browser workflow — with someone qualified to sign it off.
• Plan capacity as artifacts × measured review minutes, not 269 → 36.
• Decide up front where the freed time lands: spread across the team, or concentrated on the one reviewer the artifacts now flow through. The paper's "operator to supervisor" is the middle-management argument with a dataset behind it, and a supervisor is a job, not a saving.

My own expectation, labelled as such: measured oversight for artifacts built from write, edit and bash calls will land well above ten minutes, because checking work priced at 5–15 minutes a step is not a domain-independent constant. I could be wrong. That is what the measurement is for.

Perplexity has the logs. Measured human time per delivered artifact, by domain, from the same 10,000 pairs, would replace the constant and settle this either way. Until then the paper has measured how much the machine does and assumed how much the human still has to.