We audited our own MCP server. We failed.
Tool schemas cost more tokens than the work they describe. What we measured, and the two patterns we got wrong.
Winged Victory of Samothrace · Louvre · CC0the torso against the wind · 5.6°We settle what agents pay for, so we can measure what the money bought: which models finish the task, what they really spent, and where it went. Any evaluation we publish runs on tasks we synthesize. A private evaluation runs on a customer’s own cases and is never published.
Newest first.
Tool schemas cost more tokens than the work they describe. What we measured, and the two patterns we got wrong.
Measured savings with confidence intervals, and the cases where a router will not help you.
Ninety days of paid agent calls, described from the settlement side.
A routing benchmark run through a production payment pipeline, and the classifier we built from it.
Instruments that run on the gateway's own settled calls.
From settled calls, updated continuously.
The public feed of the gateway.
The same measurement on your workload. Never published.
Every candidate model, under a hard budget, priced per completed task.
Measured on your own tasks, at settled prices.