Blog

AI Agents

Usage Caps Are a Confession, Not a Strategy

Why enterprise AI usage caps treat the symptom instead of the cause: most agent spend buys fetching and re-sending context, not reasoning.

AI AGENTS

August 27, 2026

Davin Chia

Summarize with AI:

Usage Caps Are a Confession, Not a Strategy

Uber burned through its annual AI budget in four months, then capped employees at $1,500 a month for coding tools. Tesla set their cap at $200 a week. Both companies run world-class infrastructure teams, which makes it clear that managing AI spend is a problem even the best engineers struggle to solve. Usage caps aren't a solution to AI spend — they reveal a problem, and show even the best companies haven't found a real solution yet.

You're paying for fetching, not thinking

It's quite apparent that we are approaching agentic architecture the wrong way, and that is causing the incredible spikes in costs, mostly from context inputs, not actionable outputs. As an engineer, I love hard numbers, so here are some of my favorite ones backing this up:

Vantage measured agent coding sessions at roughly a 25-to-1 input-to-output ratio, with input tokens around 85% of session cost.
A University of Michigan, Stanford, and Microsoft study of SWE-Bench agents found input tokens were over 97% of solving cost.

You're not just wasting money; you're wasting context. Chroma found every frontier model it tested lost accuracy as context filled. So not only are you making your agents more expensive, you're also making them dumber.

You're spending your AI dollars on the part that isn't even intelligence. Less than 3 to 15 cents of every dollar buys reasoning. The rest buys fetching, packaging, and re-sending context.

Anatomy of the waste

You may be wondering where all this input volume comes from.

It starts with conversation history, fetched over and over. A 50-turn session doesn't send 50 messages. It re-sends an accumulating transcript, which is why cost grows quadratically with session length.

Tool definitions are also a major cause of bloat, loaded before each agent action. GitHub's official MCP server ships roughly 17,600 tokens of tool definitions per request. One independent developer measured a 93-tool server at 55,000 tokens before the agent said a word.

APIs aren't built for meaning-based agentic search, which results in over-fetch to find the relevant information. The agent pages through fat payloads to find one record, and every page rides along in context from then on.

And then there's retry thrash, when stale or fragmented data sends the agent back for another pass. Suffice it to say, input volume is a serious issue and needs to be optimized if we ever hope to reign in our AI spend.

Why the caps can't work

Caps don't distinguish between wasteful token consumption and productive usage. It's a cost control measure that helps the balance sheet in the short term, but doesn't improve an organization's long-term AI strategy.

Falling prices don't rescue the math either. Per-token prices have been dropping roughly 10x per year. But enterprise LLM spend went from $3.5 billion to $8.4 billion in six months, per Menlo Ventures. This is a sign that the problem starts before tokens are even consumed.

Alex Karp went on CNBC in July and said "something has gone completely wrong" with token economics, and the diagnosis is correct. The treatment is where we need a different approach.

It goes back to engineering fundamentals. When a resource is expensive, I find a way to use less of it. Model prices may fall, and cheaper options may emerge. But I don't want to rely on market forces when there are architectural solutions available.

Fixing the plumbing

So, what is the architectural solution? It derives from trends that already exist in our industry, and is showing great promise in early implementations.

Here's how we designed it at Airbyte.

The principle is pre-assembled context: do the discovery work before the agent runs, so it queries an index that already knows your systems instead of exploring them at runtime. Five exploratory calls collapse to one search and one targeted fetch.

I'll share some externally validated examples to show that we aren't the only ones thinking about this approach.

Anthropic's engineering team cut a multi-tool workflow token's consumption by 98% by mediating data access.
GitHub's engineers cut a similar workflow's token burn by 62% via pre-fetching.

Without bragging (too much), I have to admit I am pretty excited about our results so far. Our benchmark harness, which compares the Airbyte MCP against native vendor MCPs on identical prompts, shows up to 80% fewer tokens and 40% fewer tool calls, with results ranging from 90% savings on Zendesk down to 16% on Salesforce.

That 16% might seem insignificant, so why are we bringing it up? Salesforce's SOQL already lets callers request exact fields, so there's less waste for pre-assembled context to remove. Our relatively minor optimization here actually reinforces my argument.

The bill is set by access patterns, and when the access pattern is already good, there's less for us to remove.

Yes, basically all of these benchmarks, including our own, are self-measured. None of them have been reproduced. But if you're interested in reproducing ours...

Check out our harness here. I'd love to chat with anyone who finds a discrepancy or an improvement.

The reframe

If there's one takeaway from this, it's that I'd like the discussion to go from pricing to engineering. AI is expensive, but it's also a productivity multiplier. We can't afford to cap its usage for our employees, but we also can't just let costs spiral. Engineering encompasses many objectives, but designing more efficient systems is certainly a huge part of the job description. That's what we are doing at Airbyte, and I encourage you to test out what we're building.

Integrate with 700+ apps using Airbyte

Move data from 700+ sources into warehouses, lakes, and beyond. Set up pipelines in minutes with pre-built connectors and the Connector Builder.

Try it free Talk to sales

About the Author

Davin Chia

Davin Chia is an Engineering Lead at Airbyte, leading Cloud, Infrastructure & Tooling.

Integrate with 700+ apps using Airbyte

Try Airbyte for free

Usage Caps Are a Confession, Not a Strategy

Integrate with 700+ apps using Airbyte

Integrate with 700+ apps using Airbyte

Related posts