Agentic Data Engineering Resources

Resource

How to Improve Agent Productivity With Better Context, Fewer Tool Calls, and Faster Retrieval

Reduce tool calls, token cost, and latency by routing agents between indexed context and live system calls.

Pedro Lopez

July 14, 2026

Summarize with AI:

Agent slowness is not a model problem, and swapping the model rarely fixes it.

Your agent passed the demo, but in production, answers crawl in, token bills climb, and a question that should take one query fans out into five live API calls before the model says anything. The fix is to eliminate unnecessary tool calls, shape the context sent to the model, and choose the fastest retrieval path for each source.

What follows diagnoses the symptoms you actually see on production dashboards and routes each one to a concrete, repeatable fix you can ship without rebuilding the agent.

TL;DR

Agent productivity improves when speed, cost, and accuracy move together.
The biggest gains come from reducing unnecessary tool calls, shaping context before it reaches the model, and choosing faster retrieval paths.
Repeated reads should use pre-materialized or indexed context, while writes and second-fresh data should use live tool calls.
Airbyte Agents packages this hybrid pattern with a Context Store for indexed reads and Direct API execution for fresh state and writes.

What Does Agent Productivity Actually Measure?

Agent productivity is the output an agent produces per token, per second, and per dollar while holding a fixed reliability bar in production.

An agent that answers correctly but takes ten times longer is not an improvement, and LangChain's evaluation work makes the same point: a system that is 95% accurate but 10x slower rarely wins. The metric only counts when speed, cost, and accuracy move together, which means productivity must be tracked across several dimensions at once.

Tokens per task. Anthropic's multi-agent research found that token usage explains around 80% of performance variance, with tool calls and model choice as secondary factors.
Output token cost. Output tokens run 3 to 8x more expensive than input tokens across providers, so capping response length compounds savings on every turn.
Runtime context assembly. The dominant drain is assembling context live, one tool call per source, before the agent can reason.
Context length thresholds. Performance declines beyond about 32K tokens for Llama-3.1-405B and about 64K tokens for GPT-4-0125-preview, with context rot analysis flagging similar degradation near 30K tokens.
Tool-call depth. Past roughly 10 tool calls in a single task, accumulated context degrades performance by leading to worse selection and more retries.

These dimensions are tightly coupled in practice: a chatty tool inflates token counts, slows retrieval, and pushes the window toward thresholds where recall drops. That coupling is why the same complaint surfaces in four different shapes on production dashboards.

What Are the Most Common AI Agent Productivity Problems?

Most productivity complaints trace back to four recurring symptoms, each with a different root cause and a different fix. Before reaching for an approach, identify which one you actually have. The distinction between agent tools and agent connectors matters here because the same symptom can come from a chatty tool or a slow data path.

Symptom	Operator sees	Root cause	Primary fix
Repeated tool calls	Five round-trips per answer	Live assembly, one tool per source	Pre-materialize; batch fields
Slow retrieval	Seconds, not sub-second	Live APIs, pagination, big tool sets	Query a pre-indexed layer
Bloated context	Token cost climbs each turn	Raw API payloads flood the window	Filter and shape responses
Stale results	Outdated state in answers	Cached data behind a live event	Route to a live call

Two patterns make these symptoms harsher than they look on a dashboard. Tool calling fails between 3% and 15% of the time in production, so chatty agents stack retry loops on top of base latency, while bloated windows hit "lost in the middle" behavior where the model effectively uses only around 50K tokens of a 200K spec. Stale results compound both, since retrieval recall can decline over time with no alarm. The cheapest place to start is the loudest symptom: too many calls.

How Do You Reduce Unnecessary Tool Calls?

Cutting calls is the single biggest lever because each round trip adds latency, tokens, and a chance of failure. The goal is to preserve safe tool calls while removing those that exist solely to stitch data together. Three tactics do most of the work.

Design tools that fetch what a task needs in one shot instead of one call per field. Targeted reads beat list-everything queries: ranking relevant chunks rather than stuffing 200 unranked ones drops a query from roughly 50,000 tokens to 2,500 and latency from around 200ms to under 50ms.

Anthropic's Programmatic Tool Calling pushes this further by keeping intermediate results out of the model's context, with reported average token use falling by roughly a third on complex research tasks.

2. Load Tools on Demand at Scale

In Model Context Protocol (MCP) environments, a five-server setup can consume around 55K tokens before the conversation starts. Loading definitions on demand cuts token usage by around 85% in Anthropic's testing while lifting MCP evaluation accuracy on Opus 4 from 49% to 74%. Claude Code already defers MCP server tool definitions by default, so only tool names enter context until a tool is used.

3. Pre-Materialize What Gets Queried Repeatedly

When the same systems are queried repeatedly, assemble the data in advance. A pre-indexed layer collapses repeated calls from five or six live round-trip calls into one or two reads. The trade-off is worth stating plainly: pre-materialization adds refresh complexity and a freshness lag, so it fits data that tolerates minutes-old state, not payment confirmations that must reflect the last few seconds.

Once the call count is pruned, the next bottleneck is the data each remaining call returns, and that is a packaging problem.

How Do You Package Context So the Model Uses Fewer Tokens?

Shaping data before it enters the window beats cleaning it inside the window. The structural win is upstream, in the shape of the payload and the policy that governs session history.

Return Typed, Filtered Responses

Return typed, filtered JSON that the model can sort and join rather than raw API dumps. Compact JSON alone cuts payloads by 40 to 50% versus formatted JSON, and compressing tools, summarizing telemetry, and trimming context can cut a single request by about 90%.

Cap output length too: Claude Code enforces a 25,000-token hard cap on tool output for exactly this reason.

Trim History and Cap Output

For long sessions, compaction and trimming keep the window healthy. Summarize at intervals rather than waiting for the ceiling: Claude Code auto-compacts after 95% of the window, while LangChain's Deep Agents trigger summarization at 85%. Keep the last N turns intact and archive the rest behind references. These tactics help, but they treat a symptom; the durable fix is shaping the data before it ever reaches the window.

With payloads tight and sessions trim, the remaining variable is which retrieval path each source should use.

How Do You Make Retrieval Faster Across Multiple Systems?

Retrieval speed comes from choosing the right path for each source. Two principles separate fast pipelines from slow ones.

Query a Pre-Indexed Layer for Repeated Reads

For reads that repeat across the same systems, query a pre-indexed layer instead of paginating live APIs. Where semantic match drives the query, agentic vector search is the fast path. Reserve direct live calls for freshness-sensitive reads, and let your data freshness strategy decide which reads qualify. A two-mode pattern, indexed plus live fallback, removes the routing decision from the prompt.

Set Realistic Cross-System Latency Expectations

Single-corpus vector retrieval is fast: Qdrant has shown p99 around 20ms on 1M vectors in production. That speed matters less than operators expect, because retrieval is rarely the bottleneck. In a measured RAG pipeline, retrieval took milliseconds while inference took far longer. The real cost is sequential round-trips: five API calls at around 200ms each block for about a full second, and multi-step chains have been reported at 12 to 40 seconds end-to-end.

Knowing the paths exist is one thing; knowing which to use per source is the harder call.

When Should an Agent Call a Live Tool Versus Query Pre-Materialized Context?

Use this per-source rule to decide which path each data source takes at runtime. Treat the three dynamic context retrieval patterns, pre-indexed, just-in-time, and hybrid, as your menu of options.

Condition	Live tool call	Pre-materialized retrieval
Freshness need	Last few seconds (writes, payments)	Minutes-to-hours-old is fine
Query frequency	Rare or unpredictable	Repeated across sessions
Sources per answer	Single source, single record	Multiple sources joined
Latency budget	Round-trip is acceptable	Sub-second required
Operation type	Create, update, delete	Search, read, filter, sort

The durable rule: repeated cross-system reads belong in a pre-materialized layer, and writes plus second-fresh reads belong on a direct live call. Caching's cost advantage becomes material above roughly 1,000 sessions per day on the same data, and a sub-500ms budget rules out live calls in the critical path entirely. Most real agents end up hybrid, choosing between two paths automatically, one source at a time. The cost picture follows directly from those routing choices.

How Do Faster Retrieval and Fewer Calls Lower Agent Cost?

Fewer round trips cut both latency and token spend. Shaped context cuts the per-call token load. Pre-materialized reads remove repeated pagination and the redundant fetches that inflate every conversation turn.

The savings track the tactics directly. Semantic caching for high-repetition workloads has reduced costs by up to 73%. In Airbyte's launch benchmark across Gong, Linear, Salesforce, Slack, and Zendesk, pre-materialized reads delivered up to 80% fewer tokens overall, with savings reaching up to 90% on the highest-savings source. What makes those numbers reproducible is the architecture underneath.

How Do Airbyte Agents Improve Agent Productivity?

Airbyte Agents pre-materialize operational data from your connected SaaS sources into the Context Store, a managed, searchable replica of agents' queries instead of stitching context together at runtime. The work of assembling context happens in advance, so a question that would take five or six live calls collapses to one or two.

Execution runs in two modes, which the agent chooses automatically. Search queries the Context Store for fast indexed reads, while Direct calls the live API for writes and second-fresh state. One auth flow covers 50+ connectors, and the same Context Store is shared across the Web app, the Agent MCP, the Agent SDK, the Agent CLI, and the API. This allows Airbyte Agents to make around 40% fewer tool calls with sub-second search across replicated, pre-indexed enterprise data.

What's the Fastest Way to Make a Production Agent More Productive?

The agents that scale are not the ones with bigger windows or smarter models; they are the ones that stop assembling cross-system context at runtime. Once you route repeated reads to a pre-materialized layer, shape responses into typed and filtered JSON, and reserve live calls for writes and second-fresh data, latency, token spend, and accuracy move in the same direction. The architectural decision precedes the model decision.

Airbyte Agents packages that pattern end-to-end: a managed Context Store for indexed reads, Direct API execution for writes and fresh state, two-mode routing the agent handles on its own, and one auth flow across 50+ agent connectors. The same store powers Agent MCP, Agent SDK, an MCP Gateway, and an Agent CLI, so the path you start on is not the one you have to stay on.

Get a demo to see how it fits your stack, or try Airbyte Agents and ship faster, cheaper agents this week.

Frequently Asked Questions

Does Reducing Tool Calls Hurt Data Freshness?

No, as long as you route freshness-sensitive reads to a direct live call and pre-materialize only the data that tolerates a short lag. The point is to match each source to the right path, not to eliminate live calls. Writes and second-fresh state always go live.

Will a Bigger Context Window Fix Bloated Context?

No. A larger window delays the failure rather than fixing it, because accuracy still drops as the window fills with raw, unshaped data, and models effectively use only a fraction of a large context window. Shaping context before it enters the window is the durable fix.

Can a Non-Developer Improve Agent Productivity in Claude or Cursor?

Yes. Connecting the agent to a pre-materialized Context Store via an MCP-compatible client like Claude or Cursor means it queries unified data rather than chaining live lookups, with no infrastructure code required. The agent gets to the answer instead of fishing across systems.

What's the Difference Between Caching and Pre-Materializing Context?

Caching stores recent responses to avoid repeat fetches after the first query. Pre-materializing builds a unified, searchable replica across sources before the session begins, so cross-system reads resolve in a single query rather than several round trips, even on a cold start.

Try Airbyte Agents

Airbyte connects your agents to all of your data and assembles context before they run. Build agents that actually know your business.

Try it free Talk to sales

How to Improve Agent Productivity With Better Context, Fewer Tool Calls, and Faster Retrieval

Related posts

Try Airbyte Agents