Tool calling is supposed to be the hard part that got solved. Every major model provider ships it, a common protocol interface has emerged for exposing tools to models, and wiring an agent to an external system is now a weekend task rather than a quarter of work. So a reasonable engineer might conclude that reliable agents are mostly a matter of connecting enough tools. That conclusion is where production teams get burned. The interface for calling a tool is settled, but an agent's reliability depends far more on the data behind those calls than on the calls themselves. This article argues that most agent failures are data-layer problems wearing an interface-layer disguise and examines the evidence.
TL;DR Tool calling is settled infrastructure: a standard way for a model to invoke external capabilities and receive typed results. Production failures commonly involve context bloat, incorrect tool selection, cross-system joins, and authentication. Benchmarks show that frontier models fail on most multi-step tool-use and enterprise-data tasks. Pre-materialized context lets agents query unified data instead of assembling it on a call-by-call basis. Want to see this in practice? Get started and connect your first source in minutes.
What Does Tool Calling Actually Give an Agent? Tool calling gives a model a standardized way to invoke external capabilities and receive structured results. Results come back typed rather than as prose the model has to interpret. Model Context Protocol has become the interoperable interface for exposing tools across clients. This is the layer that turns a chat model into something that can act.
Most agent integrations shipping today reduce to exposing tools over a shared protocol. Practitioners document provider-to-provider differences in tool-calling behavior that make it hard to guarantee consistent agent loops. Protocol connectivity is largely settled, though OAuth lifecycle management, rate limits, and compliance all sit outside the protocol. Production reliability therefore depends on data quality, OAuth management, rate-limit handling, and compliance under production load.
Why Does Tool Calling Break Down in Production? Incident reports, issue trackers, and community threads document the same four patterns. Each one traces back to the data and context that an agent works with, rather than to the mechanics of the call itself.
Tool definitions crowd out the context window Anthropic's engineering write-up on advanced tool use finds that a catalog of 50 or more Model Context Protocol tools consumes roughly 72K tokens before the agent does any work. The scaling problem becomes concrete when connecting five MCP servers , each with 30 tools. At 200 to 500 tokens per tool description, this adds 30,000 to 75,000 tokens of schema overhead that reloads on every request. A gateway can route schemas and reduce per-request overhead, but it cannot eliminate that overhead because the schemas for the tools the agent actually uses still have to reach the model.
Selection accuracy falls as the toolset grows Anthropic's guidance on context engineering suggests teams should not expect an agent to choose correctly when a human engineer cannot definitively identify the right tool. The signature symptom is the hallucinated tool call , where the agent invokes a tool that does not exist, and it is the strongest signal that context bloat has pushed the model past its reliable operating range. Practitioner testing on a large public tool server has reported selection accuracy sliding from the mid-90s with a focused toolset toward the low-70s with a full catalog loaded. Nothing changes except the size of the catalog.
Runtime assembly cannot answer cross-system questions A Salesforce opportunity, a HubSpot contact, and a Stripe invoice are usually the same customer relationship, and nothing at the tool layer says so. A revenue-operations team will tell you that a closed deal in one system does not map cleanly onto a converted customer in another. An agent with three working MCP connections can fetch all three records and still guess at how they relate. Finance teams define revenue through business rules that no tool schema captures. The data must encode the relationship before the agent reasons over it, which makes this a job for an agentic data platform rather than a prompt.
Auth and payload failures compound over long sessions A parallel-refresh race condition can occur when ten concurrent tool calls hit an expired OAuth token . The provider issues a new token to the first refresh request and revokes the old one. The other nine fail with invalid_grant, and the connection can break permanently. Payload size poses a separate long-session risk, since single-tool responses of roughly 50 to 100 KB can drive integrated development environment (IDE) clients to full CPU usage and stall the whole session until a hard restart. Data freshness adds a third source of session failure. Stale inventory data leaves a booking agent showing rooms that hotels can no longer honor, and the agent has no way to verify that the references are current.
What Do Benchmarks Say About Tool-Calling Reliability? Benchmarks test tool-agent-user interaction, enterprise-data queries, and multi-agent traces under controlled conditions. Read together with practitioner measurements of tool selection, they show that tool-calling agents remain far from dependable on the kinds of tasks production teams care about.
Benchmark What It Tests Result τ-bench (arXiv:2406.12045) Tool-agent-user interaction on retail and airline tasks gpt-4o around 61% pass on retail, 35% on airline; consistency across repeated trials drops below 25% DAB benchmark Data agents on real enterprise data, including multi-database joins and messy keys Best frontier model around 38% pass rate across 50 trials MAST (arXiv:2503.13657) 7 open-source multi-agent frameworks, 1,600+ annotated traces Roughly 41% to 86% failure rates across 14 distinct failure patterns
The DAB benchmark matters most for teams building on business data, because it is among the first to test agents against real enterprise patterns rather than hand-crafted endpoints. The math behind these numbers explains why interface-level improvements plateau: a 95% reliable step sounds safe until you chain twenty of them, at which point the whole chain succeeds only about 36% of the time. That shortfall does not close by adding more tools, so reliability has to come from removing steps and shrinking what each remaining step has to get right.
What Has to Surround Tool Calling for Agents to Be Reliable? A production agent needs several things the tool interface does not provide on its own. Each addresses a specific way that runtime assembly breaks down.
Governed execution through approval gates and call-level policy, so risky operations pause for review. A command-line interface that scripts these policies into continuous integration keeps the rules consistent across environments. Permission-aware data access , so the agent sees only the data a given user is allowed to see, with freshness guarantees that stop it from treating stale records as current.Observability and traceability, so an engineer can reconstruct why a given call happened and where a session went wrong. Unified cross-system context, so the three records that all mean "Acme" resolve to one customer before the agent reasons over them. Context engineering , the craft of curating which tokens reach the model on a given turn, addresses the symptoms in the first two patterns above. Curating tokens at the prompt layer is a different problem from guaranteeing that the data behind those tokens is fresh, governed, and joined. The prompt layer decides what the model sees; the infrastructure layer decides whether what it sees is true. The alternative to runtime assembly is to join, reshape, and unify data from scattered sources before the request arrives, so the agent queries a pre-built context layer instead of reconstructing everything call by call. The two architectures diverge across the dimensions compared below.
Dimension Runtime Tool-Call Assembly Pre-Materialized Context Latency Each source adds a round trip before reasoning starts Single query against a pre-indexed layer Token cost Tool schemas reload on every request Typed results scoped to the question Cross-system truth Agent reconciles records across systems on the fly Agents reason across unified records at query time Failure surface Rate limits, token expiry, and partial responses land mid-task Upstream systems handle freshness; live calls reserved for writes
Whether that upstream work is worth carrying depends entirely on the shape of the workload, which is the next thing to pin down.
Which Agent Workloads Can Run on Tool Calling Alone? Tool calling alone is enough when an agent touches a single source, makes few calls, needs no cross-system joins, runs short sessions, and has a human reviewing outputs. A support agent that looks up a single ticket in a single system fits that profile, and bolting on a context replication layer would add cost with no payoff.
The picture changes when an agent draws on multiple sources whose records describe the same entities, when a question requires reconciling those entities, or when execution runs unattended, and no human catches a wrong answer. Cross any one of those thresholds and the reliability problem stops being an interface problem and becomes a data-layer problem. That shift is what the following section is built to address.
How Do Airbyte's agentic platform Add the Context Layer Around Tool Calls? Airbyte's agentic platform is the data layer for AI agents, and it maps directly onto the four failure patterns. It combines 50-plus agent connectors with Managed Auth, a single Context Store, and two execution modes, so a team can adopt as much or as little of the stack as a given agent requires.
Managed Auth holds platform and connector credentials and refreshes OAuth on Airbyte's side, so the parallel-refresh race that kills long sessions happens behind the service rather than inside your agent loop. The Context Store gives agents unified records from connected sources to reason across, which is what resolves the three-Acmes problem before the model ever sees the data. Search queries that index the layer while Direct reaches the live application programming interface (API) for writes and current state.
For teams building the agent itself, the Agent SDK exposes these capabilities programmatically, so retrieval, context scoping, and mode selection live in application code rather than in brittle glue scripts. The Agent CLI moves the same operations into the terminal and continuous integration, which is where governed execution and repeatable connector configuration belong once an agent graduates from a notebook to a pipeline. In Airbyte's launch benchmark, agents built this way made around 40% fewer tool calls and used up to 80% fewer tokens, running the same tasks against pre-materialized context rather than assembling it from each source at request time.
What Should You Build Around Tool Calling? The through-line of the evidence is that tool calling is a solved interface atop an unsolved data problem. Benchmarks fall apart on cross-system and multi-step tasks, and the failures that teams report in production trace back to context that was stale, unjoined, or ungoverned rather than to the calls themselves. Reliability comes from doing the data work upstream, so each step the agent takes has less to get right.
That upstream work is what Airbyte's agentic platform provides through governed connectors, a Context Store that unifies records across sources, Managed Auth that keeps long sessions alive, and Search and Direct modes that separate indexed reads from live writes. Together, they let an engineering team focus on agent behavior rather than on the data plumbing beneath it.
Get a demo to see how Airbyte's agentic platform keeps tool-calling agents reliable on real business data.
Frequently Asked Questions Can Code Execution Replace Tool Calling Entirely? Code execution pays off when the tool catalog is large and the workflow varies, because processing data in the execution environment rather than routing it through the model can reduce token usage. For well-defined, low-variety workflows, standard tool calling stays simpler to build and debug. The two complement each other rather than one replacing the other.
How Do I Test My Own Agent's Tool-Count Ceiling? Measure selection accuracy at your actual catalog size before shipping, since accuracy tends to decline as the candidate catalog grows. Keep the toolset focused and re-measure after each catalog change, because the ceiling moves whenever you add or retire tools.
Are Tool Descriptions Themselves a Reliability Problem? Yes. Missing functional descriptions and error-handling guidance can lead to wrong tool selection and repeated retries, so audit the quality of the description before deployment as carefully as you audit the tools themselves.
What Is the Security Exposure of Tool-Connected Agents? Tool-connected agents can expose sensitive data or run unsafe actions when they trust poisoned tool definitions. Treat every third-party tool definition as untrusted input, turn off auto-approval for risky operations, and keep human approval on destructive actions.
