Tool Calling in AI Agents: How Agents Use External Systems to Get Work Done

Tool calling is the mechanism that lets an AI agent request an action from an external system. The model receives a description of available tools, selects one that fits the task, and produces a structured request containing the tool name and arguments. Your code then executes the request and returns the result to the model.

The word "calling" is misleading, and that confusion costs money. The model never calls anything. It produces a suggestion. Your code decides whether to honour it.

The Four-Step Loop: Schema, Selection, Execution, Interpretation

Every tool call runs through the same four steps.

• Schema: You describe each tool to the model: its name, what it does, what arguments it needs, and what those arguments mean. This description is everything the model knows about the tool.
• Selection: The model picks a tool and fills in arguments. This is a probabilistic judgement, not a lookup. It can be wrong.
• Execution: Your application receives the structured request, validates it, then runs it against the real system. This step is entirely deterministic code that you own.
• Interpretation: The result goes back into the model's context. The model reads it and decides what to do next, which is usually another tool call.

The loop repeats until the task is done or something breaks. Next, here is what the model does and does not do.

What the Model Does and Does Not Do

Two things: it chooses, and it formats.

It has no execution privileges. No network access. No credentials. Every permission your agent has is granted to your application. Every database it can write to, your code is connected.

When an agent deletes the wrong record, the model makes a bad suggestion, and your infrastructure carries it out. That distinction matters because it tells you who is accountable. Tool calling reliability is an infrastructure problem, not a model-selection problem. A stronger model narrows the failure rate. It doesn't change who's responsible for the damage. That becomes clear when you look at where the tool-calling breaks first.

Most agent pilots succeed. Most agent deployments struggle. The gap opens at the tool layer because that's where an agent stops predicting text and starts changing state in systems that matter.

Reading is cheap. Writing is not.

A tool that reads data has a bounded worst case. The agent pulls something irrelevant, wastes tokens, and moves on. You lose a bit of money and a bit of latency.

A tool that writes has an unbounded worst case. It issues a refund. Sends an email. Updates a ledger. Revokes access. There's no undo button. No context window can fix this.

Most teams run read and write tools through the same validation path, with the same review standards. That's the single biggest design error we see. These deserve different validation, different logging, and different approval requirements.

Errors compound across turns.

A single tool call with 95% accuracy sounds acceptable. A ten-step workflow chaining those calls doesn't. The failure compounds at every step.

Worse, agents don't fail cleanly. A bad tool result is treated as fact in the context. The model reasons confidently forward from it. By step seven, the agent is executing a plan built on something that was never true.

That's why single-call accuracy is a poor proxy for production reliability. What matters is whether the agent finishes the whole task. That number is always lower than per-call accuracy suggests. To see why, here's how tool calling fails in production.

Wrong Tool Selection

The agent picks a plausible tool that doesn't fit the task. This happens most when tool descriptions overlap, when two tools share similar names, or when the right tool's description doesn't use the words the user used. You'll see it in logs. It's loud.

Malformed Arguments

The agent selects the correct tool but passes invalid arguments. It borrows a parameter name from a different tool's schema. Passes a string where an integer belongs. Invents an ID that looks structurally correct. Also loud.

Tool Hallucination

The agent calls a tool that doesn't exist. No partial credit here. This spikes when the tool list is long and the descriptions blur together. You'll see it immediately.

Silent Success

This is the expensive one. The agent calls a valid tool with valid arguments, gets a 200 response, and produces the wrong business outcome. Nothing throws. Nothing logs an error. Your monitoring shows a healthy system. You find out from a customer complaint, weeks later. This one will hurt you. If you remember one thing from this section, make it this: log every tool call, its arguments, and its result from day one. Every day, hidden failures surface sooner. That logging becomes even more important as tools accumulate.

There's a counterintuitive rule in agent design. Past a certain point, every tool you add makes the agent worse at using the tools it already has.

The mechanism is simple. Tool descriptions live in the context window alongside the system prompt and the conversation. They compete for the model's attention with the actual task.

Run the numbers on your own agent. Add up the token cost of every tool schema before a single user message arrives. Most teams are shocked to find that tool definitions consume a large chunk of the context budget on every request.

Two things degrade at once. Attention is spread thin across dozens of near-identical schemas, so the correct tool competes with the noise instead of standing out. Tokens spent describing tools the agent won't use in this task are tokens unavailable for reasoning about the task it actually has to perform.

The vendors know this is real. Anthropic shipped Tool Search and Programmatic Tool Calling in its API specifically to address this. When the model vendors are building infrastructure for the problem, the problem is real.

The fix isn't fewer capabilities. It has fewer capabilities visible at once. That leads to the practical ways to reduce overload.

• Scope tools per task. Route the request first, then expose only the tools relevant to that route. A billing question doesn't need file system access.

• Load tools dynamically. Retrieve a relevant subset for each task using semantic search on tool descriptions rather than registering everything up front.

• Split across agents. Give each specialised agent a small toolset instead of handing everything to one orchestrator. This is one of the stronger arguments for multi-agent design.

• Measure before you add. Build a small eval suite of representative queries and re-run it every time a tool joins the registry. If accuracy drops, you've found your ceiling. That same discipline applies as protocols like MCP change how tools are connected.

Before the Model Context Protocol, every combination of AI application and tool needed its own connector. Ten applications and one hundred tools meant up to a thousand integrations, each written and maintained separately.

BCG framed it this way: without a standard protocol, integration complexity grows quadratically as agents spread across an organisation. With MCP, it grows linearly. You build one server per resource. Any compliant client uses it.

The adoption moved fast. Anthropic reported over 10,000 active public MCP servers and 97 million monthly SDK downloads across Python and TypeScript by December 2025, when it donated MCP to the Agentic AI Foundation.

That donation matters more than the download count. The Linux Foundation announced the AAIF with Amazon, Anthropic, Block, Bloomberg, Cloudflare, Google, Microsoft, and OpenAI as platinum members. That removed the single-vendor risk that had kept enterprise architects on the sidelines. With that baseline in place, the next question is what changed in 2026.

What Changed in 2026: Enterprise Auth, Registries, Governance

Three shifts in 2026 turned MCP from a developer convenience into something an enterprise architecture review can actually approve.

1. Authorisation got centralised. The Enterprise-Managed Authorisation extension reached stable status, replacing per-server consent prompts with a single sign-in through the organisation's identity provider. Anthropic, Microsoft, and Okta support it. Asana, Atlassian, Figma, Linear, and Supabase support it on the server side. Repeated auth prompts were the most cited blocker in enterprise MCP pilots.

2. The protocol scaled. The 2026 MCP roadmap is reorganised around four priorities: transport scalability, agent communication, governance maturity, and enterprise readiness. The spec release in July 2026 delivers on it with a stateless protocol core, an extensions framework, a tasks primitive for long-running work, and authorisation aligned to OAuth and OpenID Connect.

3. Schemas got expressive. That same release lifts tool input and output schemas to full JSON Schema 2020-12, allowing composition and conditionals instead of flat parameter lists. It also documents W3C Trace Context propagation, so a tool call is traced from the host application through the client, the server, and any downstream calls the server makes. Distributed tracing is how you make silent success visible. From there, the remaining question is when custom integration still makes more sense than MCP.

When a Custom Integration Still Beats MCP

MCP isn't the answer to every integration problem. Don't treat it as one.

Start with the simpler path when it is enough. Use a direct integration when the tool is used by exactly one agent, when latency budgets are tight enough that a protocol hop costs you, or when the operation is risky enough that you want a bespoke validation and approval path. Reach for MCP when multiple clients need the same resource, when you expect to change model providers, or when the tool is maintained by another team. Choose the shared protocol when reuse justifies it.

The honest framing is that MCP solves an organisational problem more than a technical one. One team, one agent, three tools? It's overhead. Twelve teams building agents against overlapping systems? It's the difference between an integration layer and integration debt.

Tool calling is where a language model gains the ability to affect the world outside its context window. That's why security gets treated as its own category, not a subset of model risk.

The OWASP GenAI Security Project published its Top 10 for Agentic Applications for 2026, developed with over 100 contributing organisations. It exists because agentic systems break assumptions. The LLM Top 10 was built on. An agent plans, stores memory, invokes tools, and acts using real credentials. That's different.

Two entries sit directly on the tool call.

ASI02, Tool Misuse and Exploitation: Agents use connected tools in unsafe ways, or attackers exploit tool interfaces to gain access.

ASI03, Identity and Privilege Abuse: Agents misuse credentials or inherited permissions to reach data beyond intended limits.

The pattern behind both is over-permissioning. An agent gets a service account with broad access because scoping it properly was tedious during the pilot. That scope never gets tightened. The pilot becomes production.

Indirect prompt injection through tool results is the failure mode that catches people off guard.

Tool results enter the model's context as text, and the model has no reliable way to tell data from instructions. An attacker who can influence what a tool returns can influence what the agent does next. A poisoned document. A crafted support ticket. A comment field in a record the agent reads. Any of these can carry instructions.

OWASP ranks prompt injection as the top risk. The indirect variant is the one that matters for tool-using agents. In a chatbot, successful injection produces bad text. An agent with the right tools can move money.

The security community's reading is lopsided. CIO reported that fewer than 4% of MCP-related RSA Conference submissions framed the protocol as an opportunity. The rest focused on exposure: over-permissioned tooling, untrusted servers, and tool impersonation. The exposure isn't theoretical.

The controls that actually hold are controls that stack.

Least privilege per tool. Scope credentials to the specific operation, not the service. Read tools get read tokens.

Approval gates on state changes. Any tool that moves money, sends external communication, or deletes data routes through a human or a deterministic policy check. This isn't a maturity phase you graduate from. It stays.

Treat every tool result as untrusted input. Validate the structure and content before they reach the model's context, just as you'd validate user input.

Validate output before execution. The model's tool request is a suggestion. Check it against a schema and a policy layer before your code acts.

Log at the tool-call level. Arguments, results, and selection on every call. This is the only way silent success becomes visible.

Most tool calling failures trace back to the schema, not the model. Good news: the schema is entirely under your control.

Description quality is prompt engineering, not documentation.

A tool description isn't documentation for a developer who'll read it once. It's a prompt the model reads on every single request. It's the model's only basis for choosing.

Write descriptions that mirror how users actually phrase the need, not how your engineering team names the service. Say when to use the tool and when not to. If two tools could plausibly serve the same request, say explicitly in each description what distinguishes them.

Name parameters unambiguously. "id" is a trap when the agent has four tools that each take a different kind of ID. "customer_account_id" isn't verbose. It's disambiguating.

Idempotency, error contracts, and retry semantics matter because agents retry.

Agents retry on timeouts, on ambiguous results, and on their own confusion. Design for that rather than hoping against it.

Make writing tools idempotent where possible. Accept an idempotency key so a retried refund doesn't become two refunds.

Return errors the model can act on. An opaque "Error 500" teaches the model nothing. An error that names the problem and the next step tells it exactly what to do.

Return structured results, not prose. The model parses tool output as context. Structure reduces the chance of it misreading a number or a status.

Error messages are underrated as a reliability lever. A well-written error contract turns a dead end into a self-correcting step.

Our guide on how to build AI agents for enterprise covers where this fits in the delivery sequence.

You can't govern what you don't measure. Tool calling is measurable well before production.

The Berkeley Function Calling Leaderboard, maintained by UC Berkeley's Gorilla team, is the most-cited public benchmark for tool calling. It grades structural correctness using abstract syntax tree matching and executable evaluation against real runtime behaviour. V4 extends from single-turn function calling to multi-turn agentic evaluation.

BFCL is useful for model selection. It's not useful for predicting how your agent will behave. It doesn't know your tool descriptions, your schema quality, your data, or how your users phrase things. A model at the top of the leaderboard will still fail on a badly described tool.

Treat BFCL as a floor for model choice, then measure your own system.

Build your own eval suite.

Start smaller than you think you need. Fifty representative queries beat a perfect framework you never finish.

Collect real queries. Pull them from support tickets, user research, or existing logs. Invented queries test invented problems.

Label the correct tool and arguments for each. This is the tedious part. It's also the part that pays.

Measure selection separately from execution. Did it pick the right tool? Did it fill in the right arguments? These fail for different reasons and need different fixes.

Run the suite on every change. New tool, new model version, edited description. Regression here stays silent otherwise.

Track multi-turn completion, not just per-call accuracy. The number that matters is whether the whole task was finished correctly.

Three actions, in order.

• Audit what your agents can currently write to, and scope those credentials down.
• Then instrument logging at the tool-call level so silent success stops being invisible.
• Finally, build a small eval suite against real queries before you add another tool.