Tool-Calling Patterns That Actually Work in Production AI Agents

Most agent tutorials show the same thing: define one tool, let the model call it once, print the result, declare victory. That is not an agent. That is a function call with extra steps. The moment you put it in front of real traffic, the model calls three tools when it should have called one, invents a parameter that does not exist, retries a failed payment four times, and hangs for ninety seconds because a downstream API timed out and nobody told the model.

I have shipped tool-calling agents to production and spent more time on the plumbing around the tool calls than on the tools themselves. The reliability does not come from the model. It comes from five patterns that most demos skip because they only matter once things go wrong. Here they are, naive version first, then the version that survives.

The Naive Loop Everyone Starts With

Almost every tool-calling agent begins as some variation of this:

def run(task: str):
    messages = [{"role": "user", "content": task}]
    while True:
        resp = client.messages.create(
            model="claude-opus-4-8",
            max_tokens=1024,
            tools=TOOLS,
            messages=messages,
        )
        if resp.stop_reason != "tool_use":
            return resp.content[-1].text
        tool_use = next(b for b in resp.content if b.type == "tool_use")
        result = TOOL_FNS[tool_use.name](**tool_use.input)
        messages.append({"role": "assistant", "content": resp.content})
        messages.append({"role": "user", "content": [{
            "type": "tool_result",
            "tool_use_id": tool_use.id,
            "content": str(result),
        }]})

This works in the happy path and breaks everywhere else. It assumes exactly one tool call per turn, it crashes the instant a tool raises, it has no upper bound on cost, and it trusts whatever tool_use.input the model produced. Every pattern below fixes one of those assumptions.

Pattern 1: Schemas Tight Enough That Hallucination Is Rare

The single highest-leverage change you can make is tightening your tool schemas. Models hallucinate arguments when the schema leaves room for interpretation. A loose schema is an invitation.

Here is a loose schema that will get abused:

{
    "name": "search_orders",
    "description": "Search for orders",
    "input_schema": {
        "type": "object",
        "properties": {
            "status": {"type": "string"},
            "date": {"type": "string"},
            "limit": {"type": "integer"},
        },
    },
}

Nothing here tells the model what a valid status is, what date format you expect, or that limit has a ceiling. So it guesses. You will see status="completed" when your enum is fulfilled, dates in three different formats, and limit=10000.

The tight version closes every gap:

{
    "name": "search_orders",
    "description": (
        "Search orders placed by the current user. Returns at most `limit` "
        "orders sorted newest first. Use this only for the signed-in user; "
        "it cannot search other accounts."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "status": {
                "type": "string",
                "enum": ["pending", "fulfilled", "cancelled", "refunded"],
                "description": "Order status to filter by.",
            },
            "since": {
                "type": "string",
                "format": "date",
                "description": "ISO 8601 date, e.g. 2026-07-01. Orders on or after this date.",
            },
            "limit": {
                "type": "integer",
                "minimum": 1,
                "maximum": 50,
                "description": "Max orders to return. Keep small unless the user asks for more.",
            },
        },
        "required": ["status"],
        "additionalProperties": false,
    },
}

The rules that matter:

• Use enum for anything with a fixed set of values. This is the biggest single win. The model cannot invent a status that is not in the list.
• Set additionalProperties: false. It stops the model from inventing extra keys and makes validation failures loud instead of silent.
• Mark the truly required fields required and nothing else. Over-requiring makes the model fabricate values it does not have.
• Put constraints the model should respect in the description, not just the schema. “Keep small unless the user asks for more” changes behavior in a way maximum: 50 alone does not.

Then validate the arguments before you execute, because the schema is a strong suggestion to the model, not a guarantee. Reject bad input as a tool result the model can read and correct, which leads directly into the next pattern.

Pattern 2: Parallel Tool Calls for Independent Work

The naive loop handles one tool call per turn. But modern models will return several tool_use blocks in a single response when the calls are independent, and if you only execute the first one you throw away the model’s best instinct and force extra round trips.

Say the user asks “compare my last order to my subscription and tell me if I am overpaying.” The model can look up the order and the subscription at the same time. Sequential execution costs you two extra latency round trips for no reason.

Execute every tool block the response returned, and run the independent ones concurrently:

import asyncio

async def handle_turn(resp, messages):
    tool_uses = [b for b in resp.content if b.type == "tool_use"]

    async def run_one(tu):
        try:
            result = await dispatch(tu.name, tu.input)
            return tool_result(tu.id, result)
        except ToolError as e:
            return tool_result(tu.id, e.as_payload(), is_error=True)

    results = await asyncio.gather(*(run_one(tu) for tu in tool_uses))
    messages.append({"role": "assistant", "content": resp.content})
    messages.append({"role": "user", "content": results})


def tool_result(tool_use_id, content, is_error=False):
    return {
        "type": "tool_result",
        "tool_use_id": tool_use_id,
        "content": content if isinstance(content, str) else json.dumps(content),
        "is_error": is_error,
    }

Two things people get wrong here:

• You must return a tool_result for every tool_use_id in the assistant turn. If the model called three tools and you return two results, the next API call fails. Gather all of them, including the errors.
• Parallel is only safe for reads and idempotent writes. If the model tries to issue two refunds in parallel, concurrency turns a logic bug into a double charge. For anything with side effects, either serialize it or lean on Pattern 5. If you genuinely never want parallel calls, you can set tool_choice to disable it, but that is a blunt instrument that also slows down safe fan-out.

The payoff is real: fan-out reads that used to take four sequential turns collapse into one turn with four concurrent calls, and wall-clock latency drops to the slowest single call instead of the sum.

Pattern 3: Errors Are Tool Results, Not Exceptions

The naive loop crashes when a tool raises. That is the worst possible behavior, because the model is the one component in your system that can actually recover from a bad call. It cannot recover from a stack trace it never sees.

Feed the error back as a tool_result with is_error: true and a message the model can act on. There is a difference between an error that tells the model what to do next and one that just says “failed”:

class ToolError(Exception):
    def __init__(self, code, message, retryable=False):
        self.code = code
        self.message = message
        self.retryable = retryable

    def as_payload(self):
        return json.dumps({
            "error": self.code,
            "message": self.message,
            "retryable": self.retryable,
        })


def search_orders(status, since=None, limit=10):
    if status not in VALID_STATUSES:
        raise ToolError(
            "invalid_status",
            f"'{status}' is not valid. Use one of: {sorted(VALID_STATUSES)}.",
            retryable=True,
        )
    try:
        return db.search(status=status, since=since, limit=min(limit, 50))
    except TimeoutError:
        raise ToolError(
            "upstream_timeout",
            "The order service did not respond. Try again or narrow the date range.",
            retryable=True,
        )

Good error messages are prompts in disguise. “‘complete’ is not valid. Use one of: pending, fulfilled, cancelled, refunded” gets the model to self-correct on the next turn almost every time. A bare “ValueError” gets you an apology and a retry with the same bad argument.

Draw a hard line between the two failure classes:

For permanent errors, do not let the model retry. Return a non-retryable error and, if the retry budget is exhausted, stop the loop entirely. Which is Pattern 4.

Pattern 4: Retry Budgets, Not Retry Loops

“Retry on failure” without a budget is how you get an agent that calls the same broken tool forty times and bills you for it. You need budgets at two levels: per tool call and per run.

class RetryBudget:
    def __init__(self, max_steps=12, max_tool_retries=2, max_cost_usd=1.00):
        self.max_steps = max_steps
        self.max_tool_retries = max_tool_retries
        self.max_cost_usd = max_cost_usd
        self.steps = 0
        self.cost = 0.0
        self.failures = {}  # (tool_name, args_hash) -> count

    def charge(self, usd):
        self.cost += usd

    def step(self):
        self.steps += 1
        if self.steps > self.max_steps:
            raise BudgetExceeded("max_steps")
        if self.cost > self.max_cost_usd:
            raise BudgetExceeded("max_cost")

    def record_failure(self, tool_name, args):
        key = (tool_name, _hash(args))
        self.failures[key] = self.failures.get(key, 0) + 1
        return self.failures[key] <= self.max_tool_retries

The subtle part is record_failure keying on the tool name and the arguments together. If the model retries the exact same call with the exact same arguments and it fails again, that is not a transient blip worth retrying, that is a loop. Cut it off. But if the model fixes the arguments after reading your error, that is a fresh key with a fresh budget, which is exactly the behavior you want to allow.

Wire it into the loop:

def run(task, budget: RetryBudget):
    messages = [{"role": "user", "content": task}]
    while True:
        budget.step()
        resp = client.messages.create(model=MODEL, max_tokens=1024,
                                       tools=TOOLS, messages=messages)
        budget.charge(estimate_cost(resp.usage))
        if resp.stop_reason != "tool_use":
            return resp.content[-1].text
        results = []
        for tu in [b for b in resp.content if b.type == "tool_use"]:
            try:
                results.append(tool_result(tu.id, dispatch(tu.name, tu.input)))
            except ToolError as e:
                if e.retryable and budget.record_failure(tu.name, tu.input):
                    results.append(tool_result(tu.id, e.as_payload(), is_error=True))
                else:
                    results.append(tool_result(
                        tu.id, '{"error":"giving_up","message":"This tool has failed too many times. Stop and report what you have."}',
                        is_error=True))
        messages.append({"role": "assistant", "content": resp.content})
        messages.append({"role": "user", "content": results})

When the budget is exhausted, degrade gracefully instead of throwing. “I could not complete every step, here is what I found” beats a 500 error to the user every time.

Pattern 5: Idempotency So Retries and Parallelism Are Safe

Patterns 2 and 4 both create the same danger: a tool with side effects might get called more than once. Parallel execution can fire two writes at once, and retries can repeat a write whose response got lost. If your tools are not idempotent, you have built a machine that double-charges customers.

The fix is an idempotency key threaded from the agent through to the downstream system. Naive tools take just the business arguments:

def create_refund(order_id, amount):
    return payments.refund(order_id, amount)   # retry = double refund

The safe version accepts a key and makes the operation exactly-once at the boundary that actually owns the money:

def create_refund(order_id, amount, idempotency_key):
    # The payment provider dedupes on this key: same key = same refund,
    # returned from cache instead of executed twice.
    return payments.refund(order_id, amount, idempotency_key=idempotency_key)

Where does the key come from? Derive it deterministically from the intent, so a retry of the same logical action produces the same key:

def idempotency_key(run_id, tool_name, args):
    payload = json.dumps({"tool": tool_name, "args": args}, sort_keys=True)
    return hashlib.sha256(f"{run_id}:{payload}".encode()).hexdigest()

Now a retry of “refund order 123 for $40” inside the same run always hits the same key and executes once, while a genuinely new refund in a different run gets a new key. This turns “did that write go through?” from a correctness problem into a non-issue, which is what lets you retry and parallelize aggressively without holding your breath.

How the Pieces Fit

Put together, the request path looks like this:

User task
    |
    v
[Agent loop] -- checks RetryBudget.step() each turn
    |
    v
Model returns 1..N tool_use blocks
    |
    +--> validate args against tight schema (Pattern 1)
    |        |-- invalid --> structured error result (Pattern 3)
    |
    +--> dispatch independent calls concurrently (Pattern 2)
    |        |-- each write carries an idempotency key (Pattern 5)
    |
    v
Collect one tool_result per tool_use_id (errors included)
    |
    +--> on failure: RetryBudget.record_failure (Pattern 4)
    |        |-- within budget --> retryable error, let model fix
    |        |-- over budget   --> giving_up, degrade gracefully
    |
    v
Append results, loop until stop_reason != tool_use

Notice that none of these patterns are about the prompt or the model choice. They are about the contract between your code and the model: what you let the model say, what you let it do concurrently, what you tell it when things break, and how many chances it gets.

The Honest Assessment

What works. Tight schemas with enums and additionalProperties: false eliminate the large majority of hallucinated arguments on their own, and they are the cheapest change on this list. Structured, actionable errors turn the model into a self-correcting system instead of a crash generator. Retry budgets keyed on tool-plus-arguments are the difference between an agent that costs pennies and one that costs a fortune when a downstream service degrades.

What does not. Parallel tool calling is oversold. Most agent turns have a genuine data dependency between calls, so the fan-out you can actually parallelize is smaller than the demos suggest. Idempotency keys only help if the downstream system honors them, and plenty of internal services do not, in which case you are back to serializing writes and accepting the latency.

What to actually do. Start with Pattern 1, because it is a few lines of schema and it removes a whole class of failure. Add Pattern 3 next, because feeding errors back as tool results is what makes the agent resilient at all. Then add the retry budget before you ever ship, because an unbounded loop in production is not a bug you want to discover from your invoice. Parallelism and idempotency are optimizations; reach for them when a specific tool actually needs them, not because a tutorial made fan-out look impressive. The reliable agents I have seen are not the ones with the cleverest planning. They are the ones where every tool call has a tight contract, a defined failure mode, and a budget.