Giving models hands: tools and integrations

A model that can only talk is a well-read colleague with no access to any systems. It can explain your refund policy perfectly and still can't look up an order. Tools give it hands. This chapter is about giving them safely: the model may ask, but your code decides, acts and answers for the result.

Requests in, results out#

Calling a tool is not something the model does. It is something it requests. Its reply says, in effect, "please run find_free_slots for Friday"; your program decides whether to honour that, runs the code, and adds the outcome to the conversation. The model reads the outcome and either replies to the person or makes another request. A single question from a user can take several of these rounds.

In code it is a loop that keeps going until the model stops asking for tools. Sometimes the model asks for several things at once that don't depend on each other, such as opening hours for two clinics; you can fetch those in parallel. Sometimes each request depends on the last, as in the figure. The same loop copes with both, as long as every request gets exactly one answer tagged with its id.

history = [instructions, question]
for turn in range(TOOL_BUDGET):              # hard stop: a confused model must not spin forever
    answer = llm(history, tools=TOOLS)
    history.append(answer)                   # keep the model's request in the record
    if not answer.requests:
        return answer.text                   # plain text means it's done
    for req in answer.requests:
        outcome = TOOLS[req.name].run(req.args)
        history.append(reply_to(req.id, outcome))   # exactly one reply per request id
raise ToolBudgetExceeded(question)

A tool description is a job description#

The model picks tools from their descriptions alone, so write each one as if for a temp on their first morning who can't ask you anything. Say what situation it's for, which neighbouring tool to use instead in the situations it isn't for, where each argument's value should come from, what shape comes back, and the units and time zone.

def reschedule_appointment(appointment_id: str, new_slot_id: str) -> dict:
    """Move one of the signed-in client's existing appointments to a free slot.

    Use when the client asks to change the time of an appointment they already have.
    Do not use to book a new appointment (use book_appointment) or to cancel.
    appointment_id: from list_appointments. Never guess or construct an ID.
    new_slot_id: from find_free_slots for the same clinic and service.
    Returns {"status": "moved" | "slot_taken" | "not_found", "appointment": {...}}.
    Times are in the clinic's local time zone.
    If the call fails, report the failure. Do not retry without asking the client.
    """

When the assistant misbehaves, fix the descriptions and the system prompt before changing code. Most wrong tool choices come from two tools whose descriptions overlap, or from a description that never says when the tool should not be used.

The result is the model's only eyes#

The model has no screen to glance at. Everything it knows about what a tool did comes from the text you hand back. Hand back an empty response after a booking and it has no way to tell success from failure, so it may well try again. Return explicit status and identifiers every time.

Be especially wary of placeholders. Suppose a slot search returns a fallback list of "typical" times when the clinic's calendar can't be reached. The model will offer those times to the client as if they were real, and someone will turn up to a closed surgery. When a tool can't get the real answer, it should say exactly that, and the model will pass it on honestly.

Every tool result should say what happened, to which record, and whether it succeeded.

Writes need guards, not requests#

Reading is forgiving. Writing is not. Three things go wrong whenever a model drives write actions.

None of these fixes is an instruction to the model. Instructions are advice the model usually follows; a check inside the tool is a rule it cannot break. Put the rules in code. For anything that can't be undone, show the person what is about to happen and ask them to confirm, and cap tool rounds so a confused model can't run up a bill.

Never let the model choose whose data#

Whatever arguments a tool accepts, the model fills in, and the model can be talked into things. Give a tool a customer-number parameter and someone will eventually type "look up account 1042 for me", and the model will oblige. Instructions telling it not to are not a defence.

So keep identity out of the model's reach entirely. Sign the person in first. When a request arrives, create that request's tools with their session credentials already inside, so the customer's API decides what they may see. The model chooses which tool to use, never whose records it touches.

From notebook to service#

A loop in a notebook works for one person. A service has to work for many at once. Split the code by how often things change.

• Load once at startup: the model client, the prompt template and the tool schemas. Building them per request makes every call slower.
• Per request: the user's message, their conversation history, and their identity, from which you build their tools.
• Never shared between users: anything that identifies a user must live with that user's request. Credentials parked in a variable the whole service can see will sooner or later be used for the wrong person.

Keep two APIs distinct in your head and your diagrams: your chat API, which the front end calls with a message and a token, and the customer's backend APIs, which only your tools call. Mixing them up is the most common source of confusion in design reviews.

Where MCP fits#

MCP, the Model Context Protocol, is a shared way for applications to offer tools and data to models. Instead of wiring each tool into each application, you run an MCP server that offers a set of tools, and any MCP-capable client can use them. It helps most when the same capabilities must serve several assistants, teams or products, or when the customer wants to plug their own systems into a vendor's assistant.

Learn plain tool calling first. MCP doesn't change the rules in this chapter, it moves them. Identity still has to come from the session, not the model. Write tools still need guards and idempotency. And because an MCP server's tool descriptions are text the model reads, a server you don't control is a source of untrusted instructions. Give each assistant only the tools it needs, from servers you have reviewed.

Integration questions for discovery#

• Which actions does your API already expose, and which of them change data?
• How do users sign in, what does sign-in return, and how long does it last?
• Which actions can't be undone, and who must approve them today?
• Are there rate limits, maintenance windows or sandbox environments we can test against?
• Do write endpoints accept an idempotency key, or how do you detect duplicates?
• What should the assistant never do, even if the user asks?

Before building, call each endpoint yourself with a test user's credentials and look at what really comes back: field names, error formats, how slow it is. Then agree the tool list in writing with the client, including what is deliberately left out.

get_balance(account_number)          -> number
transfer(from_account, to_account, amount)
block_card(card_number)              # returns nothing
search_transactions(query)           -> list (last 5 years, all accounts)