LLM function calling is the feature that turns a chatbot into something that can actually do work: look up an order, query a database, send an email. If you're building AI agents, it's the first primitive you need to understand.
What Is LLM Function Calling?
A language model only produces text. Function calling (also called tool calling) is a protocol that lets that text be a structured request: "call this function with these arguments."
The model never runs anything. Your code does. The model just decides which tool fits the request and fills in the parameters as JSON.
That split matters. The model is the planner. Your application is the executor, and it stays in control of what really happens.
How Function Calling Works Step by Step
Every major provider follows the same loop, with minor syntax differences.
- Define tools. You send a list of functions, each with a name, a plain-language description, and a JSON Schema for its parameters.
- The model decides. Given the user message, it either answers directly or returns a tool call with arguments.
- You execute. Your code validates the arguments, runs the function, and captures the result.
- You return the result. The tool output goes back into the conversation.
- The model continues. It writes the final answer, or requests another tool call.
Repeat that loop until the model stops asking for tools and you have an agent. That's really all an agentic loop is at its core.
A concrete example
Say you define get_order_status(order_id). A user asks, "Where's my order 4821?" The model returns a call to get_order_status with {"order_id": "4821"}. You hit your database, return "shipped, arriving Friday," and the model turns that into a clean reply.
No regex, no brittle intent parsing. The model maps messy language to a clean function call.
Writing Tool Definitions That Models Actually Use Well
Most function calling failures are definition failures, not model failures. The description is a prompt, so treat it like one.
- Name tools like verbs.
search_invoicesbeatsinvoice_tool. - Say when to use it, and when not to. "Use for order lookups. Do not use for refunds" prevents wrong picks.
- Constrain parameters. Use enums, required fields, and format hints instead of free-form strings.
- Keep the toolset small. Ten sharp tools outperform fifty overlapping ones. Selection accuracy drops as options pile up.
- Return useful errors. If a call fails, send back a readable message so the model can correct itself and retry.
Parallel and forced calls
Most APIs let the model request several independent calls in one turn, which cuts latency when you need, say, three lookups at once. Most also let you force a specific tool or require some tool call, which is handy for extraction tasks where you want guaranteed structured output.
Function Calling vs Structured Outputs vs MCP
These get confused constantly.
- Function calling lets the model request an action you then perform.
- Structured outputs force the model's reply to match a schema. Use them when you just need clean JSON, not an action.
- MCP standardizes how tools are packaged and discovered across apps, so you don't rewrite integrations per provider. Underneath, the model still emits tool calls.
Think of function calling as the mechanism and MCP as one way to distribute the tools that use it.
Production Pitfalls and Safety Rules
Letting a model trigger real code is powerful, and it's where things break. A few rules that save pain:
- Always validate arguments. Treat model output as untrusted user input. Check types, ranges, and permissions before executing.
- Apply least privilege. A read-only tool can't delete data. Separate read tools from write tools.
- Require confirmation for irreversible actions. Payments, deletions, and outbound emails should pass a human or a policy check.
- Cap the loop. Set a maximum number of tool calls per request so a confused model can't spin forever and burn tokens.
- Guard against prompt injection. Content a tool fetches, like a web page or email, can contain instructions. Never let retrieved text grant new permissions.
- Log every call. Record the tool, arguments, result, and latency. You can't debug an agent you can't replay.
Where to Start
Pick one workflow you do by hand every week. Define two or three tools around it, wire up the loop, and test with ugly real inputs. Add an evaluation set of 20 realistic requests with expected tool calls, and rerun it whenever you change a description.
Once that loop is reliable, adding more tools is incremental work. The hard part isn't the API. It's choosing tools worth building and keeping them safe.