Function Calling Across Providers

"Give a model a tool and it will use it. Give it the wrong JSON schema and it will use it creatively."

Pip, Schema-Validated AI Agent

1. The Function Calling Interface

Function calling is the mechanism that transforms an LLM from a text generator into a tool-using agent. Rather than generating natural language descriptions of what tools to use, the model produces structured JSON that specifies which function to call and with what arguments. The application code then executes the function, returns the result to the model, and the model incorporates the result into its next response. This structured interface eliminates the fragile parsing of natural language tool invocations that plagued early agent systems.

Every major LLM provider now supports function calling, but the implementations differ in important ways. Understanding these differences is essential for building portable agent systems and for selecting the right provider for your use case. The core pattern is the same across all providers: define tool schemas, pass them to the model alongside the user message, handle tool call responses, execute the tools, and return results. The details of schema format, multi-tool handling, and streaming behavior vary.

OpenAI was the first major provider to ship function calling (June 2023), and their format has become the de facto standard that many open-source frameworks adopt. Anthropic's tool use implementation adds explicit thinking before tool calls. Google's Gemini API supports function calling with automatic function execution in some modes. Open-source models accessed through frameworks like Ollama or vLLM increasingly support function calling, though the reliability varies by model size and training data.

Input: user message M, tool schemas {T1, ..., Tn}, LLM model, max iterations K
Output: final text response

1. messages = [system_prompt, M]
2. for i = 1 to K:
 a. response = LLM(messages, tools={T1, ..., Tn})
 b. if response has no tool_calls:
 return response.content // final answer
 c. Append assistant message (with tool_calls) to messages
 d. for each tool_call in response.tool_calls:
 i. name = tool_call.function.name
 ii. args = parse_json(tool_call.function.arguments)
 iii. result = execute(name, args)
 iv. Append tool result message (role="tool", id=tool_call.id) to messages
3. return "Max iterations reached"
 

OpenAI Function Calling

This snippet defines a tool schema and processes function-call responses using the OpenAI function calling API.

from openai import OpenAI

client = OpenAI()

tools = [
 {
 "type": "function",
 "function": {
 "name": "get_weather",
 "description": "Get the current weather for a location",
 "parameters": {
 "type": "object",
 "properties": {
 "location": {
 "type": "string",
 "description": "City and state, e.g. San Francisco, CA",
 },
 "unit": {
 "type": "string",
 "enum": ["celsius", "fahrenheit"],
 "description": "Temperature unit",
 },
 },
 "required": ["location"],
 },
 },
 }
]

response = client.chat.completions.create(
 model="gpt-4o",
 messages=[{"role": "user", "content": "What is the weather in Paris?"}],
 tools=tools,
 tool_choice="auto",
)

# Handle the tool call
tool_call = response.choices[0].message.tool_calls[0]
print(f"Function: {tool_call.function.name}")
print(f"Arguments: {tool_call.function.arguments}")

from pydantic_ai import Agent

agent = Agent("openai:gpt-4o")

@agent.tool_plain
def get_weather(location: str, unit: str = "celsius") -> str:
 """Get the current weather for a location."""
 return f"72F, partly cloudy in {location}"

result = agent.run_sync("What is the weather in Paris?")
print(result.data)

Anthropic Tool Use

This snippet defines tools and handles tool-use responses using the Anthropic messages API.

import anthropic

client = anthropic.Anthropic()

tools = [
 {
 "name": "get_weather",
 "description": "Get the current weather for a location",
 "input_schema": {
 "type": "object",
 "properties": {
 "location": {
 "type": "string",
 "description": "City and state, e.g. San Francisco, CA",
 },
 "unit": {
 "type": "string",
 "enum": ["celsius", "fahrenheit"],
 "description": "Temperature unit (default: celsius)",
 },
 },
 "required": ["location"],
 },
 }
]

response = client.messages.create(
 model="claude-sonnet-4-20250514",
 max_tokens=1024,
 tools=tools,
 messages=[{"role": "user", "content": "What is the weather in Paris?"}],
)

# Anthropic returns tool_use blocks within the content array
for block in response.content:
 if block.type == "tool_use":
 print(f"Tool: {block.name}, Input: {block.input}")

from pydantic_ai import Agent

# Same code works with any provider; just change the model string
agent = Agent("anthropic:claude-sonnet-4-20250514") # or "openai:gpt-4o"

@agent.tool_plain
def get_weather(location: str, unit: str = "celsius") -> str:
 """Get the current weather for a location."""
 return f"22C, sunny in {location}"

result = agent.run_sync("What is the weather in Paris?")

To understand why function calling is architecturally significant (and not just a convenience feature), consider what happens without it. Before structured function calling, agents had to embed tool invocations in natural language ("I should search for 'weather Paris'"), and application code had to parse these free-form strings with regex or heuristics. This was fragile: minor rephrasing broke the parser, and the model had no formal contract specifying valid actions. Structured function calling introduces a type system for agent actions. The JSON schema defines exactly what the model can do, with what parameters, and in what format. This transforms agent tool use from "string parsing with fingers crossed" into a well-defined API contract, enabling reliable orchestration, automatic validation, and composable tool chains as explored in Section 23.2 (MCP).

2. Multi-Tool Orchestration

Real agents need multiple tools working together. A research agent might search the web, extract content from URLs, store findings in a database, and generate a report. The model must decide not only which tool to call but in what order, and it must handle the data flow between tool calls. Modern APIs support parallel tool calling, where the model can request multiple tool executions in a single response, significantly reducing the number of round trips for independent operations.

The agent loop for multi-tool orchestration follows a standard pattern: send the user message with all tool definitions, receive the model's response (which may contain one or more tool calls), execute all requested tools, return all results in a single follow-up message, and repeat until the model produces a final text response without tool calls. Managing this loop correctly, especially handling errors from individual tool calls without derailing the entire conversation, is a core engineering challenge.

3. Open-Source Function Calling

Open-source models have rapidly closed the gap in function calling capability. Models like Llama 3.1 (with tool use training), Mistral's function calling models, and Qwen 2.5 support structured tool interactions. These models can be served through vLLM, Ollama, or TGI with OpenAI-compatible API endpoints, making them drop-in replacements for many use cases. The trade-off is typically reliability: frontier models handle complex multi-tool scenarios more robustly, while open-source models may require more careful prompt engineering and schema design.

For teams that need to keep data on-premises or require custom fine-tuning for domain-specific tools, open-source function calling models provide a viable path. Fine-tuning on examples of your specific tool schemas and usage patterns can bring open-source models to near-frontier reliability for a constrained tool set. The ToolACE and Gorilla projects have demonstrated that targeted training on tool-use data can produce highly capable tool-using models from relatively small base models.

Exercises