Tool Calling

Pass function schemas in your request and Arbytra returns structured tool calls you can execute locally.

Prerequisites

An Arbytra API key
Python 3.10+ with the OpenAI SDK (pip install openai) or the arbytra SDK (pip install arbytra)
- OR Node.js 18+ with the OpenAI SDK (npm install openai) or @arbytra/sdk (npm install @arbytra/sdk)

Define tools

Define tools as JSON schemas describing the function signature:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "The city name"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["city"]
            }
        }
    }
]

Call tools

Send a request with tools and check the response:

import os
from openai import OpenAI
import json

client = OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools
)

if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Function: {tool_call.function.name}")
    print(f"Arguments: {json.loads(tool_call.function.arguments)}")

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const tools: OpenAI.ChatCompletionTool[] = [
    {
        type: "function",
        function: {
            name: "get_weather",
            description: "Get weather for a city",
            parameters: {
                type: "object",
                properties: { city: { type: "string" } },
                required: ["city"],
            },
        },
    },
];

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris?" }],
    tools,
});

if (response.choices[0].message.tool_calls) {
    const toolCall = response.choices[0].message.tool_calls[0];
    if (toolCall.type === "function") {
        console.log(`Function: ${toolCall.function.name}`);
        console.log(`Arguments: ${toolCall.function.arguments}`);
    }
}

import os
from arbytra import Client
import json

client = Client(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string"}
                },
                "required": ["city"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools
)

if response.choices[0].message.tool_calls:
    tool_call = response.choices[0].message.tool_calls[0]
    print(f"Function: {tool_call.function.name}")
    print(f"Arguments: {json.loads(tool_call.function.arguments)}")

import { Client } from "@arbytra/sdk";

const client = new Client({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseUrl: "https://api.arbytra.com/v1",
});

const tools = [
    {
        type: "function" as const,
        function: {
            name: "get_weather",
            description: "Get weather for a city",
            parameters: {
                type: "object",
                properties: {
                    city: { type: "string" },
                },
                required: ["city"],
            },
        },
    },
];

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris?" }],
    tools,
});

if (response.choices[0].message.tool_calls) {
    const toolCall = response.choices[0].message.tool_calls[0];
    console.log(`Function: ${toolCall.function.name}`);
    console.log(`Arguments: ${toolCall.function.arguments}`);
}

Execute tool calls

After receiving tool calls, execute them and send the results back:

import json

def get_weather(city: str) -> str:
    return f"Weather in {city}: 72F, sunny"

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools
)

message = response.choices[0].message
if message.tool_calls:
    messages = [
        {"role": "user", "content": "What's the weather in Paris?"},
        message.model_dump(exclude_none=True),
    ]
    
    for tool_call in message.tool_calls:
        args = json.loads(tool_call.function.arguments)
        if tool_call.function.name == "get_weather":
            result = get_weather(args["city"])
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": result
        })
    
    final_response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools
    )
    print(final_response.choices[0].message.content)

function getWeather(city: string): string {
    return `Weather in ${city}: 72°F, sunny`;
}

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris?" }],
    tools,
});

const message = response.choices[0].message;
if (message.tool_calls) {
    const messages: OpenAI.ChatCompletionMessageParam[] = [
        { role: "user", content: "What's the weather in Paris?" },
        message,
    ];

    for (const toolCall of message.tool_calls) {
        if (toolCall.type !== "function") continue;
        const args = JSON.parse(toolCall.function.arguments);
        let result = "";
        if (toolCall.function.name === "get_weather") {
            result = getWeather(args.city);
        }
        messages.push({ role: "tool", tool_call_id: toolCall.id, content: result });
    }

    const finalResponse = await client.chat.completions.create({
        model: "gpt-4o",
        messages,
        tools,
    });

    console.log(finalResponse.choices[0].message.content);
}

import json

def get_weather(city: str) -> str:
    # Your actual implementation here
    return f"Weather in {city}: 72°F, sunny"

# Step 1: Initial request
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools
)

# Step 2: Check for tool calls
message = response.choices[0].message
if message.tool_calls:
    # Build message history
    messages = [
        {"role": "user", "content": "What's the weather in Paris?"},
        message.model_dump(exclude_none=True),  # Assistant message with tool_calls
    ]
    
    # Execute each tool call
    for tool_call in message.tool_calls:
        args = json.loads(tool_call.function.arguments)
        
        if tool_call.function.name == "get_weather":
            result = get_weather(args["city"])
        
        # Add tool result
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": result
        })
    
    # Step 3: Get final response
    final_response = client.chat.completions.create(
        model="gpt-4o",
        messages=messages,
        tools=tools
    )
    
    print(final_response.choices[0].message.content)

import { Client } from "@arbytra/sdk";

const client = new Client({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseUrl: "https://api.arbytra.com/v1",
});

function getWeather(city: string): string {
    return `Weather in ${city}: 72°F, sunny`;
}

// Step 1: Initial request
const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris?" }],
    tools,
});

// Step 2: Check for tool calls
const message = response.choices[0].message;
if (message.tool_calls) {
    const messages: any[] = [
        { role: "user", content: "What's the weather in Paris?" },
        message,
    ];

    // Execute each tool call
    for (const toolCall of message.tool_calls) {
        const args = JSON.parse(toolCall.function.arguments);
        let result = "";
        if (toolCall.function.name === "get_weather") {
            result = getWeather(args.city);
        }
        messages.push({ role: "tool", tool_call_id: toolCall.id, content: result });
    }

    // Step 3: Get final response
    const finalResponse = await client.chat.completions.create({
        model: "gpt-4o",
        messages,
        tools,
    });

    console.log(finalResponse.choices[0].message.content);
}

model_dump(exclude_none=True) preserves all tool call fields while stripping None fields that some providers reject. Some providers attach a cryptographic signature to tool calls for multi-turn verification. Using model_dump(exclude_none=True) ensures the signature is echoed back correctly.

Use multiple tools

Define multiple tools in the same request:

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the web for information",
            "parameters": {
                "type": "object",
                "properties": {"query": {"type": "string"}},
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "send_email",
            "description": "Send an email",
            "parameters": {
                "type": "object",
                "properties": {
                    "to": {"type": "string"},
                    "subject": {"type": "string"},
                    "body": {"type": "string"}
                },
                "required": ["to", "subject", "body"]
            }
        }
    }
]

const tools: OpenAI.ChatCompletionTool[] = [
    {
        type: "function",
        function: {
            name: "get_weather",
            description: "Get weather for a city",
            parameters: {
                type: "object",
                properties: { city: { type: "string" } },
                required: ["city"],
            },
        },
    },
    {
        type: "function",
        function: {
            name: "search_web",
            description: "Search the web for information",
            parameters: {
                type: "object",
                properties: { query: { type: "string" } },
                required: ["query"],
            },
        },
    },
    {
        type: "function",
        function: {
            name: "send_email",
            description: "Send an email",
            parameters: {
                type: "object",
                properties: {
                    to: { type: "string" },
                    subject: { type: "string" },
                    body: { type: "string" },
                },
                required: ["to", "subject", "body"],
            },
        },
    },
];

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get weather for a city",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string"}},
                "required": ["city"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the web for information",
            "parameters": {
                "type": "object",
                "properties": {"query": {"type": "string"}},
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "send_email",
            "description": "Send an email",
            "parameters": {
                "type": "object",
                "properties": {
                    "to": {"type": "string"},
                    "subject": {"type": "string"},
                    "body": {"type": "string"}
                },
                "required": ["to", "subject", "body"]
            }
        }
    }
]

const tools = [
    {
        type: "function" as const,
        function: {
            name: "get_weather",
            description: "Get weather for a city",
            parameters: {
                type: "object",
                properties: { city: { type: "string" } },
                required: ["city"],
            },
        },
    },
    {
        type: "function" as const,
        function: {
            name: "search_web",
            description: "Search the web for information",
            parameters: {
                type: "object",
                properties: { query: { type: "string" } },
                required: ["query"],
            },
        },
    },
    {
        type: "function" as const,
        function: {
            name: "send_email",
            description: "Send an email",
            parameters: {
                type: "object",
                properties: {
                    to: { type: "string" },
                    subject: { type: "string" },
                    body: { type: "string" },
                },
                required: ["to", "subject", "body"],
            },
        },
    },
];

Use parallel tool calls

Models can request multiple tool calls in parallel:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Paris and Tokyo?"}],
    tools=tools
)

if response.choices[0].message.tool_calls:
    for tool_call in response.choices[0].message.tool_calls:
        print(f"{tool_call.function.name}: {tool_call.function.arguments}")

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris and Tokyo?" }],
    tools,
});

if (response.choices[0].message.tool_calls) {
    for (const toolCall of response.choices[0].message.tool_calls) {
        if (toolCall.type !== "function") continue;
        console.log(`${toolCall.function.name}: ${toolCall.function.arguments}`);
    }
}

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{
        "role": "user",
        "content": "What's the weather in Paris and Tokyo?"
    }],
    tools=tools
)

# May return two tool calls
if response.choices[0].message.tool_calls:
    for tool_call in response.choices[0].message.tool_calls:
        print(f"{tool_call.function.name}: {tool_call.function.arguments}")

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris and Tokyo?" }],
    tools,
});

if (response.choices[0].message.tool_calls) {
    for (const toolCall of response.choices[0].message.tool_calls) {
        console.log(`${toolCall.function.name}: ${toolCall.function.arguments}`);
    }
}

Control tool choice

Control which tools the model can use:

# Let model decide
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="auto"
)

# Force tool use
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="required"
)

# Force specific tool
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "get_weather"}}
)

# Disable tools
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="none"
)

// Let model decide
const auto = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: "auto",
});

// Force tool use
const required = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: "required",
});

// Force specific tool
const specific = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: { type: "function", function: { name: "get_weather" } },
});

// Disable tools
const none = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: "none",
});

# Let model decide
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="auto"  # default
)

# Force tool use
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="required"
)

# Force specific tool
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice={"type": "function", "function": {"name": "get_weather"}}
)

# Disable tools
response = client.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    tools=tools,
    tool_choice="none"
)

// Let model decide
const auto = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: "auto",
});

// Force tool use
const required = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: "required",
});

// Force specific tool
const specific = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: { type: "function", function: { name: "get_weather" } },
});

// Disable tools
const none = await client.chat.completions.create({
    model: "gpt-4o", messages, tools,
    tool_choice: "none",
});

tool_choice controls whether the model calls tools:

"auto" (default): The model decides whether to call a tool based on the conversation.
"required": The model must call at least one tool. Most providers support this; see below for exceptions.
{"type": "function", "function": {"name": "..."}}: The model must call the specified tool.
"none": The model won’t call any tools. Arbytra ensures no tool calls are produced, even on providers that don’t respect "none". These requests can route to more providers, improving availability.

Provider-specific behavior

`tool_choice` with reasoning models

Some providers activate reasoning by default for certain models. When reasoning is active, these providers don’t fully support tool_choice="required" or named tool_choice. Arbytra handles this automatically:

Provider	Affected models	Behavior
DeepSeek	`deepseek-v4-flash`, `deepseek-v4-pro`	Arbytra routes `"required"` requests to other providers that honor the constraint. Use `tool_choice: "auto"` directly to avoid routing constraints.
Moonshot	`kimi-k2.5`, `kimi-k2.6`	Arbytra suppresses default thinking to honor `tool_choice="required"`. If you explicitly enable thinking (`reasoning_effort` > `"off"`), Arbytra routes to alternative providers; if none are available, you receive a routing error.

These models still call tools reliably with tool_choice="auto". The constraint only affects forcing tool use. See Extensions and Thinking for details on reasoning_effort and how Arbytra translates it per provider.

These constraints originate from the providers’ APIs. Third-party hosts (e.g., Fireworks) serving the same model weights typically don’t have this restriction.

`tool_choice="required"` provider support

Arbytra filters out providers known not to honor tool_choice="required" when routing these requests.

Provider	Affected models	Reason
DeepSeek	`deepseek-v4-flash`, `deepseek-v4-pro`	Reasoning models reject `"required"` with a 400 error
MiniMax	All models	Models silently ignore `"required"` and behave as `"auto"`
SiliconFlow	All models	Models return malformed tool calls or errors with `"required"` on SiliconFlow
Z.AI	All models	Models ignore the `"required"` constraint and may respond with text
DeepInfra	`llama-3.1`	Model returns text instead of honoring the `"required"` constraint
DeepInfra	`qwen-3.5`	Model rejects `"required"` with a 400 error
DeepInfra	GLM family (`glm-4.6`, `glm-4.7`, `glm-4.7-flash`, `glm-5`, `glm-5.1`)	Models ignore the `"required"` constraint and may respond with text
Together AI	`glm-5.1`	Model ignores the `"required"` constraint and may respond with text

If no capable provider is available for the requested model, the API returns a tool_choice_required_not_supported error. To resolve this:

Use tool_choice: "auto". Models still call tools when prompted appropriately.
Remove the provider constraint to allow routing to a capable provider.

With tool_choice: "auto", models still call tools when the prompt makes it appropriate. The model isn’t forced to call a tool and may respond with text instead. In practice, well-prompted requests still produce tool calls reliably. Named tool_choice ({type: "function", function: {name: "..."}}) isn’t affected by this filtering. Only the string value "required" triggers provider filtering.

If you need guaranteed forced tool use, exclude affected providers using exclude_providers to route to a provider that fully supports tool_choice: "required".

Stream tool calls

Reassemble streamed tool call chunks into complete function calls:

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
    stream=True
)

tool_calls = {}
for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta
    
    if delta.tool_calls:
        for tc in delta.tool_calls:
            idx = tc.index
            if idx not in tool_calls:
                tool_calls[idx] = {"id": tc.id, "function": {"name": "", "arguments": ""}}
            if tc.function and tc.function.name:
                tool_calls[idx]["function"]["name"] += tc.function.name
            if tc.function and tc.function.arguments:
                tool_calls[idx]["function"]["arguments"] += tc.function.arguments

print(list(tool_calls.values()))

const stream = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris?" }],
    tools,
    stream: true,
});

const toolCalls: Record<number, { id: string; function: { name: string; arguments: string } }> = {};
for await (const chunk of stream) {
    if (!chunk.choices.length) continue;
    const delta = chunk.choices[0].delta;

    if (delta.tool_calls) {
        for (const tc of delta.tool_calls) {
            if (!(tc.index in toolCalls)) {
                toolCalls[tc.index] = { id: tc.id!, function: { name: "", arguments: "" } };
            }
            if (tc.function?.name) toolCalls[tc.index].function.name += tc.function.name;
            if (tc.function?.arguments) toolCalls[tc.index].function.arguments += tc.function.arguments;
        }
    }
}

console.log(Object.values(toolCalls));

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
    stream=True
)

tool_calls = {}
for chunk in stream:
    if not chunk.choices:
        continue
    delta = chunk.choices[0].delta
    
    if delta.tool_calls:
        for tc in delta.tool_calls:
            idx = tc.index
            if idx not in tool_calls:
                tool_calls[idx] = {"id": tc.id, "function": {"name": "", "arguments": ""}}
            if tc.function and tc.function.name:
                tool_calls[idx]["function"]["name"] += tc.function.name
            if tc.function and tc.function.arguments:
                tool_calls[idx]["function"]["arguments"] += tc.function.arguments

print(list(tool_calls.values()))

const stream = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What's the weather in Paris?" }],
    tools,
    stream: true,
});

const toolCalls: Record<number, { id: string; function: { name: string; arguments: string } }> = {};
for await (const chunk of stream) {
    if (!chunk.choices.length) continue;
    const delta = chunk.choices[0].delta;

    if (delta.tool_calls) {
        for (const tc of delta.tool_calls) {
            if (!(tc.index in toolCalls)) {
                toolCalls[tc.index] = { id: tc.id!, function: { name: "", arguments: "" } };
            }
            if (tc.function?.name) toolCalls[tc.index].function.name += tc.function.name;
            if (tc.function?.arguments) toolCalls[tc.index].function.arguments += tc.function.arguments;
        }
    }
}

console.log(Object.values(toolCalls));

See Streaming Guide for full streaming patterns including error handling and metadata access.

Convert legacy functions

Arbytra auto-converts the deprecated functions/function_call parameters to the modern tools/tool_choice format:

Legacy parameter	Converted to	Condition
`functions`	`tools`	Only if `tools` is absent
`function_call: "auto"`	`tool_choice: "auto"`	Only if `tool_choice` is absent
`function_call: "none"`	`tool_choice: "none"`	Only if `tool_choice` is absent
`function_call: {name: "fn"}`	`tool_choice: {type: "function", function: {name: "fn"}}`	Only if `tool_choice` is absent

Conversion only runs when the legacy field is present and the modern field is absent. If both are present, the modern field takes precedence. Use tools/tool_choice for new code. Arbytra supports the legacy format for backward compatibility. Arbytra also normalizes legacy message formats in chat history:

Legacy message format	Converted to	Notes
`role: "function"` message	`role: "tool"` message	`name` replaced with synthesized `tool_call_id`
`assistant.function_call`	`assistant.tool_calls` entry	Original `function_call` field removed

Your existing chat histories with legacy function messages work without changes.

Most providers support tool calling, but subfeatures like parallel_tool_calls vary. Check /v1/directory/models for current capability details.

Best practices

Clear Descriptions

Write clear, specific function descriptions so the model knows when to use them.

Validate Arguments

Always validate tool call arguments before executing.

Error Handling

Return helpful error messages in tool results when execution fails.

Limit Tools

Only include relevant tools to reduce confusion and latency.

Get Started

Guides

Response API

Frameworks

Integrations

Specifications

Platform

Changelog

Prerequisites

Define tools

Call tools

Execute tool calls

Use multiple tools

Use parallel tool calls

Control tool choice

Provider-specific behavior

`tool_choice` with reasoning models

`tool_choice="required"` provider support

Stream tool calls

Convert legacy functions

Best practices

Clear Descriptions

Validate Arguments

Error Handling

Limit Tools

​Prerequisites

​Define tools

​Call tools

​Execute tool calls

​Use multiple tools

​Use parallel tool calls

​Control tool choice

​Provider-specific behavior

​tool_choice with reasoning models

​tool_choice="required" provider support

​Stream tool calls

​Convert legacy functions

​Best practices

Clear Descriptions

Validate Arguments

Error Handling

Limit Tools

Prerequisites

Define tools

Call tools

Execute tool calls

Use multiple tools

Use parallel tool calls

Control tool choice

Provider-specific behavior

`tool_choice` with reasoning models

`tool_choice="required"` provider support

Stream tool calls

Convert legacy functions

Best practices