Structured Output

You can force models to return valid JSON, optionally conforming to a specific schema.

Prerequisites

An Arbytra API key
Python 3.10+ with the OpenAI SDK (pip install openai) or the arbytra SDK (pip install arbytra)
- OR Node.js 18+ with the OpenAI SDK (npm install openai) or @arbytra/sdk (npm install @arbytra/sdk)

Choose a response format

Arbytra supports three response format types:

Type	Description	Use case
`text`	Default. Model returns plain text.	General chat, creative writing
`json_object`	Model returns valid JSON. No schema enforcement.	Flexible JSON extraction
`json_schema`	Model returns JSON matching a provided schema.	Typed data extraction, API responses

json_schema and json_object are separate capabilities. json_schema has broader model support. Most Claude models support json_schema but not json_object. Claude Sonnet 4 and Opus 4 don’t support either mode. If you request an unsupported mode, Arbytra returns a 400 with a suggested alternative.

Check per-model support on the models page or via the Model directory. json_schema appears as Structured Output, json_object as JSON Mode.

Return JSON

To return any JSON output, set response_format to json_object:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1",
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Extract the user's name and age as JSON."},
        {"role": "user", "content": "I'm Alice and I'm 30 years old."},
    ],
    response_format={"type": "json_object"},
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [
        { role: "system", content: "Extract the user's name and age as JSON." },
        { role: "user", content: "I'm Alice and I'm 30 years old." },
    ],
    response_format: { type: "json_object" },
});

console.log(response.choices[0].message.content);

import os
from arbytra import Client

client = Client(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "Extract the user's name and age as JSON."},
        {"role": "user", "content": "I'm Alice and I'm 30 years old."}
    ],
    response_format={"type": "json_object"}
)

print(response.choices[0].message.content)
# {"name": "Alice", "age": 30}

import { Client } from "@arbytra/sdk";

const client = new Client({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseUrl: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [
        { role: "system", content: "Extract the user's name and age as JSON." },
        { role: "user", content: "I'm Alice and I'm 30 years old." },
    ],
    response_format: { type: "json_object" },
});

console.log(response.choices[0].message.content);
// {"name": "Alice", "age": 30}

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "Extract the user'\''s name and age as JSON."},
      {"role": "user", "content": "I'\''m Alice and I'\''m 30 years old."}
    ],
    "response_format": {"type": "json_object"}
  }'

The model returns valid JSON, but the structure isn’t guaranteed. For strict schema conformance, use json_schema instead.

When using json_object mode, always include the word “JSON” in your system or user message. Some providers require this and return a 400 error without it. Including it is harmless on providers that don’t enforce it. The json_schema mode does not have this requirement.The examples above include “JSON” in the system message.

Enforce schema

You can enforce a specific JSON structure by providing a schema:

import os
import json
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1",
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Extract: Alice is 30, lives in NYC, alice@example.com"},
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "ContactInfo",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                    "city": {"type": "string"},
                    "email": {"type": "string"},
                },
                "required": ["name", "age", "city", "email"],
            },
        },
    },
)

contact = json.loads(response.choices[0].message.content)
print(contact["name"])
print(contact["email"])

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [
        { role: "user", content: "Extract: Alice is 30, lives in NYC, alice@example.com" },
    ],
    response_format: {
        type: "json_schema",
        json_schema: {
            name: "ContactInfo",
            schema: {
                type: "object",
                properties: {
                    name: { type: "string" },
                    age: { type: "integer" },
                    city: { type: "string" },
                    email: { type: "string" },
                },
                required: ["name", "age", "city", "email"],
            },
        },
    },
});

const contact = JSON.parse(response.choices[0].message.content!);
console.log(contact.name);
console.log(contact.email);

import os
import json
from arbytra import Client

client = Client(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "user", "content": "Extract: Alice is 30, lives in NYC, alice@example.com"}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "ContactInfo",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "age": {"type": "integer"},
                    "city": {"type": "string"},
                    "email": {"type": "string"}
                },
                "required": ["name", "age", "city", "email"]
            }
        }
    }
)

contact = json.loads(response.choices[0].message.content)
print(contact["name"])   # Alice
print(contact["email"])  # alice@example.com

import { Client } from "@arbytra/sdk";

const client = new Client({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseUrl: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [
        { role: "user", content: "Extract: Alice is 30, lives in NYC, alice@example.com" },
    ],
    response_format: {
        type: "json_schema",
        json_schema: {
            name: "ContactInfo",
            schema: {
                type: "object",
                properties: {
                    name: { type: "string" },
                    age: { type: "integer" },
                    city: { type: "string" },
                    email: { type: "string" },
                },
                required: ["name", "age", "city", "email"],
            },
        },
    },
});

const contact = JSON.parse(response.choices[0].message.content!);
console.log(contact.name);   // Alice
console.log(contact.email);  // alice@example.com

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "user", "content": "Extract: Alice is 30, lives in NYC, alice@example.com"}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "ContactInfo",
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "city": {"type": "string"},
            "email": {"type": "string"}
          },
          "required": ["name", "age", "city", "email"]
        }
      }
    }
  }'

The json_schema object requires a name field. The schema field accepts a standard JSON Schema definition.

Arbytra automatically routes to providers that support your requested format. If no provider supports it, you get a clear error with suggestions.

Resources

Tool calling

Call functions from LLM responses

Routing options

Optimize for cost, latency, or throughput

Error handling

Handle errors and retries

Browse models

See which models support structured output and JSON mode

Get Started

Guides

Response API

Frameworks

Integrations

Specifications

Platform

Changelog

Structured Output

Prerequisites

Choose a response format

Return JSON

Enforce schema

Resources

Tool calling

Routing options

Error handling

Browse models

​Prerequisites

​Choose a response format

​Return JSON

​Enforce schema

​Resources

Tool calling

Routing options

Error handling

Browse models

Prerequisites

Choose a response format

Return JSON

Enforce schema

Resources