Advanced Routing

Fine-tune routing with suffix shortcuts, multi-model requests, quality constraints, and data policies. For basic routing, see Routing options.

Prerequisites

An Arbytra API key
Python 3.10+ with the OpenAI SDK (pip install openai) or the arbytra SDK (pip install arbytra)
- OR Node.js 18+ with the OpenAI SDK (npm install openai) or @arbytra/sdk (npm install @arbytra/sdk)
Familiarity with Routing options

How routing works

When you send a request, Arbytra’s router:

Enumerates candidates — finds all providers offering the requested model(s)
Filters by constraints — removes providers that violate your routing options (data policy, Bring Your Own Key (BYOK) requirement, performance constraints, excluded providers)
Scores by strategy — ranks remaining candidates using your optimize strategy:
- cost: Cost-optimized, well-rounded
- cost-focus: Aggressively minimize cost (default)
- ttft: TTFT-optimized, well-rounded
- ttft-focus: Aggressively minimize time to first token
- tps: Throughput-optimized, well-rounded
- tps-focus: Aggressively maximize tokens per second
- balanced: All dimensions weighted evenly
Selects and routes — selects from the ranked list, favoring higher-scored providers
Falls back if needed — if the provider fails and allow_fallbacks is true, retries with the next candidate (up to max_fallback_attempts)

See Python SDK or TypeScript SDK for routing code examples.

Use suffix shortcuts

Append a suffix to any model name for quick routing configuration:

Suffix	Strategy	Description
`:cost-focus`	`cost-focus`	Aggressively minimize cost
`:cost`	`cost`	Cost-optimized, well-rounded
`:ttft-focus`	`ttft-focus`	Aggressively minimize time to first token
`:ttft`	`ttft`	TTFT-optimized, well-rounded
`:tps-focus`	`tps-focus`	Aggressively maximize tokens per second
`:tps`	`tps`	Throughput-optimized, well-rounded
`:balanced`	`balanced`	All dimensions weighted evenly

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

response = client.chat.completions.create(
    model="gpt-4o:cost-focus",
    messages=[{"role": "user", "content": "Hello!"}]
)

response = client.chat.completions.create(
    model="claude-sonnet-4-20250514:ttft",
    messages=[{"role": "user", "content": "Hello!"}]
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o:cost",
    messages: [{ role: "user", content: "Hello!" }],
});

console.log(response.choices[0].message.content);

import os
from arbytra import Client

client = Client(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

# Cost-focused provider for gpt-4o
response = client.chat.completions.create(
    model="gpt-4o:cost-focus",
    messages=[{"role": "user", "content": "Hello!"}]
)

# Fastest time to first token
response = client.chat.completions.create(
    model="claude-sonnet-4-20250514:ttft",
    messages=[{"role": "user", "content": "Hello!"}]
)

import { Client } from "@arbytra/sdk";

const client = new Client({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseUrl: "https://api.arbytra.com/v1",
});

// Cost-focused provider for gpt-4o
const response = await client.chat.completions.create({
    model: "gpt-4o:cost-focus",
    messages: [{ role: "user", content: "Hello!" }],
});

// Fastest time to first token
const fast = await client.chat.completions.create({
    model: "claude-sonnet-4-20250514:ttft",
    messages: [{ role: "user", content: "Hello!" }],
});

Suffixes work with any HTTP client:

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o:cost-focus",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

The router parses suffixes only when the model ID contains exactly one colon. Fine-tuned models with multiple colons (for example, ft:gpt-4o:org:custom) pass through unchanged.

Route across models

Pass gateway.models instead of model to route across multiple models (mutually exclusive with model, max 10):

response = client.chat.completions.create(
    model="",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {
        "models": ["gpt-4o", "claude-sonnet-4-20250514", "gemini-flash-latest"],
        "routing": {"mode": "pool"}
    }}
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: {
        models: ["gpt-4o", "claude-sonnet-4-20250514", "gemini-flash-latest"],
        routing: { mode: "pool" },
    },
});

console.log(response.choices[0].message.content);

import os
from arbytra import Client

client = Client(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

# Pool mode (default): best provider across all models
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"models": ["gpt-4o", "claude-sonnet-4-20250514", "gemini-flash-latest"], "routing": {"mode": "pool"}}
)

# Fallback mode: try models in order
response = client.chat.completions.create(
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"models": ["gpt-4o", "claude-sonnet-4-20250514"], "routing": {"mode": "fallback"}}
)

import { Client } from "@arbytra/sdk";

const client = new Client({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseUrl: "https://api.arbytra.com/v1",
});

// Pool mode (default): best provider across all models
const response = await client.chat.completions.create({
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { models: ["gpt-4o", "claude-sonnet-4-20250514", "gemini-flash-latest"], routing: { mode: "pool" } },
});

// Fallback mode: try models in order
const fallback = await client.chat.completions.create({
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { models: ["gpt-4o", "claude-sonnet-4-20250514"], routing: { mode: "fallback" } },
});

# Pool mode (default): best provider across all models
curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"models": ["gpt-4o", "claude-sonnet-4-20250514", "gemini-flash-latest"], "routing": {"mode": "pool"}}
  }'

# Fallback mode: try models in order
curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"models": ["gpt-4o", "claude-sonnet-4-20250514"], "routing": {"mode": "fallback"}}
  }'

Mode	Behavior
`pool` (default)	Select the best-scoring provider across all requested models
`fallback`	Try all providers for the first model, then the second model, and so on

Set quality constraints

Filter providers by performance requirements. Pass constraint ceilings under gateway.routing:

Field	Type	Description
`max_cost_per_1m`	number	Maximum cost per 1 million tokens (USD)
`max_ttft_ms`	integer	Maximum time to first token in milliseconds
`min_throughput_tps`	number	Minimum throughput in tokens per second

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {"routing": {
        "optimize": "balanced",
        "min_throughput_tps": 30,
        "weights": {"cost": 0.6, "ttft": 0.4}
    }}}
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: {
        optimize: "balanced",
        min_throughput_tps: 30,
        weights: { cost: 0.6, ttft: 0.4 },
    } },
});

console.log(response.choices[0].message.content);

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"routing": {
        "optimize": "balanced",
        "min_throughput_tps": 30,
        "weights": {"cost": 0.6, "ttft": 0.4}
    }}
)

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: {
        optimize: "balanced",
        min_throughput_tps: 30,
        weights: { cost: 0.6, ttft: 0.4 },
    } },
});

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"routing": {"optimize": "balanced", "min_throughput_tps": 30, "weights": {"cost": 0.6, "ttft": 0.4}}}
  }'

Constraint ceilings (max_ttft_ms, min_throughput_tps, max_cost_per_1m) evaluate against median (p50) metrics. To rank providers by worst-case (p95) TTFT or throughput for scoring, set ttft_percentile or throughput_percentile — see Choose metric percentile.

Filter by parameter support

Not all providers support every optional parameter. By default, Arbytra drops unsupported parameters and adds a warning to the response. Set require_parameters to true to only route to providers that accept the optional parameters you sent:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    seed=42,
    extra_body={"gateway": {"routing": {
        "optimize": "cost",
        "require_parameters": True,
    }}}
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-5.4",
    messages: [{ role: "user", content: "Hello!" }],
    top_p: 0.9,
    gateway: { routing: {
        optimize: "cost",
        require_parameters: true,
    } },
});

console.log(response.choices[0].message.content);

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    seed=42,
    gateway={
        "routing": {
            "optimize": "cost",
            "require_parameters": True,
        },
    }
)

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    seed: 42,
    gateway: {
        routing: {
            optimize: "cost",
            require_parameters: true,
        },
    },
});

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}],
    "seed": 42,
    "gateway": {"routing": {"optimize": "cost", "require_parameters": true}}
  }'

The following parameters have per-provider support. When you set require_parameters to true, Arbytra checks that your provider supports each one you sent: temperature, top_p, seed, logit_bias, logprobs, top_logprobs, n, presence_penalty, frequency_penalty, user, parallel_tool_calls, web_search_options, verbosity, prompt_cache_key, safety_identifier. require_parameters composes with other constraints. A provider must pass all filters to be eligible. You can check which parameters each provider supports via the model directory endpoint, where each provider entry includes accepted_params and supported_parameters fields.

Set custom weights

You can override preset strategies with custom weights across three dimensions.

Dimension	Field	What it controls
Cost	`cost`	Favor lower-cost providers
Latency	`ttft`	Favor lower time-to-first-token
Throughput	`throughput`	Favor higher tokens-per-second

Pass routing.weights with your desired dimensions:

# Only cost matters
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {"routing": {"weights": {"cost": 1}}}}
)

# Mostly cost, some latency
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {"routing": {"weights": {"cost": 0.7, "ttft": 0.3}}}}
)

# All three dimensions
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {"routing": {"weights": {"cost": 0.85, "ttft": 0.5, "throughput": 0.5}}}}
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

// Only cost matters
const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { weights: { cost: 1 } } },
});

// Mostly cost, some latency
const response2 = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { weights: { cost: 0.7, ttft: 0.3 } } },
});

// All three dimensions
const response3 = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { weights: { cost: 0.85, ttft: 0.5, throughput: 0.5 } } },
});

console.log(response3.choices[0].message.content);

# Only cost matters
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"routing": {"weights": {"cost": 1}}}
)

# Mostly cost, some latency
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"routing": {"weights": {"cost": 0.7, "ttft": 0.3}}}
)

# All three dimensions
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"routing": {
        "weights": {
            "cost": 0.85,
            "ttft": 0.5,
            "throughput": 0.5
        }
    }}
)

// Only cost matters
const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { weights: { cost: 1 } } },
});

// Mostly cost, some latency
const response2 = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { weights: { cost: 0.7, ttft: 0.3 } } },
});

// All three dimensions
const response3 = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: {
        weights: {
            cost: 0.85,
            ttft: 0.5,
            throughput: 0.5,
        },
    } },
});

# Only cost matters
curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"routing": {"weights": {"cost": 1}}}
  }'

# Mostly cost, some latency
curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"routing": {"weights": {"cost": 0.7, "ttft": 0.3}}}
  }'

# All three dimensions
curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"routing": {
      "weights": {
        "cost": 0.85,
        "ttft": 0.5,
        "throughput": 0.5
      }
    }}
  }'

The server accepts any non-negative numbers and normalizes them proportionally.
Omitted dimensions default to 0.
At least one dimension must be greater than 0.
weights overrides the optimize preset, and the response metadata contains routing_strategy: "custom".
To score using worst-case metrics, set ttft_percentile and/or throughput_percentile to "p95". See Choose metric percentile.

Providers approaching their rate limits are automatically deprioritized.

Choose metric percentile

By default, Arbytra scores providers using median (p50) metrics. You can switch to 95th-percentile (worst-case) independently for TTFT and throughput:

Field	Controls	Default
`ttft_percentile`	TTFT scoring (which providers rank higher)	`p50`
`throughput_percentile`	Throughput scoring (which providers rank higher)	`p50`

Both scoring fields accept "p50" (median) or "p95" (worst-case). They work with presets and custom weights — no weights required. Example — rank providers by worst-case (p95) TTFT instead of median:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {"routing": {
        "optimize": "ttft",
        "ttft_percentile": "p95"
    }}}
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: {
        optimize: "ttft",
        ttft_percentile: "p95",
    } },
});

console.log(response.choices[0].message.content);

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"routing": {
        "optimize": "ttft",
        "ttft_percentile": "p95"
    }}
)

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: {
        optimize: "ttft",
        ttft_percentile: "p95",
    } },
});

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"routing": {
      "optimize": "ttft",
      "ttft_percentile": "p95"
    }}
  }'

Data policy

Control how providers handle your data: Arbytra doesn’t store prompts or responses. Set data_policy: "zdr" to route only to providers that satisfy zero data retention.

Policy	Description
`none` (default)	No restrictions
`no_training`	Provider must not use data for training
`zdr`	Zero data retention — strictest policy

The hierarchy is zdr > no_training > none. When a per-request policy intersects with an account-level policy, the most restrictive one wins.

response = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Sensitive financial data..."}],
    extra_body={"gateway": {"routing": {"data_policy": "zdr"}}}
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { data_policy: "zdr" } },
});

console.log(response.choices[0].message.content);

response = client.chat.completions.create(
    model="claude-sonnet-4-6",
    messages=[{"role": "user", "content": "Sensitive financial data..."}],
    gateway={"routing": {"data_policy": "zdr"}}
)

const response = await client.chat.completions.create({
    model: "claude-sonnet-4-6",
    messages: [{ role: "user", content: "Sensitive financial data..." }],
    gateway: { routing: { data_policy: "zdr" } },
});

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-sonnet-4-6",
    "messages": [{"role": "user", "content": "Sensitive financial data..."}],
    "gateway": {"routing": {"data_policy": "zdr"}}
  }'

Opt in to premium tiers

Premium-tier offerings are excluded from routing by default. Set tier to opt in:

Value	Effect
`"priority"`	Includes Anthropic Fast Mode offerings (2.5x speed, 6x cost)
omitted (default)	Excludes premium-tier offerings

response = client.chat.completions.create(
    model="claude-opus-4-6",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {"routing": {"tier": "priority"}}}
)

const response = await client.chat.completions.create({
    model: "claude-opus-4-6",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { tier: "priority" } },
});

console.log(response.choices[0].message.content);

response = client.chat.completions.create(
    model="claude-opus-4-6",
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"routing": {"tier": "priority"}}
)

const response = await client.chat.completions.create({
    model: "claude-opus-4-6",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: { tier: "priority" } },
});

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-opus-4-6",
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"routing": {"tier": "priority"}}
  }'

Arbytra’s “priority” tier refers to Anthropic Fast Mode, not Anthropic’s separate Priority Tier (committed capacity SLA). See Routing options for details.

Provider alias normalization

Provider names in providers and exclude_providers are case-insensitive and support aliases:

Alias	Canonical name
`google`, `google_ai`, `googleai`, `gemini`	`google_ai_studio`
`fireworks`	`fireworks_ai`
`together`	`together_ai`

Unrecognized names pass through as-is (lowercased).

Configure fallbacks

By default, Arbytra retries with alternative providers on 429 (rate limit), 5xx (server error), and timeout responses.

Setting	Default	Description
`allow_fallbacks`	`true`	Enable automatic fallback to alternative providers
`max_fallback_attempts`	19	Safety ceiling on fallback attempts beyond the primary, range 1-19 (chain length 20 total)
`timeout_ms`	`120000` (streaming) / `300000` (non-streaming)	Per-attempt timeout in milliseconds. For streaming: time to first byte. For non-streaming: time to complete response
`deadline_ms`	None (streaming) / `1080000` (non-streaming)	Hard wall-clock cap across all fallback attempts. Non-streaming requests default to 18 minutes; streaming has no default (connections are long-lived). Set explicitly to override

You can configure per-attempt timeouts with timeout_ms. To set a hard wall-clock cap across all fallback attempts, use deadline_ms. Non-streaming requests have an 18-minute default deadline; streaming requests have no deadline (opt-in only). If the deadline is exceeded, the request fails with a timeout error.

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    extra_body={"gateway": {"routing": {
        "allow_fallbacks": True,
        "max_fallback_attempts": 5
    }}}
)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: {
        allow_fallbacks: true,
        max_fallback_attempts: 5,
    } },
});

console.log(response.choices[0].message.content);

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
    gateway={"routing": {
        "allow_fallbacks": True,
        "max_fallback_attempts": 5
    }}
)

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Hello!" }],
    gateway: { routing: {
        allow_fallbacks: true,
        max_fallback_attempts: 5,
    } },
});

curl https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}],
    "gateway": {"routing": {"allow_fallbacks": true, "max_fallback_attempts": 5}}
  }'

Get Started

Guides

Response API

Frameworks

Integrations

Specifications

Platform

Changelog

Advanced Routing

Prerequisites

How routing works

Use suffix shortcuts

Route across models

Set quality constraints

Filter by parameter support

Set custom weights

Choose metric percentile

Data policy

Opt in to premium tiers

Provider alias normalization

Configure fallbacks

​Prerequisites

​How routing works

​Use suffix shortcuts

​Route across models

​Set quality constraints

​Filter by parameter support

​Set custom weights

​Choose metric percentile

​Data policy

​Opt in to premium tiers

​Provider alias normalization

​Configure fallbacks

Prerequisites

How routing works

Use suffix shortcuts

Route across models

Set quality constraints

Filter by parameter support

Set custom weights

Choose metric percentile

Data policy

Opt in to premium tiers

Provider alias normalization

Configure fallbacks