API Overview - Arbytra

The Arbytra API is OpenAI-compatible. You can use the OpenAI SDK with a base URL change.

Base URL

https://api.arbytra.com/v1

Make a request

Send a chat completion request:

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is the capital of France?"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseURL: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What is the capital of France?" }],
});

console.log(response.choices[0].message.content);

import os
from arbytra import Client

client = Client(
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1"
)

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "What is the capital of France?"}]
)

print(response.choices[0].message.content)

import { Client } from "@arbytra/sdk";

const client = new Client({
    apiKey: process.env.ARBYTRA_API_KEY,
    baseUrl: "https://api.arbytra.com/v1",
});

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "What is the capital of France?" }],
});

console.log(response.choices[0].message.content);

curl -X POST https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

To stream the response, set stream to true:

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing in one paragraph."}],
    stream=True
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

const stream = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Explain quantum computing in one paragraph." }],
    stream: true,
});

for await (const chunk of stream) {
    if (chunk.choices[0]?.delta?.content) {
        process.stdout.write(chunk.choices[0].delta.content);
    }
}

stream = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain quantum computing in one paragraph."}],
    stream=True
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

const stream = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Explain quantum computing in one paragraph." }],
    stream: true,
});

for await (const chunk of stream) {
    if (chunk.choices[0]?.delta?.content) {
        process.stdout.write(chunk.choices[0].delta.content);
    }
}

curl -X POST https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Explain quantum computing in one paragraph."}],
    "stream": true
  }'

To optimize for cost, add routing options:

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize the benefits of cloud computing."}],
    extra_body={"gateway": {"routing": {"optimize": "cost"}}}
)

print(response.choices[0].message.content)

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Summarize the benefits of cloud computing." }],
    gateway: { routing: { optimize: "cost" } },
});

console.log(response.choices[0].message.content);

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize the benefits of cloud computing."}],
    gateway={"routing": {"optimize": "cost"}}
)

print(response.choices[0].message.content)

const response = await client.chat.completions.create({
    model: "gpt-4o",
    messages: [{ role: "user", content: "Summarize the benefits of cloud computing." }],
    gateway: { routing: { optimize: "cost" } },
});

console.log(response.choices[0].message.content);

curl -X POST https://api.arbytra.com/v1/chat/completions \
  -H "Authorization: Bearer $ARBYTRA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Summarize the benefits of cloud computing."}],
    "gateway": {"routing": {"optimize": "cost"}}
  }'

Endpoints

Endpoint	Method	Description
`/v1/chat/completions`	POST	Create a chat completion
`/v1/responses`	POST	Create a response (Response API)
`/v1/models`	GET	List callable models
`/v1/models/{model_id}`	GET	Retrieve callable model
`/v1/me`	GET	Get API key identity
`/v1/registry/providers`	GET	Provider catalog
`/v1/registry/models`	GET	Canonical model records
`/v1/directory/models`	GET	Model directory
`/v1/byok/providers`	GET	BYOK provider discovery
`/v1/byok/providers/{provider}/tiers`	GET	BYOK provider account tiers
`/v1/workspaces/{workspace_id}/api-keys`	GET	List API keys
`/v1/workspaces/{workspace_id}/api-keys/{api_key_id}`	GET	Get API key
`/v1/workspaces/{workspace_id}/api-keys/{api_key_id}/usage`	GET	Get API key usage
`/v1/workspaces/{workspace_id}/api-keys`	POST	Create API key
`/v1/workspaces/{workspace_id}/api-keys/{api_key_id}`	PATCH	Update API key
`/v1/workspaces/{workspace_id}/api-keys/{api_key_id}`	DELETE	Revoke API key
`/v1/workspaces/{workspace_id}/byok-keys`	GET	List BYOK keys
`/v1/workspaces/{workspace_id}/byok-keys/{byok_key_id}`	GET	Get BYOK key
`/v1/workspaces/{workspace_id}/byok-keys`	POST	Create BYOK key
`/v1/workspaces/{workspace_id}/byok-keys/{byok_key_id}`	PATCH	Update BYOK key
`/v1/workspaces/{workspace_id}/byok-keys/{byok_key_id}`	DELETE	Delete BYOK key
`/v1/workspaces/{workspace_id}/billing/balance`	GET	Credit balance

OpenAI compatibility

Arbytra supports the same request/response format as OpenAI. Switch to Arbytra by changing two values:

Base URL: https://api.openai.com/v1 → https://api.arbytra.com/v1
API Key: Use your Arbytra API key (starts with ak_)

Arbytra extensions

Arbytra responses carry additional fields beyond the OpenAI format.

Response extensions

Every chat completion response includes a routing_metadata object with routing and cost details:

{
  "routing_metadata": {
    "provider": "openai",
    "provider_model_id": "gpt-4o-2024-08-06",
    "model_canonical": "gpt-4o",
    "routing_strategy": "balanced",
    "ttft_ms": 312,
    "cost": {
      "usd": 0.00015
    },
    "warnings": []
  }
}

When the gateway had to modify or ignore part of the request, warnings carries one or more structured entries:

{
  "routing_metadata": {
    "provider": "openai",
    "provider_model_id": "gpt-4o-2024-08-06",
    "model_canonical": "gpt-4o",
    "routing_strategy": "balanced",
    "warnings": [
      {
        "type": "unsupported_parameter",
        "code": "seed",
        "message": "Parameter 'seed' is not supported by the selected provider."
      }
    ]
  }
}

See Response metadata for the full field reference.

Request extensions

Pass routing options to optimize your requests via the gateway field:

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello!"}],
  "gateway": {
    "routing": {
      "optimize": "cost",
      "max_ttft_ms": 1000
    }
  }
}

Request metadata

Attach custom metadata to requests for tracking and observability via gateway.metadata. Arbytra strips this metadata before forwarding to the provider.

{
  "model": "gpt-4o",
  "messages": [{"role": "user", "content": "Hello!"}],
  "gateway": {
    "metadata": {
      "tags": ["production", "chatbot"],
      "user_id": "user_abc123",
      "trace_id": "trace_xyz789",
      "custom_fields": {
        "environment": "production",
        "feature": "customer-support"
      }
    }
  }
}

Field	Type	Limits
`tags`	`string[]`	Max 100 tags, each max 50 chars
`user_id`	`string`	Max 255 chars
`trace_id`	`string`	Max 255 chars
`custom_fields`	`Record<string, string>`	Max 10 fields, keys max 50 chars, values max 200 chars

Response headers

Every response carries custom headers. See Response headers for the full reference.

Authentication

Learn how to authenticate your requests

​Base URL

​Make a request

​Endpoints

​OpenAI compatibility

​Arbytra extensions

​Response extensions

​Request extensions

​Request metadata

​Response headers