LangChain

Use Arbytra as your LLM provider in LangChain with a drop-in ChatOpenAI replacement. This integration is Python-only. For TypeScript, use the Vercel AI SDK integration or configure the OpenAI SDK directly with Arbytra’s base URL.

Prerequisites

An Arbytra API key

Installation

pip install "arbytra[langchain]"

Use SDK adapter

Use the ArbytraChatOpenAI adapter:

from arbytra.frameworks.langchain import ArbytraChatOpenAI

llm = ArbytraChatOpenAI(model="gpt-5.4")

ArbytraChatOpenAI extends LangChain’s ChatOpenAI with:

use_responses_api=False set by default (ensures routing metadata and typed error mapping)
Routing injection via extra_body
OpenAI error mapping to typed Arbytra error classes

from arbytra.frameworks.langchain import ArbytraChatOpenAI

llm = ArbytraChatOpenAI(model="gpt-5.4")

# Simple invoke
response = llm.invoke("What is 2+2?")
print(response.content)

# Streaming
for chunk in llm.stream("Count to 5"):
    print(chunk.content, end="", flush=True)

# With messages
from langchain_core.messages import HumanMessage, SystemMessage

messages = [
    SystemMessage(content="You are a helpful assistant."),
    HumanMessage(content="Explain quantum computing briefly."),
]
response = llm.invoke(messages)
print(response.content)

Configure options

ArbytraChatOpenAI accepts these parameters:

Parameter	Type	Default	Description
`model`	`str`	(required, via parent)	Model ID
`api_key`	`str \| None`	`ARBYTRA_API_KEY` env	API key
`routing`	`RoutingOptions \| None`	`None`	Routing configuration
`base_url`	`str`	`"https://api.arbytra.com/v1"`	API base URL
`**kwargs`			Passed through to `ChatOpenAI` (e.g., `temperature`, `max_tokens`)

Configure routing

You can pass a RoutingOptions instance to control cost, latency, and quality trade-offs:

from arbytra.frameworks.langchain import ArbytraChatOpenAI
from arbytra.route_types import RoutingOptions

llm = ArbytraChatOpenAI(
    model="gpt-5.4",
    routing=RoutingOptions(optimize="cost", max_ttft_ms=1000),
)

response = llm.invoke("Hello!")
print(response.content)

Access routing metadata through generation_info when using generate():

result = llm.generate([[HumanMessage(content="Hello!")]])
info = result.generations[0][0].generation_info
if info and "routing_metadata" in info:
    print(f"Provider: {info['routing_metadata']['provider']}")

Configure manually

Alternative: configure LangChain manually

If you prefer to use ChatOpenAI directly:

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-5.4",
    api_key=os.environ["ARBYTRA_API_KEY"],
    base_url="https://api.arbytra.com/v1",
    use_responses_api=False,
)

use_responses_api=False is the default. Both Chat Completions and Response API streaming include routing_metadata.

Alternative: use `ArbytraAsyncOpenAI` (experimental)

If you can’t use arbytra[langchain] (for example, your project pins a different langchain-openai version), pass ArbytraAsyncOpenAI into LangChain’s async_client parameter:

from langchain_openai import ChatOpenAI
from arbytra import ArbytraAsyncOpenAI

client = ArbytraAsyncOpenAI()
llm = ChatOpenAI(
    model="gpt-4o",
    async_client=client.chat.completions,
    api_key="placeholder",
)

Pass client.chat.completions (not the whole client) and provide any string as api_key (LangChain requires it for construction). Read client.last_routing_metadata after each call. See ArbytraAsyncOpenAI for the full class reference.

Notes

OpenAI API errors map to typed Arbytra error classes (RateLimitError, PermissionDeniedError, BadRequestError, etc.).
ArbytraChatOpenAI sets use_responses_api=False by default.

Get Started

Guides

Response API

Frameworks

Integrations

Specifications

Platform

Changelog

Prerequisites

Installation

Use SDK adapter

Configure options

Configure routing

Configure manually

Alternative: use `ArbytraAsyncOpenAI` (experimental)

Notes

​Prerequisites

​Installation

​Use SDK adapter

​Configure options

​Configure routing

​Configure manually

​Alternative: use ArbytraAsyncOpenAI (experimental)

​Notes

Prerequisites

Installation

Use SDK adapter

Configure options

Configure routing

Configure manually

Alternative: use `ArbytraAsyncOpenAI` (experimental)

Notes