> ## Documentation Index
> Fetch the complete documentation index at: https://arbytra.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LlamaIndex

> Use LlamaIndex with Arbytra's routing and cost optimization

Use Arbytra as your LLM provider in LlamaIndex.

This integration is Python-only. For TypeScript, use the [Vercel AI SDK](/frameworks/vercel-ai-sdk) integration.

## Prerequisites

* An [Arbytra API key](https://arbytra.com/signup?redirectTo=%2Fdashboard%3Ftab%3Dapi-keys)

## Install

```bash theme={null}
pip install "arbytra[llamaindex]"
```

## Use SDK adapter

Use the `ArbytraLlamaIndexLLM` adapter:

```python theme={null}
from arbytra.frameworks.llamaindex import ArbytraLlamaIndexLLM

llm = ArbytraLlamaIndexLLM(model="gpt-5.4")
```

`ArbytraLlamaIndexLLM` supports chat, completion, streaming, async, per-call routing overrides, and Arbytra error mapping.

```python theme={null}
from arbytra.frameworks.llamaindex import ArbytraLlamaIndexLLM
from llama_index.core.llms import ChatMessage

llm = ArbytraLlamaIndexLLM(model="gpt-5.4")

response = llm.chat([ChatMessage(role="user", content="What is 2+2?")])
print(response.message.content)

for chunk in llm.stream_chat([ChatMessage(role="user", content="Count to 5")]):
    print(chunk.delta, end="", flush=True)
```

## Configure options

| Parameter  | Type                     | Default                      | Description                                                                 |
| ---------- | ------------------------ | ---------------------------- | --------------------------------------------------------------------------- |
| `model`    | `str`                    | (required)                   | Model ID                                                                    |
| `api_key`  | `str \| None`            | `ARBYTRA_API_KEY` env         | API key                                                                     |
| `routing`  | `RoutingOptions \| None` | `None`                       | Default routing configuration                                               |
| `api_base` | `str`                    | `"https://api.arbytra.com/v1"` | API base URL                                                                |
| `**kwargs` |                          |                              | Passed through to LlamaIndex's `OpenAI` (e.g., `temperature`, `max_tokens`) |

## Configure routing

Pass a `RoutingOptions` instance to set default routing:

```python theme={null}
from arbytra.frameworks.llamaindex import ArbytraLlamaIndexLLM
from arbytra.route_types import RoutingOptions

llm = ArbytraLlamaIndexLLM(
    model="gpt-5.4",
    routing=RoutingOptions(optimize="cost"),
)
```

Per-call routing overrides the instance default:

```python theme={null}
from arbytra.route_types import RoutingOptions

response = llm.chat(
    [ChatMessage(role="user", content="Hello!")],
    routing=RoutingOptions(optimize="tps-focus"),
)
```

Access routing metadata from the response:

```python theme={null}
response = llm.chat([ChatMessage(role="user", content="Hello!")])
metadata = response.additional_kwargs.get("routing_metadata")
if metadata:
    print(f"Provider: {metadata['provider']}")
```

## Configure manually

<Accordion title="Alternative: configure LlamaIndex manually">
  If you prefer to use LlamaIndex's `OpenAI` class directly:

  ```python theme={null}
  import os
  from llama_index.llms.openai import OpenAI

  llm = OpenAI(
      model="gpt-5.4",
      api_key=os.environ["ARBYTRA_API_KEY"],
      api_base="https://api.arbytra.com/v1",
  )
  ```

  For routing options, per-call overrides, and Arbytra error mapping, use `ArbytraLlamaIndexLLM`.
</Accordion>

## Use `ArbytraAsyncOpenAI` (experimental)

If your project pins a different `llama-index-llms-openai` version, pass `ArbytraAsyncOpenAI` as the `async_openai_client`:

```python theme={null}
from llama_index.llms.openai import OpenAI
from arbytra import ArbytraAsyncOpenAI

client = ArbytraAsyncOpenAI()
llm = OpenAI(
    model="gpt-4o",
    async_openai_client=client,
    api_key="placeholder",
)
```

<Note>
  LlamaIndex's `OpenAI` requires an `api_key` for construction. Pass any placeholder value.
</Note>

Read `client.last_routing_metadata` after each call. See [`ArbytraAsyncOpenAI`](/sdk/python-reference#arbytraasyncopenai-experimental) for the full class reference.