> ## Documentation Index
> Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# AI gateway

> The LLM proxy every model call goes through: what it enforces, where it is configured, and how to route it through your own gateway.

The AI gateway is the LLM proxy on the Agent Controller. A session gets `ANTHROPIC_BASE_URL` (Claude Code) or `OPENAI_BASE_URL` (Codex) pointing at it, plus a session token. No executor, workspace or harness ever holds a provider key.

![Diagram](https://docs.xpander.ai/images/diagrams/run-integrations-ai-gateway-b1f50681.svg)

## What passes through it

* Every turn of an agent on Claude Code, Codex or OpenCode.
* xpander's own model calls: thread titles, context compaction, evaluation judges, and the **gateway model** that filters entries ahead of the agent's model.
* Guardrail moderation, when enabled.

For each call, the proxy resolves the credential, applies the model policy and budget, records the call under the person who asked, and forwards it.

### On a self-hosted install

The controller forwards calls to the [LLM Gateway](https://docs.xpander.ai/run/llm-gateway.md) in the cluster:

| Install | Gateway address |
| - | - |
| Air-Gapped | `http://llm-gw:4000` |
| Hybrid, with the release name `xpander` | `http://xpander-llm-gw:4000` ([release names](https://docs.xpander.ai/self-hosted/hybrid.md#what-gets-deployed)) |

* The gateway is always on. It applies allowed models, provider keys, fallbacks and budgets, and logs every call.
* Every provider your agents use needs a key, saved in the organization or set on the install (`secrets.static`). Otherwise the call is refused with a message naming the missing key.
* New runs wait for a healthy gateway. See [LLM Gateway on a self-hosted install](https://docs.xpander.ai/run/llm-gateway/self-hosted.md).
* Preflight refuses an install with no provider for model calls: an organization default, a gateway provider, or a per-provider in-cluster endpoint must exist.

## Where it is configured

| What | Where |
| - | - |
| Providers, your own keys, models and who may use them, aliases | **Platform > AI governance > Models** |
| Spending limits | **Platform > AI governance > Budgets**. See [Budgets](https://docs.xpander.ai/run/llm-gateway/budgets-and-blocks.md) |
| Usage and every request | **Platform > AI governance > Usage** and **Logs** |
| Install-level defaults | `global.llm.defaults.{provider,model}` and `global.llm.gateway.{provider,model,apiBase}` in the chart values |
| Amazon Bedrock | `global.llm.bedrock.region`, then `bearerTokenSecret`, `assumeRoleArn` (+ `assumeRoleExternalId`), or nothing for EKS Pod Identity / IRSA. Use `baseUrl` for a VPC endpoint |

The organization's default provider sets a new agent's harness: Anthropic gives Claude Code, Amazon Bedrock with an Anthropic model gives Claude Code on Bedrock, OpenAI gives Codex, any other provider gives OpenCode on that model.

## Routing through a gateway you already run

Each provider's upstream is an environment variable on the Agent Controller, so your own AI gateway can sit between xpander and the vendor:

| Provider | Controller environment |
| - | - |
| Anthropic | `LLM_PROXY_ANTHROPIC_BASE_URL` |
| Amazon Bedrock | `LLM_PROXY_AMAZON_BEDROCK_BASE_URL` (a VPC or PrivateLink runtime endpoint on Air-Gapped) |
| Any OpenAI-compatible runtime (vLLM, Ollama, NVIDIA NIM, your gateway) | The provider's `apiBase` under `global.llm` |

The controller egresses to `api.anthropic.com` by default. With an override, the override is the only host it needs. xpander applies its policy, budget and record before the call leaves. Your gateway's other clients stay outside xpander's controls.

## Verify

**1. Check the Agent Controller**

```bash theme={"dark"}
kubectl -n <namespace> port-forward service/xpander-agent-controller 9016:9016 &
curl http://localhost:9016/health
```

**2. Check the LLM Gateway pods**

```bash theme={"dark"}
kubectl -n <namespace> get pods -l app=llm-gw
```

A healthy gateway shows two `llm-gw` pods (one on a single-node cluster installed with `install.sh`), each `1/1` and `Running`.

**3. Check the product**

The **Locations** tab of **Platform > Agents > Runtime** shows the installation as **Connected**. A model call appears in the task's usage and in **Platform > AI governance > Logs**.

## Next steps

> **[Set up providers and keys](https://docs.xpander.ai/run/integrations/ai-vendors.md#provider-setup-on-a-self-hosted-install)**
>
> On a self-hosted install.

> **[Run a local model air-gapped](https://docs.xpander.ai/self-hosted/airgap.md#local-llm-inference)**
>
> Local LLM inference.

> **[Manage LLM keys on Hybrid](https://docs.xpander.ai/self-hosted/hybrid.md#managing-llm-api-keys)**
>
> Where provider keys live.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.