AI gateway

Every model call from every agent goes through xpander’s LLM proxy on the Agent Controller: what it enforces, where it is configured, and how to route it through a gateway you already run.

The AI gateway is the LLM proxy on the Agent Controller, the xpander service that mediates every model and skill call. No executor, workspace or harness holds a provider key. The client inside the session is pointed at the proxy instead. The controller then resolves the credential, applies the organization’s model policy and budget, records the call under the person who asked, and forwards it to the provider.

What passes through it

• Every turn of an agent on Claude Code, Codex or OpenCode. The session receives ANTHROPIC_BASE_URL (Claude Code) or OPENAI_BASE_URL (Codex) pointing at the proxy, with a session token instead of a provider key.
• xpander’s own model calls: thread titles, context compaction, evaluation judges, and the gateway model that runs entry filtering ahead of the agent’s model.
• Guardrail moderation, when enabled.

On a self-hosted install the controller forwards these calls to the LLM Gateway in the cluster. The address is http://llm-gw:4000 on Air-Gapped. On Hybrid with the release name xpander, it is http://xpander-llm-gw:4000 (release names). The gateway is always on. It applies the organization’s allowed models, provider keys, fallbacks and budgets, and meters and logs every call. Every provider your agents use needs a key, saved in the organization or set on the install (secrets.static). A call to a provider with neither is refused with a message that names the missing key. New runs wait for a healthy gateway. See LLM Gateway on a self-hosted install.

Preflight refuses a self-hosted install that has no provider configured for model calls: an organization default provider, a gateway provider, or a per-provider in-cluster endpoint must exist.

Where it is configured

What	Where
Providers, your own keys, which models are on and who may use them, aliases	Platform > AI governance > Models
Spending limits for the organization, groups, members, keys and agents	Platform > AI governance > Budgets; see Budgets
Usage per agent and per person, and every request	Platform > AI governance > Usage and Logs
Install-level defaults on a self-hosted cluster	global.llm.defaults.{provider,model} and global.llm.gateway.{provider,model,apiBase} in the chart values
Amazon Bedrock	global.llm.bedrock.region, then bearerTokenSecret, assumeRoleArn (+ assumeRoleExternalId), or nothing for EKS Pod Identity / IRSA; baseUrl for a VPC endpoint

Which harness a new agent starts on follows the organization’s default provider: Anthropic maps to Claude Code, Amazon Bedrock with an Anthropic model to Claude Code on Bedrock, OpenAI to Codex, any other provider to OpenCode on that model.

Routing through a gateway you already run

The proxy’s upstream per provider is an environment variable on the Agent Controller, so an existing AI gateway in your network can sit between xpander and the vendor:

Provider	Controller environment
Anthropic	LLM_PROXY_ANTHROPIC_BASE_URL
Amazon Bedrock	LLM_PROXY_AMAZON_BEDROCK_BASE_URL (a VPC or PrivateLink runtime endpoint on Air-Gapped)
Any OpenAI-compatible runtime (vLLM, Ollama, NVIDIA NIM, your gateway)	The provider’s apiBase under global.llm

The controller egresses to api.anthropic.com by default; with an override set, that is the only host it needs for Anthropic. xpander’s model policy, budget and record are applied on the controller before the call leaves; the gateway sees one more client, and its other clients stay outside xpander’s controls.

Verify

kubectl -n <namespace> port-forward service/xpander-agent-controller 9016:9016 &
curl http://localhost:9016/health

kubectl -n <namespace> get pods -l app=llm-gw

A healthy gateway shows two llm-gw pods (one on a single-node cluster installed with install.sh), each 1/1 and Running.

The Locations tab of Platform > Agents > Runtime shows the installation as Connected. A model call from any agent appears in the task’s usage and in Platform > AI governance > Logs.

Deep pages: AI vendors, Air-gapped: local LLM inference, Hybrid: managing LLM API keys.