> ## Documentation Index > Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt > Use this file to discover all available pages before exploring further. # LLM Gateway on a self-hosted install > Run the LLM Gateway in the Hybrid or Air-Gapped chart: upgrade, publish its host, size it and open its egress. On Hybrid and Air-Gapped installs, the gateway runs in your cluster. Model calls, provider keys and request records stay inside your environment, and settings are managed in the Platform as on xpander cloud. ![Diagram](https://docs.xpander.ai/images/diagrams/run-llm-gateway-self-hosted-daa98089.svg) ## Always on Agents send every model call through the gateway, and it can't be turned off. Since Hybrid chart 0.22.5 and Air-Gapped chart 0.17.2, the chart ignores `llm-gw.enabled`. Agents use the in-cluster service. To publish a host for laptops, apps and SDKs, set these values and upgrade the release: ```yaml theme={"dark"} llm-gw: ingress: enabled: true # publish llm-gw. internal: false # true: serve the host on a private load balancer ``` | Value | Default | What it does | | - | - | - | | `llm-gw.ingress.enabled` | `false` | Publishes `llm-gw.`. Needs the chart-wide ingress on | | `llm-gw.ingress.host` | `llm-gw.` | Another host name | | `llm-gw.ingress.internal` | `false` | Uses a private load balancer (`internalClassName`, `internalAnnotations`) | | `llm-gw.env` | | `HTTPS_PROXY` and `NO_PROXY` behind an outbound proxy | | `llm-gw.serviceAccountAnnotations` | | Workload identity for the gateway's service account | The in-cluster address, shown as **Base URL** on the Models page when no host is published, is `http://llm-gw:4000` on Air-Gapped and `http://xpander-llm-gw:4000` on Hybrid with the release name `xpander` ([release names](https://docs.xpander.ai/self-hosted/hybrid.md#what-gets-deployed)). A published host needs a DNS record and a certificate covering it. The chart reuses `ingress.tls.secretName`. Only the model paths are published (`/v1/`, `/anthropic/`, `/gemini/`, `/openai/`). While no gateway pod is ready, a fallback pod (`llm-gw-unavailable` on Air-Gapped, `xpander-llm-gw-unavailable` on Hybrid) answers with a JSON `503` that coding tools and SDKs retry. On Air-Gapped, people point their coding tools at the published host ([Connect your tools](https://docs.xpander.ai/run/llm-gateway/connect-tools.md)). On Hybrid, their coding tools connect to xpander cloud, so the host is only for your own apps and SDKs. ### Upgrading an existing install Every upgrade starts the gateway: `install.sh upgrade` and any `helm upgrade`, including `--reuse-values`. A leftover `llm-gw.enabled: false` is ignored. If the gateway was off, the upgrade starts it and restarts agent-controller and api once. Before you upgrade: 1. **Mirror the image**, if you mirror images. On Air-Gapped, run the current `image_sync.sh --manifest-out` first. It mirrors and pins the `llm-gw` and `tool-repo-seed` images. `--icon-pack-tag` no longer exists, so remove it from your scripts. If the image is not pinned, the render stops and names the value to set. 2. **Open the gateway's [network paths](#network).** 3. **Give every provider your agents use a key**, in [Models](https://docs.xpander.ai/run/llm-gateway/providers-and-keys.md) or on the install (`secrets.static`). A provider with neither is refused with a message that names the missing key. Custom OpenAI-compatible endpoints keep working. New agent runs wait for a healthy gateway. If its pods can't start, such as when the image can't be pulled, new runs are refused with "The xpander AI gateway is unavailable right now". Check after the upgrade: ```bash theme={"dark"} kubectl -n get pods -l app=llm-gw ``` A healthy gateway shows two `llm-gw` pods (one on a single-node `install.sh` cluster), each `1/1` and `Running`. ## Long streams and size A coding session can hold one response open for many minutes. The chart sets a 1500-second idle timeout on the ingress. Set at least 1500 seconds on any load balancer in front of it (on an AWS Application Load Balancer, `idle_timeout.timeout_seconds=1500`). The gateway runs 2 to 20 servers scaled on CPU (one on a single-node `install.sh` cluster), each 0.5 to 1 vCPU and 1 to 4 GiB. It has no database of its own: request records go to your install's PostgreSQL, so size that for request volume and retention ([Sizing](https://docs.xpander.ai/self-hosted/sizing.md)). ## Keys and secrets No provider key is set on the gateway. Provider keys stay in your vault or in `secrets.static` and are added to each call ([Providers and keys](https://docs.xpander.ai/run/llm-gateway/providers-and-keys.md)). | Key | Hybrid (release name `xpander`) | Air-Gapped | | - | - | - | | The gateway's internal key, generated on install and never used by callers | Secret `xpander-llm-gw`, or `llm-gw.masterKey.existingSecret` | Secret `xpander-llm-gw-keys`, or create the Secret with keygen off | | The key that encrypts saved prompt and response text | Secret `xpander-llm-gw-content`, or `llm-gw.contentKey.existingSecret` | `keygen.overrides.LLM_GW_CONTENT_KEY` | Never delete or rotate the content key: saved text can't be read after it changes. On Hybrid, both generated Secrets survive upgrade and uninstall. With ArgoCD, set both `existingSecret` values or have ArgoCD ignore the Secrets' `/data`, as in `examples/argocd-application.yaml`. ## Network The gateway reaches your model providers or OpenAI-compatible endpoints, plus the xpander services in your cluster and Redis. With the chart's network policy on, it accepts traffic only on its serving port, can never reach the cloud metadata address, and egresses only to DNS, agent-controller, Redis and public `https`. List anything else in `llm-gw.networkPolicy`: | Key | For | Ports | | - | - | - | | `providerCidrs` | Model endpoints on your own network | Any port | | `cloudEndpointCidrs` | Private cloud endpoints such as Bedrock or Azure | 443 | | `redisCidrs` | An external Redis | | Behind an outbound proxy, set `HTTPS_PROXY` and `NO_PROXY` in `llm-gw.env`. The gateway does not read `global.env`. A custom certificate authority is trusted as for the other services ([Air-gapped](https://docs.xpander.ai/self-hosted/airgap.md)). A sealed install (no internet egress) uses only endpoints in your network, with **Bring your own key** as the only license model. [Log destinations](https://docs.xpander.ai/run/llm-gateway/logs-export.md) only need to be reachable from your cluster. ## Hybrid specifics * **Settings** (policies, models, keys, budgets) are managed in the Platform and reach your install within seconds. * **Agents** on your install call your install's gateway, on your stored keys and your install's credentials. xpander's keys are never used. * **People's coding tools** connect to xpander cloud at `https://llm-gw.xpander.ai` with their personal xpander key ([Connect your tools](https://docs.xpander.ai/run/llm-gateway/connect-tools.md)). They run on the person's own Claude plan or a key you stored in xpander, never on xpander's keys or your install's credentials, and their prompt and response text is never kept in xpander cloud. A provider with no key stored in xpander shows *No key for this provider is stored in xpander, so tools on your computer can't use these models. Agents in your own environment may still use them.* * **Records** of agents on your install stay there and show in the Platform when your browser can reach your environment. Totals for billing go with billing events. Records from people's coding tools live in xpander cloud and show in Logs. * **Budgets**: see [Budgets on self-hosted installs](https://docs.xpander.ai/run/llm-gateway/budgets-and-blocks.md#self-hosted-installs). ## Next steps > **[Add provider keys](https://docs.xpander.ai/run/llm-gateway/providers-and-keys.md)** > > A key for every provider. > **[Plan the install's size](https://docs.xpander.ai/self-hosted/sizing.md)** > > PostgreSQL for volume and retention. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.