> ## Documentation Index > Fetch the complete documentation index at: https://docs.xpander.ai/llms.txt > Use this file to discover all available pages before exploring further. # Providers, keys and model access > Connect model providers, choose which models are on and who may use them, and decide whose key pays. Set up providers, keys and model access in [Models](https://platform.xpander.ai/governance/model-access) (**Platform > AI governance > Models**). Provider keys stay in your organization's vault. The LLM Gateway adds the key to each call on its way to the provider, so no person, agent or laptop ever receives one. ![Diagram](https://docs.xpander.ai/images/diagrams/run-llm-gateway-providers-and-keys-70998346.svg) ## The Models page ![Models page with the gateway base URL and the Providers list: each provider's details, status and how many models are on](https://mintcdn.com/xpanderai-099931d1/H-WYYv839kjeMNSx/images/llm-gateway/05-models-page.png?fit=max\&auto=format\&n=H-WYYv839kjeMNSx\&q=85\&s=d10190fad6f88cbe1f4c95c237f54613) * **Base URL** is the address clients and SDKs call. On a self-hosted install that does not publish the gateway's host, it is the in-cluster address: `http://llm-gw:4000` on Air-Gapped, and `http://xpander-llm-gw:4000` on Hybrid with the release name `xpander` ([release names](https://docs.xpander.ai/self-hosted/hybrid.md#what-gets-deployed)). * **Get started** is a checklist until your first request. It hides once the required steps are done. * **Providers** lists each provider with its status: **Connected**, **Needs a key**, **All models off** or **Key error**. * **Organization defaults** sets the provider, model and credentials new agents start with. * **Who can use what** shows who can call which model ([below](#who-may-use-which-models)). ## Add a provider | Install | **Add provider** offers | Add a key to a listed provider | | - | - | - | | xpander cloud | **Model vendor** and **Custom endpoint** | **Use your own key…** in the row's menu, or **Add key** in its card | | Hybrid or Air-Gapped | A custom endpoint | **Use your own key…** in the row's menu | On xpander cloud, choose **Add provider**, pick the **Provider** (such as Anthropic, OpenAI, Google Gemini, Amazon Bedrock or Microsoft Foundry), paste the **API key**, name it in uppercase and underscores (such as `PROD_ANTHROPIC`), and choose **Add provider**. Your own provider keys are part of the Enterprise plan. Without it, the page shows *Bring-your-own LLM keys is an Enterprise feature* with **Contact us**. ### Amazon Bedrock with an IAM role Set **Credential** to **IAM role** to store no secret: xpander assumes the role for short-lived credentials on each use. Follow the policies the dialog shows, enter the **Role ARN** and **Region**, choose **Test connection**, then **Add role**. On xpander cloud, your organization ID is the external ID, so only your organization can use the role. Full steps, including self-hosted installs: [Amazon Bedrock with an IAM role](https://docs.xpander.ai/run/integrations/bedrock-iam-role.md). ### Azure endpoint Each **Microsoft Foundry** key needs the **Azure endpoint** of its resource, from **Keys and Endpoint** in the Azure portal, such as `https://my-resource.openai.azure.com`. The **API version** (such as `2024-10-21`) is optional. Change it later with **Set endpoint** or **Edit endpoint** in the key's menu. A key without one shows **No endpoint**, and calls that would use it are refused until you add one. On a self-hosted install whose Azure endpoint was set at install time, a key's own endpoint is optional. ### Custom endpoints Connect a model server or AI gateway you run, such as vLLM, Ollama or an internal gateway. Any admin who can add provider keys can add one. On xpander cloud, choose **Add provider**, then **Custom endpoint**. On Hybrid or Air-Gapped, choose **Add provider**. ![Add provider form with Name Internal vLLM, ID internal\_vllm, Protocol OpenAI-compatible, a Base URL ending in /v1, API key No key, and Models with Test connection and Add model](https://mintcdn.com/xpanderai-099931d1/H-WYYv839kjeMNSx/images/llm-gateway/06-custom-endpoint-form.png?fit=max\&auto=format\&n=H-WYYv839kjeMNSx\&q=85\&s=291624eb4f0d1060402a67822b7cc177) Adding a custom endpoint: name, ID, protocol, base URL, API key and models. **1. Name it** Enter a **Name** and the **ID (used in API calls)**. The ID can't change later. **2. Pick the protocol and base URL** Choose **OpenAI-compatible** or **Anthropic-compatible**, and enter the **Base URL** with its `/v1` path. On xpander cloud it must be public `https`, such as `https://ai-gateway.example.com/v1`. On Hybrid or Air-Gapped it can be inside your network, such as `http://vllm.ai-tools.svc.cluster.local:8000/v1`. **3. Pick a key** **No key (local or unauthenticated server)**, **Use an existing key** or **Create a new key**. **4. List the models and save** **Test connection** lists what the server offers, or **Add model** adds IDs yourself. Choose **Add provider**. The models are on for your organization right away. To track spend, set a price per model with **Edit model** under **Keys and models…**, in USD per 1M tokens. Leave it at 0 if you don't pay per token. The row's menu also has **Check now**, **Edit provider** and **Remove provider**. New models that a vLLM, SGLang or NVIDIA NIM server starts serving are found with their context length and wait for an admin to approve them. Models already in use stay approved. #### Test model **Test model** in a custom model's menu replays real Claude Code, Codex and OpenCode requests on the model and reports each check as passed, fixed or failed. It checks that the model is reachable through the gateway, streams in the client's format, makes a function call and continues after the result, accepts the client's system prompt layout, thinking and effort settings and full list of function definitions, and reports a full context in the client's own error shape. A failed check names the cause and the fix, such as the vLLM `--tool-call-parser` flag. #### Endpoint health checks Every 2 minutes, the gateway runs the same agent check on one of each custom endpoint's models (xpander's own endpoints are not checked). | When | Result | | - | - | | Two failed checks in a row | Marked **down** | | 5 or more provider errors in one 2-minute window, with no more successes than errors | Marked **down** by agent traffic. Refused keys, bad requests and full contexts don't count. | | One passing check | Healthy again | A down endpoint shows a red banner in [Models](https://platform.xpander.ai/governance/model-access) with the failed step, the cause, the fix and the last 24 checks. **Open in Test model** reruns everything. Model admins are notified in the app, by email and on Slack when it goes down and when it recovers. #### How the gateway adapts requests to each model Each custom model has a **profile** that reshapes every request to fit it. Open it with **View profile** (**Edit profile** if you may change it). | Setting | Options | | - | - | | **Thinking** | Pass, Strip, Map to reasoning effort | | **Effort** | Pass, Strip, Map to `reasoning_effort`, Map to `chat_template_kwargs` (the `enable_thinking` and `thinking` switches) | | **System prompt** | Pass, Merge into the first message | | `Max tools` | Refuse requests with more function definitions, before the model runs | | **Context length** | Refuse requests that don't fit, before the model runs | A context-full refusal uses each CLI's own error, so the CLI compacts as usual: OpenAI's `context_length_exceeded` for Codex and OpenCode, Anthropic's "prompt is too long" for Claude Code. Values come from four sources: **Family default** (built in for a model family), **Learned by Test model** (a check that found a fix saves it), **Set by an admin** (wins over the other two), and **Discovered** (the context length the endpoint reports, used when nothing else sets one). Family defaults: gpt-oss maps effort to `reasoning_effort`. Qwen, DeepSeek and GLM map it to `chat_template_kwargs`. Kimi and Nemotron strip thinking and effort, and Kimi accepts at most 128 function definitions. The engine and version are read from vLLM (`/version`), SGLang (`/get_server_info`) or NVIDIA NIM (`/v1/version`), with each model's `max_model_len`. #### A server with no key With **No key (local or unauthenticated server)**, every call carries a placeholder, never a real key. Gateway callers name the model `::`, such as `internal_vllm::llama-3.3-70b`. **Check now** and **Test connection** still work. Calls show in [Logs](https://platform.xpander.ai/governance/logs) at the price set with **Edit model**, and are not charged in credits. > [!WARNING] > > On xpander cloud the endpoint must be public, so anyone who finds its address can use it. Protect the server with a key, or connect it from a Hybrid or Air-Gapped install, where it can stay private. To connect an AI gateway your company already runs, see [Use xpander with your existing AI gateway](https://docs.xpander.ai/run/llm-gateway/existing-gateway.md). ### Model endpoints on xpander cloud | Install | Allowed | Refused, when saved and again on every call | | - | - | - | | xpander cloud | Public `https` addresses only | Private, loopback, link-local, cloud metadata and cluster addresses: *Model endpoints are called at public addresses here, so this endpoint was not saved. Use the endpoint's public https address, or keep the provider's own endpoint.* | | Hybrid or Air-Gapped | Private addresses inside your network | Link-local and cloud metadata addresses (`169.254.0.0/16`, `fe80::/10`, the cloud metadata hosts), checked against the address actually dialled | This applies to Azure endpoints, OpenAI-compatible endpoints and model endpoints set on an agent. ## Turn models on Open a provider's row (or **Keys and models…**) and tick the models the organization may use. Prices are USD per 1M tokens. **Disable provider** or **Disable model** stops it everywhere. A model blocked at the gateway shows **Off** (*No one in the organization can use this model.*) until you choose **Unblock at the gateway**. ## Who may use which models ![Access panel for Amazon Bedrock listing Organization and the groups Default, Design, Engineering and Sales, each with All models ticked](https://mintcdn.com/xpanderai-099931d1/H-WYYv839kjeMNSx/images/llm-gateway/07-who-may-use-it.png?fit=max\&auto=format\&n=H-WYYv839kjeMNSx\&q=85\&s=70ba6a0afe2e842e00fd684757fd69ff) Who may use it: the organization and each group, with which of the provider's models they may use. Shown with sample data. **Who may use it…** in a provider's or model's menu sets what the **Organization** and each group may use: **All models** (every model the provider has on) or a list. A group can use only what the organization has on. **Who can use what** shows the result **By person**, **By group** or **By model**, with the reason for each block: **Blocked for everyone**, **Group switched off**, **Not on the organization's list** or **Not on the group's list**. It applies the gateway's own rules, so use it to answer "why can't this person use this model". ## Whose key pays A provider can hold several keys, such as one per team. In the provider's card under **LLM gateway**, set the **Organization default** (or **Automatic**), assign keys in **Give a group, person or agent its own key**, and confirm with **Check a person or agent**. A key marked **Rejected by the provider** was refused. **Used by the gateway** marks the one in use. Each call uses the first key assigned to: 1. the agent making the call 2. the person making the call 3. the person's primary group 4. the organization default for that provider Whether a stored key or xpander's key pays at all depends on your [license model](https://docs.xpander.ai/run/llm-gateway/which-key-pays.md). A provider with no key of your own shows the **xpander key** row (*Metered in credits.*) on xpander cloud, **Installation credentials** on a self-hosted install with a key in `secrets.static`, and **Needs a key** otherwise. Calls to a provider that needs a key are refused with a message that names it. ## Rotating a key Add and assign the new key, or replace the value in the vault. The next call uses it. A deleted key stops being used from the next call. ## Next steps > **[Decide who pays for each call](https://docs.xpander.ai/run/llm-gateway/which-key-pays.md)** > > License models and stored keys. > **[Connect your existing AI gateway](https://docs.xpander.ai/run/llm-gateway/existing-gateway.md)** > > Keep the gateway you run. This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.