> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cloud.cdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Gateway

> LLM Gateway is a centrally managed proxy that routes LLM traffic to external providers through a single unified endpoint.

***LLM Gateway*** is a centrally managed proxy that routes LLM requests from your applications and AI agents to external providers. Instead of each application holding its own provider API keys, the CData administrator registers providers once and governs all traffic through a single endpoint. Send requests to `https://cloud.cdata.com/gateway/v1`  and Connect AI forwards them to the configured provider. Authenticate using the virtual key you issue on the **Virtual Keys** tab. Your application never holds a raw provider API key.

<Note>The LLM Gateway supports the following external providers: Anthropic, Azure OpenAI, Cohere, Custom OpenAI-compatible providers, Google Gemini, and OpenAI.</Note>

The **LLM Gateway** page contains the following tabs:

* [**Projects**](/en/LLM-Gateway/Projects): Contains a list of projects, each scoped to a routing rule and identified by a user-defined name.
* [**Virtual Keys**](/en/LLM-Gateway/Virtual-Keys): Contains a list of virtual keys for authentication. Virtual keys are the credentials a client sends to the LLM Gateway.
* [**Routing Rules**](/en/LLM-Gateway/Routing-Rules): Contains a list of routing rules that determine where a project's traffic goes: to a single provider, a priority-failover chain, or based on a model allowlist.

## Get Started Using LLM Gateway

The following is an overview of setting up the LLM Gateway. Refer to the linked sections for details.

<Steps>
  <Step>
    **Register a provider.** On the [**LLM Providers**](/en/LLM-Gateway/LLM-Providers) page, click **+ Add Provider** and enter your provider name, type, and API key. This is the credential Connect AI uses to forward requests to the LLM provider. Next, select which of the provider's models to use (such as *claude-sonnet-5* for Anthropic or *gpt-4o* for OpenAI).
  </Step>

  <Step>
    **Create a routing rule.** On the [**Routing Rules**](/en/LLM-Gateway/Routing-Rules) tab of the **LLM Gateway** page, click **+ Add Routing Rule** and select a routing name, description, and type. Select **Single provider** to route all traffic to one provider, **Priority Failover** to route to a primary provider with automatic failover, or **Model Allowlist** to approve specific model–provider pairs. With **Model Allowlist**, the client names the model in each request, and the gateway forwards it to the paired provider or refuses it if the model is not on the list. Then configure the targets for your routing rule. If the organization-level default routing rule already suits your needs, you can skip this step and assign it when creating your project.
  </Step>

  <Step>
    **Create a project.** On the [**Projects**](/en/LLM-Gateway/Projects) tab of the **LLM Gateway** page, click **+ Add Project** and follow the instructions for creating a project. Add a project name and description, and then assign the routing rule you created in the previous step, or use the default routing rule. Save the project.
  </Step>

  <Step>
    **Issue a virtual key for the project.** On the **Virtual Keys** tab of your project, or on the [**Virtual Keys**](/en/LLM-Gateway/Virtual-Keys) tab of the main **LLM Gateway** page, click **+ Issue Virtual Key** and follow the instructions for issuing a virtual key.
  </Step>

  <Step>
    **Send requests through the gateway.** Point your LLM client at the Base URL for your LLM and authenticate with the virtual key, as follows:

    <Steps>
      <Step>
        **Choose your Base URL.** On the **Projects** tab of the **LLM Gateway** page, click **View Endpoints** to open a dialog showing the Gateway URL: `https://cloud.cdata.com/gateway/v1`.

        This URL is already complete. Do not add a suffix.
      </Step>

      <Step>
        **Configure your client.** Set your client's **Base URL** to the endpoint you copied. Pass the virtual key as the `Authorization: Bearer <virtual-key>` header or as the `x-api-key` header. The gateway selects the provider and model based on the project's routing rule.
        <Note>A Connect AI Personal Access Token is not accepted here; only the virtual key binds a request to its project.</Note>
      </Step>

      <Step>
        **Make a test request.** The following examples show a minimal request using different SDKs against the same base URL. Replace `<virtual-key>` with the key you issued.

        <CodeGroup>
          ```bash OpenAI-compatible (curl) theme={null}
          curl https://cloud.cdata.com/gateway/v1/chat/completions \
            -H "Authorization: Bearer <virtual-key>" \
            -H "Content-Type: application/json" \
            -d '{
              "messages": [{"role": "user", "content": "Hello"}]
            }'
          ```

          ```python OpenAI SDK (Python) theme={null}
          from openai import OpenAI

          client = OpenAI(
              base_url="https://cloud.cdata.com/gateway/v1",
              api_key="<virtual-key>",
          )

          response = client.chat.completions.create(
              messages=[{"role": "user", "content": "Hello"}],
          )
          print(response.choices[0].message.content)
          ```

          ```python Anthropic SDK (Python) theme={null}
          import anthropic

          client = anthropic.Anthropic(
              base_url="https://cloud.cdata.com/gateway/v1",
              api_key="<virtual-key>",
          )

          message = client.messages.create(
              max_tokens=1024,
              messages=[{"role": "user", "content": "Hello"}],
          )
          print(message.content)
          ```
        </CodeGroup>
      </Step>
    </Steps>
  </Step>
</Steps>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.