Skip to main content
The LLM Providers page contains a list of available LLM providers, models, and the API keys the gateway uses to reach them. LLM Gateway supports many LLM providers, including Custom OpenAI-compatible providers.
LLM Gateway Providers
Each provider displays the following information:
  • Provider–the user-friendly name and type of the LLM provider.
  • Models–the LLM you are using (for example, gpt-4o or claude-sonnet-5).
  • Status–the status of the LLM provider: Active or Inactive.
  • API Key–a masked representation of the API key obtained from the LLM provider. The API key cannot be edited after the provider is registered. To update an API key, delete the provider and re-add it.
Each provider also contains edit and delete icons. On the edit page of the provider, you can edit the provider’s Name and add or delete Models. Each model’s costs are editable only when Override cost is selected. If you clear Override cost, the cost fields read Not set and cannot be edited. Click Save Changes when finished. Click the delete icon to remove the provider.
You must have a system role of Administrator to add, edit, or delete providers. Providers are account-level, so a Workspace Administrator cannot add, edit, or delete them. Non-administrators can view this page but cannot make any changes.

Add a Provider

For each provider, obtain your API key from that provider’s developer console before registering.
To register a new LLM provider, do the following:
1
Click + Add Provider. An Add LLM Provider dialog appears.
2
In Step 1: Details, enter the following information about the provider:
  • Name–enter a friendly name for the provider.
  • Provider–select a provider from the list, or select Custom for an OpenAI-compatible provider that is not listed. When possible, the Base URL is filled in for the selected provider.
  • Override Base URL–select this to use your own address (such as a company proxy) instead of the provider default. This option is not available for all providers.
  • Base URL–enter the base endpoint for the provider, for example, https://proxy.example.com/v1. This field is editable only when Override Base URL is selected, or if the provider allows a custom URL. Otherwise, it displays the provider default. See Base URL Requirements.
  • API Key–enter the provider’s API key.
3
In Step 2: Select Models, add the LLM models to use for the provider:
  1. Click + Add Model and select a model from the list, or type the name of a model that is not in the list.
  2. The dialog displays Cost per 1M tokens for the model: Input, Output, Cache read, and Cache write. Selecting a model from the list fills in its costs. You can edit the costs if desired. For a model you enter yourself, Input and Output start at 0 and Cache read and Cache write start blank.
  3. Click Add to add the model.
  4. Repeat to add additional models.
Costs are estimates of the AI provider’s charges, not CData billing. Connect AI uses them for spend estimates, project costing, and budget-limit checks.
4
When you are finished adding models, click Add to close the dialog. The LLM provider appears on the LLM Providers page with a status of Active. The provider’s Edit page shows estimated cost information for each of its models.

Base URL Requirements

Unless you select Override Base URL, Connect AI uses the default base URL for the provider: For other providers, the default is the Base URL that the dialog fills in when you select the provider. If you override the base URL, or use an Azure OpenAI or Custom provider, follow these rules:
  • OpenAI and Anthropic–include the version path, for example, https://proxy.example.com/v1.
  • Cohere–use the host only.
  • Google Gemini–end the URL before /v1beta.
  • For all providers, do not include a query string, fragment, or user name and password. The exception is Azure OpenAI, which uses ?api-version=.
Last modified on November 12, 2026