Configure an LLM Provider

The Identity section collects the provider’s name. Enter a Display name: a human-readable label shown in dashboards and model selectors, up to 253 characters.

Agentic Data Plane derives the provider’s resource ID from the display name in the background. The create form doesn’t show it. The resource ID is the machine identifier that rpk ai commands use. It starts with a lowercase letter, contains only lowercase letters, numbers, and hyphens, ends with a letter or number (^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$), is at most 63 characters, and is immutable after creation. It appears in the proxy URL (/llm/v1/providers/<resource-id>/…​).

In the Provider type section, select the type that matches your upstream. Each tile carries the type’s name and icon and a one-line summary of what it reaches. Hover over a tile, or select it, to show a Docs link to the upstream’s own API documentation.

Selecting a type reveals the type-specific configuration fields.

Each API key reference and credential field points at a secret-store entry, not the secret value itself. On the create form, the Credential section lists the keys already in the Agentic Data Plane secret store. Pick one, or select Bring a new API key reference to store a new key inline. Enter a Key name and the API key value, then click Save key. Agentic Data Plane encrypts the key and stores it in the secret store under an identifier derived from the key name, which appears under the field. You can’t view the value again after you save it.

The picker lists only the secrets AI Gateway can read, which are the ones held in the Agentic Data Plane secret store. A secret that lives in Redpanda Cloud for another product, such as a Redpanda Connect pipeline, isn’t listed, and a new Agentic Data Plane secret can’t reuse its name.

To change the key later, open the provider’s Settings tab. Its Credential section uses the same picker as the create form. Select another stored key, or bring a new one.

Models you select on this form become the catalog the provider exposes.

For OpenAI, Anthropic, Google Gemini API, and Amazon Bedrock, the form shows a picker backed by the provider’s catalog, with the full catalog selected by default. Keep the default, or deselect models to narrow what the provider exposes. Each model in the picker shows its input and output price per million tokens. Select from the list, or type a model identifier the catalog doesn’t show. For OpenAI-compatible, the form takes a freeform list. Type the exact identifiers your upstream serves.

For Gemini Enterprise Agent Platform, the picker lists only the models served at the provider’s location, so choose the location first. The form preselects the Gemini models there, and Claude models start unselected. Select only the Claude models you enabled in your GCP project. If you change the location, the selection resets to that location’s Gemini models.

Redpanda maintains the catalog of available models in the picker. When an upstream provider publishes a new model, it usually appears in the picker within a day or two. Admins don’t have to wait for a Redpanda release. New models aren’t enabled automatically. An admin still selects the model in the catalog to make it callable through this provider. A model the upstream provider has retired drops out of the picker list, though you can still type its identifier. A provider that already has it selected keeps serving it, its recorded spend keeps pricing at the catalog rates, and agents using it keep working.

For Bedrock, the picker lists inference-profile IDs for models that Bedrock serves only through a profile, such as Claude, and the bare model ID for models invoked in-region, such as Mistral Large 3. See Inference profiles and IAM.

The Bedrock picker groups models under the vendor that publishes them, such as Anthropic, Amazon, Google, Meta, Mistral, and OpenAI.

Beyond pricing, the catalog carries each model’s capabilities and context-window limits: max_input_tokens (the largest context the model accepts) and max_output_tokens (the most it can generate in a single response). Both are read-only catalog metadata. A limit the catalog doesn’t declare stays unset rather than reported as zero. To see a model’s limits, run rpk ai model get <model>. The context-window indicator on an agent’s Playground tab uses them to show how full the agent’s context window is.

The Tags section, last on the create form, takes key/value pairs that organize and filter providers and that access policies can read in their conditions. Enter a tag key and a tag value, then press Enter or click the add (+) button.

To change tags after you create the provider, use the Tags section on its Settings tab. A tagged provider shows its tags on the connection card of its detail page and in the Tags column of the LLM providers list.

For the limits and rules that apply to tags on every resource, see Tag Resources.

After you create the provider, its detail page organizes everything about it into tabs: Overview, Models, Connect, Playground (when enabled for your organization), and Settings. The page header carries the provider’s logo, its display name with an Enabled or Disabled badge beside it, its resource identifier, and its description if it has one.

The Transcripts section controls whether AI Gateway records the message bodies this provider proxies. It appears on the create form and on the provider’s Settings tab, and it has two independent toggles:

When you create a provider, recording starts on, so a new provider records message bodies from its first request. This holds on every create path. The Create provider form starts with both toggles on, and a create with rpk ai llm-provider create that doesn’t mention transcripts gets the same default. The bodies land on the AI Gateway’s own trace data for the calls it proxies. They don’t feed agent transcripts, which a managed agent records under its own transcript recording mode. Turn either toggle off, on the create form or later on the Settings tab, for workloads where the message body must not be retained, such as regulated PII or customer secrets. To keep that content out of agent transcripts too, set each agent’s recording mode to Metadata only or Off.

Two cases don’t get that default:

These are per-provider settings, not per-request. Applications can’t opt in or out at call time. To split sensitive from non-sensitive traffic, create one provider with recording on and another with it off, and route each application to whichever proxy URL matches its data class.

Recording settings do not affect cost and usage telemetry. Token counts, latency, and provider/model attribution are always recorded, so the Cost and usage page reports spend for traffic on the provider regardless of these toggles. Only the message bodies are withheld when the toggles are off.

Before you click Create provider, check the Transcripts section. Both toggles arrive on, so the provider starts recording prompts and responses with its first request, and what it captures is not retroactively redacted. See Configure message recording.

What a successful result proves depends on the provider type, and the panel says so before you run it:

If the check fails, see Troubleshooting.

The LLM providers list page is the at-a-glance home for every provider in your Agentic Data Plane environment. Open it from the sidebar’s LLM providers entry.

Click the 24h requests or 30d spend column header to order the list by that figure. A metric sort replaces any other column’s sort. Use the search box to find a provider by name, and the Filter button to narrow the list by provider type, model, or status. The Add provider button opens the create flow described in Open the Create LLM provider page. Each row’s actions menu can enable or disable the provider, copy its proxy URL, or delete it, and offers to copy the upstream base URL for an OpenAI or OpenAI-compatible provider. Disabling asks you to confirm, and deleting asks you to type the provider’s display name. The list paginates, with a rows-per-page selector in the footer.

The Cost and usage page reports spend, requests, and tokens for a time window against the period before it, breaks spend down by agent, model, provider, user, or tag, charts any one of them against the gateway total, and exports CSV. See Analyze Cost and Usage.

AI Gateway does not provide these capabilities. For current status, see the Agentic Data Plane release notes.