Configure OpenLM AI Proxy

After you deploy AI Proxy, connect it to OpenLM Platform, set up the providers you want to meter, and route clients through it. You do most of this configuration in the AI Proxy UI.

The primary, supported workflow is API interception: clients send LLM API requests to AI Proxy, and AI Proxy meters them. Configure that first.

Before you begin

Have the following ready:

• The base address of your OpenLM Platform, for example https://example.com.
• If your OpenLM Platform is secured, a Client ID and Client Secret for AI Proxy. The OpenLM Platform administrator who manages Identity creates them. Ask which client type to use if you are not sure.
• The API key of each provider that you want AI Proxy to hold, if you use Managed mode.

Connect to OpenLM Platform

AI Proxy sends usage data to OpenLM Platform through a receiver. You add one receiver for your OpenLM Platform.

1. Open the AI Proxy UI at http://<aiproxy-host>:<ui-port>/aiproxy/.
2. Open Receivers Configuration and add a receiver.
3. In Location, enter the base address of your OpenLM Platform, for example https://example.com. Enter the address only. AI Proxy adds the rest of the path.
4. If your OpenLM Platform is secured, enter the Client ID and Client Secret. Leave them empty only for an OpenLM Platform that needs no sign-in.
5. Optionally, change Request Timeout. The default is 30 seconds, and the allowed range is 1 to 300.
6. Select Check Connection, then save.

AI Proxy stores the Client Secret encrypted.

When the connection works, the Status of the receiver shows Ok. AI Proxy then delivers usage events and AI asset information to the platform, where they feed AI FinOps reports.

Note

Some older documents call this setting the Open Platform Connection or Broker Hub Receivers. It is the same connection to OpenLM Platform.

Set up a provider

For each LLM provider you route through AI Proxy, select an authentication mode:

• Managed: AI Proxy holds a shared provider API key and applies it to outgoing requests. Clients authenticate to AI Proxy.
• Passthrough: Each client sends its own provider API key, and AI Proxy forwards it unchanged.

To add a provider:

1. In the AI Proxy UI, open LLM Configuration, then select Add Provider.

2. Select the Provider Type.

3. Select the Operating Mode: Managed or Passthrough. The default is Passthrough.

4. Turn the provider on with the Enable switch. Turn OTel on if you collect usage from the client through OpenTelemetry.

5. Enter the provider fields and select Save:

   Provider	Fields
   Anthropic	API Key (required in Managed mode), and an optional Admin API Key.
   OpenAI	API Key.
   Vertex AI	Location (the default is us-central1) and Service Account JSON.

Changes you save in the UI apply without a restart.

The following table lists support by platform:

Platform	Managed	Passthrough	Notes
Anthropic (Claude Code)	Yes	Yes	Includes the Claude Code CLI proxy.
OpenAI	Yes	Yes	Prototype.
Google Vertex AI (Gemini)	Yes	Yes	None

Cursor and GitHub Copilot can't be metered through AI Proxy. To track them, see Cursor and GitHub Copilot.

Issue a key to Managed-mode clients

In Managed mode, clients don't use the provider key. They use a key that AI Proxy issues:

1. In the AI Proxy UI, open Api Keys Configuration, then select Add API Key.
2. Enter a Description, and select the Allowed Providers for the key.
3. Save, and give the key to the client. The key starts with lm-sk-.

The client sends the key as Authorization: Bearer <key> or as an x-api-key header.

Route the Claude Code CLI through AI Proxy

To point the Claude Code CLI at AI Proxy, set the ANTHROPIC_BASE_URL environment variable to the proxy's Anthropic endpoint:

1. Install the Claude Code CLI.

2. Sign in to Claude Code.

3. Set ANTHROPIC_BASE_URL to the AI Proxy Anthropic endpoint:

   Windows

   setx ANTHROPIC_BASE_URL "http://<aiproxy-host>:<api-port>/api/v1/llm/anthropic"

   macOS and Linux

   export ANTHROPIC_BASE_URL="http://<aiproxy-host>:<api-port>/api/v1/llm/anthropic"

   To keep the setting, add the same line to your shell profile. On macOS, the default shell is zsh, so use ~/.zshrc. On Linux with bash, use ~/.bashrc.

   echo 'export ANTHROPIC_BASE_URL="http://<aiproxy-host>:<api-port>/api/v1/llm/anthropic"' >> ~/.zshrc

   Replace <aiproxy-host> with your AI Proxy host and <api-port> with the API port you mapped during deployment.

4. Restart the terminal so the environment variable takes effect.

5. Run a Claude Code session and ask any question.

6. Confirm that the data reaches OpenLM Platform. See Confirm that data reaches AI FinOps.

To send traffic directly to Anthropic again, reset the variable to https://api.anthropic.com and restart the terminal.

Coverage

Some Claude Code activity doesn't always report token usage. To capture all traffic, route every client through AI Proxy. Network administrators can enforce this by blocking direct access to LLM providers at the firewall, so all requests flow through the proxy.

Export usage with OpenTelemetry

AI Proxy uses OpenTelemetry (OTel) to monitor usage and can export events to an external collector. It supports three patterns:

• Claude Code direct reporting: Claude Code reports usage events directly to the AI Proxy OTel endpoint through its OpenTelemetry environment variables, with no intercepting proxy in the request path.
• LiteLLM: A LiteLLM gateway sits between your clients and the LLM provider. LiteLLM aggregates usage and exports it to AI Proxy over OpenTelemetry.
• OTel export: AI Proxy forwards OpenTelemetry events to an external collector, such as Grafana or Datadog, alongside sending usage data to OpenLM Platform.

For Claude Code direct reporting, AI Proxy generates the setup scripts for you:

1. In the AI Proxy UI, open LLM Configuration and select the provider. Make sure OTel is on.
2. Select Download for Windows (.bat) or Download for Linux / macOS (.sh).
3. Run the script on each developer machine, then restart the terminal.

Route OpenAI and Vertex AI clients

Clients of other providers send their requests to AI Proxy instead of the provider, as Claude Code does. In Managed mode, the client uses an AI Proxy key. In Passthrough mode, it sends its own provider key. Set the client's base address to the AI Proxy API address that you mapped during deployment.

How AI Proxy identifies users

AI Proxy attributes each call to a consumer so that reports can break usage down by user:

• When a request carries no username, AI Proxy uses the authentication key as the consumer ID.
• You can supply usernames through the API or through the Identity integration.
• For Claude Code, AI Proxy can resolve the user's email for a usage record.

Confirm that data reaches AI FinOps

After you route a client, check each stage from the stack to the reports:

1. In the AI Proxy UI, open Receivers Configuration. The receiver Status is Ok.
2. In OpenLM Platform, open Broker Hub and select the AI Proxy tab. Your agent is in the list, with its Status and Last Update.
3. Send a request through AI Proxy from a client, for example a Claude Code question.
4. Wait at least 5 minutes. AI Proxy sends usage to OpenLM Platform every 5 minutes.
5. Open the AI FinOps reports, starting with the Details report. Your usage appears in the reports.

If a stage fails, see Troubleshoot AI Proxy.

User monitoring

Beyond API interception, AI Proxy includes user-level monitoring to support license optimization in OpenLM Platform, such as restricting access to expensive models. This feature is experimental. Use API interception as the primary, supported workflow.

Next step

Review your results in the AI FinOps reports. To upgrade or remove AI Proxy later, see Deploy AI Proxy.