OpenLM AI Proxy system requirements
AI Proxy runs as a Docker Compose stack on a single Linux host inside your network. Confirm the following requirements before you deploy.
Host requirements
The host needs the following:
- A 64-bit Linux host. AI Proxy is tested on Ubuntu 24.04 LTS.
- Docker Engine with the Docker Compose v2 plugin (
docker compose). - Outbound internet access to the large language model (LLM) providers you route through AI Proxy and to your OpenLM Platform endpoint.
Hardware requirements
The stack runs the API, the UI, a UI proxy, MongoDB, and Redis. Use the values in the following table as a starting point, and scale based on request volume:
| Resource | Minimum | Recommended |
|---|---|---|
| CPU | 2 virtual CPUs (vCPUs) | 4 vCPUs |
| Memory | 4 GB | 8 GB |
| Disk | 20 GB | 50 GB or more |
MongoDB storage grows with the number of requests you route. AI Proxy keeps usage records for a short, fixed period and doesn't have a retention setting, so size the disk for your request volume.
Network requirements
Provide the following connectivity:
- Outbound to providers: HTTPS (port 443) to each LLM provider you route through AI Proxy, for example Anthropic, OpenAI, or Google Vertex AI.
- Outbound to OpenLM Platform: HTTPS (port 443) to your OpenLM Platform endpoint, for the Open Platform Connection that delivers usage data.
- Inbound from clients: Clients reach the AI Proxy API on port 8088 and the UI on port 8092 by default. You can change the host ports in the Docker Compose file. Restrict these ports to the networks that need them.
Open no other inbound ports. MongoDB (27017) and Redis (6379) are for the containers only. Remove their ports entries from the Compose file, as described in Set your own passwords and key.
TLS
Neither the API image nor the UI proxy terminates TLS. They serve plain HTTP on ports 8088 and 8092. Provider API keys can pass through AI Proxy, so place a reverse proxy that terminates TLS in front of both ports, and give clients the HTTPS address. The examples in this guide use http:// for a first test only.
Supported AI platforms
For each platform's status and authentication modes, see OpenLM AI Proxy.
Continue to Deploy AI Proxy to run the stack.
The documentation team might draft this page using a large language model. A technical writer reviews every page and verifies it against the product before publishing it. A human makes the final decision.