---
title: Troubleshooting
description: Fix common OpenLM LicOps symptoms in OpenLM Platform, from a Router that never reaches LicOps mode to jobs that are denied or never gated.
product: OpenLM Platform
---

Work from the symptom you can see. Most LicOps problems are one of four things: LicOps is not syncing with OpenLM Platform, the Router is not in LicOps mode, a request names a pool that does not exist, or jobs are not going through LicOps.

## Why does the container not start?

LicOps runs on the OpenLM Router, so it fails at startup for the same reasons a failover Router does: a misnamed connection-string variable, a missing correlation setting, or an encryption key encoded twice. See [The container does not start](https://www.openlm.com/documentation/cloud/deployment-operations/router/troubleshooting/#why-does-the-container-not-start) in the Router troubleshooting. The following table lists the symptoms, causes, and fixes:

| Symptom | Cause | Fix |
| --- | --- | --- |
| The Router starts, but every admit fails, and the logs show Redis errors on the admit. | Redis is older than 7.0. | Upgrade Redis to 7.0 or higher. See [System requirements](https://www.openlm.com/documentation/cloud/deployment-operations/licops/system-requirements#redis-requirements). |
| On Kubernetes, the Router refuses to start and its log names `TrustedCidrs`. | The trusted range is empty. | Set `routerapi.admissionController.proxyProtocol.trustedCidrs` to your pod CIDR. See [Deploy on Kubernetes](https://www.openlm.com/documentation/cloud/deployment-operations/licops/deploy-kubernetes#install-the-router). |
| On Kubernetes, an image pull fails with a reference ending in a colon. | A chart `tag` was not set. | Set every tag explicitly. |
| On Kubernetes, the webhook's pods fail with `ImagePullBackOff`. | An image name or tag is wrong. | Use the charts from the OpenLM bundle, which name `olm-pod-sidecar`, `olm-dispatch-init`, and `olm-iptables-init`. |

## Why is LicOps not syncing with OpenLM Platform?

Until LicOps syncs with OpenLM Platform, it has no configuration, no rules, and no pools, and it denies or refuses every request. The Router UI shows a red notice under **Platform Connection** that names the reason, and `GET /api/v1/status` reports the same reason in `configurationManagement.lastSync.lastFailure`. The causes and fixes are the same as for a failover Router. See [The Router is not syncing with the Platform](https://www.openlm.com/documentation/cloud/deployment-operations/router/troubleshooting/#why-is-the-router-not-syncing-with-openlm-platform).

LicOps also needs fresh seat counts and server health, which it collects separately from its configuration. To check them, read the availability report:

```bash
curl -s http://localhost:8100/api/v1/admission-controller/availability \
  | jq '{lastPlatformSyncTime, lastReconcileTime, coldStart, staleForAdmission}'
```

The following table lists the symptoms, causes, and fixes:

| Symptom | Cause | Fix |
| --- | --- | --- |
| `coldStart` stays `true`, and admit keeps answering `503`. | LicOps has never received seat counts from OpenLM Platform. | Check that `GET /api/v1/platform/servers` lists your license servers. If it answers `503`, follow the Router troubleshooting linked above: the usual cause is a credential whose client type is not `Router`. |
| Admit was working and now denies everything with `StaleSnapshot`, and `lastReconcileTime` has stopped advancing. | The OpenLM Platform sync stopped. If Redis was restarted empty, the Router lost its job schedule, and the sync does not restart by itself. | Restart the Router, then make Redis persistent so it cannot recur. See [System requirements](https://www.openlm.com/documentation/cloud/deployment-operations/licops/system-requirements#redis-requirements). |
| `staleForAdmission` flips between `true` and `false`. | OpenLM Platform answers slowly or intermittently. | Check the network path from the LicOps host to OpenLM Platform, and the log of the Router for failed sync attempts. |

## Why does LicOps Configuration show Port Mappings?

If **LicOps Configuration** shows **Port Mappings** instead of **LicOps** and **Rules**, it is working with a failover Router. It shows the pages for the mode of the Router it is working with: **Port Mappings** for a failover Router, and **LicOps** and **Rules** for a LicOps Router. The following table lists the symptoms, causes, and fixes:

| Symptom | Cause | Fix |
| --- | --- | --- |
| **LicOps Configuration** shows **Port Mappings** for your LicOps deployment. | The Router was started without `ROUTER_CONFIG_BOOTSTRAP_MODE=loadBalancer`, or with `basic`. | Set the variable as in [Deploy OpenLM LicOps](https://www.openlm.com/documentation/cloud/deployment-operations/licops/deployment#create-the-compose-file). The mode is fixed when the Router first starts. Deploy a new Router with its own empty MongoDB database, and stop the old one. |
| The pages switch between **Port Mappings** and **LicOps** from one reload to the next. | Two Routers report to the same OpenLM account, such as a failover Router and a LicOps Router. **LicOps Configuration** works with the one that reported most recently. | Run one Router per OpenLM account. Stop the Router you do not use. See [Change, upgrade, or remove OpenLM Router](https://www.openlm.com/documentation/cloud/deployment-operations/router/manage/#remove-openlm-router). |
| The page says that no Router has reported itself yet. | LicOps has not synced with OpenLM Platform, or the page was opened before it did. The page does not refresh by itself. | Confirm the OpenLM Platform connection works, then reload the page. |
| Each page shows `LicOps License is not available with current openLM License`, and saves are refused. | Your OpenLM license does not include LicOps. | Contact OpenLM to have the license reissued. LicOps keeps running with the configuration it already has. See [System requirements](https://www.openlm.com/documentation/cloud/deployment-operations/licops/system-requirements#openlm-license). |

## Why does LicOps have no pools?

`GET /api/v1/stats` answers `[]`, and every admit is denied. The following table lists the symptoms, causes, and fixes:

| Symptom | Cause | Fix |
| --- | --- | --- |
| The list is empty within a minute of connecting LicOps. | LicOps has not received license-server data from OpenLM Platform. | Follow [Why is LicOps not syncing with OpenLM Platform?](#why-is-licops-not-syncing-with-openlm-platform). |
| A pool, or one server in a pool, is missing. | OpenLM Platform does not monitor that license server: the OpenLM Broker is not installed on it, not approved in **Broker Hub**, or the server is not approved under **License Servers**. | Follow the Broker steps in [System requirements](https://www.openlm.com/documentation/cloud/deployment-operations/licops/system-requirements#openlm-platform-prerequisites), then check that **Live Servers Statistics** shows the server as **Up**. |

## Why are requests denied?

Start from the deny `reason`, then read the availability report as described in [Find out why a request was denied](https://www.openlm.com/documentation/cloud/deployment-operations/licops/operations#find-out-why-a-request-was-denied). The following table lists the symptoms, causes, and fixes:

| Symptom | Cause | Fix |
| --- | --- | --- |
| `NoServerAvailable`, while the pool looks full of free seats. | The vendor or feature in the request does not match the pool, almost always in case or spelling. | Compare with `GET /api/v1/stats`. For Slurm, regenerate the license map rather than editing it by hand. |
| One Slurm license name started being denied with `NoServerAvailable`. The others still work. | The license was added or renamed in OpenLM Platform after the license map was generated. | Regenerate the map and schedule the refresh, as described in [Slurm](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/slurm#keep-the-license-map-current). |
| `NoCapacity`, while `free` shows seats on some servers. | A rule such as **Keep off a server** narrowed the servers, or the request asks for more seats than any one server has free. | Check the rules, and `largestAdmittableRequest` for the pool. |
| `ConstraintBlocked`. | A reservation caps or blocks every server that has free seats. | See [Cap seats on a schedule](https://www.openlm.com/documentation/cloud/deployment-operations/licops/reservations). |
| `PolicyDenied`. | A **Deny the request** rule matched. | Check the rules under **LicOps Configuration**. |
| A rule seems to have no effect. | It was saved less than a minute ago, its **Target server** does not match a server, or a condition compares names with a different case. | Wait a minute and check `GET /api/v1/rules/router-admission`. Pick targets with the picker, and copy names from `GET /api/v1/stats`. |
| A time-window rule or a reservation period starts hours early or late. | LicOps runs in UTC. | Set `TZ` on the `routerapi` container. See [Make the schedule follow your time zone](https://www.openlm.com/documentation/cloud/deployment-operations/licops/reservations#make-the-schedule-follow-your-time-zone). |

## Why are seats never returned?

The pool's `reserved` count in `GET /api/v1/stats` keeps rising, and requests start being denied with `NoCapacity` although few jobs run. The following table lists the causes and fixes:

| Cause | Fix |
| --- | --- |
| Jobs do not release their seats on every ending. | Check that the release runs from a hook or `trap` that always runs. For Slurm and LSF, confirm both adapter roles are installed and the hooks run. |
| LSF's `POST_EXEC` cannot find the session ID. | Set `OL_STATE_DIR` to a directory both `PRE_EXEC` and `POST_EXEC` can reach. See [IBM Spectrum LSF](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/lsf#when-something-goes-wrong). |
| Jobs wait in the queue longer than the admit timeout and lose their seats, then start and check out anyway. | Raise the admit timeout above your typical queue wait. See [Set the session timeouts](https://www.openlm.com/documentation/cloud/deployment-operations/licops/deployment#set-the-session-timeouts). |

Seats that were never released return when their sessions time out. To return one at once, see [Return a seat that was never released](https://www.openlm.com/documentation/cloud/deployment-operations/licops/operations#return-a-seat-that-was-never-released).

## Why are jobs not going through LicOps?

The following table lists the symptoms, causes, and fixes:

| Symptom | Cause | Fix |
| --- | --- | --- |
| Licensed jobs start without any admit in the adapter log. | Jobs were submitted with the real `sbatch`, without `-a openlm` on LSF, or from a submit host without the adapter. | See [Slurm](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/slurm#submit-licensed-jobs) or [IBM Spectrum LSF](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/lsf#submit-licensed-jobs). |
| Kubernetes pods start without the LicOps containers. | The namespace label or the `openlm.io/needs-licensing` annotation is missing. Either one missing makes the webhook skip the pod without any error. | See [Kubernetes workloads](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/kubernetes#license-pods-with-the-admission-webhook). |
| No pod can be created in an opted-in namespace. | The admission webhook is unavailable, and its failure policy is `Fail`. | Restore the webhook, or reinstall it with `webhook.failurePolicy=Ignore`. See [Deploy on Kubernetes](https://www.openlm.com/documentation/cloud/deployment-operations/licops/deploy-kubernetes#install-the-admission-webhook). |
| Jobs are admitted, but checkouts do not go to the server LicOps chose. | Without checkout routing, applications check out from their own license settings. With it, a vendor-specific variable such as `MLM_LICENSE_FILE` can still take precedence. | See [Decide whether LicOps also chooses the server](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/index#decide-whether-licops-also-chooses-the-server). |
| With checkout routing, checkouts reach LicOps but are not matched to their jobs. | The compute nodes reach LicOps through address translation, so their checkouts arrive from a different IP address than the one dispatch recorded. | Let the compute nodes reach the LicOps host directly. |

## Collect diagnostics

To collect diagnostics, run these commands:

```bash
docker compose logs routerapi --tail 200
curl http://localhost:8100/api/v1/status
curl http://localhost:8100/api/v1/stats
curl http://localhost:8100/api/v1/admission-controller/availability
curl http://localhost:8100/api/v1/rules/router-admission
```

On Kubernetes, read the logs with `kubectl -n <namespace> logs deployment/openlm-routerapi --tail 200`, and reach the endpoints through `kubectl port-forward svc/openlm-router-api 8100:8100`. For the scheduler side, include `/var/log/openlm/hpc-adapter.log` from the submit host and from the controller or execution host.