--- title: IBM Spectrum LSF description: Install the OpenLM LSF adapters so that licensed LSF jobs get a LicOps seat from OpenLM Platform before they are queued. product: OpenLM Platform --- To have LSF ask LicOps for a seat before a licensed job is queued, install the OpenLM LSF adapters. Users add an `openlm[...]` block to the job's resource request, and LSF does the rest: a job that LicOps denies is never queued. :::warning[Test on your cluster first] LSF support is in early access. Validate the adapters on a non-production queue, with the checks in [Verify](#verify), before you rely on them. ::: ## How the adapters fit into LSF The following table lists the three requests, the adapter that makes each, where it runs, and when: | Request | Adapter | Runs on | When | | --- | --- | --- | --- | | Admit | `esub.openlm`, an LSF `esub` | Each submit host | At submission, before the job reaches `mbatchd` | | Dispatch | `ol-pre-exec`, as the queue's `PRE_EXEC` | The job's first execution host | When the job starts | | Release | `ol-post-exec`, as the queue's `POST_EXEC` | The job's first execution host | When the job ends, however it ends | `esub.openlm` stores the session ID in the job's environment as `OL_SESSION_ID`, and the queue hooks read it back. Some LSF configurations strip that variable before `POST_EXEC`, so `ol-pre-exec` also writes the session ID to a small file on the execution host, and `ol-post-exec` falls back to it. Unlike Slurm, LSF needs no license map: the vendor and feature travel in the job's own resource request. ## Before you begin Confirm these prerequisites: - LicOps is running and sees your pools, as described in [Configure OpenLM LicOps](https://www.openlm.com/documentation/cloud/deployment-operations/licops/configuration). - The LSF management host, the execution hosts, and every submit host can reach LicOps on port `8100`. - `curl` and `jq` are installed on the management host, the execution hosts, and every submit host. - You can edit `lsb.queues` and run `badmin reconfig` on the management host, and you can write to `$LSF_SERVERDIR` on each submit host. - You have the OpenLM HPC adapter package for LSF and the integration kit, from OpenLM. ## Install the adapters To install the adapters: 1. Unpack the adapter package on the LSF management host. 2. Check LicOps from the management host with `verify-router.sh` from the integration kit: ```bash ./lsf/verify-router.sh --base-url http://:8100/api/v1/admission-controller ``` 3. On the management host, wire the dispatch and release hooks into a queue: ```bash sudo ./lsf/install-lsf-adapters.sh --role master --queue normal \ --base-url http://:8100/api/v1/admission-controller ``` The installer copies the adapters to `/opt/openlm/hpc`, creates `/etc/openlm/hpc-adapter.conf`, backs up `lsb.queues`, adds `PRE_EXEC` and `POST_EXEC` to the queue you name, and runs `badmin reconfig`. It refuses to overwrite a `PRE_EXEC` or `POST_EXEC` that already points at another script. It wires one queue per run: run it again with another `--queue` for every queue that runs licensed jobs. 4. On every submit host, install the `esub`: ```bash sudo ./lsf/install-lsf-adapters.sh --role submit \ --base-url http://:8100/api/v1/admission-controller ``` To install the submit hosts from the management host in one step, add `--submit-nodes host1,host2` to the `--role master` command. That needs passwordless root SSH to those hosts. 5. On every host, open `/etc/openlm/hpc-adapter.conf` and set these values. The installer creates the file from the template the first time, with `OL_BASE_URL` filled in, and never overwrites it afterwards: | Setting | Value | | --- | --- | | `OL_FAIL_MODE_ADMIT`, `OL_FAIL_MODE_DISPATCH` | `open` or `closed`, as described in [Decide what happens when LicOps is unreachable](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/index#decide-what-happens-when-licops-is-unreachable). | | `OL_CLUSTER_NAME` | Optional. The only way LicOps learns the name of an LSF cluster. | | `OL_STATE_DIR` | Optional. A directory that `PRE_EXEC` and `POST_EXEC` both reach on the execution host. See [When something goes wrong](#when-something-goes-wrong). | Run either command with `--dry-run` first to see every change it would make. Both are safe to run again. :::warning[Install both roles] Without the `esub` on the submit hosts, jobs are queued without asking LicOps. Without the queue hooks, jobs are admitted but never dispatched or released, so their seats wait for the session timeout. ::: ## Submit licensed jobs Users select the `esub` with `-a openlm` and name the pool in an `openlm[...]` block: ```bash bsub -a openlm -R "openlm[vendor=MLM:feature=MATLAB:count=1]" ./my-simulation.sh ``` The `vendor` and `feature` must match a pool exactly, including case. Copy them from `GET /api/v1/stats`. Write a space in a feature name as `%20`, such as `feature=Acrobat%20Pro`. `count` is the number of seats and defaults to `1`. The same block works as a `#BSUB -R` line at the top of the job script. To make admission non-optional, configure `openlm` as a mandatory `esub` in `lsf.conf` instead of relying on users to pass `-a openlm`. A job without an `openlm[...]` block passes straight through. Each job requests one vendor and feature. A job that needs two different products cannot ask for both in one submission. ## Route checkouts through LicOps By default, the adapters gate when a job starts, and the application checks out from the license servers in its own settings. To make the application check out through LicOps, so that its checkout goes to the server LicOps chose, add `--lb-endpoint 30001@` to both install commands. If `/etc/openlm/hpc-adapter.conf` already exists, the installer leaves it alone, so add `OL_LB_ENDPOINT="30001@"` to it yourself on each submit host. The `esub` then puts `30001@` in front of the job's `LM_LICENSE_FILE`, keeping the existing value behind it as a fallback. See [Decide whether LicOps also chooses the server](https://www.openlm.com/documentation/cloud/deployment-operations/licops/integrations/index#decide-whether-licops-also-chooses-the-server). Many applications read their own variable before `LM_LICENSE_FILE`. MATLAB reads `MLM_LICENSE_FILE`, for example. Where such a variable is set, it wins, and the application checks out without going through LicOps. ## Verify To test the adapters, run these commands from a submit host: ```bash bsub -a openlm -R "openlm[vendor=MLM:feature=MATLAB:count=1]" sleep 30 tail -f /var/log/openlm/hpc-adapter.log ``` Check that the log records the admit when you submit, the dispatch when the job starts, and the release when it ends. While the job is admitted or running, the pool's `reserved` count in `GET /api/v1/stats` is one higher. Also check that the release happens when `POST_EXEC` runs, not only after the session times out: the `reserved` count falls as soon as the job ends. ## When something goes wrong The following table lists symptoms, their causes, and fixes: | Symptom | Cause | Fix | | --- | --- | --- | | Nothing is admitted. | No `esub.openlm` in `$LSF_SERVERDIR`, or the job was submitted without `-a openlm`. | Install the submit role, or make the `esub` mandatory. | | `ol-common.sh not found` | The adapter folders were moved apart, or the `esub` in `$LSF_SERVERDIR` was replaced by a symbolic link. | Keep the installed layout under `/opt/openlm/hpc`, and keep the one-line `esub` wrapper the installer wrote. | | The hooks never run. | `badmin reconfig` was not run, or the job ran in a queue without the hooks. | Check `bqueues -l ` for `PRE_EXEC` and `POST_EXEC`. | | Seats return only after the session timeout. | `POST_EXEC` lost `OL_SESSION_ID`, and its fallback file was not readable. | Set `OL_STATE_DIR` to a directory that `PRE_EXEC` and `POST_EXEC` both reach on the execution host. | | Every licensed job is rejected at submission. | LicOps is unreachable and `OL_FAIL_MODE_ADMIT=closed`, or `OL_BASE_URL` is wrong. | Run `verify-router.sh` from that submit host. | | Jobs are requeued after they start. | LicOps is unreachable at dispatch and `OL_FAIL_MODE_DISPATCH=closed`. | Restore LicOps, or switch dispatch to `open`. | ## Remove the adapters To preview the removal and then remove the adapters, run these commands: ```bash sudo ./lsf/uninstall-lsf-adapters.sh --dry-run sudo ./lsf/uninstall-lsf-adapters.sh ``` The uninstaller reads the record the installer kept, `/etc/openlm/hpc-adapter.install`, and removes only what that installation added.