A free, open-source learning guide for Docker Compose users

Your Compose stack, on a shared cluster

On your laptop, docker compose up starts the stack on demand. A shared research cluster has no Docker daemon and gives you no root access (administrator rights). Instead, you request a scheduler allocation (a block of computer time on the cluster), wait your turn, and launch the services with a batch script. This guide shows how to make that translation.

Start with the big idea Already have a Compose file? Translate it

Support researchers or run a cluster? See what this assumes, and when not to use it

Your laptop

docker compose up

coordinator
worker
worker

Docker daemon, root access
Starts whenever you ask
Ports published to your machine

A shared cluster

sbatch slurm/baseline.sbatch

One allocation: one node, a fixed time

coordinator
worker
worker

No Docker daemon, no root access
A scheduler decides when you run
Local-only access inside the node

The same three programs in two environments. On your laptop, Docker starts them on demand. On a cluster, a scheduler grants one allocation and your batch script starts them inside it. Shown here: the single-node baseline, one job on one node.

Tested

Purdue Anvil · Oct 2026 · Test record

What this means for you

Tested: The example workflow ran start to finish and every result was correct. A second run was made to fail on purpose; it stopped at the check that the coordinator can serve (the readiness gate), before any worker started.
You need: Access to one compute node through Slurm. The test used one node on a shared partition.
Apptainer on the compute nodes.
A private folder for the job that you can write to (job-scoped scratch).
Depends on your center: Your account and partition names. You supply them when you submit the job.
How Apptainer is made available: on the default command path, or through a module your center provides.
Which scratch location the job uses. Centers provide it in different ways.
The coordinator listens only on loopback (127.0.0.1); reaching it from beyond the node needs access your center approves.
Whether each worker can run as its own job step (a program the scheduler launches and tracks inside your allocation) on a shared node.
Whether you can rebuild the container image yourself. On the newer Apptainer used for the latest test, the release’s image build script could not run without extra privileges, so the tested image is the earlier build with the same fingerprint (digest).

What hasn’t been tested yet

Pick your path

New to clusters? Start here
Understand the model
The six steps from request to results, in detail, with a short annotated script you can read without a cluster account.
Take the five-minute tour
Already have a Compose file?
Translate it
See what each part of your Compose file becomes on a cluster, then edit the example job to run your own services.
Open the Compose-to-Slurm map
Research software engineers & center staff
Evaluate and adopt
What the pattern assumes, where it was tested, and when a service your center already runs is the better choice.
Read the evaluation guide

The big idea

A scheduler (Slurm, at most centers) decides where and when your stack runs. Inside one allocation, four familiar Compose ideas change meaning.

In Compose a Compose service
On a cluster Job step
A program the scheduler launches and tracks inside your allocation (a job step).
In Compose healthcheck + depends_on
On a cluster Readiness gate
A check that the coordinator can really serve before any worker starts (a readiness gate). It does not establish that results are correct.
In Compose a named volume
On a cluster Job-scoped scratch
Temporary job storage (job-scoped scratch). It may remain until your center clears it; copy out what you need before the job ends.
In Compose a published ports entry
On a cluster Loopback
Local-only access (loopback). It is not an isolation boundary; access beyond the node requires center approval.

Four ideas that change meaning

Same idea, same color and shape, on every page of this site.

Job step: was a Compose service
Readiness gate: was healthcheck + depends_on
Job-scoped scratch: was a named volume
Loopback: was a published ports entry

docker-compose.yml · simplified · Designed, not run

services:
■Job step:   coordinator:
    build: .
    command: python3 /opt/bssw/bin/coordinator.py …
●Loopback:     # ports: ["8080:8080"]  # not used: the coordinator listens on 127.0.0.1 only
◆Readiness gate:     healthcheck:
◆Readiness gate:       test: ["CMD", "python3", "-c", "…check /health…"]
◆Readiness gate:       interval: 1s
◆Readiness gate:       retries: 30  # about READINESS_TIMEOUT=30
▲Job-scoped scratch:     volumes: [work:/work]
■Job step:   worker:
    build: .
■Job step:     deploy: {replicas: 2}
●Loopback:     network_mode: "service:coordinator"
◆Readiness gate:     depends_on: {coordinator: {condition: service_healthy}}
▲Job-scoped scratch:     volumes: [work:/work]
  verify:  # one-shot; yours to replace
    build: .
    # waits for the workers to finish; not a gate
    depends_on: {worker: {condition: service_completed_successfully}}
▲Job-scoped scratch:     volumes: [work:/work]
volumes:
▲Job-scoped scratch:   work: {}  # kept until down -v

slurm/baseline.sbatch · excerpt · line numbers from v0.1.0 · Test record

3#SBATCH --nodes=1
6#SBATCH --time=00:10:00
■Job step: 78step_exec() {
■Job step: 79  "$SRUN_CMD" --exclusive --nodes=1 --ntasks=1 --cpus-per-task=1 "$@"
80}
120trap cleanup EXIT
▲Job-scoped scratch: 173RUNTIME_DIR=${SCRATCH_ROOT%/}/bssw-${SLURM_JOB_ID}
▲Job-scoped scratch: 176mkdir -m 700 -- "$RUNTIME_DIR" || \
■▲Job step, Job-scoped scratch: 200step_exec "$APPTAINER_CMD" exec --cleanenv --bind "${RUNTIME_DIR}:/work" "$IMAGE_PATH" \
■Job step: 201  python3 "$APP_ROOT/coordinator.py" --input /work/input.json \
●Loopback: 202  --results /work/results.json --endpoint-file /work/endpoint.json \
■Job step: 203  > "$RUNTIME_DIR/logs/coordinator.log" 2>&1 &
◆Readiness gate: 219container_exec python3 "$APP_ROOT/readiness.py" --endpoint "$ENDPOINT" \
◆Readiness gate: 220  --task-count "$TASK_COUNT" --timeout "$READINESS_TIMEOUT" \
■Job step: 228for ((index=1; index<=10#$WORKER_COUNT; index++)); do
■Job step: 230  step_exec "$APPTAINER_CMD" exec --cleanenv --bind "${RUNTIME_DIR}:/work" "$IMAGE_PATH" \
■●Job step, Loopback: 231    python3 "$APP_ROOT/worker.py" --endpoint "$ENDPOINT" --worker-id "$worker_id" \
■Job step: 239  if ! wait "$worker_pid"; then
245container_exec python3 "$APP_ROOT/verify.py" --input /work/input.json \
246  --results /work/results.json --expected /work/expected.json --output /work/result.json \

bin/coordinator.py · excerpt · line numbers from v0.1.0

●Loopback: 115server = CoordinatorServer(("127.0.0.1", 0), state)

Here from the SC26 poster or the US-RSE’26 talk? Talk and poster links

Your Compose stack, on a shared cluster

Pick your path

Understand the model

Translate it

Evaluate and adopt

The big idea

Four ideas that change meaning