Scheduler as Orchestrator

Your Compose stack, before

On a laptop, one Compose file describes the whole stack. This simplified sketch has a coordinator that must be answering before the workers start, two workers, a one-shot check of the results, and a named volume for the files they share. Commands for the workers and the check are left out. It is a design sketch: Designed, not run.

# docker-compose.yml (simplified)
services:
  coordinator:
    build: .
    command: python3 /opt/bssw/bin/coordinator.py --input /in/tasks.json --results /work/results.json --endpoint-file /work/endpoint.json
    healthcheck:
      test: ["CMD", "python3", "-c", "import json, urllib.request as u; e = json.load(open('/work/endpoint.json'))['endpoint']; assert json.load(u.urlopen(e + '/health', timeout=1)) == {'status': 'ready', 'schemaVersion': 1, 'taskCount': 4}"]
      retries: 30
    volumes: [work:/work, ./inputs:/in:ro]
  worker:
    build: .
    deploy: { replicas: 2 }
    depends_on: { coordinator: { condition: service_healthy } }
    volumes: [work:/work]
  verify:
    build: .
    depends_on: { worker: { condition: service_completed_successfully } }
    volumes: [work:/work, ./inputs:/in:ro, ./expected:/expected:ro]
volumes:
  work: {}

The same stack on a cluster

A shared cluster runs no Docker daemon and gives you no administrator rights, so docker compose up does not work there. Instead you ask a batch scheduler for an allocation: a share of one machine for a fixed time. One launch script then does inside it what Compose did on your laptop. The script moves through six steps: get the allocation, start the coordinator, pass the readiness gate, run the workers, verify the results, and clean up.

Four ideas change meaning along the way:

• A Compose service becomes a program the scheduler launches and tracks inside your allocation (job step).
• depends_on plus a health check becomes a check that the coordinator can really serve before any worker starts (readiness gate).
• A named volume becomes temporary job storage (job-scoped scratch).
• A published port becomes local-only access (loopback). Reaching the service from outside the node needs a path your center approves, such as an SSH tunnel.

Two checks guard the run, and they answer different questions. The readiness gate asks whether the coordinator can serve. It belongs to the pattern, so you keep it and change only what “ready” means for your service. Verification asks whether the application’s results are correct. Only your application can say, so you replace it with your own check.

Map each responsibility

What varies by center

This mapping does not make a workflow portable unchanged. Check each of these with your center before you adapt anything:

• Scheduler syntax. Account, partition, and time-limit options differ. Read your center’s job-script guide.
• Network policy. Whether job steps on one node may reach each other over loopback, and what, if anything, can reach a node from outside.
• Storage. Which path is private and job-scoped, how much quota it has, and when it is cleared.
• Runtime modules. How your center provides Apptainer or Charliecloud, and which version.
• Accelerators. The flags and runtime options that expose GPUs.

Multi-node discovery, persistent databases, privileged ports, and production availability are outside the single-node pattern this guide teaches. For what has been run and what has not, see what hasn’t been tested yet.

Adapt it

Pick one Compose stack of your own and decide who owns each of five things: the allocation, the startup order, the runtime state, access to the service, and any accelerators. Each owner is one of scheduler policy, runtime behavior, your launch script, or your center’s local configuration. Write the answers down; the completion exercise on this page turns them into a reviewable decision record.

Next, run the baseline example, or read how it is built in Baseline Pattern: Single-Node Service + Workers.

Key points

• A scheduler allocation and one launch script, not Compose, start, order, and end your services.
• Four ideas change meaning: a service becomes a job step, a health check becomes the readiness gate, a volume becomes job-scoped scratch, and a published port becomes loopback.
• The readiness gate checks that the coordinator can serve. Verification checks that the results are correct. They are different checks.
• Scheduler syntax, network policy, storage, and runtime modules vary by center.