The Docker-Compose File Explained

A short overview over the docker compose file.

The Services

The file contains multiple services, 7 are required while the rest are optional. The ui and backend service are the QHAna UI and the QHAna backend respectively. The UI is a web frontend written in Angular. The backend stores all experiment data and is written in ballerina. The plugins use 5 services in total in the default configuration. The qhana-plugin-runner service serves the REST APIs of the plugins, while the worker perferms the background computation tasks. Redis is used as the task queue service and data is stored in the postgres database. Data is also stored in the minio instance per default (requires the minio plugin to be loaded). The muse-db service is optional and provides the muse data in a live mysql database.

Warning

The services qhana-plugin-runner and worker must have the exact same configuration apart from the concurrency settings. They must have the same plugins loaded, use the same task broker for the task queue and use the same database.

Warning

If minio is NOT configured as the default store, then the qhana-plugin-runner and worker services require a shared file system! This can be achieved by mounting the same docker volume in both containers under the path /app/instance.

services:
  qhana-plugin-runner:
    platform: linux/amd64
    image: ghcr.io/ust-quantil/qhana-plugin-runner:main
    depends_on:
      - redis
      - postgres
  redis:
    image: "redis:latest"
  postgres:
    image: "postgres:latest"
  minio:
    image: quay.io/minio/minio 
  muse-db:
    image: "muse-db"
    profiles:
      - with_db
  worker:
    platform: linux/amd64
    image: ghcr.io/ust-quantil/qhana-plugin-runner:main
    depends_on:
      - qhana-plugin-runner
  backend:
    image: ghcr.io/ust-quantil/qhana-backend:main
  ui:
    image: ghcr.io/ust-quantil/qhana-ui:main

Open Ports

services:
  qhana-plugin-runner:
    ports:
      - "5005:8080"
  redis:
    ports:
      - "6379:6379"
  postgres:
    ports:
      - "5432:5432"
  minio:
    ports:
      - "9000:9000"
      - "9001:9001"
  muse-db:
  worker:
  backend:
    ports:
      - 9091:9090
  ui:
    ports:
      - 8080:8080

The docker-compose file maps 4 ports to the outside. The user interface is available at port 8080. This port can be changed to any other open port. The QHAna UI expects the backend to answer at port 9090. If the backend port is changed in the docker-compose file, then the port must be changed in the settings page of the QHAna UI for each client using the UI. The plugin runner api service is exposed under port 5005 by default. The addresses of all known plugin runners can be edited in the settings page of the QHAna UI. This has to be done only once as the list is stored in the QHAna backend. The exposed redis port is for debugging purposes. Minio ports are exposed and can be used to inspect data in the minio store. Other plugin runners can use the redis broker for their task queue only if they also use the same database and data store. All other services are only available in the internal network created by docker-compose (using the service name as hostname). They are used internally by the plugin runner.

Tip

All URLs that get processed by any part of the QHAna application (e.g., UI, Backend, Plugin Runner (+ Worker), Plugin Registry (+ Worker), minio) need to resolve correctly from inside and outside of the docker compose network. To achieve this for a development setup without the use of static IPs, a reverse proxy is included in all relevant containers that bounces back calls to the container localhost to the host of the containers. The reverse proxy itself relys on host.docker.internal to resolve the container host!

For debugging and convenience, other connections (e.g., connections to redis/postgres) also use host.docker.internal and the target containers expose their ports to the local host. This allows for easy debugging of the database from outside and reusing the redis broker for other celery workers.

Network Connectivity

Network Connectivity

Arrows symbolize connections while lines without arrows are used for port mappings. The blue arrows directly use the host.docker.internal dns address to bounce back to the localhost. However, the proxied localhost ports also rely on host.docker.internal.

Settings

services:
  qhana-plugin-runner:
    environment:
      WAIT_HOSTS: host.docker.internal:6379, host.docker.internal:5432
      WAIT_SLEEP_INTERVAL: 5
      WAIT_TIMEOUT: 600
      CONCURRENCY: 2
      BROKER_URL: redis://host.docker.internal:6379
      RESULT_BACKEND: redis://host.docker.internal:6379
      CELERY_QUEUE: "qhana_queue1"
      SQLALCHEMY_DATABASE_URI: "postgresql+psycopg2://user:password@host.docker.internal:5432/default_db"
      DEFAULT_FILE_STORE: "minio"
      MINIO_CLIENT: '{"endpoint": "localhost:9000", "access_key": "QHANA", "secret_key": "QHANAQHANA", "secure": false}'
      LOCALHOST_PROXY_PORTS: &localhost-proxy-ports ":5005 :5006 :5007 :9000 :9001 :9091 ${EXTRA_PROXY_PORTS}"
      GIT_PLUGINS: "git+https://github.com/UST-QuAntiL/qhana-plugin-runner.git@main#subdirectory=/plugins"
      PLUGIN_FOLDERS: &plugin-folders ./git-plugins:./git-plugins/classical_ml/data_preparation
      NISQ_ANALYZER_UI_URL: http://localhost:5009
  redis:
  postgres:
    environment:
      POSTGRES_PASSWORD: password
      POSTGRES_USER: user
      POSTGRES_DB: default_db
  minio:
    environment: 
      MINIO_ROOT_USER: QHANA
      MINIO_ROOT_PASSWORD: QHANAQHANA
  worker:
    environment:
      WAIT_HOSTS: host.docker.internal:6379, host.docker.internal:5432
      WAIT_SLEEP_INTERVAL: 5
      WAIT_TIMEOUT: 600
      CONTAINER_MODE: worker
      CONCURRENCY: 2
      BROKER_URL: redis://host.docker.internal:6379
      RESULT_BACKEND: redis://host.docker.internal:6379
      CELERY_QUEUE: "qhana_queue1"
      SQLALCHEMY_DATABASE_URI: "postgresql+psycopg2://user:password@host.docker.internal:5432/default_db"
      DEFAULT_FILE_STORE: "minio"
      MINIO_CLIENT: '{"endpoint": "localhost:9000", "access_key": "QHANA", "secret_key": "QHANAQHANA", "secure": false}'
      LOCALHOST_PROXY_PORTS: *localhost-proxy-ports
      GIT_PLUGINS: *git-plugins
      PLUGIN_FOLDERS: *plugin-folders
  backend:
    environment:
      LOCALHOST_PROXY_PORTS: *localhost-proxy-ports
      QHANA_HOST: http://localhost:9091

The QHAna plugin runner and worker share mostly the same configuration. The variables BROKER_URL, RESULT_BACKEND, SQLALCHEMY_DATABASE_URI, and GIT_PLUGINS must be configured to the same values! The environent variable for CONCURRENCY can have different values. The worker container must have the environment variable CONTAINER_MODE set to worker to start the image as a worker (since the exact same image as the plugin runner is used as API server and as worker). The user and password set in the redis service must be used to construct the SQLALCHEMY_DATABASE_URI (replace the existing user:password section with the changed user and password). The variable MINIO_CLIENT is a json object that holds the client configuration used to access the minio server.

The containers that use (or can use) a database are configured with a wait script. See https://github.com/ufoscout/docker-compose-wait?tab=readme-ov-file#additional-configuration-options for config options.

Advanced Configuration

The plugin runner and worker and the QHAna backend can be configured by mapping a configuration file into the container. The configuration file must be loaded under /app/instance/config.[json|toml] for the plugin runner and worker. The backend expects the configuration under /app/data/Config.toml but all configurations can be set via env variables documented in the (README.md)[https://github.com/UST-QuAntiL/qhana-backend#configuration-handling]. For the information of what config options are available in the config files please refer to https://github.com/UST-QuAntiL/qhana-plugin-runner and https://github.com/UST-QuAntiL/qhana-backend/blob/main/Config-template.toml.

Volumes

The docker compose file configures three volumes for persistent data storage. The plugin runner and worker container share the same volume named instance that is used as a shared file system. All data generated by the plugin runner is saved to that shared file system or the configured minio instance (which has its own volume). To achieve true persistence the redis and postgres services need to be configured with volumes too (this is only needed for production setups). Please refer to the image documentations on docker hub for how these volumes should be configured.

The volume named experiments is used by the QHAna backend to persist the backend data. It contains all data stored for the experiments (as files in the file system) and an sqlite database. The backend can also be configured to use a MariaDB database with the Config.toml file.

services:
  qhana-plugin-runner:
    volumes:
      - instance:/app/instance
  minio:
    volumes:
      - minio:/data
  worker:
    volumes:
      - instance:/app/instance
  backend:
    volumes:
      - experiments:/app/data
volumes:
  minio:
  instance:
  experiments:

Hint

As a basic security measure all QHAna containers run as an unpriviledged user. This can lead to problems with mounting folders into the same space as the volumes. This stack overflow question might be a good starting point for researching a solution: https://stackoverflow.com/questions/40462189/docker-compose-set-user-and-group-on-mounted-volume

The QHAna backend uses the user ballerina while the plugin runner containers use gunicorn as the username. For completeness: the QHAna UI runs as the user nginx.