Deploy with Docker Compose ============================== ApertureDB publishes each Compose layout as an OCI package. Select a release and configure it with environment variables; the package provides the service definitions, default images, and connections between services. Run ApertureDB using Docker with Linux container support and Docker Compose 2.39.4 or later. Authenticate with ``docker login`` if your supplied images require registry access. The commands below use a POSIX shell. Choose a layout ------------------- Choose a published release from :doc:`../Deployment` and one of these layouts: .. list-table:: Layouts :header-rows: 1 :widths: 25 40 35 * - Layout - Storage - Use * - ``single-node`` - Local database files and logs - One database without distributed checkpointing. * - ``distributed-local`` - Local objects and binlogs shared by the databases and checkpointer - Exercise distributed features on one host. * - ``distributed-cloud`` - S3 or Google Cloud Storage for database content; local logs and coordination data - Use an existing object-storage bucket. Each layout has its own Docker Hub repository: * ``aperturedata/aperturedb-compose-single-node`` * ``aperturedata/aperturedb-compose-distributed-local`` * ``aperturedata/aperturedb-compose-distributed-cloud`` The package and service images use the same tag. The layout name is part of the repository name. All layouts run on one Docker host and include the Web UI and HTTP/TCP gateways. Multiple database containers on one host do not protect against host failure. Local persistence uses Docker named volumes unless you set ``DATA_DIR``. Configure and start ----------------------- Create a local file named ``deployment.env`` with the settings for your deployment: .. code-block:: text ADB_MASTER_KEY='' MONITORING_PASSWORD='' GF_SECURITY_ADMIN_PASSWORD='' GATEWAY=127.0.0.1 ``GATEWAY`` selects the host interface for published ports. Loopback restricts access to the Docker host; use a reachable interface for remote clients. Leaving it unset binds all interfaces. The initial database login is ``admin`` with ``ADB_MASTER_KEY`` as its password. For ``distributed-cloud``, also add the following settings: .. code-block:: text ADB_STORAGE_TYPE=s3 ADB_STORAGE_BUCKET_NAME= ADB_STORAGE_CREDENTIALS='{"access_key":"","secret_access_key":"","region":"us-west-2"}' All three storage settings are required for that layout. For Google Cloud Storage, use ``ADB_STORAGE_TYPE=gs`` and supply the service account credential JSON on one line. See :doc:`../Deployment` for bucket permissions. Use a dedicated bucket for each independent database deployment. Set ``ADB_COMPOSE`` to your chosen release and layout. Replace ```` with a published release, such as ``X.Y.Z`` or ``X.Y.Z-rc.N``, and replace ``single-node`` if using a distributed layout: .. code-block:: bash ADB_COMPOSE='oci://aperturedata/aperturedb-compose-single-node:' chmod 600 deployment.env docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" config --quiet docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" up -d docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" ps Docker Compose fetches the configuration directly from the registry. You do not need to download YAML files, extract a package, or check out a repository. Keep ``deployment.env``, the project name (``-p aperturedb``), and the selected reference for subsequent commands. Use a different project name for each independent deployment. Connect ----------- With the default ports and loopback interface: .. list-table:: Access points :header-rows: 1 * - Service - Address * - Web UI - ``http://localhost/`` or ``https://localhost/`` * - REST API - ``http://localhost/api/`` or ``https://localhost/api/`` * - Native database protocol - ``localhost:55555`` * - Grafana, with telemetry enabled - ``http://localhost/grafana/`` or ``https://localhost/grafana/`` Log into the Web UI as ``admin`` using ``ADB_MASTER_KEY``. Configure the ports and certificates below, then run the client example in :ref:`deployment-verify` to confirm that the database accepts queries. For remote clients, replace ``localhost`` with the Docker host's address. See :ref:`configuration-deployment-mapping` for the distinction between backend, Lenz, and web TLS settings. Published ports and request limits -------------------------------------- Add only the settings you want to change to ``deployment.env``: .. list-table:: Network settings :header-rows: 1 :widths: 35 20 45 * - Variable - Default - Purpose * - ``HTTP_PORT`` - ``80`` - Published HTTP port. * - ``HTTPS_PORT`` - ``443`` - Published HTTPS port. * - ``ADB_TCP_PORT`` - ``55555`` - Published native database port. * - ``ADB_HEALTH_PORT`` - ``58085`` - Published database health check port. * - ``HTTP_BODY_LIMIT`` - ``268435456`` - Maximum HTTP request body size in bytes. * - ``HTTP_TIMEOUT_SECONDS`` - ``300`` - Upstream inactivity timeout for API and Web UI requests. * - ``LNZ_FORCE_SSL`` - ``false`` - Require encrypted native TCP connections when set to ``true``. Only one deployment can bind a given port on the same host interface. For example, use ``HTTP_PORT=8080`` and ``HTTPS_PORT=8443`` if the default ports are already occupied. Internal container ports remain unchanged. HTTP redirects to HTTPS on the configured ``HTTPS_PORT``, preserving the request method and body. HTTPS supports TLS 1.2 and 1.3. Use ``/`` for the Web UI, ``/api/`` for the REST API, and ``/grafana/`` for dashboards. Grafana Live and streamed responses are supported. ``HTTP_BODY_LIMIT`` limits each request body; oversized requests receive 413. The HTTP service allows active requests up to five minutes to finish during shutdown. Long-lived sessions should reconnect after an update. For a host reverse proxy, bind to loopback and forward to the configured HTTPS port. Preserve the ``/api/`` and ``/grafana/`` paths. Set ``GF_SERVER_ROOT_URL`` to the public URL ending in ``/grafana/`` when accessing Grafana remotely; its default is ``https://localhost:/grafana/``. Configure native TCP TLS separately from the web proxy. Generated certificate ----------------------------- By default, the deployment generates a self-signed certificate and retains it in a named volume. It is valid for ``localhost``, ``127.0.0.1``, ``::1``, and the service names ``lenz`` and ``http-proxy``, so containers on the Compose network can verify it too. To add the DNS names or IP addresses your clients use, list them in ``deployment.env``, separated by commas: .. code-block:: text TLS_CERT_HOSTNAMES=db.example.com,203.0.113.10 The generated pair is reused on subsequent starts while this list is unchanged. After a change, the next start generates a new certificate. Clients that verify the generated certificate need a copy of it: .. code-block:: bash docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" \ cp lenz:/etc/lenz/certificate/tls.crt ./aperturedb.crt Pass that file as the client's CA certificate, such as ``ca_cert`` in the Python SDK. Use your own certificates ----------------------------- To use your own certificate instead, add these settings to ``deployment.env``: .. code-block:: text TLS_CERT_DIR=/etc/aperturedb/tls TLS_CERT_FILENAME=fullchain.pem TLS_KEY_FILENAME=privkey.pem LNZ_FORCE_SSL=true ``TLS_CERT_DIR`` must be an absolute directory on the Docker host containing a PEM certificate chain and a matching unencrypted RSA private key. ECDSA keys are not supported. The filenames default to ``tls.crt`` and ``tls.key``; use different bare filenames containing only letters, digits, dots, underscores, or hyphens. The names ``.`` and ``..`` are invalid. Keep symlink targets inside the mounted directory, and make the files readable by the container users. Leave ``TLS_CERT_HOSTNAMES`` unset; a supplied certificate defines its own names. The same startup command applies with these settings. The directory is mounted read-only and the supplied pair is validated before the gateways start. Missing or invalid files prevent gateway startup; supplied files are never generated or overwritten. Providing certificates alone does not require encrypted native TCP connections; set ``LNZ_FORCE_SSL=true`` to enforce them. Replace both certificate files in the mounted directory to rotate them. Valid replacements are loaded automatically; existing connections can finish with the previous certificate. Invalid replacements leave the last valid pair active. Leave ``TLS_CERT_DIR`` and the filename variables unset to use the generated certificate. If only one of its two files is present, restore the complete pair before starting again. Optional launch settings ------------------------ The explicit project name is optional. If you omit it, keep the working directory fixed so later commands use the same project and persistent volumes. When settings are exported in the shell, ``--env-file`` is also optional: .. code-block:: bash docker compose -f 'oci://aperturedata/aperturedb-compose-distributed-cloud:' up All optional settings below use the same single Compose reference. Set variables in ``deployment.env`` or the shell; shell values take precedence. Omitted settings use the package defaults. Automate with CI/CD or infrastructure as code ------------------------------------------------- Use the same published package for manual and automated deployments. Your pipeline or infrastructure-as-code (IaC) configuration only needs to maintain the release reference, project name, and deployment variables. You do not need to copy, template, or maintain Compose YAML files for the settings documented below. Keep non-secret settings in your environment configuration and supply passwords and credentials from your secret store. Automation can generate ``deployment.env`` or export variables directly. With ``ADB_COMPOSE`` set to a specific release and layout, and deployment variables exported, run: .. code-block:: bash docker compose -p aperturedb -f "$ADB_COMPOSE" config --quiet docker compose -p aperturedb -f "$ADB_COMPOSE" up -d Use the same package reference across your environments, changing only their variables. Keep each deployment's project name and storage locations stable so subsequent runs reuse its data. Automation must also prepare the Docker host, registry access, and any supplied certificates or storage. Image repositories ---------------------- To use a registry mirror, set a prefix including its trailing slash: .. code-block:: text IMAGE_REPOSITORY_PREFIX=registry.example.com/mirror/ This applies to every image, including third-party images. For example, ``aperturedata/aperturedb:`` becomes ``registry.example.com/mirror/aperturedata/aperturedb:`` and ``alpine/openssl:`` becomes ``registry.example.com/mirror/alpine/openssl:``. Mirror the images required by your selected layout and profiles, and authenticate to that registry before starting. Use a complete image reference when a mirror changes a repository name or when you need a digest-pinned image: .. code-block:: text APERTUREDB_IMAGE=registry.example.com/database/aperturedb:custom-release LENZ_IMAGE=registry.example.com/proxy/lenz@sha256: Complete image overrides take precedence over ``IMAGE_REPOSITORY_PREFIX``. Supported variables are ``APERTUREDB_IMAGE``, ``CHECKPOINTER_IMAGE``, ``LENZ_IMAGE``, ``WEBUI_IMAGE``, ``GRAFANA_IMAGE``, ``PROMTAIL_IMAGE``, ``ZOOKEEPER_IMAGE``, ``PROMETHEUS_IMAGE``, ``LOKI_IMAGE``, and ``OPENSSL_IMAGE``. Checkpointer and ZooKeeper apply only to distributed layouts. Keep the bundled image defaults unless you need a compatible override. Host storage ---------------- To use host directories instead of Docker named volumes, set an absolute parent path in ``deployment.env``: .. code-block:: text DATA_DIR=/srv/aperturedb Each data or log location uses ``DATA_DIR/``. Prepare the following subdirectories for your layout, with suitable ownership and permissions for the container users: .. list-table:: Host subdirectories :header-rows: 1 :widths: 25 75 * - Layout - Subdirectories * - ``single-node`` - ``aperturedb_db``, ``aperturedb_logs`` * - ``distributed-local`` - ``aperturedb_objects``, ``aperturedb_binlogs``, ``aperturedb_acl``, ``aperturedb_0_logs``, ``aperturedb_1_logs``, ``checkpointer_checkpointer_logs``, ``zookeeper_zoo1_data``, ``zookeeper_zoo2_data``, ``zookeeper_zoo3_data`` * - ``distributed-cloud`` - ``aperturedb_0_logs``, ``aperturedb_1_logs``, ``checkpointer_checkpointer_logs``, ``zookeeper_zoo1_data``, ``zookeeper_zoo2_data``, ``zookeeper_zoo3_data`` * - Any layout with telemetry - Also create ``grafana``, ``loki``, and ``prometheus``. Compose may create missing directories, but host mounts do not initialize ownership from image directories. Use a separate ``DATA_DIR`` for each independent deployment. Run the same startup command after setting it; no additional Compose file is needed. Changing ``DATA_DIR`` does not move existing data. Stop the deployment and copy its data with the required ownership and permissions before switching locations. Generated certificates remain in their named volume independently of ``DATA_DIR``; use ``TLS_CERT_DIR`` for your own certificates. Monitoring -------------- To enable Prometheus, Grafana, Loki, and Promtail log collection: .. code-block:: bash docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" \ --profile telemetry up -d ``MONITORING_PASSWORD`` authenticates metrics collection; it is separate from the Grafana login. ``GF_SECURITY_ADMIN_USER`` defaults to ``admin``. Set ``GF_SECURITY_ADMIN_PASSWORD`` before initializing Grafana. Change an existing administrator password through Grafana when retaining its data volume. Set ``GF_SERVER_ROOT_URL`` to the public Grafana URL if needed, for example ``https://database.example.com/grafana/``. Its default is ``%(protocol)s://%(domain)s:%(http_port)s/grafana/``. ``PROMETHEUS_RETENTION_TIME`` defaults to ``28d`` and ``LOKI_RETENTION_PERIOD`` to ``672h``. Adjust them to control how long metrics and logs are retained. Scraping and service discovery stay fixed at five seconds, with a four-second scrape timeout; monitoring calculations depend on that interval. Container CPU and memory limits --------------------------------- Every service supports optional CPU and memory limits. All limits default to ``0`` (unlimited), including the services in the ``telemetry`` profile. No memory reservation or swap limit is added. Add the settings you need to ``deployment.env``; limits apply only to services present in the selected layout and enabled profiles. CPU values are Docker CPU counts, such as ``0.5`` or ``2``. Memory values are Docker sizes, such as ``512m`` or ``2g``. Use ``0`` for an unlimited resource. .. list-table:: Per-service resource variables :header-rows: 1 :widths: 25 35 40 * - Service - CPU variable - Memory variable * - ``aperturedb`` (single-node) - ``ADB_CPUS`` - ``ADB_MEMORY_LIMIT`` * - ``aperturedb0`` - ``ADB0_CPUS`` - ``ADB0_MEMORY_LIMIT`` * - ``aperturedb1`` - ``ADB1_CPUS`` - ``ADB1_MEMORY_LIMIT`` * - ``checkpointer`` - ``CHECKPOINTER_CPUS`` - ``CHECKPOINTER_MEMORY_LIMIT`` * - ``lenz`` - ``LNZ_CPUS`` - ``LNZ_MEMORY_LIMIT`` * - ``zoo1`` - ``ZOO1_CPUS`` - ``ZOO1_MEMORY_LIMIT`` * - ``zoo2`` - ``ZOO2_CPUS`` - ``ZOO2_MEMORY_LIMIT`` * - ``zoo3`` - ``ZOO3_CPUS`` - ``ZOO3_MEMORY_LIMIT`` * - ``promtail`` (single-node) - ``PROMTAIL_CPUS`` - ``PROMTAIL_MEMORY_LIMIT`` * - ``promtail0`` - ``PROMTAIL0_CPUS`` - ``PROMTAIL0_MEMORY_LIMIT`` * - ``promtail1`` - ``PROMTAIL1_CPUS`` - ``PROMTAIL1_MEMORY_LIMIT`` * - ``loki`` - ``LOKI_CPUS`` - ``LOKI_MEMORY_LIMIT`` * - ``prometheus`` - ``PROMETHEUS_CPUS`` - ``PROMETHEUS_MEMORY_LIMIT`` * - ``grafana`` - ``GRAFANA_CPUS`` - ``GRAFANA_MEMORY_LIMIT`` * - ``webui`` - ``WEBUI_CPUS`` - ``WEBUI_MEMORY_LIMIT`` * - ``ca`` (certificate initialization) - ``CA_CPUS`` - ``CA_MEMORY_LIMIT`` * - ``http-proxy`` - ``HTTP_PROXY_CPUS`` - ``HTTP_PROXY_MEMORY_LIMIT`` In distributed layouts, ``ADB_CPUS`` and ``ADB_MEMORY_LIMIT`` remain the shared defaults for both database nodes. An ``ADB0_*`` or ``ADB1_*`` setting overrides the corresponding shared limit for that node. An explicit per-node ``0`` means unlimited even when the shared value is nonzero. Existing ``CHECKPOINTER_*`` and ``LNZ_*`` resource variables retain their behavior. For example: .. code-block:: text ADB_CPUS=2 ADB_MEMORY_LIMIT=4g ADB0_MEMORY_LIMIT=6g HTTP_PROXY_CPUS=0.5 Both database nodes use two CPUs. The first has a 6 GiB memory limit, and the second has a 4 GiB limit. The HTTP proxy has a half-CPU limit. Other resources remain unlimited unless configured separately. Validate every profile before starting containers: .. code-block:: bash docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" \ --profile '*' config --quiet Capacity and logging ------------------------ Use these settings when tuning a deployment for your host and workload: .. list-table:: Capacity and logging settings :header-rows: 1 :widths: 40 25 35 * - Variable - Default - Purpose * - ``ADB_KVGD_DB_SIZE`` - ``204800`` - Graph-store size setting. * - ``ADB_KVGD_DB_INC_SIZE`` - ``2048`` - Graph-store growth increment for the database; ``0`` uses the built-in increment. * - ``CHECKPOINTER_KVGD_DB_INC_SIZE`` - ``6144`` - Graph-store growth increment for the checkpointer in distributed layouts. * - ``ADB_KVGD_DB_SYNC`` - ``true`` single-node; ``false`` distributed - Flush the graph store to disk on every commit. ``false`` writes faster but can lose the latest commits if the host fails. * - ``ADB_MERGE_WRITE_REQUEST_UPTO`` - ``5`` - Number of queued write requests the database may merge into one commit; ``0`` disables merging. * - ``ADB_RESOURCES_MAX_STORAGE`` - ``4096`` - Graph-store storage limit in GiB; ``0`` means unlimited. * - ``ADB_QUERY_THREADS`` - ``16`` single-node; ``48`` distributed - Query workers per database process; independent of accepted connections. * - ``LNZ_ADB_MAX_CONCURRENCY`` - ``24`` single-node; ``48`` distributed - Database concurrency per gateway process. * - ``LNZ_TIMEOUT_FACTOR`` - ``4`` - Scales the gateway's network timeouts; raise it for slow clients or very large requests. * - ``ADB_LOG_LEVEL`` - ``1`` - Database log level. * - ``CHECKPOINTER_LOG_LEVEL`` - ``0`` - Checkpointer log level in distributed layouts. * - ``ZOOKEEPER_TICK_TIME`` - ``8000`` - ZooKeeper tick, in milliseconds, in distributed layouts. * - ``ZOOKEEPER_MIN_SESSION_TIMEOUT`` - ``20000`` - Lowest ZooKeeper session timeout, in milliseconds, in distributed layouts. A database that loses ZooKeeper for longer loses its session. * - ``ZOOKEEPER_MAX_TIME_TO_WAIT_FOR_EPOCH`` - ``30000`` - Milliseconds before an elected leader waiting for a quorum can step down when one of its voters resumes the election. Use ``0`` to allow an immediate retry on such a notification or ``-1`` to disable the early retry. CPU limits accept fractional cores, such as ``ADB_CPUS=2.5``. Memory limits accept sizes such as ``ADB_MEMORY_LIMIT=8g``. A limit of ``0`` means unlimited. These settings do not change the number of containers. Leave database client capacity above gateway concurrency to allow for internal clients and all gateway processes. See :doc:`../ConfigFile` for database configuration details. In both distributed layouts, Docker registers the ZooKeeper peer names (``zoo1``, ``zoo2``, and ``zoo3``) when the containers join the network. Health checks do not gate DNS publication. The epoch setting applies to all three peers. The single-node layout runs without ZooKeeper. .. _stop-restart-and-upgrade: Stop, restart, and apply changes ---------------------------------------- Use the same project name, environment file, and Compose reference for lifecycle commands. To stop all services, including telemetry, and start them again: .. code-block:: bash docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" \ --profile telemetry down docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" \ --profile telemetry up -d Omit ``--profile telemetry`` on startup if you do not want monitoring. ``down`` retains named volumes, host directories, and cloud bucket contents. ``down -v`` deletes managed named volumes, including generated certificates; it does not delete data in host directories or object storage. Review the compatibility notes in :doc:`../ReleaseNotes` for your target version, then complete :ref:`deployment-upgrades`. Set ``ADB_COMPOSE`` to the chosen package for the same layout, keep the project name and storage locations fixed, then run: .. code-block:: bash docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" \ --profile telemetry pull docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" \ --profile telemetry up -d --remove-orphans ``--remove-orphans`` removes services from this Compose project that are no longer part of the selected package, freeing their published ports. If startup fails, inspect status and logs: .. code-block:: bash docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" ps -a docker compose -p aperturedb --env-file ./deployment.env -f "$ADB_COMPOSE" logs --tail=100 An exited ``ca`` container with exit code 0 is expected. For failures, check registry access, host port conflicts, certificate paths and permissions, write access to storage directories, and cloud credentials where applicable.