Singularity container runtime

Overview

Arvados can be configured to use Singularity instead of Docker to execute containers on cloud nodes or a SLURM/LSF cluster. Singularity may be preferable due to its simpler installation and lack of long-running daemon process and special system users/groups.

Current limitations:

  • Even when using the singularity runtime, users’ container images are expected to be saved in Docker format using arv keep docker. Arvados converts the Docker image to Singularity format (.sif) at runtime as needed. Specifying a .sif file as an image when submitting a container request is not yet supported.
  • Singularity does not limit the amount of memory available in a container. Each container will have access to all memory on the host where it runs, unless memory use is restricted by SLURM/LSF.
  • Programs running in containers may behave differently due to differences between Singularity and Docker.
    • The root (image) filesystem is read-only in a Singularity container. Programs that attempt to write outside a designated output or temporary directory are likely to fail.
    • The Docker ENTRYPOINT instruction is ignored.
  • Arvados is tested with Singularity version 3.7.4. Other versions may not work.

Notes:

  • Docker images are converted on the fly by mksquashfs, which can consume a considerable amount of RAM. The RAM usage of mksquashfs can be restricted in /etc/singularity/singularity.conf with a line like mksquashfs mem = 512M. The amount of memory made available for mksquashfs should be configured lower than the smallest amount of memory requested by a container on the cluster to avoid the conversion being killed for using too much memory.

Configuration

To use singularity, first make sure Singularity is installed on your cloud worker image or SLURM/LSF compute nodes as applicable. Note squashfs-tools is required.

$ singularity version
3.7.4
$ mksquashfs -version
mksquashfs version 4.3-git (2014/06/09)
[...]

Then update Containers.RuntimeEngine in your cluster configuration:

      # Container runtime: "docker" (default) or "singularity"
      RuntimeEngine: singularity

Restart your dispatcher (crunch-dispatch-slurm, arvados-dispatch-cloud, or arvados-dispatch-lsf) after updating your configuration file.


Previous: Install the LSF dispatcher Next: Configure container shell access

The content of this documentation is licensed under the Creative Commons Attribution-Share Alike 3.0 United States licence.
Code samples in this documentation are licensed under the Apache License, Version 2.0.