Arvados can be configured to use Singularity instead of Docker to execute containers on cloud nodes or a SLURM/LSF cluster. Singularity may be preferable due to its simpler installation and lack of long-running daemon process and special system users/groups.
arv keep docker. Arvados converts the Docker image to Singularity format (
.sif) at runtime as needed. Specifying a
.siffile as an image when submitting a container request is not yet supported.
mksquashfs, which can consume a considerable amount of RAM. The RAM usage of mksquashfs can be restricted in
/etc/singularity/singularity.confwith a line like
mksquashfs mem = 512M. The amount of memory made available for mksquashfs should be configured lower than the smallest amount of memory requested by a container on the cluster to avoid the conversion being killed for using too much memory.
To use singularity, first make sure Singularity is installed on your cloud worker image or SLURM/LSF compute nodes as applicable. Note
squashfs-tools is required.
$ singularity version 3.7.4 $ mksquashfs -version mksquashfs version 4.3-git (2014/06/09) [...]
Containers.RuntimeEngine in your cluster configuration:
# Container runtime: "docker" (default) or "singularity" RuntimeEngine: singularity
Restart your dispatcher (
arvados-dispatch-lsf) after updating your configuration file.
The content of this documentation is licensed under the
Commons Attribution-Share Alike 3.0 United States licence.
Code samples in this documentation are licensed under the Apache License, Version 2.0.