This page describes the requirements for a compute node in a Slurm or LSF cluster that will run containers dispatched by
arvados-dispatch-lsf. If you are installing a cloud cluster, refer to Build a cloud compute node image.
These instructions apply when Containers.RuntimeEngine is set to
singularity, refer to Set up a compute node with Docker when running
Please refer to the Singularity documentation in the Architecture section.
This page describes how to configure a compute node so that it can be used to run containers dispatched by Arvados on a static cluster. These steps must be performed on every compute node.
# yum install python-arvados-fuse crunch-run squashfs-tools
# apt-get install python-arvados-fuse crunch-run squashfs-tools
Follow the Singularity installation instructions. Make sure
mksquashfs are working:
$ singularity version 3.7.4 $ mksquashfs -version mksquashfs version 4.3-git (2014/06/09) [...]
Containers.RuntimeEngine in your cluster configuration:
# Container runtime: "docker" (default) or "singularity" RuntimeEngine: singularity
Docker images are converted on the fly by
mksquashfs, which can consume a considerable amount of RAM. The RAM usage of mksquashfs can be restricted in
/etc/singularity/singularity.conf with a line like
mksquashfs mem = 256M. The amount of memory made available for mksquashfs should be configured lower than the smallest amount of memory requested by a container on the cluster to avoid the conversion being killed for using too much memory. The default memory allocation in CWL is 256M, so that is also a good choice for the
mksquashfs mem setting.
The content of this documentation is licensed under the
Commons Attribution-Share Alike 3.0 United States licence.
Code samples in this documentation are licensed under the Apache License, Version 2.0.