Set up a compute node with Singularity

Note:

This page describes the requirements for a compute node in a Slurm or LSF cluster that will run containers dispatched by crunch-dispatch-slurm or arvados-dispatch-lsf. If you are installing a cloud cluster, refer to Build a cloud compute node image.

Note:

These instructions apply when Containers.RuntimeEngine is set to singularity, refer to Set up a compute node with Docker when running docker.

Introduction
Install python-arvados-fuse and crunch-run and squashfs-tools
Set up Singularity
Singularity mksquashfs configuration

Introduction

Please refer to the Singularity documentation in the Architecture section.

This page describes how to configure a compute node so that it can be used to run containers dispatched by Arvados on a static cluster. These steps must be performed on every compute node.

Install python-arvados-fuse and crunch-run and squashfs-tools

Alma/CentOS/Red Hat/Rocky

# dnf install python-arvados-fuse crunch-run squashfs-tools

Debian and Ubuntu

# apt-get install python-arvados-fuse crunch-run squashfs-tools

Install NVIDA CUDA Toolkit (optional)

If you want to use NVIDIA GPUs, install the CUDA toolkit.

In addition, you also must install the NVIDIA Container Toolkit:

DIST=$(. /etc/os-release; echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | \
  sudo apt-key add -
curl -s -L https://nvidia.github.io/libnvidia-container/$DIST/libnvidia-container.list | \
  sudo tee /etc/apt/sources.list.d/libnvidia-container.list
sudo apt-get update
apt-get install libnvidia-container1 libnvidia-container-tools nvidia-container-toolkit

Set up Singularity

Follow the Singularity installation instructions. Note that while the latest stable version is normally expected to be compatible, Arvados is currently tested with singularity 3.10.4.

Make sure singularity and mksquashfs are working:

$ singularity version
singularity-ce version 3.10.4-dirty
$ mksquashfs -version
mksquashfs version 4.4 (2019/08/29)
[...]

Then update Containers.RuntimeEngine in your cluster configuration:

      # Container runtime: "docker" (default) or "singularity"
      RuntimeEngine: singularity

Singularity mksquashfs configuration

Docker images are converted on the fly by mksquashfs, which can consume a considerable amount of RAM. The RAM usage of mksquashfs can be restricted in /etc/singularity/singularity.conf with a line like mksquashfs mem = 256M. The amount of memory made available for mksquashfs should be configured lower than the smallest amount of memory requested by a container on the cluster to avoid the conversion being killed for using too much memory. The default memory allocation in CWL is 256M, so that is also a good choice for the mksquashfs mem setting.

Previous: Set up a compute node with Docker Next: Install the Slurm dispatcher