Configuration files

Arvados /etc/arvados/config.yml

The configuration file is normally found at /etc/arvados/config.yml and will be referred to as just config.yml in this guide. This configuration file must be kept in sync across every service node in the cluster, but not shell and compute nodes (which do not require config.yml).

Syntax

The configuration file is in YAML format. This is a block syntax where indentation is significant (similar to Python). By convention we use two space indent. The first line of the file is always “Clusters:”, underneath it at the first indent level is the Cluster ID. All the actual cluster configuration follows under the Cluster ID. This means all configuration parameters are indented by at least two levels (four spaces). Comments start with # .

We recommend a YAML-syntax plugin for your favorite text editor, such as yaml-mode (Emacs) or yaml-vim.

Example file:

Clusters:                         # Clusters block, everything else is listed under this
  abcde:                          # Cluster ID, everything under it is configuration for this cluster
    ExampleConfigKey: "fghijk"    # An example configuration key
    ExampleConfigGroup:           # A group of keys
      ExampleDurationConfig: 12s  # Example duration
      ExampleSizeConfig: 99KiB    # Example with a size suffix

Each configuration group may only appear once. When a configuration key is within a config group, it will be written with the group name leading, for example ExampleConfigGroup.ExampleSizeConfig.

Duration suffixes are s=seconds, m=minutes or h=hours.

Size suffixes are K=10 ³, Ki=2 ¹⁰ , M=10 ⁶, Mi=2 ²⁰, G=10 ⁹, Gi=2 ³⁰, T=10 ¹², Ti=2 ⁴⁰, P=10 ¹⁵, Pi=2 ⁵⁰, E=10 ¹⁸, Ei=2 ⁶⁰. You can optionally follow with a “B” (eg “MB” or “MiB”) for readability (it does not affect the units.)

Create empty configuration file

Change webserver-user to the user that runs your web server process. This is www-data on Debian-based systems, and nginx on Red Hat-based systems.

# export ClusterID=xxxxx
# umask 027
# mkdir -p /etc/arvados
# cat > /etc/arvados/config.yml <<EOF
Clusters:
  ${ClusterID}:
EOF
# chgrp webserver-user /etc/arvados /etc/arvados/config.yml

Nginx configuration

This guide will also cover setting up Nginx as a reverse proxy for Arvados services. Nginx performs two main functions: TLS termination and virtual host routing. The virtual host configuration for each component will go in its own file in /etc/nginx/conf.d/.

Synchronizing config file

The Arvados configuration file must be kept in sync across every service node in the cluster. We strongly recommend using a devops configuration management tool such as Puppet to synchronize the config file. Alternately, something like the following script to securely copy the configuration file to each node may be helpful. Replace the ssh targets with your nodes.

#!/bin/sh
sudo cat /etc/arvados/config.yml | ssh 10.0.0.2 sudo sh -c "'cat > /etc/arvados/config.yml'"
sudo cat /etc/arvados/config.yml | ssh 10.0.0.3 sudo sh -c "'cat > /etc/arvados/config.yml'"

---

Previous: Arvados package repositories Next: InternalURLs and ExternalURL