Install a shell server

There is nothing inherently special about an Arvados shell server. It is just a GNU/Linux machine with Arvados utilites and SDKs installed. For optimal performance, the Arvados shell server should be on the same LAN as the Arvados cluster, but that is not required.

Install API tokens

Please follow the API token guide to get API tokens for your Arvados account and install them on your shell server. We will use those tokens to test the SDKs as we install them.

Install the Ruby SDK and utilities

First, install the curl development libraries necessary to build the Arvados Ruby SDK. On Debian-based systems:

~$ sudo apt-get install libcurl4-openssl-dev

On Red Hat-based systems:

~$ sudo yum install libcurl-devel

Next, install the arvados-cli Ruby gem. If you’re using RVM:

~$ sudo /usr/local/rvm/bin/rvm-exec default gem install arvados-cli

If you’re not using RVM:

~$ sudo -i gem install arvados-cli

Install the Python SDK and utilities

~$ sudo yum install centos-release-scl scl-utils

~$ sudo yum-config-manager --enable rhel-server-rhscl-7-rpms

On Red Hat-based systems:

~$ echo 'exclude=python2-llfuse' | sudo tee -a /etc/yum.conf
~$ sudo yum install python-arvados-python-client python-arvados-fuse crunchrunner

On Debian-based systems:

~$ sudo apt-get install python-arvados-python-client python-arvados-fuse crunchrunner

Install Git and curl

On a Debian-based system, install the following packages:

~$ sudo apt-get install git curl

On a Red Hat-based system, install the following packages:

~$ sudo yum install git curl

Update Git Config

Configure git to use the ARVADOS_API_TOKEN environment variable to authenticate to arv-git-httpd. We use the --system flag so it takes effect for all current and future user accounts. It does not affect git’s behavior when connecting to other git servers.

~$ sudo git config --system 'credential.https://git.uuid_prefix.your.domain/.username' none
~$ sudo git config --system 'credential.https://git.uuid_prefix.your.domain/.helper' '!cred(){ cat >/dev/null; if [ "$1" = get ]; then echo password=$ARVADOS_API_TOKEN; fi; };cred'

Install arvados-login-sync

This program makes it possible for Arvados users to log in to the shell server — subject to permissions assigned by the Arvados administrator — using the SSH keys they upload to Workbench. It sets up login accounts, updates group membership, and adds users’ public keys to the appropriate authorized_keys files.

Create an Arvados virtual_machine object representing this shell server. This will assign a UUID.

apiserver:~$ arv --format=uuid virtual_machine create --virtual-machine '{"hostname":"your.shell.server.hostname.without.domain"}'
zzzzz-2x53u-zzzzzzzzzzzzzzz

Create a token that is allowed to read login information for this VM.

apiserver:~$ arv api_client_authorization create --api-client-authorization '{"scopes":["GET /arvados/v1/virtual_machines/zzzzz-2x53u-zzzzzzzzzzzzzzz/logins"]}'
{
 ...
 "api_token":"zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz",
 ...
}

Note the UUID and the API token output by the above commands: you will need them in a minute.

Install the arvados-login-sync program.

If you’re using RVM:

shellserver:~$ sudo -i `which rvm-exec` default gem install arvados-login-sync

If you’re not using RVM:

shellserver:~$ sudo -i gem install arvados-login-sync

Install cron.

On Red Hat-based distributions:

~$ sudo yum install cronie
~$ sudo systemctl enable crond
~$ sudo systemctl start crond

On Debian-based systems:

~$ sudo apt-get install cron

Configure cron to run the arvados-login-sync program every 2 minutes.

If you’re using RVM:

shellserver:~$ sudo bash -c 'umask 077; tee /etc/cron.d/arvados-login-sync' <<'EOF'
ARVADOS_API_HOST="uuid_prefix.your.domain"
ARVADOS_API_TOKEN="the_token_you_created_above"
ARVADOS_VIRTUAL_MACHINE_UUID="zzzzz-2x53u-zzzzzzzzzzzzzzz"
*/2 * * * * root /usr/local/rvm/bin/rvm-exec default arvados-login-sync
EOF

If you’re not using RVM:

shellserver:~$ sudo bash -c 'umask 077; tee /etc/cron.d/arvados-login-sync' <<'EOF'
ARVADOS_API_HOST="uuid_prefix.your.domain"
ARVADOS_API_TOKEN="the_token_you_created_above"
ARVADOS_VIRTUAL_MACHINE_UUID="zzzzz-2x53u-zzzzzzzzzzzzzzz"
*/2 * * * * root arvados-login-sync
EOF

A user should be able to log in to the shell server when the following conditions are satisfied:

• The user has uploaded an SSH public key: Workbench → Account menu → “SSH keys” item → “Add new SSH key” button.
• As an admin user, you have given the user permission to log in: Workbench → Admin menu → “Users” item → “Show” button → “Admin” tab → “Setup shell account” button.
• Two minutes have elapsed since the above conditions were satisfied, and the cron job has had a chance to run.