Arvados on GKE

This page documents setting up and running the Arvados on Kubernetes Helm chart on Google Kubernetes Engine (GKE).

Note:

This Helm chart does not retain any state after it is deleted. An Arvados cluster created with this Helm chart is entirely ephemeral, and all data stored on the cluster will be deleted when it is shut down. This will be fixed in a future version.

Prerequisites

Install tooling

Install gcloud:

Install kubectl:

$ gcloud components install kubectl

Install helm:

Create the GKE cluster

This can be done via the cloud console or via the command line:

$ gcloud container clusters create <CLUSTERNAME> --zone us-central1-a --machine-type n1-standard-2 --cluster-version 1.15

It takes a few minutes for the cluster to be initialized.

Reserve a static IP

Reserve a static IP in GCE. Make sure the IP is in the same region as your GKE cluster, and is of the “Regional” type.

Connect to the GKE cluster.

Via the web:

  • Click the “Connect” button next to your “GKE cluster”https://console.cloud.google.com/kubernetes/.
  • Execute the “Command-line access” command on your development machine.

Alternatively, use this command:

$ gcloud container clusters get-credentials <CLUSTERNAME> --zone us-central1-a --project <YOUR-PROJECT>

Test the connection:

$ kubectl get nodes

Test helm by running

$ helm ls

There should be no errors. The command will return nothing.

Clone the repository

Clone the repository and nagivate to the arvados-k8s/charts/arvados directory:

$ git clone https://github.com/arvados/arvados-k8s.git
$ cd arvados-k8s/charts/arvados

Start the Arvados cluster

Next, determine the IP address that the Arvados cluster will use to expose its API, Workbench, etc. If you want this Arvados cluster to be reachable from places other than the local machine, the IP address will need to be routable as appropriate.

$ ./cert-gen.sh <IP ADDRESS>

The values.yaml file contains a number of variables that can be modified. At a minimum, review and/or modify the values for

  adminUserEmail
  adminUserPassword
  superUserSecret
  anonymousUserSecret

Now start the Arvados cluster:

$ helm install arvados . --set externalIP=<IP ADDRESS>

At this point, you can use kubectl to see the Arvados cluster boot:

$ kubectl get pods
$ kubectl get svc

After a few minutes, there shouldn’t be any services listed with a ‘Pending’ external IP address. At that point you can access Arvados Workbench at the IP address specified

  • https://<IP ADDRESS>

with the username and password specified in the values.yaml file.

Alternatively, use the Arvados cli tools or SDKs. First set the environment variables:

$ export ARVADOS_API_TOKEN=<superUserSecret from values.yaml>
$ export ARVADOS_API_HOST=<STATIC IP>:444
$ export ARVADOS_API_HOST_INSECURE=true

Test access with:

$ arv user current

Reload

If you make changes to the Helm chart (e.g. to values.yaml), you can reload Arvados with

$ helm upgrade arvados .

Shut down

Note:

This Helm chart does not retain any state after it is deleted. An Arvados cluster created with this Helm chart is entirely ephemeral, and all data stored on the Arvados cluster will be deleted when it is shut down. This will be fixed in a future version.

$ helm del arvados
$ gcloud container clusters delete <CLUSTERNAME> --zone us-central1-a

Previous: Arvados on Minikube Next: Planning and prerequisites

The content of this documentation is licensed under the Creative Commons Attribution-Share Alike 3.0 United States licence.
Code samples in this documentation are licensed under the Apache License, Version 2.0.