Arvados™ Docs
WARNING - you are viewing the documentation for an old version of Arvados. For the latest version, click here.
  1. Topics
    1. Arvados admin overview
  2. Users and Groups
    1. User management
    2. User management at the CLI
    3. Role group management at the CLI
    4. Reassign user data ownership
    5. Link user accounts
    6. Configuring federation
    7. Migrating users to federated accounts
    8. Changing upstream login providers
    9. Synchronizing from external sources
    10. Securing API access with scoped tokens
    11. Automatic logout and token expiration
  3. Monitoring
    1. Logging
    2. Metrics
    3. Health checks
    4. Diagnostics
    5. Management token
    6. User activity report
  4. Data Management
    1. Configuring collection versioning
    2. Configuring collection's managed properties
    3. Restricting upload or download
    4. Balancing Keep servers
    5. Preventing container reuse
    6. Logs table management
    7. Metadata vocabulary
    8. Configuring storage classes
    9. Recovering data
    10. Measuring deduplication
  5. Cloud
    1. Using Preemptible instances
    2. Testing cloud configuration
  6. Other
    1. Migrating from Docker 1.9
    2. Upgrading to Containers API

Migrating from Docker 1.9

If you have an existing Arvados installation using Docker 1.9 and wish to update to Docker 1.10+, you must migrate the Docker images stored in Keep.

The arv-migrate-docker19 tool converts Docker images stored in Arvados from image format v1 (Docker <= 1.9) to image format v2 (Docker >= 1.10).

Requires Docker running on the local host (can be either 1.9 or 1.10+). Linux kernel >= 3.18-rc6 to support overlayfs.

Usage:

  1. Install arvados/migrate-docker19 image: docker pull arvados/migrate-docker19:1.0. If you’re unable to do this, you can run arvados/docker/migrate-docker19/build.sh to create arvados/migrate-docker19 Docker image.
  2. Make sure you have the right modules installed: sudo modprobe overlayfs bridge br_netfilter nf_nat
  3. Set ARVADOS_API_HOST and ARVADOS_API_TOKEN to the cluster you want to migrate.
  4. Your temporary directory should have the size of all layers of the biggest image in the cluster, this is hard to estimate, but you can start with five times that size. You can set up a different directory by using the --tempdir switch. Make sure that the user running the docker daemon has permissions to write in that directory.
  5. Run arv-migrate-docker19 --dry-run from the Arvados Python SDK on the host (not in a container). This will print out some information useful for the migration.
  6. Finally to make the migration run arv-migrate-docker19 from the Arvados Python SDK on the host (not in a container).

This will query Arvados for v1 format Docker images. For each image that does not already have a corresponding v2 format image (as indicated by a docker_image_migration tag) it will perform the following process:

i) download the image from Arvados
ii) load it into Docker
iii) update the Docker version, which updates the image
iv) save the v2 format image and upload to Arvados
v) create a migration link

Once the Docker images in Keep have been migrated, upgrade the version of Docker used across the cluster. Finally, update the API server configuration from “v1” to “v2” to reflect the supported Docker image version:

docker_image_formats: ["v2"]
Previous: Testing cloud configuration Next: Upgrading to Containers API

The content of this documentation is licensed under the Creative Commons Attribution-Share Alike 3.0 United States licence.
Code samples in this documentation are licensed under the Apache License, Version 2.0.