Background

The Datica Platform has always been based on running Docker containers, but has historically been restricted to our "git push" model in which customers would push their code to Datica directly from their git repository and rely on buildpacks to build the code for it to be run. Container services provide a way for customers to push a Docker image directly to Datica, skipping the buildpack and "git push" process entirely. There are many benefits to this alternative deployment model, but what we are most excited about is that you can push up an image and know that exactly that image will be run — without needing to rely on a build process, availability of dependencies, or anything else of that nature.

Datica's container service functionality was designed to allow customers to push pre-built images directly to Datica's Docker registry. You create your Dockerfile locally (built on top of Datica's base image), build it into an image locally, and then push it to Datica and sign the content. Once your image is pushed to the registry, a "deploy" command will run the image on the specified container service associated with your Datica environment. The deploy will fail if your image has not been signed, or if the latest signed content hash doesn’t match the content of the image in your repository. 

How do container services work?

Prerequisites

Before getting started, work with Datica on adding a container service for your environment. You can contact product@datica.com to start the process. Note that in some cases, migration of your environment may be necessary to access the new functionality.

Authentication

Authenticating to Datica's Docker registry for pushing and pulling Docker images uses the same user authentication as the rest of the Platform. You will authenticate using your existing email and password. We highly suggest using the Datica CLI for interacting with container services on our Platform since it streamlines the process of interacting with Datica’s Docker registry. Our CLI will automatically manage the registry and namespace for your images and handle content verification.

For user accounts with MFA enabled, you will have to use the Datica CLI to interact with your Datica container services. Docker uses Basic Auth which will reuse the same OTP for multiple requests, causing an authentication error. The Datica CLI wrapper functions will handle this for you.

Image Signing

Datica uses the Docker notary service to ensure that all images pushed to the registry are actually owned by the organization that pushed them. We recommend reading through this article for more background on content trust if you are unfamiliar with the process, while taking specific note of the warning about not losing your root key in the operations and keys section.

Datica does not have access to your root key, and we will be unable to retrieve or reset it for you if the key, or its password, are lost. It is important that you keep your root key in a secure location, preferably offline on a USB stick, and create multiple backups as well. The root key is an important organizational secret that dictates what content actually belongs to you. In the event that it is lost or compromised, every repository signed by your root key on the Datica registry will need to be completely wiped in order for you to resign your images with a new root key.

In addition to keeping multiple copies of your root key in secure locations, it is a good idea to have a copy on a YubiKey. Docker will automatically detect if a YubiKey is present, and search it for an appropriate signing key before looking in your local trust directory. This makes it much easier to bring your root key online for a short period of time when you need it. Otherwise you can manage your root key through the notary CLI, or by manually inserting it into your local trust directory (located at ~/.docker/trust/private). It is important to remember to remove the root key from your computer when you are done with it, but make sure it is backed up first!

Using Container Services

Part 0: Establish a plan for storing your root key

As stated above, it is imperative that you keep the root key that will be used to sign your images in a safe place. Having a specific and documented place for where your Organization will store this key, and its password, will help prevent future issues. You will also want to ensure that you document who can and does have access to the root key for your own policies. DO NOT SKIP THIS STEP.

Part 1: Create your Docker image

NOTE: It is important to remember that containers (i.e. a running instance of an image) are ephemeral. They are designed to be easily deployable, and more importantly, replaceable. Any data in your container will be lost when it is redeployed. Container services are not meant for storing data, and they will not be backed up. We will not be attaching volumes to container services at this time.

  1. Pull Datica's base image from our Docker registry. All Docker images running on Datica must inherit from our base image.
    • The location of the appropriate Docker registry can be found on the dashboard when viewing the container service. The command will look something like this:
      • With the Datica CLI:
        • datica -E "My Env" images pull registry.datica.com/base:latest
        • Note: Due to a known bug, when running the the above pull command, images are not being tagged properly. To fix this, run docker images to find the IMAGE ID. Then run docker tag <IMAGE ID> registry.datica.com/base:latest to tag the image properly. docker build should then work as expected.
      • With the Docker CLI:
        • docker pull registry.datica.com/base:latest
  2. Create your Dockerfile. There are some considerations that should be taken into account when creating a Dockerfile:
    • General Setup:
      • You must inherit from the Datica base image from Step 1.
      • You must use the PORT environment variable when running your application to make sure it can be routed to appropriately.
      • You must set environment variables on your container service for connecting to other services, like a database or cache, in your environment.
      • You must start your application using runit for logging to work appropriately. See the example and deep dive article at the links below for more information.
    • Take a look at the following example app to see how we set up a container. Code snippets in this document are pulled from this example.
    • We highly recommend following the logging and monitoring setup from our healthcheck-app repository. For a deeper dive into creating your Dockerfile, see this article.
  3. Build and tag your Docker image. We recommend that you create a build.sh script in your project to handle both building your application and the docker image. More information on why we suggest this can be found in this article.

Part 2: Push your Docker image to Datica's registry

All images pushed to Datica’s registry have to be signed. The first time you push an image, you will be required to create keys that will be used for every future image that is pushed to Datica’s registry. It is of critical importance that you keep your root key secure. Any other keys in the image signing process can be rotated out with new keys using the notary CLI in the event that they are compromised or lost, but the root key can not be replaced.

With the Datica CLI:

  1. Push and sign your image. Tagging for the Datica registry and namespace, and initializing content trust to sign your image will be handled for you based on the environment specified.
    • datica -E "My Env" images push myapp:v1

With the Docker CLI:

  1. Tag the image locally for use with the registry and namespace for your environment. It must be tagged in the form "<registry>/<namespace>/<name>:<tag>" in order to tell docker which registry you are pushing to and what organization the image belongs to.
    • docker tag myapp:v1 registry.datica.com/pod031234/myapp:v1
  2. Push and sign the tagged image using docker push with content trust enabled
    • DOCKER_CONTENT_TRUST=1 DOCKER_CONTENT_TRUST_SERVER=https://notary.datica.com docker push registry.datica.com/pod031234/myapp:v1

Part 3: Deploy your Docker image onto Datica's Platform

  1. Deploy your image. Just like with pushing through the Datica CLI, there is no need to specify the registry or namespace. This will be parsed for you based on your environment.
    • datica -E "My Env" deploy my-container-service-name myapp:v1

Part 4: Make your container service available

  1. Create a site using the CLI that points to your container service
  2. Verify you can access your running container at the site you set up