Managing secrets in Docker Compose and GitHub Actions deployments

When deploying Docker Compose applications, here's how you can manage secrets without embedding them in your containers

October 20, 2024
Managing secrets in Docker Compose and GitHub Actions deployments
Table of Contents

Not everyone has something to hide, but when it comes to database passwords, tokens and encryption keys, you should certainly keep them to yourself! Typically, secrets are stored in environment variables or configuration files embedded in Docker images. The downside to this approach is that they can be extracted from an image or viewed from a running container with docker inspect.

In this example, I will demonstrate how to securely store your secrets. That is, not in a configuration file, .env file or anywhere in plain-text — let's keep them out of our images and use Docker Compose secrets to inject them at runtime.

To illustrate this, I've developed an example project using Flask and PostgreSQL: a simple web app where users can post comments and read comments. There are secrets such as a database password shared by the Flask and Postgres containers, a Flask secret key for CSRF tokens and, a private key for reCAPTCHA. The source for the example is available on GitHub and you can view the deployed application here.

Using Docker Compose secrets

In Docker Compose, secrets are defined in the compose.yaml file. They're made available to containers at runtime by creating a read-only mount at /run/secrets/[mysecret] and can be set from either a file or environment variable.

Here's how you declare secrets in a Compose file. The secrets top-level element contains a list of secrets by name either loaded from file or environment. You must then specify which secrets are required by which service.

services:
  myapp:
    ...
    secrets:
      - my_secret

secrets:
  my_secret:
    file: ./my_secret.txt

As mentioned previously, the secrets are made available to running containers in the /run/secrets/ directory. For example, my_secret can be read like a regular text file at /run/secrets/my_secret.

Storing our secrets using Bitwarden

There are a few options for storing secrets before they are used by containers, with some advantages and disadvantages to each. The preferred option for storing application secrets is to use a secrets manager such as Bitwarden Secrets Manager, 1Password, LastPass, Doppler, etc. This makes it easy to view and update secrets as required.

In this example we're going to use Bitwarden Secrets Manager since it integrates nicely with GitHub Actions.

Storing the secrets

In Bitwarden Secrets Manager, create a project for your app and create all the secrets you need within that. Each secret has its own GUID that's used to retrieve it in our GitHub Actions workflow.

Bitwarden Secrets

Retrieving the secrets

With the Bitwarden Secrets Manager action, you can retrieve secrets by GUID and store them in pipeline environment variables. These can then be used in our Docker Compose deployment, but more on that later.

- name: Get Secrets
  uses: bitwarden/sm-action@v2
  with:
    access_token: ${{ secrets.BW_ACCESS_TOKEN }}
    secrets: |
      fc3a93f4-2a16-445b-b0c4-aeaf0102f0ff > SECRET_NAME_1
      bdbb16bc-0b9b-472e-99fa-af4101309076 > SECRET_NAME_2

You must store a Machine Account Access token in the GitHub Actions secrets and make the secrets accessible to that machine account.

Bitwarden Machine Account

Once created, you will be given the access token. You can then store it in your GitHub Actions secrets as BW_ACCESS_TOKEN, for instance.

Using the secrets in containers

In a Flask app, you can easily use Docker Compose secrets in your configuration objects by reading the secrets at startup. Of course, you can adapt this to whatever language and framework you are using.

# Simple utility function to read a secret
def _get_secret(name: str) -> str:
    with open(f"/run/secrets/{name}", "r") as file:
        return file.read()


class ProductionConfig(object):
    # Retrieve the secret and use it directly when instantiated
    SECRET_KEY = _get_secret("flask_secret_key")

    RECAPTCHA_PUBLIC_KEY = "6LcQ3GYqAAAAAC8CzGcAtO2AZ8wj1cyfYul8PNYc"
    RECAPTCHA_PRIVATE_KEY = _get_secret("recaptcha_private_key")

    # Create our database connection string dynamically using our secrets
    @property
    def SQLALCHEMY_DATABASE_URI(self):
        password = _get_secret("database_password")
        return f"postgresql+psycopg://dcs:{password}@postgres:5432/dcs"

Deploying the app

To deploy our app, we're going to need a production-ready compose.yaml file with lightweight Alpine Linux images, a restart policy and, a dedicated network.

Creating our Docker Compose file

The following Docker Compose file has two services: the app and a PostgreSQL database. Our secrets are defined as environment variables and are used by both containers.

services:
  app:
    image: ghcr.io/jamie-mh/dockercomposesecrets:latest
    container_name: dcs-app
    restart: always
    ports:
      - "8000:8000"
    networks:
      - dcs
    depends_on:
      - postgres
    environment:
      - FLASK_ENV=production
    # Use our defined secrets in the app container
    secrets:
      - flask_secret_key
      - recaptcha_private_key
      - database_password
      
  postgres:
    image: postgres:17-alpine
    container_name: dcs-postgres
    restart: always
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - dcs
    environment:
      - POSTGRES_USER=dcs
      - POSTGRES_DB=dcs
      # The Postgres image supports loading a password from Docker Compose secrets file
      # using a _FILE environment variable
      # https://github.com/docker-library/docs/blob/master/postgres/README.md#docker-secrets
      - POSTGRES_PASSWORD_FILE=/run/secrets/database_password
    # Make sure the secret is available to the Postgres container
    secrets:
      - database_password

# Populate our secrets from environment variables
# We'll fetch them from Bitwarden before deploying
secrets:
  flask_secret_key:
    environment: FLASK_SECRET_KEY
  recaptcha_private_key:
    environment: RECAPTCHA_PRIVATE_KEY
  database_password:
    environment: DATABASE_PASSWORD
    
networks:
  dcs:
    driver: bridge

volumes:
  db-data:
    driver: local

Creating a deployment user

Let's create a user for our deployment. The GitHub actions runner will log in as this user make the changes to our Docker system.

sudo adduser deployuser

Keep pressing enter when prompted for a password, as we won't be needing one. We'll log in as the user using an SSH key. You can skip the prompts and just use the default values.

info: Adding user `deployuser' ...
info: Selecting UID/GID from range 1000 to 59999 ...
info: Adding new group `deployuser' (1001) ...
info: Adding new user `deployuser' (1001) with group `deployuser (1001)' ...
info: Creating home directory `/home/deployuser' ...
info: Copying files from `/etc/skel' ...
New password: 
Retype new password: 
No password has been supplied.
New password: 
Retype new password: 
No password has been supplied.
New password: 
Retype new password: 
No password has been supplied.
passwd: Authentication token manipulation error
passwd: password unchanged
Try again? [y/N] N
Changing the user information for deployuser
Enter the new value, or press ENTER for the default
    Full Name []: 
    Room Number []: 
    Work Phone []: 
    Home Phone []: 
    Other []: 
Is the information correct? [Y/n] 
info: Adding new user `deployuser' to supplemental / extra groups `users' ...
info: Adding user `deployuser' to group `users' ...

Make sure to add the newly created user to the docker group so that it can make changes to our Docker system.

sudo usermod -aG docker deployuser

Generating an SSH key

To log in to our server remotely, we will need an SSH key. So, switch to the deployment user.

sudo su deployuser

From here, we can generate our key. You can use the default values and a passphrase is not necessary.

ssh-keygen

By default, this will save the public (id_ed25519.pub) and private (id_ed25519) keys to the .ssh directory.

Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/deployuser/.ssh/id_ed25519): 
Created directory '/home/deployuser/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/deployuser/.ssh/id_ed25519
Your public key has been saved in /home/deployuser/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:LbVVvBXUsm8ilJ0qOLQNsgPEPwB/fPaVgo3Gs80vp3k deployuser@jmh-public
The key's randomart image is:
+--[ED25519 256]--+
|  .o         .ooo|
|   .+. . +   oo o|
|   ..oo O + +o * |
|    ..++oO =o =  |
|     . =S=*. . . |
|      o +.o.o . o|
|       . ...o. o |
|           =E    |
|          o.     |
+----[SHA256]-----+

To allow logging in as this user, we need to add the public key to the authorized_keys file and adjust the file permissions accordingly.

cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Setting up the GitHub Actions workflow

The following workflow requires the following items to be defined in the GitHub Actions secrets:

The workflow will trigger on a tag that starts with 'v' (eg: v10) and will tag and deploy that commit of the repo.

name: Build and Deploy

on:
  push:
    tags:
      - "v*"

jobs:
  build:
    runs-on: ubuntu-latest

    # Build and push the Docker image for the current tag
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Login to GitHub Container Registry
        uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      # Tag the image with both 'latest' and the current Git tag
      - name: Build and push Docker image
        uses: docker/build-push-action@v5
        with:
          push: true
          context: .
          file: Dockerfile
          tags: ghcr.io/jamie-mh/dockercomposesecrets:latest,ghcr.io/jamie-mh/dockercomposesecrets:${{ github.ref_name }}

  # Deploy the app to the remote server
  deploy:
    runs-on: ubuntu-latest
    needs: build

    steps:
      - name: Checkout
        uses: actions/checkout@v4
        
      - name: Set up SSH
        run: |
          mkdir ~/.ssh
          echo "${{ secrets.DEPLOY_KEY }}" > ~/.ssh/deploy.key
          chmod 700 ~/.ssh
          chmod 600 ~/.ssh/deploy.key

          cat >>~/.ssh/config <<END
          Host deploy
              HostName ${{ secrets.DEPLOY_HOST }}
              User ${{ secrets.DEPLOY_USER }}
              IdentityFile ~/.ssh/deploy.key
              StrictHostKeyChecking no
              ControlMaster auto
              ControlPath ~/.ssh/control-%C
              ControlPersist yes
          END
         
      # Get the secrets by GUID and store them in environment variables
      - name: Get secrets
        uses: bitwarden/sm-action@v2
        with:
          access_token: ${{ secrets.BW_ACCESS_TOKEN }}
          secrets: |
            ce994dd3-ff6d-42f6-9f72-b20f011b6397 > FLASK_SECRET_KEY
            9d0855d5-7536-46e2-b2ce-b20f011b4e11 > RECAPTCHA_PRIVATE_KEY
            8e35b369-192a-408d-836e-b20f011b759e > DATABASE_PASSWORD

      # Log in as our GitHub user on the remote machine and deploy the app
      - name: Deploy
        run: |
          export DOCKER_HOST=ssh://deploy
          echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin

          # Deploy current tag
          sed -i 's/dockercomposesecrets:latest/dockercomposesecrets:${{ github.ref_name }}/' compose.yaml

          # Pull the latest image, restart the container and remove the old image
          docker compose -f compose.yaml pull app
          docker compose -f compose.yaml down app
          docker compose -f compose.yaml up --no-deps -d
          docker image prune -af

For this workflow to function, you need to grant image registry write permissions to the runner's GITHUB_TOKEN:

Workflow permissions

When it runs, the workflow will deploy both the app and Postgres containers to the remote machine. After the initial deployment, only the app container will be updated. The Postgres container will remain at the initial version, you can of course change this behaviour if required.

Conclusion

Docker Compose secrets are a useful feature of the container management tool. We can manage our secrets and perform a deployment externally without leaving our secrets in plain-text. Embedding secret values in container images is an obvious security risk for public images and could lead to leaking lots of sensitive data if your private registry is compromised.

By using a secret manager and making secrets available at runtime, we can completely avoid storing secret data in plain-text files or in environment variables.

For a real-world example, you can take inspiration from the Stratum website: a Flask app with a step to build static assets.


Got any comments or suggestions about this article? Feel free to contact me!

Latest posts

Deploy Docker Compose applications with zero downtime using GitHub Actions

This example demonstrates Blue-Green deployments using Docker Compose and GitHub Actions, deploy an app with zero downtime

July 21, 2024
Simple and privacy respecting web analytics using NGINX and GoAccess

GoAccess is a log file parser that can transform your NGINX access logs into an analytics dashboard. It provides website stats while retaining user privacy.

December 28, 2023
Building and deploying a simple PHP application using GitHub Actions

GitHub Actions is a versatile CI/CD platform that can be used for free. Here's how to build and deploy a PHP application using Composer for dependencies.

December 12, 2022