Table of Contents
In many cases, secrets are stored in environment variables or configuration files embedded in Docker images. The downside to this approach is that they can be extracted from an image or viewed from a running container with docker inspect.
In this example, I will demonstrate how to securely store your secrets. That is, not in a configuration file, .env file or anywhere in plain-text — let's keep them out of our images and use Docker Compose secrets to inject them at runtime.
To illustrate this, I've developed an example project using Flask and PostgreSQL: a simple web app where users can post comments and read comments. There are secrets such as a database password shared by the Flask and Postgres containers, a Flask secret key for CSRF tokens and, a private key for reCAPTCHA. The source for the example is available on GitHub and you can view the deployed application here.
Using Docker Compose secrets
In Docker Compose, secrets are defined in the compose.yaml file. They're made available to containers at runtime by creating a read-only mount at /run/secrets/[mysecret] and can be set from either a file or environment variable.
Here's how you declare secrets in a Compose file. The secrets top-level element contains a list of secrets by name either loaded from file or environment. You must then specify which secrets are required by which service.
services:
myapp:
...
secrets:
- my_secret
secrets:
my_secret:
file: ./my_secret.txt
As mentioned previously, the secrets are made available to running containers in the /run/secrets/ directory. For example, my_secret can be read like a regular text file at /run/secrets/my_secret.
Storing our secrets using Bitwarden
There are a few options for storing secrets before they are used by containers, with some advantages and disadvantages to each. The preferred option for storing application secrets is to use a secrets manager such as Bitwarden Secrets Manager, 1Password, LastPass, Doppler, etc. This makes it easy to view and update secrets as required.
In this example we're going to use Bitwarden Secrets Manager since it integrates nicely with GitHub Actions.
Storing the secrets
In Bitwarden Secrets Manager, create a project for your app and create all the secrets you need within that. Each secret has its own GUID that's used to retrieve it in our GitHub Actions workflow.
Retrieving the secrets
With the Bitwarden Secrets Manager action, you can retrieve secrets by GUID and store them in pipeline environment variables. These can then be used in our Docker Compose deployment, but more on that later.
- name: Get Secrets
uses: bitwarden/sm-action@v2
with:
access_token: ${{ secrets.BW_ACCESS_TOKEN }}
secrets: |
fc3a93f4-2a16-445b-b0c4-aeaf0102f0ff > SECRET_NAME_1
bdbb16bc-0b9b-472e-99fa-af4101309076 > SECRET_NAME_2
You must store a Machine Account Access token in the GitHub Actions secrets and make the secrets accessible to that machine account.
Once created, you will be given the access token. You can then store it in your GitHub Actions secrets as BW_ACCESS_TOKEN, for instance.
Using the secrets in containers
In a Flask app, you can easily use Docker Compose secrets in your configuration objects by reading the secrets at startup. Of course, you can adapt this to whatever language and framework you are using.
# Simple utility function to read a secret
def _get_secret(name: str) -> str:
with open(f"/run/secrets/{name}", "r") as file:
return file.read()
class ProductionConfig(object):
# Retrieve the secret and use it directly when instantiated
SECRET_KEY = _get_secret("flask_secret_key")
RECAPTCHA_PUBLIC_KEY = "6LcQ3GYqAAAAAC8CzGcAtO2AZ8wj1cyfYul8PNYc"
RECAPTCHA_PRIVATE_KEY = _get_secret("recaptcha_private_key")
# Create our database connection string dynamically using our secrets
@property
def SQLALCHEMY_DATABASE_URI(self):
password = _get_secret("database_password")
return f"postgresql+psycopg://dcs:{password}@postgres:5432/dcs"
Deploying the app
To deploy our app, we're going to need a production-ready compose.yaml file with lightweight Alpine Linux images, a restart policy and, a dedicated network.
Creating our Docker Compose file
The following Docker Compose file has two services: the app and a PostgreSQL database. Our secrets are defined as environment variables and are used by both containers.
services:
app:
image: ghcr.io/jamie-mh/dockercomposesecrets:latest
container_name: dcs-app
restart: always
ports:
- "8000:8000"
networks:
- dcs
depends_on:
- postgres
environment:
- FLASK_ENV=production
# Use our defined secrets in the app container
secrets:
- flask_secret_key
- recaptcha_private_key
- database_password
postgres:
image: postgres:17-alpine
container_name: dcs-postgres
restart: always
volumes:
- db-data:/var/lib/postgresql/data
networks:
- dcs
environment:
- POSTGRES_USER=dcs
- POSTGRES_DB=dcs
# The Postgres image supports loading a password from Docker Compose secrets file
# using a _FILE environment variable
# https://github.com/docker-library/docs/blob/master/postgres/README.md#docker-secrets
- POSTGRES_PASSWORD_FILE=/run/secrets/database_password
# Make sure the secret is available to the Postgres container
secrets:
- database_password
# Populate our secrets from environment variables
# We'll fetch them from Bitwarden before deploying
secrets:
flask_secret_key:
environment: FLASK_SECRET_KEY
recaptcha_private_key:
environment: RECAPTCHA_PRIVATE_KEY
database_password:
environment: DATABASE_PASSWORD
networks:
dcs:
driver: bridge
volumes:
db-data:
driver: local
Creating a deployment user
Let's create a user for our deployment. The GitHub actions runner will log in as this user make the changes to our Docker system.
sudo adduser deployuser
Keep pressing enter when prompted for a password, as we won't be needing one. We'll log in as the user using an SSH key. You can skip the prompts and just use the default values.
info: Adding user `deployuser' ...
info: Selecting UID/GID from range 1000 to 59999 ...
info: Adding new group `deployuser' (1001) ...
info: Adding new user `deployuser' (1001) with group `deployuser (1001)' ...
info: Creating home directory `/home/deployuser' ...
info: Copying files from `/etc/skel' ...
New password:
Retype new password:
No password has been supplied.
New password:
Retype new password:
No password has been supplied.
New password:
Retype new password:
No password has been supplied.
passwd: Authentication token manipulation error
passwd: password unchanged
Try again? [y/N] N
Changing the user information for deployuser
Enter the new value, or press ENTER for the default
Full Name []:
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n]
info: Adding new user `deployuser' to supplemental / extra groups `users' ...
info: Adding user `deployuser' to group `users' ...
Make sure to add the newly created user to the docker group so that it can make changes to our Docker system.
sudo usermod -aG docker deployuser
Generating an SSH key
To log in to our server remotely, we will need an SSH key. So, switch to the deployment user.
sudo su deployuser
From here, we can generate our key. You can use the default values and a passphrase is not necessary.
ssh-keygen
By default, this will save the public (id_ed25519.pub) and private (id_ed25519) keys to the .ssh directory.
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/deployuser/.ssh/id_ed25519):
Created directory '/home/deployuser/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/deployuser/.ssh/id_ed25519
Your public key has been saved in /home/deployuser/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:LbVVvBXUsm8ilJ0qOLQNsgPEPwB/fPaVgo3Gs80vp3k deployuser@jmh-public
The key's randomart image is:
+--[ED25519 256]--+
| .o .ooo|
| .+. . + oo o|
| ..oo O + +o * |
| ..++oO =o = |
| . =S=*. . . |
| o +.o.o . o|
| . ...o. o |
| =E |
| o. |
+----[SHA256]-----+
To allow logging in as this user, we need to add the public key to the authorized_keys file and adjust the file permissions accordingly.
cat ~/.ssh/id_ed25519.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
Setting up the GitHub Actions workflow
The following workflow requires the following items to be defined in the GitHub Actions secrets:
DEPLOY_KEY: the SSH private key generated previously (.ssh/id_ed25519)DEPLOY_HOST: the domain or IP address pointing to the remote serverDEPLOY_USER: the name of the deployment user on the serverBW_ACCESS_TOKEN: the Machine Account Access Token for our Bitwarden secrets
The workflow will trigger on a tag that starts with 'v' (eg: v10) and will tag and deploy that commit of the repo.
name: Build and Deploy
on:
push:
tags:
- "v*"
jobs:
build:
runs-on: ubuntu-latest
# Build and push the Docker image for the current tag
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
# Tag the image with both 'latest' and the current Git tag
- name: Build and push Docker image
uses: docker/build-push-action@v5
with:
push: true
context: .
file: Dockerfile
tags: ghcr.io/jamie-mh/dockercomposesecrets:latest,ghcr.io/jamie-mh/dockercomposesecrets:${{ github.ref_name }}
# Deploy the app to the remote server
deploy:
runs-on: ubuntu-latest
needs: build
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Set up SSH
run: |
mkdir ~/.ssh
echo "${{ secrets.DEPLOY_KEY }}" > ~/.ssh/deploy.key
chmod 700 ~/.ssh
chmod 600 ~/.ssh/deploy.key
cat >>~/.ssh/config <<END
Host deploy
HostName ${{ secrets.DEPLOY_HOST }}
User ${{ secrets.DEPLOY_USER }}
IdentityFile ~/.ssh/deploy.key
StrictHostKeyChecking no
ControlMaster auto
ControlPath ~/.ssh/control-%C
ControlPersist yes
END
# Get the secrets by GUID and store them in environment variables
- name: Get secrets
uses: bitwarden/sm-action@v2
with:
access_token: ${{ secrets.BW_ACCESS_TOKEN }}
secrets: |
ce994dd3-ff6d-42f6-9f72-b20f011b6397 > FLASK_SECRET_KEY
9d0855d5-7536-46e2-b2ce-b20f011b4e11 > RECAPTCHA_PRIVATE_KEY
8e35b369-192a-408d-836e-b20f011b759e > DATABASE_PASSWORD
# Log in as our GitHub user on the remote machine and deploy the app
- name: Deploy
run: |
export DOCKER_HOST=ssh://deploy
echo ${{ secrets.GITHUB_TOKEN }} | docker login ghcr.io -u ${{ github.actor }} --password-stdin
# Deploy current tag
sed -i 's/dockercomposesecrets:latest/dockercomposesecrets:${{ github.ref_name }}/' compose.yaml
# Pull the latest image, restart the container and remove the old image
docker compose -f compose.yaml pull app
docker compose -f compose.yaml down app
docker compose -f compose.yaml up --no-deps -d
docker image prune -af
For this workflow to function, you need to grant image registry write permissions to the runner's GITHUB_TOKEN:
When it runs, the workflow will deploy both the app and Postgres containers to the remote machine. After the initial deployment, only the app container will be updated. The Postgres container will remain at the initial version, you can of course change this behaviour if required.
Conclusion
Docker Compose secrets are a useful feature of the container management tool. We can manage our secrets and perform a deployment externally without leaving our secrets in plain-text. Embedding secret values in container images is an obvious security risk for public images and could lead to leaking lots of sensitive data if your private registry is compromised.
By using a secret manager and making secrets available at runtime, we can completely avoid storing secret data in plain-text files or in environment variables.
For a real-world example, you can take inspiration from the Stratum website: a Flask app with a step to build static assets.
