Docker and symlinks

Question

Docker doesn’t support symlinking files outside the build context.

Here are some different methods for using a shared file in a container:

Build Time

Copy from a config image (Docker buildkit)

Recent versions of Docker allow RUN steps to bind mount from a named image or previous build stage with the --mount=type=bind,target=/dir,source=/dir,from=image-or-stage-name

Create a Dockerfile for the base me/worker-config image that includes the shared config/files.

FROM scratch
COPY config.json /config.json

Build and tag the config image me/worker-config

docker build -t me/worker-config:latest .

Mount the me/worker-config image during the real build

RUN --mount=type=bind,target=/worker-config,source=/,from=me/worker-config:latest \
    cp /worker-config/config.json /app/config.json;

Share a base image

Create a Dockerfile for the base me/worker-config image that includes the shared config/files.

COPY config.json /config.json

Build and tag the image me/worker-config

docker build -t me/worker-config:latest .

Source the base me/worker-config image for all your worker Dockerfiles

FROM me/worker-config:latest

Build script

Use a script to push the common config to each of your worker containers.

./build worker-n

#!/bin/sh
set -uex 
rundir=$(readlink -f "${0%/*}")
container=$(shift)
cd "$rundir/$container"
cp ../config/config.json ./config-docker.json
docker build "$@" .

Build from URL

Pull the config from a common URL for all worker-n builds.

ADD http://somehost/config.json /

Increase the scope of the image build context

Include the symlink target files in the build context by building from a parent directory that includes both the shared files and specific container files.

cd ..
docker build -f worker-a/Dockerfile .

All the source paths you reference in a Dockerfile must also change to match the new build context:

COPY workerathing /app

becomes

COPY worker-a/workerathing /app

Using this method can make all build contexts large if you have one large build context, as they all become shared. It can slow down builds, especially to remote Docker build servers. Note that only the .dockerignore file from the base of the build context is referenced.

Alternate build that can mount volumes

Other projects that strive for Dockerfile compatibility may support volumes at build time. For example a podman build / buildah support a --volume option to bind mount files from the host into a build container.

podman build --volume /project/config:/worker-config:ro,Z -t me/worker-a .

Then the build can reference the mounted volume

COPY /worker-config/config.json /app

Run time

Mount a config directory from a named volume

Volumes like this only work as directories, so you can’t specify a file like you could when mounting a file from the host to container.

docker volume create --name=worker-cfg-vol
docker run -v worker-cfg-vol:/config worker-config cp config.json /config

docker run -v worker-cfg-vol:/config:/config worker-a

Mount config directory from data container

Again, directories only as it’s basically the same as above. This will automatically copy files from the destination directory into the newly created shared volume though.

docker create --name wcc -v /config worker-config /bin/true
docker run --volumes-from wcc worker-a

Mount config file from host at runtime

docker run -v /app/config/config.json:/config.json worker-a