Build Docker image rootless in GitLab CI with Buildx

As a developer or DevOps person, a common use case is to build Docker images as part of your CI/CD pipeline. No matter whether you use Jenkins, Wookpecker CI, GitLab CI or any other of the many existing CI servers, a typical setup is to have the build runners themselves be Docker containers as well. You’ll end up having to build Docker in Docker. While the easiest and most straightforward way to implement this is to mount the Docker daemon into the container itself, it’s not necessarily the most secure way. Instead, you may want to build your images in a rootless fashion and without requiring a Docker daemon.

For a long time, Google’s Kaniko has been de-facto standard there, but at latest after the project has been discontinued, an alternative was needed. I had to migrate most of my CI pipelines and my particular requirements were the following.

Requirements:

  • Build must run inside Docker, but rootless and without access to the host daemon
  • Images shall be pushed to a private registry upon successful build
  • If a Git tag is given, use that tag (alongside latest) for the image as well, otherwise use the commit hash and branch name as tags
  • Intermediate layers shall be cached for efficiency and speed
  • Upstream images shall be pulled from a local mirror registry

Turns out this is harder than you would think. After a bit of research I found that using Docker Buildx is the recommended approach today – even though not necessarily the simplest.

After I worked out a solution, I thought it might be helpful to share it. So here is a GitLab CI pipeline that fulfills the above requirements.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
stages:
  - publish

# Run pipeline jobs for merge requests, tags, and pushes by default
workflow:
  rules:
    - if: $CI_COMMIT_TAG
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - if: $CI_MERGE_REQUEST_ID

build-docker-image:
  stage: publish
  image:
    name: moby/buildkit:rootless
    entrypoint: [ "" ]

  before_script:
    - mkdir -p ~/.docker
    
    # Log in to private GitLab registry
    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > ~/.docker/config.json

    # Retrieve commit hash / tag from repo and (and current date)
    - COMMIT_REF=$(echo $COMMIT_REF | sed -e 's/.*\///')
    - BUILD_DATE=$(date -Iseconds)

    # Configure internal registry mirror to use (optional)
    - mkdir -p /home/user/.config/buildkit
    - |
      cat > /home/user/.config/buildkit/buildkitd.toml <<EOF
      [registry."docker.io"]
        mirrors = ["docker-mirror01.example.org"]
      EOF

    # Start buildkit daemon inside the container 
    - rootlesskit buildkitd --oci-worker-no-process-sandbox &
    - export BUILDKIT_HOST=unix:///run/user/$(id -u)/buildkit/buildkitd.sock
    - while ! buildctl debug workers; do sleep 1; done

  script:
    # Build image, tag it with commit hash, branch name and - if given - commit tag and `latest`
    # Push image to same GitLab project's container registry
    - |
      buildctl build \
        --frontend dockerfile.v0 \
        --local context=$CI_PROJECT_DIR \
        --local dockerfile=$CI_PROJECT_DIR \
        --opt filename=Dockerfile \
        --opt build-arg:VERSION=$VERSION \
        --opt build-arg:BUILD_DATE=$BUILD_DATE \
        --output type=image,\"name=$CI_REGISTRY_IMAGE:${CI_COMMIT_TAG:-$COMMIT_REF},$CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA${CI_COMMIT_TAG:+,${CI_REGISTRY_IMAGE}:latest}\",push=true \
        --export-cache type=registry,ref=$CI_REGISTRY_IMAGE:buildcache \
        --import-cache type=registry,ref=$CI_REGISTRY_IMAGE:buildcache

Hope that helps! 🤓

Comments