Using BuildKit Inline Cache with a Remote Registry

Every CI job gets a fresh runner with an empty Docker layer store, so a build that takes forty seconds on a developer laptop takes six minutes in CI — reinstalling the same dependencies into the same layers, every run. Registry-backed BuildKit cache fixes that by treating the registry as the shared layer store.

When to use this pattern

  • Your CI builds container images on ephemeral runners with no persistent Docker state.
  • The image is multi-stage, and the expensive stage (dependency install or compile) rarely changes.
  • You already push to a registry the runners can read and write.

Prerequisites

Where the cache actually lives

Without an import source, BuildKit starts from nothing on every runner. The two export modes differ in what they publish and therefore in how much a later build can resume.

Inline Cache vs Registry Cache with mode=max A four-stage Dockerfile is shown as a column of stages: base, dependency install, build, and runtime. Inline cache records only the runtime stage's layers, so a later build resumes late. Registry cache in max mode records all four stages, so a later build resumes at the first genuinely changed step. DOCKERFILE STAGES 1 · base image 2 · npm ci (2m 40s) 3 · npm run build (1m 50s) 4 · runtime image INLINE CACHE recorded stages 1–3 not recorded a cold build repeats 4m 30s REGISTRY CACHE, mode=max recorded recorded — the expensive one recorded recorded Inline is free and simple; max mode costs a separate manifest and is the only one that saves the install stage. For a multi-stage frontend image, the install stage is 60–70% of the build — which is exactly what inline mode cannot record. Use inline only when the image is single-stage or the build stage is trivial.

Complete working example

# Dockerfile — ordered so the volatile COPY comes as late as possible.
# syntax=docker/dockerfile:1.7
FROM node:20-alpine AS deps
WORKDIR /app
# Only the manifests: this layer is invalidated by a dependency change, nothing else.
COPY package.json package-lock.json ./
# A cache mount keeps the npm store between builds even when this layer is rebuilt.
RUN --mount=type=cache,target=/root/.npm \
    npm ci --ignore-scripts --prefer-offline

FROM deps AS build
# Source code changes on every commit — everything above this line stays cached.
COPY . .
RUN npm run build

FROM node:20-alpine AS runtime
WORKDIR /app
ENV NODE_ENV=production
COPY --from=build /app/dist ./dist
COPY --from=deps /app/node_modules ./node_modules
CMD ["node", "dist/server.js"]
# .github/workflows/image.yml
name: Image
on:
  push:
    branches: [main]
  pull_request:

permissions:
  contents: read
  packages: write

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ghcr.io
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/build-push-action@v6
        with:
          context: .
          push: ${{ github.event_name != 'pull_request' }}
          tags: ghcr.io/acme/shop:${{ github.sha }}
          # Import the branch cache first, then main's as a fallback: a new branch
          # starts warm from main instead of from nothing.
          cache-from: |
            type=registry,ref=ghcr.io/acme/shop:buildcache-${{ github.ref_name }}
            type=registry,ref=ghcr.io/acme/shop:buildcache-main
          # Only trusted runs write cache — a pull request must not be able to
          # publish layers that later builds will trust.
          cache-to: ${{ github.event_name == 'pull_request' && '' ||
                        format('type=registry,ref=ghcr.io/acme/shop:buildcache-{0},mode=max',
                               github.ref_name) }}

Step-by-step walkthrough

mode=max is the setting that matters. The default mode=min exports only the layers present in the final image, which for a multi-stage build excludes the dependency-install stage entirely — the one worth caching. max exports every intermediate stage and costs a larger cache manifest, typically a few hundred megabytes.

Two cache-from sources, ordered. BuildKit tries each in order and uses the first that matches. Putting the branch’s own cache first gives an incremental build on repeated pushes; falling back to main means a freshly created branch starts warm rather than cold. This mirrors the fallback ordering in using restore keys for safe cache fallbacks.

Pull requests import but do not export. A build cache is a channel through which one job’s output becomes another job’s input. Untrusted pull-request code that can write to it can publish a poisoned layer under a legitimate cache key, which is the same failure explored in fixing cache poisoning issues in distributed CI runners.

How the Two cache-from Sources Resolve Three scenarios. A repeat push on an existing branch matches the branch cache and resumes at the changed layer. A first push on a new branch misses the branch cache, falls back to the main cache and still skips the dependency install. A build where neither exists starts from scratch. SCENARIO RESOLUTION COLD-START COST repeat push, same branch buildcache-feature-x → hit on source layer ~40s first push, new branch branch miss → falls back to buildcache-main dependency stage still reused ~2m 05s no cache published yet both miss → full build, then exports cache ~6m 20s, once Ordering the sources branch-first, main-second is what makes the middle row cheap instead of cold.

Cache mounts and layer cache solve different problems. --mount=type=cache,target=/root/.npm persists the package manager’s own store across builds on the same BuildKit instance. Registry cache persists finished layers across machines. Together they mean that even when a lockfile change forces the install layer to rebuild, the packages are already downloaded.

Dockerfile order is still the dominant factor. No cache configuration recovers from a COPY . . before npm ci, because the copy changes on every commit and invalidates everything after it. Get the ordering right first; registry cache multiplies the benefit but cannot create it.

Verification

# 1. The build log tells the truth: count CACHED steps.
docker buildx build --progress=plain \
  --cache-from type=registry,ref=ghcr.io/acme/shop:buildcache-main . 2>&1 \
  | grep -cE '^#[0-9]+ CACHED'

Expected on a source-only change — the dependency stage reused, the build stage rebuilt:

#7 [deps 3/3] RUN npm ci --ignore-scripts --prefer-offline
#7 CACHED
#9 [build 2/2] RUN npm run build
#9 DONE 108.4s
# 2. The cache manifest exists and is the size you expect.
docker buildx imagetools inspect ghcr.io/acme/shop:buildcache-main \
  --format '{{json .Manifest}}' | jq '{mediaType, layers: (.layers | length)}'

# 3. Prove a cold runner benefits: wipe local state and rebuild.
docker buildx prune -af
time docker buildx build --cache-from type=registry,ref=ghcr.io/acme/shop:buildcache-main .

The third command is the honest test. A build that is fast because your laptop already has the layers proves nothing about CI; pruning first reproduces what an ephemeral runner sees.

Reading the cost side honestly

Registry cache is not free, and the trade is worth stating before you enable mode=max across every image.

Each export writes a cache manifest and its layer blobs to the registry — for a typical Node application image, 300–600 MB, rewritten on every build that changes anything. That is registry storage you pay for and network transfer on both ends. A build that imports 500 MB of cache to skip 160 seconds of npm ci is a clear win on a runner with fast registry access; the same build on a runner in a different region can spend longer downloading cache than it saved.

The number to check is import time against the time saved. BuildKit prints both — the cache import appears as its own step in the plain progress output. If importing takes more than about a third of what it saves, either move the cache closer to the runners or drop back to caching only the dependency stage by exporting mode=min from a build that has deps as its final target.

Build Time by Cache Configuration on a Cold Runner Three stacked bars. Without cache, the build spends 160 seconds on dependency install, 110 on the application build and 110 on other steps, totalling six minutes twenty. Inline cache saves only the final stage. Registry cache with max mode reuses the dependency stage, leaving 22 seconds of cache import plus the application build, for two minutes five. import npm ci npm run build other steps no cache 6m 20s inline cache 5m 10s — the install stage was never recorded registry, mode=max 2m 05s — 22s import buys back 160s of install

Common pitfalls

Using mode=max on a registry that does not support OCI artifacts. Docker Hub’s free tier historically rejects the separate cache manifest, and the export fails late in the build with a push error. Test the export path once before relying on it.

Letting the cache tag accumulate forever. Cache manifests are ordinary tags and will grow without a lifecycle rule. Expire buildcache-* tags older than 30 days.

Caching a build that embeds a timestamp. If a layer’s command output changes every run — a build ID, a date, a random seed — that layer and everything after it can never be reused. Move it into an environment variable read at runtime instead, following the determinism argument in reducing Docker image size for frontend containers.

← Back to Docker Layer Caching for Full-Stack Applications