Why Your Docker Build Reinstalls Every Dependency Every Time

A build that takes eight minutes when it should take forty seconds is almost always a cache ordering problem. The dependency install runs every time because something above it changed, and the thing above it changes on every commit.

Understanding the cache key makes this predictable rather than mysterious.

How the key is computed

Each instruction produces a layer, and each layer's cache key includes the key of the layer before it. Change layer three and layers four onward are invalidated regardless of whether their own content changed. The chain is the whole mechanism.

What goes into the key depends on the instruction:

The RUN case is the one that surprises people. RUN apt-get update is cached on the string apt-get update. Docker has no idea the upstream package index changed. It will happily reuse a six week old layer, which is why update and install must be in the same instruction.

# broken: update is cached, install gets stale indexes
RUN apt-get update
RUN apt-get install -y curl

# correct
RUN apt-get update && apt-get install -y curl \
    && rm -rf /var/lib/apt/lists/*

Tested on Docker 27.5, BuildKit 0.20.

The ordering rule

Put instructions in order of how often they change, least frequently first.

# bad: source copied before install, so every commit reinstalls
FROM node:22-slim
WORKDIR /app
COPY . .
RUN npm ci
RUN npm run build

COPY . . includes every source file. Any edit changes that layer's key, npm ci is invalidated, and you reinstall the entire dependency tree to change one line of a component.

# good: manifests first, then install, then source
FROM node:22-slim
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

Now npm ci is only invalidated when the manifests change, which is when dependencies actually changed. Source edits invalidate only the last two layers.

The same shape applies everywhere: requirements.txt before source in Python, go.mod and go.sum before source in Go, Gemfile and Gemfile.lock in Ruby, pom.xml in Java.

The invisible cache buster

Even with correct ordering, COPY . . can invalidate on files you do not care about. The COPY key includes file metadata, so a rebuilt node_modules, a fresh .git directory, or a mutated build artefact all change it.

# .dockerignore
.git
node_modules
dist
coverage
*.log
.env*
.DS_Store
**/__pycache__

.dockerignore does two jobs: it shrinks the build context sent to the daemon, which is often the slowest part of a build on a large repository, and it stops irrelevant files from breaking the cache.

Check what you are actually sending:

# the first line of build output reports context size
docker build . 2>&1 | head -3

A context of several hundred megabytes means .dockerignore is missing or wrong. I have seen a 1.2 GB context that was entirely .git history.

ARG placement matters more than people expect

An ARG invalidates every layer after its first use. Put a build argument that changes per build near the top and you have disabled caching for the entire file.

# bad: GIT_SHA changes every commit, invalidating install
FROM node:22-slim
ARG GIT_SHA
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .

# good: declared where it is needed, after the expensive layers
FROM node:22-slim
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
ARG GIT_SHA
ENV GIT_SHA=$GIT_SHA
RUN npm run build

Cache mounts for package managers

BuildKit can mount a persistent cache directory that survives across builds and is not part of any layer. This is the single biggest improvement available for dependency heavy builds.

# syntax=docker/dockerfile:1
FROM node:22-slim
WORKDIR /app
COPY package.json package-lock.json ./
RUN --mount=type=cache,target=/root/.npm \
    npm ci

Even when the layer is invalidated, npm reuses the downloaded tarballs and only relinks. A full reinstall drops from minutes to seconds.

# pip
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# go
RUN --mount=type=cache,target=/go/pkg/mod \
    --mount=type=cache,target=/root/.cache/go-build \
    go build -o /out/app ./cmd/app

# apt, needs the default cleanup disabled
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked \
    apt-get update && apt-get install -y curl

Why CI never hits the cache

The most common complaint. Local builds are fast and CI rebuilds everything, because each CI run starts on a clean runner with no local layer store.

Export and import the cache explicitly:

docker buildx build \
  --cache-from type=registry,ref=registry.example.com/app:buildcache \
  --cache-to   type=registry,ref=registry.example.com/app:buildcache,mode=max \
  -t registry.example.com/app:$GIT_SHA \
  --push .

mode=max exports intermediate layers as well as the final one, which is what makes the cache useful for a partial rebuild. The default mode=min only exports the final layers and rarely helps.

There is a subtlety: cache lookups fail across architectures. A build on arm64 will not reuse layers built on amd64, which is the same underlying mismatch that produces exec format errors. Scope the cache ref per platform if you build both.

Note also that a cache that produces a wrong result is worse than no cache. If your CI cache is keyed loosely enough to reuse stale artefacts, you get builds that pass on nothing real, which I covered in the CI cache that made builds slower and wrong.

Seeing what actually happened

# per step timing and whether each was CACHED
docker build --progress=plain --no-cache=false . 2>&1 | grep -E "^#[0-9]+ (CACHED|DONE)"

# layer sizes, to find what is worth optimising
docker history myimage:latest --no-trunc --format "{{.Size}}\t{{.CreatedBy}}"

The first CACHED that stops appearing is your invalidation point. Everything below it is being rebuilt because of that step, so that is the one to fix.

Prevention

• Order instructions from least to most frequently changing. This one rule fixes most builds.
• Maintain .dockerignore as carefully as .gitignore. Check the reported context size occasionally.
• Never split apt-get update from apt-get install.
• Declare ARG as late as possible.
• Use BuildKit cache mounts for every package manager, and registry cache export in CI.
• Use multi stage builds so build tooling never reaches the runtime image. Smaller images pull faster and carry less to patch.