beginner30 minby DevFox

Layers, Caching and Build Context

Why your second build is instant and your third is not. How layers are cached, why instruction order decides your build time, and what .dockerignore is really for.

Docker
Performance

Build your image once and it takes a minute. Build it again without changing anything and it takes half a second. Change one line of your application and it takes a minute again — or half a second, depending entirely on which line you changed and where the COPY sits in your Dockerfile.

That difference is the build cache, and it is the most immediately useful thing in this course. By the end of this lesson you will know why a rebuild is fast or slow, how to order a Dockerfile so it is nearly always fast, and what your machine is quietly sending to the Docker daemon every time you type docker build.

Every instruction is a layer

You met layers as a fact about images: a stack of filesystem changes, presented to the container as one merged tree. Now for the part that matters while you work.

Each instruction in your Dockerfile produces one layer. FROM brings in the base image's layers, and every RUN, COPY and ADD after it adds another on top, recording exactly what changed on the filesystem.

dockerfile

FROM python:3.12-slim          # base layers
WORKDIR /app                   # metadata, no real layer
COPY requirements.txt .        # layer: one small file
RUN pip install -r requirements.txt   # layer: site-packages
COPY app.py .                  # layer: one small file
CMD ["python", "app.py"]       # metadata, no real layer

Instructions that only set metadata — WORKDIR, ENV, CMD, EXPOSE, LABEL — cost nothing on disk. The ones that touch files are the ones with weight.

The cache, and the rule that governs it

Docker keeps every layer it builds. On a rebuild, it walks your Dockerfile from the top and asks, for each instruction: have I built this exact instruction, on top of this exact parent layer, before? If yes, it reuses the stored layer instead of doing the work. That is a cache hit, and it appears in the output as CACHED:

text

 => [1/5] FROM docker.io/library/python:3.12-slim     0.0s
 => CACHED [2/5] WORKDIR /app                         0.0s
 => CACHED [3/5] COPY requirements.txt .              0.0s
 => CACHED [4/5] RUN pip install -r requirements.txt  0.0s
 => [5/5] COPY app.py .                               0.1s

Now the rule that decides everything:

The cache is a chain, not a set.

COPY requirements.txt — hit
Nothing about that file changed, so the stored layer is reused.
COPY app.py — miss
You edited one character. Different contents, different checksum, so this layer has to be built.
Everything after it — miss too
Even instructions whose text has not changed at all. A layer is defined as changes on top of a specific parent, and the parent is now new, so the layer has never existed before.

You changed one file. Three instructions rebuilt.

That single fact is why Dockerfile order is a performance decision rather than a matter of taste.

For RUN, "changed" means the command text changed — Docker does not inspect what the command would do. For COPY and ADD, it means the contents of the copied files changed, compared by checksum. Renaming a file, editing one character, or touching permissions all count.

Ordering for the cache

Here is the whole technique, as a pair. Same base image, same dependencies, same application — one line moved.

Bad — reinstalls every dependency whenever any source file changes:

dockerfile

FROM python:3.12-slim
WORKDIR /app
COPY . .
RUN pip install --no-cache-dir -r requirements.txt
CMD ["python", "app.py"]

Good — installs dependencies from the cache unless the dependency list itself changed:

dockerfile

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["python", "app.py"]

In the first version, COPY . . brings in your source code, so editing a single line changes that layer — and the pip install that follows it must then run again from scratch. You pay the full install on every code change, which for a real project is minutes each time and can be tens of thousands of packages in a JavaScript build.

The principle generalises to any language, and it is worth stating plainly: put the instructions that rarely change above the instructions that change constantly. Your dependency manifest changes weekly; your source code changes every few minutes.

dockerfile

# Node.js: the same shape
COPY package.json package-lock.json ./
RUN npm ci
COPY . .

dockerfile

# Go: modules first, then the code
COPY go.mod go.sum ./
RUN go mod download
COPY . .

Keeping a RUN line honest

Two more habits make cached layers behave, both about RUN.

The first is to combine steps that only make sense together, so no layer is left holding rubbish:

dockerfile

RUN apt-get update \
  && apt-get install -y --no-install-recommends curl \
  && rm -rf /var/lib/apt/lists/*

Splitting that into three RUN lines would create three layers, and the deletion in the third could not shrink the first two — a layer only ever adds to the stack, so a file added in one layer and deleted in a later one is still shipped inside the image. The combined form never writes the package lists into a layer at all.

The second is that apt-get update in its own layer is a trap: it caches, so weeks later you get a stale package index with a fresh install command and the build fails on a version that no longer exists. Chained to the install, as above, they always run together.

The build context

Now the argument you have been typing without thinking about:

bash

docker build -t hello-docker .

That . is the build context — the directory Docker packs up and sends to the daemon before the build starts. The daemon is a separate process, possibly inside a virtual machine, so it cannot reach into your filesystem. COPY copies out of the context, not out of your working directory.

Two consequences follow, and both bite people.

You cannot copy from outside the context. COPY ../shared/lib . fails, because ../shared was never sent. The fix is to build from a directory that contains everything you need and point at the Dockerfile explicitly:

bash

docker build -f service/Dockerfile -t api .

Everything in the context is transferred, whether you copy it or not. The first line of a build reports how much:

text

 => [internal] load build context
 => => transferring context: 412.83MB

Four hundred megabytes to build a small service means the context is full of things that have no business in a build: node_modules, a .venv, .git history, build output, log files. That transfer happens on every single build.

.dockerignore

The fix is a .dockerignore file beside your Dockerfile. It uses the same style as .gitignore and excludes paths from the context entirely:

text

.git
.gitignore
node_modules
__pycache__
*.pyc
.venv
venv
dist
build
.pytest_cache
.coverage
*.log
.env
.DS_Store
Dockerfile
.dockerignore
README.md

Three separate wins, and the middle one is the surprising one.

Speed

A 400 MB context becomes kilobytes.

That transfer happened on every single build, before any instruction ran.

Cache stability

The surprising one.

COPY . . hashes the files it copies. Anything under .git changes on every commit, so without a .dockerignore that layer — and everything after it — misses the cache constantly.

Safety

Secrets never enter a layer.

A .env file or a private key in your working directory is copied in by COPY . ., travels to every machine that pulls the image, and is visible to anyone who runs docker history.

Twenty lines of text, and all three come free

Working with and against the cache

Occasionally you want the cache out of the way — to prove a build works from nothing, or because something outside Docker's view changed:

bash

docker build --no-cache -t hello-docker .
docker build --pull -t hello-docker .      # refresh the base image

--pull is the more common need. FROM python:3.12-slim resolves to whatever your local copy of that tag is; if the tag has moved upstream, only --pull notices.

The cache is not free, either. Every stored layer occupies disk, and on a machine that builds often it accumulates fast:

bash

docker system df               # what the build cache is costing
docker builder prune           # reclaim it

Cheat sheet

dockerfile

# Order: rarely-changing things first
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .                  # changes weekly
RUN pip install --no-cache-dir -r requirements.txt
COPY . .                                 # changes constantly
CMD ["python", "app.py"]

# One RUN for steps that belong together
RUN apt-get update \
  && apt-get install -y --no-install-recommends curl \
  && rm -rf /var/lib/apt/lists/*

text

# .dockerignore — keep churn and secrets out of the context
.git
node_modules
__pycache__
.venv
dist
*.log
.env

bash

# Building
docker build -t app .                # context is "."
docker build -f svc/Dockerfile -t app .   # Dockerfile elsewhere
docker build --no-cache -t app .     # ignore every cached layer
docker build --pull -t app .         # re-pull the base image
docker build --progress=plain -t app .    # full, unfolded output

# Inspecting and reclaiming
docker history app                   # layers and their sizes
docker system df                     # images, containers, cache
docker builder prune                 # drop the build cache

Where to go next

Reorder one of your own Dockerfiles the way this lesson describes, add a .dockerignore, and time the rebuild after a one-line code change. The difference is usually large enough to feel.

Next comes the other half of the filesystem story: the container's writable layer disappears when the container does, so where does a database put its data? That is volumes. After it, networking — how one container reaches another — and then the two of them together in Compose.