intermediate30 minby DevFox

Visual and Snapshot Testing

Catching the change no assertion describes. Snapshot tests and their rot, visual diffs and their noise, and how to keep either from becoming a rubber stamp.

Testing

Some changes have no assertion that would catch them. A stylesheet edit shifts a button behind a header. A refactor drops a field from an API response nobody was asserting on. A component renders correctly and the spacing is now wrong everywhere.

Snapshot and visual testing answer these by recording what the output looks like and comparing later runs against the recording. It is a genuinely useful technique with a specific failure mode: because updating the recording is one keystroke, both kinds quietly degrade into a rubber stamp. By the end of this lesson you will know how to use them and how to stop that happening.

A different kind of assertion

Ordinary tests state the expected value in the test. Snapshot tests record it the first time and compare afterwards.

typescript

// stated: you wrote the expectation
expect(receipt.totalLine).toBe('Total: £40.50');

// recorded: the first run captured it
expect(receipt).toMatchSnapshot();

A stated expectation

Says why the value is correct.

expect(receipt.totalLine).toBe('Total: £40.50') encodes a decision somebody made. A failure names the rule that broke.

A recorded snapshot

Covers everything, explains nothing.

One line covers the entire structure, including fields you would never have thought to assert on.

And it tells you nothing about why any particular value is right — only that it used to be that.

One trade: coverage for precision

That trade is right in a specific situation: when the output is large, and what you care about is that it does not change unintentionally.

Snapshot testing

The first run writes a file; later runs compare against it.

typescript

test('the receipt has the expected shape', () => {
  const receipt = receiptFor(order({ total: 45.0, discount: 10 }));
  expect(receipt).toMatchSnapshot();
});

text

// __snapshots__/receipt.test.ts.snap
exports['the receipt has the expected shape 1'] = `
{
  "currency": "GBP",
  "discountLine": "Member discount: -£4.50",
  "lines": [...],
  "subtotalLine": "Subtotal: £45.00",
  "totalLine": "Total: £40.50",
}
`;

Change the code and the diff appears in the failure:

text

- "totalLine": "Total: £40.50",
+ "totalLine": "Total: £40.5",

That is the technique working: a formatting regression nobody had an assertion for, caught by a one-line test.

Inline snapshots are usually better than file-based ones, because the expected value sits in the test where a reviewer sees it:

typescript

expect(receipt.totalLine).toMatchInlineSnapshot(`"Total: £40.50"`);

Where snapshots fit well:

text

API response shapes            catches a dropped or renamed field
error message catalogues       all of them at once
generated output               SQL, HTML, a config file, a CSV
serialised data structures     large, and you care that it is stable

Where they do not:

text

anything with a rule           "the total is 10% off" deserves a real
                               assertion; a snapshot hides the rule
values that change             timestamps, ids, random values —
                               unless masked
large UI component trees       hundreds of lines nobody reads, and a
                               diff nobody can evaluate

The rubber stamp

Bad — a failing snapshot, updated without being read:

bash

npm test
# 14 snapshots failed
npx jest -u
git commit -m "update snapshots"

Good — the diff read, then updated deliberately:

bash

npm test
# 14 snapshots failed
# ...read the diff. 13 are the intended spacing change.
# ...the 14th changed "Total: £40.50" to "Total: £40.5". That is a bug.
# fix the rounding, then:
npx jest -u
git commit -m "fix currency formatting; update snapshots"

The bad sequence is the entire failure mode of snapshot testing, and it is the default behaviour under time pressure. Fourteen failures, one command to make them all pass, and the one real regression among them is committed as an approved expectation. The test now enforces the bug.

Three defences, and they are worth putting in place before the first snapshot.

Keep snapshots small
A twenty-line snapshot gets read; a four-hundred-line one does not. If it is too big to review, assert on the part you care about instead.
Never update in bulk
-u on a whole suite is the dangerous form. Update one test at a time when several fail.
Review snapshot files in code review
They are committed, so a reviewer sees the diff — and a pull request that changes fourteen snapshot files with no explanation deserves a question.

All three exist to keep a human between the failure and the new baseline

Visual regression testing

The same idea applied to pixels: capture a screenshot, compare it with the approved one, fail on a difference.

typescript

test('the contacts page looks right', async ({ page }) => {
  await page.goto('/contacts');
  await expect(page).toHaveScreenshot('contacts.png');
});

test('an empty contacts list looks right', async ({ page }) => {
  await page.goto('/contacts?empty=1');
  await expect(page.getByRole('main')).toHaveScreenshot('empty.png');
});

Playwright stores the baseline, and on failure produces three images — expected, actual, and a diff — which makes the review immediate.

bash

npx playwright test --update-snapshots     # after an intended change

This is the only technique that catches an entire class of defect: a layout that broke, an element that moved behind another, a font that failed to load, a component that renders at the wrong size, a dark-mode colour that became unreadable. No assertion describes any of them.

Why visual tests get abandoned

They are also the flakiest thing in testing, and the reasons are mechanical rather than mysterious.

text

font rendering    differs between operating systems and versions
antialiasing      a one-pixel edge difference on every glyph
animations        captured mid-transition
lazy images       captured before they load
scrollbars        present or absent depending on content
dynamic data      a timestamp, a name, a count
GPU differences   your laptop and the CI runner render differently

The last one is the killer, and it has one real answer: run visual tests in a container, so the rendering environment is identical everywhere.

bash

# Playwright's own image, matching the version you use
docker run --rm -v "$PWD":/work -w /work \
  mcr.microsoft.com/playwright:v1.49.0-jammy \
  npx playwright test --project=visual

Baselines generated on a developer's Mac and compared on a Linux runner will never match. Generating and comparing inside the same image is what makes the technique viable at all.

The rest are handled in configuration and in the test:

typescript

await expect(page).toHaveScreenshot('contacts.png', {
  animations: 'disabled', // no mid-transition captures
  caret: 'hide', // no blinking text cursor
  maxDiffPixelRatio: 0.01, // a tolerance for antialiasing
  mask: [page.getByTestId('last-updated')], // hide what varies
});

typescript

// wait for what the screenshot depends on
await page.waitForLoadState('networkidle');
await expect(page.getByRole('table')).toBeVisible();

Choosing what to capture

A whole-page screenshot of every page is the version that gets abandoned: every unrelated change fails everything, and no diff is reviewable.

Two better strategies:

Component-level snapshots. Capture each component in its states — default, loading, error, empty, with a very long label — in isolation. Storybook plus a visual tool does this well. A failure names one component, and the state matters as much as the component.

A small set of key pages, in a few viewports. Five to ten pages at mobile and desktop widths. Enough to catch a layout collapse; small enough that a redesign is a reviewable batch.

typescript

// the states that break, not just the happy one
for (const state of ['default', 'loading', 'error', 'empty']) {
  test(`the contact list in the ${state} state`, async ({ page }) => {
    await page.goto(`/contacts?state=${state}`);
    await expect(page.getByRole('main')).toHaveScreenshot(`${state}.png`);
  });
}

That loop is where visual testing earns its keep: the empty state and the error state are the ones nobody looks at manually, and they are the ones that break.

Cheat sheet

text

# Snapshot vs stated assertion
stated     expect(total).toBe('Total: £40.50')   precise, explains why
recorded   expect(receipt).toMatchSnapshot()     broad, explains nothing
#   use recorded when the output is LARGE and you care that it does
#   not change unintentionally

# Snapshots fit
API response shapes / error catalogues / generated output (SQL, HTML,
CSV) / large stable data structures
# Snapshots do not fit
anything with a RULE — assert the rule instead
values that vary — mask them or the snapshot rots
huge component trees — a diff nobody can evaluate
#   prefer INLINE snapshots: the expectation is in the test, visible
#   to a reviewer

# The failure mode: the rubber stamp
#   14 failures, `jest -u`, commit. The one real regression among
#   them is now the approved expectation.
defences
  keep snapshots small enough to read
  never bulk-update without reading the diff
  review snapshot files in code review — they are committed

typescript

// Visual regression
await expect(page).toHaveScreenshot('contacts.png', {
  animations: 'disabled',
  caret: 'hide',
  maxDiffPixelRatio: 0.01, // small — a large tolerance hides bugs
  mask: [page.getByTestId('last-updated')],
});

text

# Why visual tests flake, and the fixes
fonts, antialiasing, GPU  -> RUN IN A CONTAINER. This is the big one.
animations                -> animations: 'disabled'
lazy images, late data    -> wait for a condition first
scrollbars                -> fixed viewport, or capture a region
dynamic content           -> mask it
#   baselines made on a Mac and compared on Linux will never match

# What to capture
component states: default / loading / error / empty / long label
  ...plus a handful of key pages at mobile and desktop widths
#   NOT: every page, full-page. Nothing about that diff is reviewable.

# Updating a baseline is a deliberate act
npx playwright test --update-snapshots
npx jest -u                # one test at a time when several fail

Where to go next

The next lesson takes the machine-checkable part of accessibility into the pipeline — what an automated audit can catch, the much larger part it cannot, and how to wire the first in without claiming the second.

Before that, add one visual test to something you work on: a single component in its empty and error states, generated and compared inside a container. Those two states are where visual testing pays for itself fastest.