Accessibility Checks in the Pipeline
What an automated audit can catch, the much larger part it cannot, and how to wire the machine-checkable half into CI without pretending the job is done.
What an automated audit can catch, the much larger part it cannot, and how to wire the machine-checkable half into CI without pretending the job is done.
An automated accessibility scan is one of the cheapest checks you can add to a pipeline, and one of the easiest to over-claim. It reliably finds a missing form label and a contrast failure. It cannot tell you whether the tab order makes sense, whether an alt text is meaningful, or whether an error is announced to somebody who cannot see it.
By the end of this lesson you will have the automatable part running on every change, and — just as importantly — a clear account of what remains, so a green pipeline is not mistaken for an accessible product.
The commonly cited figure is that automated tooling finds roughly a third of accessibility issues. The exact proportion is arguable; the split is not, and it follows from what a machine can evaluate.
a scanner CAN detect
an input with no associated label
an image with no alt attribute
text below the contrast threshold
a button with no accessible name
a page with no lang attribute
duplicate ids, invalid ARIA attributes
a heading level that skips
a form control outside a form landmark
a scanner CANNOT detect
an alt text that says "image1.png"
a tab order that jumps around the page
a focus indicator that is invisible on this background
an error announced nowhere a screen reader will reach
a modal that does not trap focus
a control that works with a mouse and not a keyboard
content that only makes sense if you can see the layout
a time limit nobody can meetA rule, applied to the DOM.
Is there a label? Is there an alt? Is the contrast ratio
above the threshold? Does this heading level skip?
Mechanical, cheap to run on every commit, and roughly a third of real issues.
Judgement about meaning and experience.
An alt that says "image1.png" passes. A tab order that jumps
around the page passes. A modal that does not trap focus
passes.
Every one of those is a person unable to use your product, and the scanner is green.
axe-core is the standard engine, and it plugs into whatever you already
run. With Playwright:
A failure is unusually readable — each violation names the rule, the elements, the impact, and a link to how to fix it:
The withTags call is worth being deliberate about. It selects which
standard you are checking against — wcag2aa and wcag21aa are the usual
target, since AA is what most legislation references. Adding
best-practice catches more and includes opinions you may not want to
block a build on.
For unit-level component tests, jest-axe and vitest-axe do the same
against rendered markup, which is faster and catches problems before a
page exists:
The most common gap in an accessibility suite is scanning one static view of each page. Most accessibility failures are in states.
The states worth scanning: error, loading, empty, modal open, menu expanded, and dark mode. Contrast in particular is frequently fine in one theme and failing in the other, and nobody notices because the scan ran in the default.
include scopes the scan to a region, which is how you get a useful result
from a dialog rather than a page-wide report.
Bad — turn it on, get 340 violations, disable it:
Good — block new violations, and burn down the existing ones:
The bad version is the usual reason a team has no accessibility testing: a zero-tolerance check on a product with existing problems fails immediately, cannot be fixed this sprint, and gets commented out — after which nothing is checked and new violations arrive freely.
The good version draws a line. Existing violations are catalogued and excluded explicitly, so they are visible and countable rather than forgotten; everything else blocks. Then the exclusion list is a backlog, and it shrinks.
Two rules keep that honest: the list lives in a file with an owner and a rough date per entry, and nothing is added to it — a new violation is fixed, not appended.
The other two thirds cannot be automated, and pretending otherwise is the mistake this lesson exists to prevent. What can be done is making them routine rather than heroic.
Keyboard only, once per feature. Put the mouse aside. Tab through the new work: is everything reachable, is the focus always visible, is the order sensible, does Esc close what it should, is focus trapped inside a modal and returned afterwards? Five minutes, and it finds more than the scanner does.
A screen reader, once per significant feature. VoiceOver on macOS (Cmd + F5), NVDA on Windows — both free. Try one flow. Uncomfortable at first and remarkably informative: an error that is never announced, a button read as "button", a table with no headers.
Zoom to 200%, and 400%. Text should reflow rather than clip. This is a WCAG requirement and takes ten seconds.
Check that colour is never the only signal. A red border with no message or icon conveys nothing to a colour-blind user.
A workable arrangement, which mirrors the CI gates:
That last line is worth arguing for. An audit by someone who relies on a screen reader finds things no checklist contains, and it is the only way to learn whether the product is genuinely usable rather than merely conformant.
The last lesson of this course is about the process around all of it: bug triage that keeps the queue meaningful, and quality metrics that survive contact with incentives — the honest answer to "how do we know testing is working?"
Before that, add one axe check to a page you work on and run it. Whatever it reports is real, actionable, and probably fixable this afternoon — missing labels and contrast failures usually are.
1) serious: Form elements must have labels (label)
Element: <input type="email" id="email-field">
Fix: add a <label for="email-field">, or an aria-label
https://dequeuniversity.com/rules/axe/4.10/labelon every change axe on components and key pages, in their states
per feature a keyboard pass, and a zoom check
per significant a screen-reader pass on the main flow
feature
periodically a full audit, ideally including someone who
uses assistive technology daily# A scanner finds ~a third. It CAN detect
missing labels / missing alt / contrast / no accessible name /
missing lang / duplicate ids / invalid ARIA / skipped heading levels
# It CANNOT detect
alt text that says "image1.png" / a nonsensical tab order /
an invisible focus ring / an error announced nowhere /
a modal that does not trap focus / mouse-only controls /
content that needs sight to make sense
# NEVER report a green scan as "accessible"
# Scan the STATES, not one static view
error / loading / empty / modal open / menu expanded / DARK MODE
# contrast passes in one theme and fails in the other constantly
# Introducing it to an existing product
# zero tolerance on a product with 340 violations = the test gets
# disabled and nothing is checked
# instead: catalogue the existing ones in a list with owners,
# block everything else, and never ADD to the list
# The manual checks, made routine
every change axe on components and key pages, in their states
per feature keyboard-only pass; zoom to 200% and 400%
per big feature a screen-reader pass on the main flow
(VoiceOver: Cmd+F5. NVDA: free on Windows.)
periodically a full audit, ideally with someone who uses
assistive technology daily
# Free win you may already have
# getByRole locators fail when a control loses its label or stops
# being a button — your journey tests enforce the basics alreadynpm install --save-dev @axe-core/playwrightimport { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';
test('the contacts page has no automatic accessibility violations', async ({
page,
}) => {
await page.goto('/contacts');
const results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
.analyze();
expect(results.violations).toEqual([]);
});import { axe } from 'vitest-axe';
test('the contact form has no violations', async () => {
const { container } = render(<ContactForm />);
expect(await axe(container)).toHaveNoViolations();
});test('the form is accessible in its error state', async ({ page }) => {
await page.goto('/contacts/new');
await page.getByRole('button', { name: 'Save' }).click(); // trigger errors
const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]);
});
test('the delete dialog is accessible while open', async ({ page }) => {
await page.goto('/contacts');
const row = page.getByRole('row', { name: /Ada Lovelace/ });
await row.getByRole('button', { name: 'Delete' }).click();
const results = await new AxeBuilder({ page })
.include('[role="dialog"]')
.analyze();
expect(results.violations).toEqual([]);
});const results = await new AxeBuilder({ page }).analyze();
expect(results.violations).toEqual([]); // 340 failures. Test skipped.const KNOWN = ['color-contrast', 'landmark-one-main'];
test('no new accessibility violations on the contacts page', async ({
page,
}) => {
await page.goto('/contacts');
const results = await new AxeBuilder({ page })
.disableRules(KNOWN) // tracked in ACCESSIBILITY.md, with owners
.analyze();
expect(results.violations).toEqual([]);
});npm install --save-dev @axe-core/playwright # browser tests
npm install --save-dev vitest-axe # component testsconst results = await new AxeBuilder({ page })
.withTags(['wcag2a', 'wcag2aa', 'wcag21aa']) // AA is the usual target
.include('[role="dialog"]') // scope to a region
.disableRules(KNOWN_ISSUES) // a tracked, shrinking list
.analyze();
expect(results.violations).toEqual([]);