Levels of Testing
Unit, integration, system and end-to-end — what each level can see, what it cannot, and why the pyramid is a statement about cost and speed rather than a rule.
Unit, integration, system and end-to-end — what each level can see, what it cannot, and why the pyramid is a statement about cost and speed rather than a rule.
A team I know had 900 browser tests. The suite took fifty-one minutes, failed for no reason about twice a week, and everybody had learned to press "re-run" until it went green.
It was testing an email validation rule by opening a browser, loading a page, typing into a field and reading an error message. Twenty times, for twenty invalid addresses. The same rule, in the right place, would have been twenty checks running in under a millisecond.
By the end of this lesson you will know the four levels of testing, what each can and cannot see, and how to decide which level a given check belongs at. The pyramid you may have seen drawn is here too, along with what it is actually claiming.
A level is how much of the system a test exercises at once. There are four, and they are not competing approaches — they are different distances, and what you can see changes with distance.
The product as a user meets it. Seconds to minutes each.
The whole application through its own API. Seconds.
Two or three parts together, at least one of them real. Milliseconds to seconds.
One function or class, alone. Under a millisecond.
The shape is a claim about how many of each you want, and the rest of this lesson is what that claim rests on.
A unit test exercises the smallest meaningful piece of code in isolation — a function, a method, a class — with nothing real around it.
def test_discount_never_makes_a_price_negative():
assert apply_discount(price=10.00, percent=150) == 0.00That runs in microseconds. No database, no network, no browser. You can run ten thousand of them in the time a browser takes to open one page.
What they see well: logic. Calculations, branches, edge cases, validation rules, error handling. Anything where one input should produce one output, and where the interesting cases are numerous — which is exactly where the input shapes from the last lesson apply.
What they cannot see: whether the pieces fit together. Every unit can be individually correct while the application is broken, because correctness of parts does not add up on its own.
An integration test exercises two or more parts together, with at least one of them real. Your code and a real database. Your code and a real HTTP client. Two modules and the interface between them.
Slower — this one needs a database — and it sees a whole category of bug that no unit test can.
What they see well: the seams. The SQL that is valid in your
head and invalid in the database. The column that does not allow
null. The transaction that never commits. The serialisation that
drops a field. The API client that sends snake_case where the
server wants camelCase.
Those were shape four from the last lesson, and they are the most common source of real production failures — because each side was tested against the other side's imagined behaviour.
What they cannot see: whether a user can complete a task.
A system test exercises the complete application through its own interface — usually its API — with its real database and its real configuration, but without the browser, and often with external services faked.
What they see well: the contract the application presents. Status codes, response shapes, error messages, authentication and authorisation, and whether a sequence of requests behaves — create, then read, then delete.
For a backend service this is the best value per test you can buy: nearly all the coverage of an end-to-end test, at a fraction of the cost and without a browser's flakiness.
An end-to-end test drives the real product the way a person does — a real browser against a real deployment, front end and back end and database together.
What they see well: that the thing actually works. Every layer, integrated, through the interface a user meets. It is the only level that catches a broken front-end build, a misconfigured URL, a button hidden behind a cookie banner, or a login flow that fails on a redirect.
What they cost: everything. Slow — seconds to minutes each. Fragile, because a hundred moving parts can each wobble. Vague on failure: a red end-to-end test tells you the journey broke, not where. And expensive to maintain, because one redesigned page invalidates dozens of them.
Not "unit tests are better". The claim is economic, and it has two halves.
Cost rises as you go up. Slower to run, harder to write, more fragile, more expensive to maintain.
Precision falls as you go up. A failing unit test names the function. A failing end-to-end test names the journey and leaves you to work out which of forty components was at fault.
Put those together and you get the one rule worth memorising: push each check down to the lowest level that can catch it.
Two well-known counter-shapes are worth being able to name.
Mostly manual and end-to-end, few unit tests.
What you get when testing was added late, from the outside.
Feedback takes an hour, regressions are frequent, and nobody believes a red result until they have re-run it twice.
A bulge at the integration level instead of the unit level.
The argument: in a typical web application the bugs that actually happen are at the seams, and unit tests of thin glue code prove very little.
Fair for that kind of application — and it does not change the rule. Push each check to the lowest level that can catch it; here the seams are the thing.
When you know what to check, one question settles where it goes: what is the smallest amount of system that could get this wrong?
Could one function get this wrong on its own?
Then it is a unit test. Validation rules, calculations, branches, edge cases — nearly all of them live here.
Does it need my code and something real?
A database, a queue, an HTTP client. Integration test. This is where schemas, transactions and serialisation break.
Is it about what the API promises?
Status codes, permissions, a sequence of requests. System test, no browser needed.
Does it genuinely need all of it, in a browser?
Then you have found a real end-to-end test. There should not be many.
Here is what happens when that question does not get asked.
Bad — checking a validation rule through the browser:
Good — the rule in a unit test, the wiring in one browser test:
The bad version takes several seconds, needs a browser and a running application, and breaks when the page is redesigned. Multiply it by the twenty invalid-address cases worth checking and you have a slow, brittle suite testing a pure function through six layers of software.
The good version checks the twenty cases in milliseconds and spends one expensive test confirming the plumbing exists. Same coverage, a fraction of the cost, and a failure that says which part is wrong.
Level is about scope. Several other words describe purpose or technique, and they cut across the levels rather than sitting inside them:
| Term | What it means |
|---|---|
| Smoke test | A quick check that the build is worth testing |
| Sanity test | A narrow check that one fix worked |
| Regression test | Re-checking that what worked still works |
| Acceptance test | Would the person who asked for it accept it? |
| Component test | One deployable service, its dependencies faked |
| Contract test | Two services still agree on their interface |
Smoke, sanity and acceptance get their own lesson shortly, and regression gets one too. The distinction to hold on to now: a smoke test is a purpose that might be implemented at any level, while "unit" and "end-to-end" are scopes. Mixing the two vocabularies is why testing conversations go in circles.
You know where to look and at what distance. The next lesson is about what to look for: turning a requirement — usually vaguer than it appears — into concrete test cases somebody else could run and get the same answer.
Before that, take a feature you know and place three checks at three levels: something that belongs in a unit test, something that needs the database, and the one journey worth driving through a browser. Doing that deliberately a few times is what makes the choice automatic.
1. Open the site
2. Sign in as a real account
3. Click New Note, type a body, press Save
4. Reload the page
5. The note is visible in the list1. Open the registration page
2. Type "not-an-email" into the email field
3. Press Submit
4. Verify "Please enter a valid email address" appears# The four levels — scope, not quality
unit one function/class, isolated <1ms
sees: logic, branches, edge cases
blind to: whether the parts fit together
integration your code + something real ms to seconds
sees: the seams — SQL, schemas, serialisation,
transactions, API shapes
blind to: whether a user can finish a task
system the whole app via its own API seconds
sees: status codes, auth, response shapes, sequences
best value per test for a backend service
end-to-end the real product in a browser seconds to minutes
sees: that it genuinely works, all layers integrated
costs: slow, fragile, vague on failure, costly to keep
# The pyramid's actual claim
# cost rises as you go up; precision falls as you go up
# therefore: push each check DOWN to the lowest level that
# can catch it
# many unit / fewer integration / fewer system / a few e2e
# Shapes to recognise
# ice cream cone mostly manual and e2e — slow, untrusted
# testing trophy integration-heavy — a fair argument for web apps
# Choosing a level, in one question
# what is the SMALLEST amount of system that could get this wrong?
# that is where the test belongs.
# Purpose words, which cut across levels
# smoke is this build worth testing at all
# sanity did this one fix work
# regression does what used to work still work
# acceptance would the requester accept it
# contract do two services still agreedef test_saving_a_note_can_be_read_back(database):
note_id = notes.create(database, body="hello")
assert notes.get(database, note_id).body == "hello"def test_creating_a_note_requires_authentication():
response = client.post("/notes", json={"body": "hello"})
assert response.status_code == 401def test_rejects_an_address_with_no_at_sign():
assert not is_valid_email("not-an-email")
# ...and separately, one end-to-end test that submitting an
# invalid form shows the error message at all.