Smoke, Sanity and Acceptance Testing
The same software tested for different purposes: is the build worth testing, did this fix work, and would the person who asked for the feature accept it?
The same software tested for different purposes: is the build worth testing, did this fix work, and would the person who asked for the feature accept it?
The build landed at two o'clock. The release call is at five. The full suite takes a day.
Somebody has to decide what to run in those three hours, and the wrong decision is easy to make: start at the top of the suite and work down. Do that and at five o'clock you will have run a fifth of it, in alphabetical order, and still not know whether anyone can log in.
By the end of this lesson you will know the three questions people routinely confuse — is this build worth testing at all, did that one fix work, and would the person who asked for this accept it — and the order to ask them in, which is what makes three hours enough.
The levels lesson was about scope: how much of the system a test touches. This one is about purpose: the question a test is asked in order to answer.
The two are independent, and mixing the vocabularies is why testing conversations go in circles. A smoke test might be a single API call or a whole browser journey — what makes it a smoke test is the question it answers, not its size.
| Kind | The question | Size | When |
|---|---|---|---|
| Smoke | Is this build worth testing? | Minutes | Every build |
| Sanity | Did this specific fix work? | Minutes | After a fix |
| Regression | Does what worked still work? | Hours | Before release |
| Acceptance | Would the requester accept it? | Varies | Before sign-off |
The name comes from hardware: power up the board, and if smoke comes out, stop — there is no point checking whether the display works.
A smoke test is a short, broad check that the build is fundamentally functional. Not thorough, deliberately: it touches many areas shallowly rather than one area deeply, and its only job is to decide whether detailed testing would be a waste of time.
A realistic smoke test for a web application:
1. The application loads without an error page
2. A known account can sign in
3. The main list page shows data
4. Creating one item works
5. Opening a detail page works
6. Sign out works
7. No 500 errors appeared in the logs during the aboveSeven steps, five minutes, and it catches the whole class of failure where something is broken so fundamentally that nothing else matters — a bad deployment, a missing environment variable, a database that is not reachable, a front-end build that did not compile.
Three properties define a good one:
Broad and shallow. One case per critical area, never several.
Fast. If it takes longer than about ten minutes it is not a smoke test any more, and people will skip it.
Unambiguous. A smoke test that fails means stop and tell someone, not investigate for an hour. It should be reliable enough that a failure is always real.
Smoke testing is also called build verification, and after a deployment specifically, a post-deployment check. Same thing.
A sanity test is narrow and deep: one specific thing was changed, and you check that particular thing and its immediate surroundings.
The typical use is a returned bug fix. A defect said the reset link worked twice; a fix arrives; you check that specific behaviour and the few things closest to it:
1. Use a reset link once — it works
2. Use the same link again — rejected with the right message
3. Request a new link and use it — it works
4. An unexpired unused link still worksSteps 3 and 4 are the important part, and the reason a sanity test is not just "re-run the failed case". A fix can make the reported symptom go away and break something adjacent — the classic being a fix that rejects the second use by invalidating all links.
The contrast with a smoke test is exact, and worth holding on to, because these two get confused more than any other pair in testing.
Broad and shallow.
Many areas, one check each. Asked of every build.
Is there any point testing this at all?
Narrow and deep.
One area, several checks, including its neighbours. Asked after a specific change.
Did that particular thing get fixed, without breaking what sits next to it?
The other two ask whether the software works. Acceptance testing asks whether it is the right software — which is the validation question from the first lesson, made into an activity.
It is testing against the need, not against the specification, and that distinction is what gives it value. A feature can satisfy every written requirement and still not be acceptable:
Requirement met The report can be exported as CSV
Not acceptable It takes nine minutes and the user has to
stay on the pageNobody wrote down "and it should be quick enough to use", because nobody writes that down. The acceptance question surfaces it.
Several forms exist, and the names come up in real work:
User acceptance testing (UAT) — the people who will use the feature try it, usually in a staging environment, usually with their own real scenarios rather than a script. It reliably finds things no internal tester would: the workflow does not match how they actually work, a term means something different in their world, a step that seems minor happens four hundred times a day.
Business acceptance — does it satisfy the commercial need? Does the reporting produce the numbers finance expects?
Contract acceptance — for work delivered against an agreement, the formal check that the deliverables are met.
Operational acceptance — can it be run? Is it monitored, is there a backup, can it be restored, does it log usefully, can it be deployed and rolled back? This one is routinely skipped and is worth insisting on: a feature that works and cannot be operated is not finished.
Alpha and beta — testing by internal users, then by real users outside the organisation, before a full release.
Bad — the whole suite, started at the front:
14:00 Begin the full regression suite, alphabetically
16:30 Case 88 of 400. Still on account settings.
17:00 Release decision needed. No idea whether login works.Good — the questions in order of what they rule out:
14:00 Smoke test — 6 minutes. Build is usable.
14:10 Sanity-check the three fixes in this build — 20 minutes.
14:30 Regression on the areas the change touched — 90 minutes.
16:00 Highest-priority cases in untouched areas — 60 minutes.
17:00 Release decision, with a clear statement of what was
not covered.The first version spends two and a half hours on whatever happens to sort first and arrives at the decision point with no view of the critical paths. It also fails the honesty test: "we ran 88 of 400 cases" tells the decision-maker almost nothing.
The second answers the cheapest, most decisive question first, then spends the remaining time by risk, and ends with a statement someone can actually make a decision from — including what was not tested, which is the part people leave out.
The sequence is not convention; each step exists to avoid wasting the next one.
Smoke — is the build usable at all?
Minutes. If it fails, stop: everything after this was going to fail too, and you would have spent the afternoon finding that out one case at a time.
Sanity — did the changes in this build work?
Minutes. If they did not, it goes back to development. There is no point testing carefully around a fix that is not finished.
Regression — does what used to work still work?
Hours, and the expensive one. Start where the change landed, then keep going by risk rather than by the order the suite happens to be in.
Acceptance — is this the right thing?
By the people who asked for it, and including whether it can actually be operated.
That is the same cost-curve argument from the first lesson, applied to an afternoon instead of to a project.
# Purpose, not scope. Any of these can be any size.
SMOKE — is this build worth testing?
broad and shallow: one check per critical area
under 10 minutes; automated; runs after every deployment
a failure means STOP and tell someone, not investigate
also called: build verification, post-deployment check
1. the app loads 2. sign in works
3. main list has data 4. create one item
5. a detail page 6. sign out
7. no 500s in the log
SANITY — did this specific change work?
narrow and deep: one area, several checks
always include the neighbours, not just the reported case
a fix that stops the second use by breaking ALL uses passes
a re-run of the bug and fails a sanity test
REGRESSION — does what used to work still work?
broad and deep; hours; the expensive one
start where the change landed, then continue by risk
ACCEPTANCE — would the requester accept it?
tests the NEED, not the specification
UAT real users, real scenarios, staging
business does it satisfy the commercial need
contract are the agreed deliverables met
operational monitored? backed up? deployable? rollback?
alpha / beta internal users, then external
never let UAT be the first time anyone tried the feature
# The order, and why
# smoke fails -> stop; everything else would have failed
# sanity fails -> back to development; do not test around it
# then regression -> where the change landed, then by risk
# then acceptance -> by the people who asked
# Reporting honestly
# "88 of 400 cases" is not information
# "smoke passed, three fixes verified, regression on billing and
# auth complete, settings and reporting NOT covered" isEverything so far has assumed there is software to run. The next lesson looks at the testing you can do before that — reviewing requirements, reviewing code, and what a linter or a type checker is actually doing — which is the cheapest point on the cost curve and the one most often skipped.
Before that, write a smoke test for something you work on: five to eight steps, under ten minutes, covering the areas where a failure would mean nothing else is worth checking. It is a short exercise and the artefact is immediately useful.