Exploratory Testing
Testing without a script, done properly: charters, time-boxing, note-taking and heuristics. The technique that finds the bugs your written cases were never going to.
Testing without a script, done properly: charters, time-boxing, note-taking and heuristics. The technique that finds the bugs your written cases were never going to.
Five hundred test cases passed. An hour of unscripted poking found five bugs, and the worst of them was this: a CSV import that failed halfway left the first forty-nine rows in the database while telling the user nothing had been imported. Run the same file again and those forty-nine arrived a second time.
No case covered it, and no case was ever going to. Somebody would have had to imagine a partial failure before the feature existed.
That is the ceiling on a written suite, and exploratory testing is how you get above it: designing and running tests at the same time, letting what you find decide what to try next. It is not "clicking around", and the difference between the two is the whole content of this lesson.
By the end you will know how to run a proper session, how to keep notes that make your findings reproducible, and a set of prompts for when you run out of ideas.
A test case is a prediction. Writing "the reset link should be rejected the second time" means somebody thought of second uses before the software existed.
So five hundred cases cover five hundred predictions, and the defects that reach production are almost always outside that set — the combination nobody put together, the state nobody imagined, the sequence no requirement described.
Repeatable, reportable, automatable.
Anyone can pick it up and get the same answer. It gives you a baseline you can point at.
It can only ever find bugs somebody already imagined.
Finds what the scripts do not know about.
What you find in minute three changes what you try in minute four — something a script structurally cannot do.
It gives no coverage number and no repeatability on its own.
Undirected clicking is what people picture, and it has three specific problems. No boundary, so you drift and re-cover the same ground. No record, so an interesting finding cannot be reproduced. No accountability, so nobody can say afterwards what was examined.
Three pieces of structure fix all three, and everything inside them is free.
One sentence naming the mission.
So the session has a subject, and so you can tell whether an interesting thread is on it or not.
Usually 60 or 90 minutes.
So it has an end, can be planned around, and does not quietly become the whole day.
Written as you go, not afterwards.
So a finding is reproducible and the session is reportable — including what you did not get to.
A charter is one sentence naming what you will explore, and often how. Specific enough to give direction, loose enough to allow discovery.
Explore the CSV import
with files that are malformed in different ways
to discover how partial failures are handled
Explore the shared-project permissions
with users at each permission level
to discover whether anyone can see or change what they should not
Explore the checkout flow
by interrupting it — closing the tab, going back, losing the
network
to discover whether an order can end up in a broken stateThe pattern is: explore X with Y to discover Z.
Compare that with "test the import feature". The charter says what counts as being on-task, which is what lets you follow an interesting thread without losing the thread you were on.
Where charters come from: an area that has just changed, an area that produced defects before, a feature whose requirements were thin, a part of the product nobody has looked at in months, or a specific worry somebody has voiced.
Ninety minutes, one charter, notes as you go.
You will find yourself doing the same three things over and over. That loop is the engine of the whole technique.
A question
Something you do not know about the software yet. "What happens if the file is empty?"
An experiment
The smallest thing that would answer it. Upload an empty file.
A result
Which is interesting whether or not it is a bug — because it is what tells you the next question worth asking.
While you work, note four kinds of thing:
BUG something wrong. Enough detail to reproduce it, now, while
you still remember the sequence.
QUESTION something you do not understand, or behaviour you cannot
tell is intended. Ask someone afterwards.
IDEA something worth trying that is off this charter. Write it
down and keep going — it becomes a future charter.
NOTE what you covered, and anything a reader would need to know.Real notes from a session, and they look like this rather than like prose:
14:02 Charter: CSV import, malformed files, partial failures
14:05 Happy path: 100-row file imports, "100 records imported"
14:09 Row 50 has a bad date. Import stops. 49 records ARE saved,
message says "Import failed". BUG-1: partial import left in
place, message implies nothing happened.
14:15 Same file again. Now 98 records — the 49 were imported
twice. BUG-2: no idempotency, no duplicate detection.
14:22 Empty file (0 bytes): "Import successful, 0 records". Odd
but defensible. QUESTION: should it warn?
14:26 Header row only: same as above.
14:31 Missing header row: first data row consumed as headers, 99
imported silently. BUG-3.
14:38 10 MB file: browser hangs ~40s, no progress indicator.
BUG-4. IDEA: what is the actual size limit? Separate
charter.
14:45 File with a UTF-8 BOM: first column name unmatched, whole
import fails with "Unknown column". BUG-5.
14:52 Non-CSV renamed to .csv: rejected cleanly. Good.
15:00 Stop. 5 bugs, 1 question, 1 idea. Not covered: concurrent
imports, permissions, files over 10 MB.Two things about those notes. The timestamps make the sequence reproducible, which matters when a bug depends on order — as BUG-2 did. And the last line reports what was not covered, which is what makes the session honest and lets someone decide whether to charter another.
Twenty minutes in, the obvious things are done. Heuristics are prompts that generate more, and having a list is what separates a productive session from a stalled one.
The input shapes from the earlier lesson are the first resort: empty, zero, negative, enormous, unicode, apostrophes, boundaries.
Interrupt it. Close the tab mid-action. Press back. Refresh during a save. Turn the network off halfway. Open two tabs and act in both.
Do it twice. Submit twice, quickly. Then again after a failure. Then again tomorrow.
Go the wrong way. Skip a step. Reach step three by its URL without doing steps one and two. Use the back button instead of a Cancel link.
Be the wrong person. Log out and try. Log in as someone who should not have access. Use a URL belonging to another user's record. Change an id in the address bar.
CRUD it. For any object: create it, read it, update it, delete it, then try to read it again. Then delete it twice.
Follow the data. Create something, then look for it everywhere it should appear — the list, the search, the export, the report, the email, the audit log. One of those places is usually forgotten.
Change the environment. A narrow window. A phone. Zoom to 200%. Keyboard only, no mouse. Dark mode. A different time zone.
Break the assumption. Whatever the feature assumes is true — that there is data, that the user has one of something, that the file is small, that the request arrives in order — make it false.
Bad — a session with no charter and no notes:
Spent the afternoon testing the new import feature.
Found some issues, mostly around error handling.
Will look into it more tomorrow.Good — a charter, a time box, and what came out:
Charter: CSV import, malformed files, partial-failure handling
Duration: 60 minutes
Found 5 bugs (BUG-1 to BUG-5). The serious one is BUG-1: a
failed import leaves partial data behind while telling the user
nothing was imported. BUG-2 follows from it — re-running the
same file duplicates the rows that did import.
Not covered: concurrent imports, permission checks, files over
10 MB. Suggest a second charter for size limits.The first report is a real cost, not just a missed opportunity. The issues cannot be fixed because they cannot be reproduced. Nobody can tell what was examined, so the area has to be tested again from nothing. And "mostly around error handling" gives a developer no way to start.
The second is the same afternoon, reported so that five defects are actionable, the biggest risk is named first, and the gap is explicit.
Exploratory testing is at its most valuable in four situations:
A new feature, before writing cases. Explore first, then write cases for what you learned. Cases written from real experience of the software are much better than cases written from a requirement.
When requirements are thin. If nobody can tell you what it should do, you cannot write a case yet — but you can go and find out what it does.
After a fix, around the fix. The sanity-check neighbourhood from the previous lesson is a small exploratory session.
On a schedule, in high-risk areas. A recurring session on payments, permissions or data export finds things the regression suite has stopped being able to see.
It is also worth pairing occasionally: two people at one screen, one driving and one asking questions, find things neither would alone.
# Exploratory testing: designing and running tests at the same
# time, letting each finding decide the next experiment.
# scripted only finds bugs somebody already imagined
# exploratory finds the ones nobody did
# do both
# Three pieces of structure — this is what makes it not clicking
charter explore X with Y to discover Z
time box 60 or 90 minutes; it has an end
notes written AS YOU GO; timestamps; what you did NOT cover
# Charter examples
explore the CSV import with malformed files to discover how
partial failures are handled
explore shared-project permissions as each role to discover
whether anyone can see what they should not
# Note four kinds of thing
BUG enough detail to reproduce, written down immediately
QUESTION behaviour you cannot tell is intended — ask afterwards
IDEA worth trying, off-charter — a future session
NOTE what you covered
# Heuristics, for when the obvious ideas run out
inputs empty, zero, negative, huge, unicode, apostrophes,
boundaries
interrupt it close the tab, press back, refresh mid-save, drop
the network, act in two tabs
do it twice submit twice fast; again after a failure
wrong way skip a step; reach step 3 by URL; back instead of
cancel
wrong person logged out; another role; another user's id in the
URL
CRUD it create, read, update, delete, read again, delete
again
follow data list, search, export, report, email, audit log —
one is forgotten
environment narrow window, phone, 200% zoom, keyboard only,
dark mode, another time zone
assumptions whatever it assumes is true, make it false
# Reporting a session
# the charter, the duration, the bugs, the biggest risk first,
# and what was NOT covered
# When it is most valuable
# a new feature, BEFORE writing cases
# when requirements are thin
# around a fix (the sanity neighbourhood)
# on a schedule, in high-risk areasYou have found something. The next lesson is about what happens then — writing a bug report that gets fixed rather than closed, which is a specific skill and the point at which your work either becomes a change in the software or does not.
Before that, run one real session: pick a charter, set a timer for sixty minutes, and keep notes in the format above. The discipline feels constraining for the first ten minutes and then does not, and the difference in what you find is immediate.