Unit Tests That Earn Their Keep
The difference between testing behaviour and testing implementation, why the second kind fails on every refactor, and how to pick the unit worth isolating.
The difference between testing behaviour and testing implementation, why the second kind fails on every refactor, and how to pick the unit worth isolating.
Most codebases with poor test suites do not have too few unit tests. They have plenty, and the tests break every time anyone restructures anything while catching almost no defects — so refactoring becomes expensive and the suite gets blamed for both.
The cause is almost always one thing: the tests are coupled to how the code works rather than to what it does. By the end of this lesson you will be able to tell those apart, and you will know how to pick the unit worth isolating in the first place.
A unit test can assert two very different kinds of thing.
Given this input, the result is that.
What a caller would notice.
Split the function in two, rename its helpers, inline the calculation — the test still passes as long as the answer is still right.
The code called these methods, in this order.
How the result was arrived at.
Fails while nothing is broken — which teaches the team that a red suite does not mean a real problem, and that is the beginning of every ignored test suite.
# behaviour: the caller's view
def test_a_forty_five_pound_order_gets_ten_percent_off():
assert final_price(order_total=45.00) == 40.50
# implementation: the internal call sequence
def test_final_price_calls_the_discount_service(mocker):
discount = mocker.patch("pricing.discount_percent")
final_price(order_total=45.00)
discount.assert_called_once_with(45.00)The second test says nothing about whether the price is right. It would
pass with discount_percent returning garbage, and it fails the moment
somebody improves the code without changing what it does.
Bad — three mocks, and it verifies the plumbing rather than the result:
def test_import_saves_rows(mocker):
parser = mocker.patch("importer.parse_csv")
validator = mocker.patch("importer.validate_row")
repository = mocker.patch("importer.save")
parser.return_value = [{"name": "Ada"}]
validator.return_value = True
import_contacts("contacts.csv")
parser.assert_called_once_with("contacts.csv")
validator.assert_called_once_with({"name": "Ada"})
repository.assert_called_once_with({"name": "Ada"})Good — one fake at the real boundary, and it asserts what happened:
def test_import_saves_a_valid_row(in_memory_repository):
result = import_contacts(
csv_text="name,email\nAda,ada@example.com\n",
repository=in_memory_repository,
)
assert result.imported == 1
assert in_memory_repository.all()[0].name == "Ada"The first test passes against an implementation that never saves
anything — because save is a mock, and asserting that a mock was called
proves only that your test called your mock. It is a test of the test. It
also breaks if parse_csv gains a second argument, if the validation
moves inside the parser, or if anything is renamed.
The second test would fail if importing did not work. It has one fake, at the point where the code genuinely leaves the process, and it asserts on an outcome a user would care about. Rewrite the internals however you like; it keeps telling the truth.
"Unit" does not mean "one function". It means the smallest piece with a meaningful boundary — something with an interface a caller uses and internals a caller does not care about.
That is usually larger than one function, and the practical consequence is important: test through the public interface, not every private helper.
# the module's surface
def import_contacts(csv_text, repository): ...
# helpers it uses
def _parse_row(line): ...
def _normalise_email(value): ...
def _is_valid_date(value): ...Test import_contacts. The helpers are exercised through it, and they
stay free to change — which is exactly what you want, because they are the
parts most likely to be restructured.
The exception is a helper with real complexity of its own. _is_valid_date
with eight formats and a leap-year rule has enough logic to deserve direct
tests; the cost is that those tests pin its signature, so make the
decision deliberately rather than by habit.
Not all code is equally worth unit testing, and knowing where the return is highest keeps a suite valuable per line.
Pure logic. A function of its inputs, with no database or network: pricing, validation, parsing, permission rules, date arithmetic, state transitions. Every equivalence class and boundary from the foundations course, at almost no cost.
Anything with many cases. Where you identified ten values worth checking, ten unit tests are nearly free and ten browser tests are unaffordable.
Code that has broken before. The bug-clustering argument. A defect here means the difficulty is real.
Complex conditions. Four branches and a special case for one customer. Unit tests are the only affordable way to cover the combinations.
And where they pay worst:
Thin glue. A function that takes three values and passes them to another function. A test for it asserts that the code says what it says, and it will fail whenever the code is rearranged.
Configuration. Testing that a constant equals its own value.
Framework behaviour. Whether the framework routes a request or serialises a field is the framework's own tested concern.
Anything whose bugs live in the seams. A repository class whose logic is entirely SQL cannot be usefully unit tested — the interesting failures are in the database's opinion of the query, and that needs an integration test. That is the next lesson.
Often the reason a unit test needs six mocks is the code, not the test. Two changes do most of the work.
Separate deciding from doing. Pull the logic out of the function that performs input and output, and the logic becomes trivially testable:
# hard to test: decides and acts in one place
def process_order(order_id):
order = db.fetch_order(order_id)
if order.total >= 100:
discount = 20
elif order.total >= 20:
discount = 10
else:
discount = 0
db.save_discount(order_id, discount)
email.send_confirmation(order.customer_email, discount)
# easy to test: a pure decision, and a thin shell around it
def discount_for(order_total: float) -> int:
if order_total >= 100:
return 20
if order_total >= 20:
return 10
return 0
def process_order(order_id):
order = db.fetch_order(order_id)
discount = discount_for(order.total)
db.save_discount(order_id, discount)
email.send_confirmation(order.customer_email, discount)discount_for needs no mocks at all and takes every boundary value in
milliseconds. process_order is now thin enough that an integration test
covers it properly.
Pass dependencies in. A function that reaches out to a global connection can only be tested by patching that global. One that accepts its dependency can be given a fake:
# only testable by patching a module-level object
def import_contacts(csv_text):
return _save_all(parse(csv_text), repository=global_repository)
# testable by passing anything with the same shape
def import_contacts(csv_text, repository):
return _save_all(parse(csv_text), repository=repository)This is dependency injection, and in its useful form it is nothing more elaborate than that: make the dependency an argument.
Five properties, and they are worth checking a test against:
Fast. Under a millisecond. If it is not, something real is involved and it is a different level of test.
Isolated. No shared state, no order dependence, safe in parallel.
Focused. One behaviour, named for it, failing for one reason.
Readable. A colleague understands what is claimed without opening another file.
Behavioural. It survives a rewrite of the internals.
def test_a_discount_never_makes_a_price_negative():
assert final_price(order_total=10.00, discount_percent=150) == 0.00Fast, isolated, one claim, readable, and indifferent to how the clamping is implemented. That is the whole target.
# Behaviour vs implementation
behaviour given this input, the observable result is that
implementation the code called these methods in this order
# THE TEST: if someone rewrote the internals without changing
# observable behaviour, should this test fail?
# yes -> it is testing implementation, and it will cost you
# The mock trap
# asserting a mock was called proves your test called your mock
# a test with 3 mocks can pass against code that saves nothing
# more than 1-2 mocks in a unit test is a DESIGN signal
# Choosing the unit
# the smallest piece with a meaningful BOUNDARY, not one function
# test through the public interface; let helpers stay free to change
# exception: a helper with real logic of its own — deliberately
# Where unit tests pay best
pure logic pricing, validation, parsing, permissions, dates
many cases your equivalence classes and boundaries
code that broke bugs cluster where they have appeared before
complex conditions branches and special cases
# Where they pay worst
thin glue asserts that the code says what it says
configuration a constant equals itself
framework behaviour already tested by the framework
seam-bound code SQL-heavy repositories need integration tests
# Making code testable (fix the code, not the test)
separate deciding from doing
discount_for(total) -> int pure, no mocks, all boundaries
process_order(id) thin shell, integration-tested
pass dependencies in
import_contacts(csv_text, repository) not a module-level global
# Five properties of a good unit test
fast under a millisecond
isolated no shared state, any order, parallel-safe
focused one behaviour, one reason to fail
readable understandable without opening another file
behavioural survives a rewrite of the internalsUnit tests cover logic and are blind to the joins. The next lesson is about those joins: integration tests, where to cut the system so a test is both realistic and fast, and which seams are worth testing across.
Before that, take three unit tests from a project you work on and apply the refactoring question to each. The ones that would fail on a pure rewrite are the ones costing you, and rewriting one to assert on an outcome instead is a useful half hour.