intermediate30 minby DevFox

Testing with Pytest

Tests that fail usefully: plain assertions, fixtures, parametrisation, and choosing what to test so the suite catches regressions instead of restating the implementation.

Python
Testing

You change how captions handle hyphens. Six weeks later someone reports that filenames with a trailing digit come out wrong. The change that broke it was yours, it was small, and nothing told you at the time — because the only thing that would have told you is a test, and there wasn't one.

Tests are not about proving code correct. They are about being told, in seconds rather than weeks, when a change breaks something you were not thinking about. By the end of this lesson you will write tests that fail usefully, share setup without hiding it, and know how to choose what to test — which matters far more than the syntax.

A test is a function that asserts

python

# tests/test_captions.py
from photo_tools.captions import make_caption


def test_underscores_become_spaces():
    assert make_caption("sunrise_over_lisbon.jpg") == "Sunrise Over Lisbon"

bash

pytest

text

tests/test_captions.py .                                    [100%]
1 passed in 0.01s

That is the whole framework. A file named test_*.py, a function named test_*, and a plain assert. No class to inherit from, no special assertion methods to memorise.

The plain assert is pytest's best feature, because it rewrites failures to show you the values:

text

E       AssertionError: assert 'Sunrise Over  Lisbon' == 'Sunrise Over Lisbon'
E         - Sunrise Over Lisbon
E         + Sunrise Over  Lisbon
E         ?              +

The double space is marked. You did not add a message, and the failure still tells you exactly what differs.

Arrange, act, assert

A test reads best in three parts, in order:

Arrange
Build exactly the situation the test is about, and nothing else.
Act
One call. This is the line the test's name describes.
Assert
One behaviour. Five assertions stop at the first failure.

Three parts, in this order, with a blank line between them — a reader finds each without looking for it

python

def test_large_photos_are_flagged():
    photo = Photo(name="dawn.jpg", city="Lisbon", size=1450)   # arrange

    result = is_large(photo)                                    # act

    assert result is True                                       # assert

Keep one behaviour per test. A test asserting five things stops at the first failure and tells you nothing about the other four, and its name cannot describe what it checks.

Which brings up the name. test_large_photos_are_flagged is a sentence you can read in a failure report. test_is_large_1 is not, and a failing test whose name you have to decode has lost most of its value before you start.

Testing failure

Code that raises needs testing too, and pytest.raises is how:

The second is worth the extra line. The exceptions lesson argued that a message should carry the offending value; match is how you keep that true, since a message nobody asserts on will drift.

One test, many cases

When the same logic needs several inputs, do not copy the test:

Five tests from one function, each reported separately — so a failure names the case that broke, not the function. Adding a sixth is one line, which is the point: cheap cases get written, and the awkward ones are exactly the ones that go untested when each costs a copy-paste.

Fixtures

A fixture is setup that tests request by naming it as a parameter:

Each test gets a fresh album, because the fixture function runs again for each. That isolation is the point — tests that share mutable state fail depending on the order they run in, which is the most demoralising kind of failure there is.

Fixtures can clean up, using the generator form from earlier in this course:

Put shared fixtures in conftest.py and every test file in that directory can use them with no import.

Two built-in fixtures earn their keep immediately:

tmp_path is a fresh temporary directory per test, cleaned up afterwards. capsys captures printed output. Between them, most "but it touches the filesystem" objections disappear.

Test behaviour, not implementation

This is the decision that determines whether a suite helps or hinders.

Bad — asserting on how the work is done.

Good — asserting on what comes out.

The first test passes if build_report calls two functions the right number of times — and keeps passing if it discards every result and returns an empty report. It also fails the moment you rename find_files or restructure the internals, even though behaviour is unchanged. That combination is the worst possible: it does not catch bugs and it does block refactoring, so the team learns that tests are an obstacle.

The second says what the function is for. It survives any rewrite that keeps the behaviour and fails on any change that does not.

The rule: assert on return values and observable effects. If a test needs to know the names of the functions being called, it is testing the implementation.

Choosing what to test

You cannot test everything, and trying produces a slow suite nobody runs. Prioritise:

Anything with a decision in it
Branches, edge cases, the boundary conditions — zero, one, empty, the maximum. Bugs live at boundaries.
Anything you got wrong before
Every bug is proof that a case was untested. Write the failing test first, then fix it — that way you know the test would have caught it, which is the only way to be sure.
The parts other code depends on
A helper used in thirty places earns thirty times the coverage of one used once.

Three claims on your time, in the order they repay it

Do not bother testing that Python works, that a getter returns what was set, or that a dataclass constructs. Those tests cost maintenance and catch nothing.

Coverage measures the code your tests execute:

It is useful in one direction only. Low coverage reliably tells you something is untested. High coverage does not tell you the tests are good — the bad test above achieves full coverage of build_report while asserting nothing about its output.

Running them usefully

--lf is the one that changes your habits: fix, re-run only the failures, repeat. It turns a slow suite into a fast loop while you work.

Mark the slow ones so the fast ones stay fast:

A cheat sheet to keep

Where to go next

You can write tests that fail with a useful message, cover many cases cheaply, share setup without hiding what matters, and — the part that decides whether the suite is worth having — assert on behaviour rather than on how the behaviour is produced.

Next is Test Doubles and When to Mock, which takes on the part this lesson deliberately avoided. Every example here tested something that takes values and returns values. Real code calls databases and networks, and the tools for that are powerful enough to produce tests that pass while everything is broken.

Before you move on, find a bug in something you have written — or introduce one deliberately. Write the test that catches it, watch it fail, then fix the code and watch it pass. That sequence, in that order, is the entire discipline, and doing it once makes the value obvious in a way that reading about it does not.