beginner30 minby DevFox

Your First Automated Test

Write, run and read a real automated test: arrange, act, assert; what a good failure message looks like; and why a test that never fails is not a test.

Testing
Automation

Every automated test you will ever read — in any language, in any framework, written by anybody — is the same three lines wearing different clothes. Put the world into a known state. Do one thing. Check what came out.

That is the whole idea, and this lesson is you writing one. There is code here and you do not need to be a programmer to follow it: a test is one of the simplest useful programs there is.

By the end you will have written, run and read a real test, you will know the three-part shape every test has, and you will know why a test that has never failed is not yet a test.

What a test actually is

Strip away the tooling and a test is a small program that sets up a situation, performs one action, and compares the result with what was expected. If they differ, it reports a failure.

No magic. The framework's entire contribution is finding your tests, running them, and printing something readable when one fails.

Arrange
Put the world into the state this test needs. An account, a file, a value.
Act
Do the one thing being tested. One thing — if there are two, it is two tests.
Assert
Compare what happened with what should have happened.

The three parts, in every test, in every language

Some people say given / when / then instead, which is the same division you met in acceptance criteria. Every test you read for the rest of your career has these three parts, labelled or not.

Your first test

We will test the discount rule from the boundary-values lesson. Here is the function under test, in discounts.py:

python

def discount_percent(order_total: float) -> int:
    """Discount for an order total, in whole percent."""
    if order_total >= 100:
        return 20
    if order_total >= 20:
        return 10
    return 0

And the test, in test_discounts.py:

python

from discounts import discount_percent


def test_no_discount_below_twenty_pounds():
    # Arrange
    order_total = 19.99

    # Act
    result = discount_percent(order_total)

    # Assert
    assert result == 0

Install the runner and run it:

bash

pip install pytest
pytest

text

============================= test session starts =====================
collected 1 item

test_discounts.py .                                             [100%]

============================== 1 passed in 0.01s ======================

That single dot is your test passing. Two conventions did the work of configuration: the file is named test_*.py and the function is named test_*, which is how pytest found it without being told.

Making it fail on purpose

A passing test tells you very little on its own. Break it deliberately — change the expected value to 5 — and run again:

text

================================== FAILURES ===========================
___________________ test_no_discount_below_twenty_pounds ______________

    def test_no_discount_below_twenty_pounds():
        order_total = 19.99
        result = discount_percent(order_total)
>       assert result == 5
E       assert 0 == 5

test_discounts.py:10: AssertionError
========================= 1 failed in 0.02s ===========================

Read that output, because reading failures is most of the skill. It names the test, shows the line that failed, and shows both values: 0 == 5 is false, and it tells you which side was which.

Change it back. And take the habit from this: whenever you write a test, make it fail once, by breaking the expectation or the code. A test you have never seen fail might be asserting nothing at all — which was the regression-test mistake from earlier in this course, and it is the single most common way an automated suite provides false comfort.

The boundaries, cheaply

Now the payoff from the boundary lesson. Every value you identified, checked in milliseconds:

python

def test_no_discount_just_below_the_first_threshold():
    assert discount_percent(19.99) == 0


def test_ten_percent_exactly_on_the_first_threshold():
    assert discount_percent(20.00) == 10


def test_ten_percent_just_above_the_first_threshold():
    assert discount_percent(20.01) == 10


def test_ten_percent_just_below_the_second_threshold():
    assert discount_percent(99.99) == 10


def test_twenty_percent_exactly_on_the_second_threshold():
    assert discount_percent(100.00) == 20


def test_twenty_percent_just_above_the_second_threshold():
    assert discount_percent(100.01) == 20


def test_no_discount_on_a_zero_total():
    assert discount_percent(0) == 0

bash

pytest -v

text

test_discounts.py::test_no_discount_just_below_the_first_...  PASSED
test_discounts.py::test_ten_percent_exactly_on_the_first_...  PASSED
test_discounts.py::test_ten_percent_just_above_the_first_...  PASSED
test_discounts.py::test_ten_percent_just_below_the_second...  PASSED
test_discounts.py::test_twenty_percent_exactly_on_the_sec...  PASSED
test_discounts.py::test_twenty_percent_just_above_the_sec...  PASSED
test_discounts.py::test_no_discount_on_a_zero_total           PASSED

============================ 7 passed in 0.02s ========================

Seven boundary cases in two hundredths of a second, on every commit, forever. Compare that with checking them by hand through a browser — and note that the two "exactly on" cases are the ones that catch an off-by-one, and they cost nothing to include.

When the same check repeats with different values, most frameworks let you write it once:

python

import pytest


@pytest.mark.parametrize(
    "order_total,expected",
    [
        (0.00, 0),
        (19.99, 0),
        (20.00, 10),
        (20.01, 10),
        (99.99, 10),
        (100.00, 20),
        (100.01, 20),
    ],
)
def test_discount_thresholds(order_total, expected):
    assert discount_percent(order_total) == expected

Seven tests, one function, and each row still fails independently with its own values shown.

A test that reads badly

Bad — four checks, one name, and a failure that says nothing:

python

def test_discounts():
    assert discount_percent(19.99) == 0
    assert discount_percent(20.00) == 10
    assert discount_percent(100.00) == 20
    assert discount_percent(0) == 0

Good — one behaviour per test, named for what it claims:

python

def test_ten_percent_starts_exactly_at_twenty_pounds():
    assert discount_percent(20.00) == 10

The bad version has two specific problems. It stops at the first failure, so if the £20 boundary is wrong you never learn whether £100 is — one bug hides the next. And it produces test_discounts failed, which tells a reader nothing; they have to open the file to find out which rule broke.

A test name is documentation that cannot go out of date, because it runs. test_ten_percent_starts_exactly_at_twenty_pounds states a business rule, and if that name appears in a failure list, everyone in the room knows what is broken without reading any code.

Assertions worth knowing

A handful of shapes covers nearly everything:

python

assert result == 10                    # equality
assert result is None                  # identity
assert "error" in message              # membership
assert result > 0                      # comparison
assert isinstance(result, int)         # type
assert not user.is_active              # boolean

# a float comparison — 0.1 + 0.2 is not exactly 0.3
assert total == pytest.approx(43.00)

# that something raises, which is a test of the failure path
with pytest.raises(ValueError):
    discount_percent(-10)

That last one deserves attention. Testing that invalid input is rejected is as important as testing that valid input works — it is the "what happens when it fails" question from the requirements lesson, in code. And note it also asks a real question of the code above: what should discount_percent(-10) do? The function as written returns 0 silently, and whether that is right is a requirements gap the test just found.

A test with setup

Real tests usually need something to exist first. When several share the same arrangement, a fixture provides it:

python

import pytest


@pytest.fixture
def basket():
    """A basket with two items, totalling £45.00."""
    return Basket(items=[Item("book", 25.00), Item("pen", 20.00)])


def test_a_forty_five_pound_basket_gets_ten_percent(basket):
    assert basket.discount_percent() == 10


def test_adding_an_item_can_reach_the_next_tier(basket):
    basket.add(Item("lamp", 60.00))
    assert basket.discount_percent() == 20

Two things make fixtures safe. Each test gets a fresh one — the second test's addition cannot affect the first — which is the structural answer to the shared-mutable-data problem from the last lesson. And the fixture's name in the test's arguments says what the test needs, so it is readable without scrolling.

Running them usefully

bash

pytest                          # everything
pytest -v                       # one line per test
pytest test_discounts.py        # one file
pytest -k boundary              # tests whose names match "boundary"
pytest -x                       # stop at the first failure
pytest --lf                     # only the ones that failed last time
pytest -q                       # quiet

--lf — last failed — is the one that changes your working rhythm. Fix something, re-run only what was broken, get an answer in a second.

The final step is out of your hands and worth asking for: the suite runs on every change, in the pipeline, and a failure blocks the merge. A suite that only runs when someone remembers is a suite that is red for three days before anybody notices.

Cheat sheet

python

# Every test, three parts
def test_ten_percent_starts_exactly_at_twenty_pounds():
    order_total = 20.00                       # Arrange
    result = discount_percent(order_total)    # Act
    assert result == 10                       # Assert

# Conventions that make it discoverable
#   file: test_*.py     function: test_*      (pytest)

# Many values, one test
@pytest.mark.parametrize(
    "order_total,expected",
    [(19.99, 0), (20.00, 10), (99.99, 10), (100.00, 20)],
)
def test_discount_thresholds(order_total, expected):
    assert discount_percent(order_total) == expected

# Shared setup, fresh for every test
@pytest.fixture
def basket():
    return Basket(items=[Item("book", 25.00)])

# Assertions
assert result == 10
assert result is None
assert "error" in message
assert total == pytest.approx(43.00)      # floats are not exact
with pytest.raises(ValueError):           # the failure path
    discount_percent(-10)

bash

# Running
pytest                  # everything
pytest -v               # a line per test
pytest -k boundary      # names matching
pytest -x               # stop at the first failure
pytest --lf             # only what failed last time

text

# Rules
one behaviour per test    a multi-assert test hides the second bug
name the behaviour        test_no_discount_on_a_zero_total
                          not test_discount_percent_2
MAKE IT FAIL ONCE         a test never seen failing may assert nothing
test the failure path     rejection matters as much as acceptance
run it on every change    in the pipeline; a failure blocks the merge

Where to go next

You can write a test. The next lesson widens the definition of "working" past correctness: usability, accessibility, compatibility, performance and security — the requirements nobody writes down, and the cheapest first check for each.

Before that, write three tests against something real: a function you have, or one you write in five minutes. Put a boundary value in each, and break one on purpose to see it fail. That last part is the habit worth forming today.