Integration Tests and Their Seams
Where to cut a system so an integration test is both realistic and fast: the seams worth testing across, and the ones that are really end-to-end tests in disguise.
Where to cut a system so an integration test is both realistic and fast: the seams worth testing across, and the ones that are really end-to-end tests in disguise.
Every unit passing and the application broken is not a paradox — it is the normal state of a system whose parts were each tested against an imagined version of their neighbours. The SQL is valid in your head. The API client sends the field name the server used to expect. The transaction never commits.
Integration tests exist to find exactly that class of defect, and the whole craft is in deciding where to cut. Cut too small and you have another unit test; cut too large and you have a slow, vague end-to-end test wearing the wrong name. By the end of this lesson you will know how to choose, and how to make the cut cheap enough to run constantly.
A seam is a boundary between two things that were built or configured separately, where each side holds an assumption about the other.
The seams that produce real defects:
your code <-> the database SQL, schema, types, transactions
your code <-> another service request and response shapes
your code <-> the file system paths, permissions, encodings
your code <-> a queue delivery, ordering, duplication
your code <-> the framework lifecycle, serialisation, middleware
one module <-> another in-process contractsThe reason these break is that both sides were tested against a description of the other. Your repository test mocked the database, so it never learned that the column does not allow null. The database has no test at all for your query.
An integration test removes the description and uses the real thing.
The useful question is: what is the smallest set of real components that could get this wrong?
Three cuts cover most needs, and they are worth naming because teams argue past each other without them.
One seam, so a failure names it.
A repository against a real database; a client against a real HTTP server.
The narrowest and the most diagnostic.
Best value per test.
Driven through the application's own HTTP interface, so one test covers routing, serialisation, validation, authorisation, the database and the transaction boundary — and is still fast.
Expensive, and usually avoidable.
Contract testing, in the advanced course, gets most of the benefit without running both.
Your code plus one real dependency. The narrowest and the most valuable. A repository against a real database; a client against a real HTTP server. It tests one seam, so a failure names it.
def test_a_contact_can_be_saved_and_read_back(database):
repository = ContactRepository(database)
contact_id = repository.save(Contact(name="Ada", email="a@b.com"))
assert repository.get(contact_id).name == "Ada"Your whole application, its own dependencies real, external services faked. Usually driven through the application's HTTP interface. This is the highest-value cut for a backend service: it covers routing, serialisation, validation, authorisation, the database and the transaction boundary in one test, and it is still fast.
def test_importing_a_csv_creates_contacts(client, admin_token):
response = client.post(
"/contacts/import",
headers={"Authorization": f"Bearer {admin_token}"},
files={"file": ("c.csv", b"name,email\nAda,a@b.com\n")},
)
assert response.status_code == 201
assert response.json()["imported"] == 1
listing = client.get("/contacts", headers=...)
assert listing.json()["contacts"][0]["name"] == "Ada"Two of your own services together. Expensive, and usually the wrong answer — contract testing, in the advanced course, gets most of the benefit without running both.
The most common way to spoil an integration test is to swap the real database for an easier one — SQLite in place of Postgres, an in-memory store in place of the file system.
Bad — the test passes and production fails:
@pytest.fixture
def database():
"""SQLite in memory — fast and convenient."""
return create_engine("sqlite:///:memory:")Good — the same engine and version as production:
@pytest.fixture(scope="session")
def database():
"""Postgres 16, the same as production, in a container."""
with PostgresContainer("postgres:16") as postgres:
engine = create_engine(postgres.get_connection_url())
run_migrations(engine)
yield engineSQLite and Postgres disagree about things that matter: string comparison
is case-insensitive by default in one and not the other, JSONB and array
columns do not exist, ON CONFLICT differs, constraint enforcement
differs, and time zone handling differs. Each of those is a seam bug your
test was written to catch and now cannot.
The point of an integration test is that the dependency is real. A substituted database converts it into a slower unit test with a false claim attached.
Containers make the honest version affordable — Testcontainers for Python, Java, Go and Node, or a service in your CI configuration. Start one per test session, not per test.
A real database is shared state, and the structuring lesson insisted every test be independent. Three ways to have both, in order of preference.
A transaction per test, rolled back. The test runs inside a transaction that is never committed, so the database is untouched afterwards. Fast — no data is ever written — and it works for the large majority of tests.
@pytest.fixture
def database(engine):
connection = engine.connect()
transaction = connection.begin()
session = Session(bind=connection)
yield session
session.close()
transaction.rollback()
connection.close()Truncate between tests. Delete the data after each test. Slower, and necessary when the code under test manages its own transactions.
A schema or database per test. Complete isolation, safe in parallel, the most setup cost. Worth it for a large parallel suite.
Given a real database and a way to isolate, these are the checks that earn their runtime — each one a defect a unit test cannot see:
save then read back does what came out match what went in?
catches serialisation, truncation, type
coercion, dropped fields
a constraint fires insert a duplicate; expect the error the
code claims to handle
a transaction rolls back make step two fail; assert step one is
NOT in the database
a query with real data 10,000 rows: does the query still work,
and how many queries does the page make?
the migration applies run migrations from zero, then the tests
a null where the code a column that allows null and a model
assumed a value that does not
ordering and pagination page 1 and page 2 with an unstable sort
key: is any record shown twice or never?That last one deserves a mention: keyset pagination with a non-unique sort column silently skips records, and it is invisible until somebody compares page boundaries. It is a pure seam bug — the query is valid, the code is reasonable, the result is wrong.
Your database is yours; a payment provider is not. Calling a third party in a test makes the test slow, non-deterministic and occasionally expensive.
The line worth drawing: real for what you own, faked for what you do not. Your database, your cache, your queue — real. A payment provider, an email service, a mapping API — faked.
How to fake them well is the next lesson but one. The relevant point here is that the boundary between "real" and "faked" should be the boundary of your system, not an arbitrary line drawn for convenience.
Integration tests are seconds where unit tests are milliseconds, and a suite of four hundred of them is a suite nobody runs. Four things keep the cost down:
Share the expensive setup. One container and one migration run per session, not per test. This is usually the single biggest win.
Do not test logic here. Ten boundary values belong in unit tests. An integration test's job is one round trip through a seam.
Create only what the test needs. A factory with sensible defaults, rather than a fixture that builds a whole customer with orders and invoices for a test about email formatting.
Run them in parallel, which requires the independence from the structuring lesson and pays for it immediately.
A useful target: the whole integration suite under two minutes. Past that, it moves out of the pre-push gate and into the pull-request gate, and the feedback loop lengthens.
# A seam: a boundary where each side assumed something about the other
your code <-> database SQL, schema, types, transactions
your code <-> a service request and response shapes
your code <-> file system paths, permissions, encodings
your code <-> a queue delivery, ordering, duplication
your code <-> framework lifecycle, serialisation, middleware
# both sides were tested against a DESCRIPTION of the other
# Where to cut: the smallest set of REAL components that could
# get this wrong
your code + one real dependency narrowest; a failure names the seam
your whole app, externals faked best value for a backend service
two of your own services usually the wrong answer — see
contract testing instead
# Test through the interface real callers use
# HTTP for a web service, not the handler function
# calling the handler skips routing, auth, deserialisation, errors
# Use the REAL dependency
# SQLite instead of Postgres disagrees about: case sensitivity,
# JSONB and arrays, ON CONFLICT, constraints, time zones
# -> every one of those is a bug the test existed to catch
# Testcontainers, or a CI service. One container per SESSION.
# Isolation, in order of preference
transaction per test, rolled back fast; works for most tests
truncate between tests needed when the code commits
schema or database per test full isolation, most setup
# The savepoint trap
# a test inside a transaction sees its own uncommitted writes,
# so code that never commits still passes
# testing persistence? use truncation for that test
# The checks worth the runtime
save then read back serialisation, truncation, type coercion
a constraint fires insert a duplicate; expect the real error
a transaction rolls back fail step 2; assert step 1 is NOT stored
a query with real volume 10,000 rows; and how many queries?
migrations from zero
a null where code assumed a value
pagination with an unstable sort key records skipped or repeated
# Real vs faked
# real for what you own: database, cache, queue
# faked for what you do not: payments, email, third-party APIs
# Keeping them fast — target: whole suite under 2 minutes
share expensive setup per session
no logic testing here — boundaries belong in unit tests
create only what the test needs
run in parallelThe next lesson is about the other side of that boundary: faking external services properly — stubs, recorded responses, local doubles — and how to notice when your fake has drifted away from the real thing it stands in for, which is the failure mode that makes fakes dangerous rather than merely imperfect.
Before that, check one thing in your own suite: does an integration test run against the same database engine and version as production? If not, that is the highest-value change available in this lesson, and it is a fixture edit.