advanced35 minby DevFox

Performance Testing and Load Models

A load test is only as good as its model of a user. Arrival rates, think time, percentiles over averages, open versus closed models, and finding the knee rather than a number.

Testing
Performance

"We load tested it — it handled a thousand users." That sentence contains almost no information, and acting on it is how systems fall over at launch.

A thousand of what, arriving how? Doing what, at what intervals? And "handled" measured how — an average, which hides everything, or a percentile, which is what users experience? By the end of this lesson you will be able to build a load model that means something, choose the right test for the question you have, and read a result without fooling yourself.

The model is the test

A load test applies a model of demand to a system and measures the response. Everything useful depends on that model being a reasonable description of reality, and most bad load tests are bad models rather than bad tooling.

Five things a model has to state:

text

who        which user types, in what proportions? A browsing visitor
           and a bulk-importing admin are not interchangeable.
what       which operations, in what mix? 90% reads and 10% writes
           behaves nothing like the reverse.
how often  the ARRIVAL RATE — requests or sessions per second — not
           a number of simulated users.
with what  realistic data volumes, cardinality and cache-hit rates.
           A test where every request asks for the same record
           measures your cache, not your system.
for how    a spike, an hour, a day. Behaviour changes with duration.
long

The most common modelling error is skipping the third and thinking in concurrent users instead, which brings us to the distinction that matters most.

Open and closed models

A closed model has a fixed number of virtual users, each looping: make a request, wait for the response, think, repeat. Concurrency is capped by the user count.

An open model has requests arriving at a specified rate regardless of what the system is doing. Fifty per second means fifty per second whether responses take 10 milliseconds or 10 seconds.

The difference is not academic — it determines whether your test can reproduce the failure you care about.

Closed — 100 looping users

Slowness reduces the load.

Each user waits for a response before making the next request, so when the system slows down the arrival rate drops.

The test protects the system from itself. Models a fixed pool of clients — a batch job, an internal tool with a hundred employees.

Open — 50 requests per second

Slowness makes requests pile up.

They arrive on schedule whether responses take 10 milliseconds or 10 seconds, so queues grow, memory grows, and timeouts cascade.

Models the internet: users arrive because they arrive, not because you finished serving the last one.

Real web traffic is open. Only one of these can reproduce a queue collapsing a system.

Real web traffic is open. A closed-model test therefore cannot reproduce the most important failure mode there is — the one where slowness causes a queue, the queue causes more slowness, and the system collapses rather than degrading. That collapse is what happens in production, and a closed model smooths it into a gentle slope.

Percentiles, not averages

An average response time is the most misleading number in performance work. Ninety-five requests at 100ms and five at 8s produce this:

average0ms

p50 — the typical request0ms

p950ms

p990ms

max0ms

One distribution, five summaries. Only the first one sounds acceptable.

The average implies a system that is fine. One user in twenty is having a terrible experience, and if that user is doing something important — checking out, submitting a form — the business impact is not one twentieth of anything.

So report percentiles, and know what each is for:

Two subtleties worth knowing. Percentiles do not average: you cannot mean the p95 of two machines to get the overall p95, so aggregation has to happen over the raw distribution. And a page is many requests: if a page load makes 20 calls, the probability that all 20 are under the p95 is about 36%, so the tail dominates the user's experience far more than its label suggests.

The kinds, and the question each answers

"Performance testing" covers several activities with different purposes, and naming them prevents running the wrong one.

The two most neglected are soak and spike. A memory leak is invisible in a ten-minute run and obvious in eight hours — and it is one of the most common real production failures. A spike exposes cold caches, connection-pool limits and autoscaling that reacts too slowly, none of which a steady ramp shows.

Finding the knee

A capacity test is not looking for a number; it is looking for a shape.

The knee is where latency starts rising much faster than load — around 500 here. That is your real capacity, not 800. Past the knee the system is already failing; it has simply not admitted it yet.

Two things to take from that table.

Report the knee, with headroom. "We are comfortable to 400 per second, degrading from 500, collapsing at 700" is actionable. "It handled 800" is false.

Watch the shape of the failure. Does it degrade — slower but correct — or collapse, with timeouts cascading? Graceful degradation is a design property worth having, and the resilience lesson is about building it.

The test that measures the wrong thing

Bad — one endpoint, one record, no think time, an average:

Good — a modelled mix, realistic data, percentile thresholds:

The bad test produces a large, meaningless number. Every request asks for the same record, so after the first one it is served from cache and never touches the database — the test measures the cache. There is no think time, so the traffic pattern resembles nothing. It is one endpoint, so the expensive listing query and the import job are untested. And the average hides whatever the tail is doing.

The good test has two user types in realistic proportions, a hundred thousand distinct ids so cache-hit rates resemble production, think time between actions, and thresholds on percentiles that fail the test rather than being read afterwards. It will produce a much smaller number, and the number will be true.

Interpreting a result

A load test result is a starting point for a diagnosis, not a conclusion. The result tells you that it slowed; the system tells you why.

That last check is the one people skip and it is frequently the most informative. A system that does not recover has leaked something, and the soak test is where you find out what.

And a caveat about environments: a load test against a staging system with a tenth of the data and one instance instead of twelve tells you very little in absolute terms. It is still useful for comparison — this version against the last — which is why benchmark runs on a modest environment are worth having even when capacity testing needs a production-like one.

Cheat sheet

Where to go next

Load testing asks what happens under pressure. The next lesson asks what happens when a dependency fails: resilience testing, injecting the latency, partitions and outages production will deliver anyway — and doing it as an experiment with a hypothesis and a blast radius.

Before that, look at the last performance number your team quoted and check whether it came from an open or a closed model, and whether it was an average or a percentile. Those two questions frequently reveal that the number means something quite different from what everyone believes.