Performance Testing and Load Models
A load test is only as good as its model of a user. Arrival rates, think time, percentiles over averages, open versus closed models, and finding the knee rather than a number.
A load test is only as good as its model of a user. Arrival rates, think time, percentiles over averages, open versus closed models, and finding the knee rather than a number.
"We load tested it — it handled a thousand users." That sentence contains almost no information, and acting on it is how systems fall over at launch.
A thousand of what, arriving how? Doing what, at what intervals? And "handled" measured how — an average, which hides everything, or a percentile, which is what users experience? By the end of this lesson you will be able to build a load model that means something, choose the right test for the question you have, and read a result without fooling yourself.
A load test applies a model of demand to a system and measures the response. Everything useful depends on that model being a reasonable description of reality, and most bad load tests are bad models rather than bad tooling.
Five things a model has to state:
who which user types, in what proportions? A browsing visitor
and a bulk-importing admin are not interchangeable.
what which operations, in what mix? 90% reads and 10% writes
behaves nothing like the reverse.
how often the ARRIVAL RATE — requests or sessions per second — not
a number of simulated users.
with what realistic data volumes, cardinality and cache-hit rates.
A test where every request asks for the same record
measures your cache, not your system.
for how a spike, an hour, a day. Behaviour changes with duration.
longThe most common modelling error is skipping the third and thinking in concurrent users instead, which brings us to the distinction that matters most.
A closed model has a fixed number of virtual users, each looping: make a request, wait for the response, think, repeat. Concurrency is capped by the user count.
An open model has requests arriving at a specified rate regardless of what the system is doing. Fifty per second means fifty per second whether responses take 10 milliseconds or 10 seconds.
The difference is not academic — it determines whether your test can reproduce the failure you care about.
Slowness reduces the load.
Each user waits for a response before making the next request, so when the system slows down the arrival rate drops.
The test protects the system from itself. Models a fixed pool of clients — a batch job, an internal tool with a hundred employees.
Slowness makes requests pile up.
They arrive on schedule whether responses take 10 milliseconds or 10 seconds, so queues grow, memory grows, and timeouts cascade.
Models the internet: users arrive because they arrive, not because you finished serving the last one.
Real web traffic is open. A closed-model test therefore cannot reproduce the most important failure mode there is — the one where slowness causes a queue, the queue causes more slowness, and the system collapses rather than degrading. That collapse is what happens in production, and a closed model smooths it into a gentle slope.
// k6: an open model — arrival rate, not user count
export const options = {
scenarios: {
steady: {
executor: 'constant-arrival-rate',
rate: 50,
timeUnit: '1s',
duration: '10m',
preAllocatedVUs: 100,
maxVUs: 1000, // allowed to grow if responses slow
},
},
};An average response time is the most misleading number in performance work. Ninety-five requests at 100ms and five at 8s produce this:
The average implies a system that is fine. One user in twenty is having a terrible experience, and if that user is doing something important — checking out, submitting a form — the business impact is not one twentieth of anything.
So report percentiles, and know what each is for:
p50 the typical experience
p95 the edge of normal — what a regular user hits occasionally
p99 the unhappy tail; on a page making 20 requests, roughly one
in five page loads contains a p99 request
p99.9 what the loudest complaints are about
max one number, usually an outlier, occasionally the whole storyTwo subtleties worth knowing. Percentiles do not average: you cannot mean the p95 of two machines to get the overall p95, so aggregation has to happen over the raw distribution. And a page is many requests: if a page load makes 20 calls, the probability that all 20 are under the p95 is about 36%, so the tail dominates the user's experience far more than its label suggests.
export const options = {
thresholds: {
http_req_failed: ['rate<0.01'], // under 1% errors
http_req_duration: ['p(95)<500', 'p(99)<1500'],
},
};"Performance testing" covers several activities with different purposes, and naming them prevents running the wrong one.
LOAD does it meet its targets at expected demand?
run at the modelled rate for a realistic duration
STRESS where does it break, and HOW?
increase the rate until it fails; watch the shape
SPIKE can it absorb a sudden jump?
10x for two minutes — a marketing email, a news story
SOAK does it survive time?
expected load for 8-24 hours. Finds leaks, connection
exhaustion, log disks filling, cache growth
CAPACITY how much headroom is there, and what limits it?
find the knee, then identify the bottleneck
BENCHMARK did this change make it faster or slower?
the same test, two versions, comparedThe two most neglected are soak and spike. A memory leak is invisible in a ten-minute run and obvious in eight hours — and it is one of the most common real production failures. A spike exposes cold caches, connection-pool limits and autoscaling that reacts too slowly, none of which a steady ramp shows.
A capacity test is not looking for a number; it is looking for a shape.
rate (req/s) p95 errors what is happening
100 80ms 0% comfortable
200 95ms 0% still linear
400 140ms 0% queues forming, still fine
500 310ms 0% the KNEE — latency rising fast
600 2.4s 0.5% queues growing faster than draining
700 9.8s 14% timeouts cascading
800 timeout 61% collapseThe knee is where latency starts rising much faster than load — around 500 here. That is your real capacity, not 800. Past the knee the system is already failing; it has simply not admitted it yet.
Two things to take from that table.
Report the knee, with headroom. "We are comfortable to 400 per second, degrading from 500, collapsing at 700" is actionable. "It handled 800" is false.
Watch the shape of the failure. Does it degrade — slower but correct — or collapse, with timeouts cascading? Graceful degradation is a design property worth having, and the resilience lesson is about building it.
Bad — one endpoint, one record, no think time, an average:
export default function () {
http.get('https://api.test/contacts/42');
}
// 5,000 req/s, average 12ms. "The API handles 5,000 per second."Good — a modelled mix, realistic data, percentile thresholds:
export const options = {
scenarios: {
browsing: {
executor: 'constant-arrival-rate',
rate: 45,
timeUnit: '1s',
duration: '30m',
preAllocatedVUs: 200,
exec: 'browse',
},
importing: {
executor: 'constant-arrival-rate',
rate: 1,
timeUnit: '60s',
duration: '30m',
preAllocatedVUs: 10,
exec: 'bulkImport',
},
},
thresholds: {
'http_req_duration{scenario:browsing}': ['p(95)<500'],
http_req_failed: ['rate<0.01'],
},
};
export function browse() {
const id = randomFrom(CONTACT_IDS); // 100k distinct ids
http.get(`${BASE}/contacts?page=${randomInt(1, 500)}`);
sleep(randomBetween(2, 8)); // real think time
http.get(`${BASE}/contacts/${id}`);
sleep(randomBetween(3, 15));
}The bad test produces a large, meaningless number. Every request asks for the same record, so after the first one it is served from cache and never touches the database — the test measures the cache. There is no think time, so the traffic pattern resembles nothing. It is one endpoint, so the expensive listing query and the import job are untested. And the average hides whatever the tail is doing.
The good test has two user types in realistic proportions, a hundred thousand distinct ids so cache-hit rates resemble production, think time between actions, and thresholds on percentiles that fail the test rather than being read afterwards. It will produce a much smaller number, and the number will be true.
A load test result is a starting point for a diagnosis, not a conclusion. The result tells you that it slowed; the system tells you why.
[ ] what saturated first? CPU, memory, disk, network, connection
pool, a lock, a downstream service
[ ] where did the time go? Application traces, not guesses. A p99
of 8s is usually one slow dependency or one lock.
[ ] how many queries per request? An N+1 that is fine at 10 rows
is fatal at load.
[ ] did errors change kind? Timeouts, connection refused, 500s and
429s mean different things.
[ ] did it recover? After the load stops, does latency return to
baseline — or has something stayed broken?That last check is the one people skip and it is frequently the most informative. A system that does not recover has leaked something, and the soak test is where you find out what.
And a caveat about environments: a load test against a staging system with a tenth of the data and one instance instead of twelve tells you very little in absolute terms. It is still useful for comparison — this version against the last — which is why benchmark runs on a modest environment are worth having even when capacity testing needs a production-like one.
# The model IS the test. State five things:
who user types and their proportions
what operation mix — 90/10 read/write is not 10/90
how often the ARRIVAL RATE, not a virtual user count
with what realistic data volume, cardinality, cache-hit rates
for how a spike, an hour, a day
long
# Open vs closed — this determines what you can reproduce
closed fixed users, each waiting for a response. Slowness REDUCES
load, so the test protects the system from itself.
models a fixed client pool: a batch job, 100 employees
open requests arrive at a rate regardless. Slowness makes them
PILE UP — queues, memory, cascading timeouts.
models the internet. Use this for web traffic.
# closed-model tests are optimistic by construction, and that is
# the mechanism behind "we tested 1,000 and it fell over at 400"
# Percentiles, never averages
95 x 100ms + 5 x 8s = average 495ms, p95 8s
p50 typical / p95 edge of normal / p99 the unhappy tail /
p99.9 the loudest complaints
# percentiles do not average across machines
# a page making 20 requests: only ~36% have all 20 under the p95
# Kinds of test
load does it meet targets at expected demand
stress where does it break, and HOW
spike can it absorb 10x for two minutes
soak 8-24 hours: leaks, connection exhaustion, disks, caches
capacity find the KNEE, then the bottleneck
benchmark this version vs the last
# soak and spike are the most neglected and find the most
# Report the KNEE, with headroom
# "comfortable to 400/s, degrading from 500, collapsing at 700"
# not "it handled 800" — past the knee it is already failing
# Set a real client timeout and count timeouts as errors
# 0% errors with 10-second latency measures patience nobody has
# After a run, diagnose
what saturated first? CPU / memory / disk / pool / lock / downstream
where did the time go? traces, not guesses
queries per request? an N+1 fine at 10 rows is fatal at load
did the error KIND change? timeout vs refused vs 500 vs 429
DID IT RECOVER? if not, something leaked — go and soak testLoad testing asks what happens under pressure. The next lesson asks what happens when a dependency fails: resilience testing, injecting the latency, partitions and outages production will deliver anyway — and doing it as an experiment with a hypothesis and a blast radius.
Before that, look at the last performance number your team quoted and check whether it came from an open or a closed model, and whether it was an average or a percentile. Those two questions frequently reveal that the number means something quite different from what everyone believes.