Performance Engineering
Working from a profile: algorithmic wins first, then data structures, then the boundary where native code or vectorisation is the honest answer — with the measurement to justify it.
Working from a profile: algorithmic wins first, then data structures, then the boundary where native code or vectorisation is the honest answer — with the measurement to justify it.
The profile says forty percent of the time is in
Decimal.__add__. So you replace Decimal with float, the
job gets twenty percent faster, and six weeks later an
accountant finds that the monthly totals are out by a few pence.
The optimisation worked and the change was wrong.
The practice course covered measuring. This one is about what to do with the measurement — where speedups actually come from, which trades are safe, when the answer is native code, and how to keep a fast version from silently becoming a wrong one. By the end you will have an order of operations that starts well before micro-optimisation.
Do less
Cache it, batch it, skip it, do it once.
A better shape
A set, an index, a different data structure.
A better tool
numpy, a real database, a compiled library.
Faster code
Micro-optimisation, and it is last.
That is why the order is not a matter of taste: no constant factor rescues a quadratic loop, and a constant-factor win on something you could have skipped entirely is wasted either way.
The practice course's example — a set instead of a list, one word, hours to seconds — is level two. Nothing at level four competes with it.
The largest wins usually come from removing work rather than speeding it up.
That last one is the most underrated. A pipeline that reprocesses unchanged input is doing work with a known-correct answer already available, and skipping it is a bigger win than any amount of tuning the processing.
Caching deserves its warning from the memory lesson: bounded, or
it becomes a leak. @lru_cache(maxsize=...), never a bare
@cache in a long-running process.
Most Python performance problems are data-structure problems wearing a costume.
The string one is worth showing because it looks harmless:
Strings are immutable, so += copies everything so far each
time. On ten thousand lines that is fifty million characters
copied to produce one string.
And the grouping shape, which turns a nested loop into two flat ones:
Python's per-operation overhead is roughly a hundred times C's. That matters only when you are performing very many operations, and the fix is usually to hand the loop to something else rather than to rewrite it.
The vectorised version is not "numpy is faster". It is that the loop now happens in C, once, over contiguous memory — and, usefully, it releases the GIL while it runs, so it is already using more of the machine.
When there is no library, the options in increasing order of cost:
Each step adds build complexity, platform-specific wheels and a smaller pool of people who can maintain it — the distribution lesson covers what native code does to your packaging. Take the step only when a profile says the hot loop is genuinely Python overhead, and expect the maintenance to outlive the speedup.
Bad — an optimisation that changes the answer.
Good — an optimisation that cannot change the answer.
Converting to float is faster and reintroduces exactly the
representation error the numbers lesson warned about — invisible
per row, material across a month, and discovered by an
accountant rather than by a test. The speedup was real; the
trade was never stated.
The rule: know what an optimisation costs before you take it. Approximations, weaker consistency, stale caches, dropped precision and lost error detail are all legitimate trades — and each must be a decision someone made and wrote down, not a side effect of a performance ticket.
A tuned path with no test regresses silently. Six months later someone makes a reasonable change and nothing fails.
Assert a generous bound. A test asserting 0.8 seconds fails
on a loaded CI machine and gets deleted; one asserting five
seconds catches an accidental return to the quadratic version
and survives noise. pytest-benchmark does this properly, with
statistics and comparison against stored results.
And leave the explanation in the code:
Without that comment the dictionary looks like an unnecessary intermediate, and the next tidy-up removes it.
Four ways a benchmark lies, worth knowing before you trust one.
Warm caches
The second run reads from the filesystem cache, the
connection pool is established, and lru_cache is populated.
Measure a cold start too, since production restarts.
Small inputs
Anything is fast on a thousand rows. The quadratic and the linear version diverge only at scale, so a benchmark on test data cannot tell them apart.
Averages
A mean hides the tail, and the tail is what users experience. Report the median and the 95th or 99th percentile.
The profiler's own overhead
cProfile charges per call, so functions called very often
look worse than they are. Confirm a fix with a plain timed
run, and prefer a sampling profiler in production.
You now have an order of operations that begins with removing work, a list of shape changes that account for most real speedups, and — the part that separates this from a list of tricks — the habit of naming what an optimisation costs before taking it. Fast and wrong is a worse outcome than slow.
Next is Typing at the Advanced Level, which returns to the type system with everything this course has established. Once you are writing decorators, protocols and generic containers, the annotations that describe them stop being obvious, and a checker that cannot follow your abstractions is a checker switched off in the places that need it most.
Before you move on, take the slowest thing you own and work down the four levels in order. Ask what work could be skipped entirely, then what shape the data should be in, and only then whether any code needs to be faster. Most of the time you will stop at level two, which is the point.
# once, not per item
compiled = re.compile(pattern)
for line in lines:
compiled.search(line)
# in bulk, not per item
metadata = fetch_all([p.id for p in photos])
# only when needed
@cached_property
def total_size(self) -> int: ...
# not at all
if not photo.changed_since(last_run):
continue`in` a list -> a set or dict scan becomes a lookup
repeated sorting -> sort once, or bisect
list.pop(0) -> collections.deque the front is O(n)
string += in loop -> "".join(parts) builds a new string each
time
lookup by field -> a dict keyed on it built once, used oftena C-implemented stdlib function bisect, heapq, itertools, array
numpy / polars / pyarrow array and dataframe workloads
Cython or mypyc annotate Python, compile it
Rust via PyO3, or C a genuine extensionTHE ORDER
1 do less cache (bounded), batch, skip unchanged input
2 a better shape set/dict, deque, index, group in one pass
3 a better tool numpy, a database index, a compiled library
4 faster code last, and rarely worth it
1 and 2 change how the work GROWS
3 and 4 change the constant in front of it
SHAPE FIXES
`in` a list -> a set
list.pop(0) -> deque
s += in a loop -> "".join(parts)
repeated sorting -> sort once / bisect
lookup by field -> a dict built once
nested loop over two -> group into a defaultdict, then two passes
NATIVE CODE, IN ORDER OF COST
bisect/heapq/itertools/array already C
numpy / polars / pyarrow one C loop, releases the GIL
Cython / mypyc annotate and compile
Rust via PyO3 / C a real extension
check a newer CPython, and PyPy, before rewriting anything
native code changes your packaging story - see distribution
TRADES
every optimisation has a cost - name it before taking it
float instead of Decimal is faster AND wrong
approximations, stale caches, weaker consistency: all fine,
all decisions somebody must make on purpose
KEEPING IT
a benchmark test with a GENEROUS bound (catches 10x, not noise)
a comment saying what the slow version cost, and the ticket
BENCHMARKS THAT LIE
warm caches measure a cold start too
small inputs the curves only diverge at scale
averages report median and p95/p99
profiler overhead confirm with a plain timed run; py-spy in prod
Amdahl: a 10x win on 10% of the path is 9%report = ""
for line in lines: # a NEW string per iteration
report += line
report = "".join(lines) # one allocationby_album = defaultdict(list)
for photo in photos: # one pass
by_album[photo.album_id].append(photo)
for album in albums: # one pass, lookup is instant
process(album, by_album[album.id])total = 0
for value in values: # 10 million interpreter steps
total += value * 1.05
import numpy as np
total = (np.array(values) * 1.05).sum() # one C loopdef total(prices: list[Decimal]) -> float:
return sum(float(p) for p in prices) # 20% fasterdef total(prices: list[Decimal]) -> Decimal:
return sum(prices) # keep Decimal; cache the CALLERdef test_captions_ten_thousand_within_budget(benchmark_photos):
start = time.perf_counter()
build_report(benchmark_photos)
elapsed = time.perf_counter() - start
assert elapsed < 5.0 # generous: catches 10x, ignores noise# Keyed by name rather than scanned: this runs per photo, and
# the linear version was 34s of a 38s job. See PERF-214.
by_name = {photo.name: photo for photo in photos}