advanced35 minby DevFox

Performance Engineering

Working from a profile: algorithmic wins first, then data structures, then the boundary where native code or vectorisation is the honest answer — with the measurement to justify it.

Python
Performance

The profile says forty percent of the time is in Decimal.__add__. So you replace Decimal with float, the job gets twenty percent faster, and six weeks later an accountant finds that the monthly totals are out by a few pence. The optimisation worked and the change was wrong.

The practice course covered measuring. This one is about what to do with the measurement — where speedups actually come from, which trades are safe, when the answer is native code, and how to keep a fast version from silently becoming a wrong one. By the end you will have an order of operations that starts well before micro-optimisation.

The order, and why it is that order

Do less
Cache it, batch it, skip it, do it once.
A better shape
A set, an index, a different data structure.
A better tool
numpy, a real database, a compiled library.
Faster code
Micro-optimisation, and it is last.

The first two change how the work grows. The last two change the constant in front of it.

That is why the order is not a matter of taste: no constant factor rescues a quadratic loop, and a constant-factor win on something you could have skipped entirely is wasted either way.

The practice course's example — a set instead of a list, one word, hours to seconds — is level two. Nothing at level four competes with it.

Doing less

The largest wins usually come from removing work rather than speeding it up.

python

# once, not per item
compiled = re.compile(pattern)
for line in lines:
    compiled.search(line)

# in bulk, not per item
metadata = fetch_all([p.id for p in photos])

# only when needed
@cached_property
def total_size(self) -> int: ...

# not at all
if not photo.changed_since(last_run):
    continue

That last one is the most underrated. A pipeline that reprocesses unchanged input is doing work with a known-correct answer already available, and skipping it is a bigger win than any amount of tuning the processing.

Caching deserves its warning from the memory lesson: bounded, or it becomes a leak. @lru_cache(maxsize=...), never a bare @cache in a long-running process.

A better shape

Most Python performance problems are data-structure problems wearing a costume.

text

`in` a list       -> a set or dict          scan becomes a lookup
repeated sorting  -> sort once, or bisect
list.pop(0)       -> collections.deque      the front is O(n)
string += in loop -> "".join(parts)         builds a new string each
                                            time
lookup by field   -> a dict keyed on it     built once, used often

The string one is worth showing because it looks harmless:

python

report = ""
for line in lines:                # a NEW string per iteration
    report += line

report = "".join(lines)           # one allocation

Strings are immutable, so += copies everything so far each time. On ten thousand lines that is fifty million characters copied to produce one string.

And the grouping shape, which turns a nested loop into two flat ones:

python

by_album = defaultdict(list)
for photo in photos:              # one pass
    by_album[photo.album_id].append(photo)

for album in albums:              # one pass, lookup is instant
    process(album, by_album[album.id])

Where native code helps

Python's per-operation overhead is roughly a hundred times C's. That matters only when you are performing very many operations, and the fix is usually to hand the loop to something else rather than to rewrite it.

python

total = 0
for value in values:              # 10 million interpreter steps
    total += value * 1.05

import numpy as np
total = (np.array(values) * 1.05).sum()      # one C loop

The vectorised version is not "numpy is faster". It is that the loop now happens in C, once, over contiguous memory — and, usefully, it releases the GIL while it runs, so it is already using more of the machine.

When there is no library, the options in increasing order of cost:

text

a C-implemented stdlib function   bisect, heapq, itertools, array
numpy / polars / pyarrow          array and dataframe workloads
Cython or mypyc                   annotate Python, compile it
Rust via PyO3, or C               a genuine extension

Each step adds build complexity, platform-specific wheels and a smaller pool of people who can maintain it — the distribution lesson covers what native code does to your packaging. Take the step only when a profile says the hot loop is genuinely Python overhead, and expect the maintenance to outlive the speedup.

Fast and wrong

Bad — an optimisation that changes the answer.

python

def total(prices: list[Decimal]) -> float:
    return sum(float(p) for p in prices)      # 20% faster

Good — an optimisation that cannot change the answer.

python

def total(prices: list[Decimal]) -> Decimal:
    return sum(prices)          # keep Decimal; cache the CALLER

Converting to float is faster and reintroduces exactly the representation error the numbers lesson warned about — invisible per row, material across a month, and discovered by an accountant rather than by a test. The speedup was real; the trade was never stated.

The rule: know what an optimisation costs before you take it. Approximations, weaker consistency, stale caches, dropped precision and lost error detail are all legitimate trades — and each must be a decision someone made and wrote down, not a side effect of a performance ticket.

Keeping it fast, and keeping it correct

A tuned path with no test regresses silently. Six months later someone makes a reasonable change and nothing fails.

python

def test_captions_ten_thousand_within_budget(benchmark_photos):
    start = time.perf_counter()
    build_report(benchmark_photos)
    elapsed = time.perf_counter() - start

    assert elapsed < 5.0        # generous: catches 10x, ignores noise

Assert a generous bound. A test asserting 0.8 seconds fails on a loaded CI machine and gets deleted; one asserting five seconds catches an accidental return to the quadratic version and survives noise. pytest-benchmark does this properly, with statistics and comparison against stored results.

And leave the explanation in the code:

python

# Keyed by name rather than scanned: this runs per photo, and
# the linear version was 34s of a 38s job. See PERF-214.
by_name = {photo.name: photo for photo in photos}

Without that comment the dictionary looks like an unnecessary intermediate, and the next tidy-up removes it.

The measurements that mislead

Four ways a benchmark lies, worth knowing before you trust one.

Warm caches
The second run reads from the filesystem cache, the connection pool is established, and lru_cache is populated. Measure a cold start too, since production restarts.
Small inputs
Anything is fast on a thousand rows. The quadratic and the linear version diverge only at scale, so a benchmark on test data cannot tell them apart.
Averages
A mean hides the tail, and the tail is what users experience. Report the median and the 95th or 99th percentile.
The profiler's own overhead
cProfile charges per call, so functions called very often look worse than they are. Confirm a fix with a plain timed run, and prefer a sampling profiler in production.

Each of these produces a number that is genuinely correct and answers the wrong question

A cheat sheet to keep

text

THE ORDER
  1 do less        cache (bounded), batch, skip unchanged input
  2 a better shape set/dict, deque, index, group in one pass
  3 a better tool  numpy, a database index, a compiled library
  4 faster code    last, and rarely worth it

  1 and 2 change how the work GROWS
  3 and 4 change the constant in front of it

SHAPE FIXES
  `in` a list          -> a set
  list.pop(0)          -> deque
  s += in a loop       -> "".join(parts)
  repeated sorting     -> sort once / bisect
  lookup by field      -> a dict built once
  nested loop over two -> group into a defaultdict, then two passes

NATIVE CODE, IN ORDER OF COST
  bisect/heapq/itertools/array   already C
  numpy / polars / pyarrow       one C loop, releases the GIL
  Cython / mypyc                 annotate and compile
  Rust via PyO3 / C              a real extension

  check a newer CPython, and PyPy, before rewriting anything
  native code changes your packaging story - see distribution

TRADES
  every optimisation has a cost - name it before taking it
  float instead of Decimal is faster AND wrong
  approximations, stale caches, weaker consistency: all fine,
  all decisions somebody must make on purpose

KEEPING IT
  a benchmark test with a GENEROUS bound (catches 10x, not noise)
  a comment saying what the slow version cost, and the ticket

BENCHMARKS THAT LIE
  warm caches         measure a cold start too
  small inputs        the curves only diverge at scale
  averages            report median and p95/p99
  profiler overhead   confirm with a plain timed run; py-spy in prod

  Amdahl: a 10x win on 10% of the path is 9%

Where to go next

You now have an order of operations that begins with removing work, a list of shape changes that account for most real speedups, and — the part that separates this from a list of tricks — the habit of naming what an optimisation costs before taking it. Fast and wrong is a worse outcome than slow.

Next is Typing at the Advanced Level, which returns to the type system with everything this course has established. Once you are writing decorators, protocols and generic containers, the annotations that describe them stop being obvious, and a checker that cannot follow your abstractions is a checker switched off in the places that need it most.

Before you move on, take the slowest thing you own and work down the four levels in order. Ask what work could be skipped entirely, then what shape the data should be in, and only then whether any code needs to be faster. Most of the time you will stop at level two, which is the point.