advanced35 minby DevFox

Memory, Reference Counting, and Garbage Collection

When an object is actually freed, the cycles reference counting cannot collect, weak references, and why measuring memory in Python is harder than it looks.

Python
Performance

The worker processes a queue for six hours and its memory climbs the whole time. Nothing visibly leaks — every function returns, every file closes, and there is no global list being appended to. Restarting fixes it until tomorrow.

Python manages memory for you, which means you rarely think about it and are therefore poorly equipped when it goes wrong. By the end of this lesson you will know exactly when an object is freed, the one case the primary mechanism cannot handle, why sys.getsizeof lies to you, and how to find what is actually holding on.

Reference counting

Every object carries a count of how many references point at it. When the count reaches zero, the object is freed immediately — not eventually, not at the next collection.

python

import sys

photos = [1, 2, 3]
print(sys.getrefcount(photos))     # 2: the name, and the argument
                                   # getrefcount itself received

backup = photos
print(sys.getrefcount(photos))     # 3

del backup
print(sys.getrefcount(photos))     # 2

getrefcount always reports one more than you expect, because passing the object to it creates a reference.

Immediacy is the useful property. This is why files close promptly in CPython even without with — the file object's count drops to zero when the name goes out of scope. It is also why you should still use with: that behaviour is an implementation detail of CPython, and PyPy and others free objects later.

References are created by more than assignment: putting something in a list, passing it to a function, capturing it in a closure, storing it as an attribute, and — the one people forget — an exception's traceback, which holds every frame and therefore every local variable in the call chain.

Cycles, and the collector

Reference counting has exactly one blind spot:

python

class Node:
    def __init__(self):
        self.parent = None
        self.children = []

parent = Node()
child = Node()
parent.children.append(child)
child.parent = parent          # a cycle

del parent, child              # both counts are still 1

Neither count reaches zero, because each is referenced by the other. Nothing else can reach them and nothing frees them.

The cyclic garbage collector exists for this. It runs periodically, finds groups of objects that reference each other but are unreachable from anywhere else, and frees them.

python

import gc

gc.collect()                   # run it now; returns how many freed
gc.get_stats()                 # per-generation statistics
gc.get_count()                 # allocations since the last pass

It is generational, on the observation that most objects die young:

Generation 2 — survivors, scanned rarely
Generation 1 — scanned sometimes
Generation 0 — new objects, scanned often

Which is why a long-lived cycle can sit around for a while — the collector is barely looking at that generation

Cycles are common and mostly harmless: parent–child trees, doubly linked lists, an object holding a callback that closes over it. You do not need to avoid them. You need to know they exist, because they change when memory is released, and because objects in a cycle with a __del__ used to be uncollectable — no longer true since Python 3.4, but plenty of advice on the internet still assumes it.

Weak references

A weak reference points at an object without keeping it alive:

python

import weakref

class Photo:
    def __init__(self, name):
        self.name = name

photo = Photo("dawn.jpg")
ref = weakref.ref(photo)

print(ref())          # <Photo object>
del photo
print(ref())          # None - it is gone

The two places this matters.

Breaking a cycle

A child that points back at its parent.

Two strong references in a loop keep each other alive until the cycle collector notices. One of them made weak means ordinary reference counting frees both immediately.

A cache that does not hold on

Entries vanish when the last real user lets go.

WeakValueDictionary drops an entry once the value is freed elsewhere, so the cache speeds up access to objects that are alive rather than deciding what stays alive.

Both are cases where holding a reference is not the same as wanting to keep the object alive

The cycle case, concretely:

python

class Node:
    def __init__(self, parent=None):
        self._parent = weakref.ref(parent) if parent else None

    @property
    def parent(self):
        return self._parent() if self._parent else None

And the cache:

python

cache: weakref.WeakValueDictionary[str, Photo] = (
    weakref.WeakValueDictionary()
)

Note that not everything supports weak references — list, dict, int and str do not, and a class with __slots__ needs "__weakref__" declared, which is the catch the MRO lesson mentioned.

The unbounded cache

Bad — a cache with no bound.

python

from functools import cache

@cache
def render_thumbnail(photo_id: str, width: int) -> bytes:
    ...

Good — a cache that forgets.

python

from functools import lru_cache

@lru_cache(maxsize=512)
def render_thumbnail(photo_id: str, width: int) -> bytes:
    ...

@cache keeps every result forever, keyed on the arguments. In a long-running service with a million photo ids and several widths, that is a million entries of image bytes and the arguments that produced them, none of which are ever released — and the process looks like it has a leak because functionally it does. Nothing appends to a global list; the decorator holds the dictionary.

maxsize bounds it. This is the six-hour worker from the opening: an unbounded cache is by far the most common cause of slow, steady memory growth in Python services, ahead of anything exotic.

The same shape appears in hand-written caches, in a module-level dict that only ever gains keys, and in a logger configured to keep every record in memory.

Measuring, and why getsizeof lies

python

import sys

sys.getsizeof([1, 2, 3])          # 88 - the LIST only
sys.getsizeof([photo_a, photo_b]) # 72 - regardless of the photos

getsizeof reports the size of the object itself, not what it references. A list of a million large objects reports the size of a million pointers. It is nearly useless for the question people ask it.

tracemalloc answers a better one — where allocations came from:

python

import tracemalloc

tracemalloc.start()
snapshot_before = tracemalloc.take_snapshot()

process_everything()

snapshot_after = tracemalloc.take_snapshot()
for stat in snapshot_after.compare_to(snapshot_before, "lineno")[:10]:
    print(stat)

text

loading.py:44: size=412 MiB (+412 MiB), count=1200000 (+1200000)

The line number and the count together usually identify the problem immediately.

For a live process, objgraph shows what is holding a reference — which is the question when you know what is growing and not why:

python

import objgraph
objgraph.show_most_common_types(limit=10)
objgraph.show_backrefs([suspect], max_depth=5)

And gc.get_referrers(obj) is the standard-library version of the same question, though its output takes reading.

A cheat sheet to keep

text

REFERENCE COUNTING
  count hits zero -> freed IMMEDIATELY, deterministically
  sys.getrefcount(x) reports one extra (its own argument)
  references come from: names, containers, arguments, closures,
    attributes, and exception tracebacks
  CPython-specific: still use `with`, do not rely on it

CYCLES
  a <-> b keeps both counts above zero forever
  the cyclic collector finds and frees unreachable groups
  generational: new objects checked often, survivors rarely
  cycles are normal - they change WHEN memory is released
  never gc.disable() for speed; tune thresholds and measure

WEAK REFERENCES
  weakref.ref(obj)         does not keep it alive; ref() -> None
  a weak parent link breaks a tree cycle
  WeakValueDictionary      a cache that never keeps things alive
  not supported by list/dict/int/str
  __slots__ needs "__weakref__" declared

THE COMMON LEAK
  @cache            unbounded - keeps every result forever
  @lru_cache(maxsize=512)                                 <- bound it
  also: a module-level dict that only ever gains keys
  this is the usual cause of steady growth in a service

MEASURING
  sys.getsizeof(x)     the object ONLY, not what it references
  tracemalloc          snapshots + compare_to -> file, line, count
  objgraph             what is holding a reference, in a live process
  gc.get_referrers(x)  the standard-library version

  freed memory often stays with the process - that is not a leak
  the question is whether it grows WITHOUT BOUND

Where to go next

You now know when an object is freed, what reference counting cannot handle on its own, and how to find what is holding on. The practical takeaway is narrow and worth acting on: check your caches. An unbounded one accounts for more slow memory growth in Python services than every other cause combined.

Next is The GIL and What It Really Blocks, which is the other thing people believe wrongly about CPython. It explains what the lock does and does not prevent, why threads still help for I/O, and what free-threaded builds change.

Before you move on, take a service or long-running script and find every cache in it — decorators, module-level dictionaries, anything memoised. Check each has a bound. That audit takes fifteen minutes and is the highest-value memory work available in most codebases.