intermediate30 minby DevFox

Decorators

Wrapping a function to add behaviour around it: closures, functools.wraps and the debugging misery of forgetting it, decorators that take arguments, and when a decorator is the wrong tool.

Python

Twelve functions in your service call an external API. Someone asks for timing on all of them. You add two lines to each — a timestamp before, a log line after — and now twelve functions have four extra lines that have nothing to do with what they do. Next month it is retries, then a cache, then an audit trail.

The behaviour you keep adding is the same every time and belongs to none of them. A decorator lets you write it once and attach it. By the end of this lesson you will write decorators that take arguments, know the one line that stops them breaking your debugging, and know when a decorator is the wrong answer.

Functions are values

Everything here rests on one fact you have already used without naming it:

python

def shout(text):
    return text.upper()

loud = shout               # no parentheses - the function itself
print(loud("hello"))       # HELLO

A function is a value. It can be assigned, passed to another function, stored in a list, and returned:

python

def make_multiplier(factor):
    def multiply(number):
        return number * factor      # remembers `factor`
    return multiply

double = make_multiplier(2)
print(double(21))          # 42

multiply is defined inside make_multiplier and uses factor, which belongs to the enclosing call. When make_multiplier returns, factor does not disappear — the inner function keeps it. That is a closure, and it is the whole machinery a decorator needs.

A decorator

A decorator is a function that takes a function and returns a replacement:

python

import functools
import time

def timed(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        try:
            return func(*args, **kwargs)
        finally:
            elapsed = time.perf_counter() - start
            print(f"{func.__name__} took {elapsed:.3f}s")
    return wrapper

python

@timed
def fetch_photos(album_id):
    ...

The @ is shorthand. This:

python

@timed
def fetch_photos(album_id): ...

means exactly this:

python

def fetch_photos(album_id): ...
fetch_photos = timed(fetch_photos)

The name fetch_photos now refers to wrapper. Calling it runs the timing code, calls the original, and returns its result.

*args, **kwargs is what makes the decorator work on any function regardless of its parameters — collect whatever came in, pass it all through untouched.

The try/finally matters for the same reason it did in the last lesson: without it, a function that raises is never timed, so the slow failures — the ones you most want to see — are the ones missing from your logs.

The line people leave out

@functools.wraps(func) looks like ceremony. It is not.

Bad — a wrapper that replaces the function's identity.

python

def timed(func):
    def wrapper(*args, **kwargs):
        ...
    return wrapper

@timed
def fetch_photos(album_id):
    """Fetch every photo in an album."""

print(fetch_photos.__name__)    # wrapper
print(fetch_photos.__doc__)     # None
help(fetch_photos)              # useless

Good — a wrapper that keeps it.

python

def timed(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        ...
    return wrapper

print(fetch_photos.__name__)    # fetch_photos
print(fetch_photos.__doc__)     # Fetch every photo in an album.

Without wraps, every decorated function in your codebase is called wrapper. Tracebacks say wrapper, logs say wrapper, help() shows nothing, and documentation tools produce a page of identical entries. The cost lands during a production incident, when the stack trace names the decorator twelve times and none of the actual functions. One line prevents all of it, and it is the line most hand-written decorators are missing.

Decorators that take arguments

To write @retry(times=3), you need one more layer: a function that takes the arguments and returns a decorator.

python

def retry(times=3, delay=1.0, catching=(OSError,)):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(1, times + 1):
                try:
                    return func(*args, **kwargs)
                except catching as error:
                    if attempt == times:
                        raise
                    log.warning(
                        "%s failed (attempt %d/%d): %s",
                        func.__name__, attempt, times, error
                    )
                    time.sleep(delay * 2 ** (attempt - 1))
        return wrapper
    return decorator

python

@retry(times=5, delay=0.5)
def fetch_photos(album_id):
    ...

Three levels, each with a job: retry takes the configuration, decorator takes the function, wrapper takes the call. It reads as a lot of nesting the first time and is a fixed pattern you will recognise from then on.

Note catching=(OSError,) — the decorator does not retry everything, which is the rule from the foundations course applied here. A decorator that retries ValueError will call a function with bad arguments five times and fail five times.

Order matters, bottom up

Stacked decorators apply from the bottom:

python

@timed
@retry(times=3)
def fetch(): ...

retry wraps fetch, then timed wraps that — so the timing covers all three attempts. Swap them and you time each attempt separately. Both are reasonable; know which you asked for.

The ones already written

Before writing your own, check the standard library.

python

from functools import lru_cache, cache

@cache
def expensive_lookup(photo_id: str) -> Photo:
    ...

Called again with the same arguments, it returns the stored result. lru_cache(maxsize=256) bounds it; cache is unbounded — and unbounded means a memory leak if the arguments vary without limit, so prefer the bounded one for anything long- running. Arguments must be hashable, which is another place the immutability rule shows up.

python

from functools import cached_property

class Album:
    @cached_property
    def total_size(self) -> int:
        return sum(p.size for p in self.photos)   # computed once

And the ones you have already met: @property, @dataclass, @contextmanager, @staticmethod, @classmethod. Decorators are not an exotic feature; you have been using them since the foundations course.

What decorators are bad at

Three limitations worth knowing before you reach for one.

They hide control flow
A reader cannot see that a function retries, caches or logs unless they look above it and then go and read that decorator.
One is fine. Four stacked is a function whose actual behaviour is assembled elsewhere.
They complicate types
A checker follows a decorated function's signature only if the decorator was written to preserve it.
An untyped decorator quietly erases the types of everything it wraps — the Any-is-contagious problem arriving through the back door.
They are awkward to make conditional
Turning one off for a single call, or configuring it at runtime rather than import time, means working against the design.

None of these is a reason never to use one. All three are reasons not to stack four.

The test that usually decides it: is this behaviour orthogonal to what the function does? Timing, retrying, caching, logging and access checks are — you could describe the function completely without mentioning them. If the behaviour is part of what the function means, it belongs in the body:

python

@validate_photo_exists          # is this the function's job?
def caption(photo_id): ...

def caption(photo_id):
    photo = load(photo_id)      # or is it just what it does?
    ...

And a decorator applied to exactly one function is usually a worse version of two lines in that function.

A cheat sheet to keep

text

THE IDEA
  functions are values; a closure remembers enclosing names
  a decorator takes a function and returns a replacement

  @timed              is exactly
  def f(): ...        f = timed(f)

THE SHAPE
  def timed(func):
      @functools.wraps(func)          <- never omit this
      def wrapper(*args, **kwargs):
          try:
              return func(*args, **kwargs)
          finally:
              ...
      return wrapper

  functools.wraps keeps __name__, __doc__ and the signature
  without it every decorated function is called "wrapper"

WITH ARGUMENTS - three levels
  def retry(times=3):        takes the configuration
      def decorator(func):   takes the function
          def wrapper(...):  takes the call
          return wrapper
      return decorator

  @retry(times=5)

STACKING
  applied bottom up; the nearest decorator wraps first

ALREADY WRITTEN
  @cache  @lru_cache(maxsize=256)     bound it in long-running code
  @cached_property                    computed once per instance
  @property @dataclass @contextmanager @staticmethod @classmethod

WHEN NOT TO
  behaviour that is part of what the function MEANS
  applied to exactly one function
  four stacked - the behaviour now lives elsewhere
  test: is it orthogonal to the function's job?

TRAPS
  decorators run at IMPORT time - no I/O in the decorator body
  untyped decorators erase the types of everything they wrap
  a retry that catches everything retries the unretryable

Where to go next

You can now attach behaviour to functions without editing them, including configurable behaviour, and you know that wraps is what keeps your tracebacks readable. The judgement to keep is the orthogonality test — a decorator is for behaviour you could describe the function completely without mentioning.

Next is Testing with Pytest. Everything in this course so far has been about writing code that is easy to reason about; that lesson is about proving it does what you think, which is the other half — and the decorated, generator-based, context-managed code you have been writing turns out to be considerably easier to test than the alternative.

Before you move on, write the timed decorator and apply it to something real. Then delete the @functools.wraps line, trigger an exception inside a decorated function, and read the traceback. The difference between the two tracebacks is the argument for that line, and it is more persuasive than this lesson is.