advanced35 minby DevFox

Typing at the Advanced Level

Protocols for structural typing, TypeVar and ParamSpec, overloads, variance, Self, and writing annotations that make a checker prove something rather than nod along.

Python
Types

You add a @timed decorator to a function. The checker stops complaining about that function entirely — not because it is correct, but because the decorator returns something typed Callable[..., Any], and everything downstream is now Any. One convenience wrapper switched off checking across a whole module, and nothing reported it.

The practice course covered annotations that describe ordinary functions. This lesson covers the constructs this course has been building — decorators, generic containers, protocols, overloads — where the annotation stops being obvious and a careless one silently disables the tool. By the end you will be able to type your own abstractions without erasing what they wrap.

Generics, and where the type flows

A type variable links types together across a signature:

python

def first[T](items: Sequence[T]) -> T:
    return items[0]

python

first([1, 2, 3])          # int
first(["a", "b"])         # str

That syntax is Python 3.12+. Older code declares the variable separately, and you will see both:

python

from typing import TypeVar

T = TypeVar("T")

def first(items: Sequence[T]) -> T: ...

The point is the link: T appears in the parameter and the return, so the checker knows a list of int yields an int. An annotation of -> Any would accept the same calls and tell the caller nothing.

Constraining it:

python

def largest[T: (int, float, Decimal)](items: Sequence[T]) -> T: ...

def total[T: Number](items: Sequence[T]) -> T: ...      # a bound

A bound means "this type or a subtype". Constraints mean "exactly one of these". Bounds are usually what you want.

Generic classes:

python

class Repository[T]:
    def __init__(self) -> None:
        self._items: dict[str, T] = {}

    def get(self, key: str) -> T | None:
        return self._items.get(key)

    def add(self, key: str, item: T) -> None:
        self._items[key] = item

python

photos: Repository[Photo] = Repository()
photo = photos.get("dawn.jpg")        # Photo | None

Typing a decorator

Bad — a wrapper that erases the signature.

python

def timed(func: Callable) -> Callable:
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        ...
    return wrapper

Good — a wrapper that preserves it.

python

def timed[**P, R](func: Callable[P, R]) -> Callable[P, R]:
    @functools.wraps(func)
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
        ...
    return wrapper

A bare Callable means "some function taking anything and returning anything". So after decoration, the checker will accept fetch_photos() with no arguments, with the wrong arguments, and will treat the result as Any — which then spreads to everything computed from it. The decorator did not just lose type information about itself; it disabled checking for every caller.

**P is a ParamSpec: it captures the whole parameter list as a unit, so P.args and P.kwargs reproduce it exactly. R carries the return type through. The decorated function keeps its real signature.

This matters more than it looks, because functools.wraps preserves the runtime metadata — name, docstring — and does nothing for the checker. wraps and ParamSpec are the two halves of the same repair, and most hand-written decorators have neither.

Protocols

The type system's version of "anything with these methods":

python

from typing import Protocol


class SupportsCaption(Protocol):
    name: str

    def caption(self) -> str: ...


def render(item: SupportsCaption) -> str:
    return f"{item.name}: {item.caption()}"

Nothing has to inherit from SupportsCaption. Any class with a name attribute and a caption method satisfies it, including classes from libraries you do not control and classes written before the protocol existed.

That is structural typing, and it matches how Python code actually works — the data-model lesson made the same point about protocols at runtime. A nominal base class would force every provider to import and inherit from you.

Runtime checking is opt-in and limited:

python

@runtime_checkable
class SupportsCaption(Protocol):
    def caption(self) -> str: ...

isinstance(photo, SupportsCaption)     # checks method NAMES only

It verifies that the attributes exist, not their signatures, so it is a weaker check than the static one. Use it sparingly.

Overloads

When a function's return type depends on its arguments, one signature cannot express it:

python

from typing import overload


@overload
def load(path: Path, *, parse: Literal[True]) -> dict: ...
@overload
def load(path: Path, *, parse: Literal[False] = False) -> str: ...

def load(path: Path, *, parse: bool = False) -> dict | str:
    text = path.read_text(encoding="utf-8")
    return json.loads(text) if parse else text

python

raw = load(path)                    # str
data = load(path, parse=True)       # dict

Without the overloads, both calls return dict | str and every caller has to narrow a union that is already decided. The implementation is written once; the @overload stubs describe how it looks from outside, and are checked against each other.

Order matters — the checker takes the first match, so put the more specific signatures first.

Variance, briefly

The rule that explains an error people find baffling:

python

def process(photos: list[Photo]) -> None: ...

raw_photos: list[RawPhoto] = [...]     # RawPhoto subclasses Photo
process(raw_photos)                    # error!

list[Photo] — invariant

A list[RawPhoto] is rejected.

Because process is free to append a plain Photo, and the caller believes their list holds only RawPhoto.

The checker is not being pedantic. It is stopping you writing a Photo into a list[RawPhoto].

Sequence[Photo] — covariant

A list[RawPhoto] is fine.

Sequence offers no way to add anything, so the problem above cannot arise and the substitution is safe.

The difference is not the element type. It is whether the parameter type lets you put things in.

So the fix is to ask for what you actually need:

python

def process(photos: Sequence[Photo]) -> None: ...     # read-only
process(raw_photos)                                   # fine

This is the "accept the broadest type your body handles" rule from the practice course, with the reason underneath it: if you only read, say so, and the variance follows.

Self, for methods that return their own type

Self annotates a method that returns an instance of the actual class, which matters for subclasses and for fluent interfaces:

python

from typing import Self

class Query:
    def where(self, **kwargs) -> Self:
        ...
        return self

Annotating -> "Query" instead would make a subclass's chained call come back as the base type.

Narrowing on purpose

Two constructs that let you teach the checker something it cannot infer.

python

from typing import TypeGuard


def all_captioned(photos: list[Photo]) -> TypeGuard[list[Captioned]]:
    return all(p.caption is not None for p in photos)


if all_captioned(photos):
    render(photos)             # photos is list[Captioned] here

python

def assert_valid(photo: Photo | None) -> None:
    if photo is None:
        raise ValueError("no photo")


assert_valid(photo)
photo.name                     # still Photo | None - not narrowed

python

def assert_valid(photo: Photo | None) -> TypeIs[Photo]: ...

An ordinary function that raises does not narrow, because the checker cannot know it always raises. TypeGuard and TypeIs say so explicitly — TypeIs being the newer and usually better choice, since it also narrows the negative branch.

And the escape hatch to use as rarely as possible:

python

photo = cast(Photo, raw)       # "trust me" - checked by nobody

cast is not a conversion and performs no check. It is an assertion to the checker, and it is a lie you are choosing to tell. Prefer a TypeIs function that actually looks.

A cheat sheet to keep

text

GENERICS
  def first[T](xs: Sequence[T]) -> T          links the types
  def f[T: Number](...)                       a bound: this or a subtype
  def f[T: (int, str)](...)                   constraints: exactly one
  class Repository[T]:                        a generic class
  older syntax: T = TypeVar("T")

DECORATORS
  def timed[**P, R](f: Callable[P, R]) -> Callable[P, R]:
      def wrapper(*args: P.args, **kwargs: P.kwargs) -> R:

  a bare Callable erases the signature AND makes the result Any,
  which spreads to every caller
  functools.wraps fixes the runtime metadata; ParamSpec fixes
  the types - you need both

PROTOCOLS
  class SupportsX(Protocol):
      def method(self) -> str: ...
  structural: nothing needs to inherit from it
  @runtime_checkable + isinstance checks NAMES only

OVERLOADS
  @overload stubs, then one implementation
  for a return type that depends on the arguments
  most specific first - the checker takes the first match

VARIANCE
  list[Sub] is NOT list[Base]      invariant, because it is writable
  Sequence[Base] accepts it        covariant, because it is read-only
  ask for the narrowest capability you need

NARROWING
  TypeIs[X]     narrows both branches      <- prefer
  TypeGuard[X]  narrows the positive only
  a function that raises does NOT narrow on its own
  cast(X, v)    checked by nobody; a lie you chose to tell

ALSO
  Self          a method returning its own type
  -> None       says it returns nothing, deliberately

Where to go next

You can now annotate the abstractions this course taught you to write, without the annotation quietly disabling the checker. The one to act on today is the decorator: a bare Callable in a widely used wrapper is among the most effective ways to lose type coverage across a codebase, and it looks like nothing.

Next is Writing a Library Others Depend On, where types stop being a private matter. A published API's annotations are part of its contract — changing one is a breaking change — and the lesson covers what that implies for design, deprecation and versioning.

Before you move on, find a decorator in your code and check whether it uses ParamSpec. Then run your type checker with --warn-return-any and see how much Any is flowing out of places you thought were checked. In most codebases the answer is a surprise.