advanced35 minby DevFox

Writing a Library Others Depend On

Designing a public surface you can live with: what to expose, deprecation that gives people time, semantic versioning applied honestly, and shipping types with your package.

Python

You rename a parameter from folder to directory, because directory is clearer. The tests pass. The release goes out. Within a day there are eleven issues open, because everyone who wrote process(folder=...) — the keyword form your own README recommended — is broken.

Once other people import your code, every visible name is a promise. By the end of this lesson you will know which parts of a library are actually public, how to design a surface you can live with, how to remove something without breaking people, and what semantic versioning obliges you to do.

Everything visible is public

Unless you say otherwise, users will import anything they can reach, and they will do it because they needed something and it was there.

Say otherwise, in three ways.

A leading underscore marks a name as internal. It is a convention rather than a mechanism, and it is understood universally:

python

def _normalise(name: str) -> str: ...      # not yours to call

__all__ declares what the module exports:

python

__all__ = ["make_caption", "Photo", "UnsupportedFormatError"]

It controls from module import *, and — more usefully — it is what documentation tools and linters read to decide what is public. Being absent from __all__ is a much clearer signal than being present in the file.

The package's __init__.py is the real front door:

python

# photo_tools/__init__.py
from photo_tools.captions import make_caption
from photo_tools.errors import UnsupportedFormatError

__all__ = ["make_caption", "UnsupportedFormatError"]

Users write from photo_tools import make_caption and never learn which module it lives in — which means you can move it later. That indirection is the single cheapest piece of future freedom available, and it costs two lines.

The corollary, from the import lesson: keep __init__.py cheap. Re-exports are free; work is paid on every import by everyone.

Designing the surface

Keyword-only arguments are the most valuable habit here:

python

def process(folder: Path, *, limit: int = 100,
            dry_run: bool = False) -> Report: ...

Everything after * must be passed by name. That means you can reorder them, add new ones anywhere, and rename nothing without breaking anyone — and the call site reads process(path, limit=50) rather than process(path, 50, False).

Positional parameters are a commitment to their order, and keyword parameters are a commitment to their names. Choose which commitment you want:

python

def process(folder: Path, /, *, limit: int = 100) -> Report: ...

/ makes folder positional-only, so its name is free to change. Use it for the obvious first argument, where nobody would pass it by keyword anyway.

Return something extensible. A tuple is fixed forever:

python

def analyse(path: Path) -> tuple[int, int]:      # count and size
    ...

Adding a third value breaks every count, size = analyse(path). A dataclass does not:

python

@dataclass(frozen=True)
class Analysis:
    count: int
    total_size: int
    # added later; nothing breaks
    skipped: int = 0

Do not return internal objects. Handing back your own mutable list lets a caller modify your state, and it commits you to that type. Return a copy, a tuple, or a frozen view.

Raise your own exceptions, with a package base class, as the errors lesson set out. ValueError from your library is indistinguishable from ValueError from anything else in the call.

What semantic versioning obliges

text

MAJOR   something that used to work no longer does
MINOR   new capability, everything old still works
PATCH   a fix, nothing else changed

The obligation is entirely in the first line: a major bump is you telling users their code may break. That is what makes >=2.4,<3 a safe thing for them to write.

What counts as breaking is broader than people expect:

text

breaking   removing or renaming anything public
           renaming a parameter (the keyword form is an API)
           making an optional parameter required
           narrowing an accepted type
           widening a RETURN type
           changing an exception type that is raised
           changing documented behaviour
           raising the minimum Python version

not        adding an optional keyword-only parameter
           adding a new function or class
           widening an accepted type
           narrowing a return type
           any change entirely behind an underscore

Two of those surprise people. Widening a return type is breaking, because callers wrote code for the narrower one — returning Photo | None where you used to return Photo breaks everyone who did not check. And type annotations are part of the contract: a user running a type checker in CI has their build fail on your annotation change, which is a breaking change delivered in a patch release.

Removing something

Bad — removing it in the next release.

python

# 2.4.0
def process_folder(path): ...

# 2.5.0 - process_folder is gone, use process() instead

Good — deprecate, warn, then remove in the next major.

python

# 2.5.0
import warnings
from typing import deprecated


@deprecated("Use process() instead; removed in 3.0")
def process_folder(path: Path) -> Report:
    warnings.warn(
        "process_folder() is deprecated and will be removed in "
        "3.0; use process() instead.",
        DeprecationWarning,
        stacklevel=2,
    )
    return process(path)

Removing a public name in a minor release breaks users who followed the contract — they pinned >=2.4,<3 precisely because you promised that was safe, and their build breaks on an upgrade they were entitled to make automatically.

The deprecation path gives them a release where their code still works and tells them what to do. Three details make it effective:

stacklevel=2
Points the warning at their line rather than at yours. Without it the message names a file they have never opened.
A message naming the replacement
And the version it disappears in. "Deprecated" on its own tells nobody what to do next.
The @deprecated decorator
From 3.13, or typing_extensions. It makes a type checker flag the call, which is how the people who have warnings switched off find out.

Leave any one of these out and the deprecation reaches fewer people than you think

Then actually remove it at the major version. A deprecation that never completes teaches people to ignore warnings.

Shipping types

Annotations do nothing for your users unless you tell packaging they exist:

text

src/photo_tools/
├── __init__.py
├── captions.py
└── py.typed          <- an empty marker file

toml

[tool.setuptools.package-data]
photo_tools = ["py.typed"]

Without py.typed, checkers treat your package as untyped and every call into it produces Any — the contagion from the typing lesson, applied to everyone who depends on you.

Then keep the annotations honest, because they are now a contract you can break.

The documentation that matters

Three things, in order of value.

A README that gets someone running in four lines. Install, the smallest useful example, and where to look next. Most readers never go further.

A CHANGELOG, written for humans. What changed, what broke, what to do about it — grouped by version, newest first. This is what someone reads when deciding whether to upgrade, and a generated commit log does not answer that question.

Docstrings on everything public. Including the exceptions raised, which are the part people omit and the part callers need in order to handle them.

Then the practices that make all of it survive:

text

test on every Python version you claim to support
run your own README examples in CI
put your public API in a test, so a rename fails a test
keep a deprecation policy in writing

That third one is worth doing literally:

python

def test_public_api_is_unchanged():
    assert set(photo_tools.__all__) == {
        "make_caption", "process", "Photo", "PhotoToolsError",
    }

It looks trivial and it turns an accidental removal from a released bug into a failing test.

A cheat sheet to keep

text

WHAT IS PUBLIC
  everything reachable, unless you say otherwise
  _leading_underscore    internal, by convention
  __all__ = [...]        the declared exports
  __init__.py re-exports the front door, so modules can move
  keep __init__.py cheap - everyone pays for it

DESIGNING
  def f(path, /, *, limit=100)
    positional commits to ORDER; keyword commits to NAMES
    `*` for everything optional - reorder and add freely
    `/` for the obvious first argument - frees its name
  return a dataclass, never a tuple - tuples cannot grow
  never return your own mutable internals
  raise your own exceptions, under a package base class

SEMVER - what breaks
  breaking  remove/rename anything public
            rename a parameter (the keyword IS the API)
            make an optional parameter required
            narrow what you accept
            WIDEN what you return          <- surprises people
            change an exception type, or documented behaviour
            raise the minimum Python
            change an annotation           <- it is a contract

  safe      add an optional keyword-only parameter
            add new functions or classes
            widen what you accept, narrow what you return
            anything behind an underscore

REMOVING
  deprecate in a minor, remove in the next MAJOR
  warnings.warn(..., DeprecationWarning, stacklevel=2)
    stacklevel=2 points at the CALLER's line
  @deprecated so type checkers flag it
  name the replacement and the removal version
  DeprecationWarning is hidden by default - use the changelog too

TYPES
  an empty py.typed file, included as package data
  without it, every call into you is Any for your users

DOCS AND HABITS
  README: install + smallest example, in four lines
  CHANGELOG for humans, newest first
  docstrings including what is RAISED
  test every version you claim; run README examples in CI
  assert on __all__ so a rename fails a test

Where to go next

You can now decide what is public and defend it, design signatures that leave room to change, remove things without breaking people, and ship annotations that help rather than mislead. The habit worth adopting immediately is keyword-only parameters — one asterisk, and most future additions stop being breaking changes.

Next is Distributing and Versioning, the mechanics beneath this. Wheels and source distributions, what native dependencies do to your build matrix, publishing safely, and the supply-chain responsibilities that come with being something other people install.

Before you move on, take a module you have written and write down its public API — the names you would be obliged to keep. Then compare that list with what is actually importable. The gap between the two is the surface you did not intend to promise.