Packaging Your Project
pyproject.toml, the src layout, editable installs and console entry points — turning a folder of scripts into something that can be installed, imported and shipped.
pyproject.toml, the src layout, editable installs and console entry points — turning a folder of scripts into something that can be installed, imported and shipped.
A colleague wants to use your photo tool. You send them the
folder. They run it from the wrong directory and get
ModuleNotFoundError: No module named 'photo_tools'. They move
it, and now it works — until they try to use one of your
functions from their own script, from somewhere else on disk,
and the import fails again.
The problem is that a folder of files is not installable.
Nothing tells Python where it lives, what it is called, or what
it needs. By the end of this lesson your project will install
with pip install, expose a command anyone can run, and use the
layout that makes a specific and nasty class of bug impossible.
Right now, import photo_tools works only when Python happens
to be looking at the folder above it — which means it depends on
your current directory, and breaks the moment anything runs from
elsewhere. Tests, scheduled jobs and other people's scripts all
run from elsewhere.
Installing a package puts it somewhere Python always looks. Then
import photo_tools works from any directory, in any script, in
your tests, in a container.
The file that makes a project installable is pyproject.toml.
One file at the root of the project, declaring what this is:
[build-system]
requires = ["setuptools>=68"]
build-backend = "setuptools.build_meta"
[project]
name = "photo-tools"
version = "0.1.0"
description = "Caption photographs from their filenames."
readme = "README.md"
requires-python = ">=3.11"
license = { text = "MIT" }
authors = [{ name = "Ana Duarte", email = "ana@example.com" }]
dependencies = [
"pillow>=10.0",
]
[project.optional-dependencies]
dev = ["pytest>=8.0", "ruff>=0.5"]
[project.scripts]
photo-tools = "photo_tools.main:main"Three sections do the real work.
[build-system] names the tool that turns your source into
something installable. setuptools is the default and fine;
hatchling and flit are common alternatives.
[project] is the metadata. name is what people
pip install; dependencies is what gets installed alongside
it. Note that the install name uses hyphens (photo-tools)
while the import name uses underscores (photo_tools) — they do
not have to match, and the difference confuses people forever.
[project.scripts] is the one that feels like magic. That
line creates a command called photo-tools that runs the
main function in photo_tools/main.py. After installing,
your user types photo-tools photos/ rather than python -m photo_tools.main photos/.
Where you put your package matters more than it looks.
Use the one on the right, and the reason is precise.
Imports find the folder.
photo_tools sits in the directory you run commands from,
and Python looks there first — so import photo_tools works
whether or not the package is installed.
Your tests exercise the folder, pass, and say nothing about a
file you forgot to include or a subpackage missing its
__init__.py. The user gets ModuleNotFoundError.
Imports can only find the install.
There is no photo_tools in the current directory, so an
import succeeds only if the package is genuinely installed.
Your tests exercise what ships.
-e means editable: instead of copying files, it points the
installation at your source directory. Edit a file and the next
run uses the change, with no reinstall. That is what you want
while developing.
".[dev]" installs the optional group from earlier, so a new
contributor gets pytest and the linter with the same command.
The whole project setup becomes:
Four lines in your README, and a new starter is running your tests. Compare that to the version where they have to guess.
Semantic versioning gives the three numbers meaning:
The contract is entirely about the third case: a major bump is
you telling users their code may break. That is what lets them
depend on >=2.4,<3 and upgrade minor releases without reading
anything.
Which makes the specifier you write for your dependencies a real decision:
Bad — an exact pin in a library you publish.
Good — a compatible range.
An exact pin in a library is not caution, it is a conflict waiting to happen: anyone who depends on your package and anything else using Pillow now has an unsatisfiable requirement, and pip refuses to install. A library declares the range it works with and lets the application resolve it. Exact pins belong in the lock file of an application, which is the distinction the previous lesson drew — applications pin, libraries constrain.
Two artefacts come out of a build:
A wheel is the built form — installing it is unpacking, with no build step, which is why it is fast. A source distribution is your source, built on the user's machine if no suitable wheel exists. Publish both.
Publishing goes to PyPI, the index pip install reads by
default:
Rehearse on TestPyPI first. A version number on PyPI can never be reused or replaced — you can only yank it and publish a new one — so the cost of finding a mistake afterwards is a permanent record of it.
Packaging is also the point where a project starts looking like one:
The README is the single highest-value file. What this does, how to install it, and the four lines that get someone from clone to running tests. Everything else can be discovered from the code; that cannot.
The LICENSE is not optional if anyone else will touch this. Code with no licence is code nobody has permission to use, and "it's on GitHub" grants nothing.
Your project is now installable, runnable as a command, and laid out so your tests exercise what you actually ship. The application-pins-libraries-constrain distinction is the one to carry forward; it explains most dependency conflicts you will ever have to untangle.
Next is Type Hints That Earn Their Keep. Packaging made your code usable by other people; type hints make it checkable — by a tool, before it runs, and by a reader who wants to know what a function expects without reading its body.
Before you move on, take the photo tool from the last course and
package it. Move it under src/, write the pyproject.toml,
add a [project.scripts] entry, and install it with -e. Then
change directory to somewhere completely unrelated and run the
command. Watching your own tool work from a folder that knows
nothing about it is the moment packaging stops being paperwork.
project/ project/
├── photo_tools/ ├── src/
│ └── __init__.py │ └── photo_tools/
├── tests/ │ └── __init__.py
└── pyproject.toml ├── tests/
└── pyproject.toml
"flat" layout "src" layoutMAJOR.MINOR.PATCH 2.4.1
PATCH a bug fix, nothing else changed
MINOR new things added, old things still work
MAJOR something that used to work no longer doesdist/
├── photo_tools-0.1.0-py3-none-any.whl a wheel
└── photo_tools-0.1.0.tar.gz a source distributionproject/
├── src/photo_tools/
├── tests/
├── pyproject.toml
├── README.md what it is, and how to run it
├── LICENSE without this, nobody may legally use it
├── CHANGELOG.md what changed in each version
└── .gitignore .venv/, __pycache__/, dist/, *.egg-info/THE FILE
pyproject.toml at the root
[build-system] which tool builds it
[project] name, version, dependencies, requires-python
[project.optional-dependencies] dev = [...]
[project.scripts] command = "package.module:function"
install name uses hyphens, import name uses underscores
LAYOUT - use src
src/photo_tools/ cannot be imported unless installed
tests/
flat layout lets tests import the folder, so they pass
while the shipped package is broken
INSTALLING
pip install -e . editable - edits take effect at once
pip install -e ".[dev]" plus the dev extras
VERSIONS
MAJOR.MINOR.PATCH
PATCH bug fix MINOR additive MAJOR breaks something
applications PIN exact versions in a lock file
libraries CONSTRAIN ranges: >=10.0,<11
an exact pin in a library is an unsatisfiable conflict
BUILD AND PUBLISH
python -m build -> dist/*.whl and *.tar.gz
twine check dist/*
twine upload --repository testpypi dist/* rehearse
twine upload dist/* permanent
IN THE REPOSITORY
README.md LICENSE CHANGELOG.md .gitignore
no LICENSE means nobody may legally use itpython -m pip install -e . # editable, from this folder
python -m pip install -e ".[dev]" # and the dev extraspython3 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
pytestdependencies = ["pillow==10.3.0"]dependencies = ["pillow>=10.0,<11"]python -m pip install build
python -m buildpython -m pip install twine
python -m twine upload --repository testpypi dist/* # rehearse
python -m twine upload dist/* # for real