advanced40 minby DevFox

Submodules and Subtrees

Several repositories, one build. Gitlinks and .gitmodules, the full submodule lifecycle and its honest list of pain, subtrees as the alternative, and when a registry beats both.

Worktrees solved one repository, several working directories. This lesson is the mirror image and the harder problem: several repositories, one build. A shared protocol definition, a design system owned by the team next door, a fork you patch twice a year — code that belongs to someone else's history but must be present in your checkout for anything to compile.

By the end you can read a submodule as what it actually is, one line in a tree, drive its whole lifecycle without leaving broken clones behind, resolve the conflicts it causes, and say with confidence when a subtree, a monorepo, or a package registry is the better answer.

Pointer or copy: the only two native answers

Git has exactly two built-in answers, and they are opposites.

A submodule stores a pointer

A bibliography entry.

Your repository records "we build against commit 9e1f4d of that other repository" and no file contents at all.

An exact citation of an exact edition — and useless until the reader goes and fetches the book.

A subtree stores the files

An appendix, bound in.

Merged into a subdirectory of yours and committed as ordinary blobs.

Heavier, and nobody needs a library card to read it — a plain git clone gets everything.

Every flag and failure mode in this lesson follows from this one choice

A submodule is one line in a tree

Strip away the tooling and a submodule is startlingly small. Lesson 1 showed a tree object listing entries with a mode, a type, an object id, and a name. A submodule is one such entry, mode 160000 — a gitlink. Look at a real one:

bash

git cat-file -p HEAD^{tree}

text

100644 blob 5f2a1c8e...  .gitmodules
040000 tree 3f0c7a91...  src
160000 commit 9e1f4d3a...  vendor/protocol

Every other line points at content Git can hand you. The 160000 commit line points at a commit that does not exist in this repository's object database at all — an address in a foreign country. The committed .gitmodules file supplies the postal service:

ini

[submodule "vendor/protocol"]
	path = vendor/protocol
	url = https://github.com/acme/protocol.git
	branch = main

Note what the gitlink is not: a branch name. Git records a commit id, and it stays frozen until a human commits a new one. Check out a two-year-old commit of the parent and git submodule update reconstructs exactly the dependency that existed that day. That reproducibility is the entire selling point, and the reason submodules never update themselves — one property seen from two sides.

The submodule lifecycle

Adding one writes both halves, the gitlink and the .gitmodules entry, and leaves them staged for you to commit:

The optional branch = main line, written by git submodule set-branch, tells --remote below which branch to follow instead of the remote's default.

Cloning is where most people meet their first empty directory: a plain git clone creates vendor/protocol and leaves it bare, because the gitlink names a commit it never fetched.

--init copies the URLs from .gitmodules into .git/config — which is why editing a URL upstream changes nothing in existing clones until someone runs git submodule sync --recursive. --recursive repeats the process for submodules that themselves have submodules. Day to day, three verbs carry the weight:

Read status output carefully, because the first character of each line is the whole message:

A space means the checkout matches the recorded gitlink; - means the submodule is not initialised — the empty directory; + means it sits at a different commit than the parent records, which is what you see after --remote or after editing inside it. That + is a change waiting to be committed: --remote moves only the working copy, and staging the path makes the new pointer real.

Removing a submodule, all three parts

A submodule's state lives in three places, so removal takes three steps:

deinit empties the working directory and clears the submodule.vendor/protocol.* keys from .git/config. git rm removes the gitlink from the index and the stanza from .gitmodules, staging both. Neither touches .git/modules/, where Git keeps the submodule's own object database so deinit and re-init stay cheap.

That third line is the one that bites. Leave the directory behind and a later git submodule add at the same path fails with a git directory for 'vendor/protocol' is found locally — or silently reuses those stale objects, giving you a submodule pointing at a fork nobody remembers cloning.

The config that makes submodules bearable

Git's submodule defaults are hostile, dating from when submodules were rare and explicit. Four settings fix them:

submodule.recurse makes pull, switch, checkout, and reset update submodules automatically, as if you passed --recurse-submodules every time. Without it, switching from a branch that pins 9e1f4d to one that pins a13c9f leaves the submodule on the old commit and your build quietly wrong.

diff.submodule log turns the useless default hunk — -Subproject commit 9e1f4d… over +Subproject commit a13c9f… — into the list of commits crossed, so a pointer bump becomes reviewable; status.submoduleSummary does the same for git status. And push.recurseSubmodules=on-demand pushes the submodule commits the parent depends on first.

The honest list of submodule pain

The first cost surprises everyone. Because the parent pins a commit and not a branch, git submodule update checks that commit out detached — HEAD points straight at a commit, with no branch attached. That is logically right, but it means edits you make inside land on no branch, and the next git submodule update moves HEAD away and leaves them reachable only through the reflog. Switch to a real branch before you edit anything.

The second is the unpushed pointer above, worth stating as a rule: the submodule is always pushed first. The gitlink is a promise that a commit is fetchable; make it true before you make the promise.

The third is CI. Every runner needs credentials for every submodule URL, not only the parent's. A private submodule over SSH fails on a fresh runner with a public key error even though the parent cloned fine, because the runner's token was scoped to one repository.

Resolving a gitlink conflict

Two branches move the submodule pointer to different commits, you merge them, and Git says:

This looks alarming and is usually trivial. Git cannot merge two commit ids — there is no text to combine. If one side's commit is an ancestor of the other's and both are present locally, Git fast-forwards the submodule for you; otherwise it stops and hands you the decision. Start by reading what the two sides point at, straight out of the trees:

Then go inside and resolve it there, as an ordinary Git problem in an ordinary repository — because that is what it is:

The key mechanic is that last line. git add on a submodule path stages no file contents; it reads whatever commit the submodule is checked out at right now and writes that id into the parent's index. Whatever you leave HEAD at inside is what the parent records.

One trap: resolving with a new merge commit inside the submodule leaves that commit on your machine alone. Push the submodule before you conclude the parent's merge, or you have built an unfetchable pointer on purpose.

Subtrees keep the files, not the pointer

A subtree takes the opposite bet: instead of an address, it merges the other repository's files into a subdirectory of yours as real tracked content.

The remote alias is optional, but every later command wants the same URL. --prefix is the subdirectory, and must be spelled identically forever after; subtree uses it to find its own previous work.

--squash is the flag worth understanding. Without it, every commit of the imported project's history is grafted into your log — for a mature library, thousands of entries in someone else's commit style. With it, Git writes one commit holding the whole imported tree, plus a merge commit whose message carries git-subtree-dir: and git-subtree-split: trailers. Those trailers are how the next pull finds where the last import stopped, which is why --squash must be passed every time once you have started.

pull brings in upstream changes. split synthesises a branch holding only the commits that touched vendor/lib, rewritten so the prefix is the repository root — a standalone history you can push anywhere. push is split plus a push, and it is slow on a large repository, because it walks every commit that ever touched the prefix.

The trade is clean and honest. Because the files are genuinely yours, git clone works with no flags, no init step, and no second set of credentials; a contributor who has never heard the word "subtree" can build the project. The cost is repository size — you carry the vendored code in every clone forever — and an awkward path upstream, since push rewrites history to produce something upstream can accept.

Choosing, with your eyes open

Approach	Problem it solves	Clone	Upgrade friction
Worktree	Many branches at once	Same repo	None
Submodule	Exact pinned pointer	Flags + extra auth	Manual per bump
Subtree	Vendored source in-tree	Plain clone	One command
Monorepo	Actively co-developed code	Plain, but large	None
Registry	Released, versioned deps	Plain clone	Tooling-managed

The opinionated version. For anything that ships releases, use a package registry — npm, PyPI, Cargo, Maven and their lockfiles were built for versioned dependencies and do it better than Git ever will, including transitive resolution, which neither submodules nor subtrees attempt.

For code you actively co-develop, where one change spans both repositories in an afternoon, use a subtree or a monorepo. Atomic commits beat repository purity, and the review showing both halves together is the one that catches the bug.

Reach for submodules when what you need is precisely what they are: an auditable, pinned pointer to a repository you control — infrastructure definitions, a fork you patch rarely, a compliance artefact where "exactly this commit" is the requirement. Outside that narrow band they mostly generate support requests from your own teammates.

A cheat sheet to keep

Where to go next

You can now read a gitlink out of a raw tree, take a submodule through its whole life without stranding a colleague on a pointer they cannot fetch, and argue from mechanism rather than folklore about how outside code enters a repository.

The natural next step is scaling Git for large repositories — partial clone, blobless and shallow fetches, sparse-checkout, and git maintenance. It follows directly from here: vendoring with subtrees is one of the fastest ways to make a repository big enough to need those tools.

Before you move on, run git cat-file -p HEAD^{tree} on a project with submodules until the 160000 line looks ordinary. Then break one on purpose in a scratch clone: commit inside the submodule, push only the parent, and clone it fresh elsewhere. Seeing that failure once beats reading about it ten times.