advanced45 minby DevFox

Rewriting History at Scale

A runbook for purging a secret or a huge binary from every commit with git filter-repo, extracting a subdirectory, and coordinating the force-push without breaking the team.

Git
Security

A teammate pushes an .env file with a live cloud key. Someone commits a 400 MB render that has sat in every clone for two years. Legal asks you to delete a vendor's file from a 2016 repository. No new commit fixes these — the bytes are still in history, one git show away from anyone who has cloned you.

By the end of this lesson you can purge a secret or a huge binary from every commit, split a subdirectory into its own project with its history intact, and run the whole thing as a coordinated operation that does not break your team's clones.

When a whole-history rewrite is justified

Rewriting every commit is a destructive migration, closer to a schema change with downtime than to a normal Git operation. Three cases justify it.

A leaked credential

Committed and pushed.

And the rewrite is cleanup, not remediation — the next section is about why.

A file that must not exist

Legal, or simply too large.

A licence violation, a takedown, a deletion request — or a few blobs that make every clone unusable.

Restructuring repositories

Splitting or merging projects.

Extracting a directory into its own repository, or combining two while keeping both histories.

That is the whole list. "The history is ugly" is not on it.

Cosmetic complaints do not justify a cost that everyone pays.

The blast radius, before you touch anything

A commit's hash covers its tree, message, author, and parents. Change one commit and its hash changes, which changes the next commit's parent, which changes that hash too. Every commit after the earliest change gets a new identity.

Think poured concrete, not stacked blocks: you cannot chip one piece out, you re-pour what sits above it.

Open pull requests
Their base and head commits no longer exist server-side, so hosts show them as wildly conflicted.
Tags, release tags included
They point at commits that are gone, unless you re-push them too.
Every clone, plus CI caches
Including the laptop of whoever is offline today, who will come back and push the old history straight into the repo.
Every recorded hash
Tracker links, changelog SHAs, pinned commits in deploy scripts, submodule pointers, blame permalinks.
Signed commits
Their signatures no longer verify, because the thing that was signed no longer exists.

Walk this list before you run anything. Every item is somebody else's afternoon.

Rotate the credential first

If the reason is a leaked secret, the rewrite is not the fix. It was pushed to a server, fetched by CI, cloned by people who have since left. You cannot un-distribute it.

The host does not forget on your schedule either. On GitHub a commit unreachable from any branch or tag is still served by its hash until garbage collection runs, on their timetable — so anyone holding the SHA still reads the blob after your rewrite. Purging it for certain takes a support request.

The tool is git filter-repo

git filter-repo is the tool for this job: a rewriter built on git fast-export, fast and correct where it counts — tags get rewritten, merges handled, empty commits pruned. It is not part of core Git, so install it:

For the narrow "delete these files, scrub these strings" case, BFG Repo-Cleaner is a simpler alternative: a small Java tool (--delete-files, --replace-text, --strip-blobs-bigger-than 100M) that refuses to touch the commit HEAD points at. It cannot rename paths or extract directories, so structural work stays here.

Run filter-repo in your everyday working copy and it stops: this is not a fresh clone. That is deliberate, because a working copy holds state the rewrite cannot fix — stashes, reflogs, extra remotes, worktrees — any of which can resurrect the blob.

--no-local copies objects rather than hardlinking into the source, so a mistake in the rewrite cannot damage the original.

Removing a file from every commit

--invert-paths flips --path from "keep only this" to "drop it":

Paths match from the repository root. For patterns use --path-glob; any number of path options combine:

Quote globs so the shell does not expand them locally. Run git filter-repo --analyze first: it ranks blobs by size under .git/filter-repo/analysis.

Redacting a secret in place

Sometimes the file stays and only one string inside it must go — a key in a config sample, a password in a fixture. --replace-text takes a rules file, one rule per line:

A bare line is a literal. Without ==> the match becomes ***REMOVED***; with it, the text after the arrow is used.

Be honest about what this does: it changes file content in every historical commit. A build from an old tag now compiles different bytes and a test asserting on that literal fails, so if you pin old revisions, re-check them before pushing.

Extracting a subdirectory into its own repo

--subdirectory-filter keeps one directory and lifts it to the root:

What about commits that never touched packages/ui? After filtering, their tree matches their parent's, so filter-repo prunes them by default — as it does merge commits whose sides collapse to one tree. You keep the commits that changed the UI package, authors and dates intact.

To land the code somewhere other than the root, add a rename:

Merging repositories reverses the manoeuvre: rewrite the incoming project into its target subdirectory, then merge with --allow-unrelated-histories.

Fixing author and committer identities

A mailmap file maps old identities to new ones, one per line, and applying it rewrites the author and committer of each commit:

The coordinated rewrite runbook

Run it in this order.

Rotate any leaked credential. Nothing below substitutes.
Freeze the repository. Announce a window and lock the branches in your host's settings rather than trusting everyone to read the email. Ask people to push or drop open work.
Notify with specifics — the time, the reason, the commands each person runs afterwards. Surprised people force-push the old history back.
Rewrite on a fresh clone.
Verify before pushing: shape, absence, size.
Push every branch and every tag.
Unfreeze, reopen PRs, purge caches. SHA-keyed CI caches are stale, and only your host can collect the unreachable objects.

Every teammate now holds a mismatched history. The safe answer is a fresh clone; for in-flight work, replant the branch:

That replays the commits after <old-main-sha> — the last commit shared with the old main — onto the new one. Nobody should run git pull after a rewrite: it merges the two histories and restores every commit you deleted.

Grafts and git replace: rewriting nothing

git replace records a substitution — "when you read commit A, use commit B" — leaving objects untouched and hashes valid, and travelling only if you push refs/replace/* deliberately. The classic use is stitching a truncated history onto an archived one: your repo opens with an import, the prehistory lives elsewhere.

git log, git blame and git bisect all follow the graft, since replacement is on by default (--replace-objects). To see the raw truth, run git --no-replace-objects log or set GIT_NO_REPLACE_OBJECTS=1. Being a view rather than a change, grafting is right when the goal is readability and wrong when the goal is that bytes stop existing.

Confirming the object is really gone

filter-repo expires reflogs and repacks for you; in a clone you rewrote with --force, do it yourself:

git gc --prune=now drops unreachable objects immediately rather than honouring the usual two-week grace period, and it permanently destroys anything not reachable from a ref, including work you stashed and forgot. Set the resulting size-pack against the number you recorded before: 400 MB down to 40 MB is your receipt — proof your copy is clean, not the internet's.

A cheat sheet to keep

Where to go next

You can now strip anything from a repository's history, split a project out of a monorepo, and coordinate the migration without stranding your team. The judgement is the harder half: most requests to rewrite history are best answered with "no, and here is the .mailmap or .gitignore that solves it."

Next comes merge strategies, drivers, and rerere — how Git decides what a merge means, how to teach it to resolve project-specific files on its own, and how to stop re-solving the same conflict on every rebase. Before that, practise on something disposable: clone a public repository, strip its largest blob, compare git count-objects -vH, then graft the old history back on. Doing it once where nobody depends on the result is what makes it calm the day it matters.