advanced40 minby DevFox

Git in CI/CD Pipelines

Make Git behave inside automation: shallow detached-HEAD checkouts and their four symptoms, diffing against the right base, porcelain output, credentials, and bot commits.

Git
DevOps

The build fails with fatal: ambiguous argument 'origin/main', and nothing about your code is wrong. The same script runs fine on your laptop. It fails on the runner because the runner was handed a deliberately incomplete copy of your repository.

By the end you can say exactly what a CI workspace contains, repair it with the smallest fetch that works, diff against the base branch the way code review does, and get a pipeline reading and writing Git safely.

What the runner actually cloned

Cloning a full repository on every push is expensive, so almost every CI system takes the same three shortcuts, usually without saying so.

A shallow clone (--depth=1)
History stops after one commit. Anything that walks backwards — git describe, git log, a merge base — has nothing to walk.
A single branch
No other ref exists locally, so origin/main is not there to diff against.
A detached HEAD
Checked out at a commit rather than on a branch, because the job builds a specific SHA and not "whatever main is right now".

Being handed one page torn out of a book. It is the page you need — until you follow a footnote.

Every classic CI-Git failure is one of those three, met by code that assumed a normal clone.

Three commands tell you what you are standing in. Run them first whenever a pipeline does something inexplicable.

bash

git rev-parse --is-shallow-repository   # true = truncated history
git rev-parse --abbrev-ref HEAD         # "HEAD" = detached
git for-each-ref refs/remotes/          # which refs exist at all

The third is the most revealing: on a fresh runner it usually prints a single line.

The four symptoms, and how to recognise them

The shortcuts do not announce themselves. They surface later as four errors that look unrelated and are the same problem wearing different hats.

git describe fails with fatal: No names found, cannot describe anything. Your version string comes from the nearest tag, and a shallow single-branch fetch pulls no tags at all — so the command works locally and fails only in CI.

git log is truncated. At --depth=1 you get exactly one commit, so a changelog generator walking back to the previous release finds nothing and emits an empty changelog. This one does not error — it produces a wrong answer, which is worse.

Diffing against the base branch fails with fatal: ambiguous argument 'origin/main': unknown revision or path not in the working tree. origin/main genuinely does not exist here, because only the feature branch was fetched. Git cannot tell a missing ref from a misspelled path, so it reports the ambiguity.

git merge-base origin/main HEAD prints nothing and exits non-zero. Even when both refs exist, a shallow clone truncates history before the two lines diverged, so as far as Git can see they share no ancestor. A script doing base=$(git merge-base ...) carries on with an empty variable, and the failure surfaces somewhere else entirely.

Giving the runner the history it needs

You have two moments to fix this: ask for more at checkout time, or deepen what you already have.

At checkout time, GitHub Actions spells the lever fetch-depth, where 0 means "no limit" and also brings tags:

GitLab CI spells it GIT_DEPTH, set on the job or globally:

On a repository with a decade of history, though, 0 adds minutes to every job. Prefer the surgical version: keep the shallow checkout and fetch only what you need.

The main:refs/remotes/origin/main part is a refspec: a source:destination pair telling Git to fetch the remote's main and write it to the ref name your laptop uses. Without the destination half, the commits arrive but origin/main still does not resolve — the same error, more network used.

--deepen=N adds N more commits to an existing shallow clone; --unshallow converts it to a complete one. A --depth=50 fetch of the base branch resolves the merge base for nearly every real pull request, at a fraction of the cost of full history.

Diffing against the right base

Once the base branch exists locally, one command answers "what did this branch touch?" — and the number of dots decides whether the answer is true.

Two-dot diff compares the two commits exactly as they stand. If main moved on after you branched — and on a busy repository it moved this morning — everything that landed there shows up in your diff, reported as your change. A path filter built on that triggers jobs for code you never touched.

Three-dot diff compares HEAD against the merge base: the most recent commit the two branches share. That is the same comparison your code review UI shows, which is why the two agree and the two-dot version does not. git merge-base origin/main HEAD prints it on its own.

One more wrinkle catches people who tested only on pushes. A push build checks out your actual commit. A pull-request build on most platforms checks out a synthetic merge commit — a temporary merge of your branch into the target, existing nowhere else. On that commit, HEAD^1 is the target branch's tip and HEAD^2 is your branch head, and HEAD itself is code no human ever wrote.

Handle both shapes and you have the monorepo job filter:

Output a script can trust

Git's everyday output is a user interface for a human at a keyboard: it gains columns between versions, honours your colour config, and is translated. Parsing it is how a pipeline that worked for a year breaks on a runner image with a different locale.

The --porcelain flag means the opposite of what it sounds like: it asks for the documented, stable, machine-oriented format Git promises not to break. -z is not a nicety either — filenames can contain spaces, quotes and newlines, and NUL is the one byte a path cannot hold, so NUL separation is the only framing a filename cannot spoof.

git ls-remote queries a remote's refs over the network without creating a repository at all — the right tool for "does tag v2.1.0 exist yet?" in a job with no checkout. And git for-each-ref --format lets you name the fields you want in the order you want them, so there is nothing left to parse.

Credentials that stay out of your logs

Git asks for credentials through the credential helper interface, which is simpler than it sounds: Git runs a program, writes protocol, host and path to its stdin, and reads username and password back. Any executable following that protocol is a valid helper — which is how CI injects tokens that live for exactly one job.

The alternative you see most often is a token embedded in the remote URL — https://x-access-token:$TOKEN@github.com/org/repo. It works, and it puts the secret into git remote -v, into any set -x trace, and into the error message printed when the host is unreachable.

A third option avoids tokens entirely. A deploy key is an SSH key authorised for one repository, ideally read-only — a far narrower blast radius than a personal access token with organisation-wide reach. Select it per command:

Sometimes the repository works against you: submodules or lockfiles pin git@github.com: SSH URLs and your runner only has an HTTPS token. Rewrite the URLs rather than editing the files:

insteadOf substitutes one URL prefix for another before Git connects, so every SSH URL in the tree — including ones in .gitmodules you do not control — becomes HTTPS.

Making the clone cheap

The fastest clone is the one you already have. If your runners keep a workspace between jobs, fetch into the existing repository rather than cloning fresh, combined with the partial-clone tools from the scaling lesson.

--filter=blob:none gives you a partial clone: all commits and trees arrive, file contents are downloaded on demand. Unlike a shallow clone it keeps full history, so git log and merge-base still work — the better default for CI when the server supports it.

--no-tags matters more than it looks on a repository carrying thousands of release tags, and --prune stops a long-lived cache hoarding refs for deleted branches. git maintenance run repacks and refreshes the commit-graph so a cached clone does not decay into something slower than a fresh one.

When the pipeline commits back

Some jobs legitimately write to the repository: a version bump, a regenerated lockfile, formatted code, published docs. Done carelessly, this is how you get a pipeline that triggers itself forever.

Give the bot its own identity so git log stays readable. Set it inline with -c rather than globally, so nothing leaks into later jobs:

The [skip ci] marker is honoured by GitHub Actions, GitLab and most others, and it is your defence against the infinite loop: a build that commits, which triggers a build, which commits. Guard the job too: skip when the last commit's author is the bot, or when git status --porcelain reports nothing to commit.

Push with HEAD:main rather than a bare git push. On a detached HEAD there is no upstream to infer, and the explicit refspec says which local commit goes to which remote branch.

A cheat sheet to keep

Where to go next

You can now read a CI checkout instead of arguing with it, and hand a script output it will not misparse.

Next up is Git notes, archives, and bundles — three lesser-known tools that pair naturally with everything here. Notes attach build numbers and benchmark results to a commit after it exists, without rewriting it. git archive produces a clean release tarball with no .git directory. git bundle packs real history into one file that crosses an air gap.

Before that, put one thing from this lesson into a real pipeline this week. Take a job you already have, add the three diagnostic commands at the top, and read what they print. Most people are surprised — and that surprise is the lesson landing.