advanced30 minby DevFox

Quality Gates and Release Decisions

Turning test results into a decision: which gates block, which only inform, what happens on a red main, and why an unreliable gate is worse than no gate.

Testing
CI/CD

Every test in this course produces information. A gate is where that information becomes a decision: this change may merge, this build may deploy, this release may go out.

Gates are where testing meets delivery, and where most of the friction between the two lives. By the end of this lesson you will know which checks should block and which should only inform, why an unreliable gate is worse than no gate, and how to make a release decision that is a decision rather than a feeling.

Blocking and informing

The first distinction, and getting it wrong is the source of most gate problems.

Blocking

Prevents progress.

Must be fast, reliable, and about something genuinely unacceptable.

If a production incident is waiting and this check is red, do you stop? If the honest answer is no, it must not block.

Informing

Reports, and stops nothing.

Can be slow, occasionally noisy, and about something worth knowing rather than something worth halting for.

This is also where a flaky check belongs until it is fixed.

The test for which column something belongs in: would you hold a critical fix for this?

text

BLOCK on
  compilation and type errors
  lint errors (not warnings)
  unit and integration test failures
  a required security scan at high severity
  a contract test that a consumer would fail

INFORM on
  coverage change
  bundle size change
  performance benchmark drift
  new low-severity dependency advisories
  complexity and code-smell metrics
  visual differences awaiting review

A gate that gets overridden trains everyone that gates are advisory, and that lesson is not one you can un-teach.

Where the gates are

Four places, with different costs and therefore different contents.

Gate two is where most of the value is, and gate four is the one most often missing. A deployment that succeeded and a system that works are different claims, and only a post-deploy check distinguishes them.

Making a gate real

A passing job that nobody enforces is not a gate. Three things have to be true:

The second is easy to overlook and causes a specific failure: two changes each pass on their own and break when combined. This is semantic conflict — no textual conflict, no merge marker, and a broken main branch. Requiring the branch to be current before merge is what catches it, and merge queues are the scaled-up version: each change is tested against the actual merge result, serially.

The third matters because emergencies are real. An override that exists, requires a reason, and is recorded is honest. One that is impossible gets worked around in ways nobody sees.

A red main branch is an emergency

The rule that makes everything else work: when the main branch is red, that is the team's most urgent problem.

The reason is compounding. Every subsequent pull request now inherits a failure that is not theirs, so nobody can tell whether their own change is safe, and within an hour three people are debugging the same unrelated failure. Two more changes merge on top and the original cause is buried.

"Revert first" is worth adopting explicitly, because the instinct is to fix. Fixing takes an unknown amount of time; reverting takes two minutes and returns everyone else to a working baseline.

The release decision

For anything not continuously deployed, someone decides a release goes out. That decision should rest on evidence, and the plan lesson's exit criteria are where the evidence is defined.

Bad — a decision made on confidence:

Good — a decision made on stated criteria:

The bad version has three problems beyond vagueness. "Pretty thoroughly" is not checkable, so nobody can disagree with it on evidence. "Apart from the flaky ones" means a real failure is indistinguishable from noise. And the deadline did the deciding, which means the same conversation will produce the same outcome next time regardless of what the tests found.

The good version states what was agreed weeks earlier, shows one criterion unmet with the specific defect, quantifies what remains unknown, and proposes a mitigation. Then the decision is a recorded choice by the person entitled to make it. Note that it still ships — the point of a gate is not to prevent shipping, it is to make what is being accepted visible.

Gates driven by production

The most advanced form is a gate on how the last release actually behaved, which the observability lesson set up.

An error budget policy turns reliability into a gate automatically:

That is a quality gate whose input is what users experienced rather than what a test asserted, and it has a property no test-based gate has: it cannot be gamed by writing better tests.

Two related production-driven gates worth knowing:

Automatic rollback on canary failure. If the canary's error rate or business metric is worse than the current version by a threshold, the rollout reverses without a human. Argo Rollouts and Flagger do this.

A deployment freeze window, for a genuinely risky period — a launch, a peak trading day, a holiday when nobody is on call. Worth having as a policy rather than as an argument each time.

Gates that damage delivery

Four patterns to recognise and avoid, because each converts a gate into a tax.

A gate on a metric rather than on a defect. "Coverage must be 80%" blocks a correct change for a number, and the coverage lesson explained what that produces. Gate on failures, inform on metrics.

A manual approval that is always given. A sign-off nobody has ever withheld is a delay pretending to be a control. Either it can say no, or it should be removed.

A gate slower than the work it protects. A four-hour suite on every pull request means people batch changes to avoid it, and large batches are riskier than small ones — so the gate has made delivery less safe.

A gate that reports what nobody reads. An informing check whose output nobody opens is pure cost. Either surface it where a decision is made, or turn it off.

Cheat sheet

Where to go next

The next lesson takes on the case where the fundamental assumption of every gate breaks down: systems whose output is not deterministic, where the same input gives a different answer and equality assertions stop working.

Before that, list your own blocking checks and apply the critical-fix test to each. Anything you would wave through during an incident is not really a gate, and moving it to informing makes the remaining set mean something.