Skip to content

CI/CD · Security · GitOps · 8 min read ·

A pipeline that proves what it ships

Scanning, software bills of materials, signatures and admission control are four separate controls that only mean something together. How to build a delivery path where the cluster refuses anything the pipeline did not vouch for.

Most pipelines can tell you that a build passed. Far fewer can tell you, a month later, exactly what is running in production, what went into it, whether it was scanned, and that nothing could have reached the cluster by another route. The difference is not more stages; it is a handful of controls arranged so that each one makes a claim the next one checks.

The claims, in order

A supply-chain control is a claim attached to an artifact. Arranged well, they form a chain:

  1. This image was built from this commit. The tag names the commit; the image's labels carry the commit and the build number; the build is reproducible enough that a second build of the same commit produces the same contents.
  2. This image was scanned and the scan was acted on. A vulnerability scan runs before the push, not after. The gate is a policy, not a number on a dashboard: no critical finding with a fix available, and every exception written down with an owner and an expiry.
  3. This is what is inside it. A software bill of materials, generated from the image rather than the source tree, so it describes what actually shipped, including what the base image brought with it.
  4. We said so. The image, the bill of materials and the scan report are signed by the pipeline's identity, not by a key a person holds. Keyless signing with a workload identity means the signature names the workflow, the repository and the branch that produced it.
  5. Nothing else runs. Admission control in the cluster verifies the signature and the attestations before a pod is scheduled, and refuses anything that lacks them.

Each step on its own is a checkbox. Together they let you answer the auditor's question with a query rather than a meeting.

Why the scan comes before the push

Scanning after the push is the common shape, because the registry integrates with a scanner and it is one fewer stage to maintain. It means a critical finding exists in the registry, pullable, for as long as it takes someone to read the report. Scanning in the pipeline, with the push conditional on the gate, means the finding never becomes an artifact. The exception file lives in the repository next to the code it excuses, is reviewed like code, and the build fails when an exception has expired.

Immutable tags, and why latest is a bug

A floating tag is a promise that the thing it points to will change. Every control above attaches to a digest; a floating tag quietly moves the deployment away from the thing that was verified. Tag each build with something ordered and unique (a build number and a short commit hash), never rewrite a tag, and have the deployment tooling resolve a tag to a digest at promotion time and pin that.

Promotion is a separate decision

A green build is not a deployment decision. The build produces a candidate; a promotion controller moves the candidate from environment to environment, recording what was promoted, from where, by what policy, and whether the environment's verification passed. The verification is a real test of the running environment, not a repeat of the unit tests: a smoke run through the public edge, a probe that one tenant cannot see another, an analysis of error rates during a canary. If the promotion controller writes the image digest into the environment's configuration in Git, every environment's state is a commit, and rolling back is promoting the previous one.

Where it usually goes wrong

  • The pipeline's identity is a long-lived secret. Federate it. A pipeline that authenticates to the registry and the cloud with a token issued for that run cannot leak a credential that lasts.
  • Admission is enabled in audit mode and never enforced. Audit mode is where it should start and where it will stay unless a date is set. Set the date.
  • The exception file is a graveyard. Every exception needs an expiry, and the build must fail when one lapses. Otherwise the gate is a formality within a quarter.
  • Someone can push to the registry by hand. Then the signature proves only that some images came from the pipeline. Registry write access belongs to the pipeline identity alone; humans get read.
  • The bill of materials is generated and never read. Its value arrives the day a widely used library is found vulnerable and you need to know, in minutes, which running images contain it. Index the bills of materials so that question is a query.

What it costs

A scan, a bill of materials and two signatures add a few minutes to a build. Admission verification adds milliseconds to pod scheduling. The real cost is the first month, when the exception file is long and the base images need updating; after that the gate is quiet because the images are clean, which is the point.

The shape above is what we run for our own platform: every image scanned before it is pushed, signed keyless by the workflow's identity with its bill of materials and scan report attached, promoted by digest through verified rings, and refused by the cluster without the signature. A DevSecOps engagement typically starts by measuring which of the five claims an existing pipeline can already make.