Your Tests Passed. Which Revision Is Running?

Do not mark a release verified until the evidence connects the tested candidate to the deployment and the production route you checked.

Consider a synthetic release. An agent tests candidate B successfully. It then checks the production homepage, which returns a healthy response from deployment A. Both checks passed. The new code has never served that request.

The review bundle I described in September connects a requirement to evidence for a candidate. Releasing that candidate introduces further transitions: source becomes an artifact, an artifact becomes a deployment, and a route directs users to it. Each transition needs its own evidence.

Follow the artifact

A commit identifies source. The release process also needs to identify what it built from that source and what the tests exercised. Tests against a checkout and tests against the packaged application answer different questions; record which one ran.

For containers, Kubernetes documents the distinction between tags and digests: a tag can move to a different image, while a digest identifies immutable image content. Recording app:release leaves the image ambiguous after the tag moves. Recording the resolved digest makes the selected content identifiable.

Where a hosted platform supplies a deployment identifier rather than an accessible artifact digest, record that identifier and its build metadata. An empty field labelled “digest unavailable” is more useful than an invented equivalence between a Git SHA and the deployed bytes.

GitHub artifact attestations can connect an artifact to signed provenance claims, including the workflow and source commit. Those attestations must be verified to provide their benefit. GitHub also explicitly separates provenance from a guarantee that the artifact is secure.

Keep the test evidence beside the provenance. Knowing where an artifact came from does not establish that its publication rules work, or that a particular test suite ever exercised it.

Check the route users reach

A deployment-specific URL and a production URL are different targets. Vercel’s generated-URL documentation makes this distinction concrete: an individual deployment has a unique URL, a branch URL follows the latest deployment from that branch, and the production URL points to the production deployment.

Extend the synthetic example. Candidate B produces deployment B, and a browser check against B’s unique URL passes. The production domain still points to A. The deployment-specific check establishes something useful about B, but promotion and production routing remain unverified.

I would keep that release incomplete until the production observation is tied to the intended deployment. Record the requested URL, the observation time and the available evidence of which deployment served the response. A version header or diagnostic endpoint can help correlate those records. Treat a self-reported identifier as a consistency check; it does not independently prove the whole request path.

Choose the behavior assertion deliberately. In this example, checking that the homepage loads would miss a broken publication rule. A read-only check that a designated draft remains absent from the public feed would address that rule more directly, provided the fixture and expected result were agreed in advance.

Keep a release receipt

This proposed receipt is a template, not a completed release report. Its placeholders should become links and identifiers emitted by the actual build and deployment process:

candidate_commit: <full SHA>
validation:
  tested_subject: <commit or artifact identity>
  run: <test-run URL>
  assertions: <requirements exercised>
build:
  identity: <artifact digest or platform build identity>
  provenance: <verified attestation or build record>
deployment:
  id: <deployment ID>
  build_identity: <resolved build identity>
  environment: production
  unique_url: <deployment-specific URL, if available>
observation:
  requested_url: <production URL>
  time: <UTC timestamp>
  served_identity: <observed identity and evidence source>
  assertion: <specific behavior checked>
  result: <passed, failed, or not run>
config_revision: <non-secret configuration version>
unverified: <remaining gaps>

The useful operation is comparing the fields. Does the build record name the candidate? Does the deployment select that build? Does the production observation identify that deployment? When a link is missing, preserve the gap instead of compressing the record into “all checks passed.”

If production rebuilds the candidate, record the new build identity. The same source commit is insufficient evidence that two builds produced identical output. Decide which validation the production artifact needs, and attach the result to that artifact rather than silently carrying over a different build’s verdict.

This can be a small record alongside the existing release job. It need not become another service. Its purpose is to let a reviewer follow the evidence without reconstructing the agent’s conversation.

Account for configuration and sampling

Artifact identity still leaves runtime state outside the record. Kubernetes ConfigMaps explicitly decouple environment-specific configuration from container images. Their update behavior also depends on how they are consumed: ConfigMaps used as environment variables require a Pod restart to update.

That is one concrete reason identical image content does not establish identical configuration. Record a non-secret configuration revision and relevant known differences. Keep credentials out of the receipt. If configuration identity is unavailable, say so.

Finally, bound the claim to the observation. A successful request supports that assertion, at that target and time. It does not establish that every replica, region or later request behaves identically. Choose further samples according to the release risk; a receipt does not require rerunning the entire suite against every request.

An agent’s completion report should therefore say which deployment was observed through which production route, which behavior passed, and what remains unverified. If the only production evidence belongs to deployment A, candidate B’s release is still waiting for its check.