Skip to content

CI: a release that ships nothing reports success #1012

Description

@simonoppowa

Merging #988 to main produced no release, and the pipeline reported success.

Run 33547853809: every check green, conclusion success, and ios-package, ios-deploy, android-package, android-deploy and github-release all skipped. No v2.2.0 tag, no GitHub release, nothing to TestFlight, no AAB. Nothing anywhere said so — the run looked exactly like a successful release.

The cause is fixed (#1008, a8a78997), but the silence is a separate defect and it will outlive that fix. Any future condition that skips the chain — a new needs: entry, another conditional job, a guard edit — fails the same way: green, quiet, and empty.

There is a second flavour of the same problem already shipping: android-deploy deliberately exits 0 when Play rejects the upload (#942). It emits a ::warning::, which is easy to miss on a green run, and Play genuinely does not have the bundle afterwards. Confirmed again on the v2.2.0 run 33554300243.

What would have caught it

A final assertion step that checks the artifact, not the job colour. Options, roughly in order of effort:

  1. github-release asserts the tag now exists at the expected version, and fails loudly if not. Cheap, catches the exact 2.2.0 failure.
  2. A release-summary job with if: !cancelled() that runs regardless, reads the outcome of each deploy job, and fails when a push to main produced no release. Catches skips too, because it does not sit downstream of them in a way that inherits the skip.
  3. Assert the version was actually published — query the Play and App Store Connect APIs for the build number and fail if absent. Most thorough, most moving parts, and would also surface the Play's API rejects health-permission bundles regardless of the declaration #942 gap as a real failure rather than a warning.

(1) plus (2) would cover both flavours without touching the deploy jobs.

Why it is worth doing

Two consecutive releases were lost. v2.1.0 to a genuine android-deploy failure, v2.2.0 to a silent skip. In both cases the gap between "pipeline finished" and "users can install it" was invisible from the run page — and in the 2.2.0 case, invisible for as long as nobody thought to check gh release list.

Metadata

Metadata

Assignees

No one assigned

    Labels

    backlog:mediumMedium priority — moderate UX gap or contained bugenhancementNew feature or requestgithub_actionsPull requests that update GitHub Actions code

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions