ci: fail a PR that moves an existing flake input backwards #257

Merged

ymrtech merged 1 commit from fix/flake-lock-pr-guard into main

2026-09-15 17:43:29 -06:00

hermes commented

2026-09-15 17:38:28 -06:00

Collaborator

Follows #253 / #255. #255 stopped a non-main ref from reverting inputs at deploy time; nothing stopped a stale flake.lock reaching main — a PR that re-locks one input on an old base (or a merge resolving a flake.lock conflict in favour of a stale copy) drags every other input main has moved back to that base's state.

The rule

A new flake-lock-guard job in the PR gate, two-step so unrelated PRs are never blocked:

Did THIS change modify flake.lock at all (vs its merge base for a PR, vs the previous main tip for a push)? If not, the merge keeps main's inputs — pass. A branch merely based on an older main is not a regression, and must not be flagged (that's why this step exists).
If it did, compare against the lock the merge targets (main for a PR) and FAIL on any input that moved backwards.

Backwards = a lower revCount (commit count, monotonic per repo), or a lower lastModified where revCount is absent.

Adding or removing an input is explicitly not a regression — an input nobody references cannot regress anything in use — and input bumps belong to the weekly flake-update pipeline, which moves inputs forward and therefore passes.

Fail-closed: missing jq, or a classification that fails, exits non-zero rather than reporting "no regression" — a silent pass is exactly the failure mode this job exists to prevent. Added to alert-hermes's needs: so a failure reaches the webhook spool.

Verification

Classifier (extracted from this file) against real locks:

case	result
forward bump → pass	✓
current main vs the pre-bump lock	FAIL, naming `clientsv` and `nix-config-private` — exactly what the incident reverted ✓
identical locks	pass ✓
add + remove an input	informational only, no failure ✓
forward bump by `revCount`	pass ✓

The full step, run on throwaway branches in a scratch clone: untouched main → 0; unrelated config change → 0 (step-1 shortcut); regressed flake.lock → 1 with the diagnosis; a change that doesn't touch flake.lock → 0.

Also: bash -n on the step both raw and with ${{ }} substituted; PyYAML job-structure assert (8 jobs, new job top-level, alert-hermes needs updated); no stray escapes or truncation markers in the new job.

Two defects were caught by that validation before committing — a stray " that swallowed the rest of the script, and \" escapes that made the jq invocations unparseable. Both are the file-corruption class warned about in the workflow-authoring notes.

Scope note

The guard is a check, not a reconcile: it refuses the change and tells the author to merge main + re-lock. The deploy-side reconcile (#255) still does the automatic thing for non-main refs.

Follows #253 / #255. #255 stopped a non-main **ref** from reverting inputs at deploy time; nothing stopped a stale `flake.lock` reaching **main** — a PR that re-locks one input on an old base (or a merge resolving a `flake.lock` conflict in favour of a stale copy) drags every *other* input main has moved back to that base's state. ## The rule A new `flake-lock-guard` job in the PR gate, two-step so unrelated PRs are never blocked: 1. **Did THIS change modify `flake.lock` at all** (vs its merge base for a PR, vs the previous main tip for a push)? If not, the merge keeps main's inputs — **pass**. A branch merely *based* on an older main is not a regression, and must not be flagged (that's why this step exists). 2. If it did, compare against the lock the merge targets (main for a PR) and **FAIL on any input that moved backwards**. Backwards = a lower `revCount` (commit count, monotonic per repo), or a lower `lastModified` where `revCount` is absent. **Adding or removing an input is explicitly not a regression** — an input nobody references cannot regress anything in use — and input *bumps* belong to the weekly flake-update pipeline, which moves inputs forward and therefore passes. Fail-closed: missing `jq`, or a classification that fails, exits non-zero rather than reporting "no regression" — a silent pass is exactly the failure mode this job exists to prevent. Added to `alert-hermes`'s `needs:` so a failure reaches the webhook spool. ## Verification Classifier (extracted from this file) against real locks: | case | result | |---|---| | forward bump → pass | ✓ | | current main vs the pre-bump lock | **FAIL, naming `clientsv` and `nix-config-private`** — exactly what the incident reverted ✓ | | identical locks | pass ✓ | | add + remove an input | informational only, no failure ✓ | | forward bump by `revCount` | pass ✓ | The full step, run on throwaway branches in a scratch clone: untouched main → 0; unrelated config change → 0 (step-1 shortcut); **regressed `flake.lock` → 1** with the diagnosis; a change that doesn't touch `flake.lock` → 0. Also: `bash -n` on the step both raw and with `${{ }}` substituted; PyYAML job-structure assert (8 jobs, new job top-level, `alert-hermes` needs updated); no stray escapes or truncation markers in the new job. Two defects were caught by that validation before committing — a stray `"` that swallowed the rest of the script, and `\"` escapes that made the `jq` invocations unparseable. Both are the file-corruption class warned about in the workflow-authoring notes. ## Scope note The guard is a *check*, not a reconcile: it refuses the change and tells the author to merge main + re-lock. The deploy-side reconcile (#255) still does the automatic thing for non-main refs.

hermes added 1 commit

2026-09-15 17:38:28 -06:00

ci: fail a PR that moves an existing flake input backwards (issue #253 )

NixOS Build Check / Scope changed deployable hosts (pull_request) Successful in 2s

Details

NixOS Build Check / Flake input regression guard (pull_request) Successful in 3s

Details

NixOS Build Check / Eval giga (scope + liveness + offline fallback) (pull_request) Successful in 2s

Details

NixOS Build Check / Build public closure (on public) (pull_request) Has been skipped

Details

NixOS Build Check / Eval giga natively (authoritative, on giga) (pull_request) Has been skipped

Details

NixOS Build Check / Build mail closure (on mail) (pull_request) Has been skipped

Details

Secret Scan / gitleaks scan (pull_request) Successful in 3s

Details

NixOS Build Check / Build vpn closure (on vpn) (pull_request) Has been skipped

Details

NixOS Build Check / Alert Hermes on failure (pull_request) Has been skipped

Details

d58d3cccec

#255 stopped a non-main REF from reverting inputs at deploy time, but nothing
stopped a stale flake.lock reaching MAIN: a PR that re-locks one input on an
old base — or a merge that resolves a flake.lock conflict in favour of a stale
copy — drags every OTHER input main has moved back to that base's state.

New `flake-lock-guard` job in the PR gate, two-step so unrelated PRs are never
blocked:

  1. Did THIS change modify flake.lock at all (vs its merge base for a PR, vs
     the previous main tip for a push)? If not, the merge keeps main's inputs —
     pass. A branch merely based on an older main is not a regression.
  2. If it did, compare against the lock the merge targets (main for a PR) and
     FAIL on any input that moved BACKWARDS.

Backwards = a lower revCount (commit count, monotonic per repo) or, where that
is absent, a lower lastModified. Adding or removing an input is explicitly not
a regression — an input nobody references cannot regress anything in use — and
input bumps belong to the weekly flake-update pipeline (forward bumps pass).

Fail-closed: missing jq or a failed classification exits non-zero rather than
reporting "no regression" (a silent pass is the failure mode this job exists to
prevent). Added to alert-hermes' needs so a failure reaches the webhook spool.

Verification:
  - the classifier (extracted from this file) against real locks: forward bump
    -> pass; current-main vs the pre-bump lock -> FAIL naming clientsv and
    nix-config-private (exactly what the incident reverted); identical locks ->
    pass; add+remove -> informational, no failure; forward bump by revCount ->
    pass.
  - the full step run on throwaway branches: a regressed flake.lock exits 1
    with the diagnosis; an unrelated change exits 0 via the step-1 shortcut.
  - bash -n on the step (raw and with ${{ }} substituted), python-yaml job
    structure assert, no stray escapes in the new job.