Skip to content

ci: census the scheduled and manual lanes and CI scripts; fix the ones that are silently dead, delete the ones nothing reads #1671

Description

@DecisionNerd

Problem

Only five of the 24 workflows run on pull requests. The other 19 are scheduled or manual, and nobody reads them. A census on main at edc91bc9e (2026-09-30), using the Actions API run counts rather than recent history:

Lane Trigger Runs Successes State
fuzz.yml daily schedule 63 11 Red since 2026-08-10. dtolnay/rust-toolchain is pinned to a master SHA with no toolchain input: 'toolchain' is a required input. No fuzzing has run for 51 days.
concurrency-stress-gate.yml weekly schedule 9 0 Never passed. gate-registry.py run executes the registry command with a literal python3, dropping the virtualenv the workflow installed the wheel into: ModuleNotFoundError: No module named 'graphforge'.
codspeed.yml daily schedule 383 301 Red since 2026-09-17, every night since #1406 added the ingest floor gate. Tracked separately; not fixed here.
native-local-admission.yml manual 11 0 Cannot pass. It asks hosted runners to qualify as BenchExec admission hosts and they are disqualified by design. M5 names OVHC-AGENCY as the system under test.
fly-tiny-qualification.yml manual 1 0 Echo-only: prints gate-registry.py command … and performs no operation.
fly-tiny-recovery.yml manual 0 0 Echo-only, same shape.
progressive-ladder.yml manual 0 0 Echo-only, same shape.
checkpoint-recovery-gate.yml manual 0 0 Never run since 2026-07-28. Dispatched for the first time for this census.
visualization-limits-stress.yml manual 0 0 Never run since 2026-08-01. Dispatched for the first time for this census.
release-certification.yml, clean-env-verify.yml manual 0 0 Never run. Release path for M13; out of scope here, flagged to the release owner.

Scripts, 151 tracked files under scripts/:

  • Five test files that no workflow, Makefile target, or doc invokes, 303 passing tests that protect nothing because nothing runs them: test-validate-g500-certification.py (248 tests), test-fly-filesystem-qualification.py (23), test-release-workflows.py (15), test-publish-track.py, test-binding-parity-policy.py. Their subjects are live. test.yml lists 50 script tests by hand, which is how five fell out.
  • Three dead scripts last touched 2026-07-28 with no reference anywhere: scripts/inventory_tck.py, scripts/tck_metrics.py, scripts/validate_datasets.py (its docstring says "for v0.3.0 release").

SQLite, DuckDB, and the DataFusion projects run fuzzing and stress on a schedule too. The difference is that a red scheduled lane there is somebody's problem within a day. A lane that is red for seven weeks, or has never been green, is cost with no signal, and a fuzz lane that does not fuzz is worse than none because the repo claims the coverage.

Scope

  1. Fix fuzz.yml: pass the toolchain explicitly. Prove it with a dispatched run on the branch that reaches the fuzz targets.
  2. Fix the registry run path so a registry command starting with python3 executes under the invoking interpreter. command output is unchanged. Add a regression test in test-gate-registry.py. Prove it with a dispatched concurrency-stress-gate.yml run on the branch.
  3. Delete the three echo-only handoff workflows. The registry commands they print remain available through gate-registry.py command <gate>.
  4. Delete native-local-admission.yml. The admission validator and the local-linux-cgroups-v2 profile stay; they run on the designated host.
  5. Dispose checkpoint-recovery-gate.yml and visualization-limits-stress.yml on the result of their first run: passing lanes stay; a failing lane gets its root cause fixed here if bounded, otherwise its own issue.
  6. Wire the five orphan test files into test.yml beside their subjects.
  7. Delete the three dead scripts.
  8. Guard: an existing policy test fails when a scripts/ci/test-* file is invoked by no workflow or Makefile target, so tests cannot be orphaned again.
  9. Update config/gate-registry.json, .github/workflows/README.md, and any policy test that pins the workflow list. Record in the README that a scheduled lane red for seven days is fixed or removed.

Non-goals

Acceptance criteria

  • A dispatched fuzz.yml run on the PR head completes the toolchain step and runs the fuzz targets.
  • A dispatched concurrency-stress-gate.yml run on the PR head passes, or fails on a stress finding rather than an import error; any such finding is filed.
  • test-gate-registry.py has a test that fails on the old python3 behaviour.
  • The four deleted workflows are gone from the tree, the registry, and the README; make gate-registry-check passes.
  • All five orphan tests run in test.yml; the guard test fails when one is removed from the workflow.
  • The three dead scripts are deleted and nothing references them.
  • Before and after counts recorded on the PR: workflows, workflow YAML lines, tracked scripts, script tests executed by CI.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    ci-cdCI/CD configuration changestoolingDeveloper tooling and automation

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions