Skip to content

[Phase 1] Drift-guard CI: fail when template stubs diverge from standards/ #969

Closed

Description

@github-actions

Story

As a standards maintainer,
I want have a CI check that fails if repo-template's stubs/config diverge from the canonical standards/,
so that the template stays a thin distribution layer and never becomes a competing fork of the standards.

Acceptance Criteria

  1. A CI check compares each templated stub's git blob SHA against the canonical standards/workflows/ SHA and fails on any DRIFTED stub, reusing the ALIGNED/DRIFTED/MISSING classification model from scripts/fleet_stub_drift.sh.
  2. The check covers the workflow stubs plus the verbatim baseline (dependabot.yml, CODEOWNERS, CLAUDE.md); intentionally per-stack-customizable files (ci.yml) are excluded via a documented allowlist with a recorded rationale.
  3. The check is wired into an existing PR-triggered job (no new scheduled/cron workload) and emits a clear annotation naming the drifted file and both SHAs.
  4. A bats test exercises the drift classification (ALIGNED / DRIFTED / MISSING) for the template's stub set.

Tasks / Subtasks

Dev Notes

  • scripts/fleet_stub_drift.sh ALREADY implements exactly this byte-identity drift model (git blob SHA equality) for the initiative-planner.yml stub across fleet repos: classify_stub_drift <canonical_sha> <repo_sha> -> ALIGNED|DRIFTED|MISSING; stub_drift_row emits repo<TAB>status<TAB>repo_sha<TAB>canonical_sha. Its functions are PURE (no network) — the network fetch lives in fleet_monitor.sh. Reuse them; do not reinvent a diff.
  • ci.yml is the ONE stub carrying real per-stack build/test steps and MUST be excluded from byte-identity drift (it is intentionally customized per consumer). Document the exclusion the same way AGENTS.md documents its lint.yml / token-report.yml / pr-review-sweep.yml exceptions.
  • Honor the initiative cost cap: attach to PR triggers within the existing lint surface — add NO new cron/scheduled workload.
  • Per the discussion's adversarial-review fallback: if a committed-stub drift guard proves too costly, the alternative is generating the template contents from standards/ in CI; keep that option open but the primary design is the drift check.
  • Testing standard: bats in tests/, listed in the lint.yml bats job; mirror tests/fleet_stub_drift.bats.

Project Structure Notes

The drift check may live in repo-template's CI, in petry-projects/.github, or here; the reusable comparison logic and its bats test belong in this repo's scripts/ + tests/ to leverage the existing fleet_stub_drift.sh primitives.

References

Likely target surface

  • scripts/fleet_stub_drift.sh
  • tests/fleet_stub_drift.bats
  • .github/workflows/lint.yml

Story prepared by the BMAD Scrum Master (Bob) for epic #964. Status: ready-for-dev.

Activity

  1. don-petry commented on Jun 28, 2026

    @don-petry
    Collaborator

    Dev-Lead Implementation Plan

    Issue: #969 — [Phase 1] Drift-guard CI: fail when template stubs diverge from standards/

    Scope

    Add a PR-triggered CI guard that fails when any repo-template stub/baseline file
    drifts from its standards-derived baseline, reusing fleet_stub_drift.sh's
    byte-identity (blob-SHA equality) ALIGNED/DRIFTED/MISSING model and excluding the
    intentionally per-stack-customized ci.yml via a documented allowlist.

    Design

    The repo-template scaffold is a distribution artifact of standards/ produced by
    seed-repo-template.sh (callers repinned to @<name>/stable, inline stubs/baseline
    shipped from standards). The guard compares, per covered file:

    • expected SHA = git hash-object of seed-repo-template.sh --emit-* output
      (the standards-derived content, repin included), and
    • committed SHA = the contents-API .sha of the file in petry-projects/repo-template.
      Equal SHAs ⇒ the template still tracks standards. The pure classification reuses
      classify_stub_drift / stub_drift_row; network (gh + git hash-object) lives only
      in the CLI driver, mirroring fleet_stub_drift.sh ↔ fleet_monitor.sh.

    Implementation Checklist

    • scripts/template_stub_drift.sh — sources fleet_stub_drift.sh; covered-file
      manifest (9 workflow stubs + dependabot.yml/CODEOWNERS/CLAUDE.md); documented
      ci.yml allowlist; template_drift_allowlisted, template_drift_covered,
      template_drift_annotate (emits ::error:: naming file + both SHAs, fails on DRIFTED);
      network CLI driver main (source-guarded).
    • .github/workflows/lint.yml — new PR-triggered template-drift job running the
      script (no cron added); add tests/template_stub_drift.bats to the bats job list.
    • AGENTS.md — record the ci.yml allowlist exclusion + rationale alongside the
      existing exception list.

    Tests to Write

    • tests/template_stub_drift.bats (mirrors tests/fleet_stub_drift.bats):
      classification ALIGNED/DRIFTED/MISSING over the template's covered set;
      ci.yml is allowlisted/excluded from template_drift_covered;
      template_drift_annotate emits a ::error:: annotation naming the drifted file
      and both SHAs and returns non-zero on DRIFTED, zero on a clean set.

    Test Command

    scripts/run-bats.sh tests/template_stub_drift.bats (full suite via lint.yml bats job);
    lint via bash .dev-lead/scripts/dev-lead-lint.sh + shellcheck scripts/*.sh.

  2. don-petry commented on Jun 28, 2026

    @don-petry
    Collaborator

    Dev-Lead: Implementation Complete

    Plan Execution

    Design

    repo-template is a distribution artifact of standards/ produced by seed-repo-template.sh (caller stubs repinned to @<name>/stable, inline stubs + baseline shipped from standards). The guard compares, per covered file:

    • expected SHA = git hash-object of seed-repo-template.sh --emit-* output (the standards-derived content, repin included), vs
    • committed SHA = the contents-API .sha of the file in petry-projects/repo-template.

    Equal blob SHAs ⇒ the template still tracks standards/. Pure classification reuses fleet_stub_drift.sh; network (gh + git hash-object) lives only in the CLI driver main, mirroring fleet_stub_drift.sh ↔ fleet_monitor.sh. On any DRIFTED file it emits a ::error file=…:: annotation naming the file and both SHAs and exits non-zero.

    Covered set: 9 workflow stubs (agent-shield, auto-rebase, copilot-setup-steps, dependabot-automerge, dependabot-rebase, dependency-audit, dev-lead, pr-review-mention, sonarcloud) + verbatim baseline (.github/dependabot.yml, .github/CODEOWNERS, CLAUDE.md). Excluded: .github/workflows/ci.yml (per-stack customizable).

    Test Results

    # scripts/run-bats.sh tests/template_stub_drift.bats
    1..13
    ok 1 classify_stub_drift is available (sourced from fleet_stub_drift.sh)
    ok 2 template stub set: a byte-identical committed file is ALIGNED
    ok 3 template stub set: a hand-edited committed file is DRIFTED
    ok 4 template stub set: a missing committed file is MISSING
    ok 5 covered set: includes the workflow stubs plus the verbatim baseline
    ok 6 covered set: excludes the allowlisted ci.yml (per-stack customizable)
    ok 7 template_drift_allowlisted: ci.yml is allowlisted
    ok 8 template_drift_allowlisted: a covered stub is NOT allowlisted
    ok 9 annotate: a DRIFTED row emits a ::error:: naming the file and both SHAs, and fails
    ok 10 annotate: an all-ALIGNED set passes (exit 0, no error annotation)
    ok 11 annotate: an absent TSV passes (nothing to check)
    ok 12 annotate: MISSING alone does not fail the job (only DRIFTED fails, AC #1)
    ok 13 stub_drift_row: classifies a drifted template file into a 4-field TSV row
    
    Full lint.yml bats suite: 883 tests, 0 failures.
    shellcheck --severity=warning -x scripts/*.sh: clean.
    .dev-lead/scripts/dev-lead-lint.sh: all checks passed.
    Driver smoke test (stubbed gh): DRIFTED → exit 1 w/ ::error annotations; ALIGNED → exit 0.
    

    Files Changed

    • scripts/template_stub_drift.sh (new): pure drift helpers (allowlist, covered set, annotate) reusing fleet_stub_drift.sh + a source-guarded network CLI driver.
    • tests/template_stub_drift.bats (new): ALIGNED/DRIFTED/MISSING classification, allowlist, and annotation tests.
    • .github/workflows/lint.yml: new template-drift PR job + template_stub_drift.bats added to the bats list.
    • AGENTS.md: new "Template drift guard (repo-template)" section documenting the ci.yml allowlist exception + rationale.

    Notes

    • Only DRIFTED fails the job (per AC test issue from agent #1); MISSING is surfaced as a non-fatal ::warning:: (matches the informational treatment in fleet_stub_drift.sh).
    • Graceful degradation: a SHA that cannot be computed/read (e.g. a transient cross-repo read failure, or repo-template not being readable by github.token if private) becomes a ::warning:: skip rather than a hard failure, so the PR lint job never flakes on infrastructure — only real drift fails it. The drift check could alternatively live in repo-template's own CI (where it has native read access); the reusable logic + bats test live here per the story's structure note.
    • dependabot.yml drift assumes the template ships the default node stack (DEPENDABOT_STACK); a different stack would need that env set.
  3. added a commit that references this issue on Jun 28, 2026
  4. added 10 commits that reference this issue on Aug 2, 2026
  5. added a commit that references this issue on Aug 18, 2026
  6. added 2 commits that reference this issue on Sep 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    dev-leadFor dev-lead agent pickupinitiativeEpic / initiative tracking issue

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions