Repository navigation
Reconcile workflow-stub pin drift: templates vs compliance-audit vs fleet are three-way inconsistent #482
Copy link
Copy link
Closed
Labels
Description
Summary
A controlled verification of the PR-based deploy (#478/#480) surfaced a three-way inconsistency between (a) the deploy templates in standards/workflows/, (b) the compliance audit's expected pins in check_centralized_workflow_stubs (scripts/compliance-audit.sh), and (c) the actual pins deployed in consumer repos. As a result, deploy-standard-workflows.sh reports 28 stubs as "drifted" — but a blind re-sync would push downgrades, pin-loosenings, and audit-breaking changes. The drift cannot be safely swept until the canonical pin per reusable is reconciled across these three sources.
Evidence
Audit (check_centralized_workflow_stubs) expected pins vs. template uses: vs. fleet (the 3 repos sampled — google-app-scripts, markets, ContentTwin):
| Workflow | Audit expects | Template uses: |
Fleet (sampled) | Problem |
|---|---|---|---|---|
pr-review-mention |
@v2 |
@v2 (tag) |
SHA # v2 / @v2 |
re-sync would loosen SHA→tag |
agent-shield |
@v1 |
@208ec2… # v1 (SHA) |
SHA #v1 / @v1 |
template SHA-pinned ≠ audit @v1 → deploying it fails the centralized-stub audit |
auto-rebase |
@v1 |
@v1 (tag) |
SHA #v1 / SHA #v2 |
re-sync loosens SHA→tag; markets is on a v2-SHA |
dependabot-automerge |
@v1 |
@v1 (tag) |
@v2 (all 3) |
fleet ahead (v2) of template+audit (v1) → re-sync = downgrade |
dependabot-rebase |
@v1 |
@9a694e5… # main (SHA→main) |
SHA #v1 |
template pins a main-branch SHA ≠ audit @v1 |
dependency-audit |
@v1 |
@v1 (tag) |
@v2 (all 3) |
fleet ahead (v2) → re-sync = downgrade |
Root causes
- No single source of truth for the per-reusable pin. Three places encode it independently (template, audit array, deployed stubs) and they have diverged.
- SHA-pin vs tag-pin tension is unresolved. Security/SonarCloud pushed SHA pins (fix(standards): pin auto-rebase stub to SHA to satisfy SonarCloud security gate #166), the centralized-stub check was relaxed to exempt internal reusable refs (Compliance: unpinned-actions-dependabot-automerge.yml #239), but templates and the audit still mostly encode
@vXtags — so the fleet, templates, and audit each landed in different states. - The deploy script's
is_already_compliantis a crude substring match on the template'suses:line — it disagrees withcheck_centralized_workflow_stubs, so "drift" per the deploy script ≠ "non-compliant" per the audit. This is why 28 stubs look drifted while the fleet is largely functional.
Why it matters
- A blind
deploy-standard-workflows.shrun (now that it opens real PRs) would open ~28 PRs, several of which are downgrades (dependency-audit/dependabot-automergev2→v1) or pin-loosenings, and theagent-shield/dependabot-rebasetemplates would deploy stubs the audit then flags non-compliant. So the deploy tooling is safe to run only per-workflow after the canonical pin is settled, not as a fleet sweep.
Recommendation
- Pick one source of truth for each reusable's canonical pin (proposal: the
<name>/stablechannel where it exists; otherwise a single@vXtag), and make the template, the audit array, and the fleet agree. - Decide SHA vs channel/tag deliberately and apply uniformly (reconcile fix(standards): pin auto-rebase stub to SHA to satisfy SonarCloud security gate #166 ↔ Compliance: unpinned-actions-dependabot-automerge.yml #239), updating the action-pinning exemption +
check_centralized_workflow_stubsaccordingly. - Align
is_already_compliantindeploy-standard-workflows.shwithcheck_centralized_workflow_stubs(ideally share the expected-pin logic) so "drift" and "non-compliant" mean the same thing. - Only then run the deploy per-workflow to reconcile the fleet (it will now open auditable PRs).
Acceptance
- One documented canonical pin per reusable; template + audit + fleet agree.
-
deploy-standard-workflows.shdrift detection matches the compliance audit. - A dry-run reports
compliantfor the fleet (no spurious "would update"); intentional updates are real upgrades, not downgrades.
References
- Surfaced from the deploy-standard-workflows.sh should deploy via PRs, not direct Contents-API pushes (breaks on ruleset-protected repos) #478/feat: implement issue #478 — deploy standard workflows via PRs, not direct pushes #480 PR-based deploy verification (see also fix(deploy): ensure standards-sync label exists + document the deploy standard #481).
standards/workflows/*.yml,scripts/deploy-standard-workflows.sh,scripts/compliance-audit.sh(check_centralized_workflow_stubs).- Prior pin decisions: fix(standards): pin auto-rebase stub to SHA to satisfy SonarCloud security gate #166 (SHA-pin for SonarCloud), Compliance: unpinned-actions-dependabot-automerge.yml #239 (exempt internal reusable refs from action-pinning).
Reactions are currently unavailable
Activity
chore(standards): migrate reusable caller templates to <name>/stable …
fix(audit): accept <name>/stable channel pins with transitional legac…
feat(deploy): batch per-repo stub sync into one PR (#482) (#493)
feat(audit): make centralized-stub check ring-aware (#870) (#529)
fix: share canary-ring pin model between audit and deploy sweep (#482) (
don-petry commentedon Jun 25, 2026
Resolved ✅ — three-way pin inconsistency closed
All three acceptance criteria met; the full-fleet
deploy-standard-workflows.sh --dry-runnow reports 0 non-compliant stubs.Root cause & fix
The per-reusable canonical pin was encoded independently in three places (template, compliance audit, deploy sweep) and diverged. The audit became ring-aware in #529/#870, but the deploy sweep's
is_already_compliantwas still a literal substring match on the template's@<name>/stablepin — so it falsely flagged every ring1/next repo and would have reverted their intentional ring pins.#531 — extracted the canary-ring model into
scripts/lib/ring-pins.sh(single source of truth:ring_tier_for_repo,ring_canonical_ref,ring_accepted_refs,ring_legacy_csv,RING_REUSABLES) and sourced it from bothcompliance-audit.shanddeploy-standard-workflows.sh. "Drift" (deploy) and "non-compliant" (audit) are now computed from the same code and can't diverge again. +7 bats.Fleet reconciliation (the genuine off-channel pins the dry-run surfaced)
@376a4fcb # <name>/stable(frozen SHA) →@<name>/stablechannel tags.dev-lead.yml@3f9e5808 # dev-lead/stable(frozen SHA, stalled behindded84ce4) →@dev-lead/stable.Both needed the org-standard SonarCloud S7637 rule exemption (
sonar.issue.ignore.multicriteria, per #498/#502) added to theirsonar-project.properties— a channel tag is a mutable action ref, andsonar.exclusionsalone quiets the quality gate but not the GitHub-Advanced-Security code-scanning SARIF.Acceptance
ring-pins.sh).Closing.
chore: drop the transitional @v1/@v2 pin grace (#870) (#540)
fix(pr-review-mention): guard ack-comment post against the 2500-comme…
fix(#613): address advisory-bot findings (Gemini/Copilot/CodeRabbit)
feat(#613): relocate canary-rollout engine + bootstrap into .github (…
feat(#613): relocate canary-rollout engine + bootstrap into .github (…
fix(audit): accept <name>/stable channel pins with transitional legac…
feat(deploy): batch per-repo stub sync into one PR (#482) (#493)
feat(audit): make centralized-stub check ring-aware (#870) (#529)
fix: share canary-ring pin model between audit and deploy sweep (#482) (
feat(#613): relocate canary-rollout engine + bootstrap into .github (…
feat(#613): relocate canary-rollout engine + bootstrap into .github (…
feat(deploy): batch per-repo stub sync into one PR (#482) (#493)
feat(audit): make centralized-stub check ring-aware (#870) (#529)
fix: share canary-ring pin model between audit and deploy sweep (#482) (
feat(#613): relocate canary-rollout engine + bootstrap into .github (…
feat(deploy): batch per-repo stub sync into one PR (#482) (#493)
feat(audit): make centralized-stub check ring-aware (#870) (#529)
fix: share canary-ring pin model between audit and deploy sweep (#482) (
feat(#613): relocate canary-rollout engine + bootstrap into .github (…