From c3977ae59d952de953df1cfef7b6d85744c7351d Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Fri, 24 Jul 2026 09:37:10 -0700 Subject: [PATCH 1/5] Genericize verbatim sections for clean carry; mask needs; section advisories The verbatim sweep (#420) exposed that the canonical AGENTS sections carried hub-local content that breaks when byte-carried to the fleet, surfaced by the PlexCleaner convergence. - De-link 31 hub-only relative links (registry/catalog/spec/reports/STANDUP/ operational) - 24 inside verbatim sections (they 404 downstream and violate the carried-files rule that now sits verbatim in the same file); the preamble and the two intent sections keep their valid hub-navigation links. - Mask a job's owned needs: list in normalize() (like the action pin), so a correctly-subsetted github-release byte-matches - an unvendored needs name fails the workflow to load, so pruning is mandatory. Selftest covers it. - Operational Repositories -> appliesTo * so the 3 cross-references to it from Foundational Principles, Branching Model, and Release Model resolve everywhere. - Reclassify Repository Onboarding and Conformance as hub-only (drop from the carried manifest): it is hub-audit machinery (registry reconciliation, STANDUP cold-start, conformance matrix) a downstream agent never runs, and its one universal rule lives in AUDIT.md. It stays in the hub AGENTS.md as context. - section-model.md: reconcile a content-duplicate section (heading differs, content duplicates a verbatim section) by content not heading; audit.py now lists a repo's undeclared H2 sections as an advisory so the reconciliation is not missed. A repo may still carry its own project-specific sections. Co-Authored-By: Claude Opus 4.8 --- AGENTS.md | 38 +++++++++++++++++++------------------- spec/audit.py | 41 ++++++++++++++++++++++++++++++++++++----- spec/fidelity-model.md | 4 ++-- spec/files.json | 2 +- spec/section-model.md | 7 ++++--- 5 files changed, 62 insertions(+), 30 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 901084cb..0c3f70a6 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -35,7 +35,7 @@ A state-changing GitHub call is the highest-blast-radius thing an agent does her ## Branching Model -- **Two workflow models, set per repo by the registry [`workflowModel`](./registry/repos.json) field.** Most repos are `release`: they ship versioned units of delivery through the feature -> `develop` -> `main` flow this section describes. **Operational** repos instead track a live service's running state and differ substantially - direct-to-`develop`, advisory CI, dispatch-only release - see "Operational Repositories". The rest of this section is the `release` model unless noted. The promotion mechanics (never delete `develop`, EOL-conflict resolution) apply to both. +- **Two workflow models, set per repo by the registry `workflowModel` field.** Most repos are `release`: they ship versioned units of delivery through the feature -> `develop` -> `main` flow this section describes. **Operational** repos instead track a live service's running state and differ substantially - direct-to-`develop`, advisory CI, dispatch-only release - see "Operational Repositories". The rest of this section is the `release` model unless noted. The promotion mechanics (never delete `develop`, EOL-conflict resolution) apply to both. - `develop` is the integration branch. Feature branches -> `develop` is **squash-only**; develop is kept linear. - `develop` -> `main` is **merge-commit only** (no squash, no rebase). Merge commits preserve develop's commit list as a real second-parent reference on main, which lets the release model attribute releases to the develop commits that produced them (see "Release Model" below). Branch protection enforces this: the develop ruleset allows only `squash`, the main ruleset allows only `merge`. - All commits on both branches must be cryptographically signed (SSH or GPG). Squash and merge commits created via the GitHub UI are signed by GitHub's web-flow key. @@ -47,12 +47,12 @@ A state-changing GitHub call is the highest-blast-radius thing an agent does her - *Main:* the check is graph-based - it asks whether main's tip commit is reachable from develop, not whether the two branches have the same content. After any develop -> main release, main's tip is a brand-new merge commit that develop's history doesn't contain. Forward-only develop never adds it (no back-merge of main into develop), so the check would fail on every subsequent release. Other technical workarounds - rebasing develop onto main, or rewriting develop's history - exist but contradict the squash-only develop ruleset and the linearity invariant. - *Develop:* the check stalls bot auto-merge when two bot PRs against develop land within the same window. As soon as the first merges, the second flips to `mergeStateStatus: BEHIND` and GitHub's auto-merge will not fire while strict is on. The merge-bot only *enables* auto-merge on `opened`/`reopened` (see below) and never auto-updates bot branches, and Dependabot's rebase isn't real-time, so the second PR sits OPEN with all checks green indefinitely. Squash mechanics still rebase the diff onto develop's tip on merge, `required_linear_history` still enforces linearity, textual conflicts still block `mergeable: CONFLICTING`, and the required `Check pull request workflow status job` still gates merges - the only thing lost is pre-merge detection of *semantic-but-not-textual* conflicts, which the post-merge develop CI run catches anyway. - See [`repo-config/README.md`](./repo-config/README.md) "Rulesets" for the configured state. -- **Configuring branch protection on a fleet repo: don't hand-build the rules.** Reconstructing the rules by hand is error-prone and has gone wrong on past ports. First delete **all** legacy classic branch-protection rules and any stray rulesets (rulesets are the *only* mechanism used), then create **exactly two rulesets named `develop` and `main`** by importing the committed `repo-config/*.json` ruleset payloads via `gh api -X POST "repos///rulesets"` (`gh ruleset` is read-only). The names are load-bearing - this file and the workflows reference them. Operational repos import [`operational/develop.json`](./repo-config/operational/develop.json) as their `develop` ruleset (the `main` ruleset is shared); [`configure.sh`](./repo-config/configure.sh) selects the right develop payload from the registry `workflowModel` automatically. **Brownfield repos** (pre-existing history) need an extra step: `Require signed commits` rejects legacy unsigned commits and the admin bypass does not cover `git push --force`, so re-signing requires temporarily disabling the ruleset. -- **Bots (Dependabot and codegen) target both `main` and `develop` in parallel.** [`.github/dependabot.yml`](./.github/dependabot.yml) duplicates every ecosystem entry (one per branch) and [`catalog/snippets/workflows/run-codegen-pull-request-task.yml`](./catalog/snippets/workflows/run-codegen-pull-request-task.yml) runs as a matrix over both branches with branch names `codegen-main` and `codegen-develop`. Each branch absorbs its own bot PRs independently, so neither falls behind, and the forward-only rule still holds (nothing is back-merged from main to develop - both branches receive their updates directly). The merge-bot ([`.github/workflows/merge-bot-pull-request.yml`](./.github/workflows/merge-bot-pull-request.yml)) dispatches `--squash` or `--merge` from each PR's base ref via a `case` statement so the form matches the ruleset on either base. Dependabot **security** PRs (CVE-driven) always open against the repo default branch (`main`) regardless of `target-branch` - the same `case` statement covers them. The merge-bot auto-merges **every** Dependabot tier including semver-major (no ecosystem or update-type guard): the required CI checks are the gate, not the bump magnitude, so a major that breaks the build fails its checks and never merges. +- **Configuring branch protection on a fleet repo: don't hand-build the rules.** Reconstructing the rules by hand is error-prone and has gone wrong on past ports. First delete **all** legacy classic branch-protection rules and any stray rulesets (rulesets are the *only* mechanism used), then create **exactly two rulesets named `develop` and `main`** by importing the committed `repo-config/*.json` ruleset payloads via `gh api -X POST "repos///rulesets"` (`gh ruleset` is read-only). The names are load-bearing - this file and the workflows reference them. Operational repos import `operational/develop.json` as their `develop` ruleset (the `main` ruleset is shared); [`configure.sh`](./repo-config/configure.sh) selects the right develop payload from the registry `workflowModel` automatically. **Brownfield repos** (pre-existing history) need an extra step: `Require signed commits` rejects legacy unsigned commits and the admin bypass does not cover `git push --force`, so re-signing requires temporarily disabling the ruleset. +- **Bots (Dependabot and codegen) target both `main` and `develop` in parallel.** [`.github/dependabot.yml`](./.github/dependabot.yml) duplicates every ecosystem entry (one per branch) and `catalog/snippets/workflows/run-codegen-pull-request-task.yml` runs as a matrix over both branches with branch names `codegen-main` and `codegen-develop`. Each branch absorbs its own bot PRs independently, so neither falls behind, and the forward-only rule still holds (nothing is back-merged from main to develop - both branches receive their updates directly). The merge-bot ([`.github/workflows/merge-bot-pull-request.yml`](./.github/workflows/merge-bot-pull-request.yml)) dispatches `--squash` or `--merge` from each PR's base ref via a `case` statement so the form matches the ruleset on either base. Dependabot **security** PRs (CVE-driven) always open against the repo default branch (`main`) regardless of `target-branch` - the same `case` statement covers them. The merge-bot auto-merges **every** Dependabot tier including semver-major (no ecosystem or update-type guard): the required CI checks are the gate, not the bump magnitude, so a major that breaks the build fails its checks and never merges. - **Maintainer-pushed commits on a bot PR auto-disable auto-merge.** The merge-bot's `merge-dependabot` and `merge-codegen` jobs only fire on `opened` / `reopened` events (auto-merge is enabled exactly once per PR). When a maintainer pushes commits to a bot's branch (a `synchronize` event with an actor that isn't the same bot), the merge-bot's `disable-auto-merge-on-maintainer-push` job fires and calls `gh pr merge --disable-auto`. The maintainer's commits stay in the PR but won't auto-merge with the bot's content; re-enable auto-merge manually (`gh pr merge --auto ` or the GitHub UI) when ready. - **Why parallel dual-target rather than develop-only with eventual flow-through:** push-distribution channels (HACS for Home Assistant integrations, Linux distros that vendor from `main`, etc.) consume `main` directly. A develop-only model would leave `main` running stale code during long-running develop features. Codegen content can also be production-critical (live API-derived data, language lists, build catalogs) rather than just sample/demo content, so both branches need fresh codegen on their own cadence. - **Codegen regenerates committed files; its output must be deterministic from its inputs, never per-run state.** The codegen workflow is a mechanism to refresh files that are checked into the repo: it runs a matrix over `main` and `develop`, each leg regenerating against its own checkout and opening its own PR (`codegen-main -> main`, `codegen-develop -> develop`). For the two legs not to conflict on `develop -> main`, the generated output must depend only on its inputs - never on per-invocation state (timestamps, GUIDs, build IDs), which would diverge every run and conflict on every release. **What** a repo regenerates (data files, source, or both; code changes or pure data) and **how** (download and process an external source, transform local inputs, whatever) is entirely its own concern - the constraint is only that the output be input-deterministic, not how it is produced. - - *Reference:* the codegen workflow tasks are kept under [`catalog/snippets/workflows/`](./catalog/snippets/workflows/) (`run-codegen-pull-request-task.yml` and its scheduler). A repo adopting codegen supplies its own input-deterministic generator; this repo ships none. + - *Reference:* a repo adopting codegen supplies its own input-deterministic generator and wires the codegen reference workflow (`run-codegen-pull-request-task.yml` and its scheduler). - **App-token workflows use Client ID, not App ID.** `actions/create-github-app-token` deprecated the numeric `app-id` input in v3.0.0; use `client-id: ${{ secrets.CODEGEN_APP_CLIENT_ID }}`. When adding new App-token call sites, use the same form - do not reintroduce `app-id` / `CODEGEN_APP_ID`. See [`repo-config/README.md`](./repo-config/README.md) "Secrets" for which secrets each mechanism needs. ## Release Model @@ -60,16 +60,16 @@ A state-changing GitHub call is the highest-blast-radius thing an agent does her The **two-phase model is the default**: PRs build fast, publishing is batched. See [README "Release Distribution Model"](./WORKFLOW.md) for the full rationale. The load-bearing rules follow. The auto-publish paths (bot push, schedule) apply to `release` repos. **Operational** repos differ - dispatch-only release, no auto-publish - see "Operational Repositories". - **PRs smoke-test only.** [`test-pull-request.yml`](./.github/workflows/test-pull-request.yml) always runs unit tests, then a `dorny/paths-filter` `changes` job gates a **reduced** build of only the changed targets (Docker `linux/amd64` only, executable on a representative runtime subset), never pushing. Build-workflow files are intentionally not in the path filters - a filter can't tell a logic change from an action-version bump - so a workflow-only change isn't smoke-built; the reusable workflows are exercised by the next run that uses them (a later code PR's smoke build, or the scheduled/publish run). Workflow YAML is still linted in CI by the lint job's `actionlint` step; also run `actionlint` locally before pushing to catch issues early. -- **A human merge never auto-publishes.** [`publish-release.yml`](./.github/workflows/publish-release.yml) is the sole publisher; each run builds the **single trigger branch** (`main` a release, `develop` a prerelease). A first [`plan`](./catalog/snippets/workflows/publish-plan-task.yml) job decides once whether the run publishes and every other job gates on its output. It publishes on a **`workflow_dispatch`** of `main`/`develop` (the human-initiated release), a **code-affecting bot push to `main`** (the codegen App merges every Dependabot/codegen PR, so `github.actor` is the gate - a human merge/promotion to `main` skips), or a **weekly `schedule`** (Docker only, to refresh the base image). The `push` is main-only and paths-filtered, so a develop bot merge and an Actions-only bump publish nothing. A source-only repo publishes on dispatch only. +- **A human merge never auto-publishes.** [`publish-release.yml`](./.github/workflows/publish-release.yml) is the sole publisher; each run builds the **single trigger branch** (`main` a release, `develop` a prerelease). A first `plan` job decides once whether the run publishes and every other job gates on its output. It publishes on a **`workflow_dispatch`** of `main`/`develop` (the human-initiated release), a **code-affecting bot push to `main`** (the codegen App merges every Dependabot/codegen PR, so `github.actor` is the gate - a human merge/promotion to `main` skips), or a **weekly `schedule`** (Docker only, to refresh the base image). The `push` is main-only and paths-filtered, so a develop bot merge and an Actions-only bump publish nothing. A source-only repo publishes on dispatch only. - **Required check.** The `changes` job is in the `Check pull request workflow status job` aggregator's `needs` and **must succeed** (not just "not fail") - a paths-filter error must never let a target-changing PR merge with its smoke build silently skipped. Skipped smoke jobs (no matching change) pass; `failure`/`cancelled` blocks. - **Reusable-task parameter contract.** Every `build-*-task.yml` and `build-release-task.yml` takes `ref` (git ref to check out/version), `branch` (logical branch driving config/tags/prerelease - `main` => Release/`latest`/non-prerelease, else Debug/`develop`/prerelease), and where relevant `smoke`. **Branch-derived config keys off `inputs.branch`** - each run builds one branch; the top-level publisher passes `branch: ${{ github.ref_name }}`, which the tasks forward and read as `inputs.branch` (not `github.ref_name`) for config/tags/prerelease. `get-version-task.yml` takes a `ref` so NBGV versions the right branch. - **Per-target subsetting.** `build-release-task.yml` has per-target `enable_*` gates and self-contained leaf tasks, so a project that drops a target deletes: its `build--task.yml`, the matching job + `github-release` `needs` entry in `build-release-task.yml`, its path-filter entry in `test-pull-request.yml`, and (for PyPI) the `publish-pypi` job in `publish-release.yml`. CodeGen, versioning, badge, merge-bot, and Dependabot are target-agnostic. -- **Orchestration vs. build - the override seam.** The pipeline splits into two layers. The **orchestration** layer is generic and is the standardization baseline: [`publish-release.yml`](./.github/workflows/publish-release.yml) (single-branch publish plan), the `get-version` + `github-release` jobs inside [`build-release-task.yml`](./catalog/snippets/workflows/build-release-task.yml), [`get-version-task.yml`](./catalog/snippets/workflows/get-version-task.yml), [`build-datebadge-task.yml`](./catalog/snippets/workflows/build-datebadge-task.yml), and the aggregator shape of [`test-pull-request.yml`](./.github/workflows/test-pull-request.yml). Within `test-pull-request.yml`, only the `changes -> smoke-build -> check-workflow-status` aggregator wiring and the ruleset-bound job name are verbatim orchestration; the `unit-test` job and the `dorny/paths-filter` entries are owned/per-target. The **build** layer - the `build--task.yml` leaf tasks - is what a derived project owns and replaces. The contract that keeps the seam clean: **a target contributes files to the GitHub release by uploading a workflow artifact named `release-asset--`.** The `github-release` job collects every `release-asset--*` artifact by pattern - its `download-artifact` step uses `pattern:`/`merge-multiple:`, **never an `artifact-ids:` that names a build job's output** (the producing build jobs still appear in `needs` for sequencing) - so it (the tag-the-commit + create-the-release + attach-the-assets logic) is reusable **as-is** across repos. **This name-pattern handoff is canonical for every repo, single-target included** - name your one asset `release-asset--` and the verbatim `github-release` globs it; do not switch a single-target repo to an `artifact-id` output plus `download-artifact` `artifact-ids:`, which looks tidier for 1:1 but forks the `github-release` download (`pattern:`/`merge-multiple:`) and breaks its verbatim carry. +- **Orchestration vs. build - the override seam.** The pipeline splits into two layers. The **orchestration** layer is generic and is the standardization baseline: [`publish-release.yml`](./.github/workflows/publish-release.yml) (single-branch publish plan), the `get-version` + `github-release` jobs inside `build-release-task.yml`, `get-version-task.yml`, `build-datebadge-task.yml`, and the aggregator shape of [`test-pull-request.yml`](./.github/workflows/test-pull-request.yml). Within `test-pull-request.yml`, only the `changes -> smoke-build -> check-workflow-status` aggregator wiring and the ruleset-bound job name are verbatim orchestration; the `unit-test` job and the `dorny/paths-filter` entries are owned/per-target. The **build** layer - the `build--task.yml` leaf tasks - is what a derived project owns and replaces. The contract that keeps the seam clean: **a target contributes files to the GitHub release by uploading a workflow artifact named `release-asset--`.** The `github-release` job collects every `release-asset--*` artifact by pattern - its `download-artifact` step uses `pattern:`/`merge-multiple:`, **never an `artifact-ids:` that names a build job's output** (the producing build jobs still appear in `needs` for sequencing) - so it (the tag-the-commit + create-the-release + attach-the-assets logic) is reusable **as-is** across repos. **This name-pattern handoff is canonical for every repo, single-target included** - name your one asset `release-asset--` and the verbatim `github-release` globs it; do not switch a single-target repo to an `artifact-id` output plus `download-artifact` `artifact-ids:`, which looks tidier for 1:1 but forks the `github-release` download (`pattern:`/`merge-multiple:`) and breaks its verbatim carry. - **What a repo still curates** (this is by design, not a leak): the *list* of leaf jobs in `build-release-task.yml`. Per **Per-target subsetting** above, you delete the target jobs you don't ship and add the one(s) you do - `build-release-task.yml`'s `github-release` job is untouched, but the file is not byte-identical because its `needs`/job list reflects your targets. Making that list itself target-agnostic is a larger "factor build from orchestration" refactor that is intentionally **not** done. - **Map your outputs to the right seam** - pick by where each artifact *goes*, not by language: - *Files attached to the GitHub Release* (zips, binaries, packaged libraries): one leaf task per output, each uploading `release-asset--`. A data-only repo (e.g. a symbol library) has exactly one such task: validate -> `zip` -> upload `release-asset--library`; it deletes the nuget/pypi/executable/docker jobs and the `publish-pypi` job, keeps `github-release` as-is. This is also where the .NET `build-executable-task` lives - it is *not* a generic file step, it is specifically `dotnet publish` of the console app; replace it wholesale, don't adapt it. - *Package-registry pushes* (NuGet.org, PyPI): the leaf task both builds **and** publishes to its registry. NuGet pushes from inside `build-nugetlibrary-task` (`dotnet nuget push --skip-duplicate`) *and* also uploads a `release-asset-*` (.7z) for the GitHub release. PyPI is split: `build-pypilibrary-task` only builds + uploads the `pypilibrary-build-` artifact, and the separate `publish-pypi` job in `publish-release.yml` does the OIDC Trusted-Publishing upload (so `id-token: write` is granted only at that one entry point) - PyPI contributes **no** `release-asset-*`. - - *Image-registry pushes* (Docker Hub): `build-docker-task` pushes multi-arch tags directly; contributes **no** `release-asset-*`. The image tag is build-layer-owned - drive it from whatever version source fits (NBGV `SemVer2`, an upstream-release pin, or a per-image matrix). To publish the Docker Hub repository overview, [`publish-docker-readme-task.yml`](./catalog/snippets/workflows/publish-docker-readme-task.yml) pushes `Docker/README.md` via `peter-evans/dockerhub-description` (single-repo by default; matrix per image for multi-image repos), wired into `publish-release.yml` and gated to `main`. + - *Image-registry pushes* (Docker Hub): `build-docker-task` pushes multi-arch tags directly; contributes **no** `release-asset-*`. The image tag is build-layer-owned - drive it from whatever version source fits (NBGV `SemVer2`, an upstream-release pin, or a per-image matrix). To publish the Docker Hub repository overview, `publish-docker-readme-task.yml` pushes `Docker/README.md` via `peter-evans/dockerhub-description` (single-repo by default; matrix per image for multi-image repos), wired into `publish-release.yml` and gated to `main`. - *Source-only / no build* (validate + tag + release): this seam does not apply. A source-only repo carries **no** `build-release-task.yml` (its `appliesTo` excludes it), so there are no leaf tasks and no `get-version`/`github-release`/`date-badge` jobs to curate. Its whole release is the standalone [`publish-release.yml`](./.github/workflows/publish-release.yml) on `workflow_dispatch`: a `validate` job (the repo's reusable validation task) gates a publish job that **inlines** NBGV for the tag and `action-gh-release` for the release (tag + auto source archive + README + LICENSE). - `get-version-task.yml` installs the .NET SDK only because NBGV needs the runtime to compute the version/tag - heavyweight but expected even for a non-.NET repo; acceptable as-is. - **No-op republish guarantee.** A weekly/dispatch publish where NBGV `SemVer2` is **unchanged** (no new commit since the last publish) re-pushes **nothing** to GitHub Releases (the `github-release` job's `release-exists` check skips the create step), NuGet (`dotnet nuget push --skip-duplicate`), or PyPI (`gh-action-pypi-publish` `skip-existing: true`) - all three key on the version string. **Docker always re-pushes** by design: it picks up upstream base-image refreshes (e.g. `ubuntu:rolling`) that aren't visible in the repo. Boundary: `version.json` has **no `pathFilters`**, so *any* commit - including a CI/workflow-only or docs-only change - advances the NBGV git height and therefore `SemVer2`, and the next publish *does* create a fresh release for it even when the shipped binary is byte-identical. This is accepted NBGV behavior; `pathFilters` are intentionally not added. @@ -78,15 +78,15 @@ The **two-phase model is the default**: PRs build fast, publishing is batched. S - **No post-release bump; no develop-ahead requirement.** NBGV advances the patch (git height) on every commit, so a release always gets a fresh build version with no `version.json` edit and there is no `bump-version-X.Y` PR after a release. A `develop -> main` promotion carries whatever `version.json` is current: a promotion with a functional bump releases that new version on `main`; a maintenance-only promotion carries the unchanged `version.json` and `main` advances only its NBGV height. - **Docs reference the 2-digit `major.minor` line, never a 3-digit build.** `README.md`, `HISTORY.md`, and release notes name the version as `Version 1.0` (the `version.json` floor); NBGV owns the patch/build position, so a concrete three-part number in a doc is both wrong (the real build height differs) and a maintenance trap. "Correcting" `1.0` to `1.0.0` is a defect, not a fix - it has blocked a release. - **Issue-closing keywords (`Closes #N`, `Fixes #N`) go in the `develop -> main` promotion PR, not the feature -> develop PR.** GitHub auto-closes an issue only when the closing keyword merges into the **default branch** (`main`); a feature/develop PR merges into `develop`, so the keyword never fires there. Reference the issue in the develop PR body if useful, but put the actual closing keyword on the promotion PR. -- **Wrapper repos that track an upstream release.** A repo wrapping an upstream release uses [`check-upstream-version-task.yml`](./catalog/snippets/workflows/check-upstream-version-task.yml): a resolver command prints the upstream version(s) as a **JSON object of `name -> version`**, written to a committed state file at the **repo root beside `version.json`** (default `upstream-version.json` - it is a build-input version source, not GitHub-platform config, so it does not belong under `.github/`), and opens a rolling App-signed bump PR per branch that the merge-bot auto-merges (`merge-upstream-version`). The object carries one key for the common single-version case (`{"version":"X"}`) or N keys for a wrapper that pins several upstream components (e.g. an image plus a companion tool), and the build reads each component by key; the bump PR's title/body name only the keys that actually moved. Call it from a scheduled entry-point workflow and matrix only the branches that ship the version (a CI-only version uses `["develop"]`). A merged bump ships on the **next publish**, not immediately - the two-phase latency tradeoff. +- **Wrapper repos that track an upstream release.** A repo wrapping an upstream release uses `check-upstream-version-task.yml`: a resolver command prints the upstream version(s) as a **JSON object of `name -> version`**, written to a committed state file at the **repo root beside `version.json`** (default `upstream-version.json` - it is a build-input version source, not GitHub-platform config, so it does not belong under `.github/`), and opens a rolling App-signed bump PR per branch that the merge-bot auto-merges (`merge-upstream-version`). The object carries one key for the common single-version case (`{"version":"X"}`) or N keys for a wrapper that pins several upstream components (e.g. an image plus a companion tool), and the build reads each component by key; the bump PR's title/body name only the keys that actually moved. Call it from a scheduled entry-point workflow and matrix only the branches that ship the version (a CI-only version uses `["develop"]`). A merged bump ships on the **next publish**, not immediately - the two-phase latency tradeoff. ## Operational Repositories -The registry [`workflowModel`](./registry/repos.json) field is `release` (the default) or `operational`. This section is the operational delta - every other rule in this file is the `release` model unless it says otherwise. +The registry `workflowModel` field is `release` (the default) or `operational`. This section is the operational delta - every other rule in this file is the `release` model unless it says otherwise. **Operational** repos track a live service's running state rather than shipping versioned units of delivery - live-service config such as Home Assistant, ESPHome, Vantage, and home automation. -- **Commit configuration directly to `develop`.** There is no feature branch - the maintainer commits straight to `develop` and *occasionally* opens a `develop -> main` PR to bless a known-good snapshot. The [`develop` ruleset](./repo-config/operational/develop.json) drops the PR and status-check gate, so direct signed pushes are allowed (force-push, deletion, and unsigned commits are still blocked) and CI runs on the push as **advisory** feedback that never rejects a commit. +- **Commit configuration directly to `develop`.** There is no feature branch - the maintainer commits straight to `develop` and *occasionally* opens a `develop -> main` PR to bless a known-good snapshot. The `develop` ruleset drops the PR and status-check gate, so direct signed pushes are allowed (force-push, deletion, and unsigned commits are still blocked) and CI runs on the push as **advisory** feedback that never rejects a commit. - **The `main` promotion gate is unchanged.** The [`main` ruleset](./repo-config/main.json) is shared with `release` repos, so the `develop -> main` PR still **enforces** the required `Check pull request workflow status job`. For an operational repo that check is lint/validation only - editorconfig/EOL plus domain linters (a Home Assistant or ESPHome config validation, a firmware build), never unit tests - so `develop` stays the live surface and a broken config can never reach `main`. - **Release only by manual dispatch.** Operational repos carry `releaseTrigger: dispatch-only` and run no codegen or auto-publish bots, so they publish **only** on a manual `workflow_dispatch` - the source-only release the publisher already supports (tag + source zip + README + LICENSE, NBGV-versioned), never automatically. The `develop -> main` promotion just blesses a known-good snapshot, and a release is a separate, deliberate dispatch. - **Fleet sync still applies.** Dependabot's dual-target sync and the App-signed merge-bot run on **every** tier, operational included, so both branches stay in sync and a promotion stays a clean forward merge. @@ -97,9 +97,9 @@ Line-ending governance for an operational repo is in [Line Endings](#line-ending Every fleet repo is a standard-style repo the hub audits **downward** against its declared type - the model the fleet uses because managing downstream divergence is too costly. Three obligations follow, and they are not optional: -- **Definition of done is the audit.** Onboarding or materially changing a repo is complete only when it passes [`AUDIT.md`](./AUDIT.md) for its type, **or** carries a committed `reports//audit.md` plus a tracking issue for the residual deltas. Do not leave a repo partially stood up and unrecorded - that is itself a defect. Stand a repo (or a new type) up with [`STANDUP.md`](./STANDUP.md); verify it with [`AUDIT.md`](./AUDIT.md); the two share the same manifests, so a repo stood up correctly passes the audit by construction. -- **The registry is ground truth about reality, not intent.** After any conformance change, reconcile the repo's [`registry/repos.json`](./registry/repos.json) entry - `status`, `types`, `releaseTrigger`, `workflowModel`, `driftNotes` - to what the repo actually is. [`spec/validate.py`](./spec/validate.py) proves the catalog is self-consistent; it does **not** prove the catalog matches the live repo. A `validate.py`-clean entry can still be false, and a stale entry is a finding. -- **Prove the docs suffice.** The onboarding docs are sufficient only when a context-free agent stands each supported repo shape - project type(s) plus workflow model - up from them alone; the cold-start self-test in [`STANDUP.md`](./STANDUP.md) and the [conformance matrix](./reports/conformance-matrix.md) track that. A shape that cannot be stood up cold is a documentation defect, fixed in the hub, not worked around per repo. +- **Definition of done is the audit.** Onboarding or materially changing a repo is complete only when it passes [`AUDIT.md`](./AUDIT.md) for its type, **or** carries a committed `reports//audit.md` plus a tracking issue for the residual deltas. Do not leave a repo partially stood up and unrecorded - that is itself a defect. Stand a repo (or a new type) up with `STANDUP.md`; verify it with [`AUDIT.md`](./AUDIT.md); the two share the same manifests, so a repo stood up correctly passes the audit by construction. +- **The registry is ground truth about reality, not intent.** After any conformance change, reconcile the repo's `registry/repos.json` entry - `status`, `types`, `releaseTrigger`, `workflowModel`, `driftNotes` - to what the repo actually is. `spec/validate.py` proves the catalog is self-consistent; it does **not** prove the catalog matches the live repo. A `validate.py`-clean entry can still be false, and a stale entry is a finding. +- **Prove the docs suffice.** The onboarding docs are sufficient only when a context-free agent stands each supported repo shape - project type(s) plus workflow model - up from them alone; the cold-start self-test in `STANDUP.md` and the conformance matrix track that. A shape that cannot be stood up cold is a documentation defect, fixed in the hub, not worked around per repo. ## Pull Request Title and Commit Message Conventions @@ -182,7 +182,7 @@ Sub-topics take a `-` after the comment marker, each elaborating a distinct item - **[`.editorconfig`](./.editorconfig) sets the line ending:** `[*] end_of_line = crlf` is the **default** - every file type is CRLF unless pinned otherwise - with **LF** pinned for the execution-sensitive exceptions - `*.sh`, Dockerfiles, and any individual `.py` executed directly via its shebang (pinned **by path**, e.g. `spec/validate.py`; vanilla `.py` stays CRLF, since Python's universal newlines accept it and it is commonly edited on Windows). Only the LF exceptions are declared; the redundant per-type CRLF rules are intentionally omitted. `.gitattributes` mirrors it: `* -text` (git stores the exact bytes you commit and will **not** normalize) plus the matching LF pins. - **Choosing an ending for a new file type:** CRLF is the **default** - cross-platform editors on Windows produce it, and it is harmless on Linux for everything except shell. Use LF only when the type **requires** it or CRLF **breaks how it is consumed**: executable scripts/shebangs (`*.sh`, s6, husky), Dockerfiles (CRLF breaks `RUN` heredocs/continuations), and tool-owned formats with a native LF ending (KiCad). **Non-workflow YAML stays CRLF** - GitHub Actions' parser tolerates it (a repo that also runs yamllint sets `new-lines: disable` to defer to `.editorconfig`). **Workflow YAML (`.github/workflows/*.{yml,yaml}`) is pinned LF** in `.editorconfig` - Dependabot and Actions rewrite it with LF, so declaring LF keeps it consistent instead of mixed on every bump. This (and the catalog snippet workflows in `catalog/snippets/workflows/*`, pinned LF the same way) is an LF class **not** backed by a `.gitattributes` pin: git keeps `* -text` (no normalization), and CI's `editorconfig-checker` (EOL-only) catches a mismatch instead. Distinguish where a file is *consumed* from where it is *edited*: consumption on Linux alone does not force LF. A config or pattern file consumed by a Linux tool stays CRLF when the tool tolerates a trailing CR: `.dockerignore` and `.gitignore` are CRLF (their parsers strip the CR), and only a *Dockerfile* - interpreted, where a CR breaks `RUN` heredocs and line continuations - is LF. -- **Operational (config) repos: the global default follows the consuming application's native platform, not the fleet CRLF default.** A config repo (registry `workflowModel: operational`) is a *view into an application's configuration directory* - often the exact tree mounted into that app's container - so its files must use the ending the app itself reads and writes, and forcing the fleet CRLF default would fight the app. Set the `[*] end_of_line` default to the app's native ending and record it in the registry [`lineEndings`](./registry/repos.json) field (`lf` | `crlf`): **LF** for a Linux-native app whose config lives in a Linux container - ESPHome, Home Assistant, a devcontainer-only or HACS config - and **CRLF** for a Windows-native editor - e.g. Vantage InFusion config edited by Design Center on Windows. The execution-sensitive LF pins (`*.sh`, Dockerfiles, workflow YAML) still apply on top, and `.gitattributes` still mirrors the chosen default. This override is for operational repos only; `release` repos keep the `[*] end_of_line = crlf` fleet default above. Do **not** re-normalize such a repo to the fleet default - that is exactly the over-normalization these per-repo endings prevent. +- **Operational (config) repos: the global default follows the consuming application's native platform, not the fleet CRLF default.** A config repo (registry `workflowModel: operational`) is a *view into an application's configuration directory* - often the exact tree mounted into that app's container - so its files must use the ending the app itself reads and writes, and forcing the fleet CRLF default would fight the app. Set the `[*] end_of_line` default to the app's native ending and record it in the registry `lineEndings` field (`lf` | `crlf`): **LF** for a Linux-native app whose config lives in a Linux container - ESPHome, Home Assistant, a devcontainer-only or HACS config - and **CRLF** for a Windows-native editor - e.g. Vantage InFusion config edited by Design Center on Windows. The execution-sensitive LF pins (`*.sh`, Dockerfiles, workflow YAML) still apply on top, and `.gitattributes` still mirrors the chosen default. This override is for operational repos only; `release` repos keep the `[*] end_of_line = crlf` fleet default above. Do **not** re-normalize such a repo to the fleet default - that is exactly the over-normalization these per-repo endings prevent. - **Mixed-consumer config: prefer to split by platform into single-platform repos, not one mixed repo.** When a config repo would be consumed on two platforms (a Linux app plus a Windows-edited subtree), the clean answer is a repo per consumer, each single-platform with its own `lineEndings` - e.g. a controller config edited by a Windows-native editor (CRLF) lives in its own repo, **not** as a subtree inside a Linux-`lf` config repo. That keeps each repo's default, CI, and checkout matched to one platform and avoids per-path EOL machinery entirely. **Fallback only if a subtree genuinely cannot be split out:** keep the global default at the primary consumer and pin the odd subtree with an `.editorconfig` path override (e.g. `[/**] end_of_line = crlf`) matching its consumer, treated like any tool-owned format; the global `* -text` in `.gitattributes` already preserves those bytes, so no extra git pin is needed. - **Scripts and extensionless executables must be LF - and pinned in `.gitattributes`, not just configured.** A CRLF shebang (`#!/usr/bin/env bash\r`) breaks execution. `.editorconfig` sets `[*.sh] = lf`, but that extension-based rule does not match **extensionless** executables (s6 service scripts `run`/`up`/`finish`, husky/git hook scripts like `.husky/pre-commit`), and `* -text` enforces nothing - so a broad normalization pass or an editor can silently flip them to CRLF (it has). `.gitattributes` is the enforcement layer: it carries `*.sh text eol=lf`, and any repo whose tooling ships extensionless scripts **adds the matching path pin** - e.g. `Docker/s6-overlay/** text eol=lf` for s6 init, `.husky/pre-commit text eol=lf` for husky hooks - so git holds them at LF on checkout and `--renormalize`. This pin is mandatory for any repo that overrides s6 init, uses husky/git hooks, or otherwise ships executable scripts. The same explicit-pin rule extends to **tool-owned file formats the base config doesn't key on**: pin them to whatever ending the tool reads and writes so a normalization sweep can't churn them - e.g. KiCad project/footprint/3D files (`*.kicad_mod`, `*.kicad_sym`, `*.step`), which KiCad writes LF (`*.kicad_mod text eol=lf`, ...). The principle is general: a file class the `.editorconfig` extension rules and `* -text` don't cover needs an explicit `.gitattributes` pin matching its tool's native ending. - **Pair each such pin with a matching `.editorconfig` override - the git pin alone is not enough.** `.gitattributes` governs **git** (checkout, commit, `--renormalize`); the **editor** follows `.editorconfig`, where the `[*] end_of_line = crlf` default still applies to any file no extension rule covers. So even with the git pin, the editor writes a CRLF shebang into an extensionless hook (breaking it when run from the working tree) or re-ends/trims a byte-sensitive data file. Give every extensionless **executable** an editorconfig LF override beside its `.gitattributes` pin (`[.husky/pre-commit] end_of_line = lf`); and for a **byte-preserve data directory** (downloaded or opaque source whose exact bytes the consumer may depend on) disable *all* editor normalization, not just EOL - `[/*]` with `charset = unset`, `end_of_line = unset`, `insert_final_newline = false`, `trim_trailing_whitespace = false` (`unset` is EditorConfig's spec-defined special value that removes an inherited property, so the editor enforces neither the global `charset` nor `end_of_line` on that path). Keep these overrides with the line-ending governance (above any `.NET-only` divider), not in the language-style section. @@ -277,7 +277,7 @@ Anti-pattern: don't keep flipping the code on the same style point. Flip the rul These conventions describe the target state. New and modified workflows must respect them; the rest of the repo is expected to be brought up to the same standard. Sweep PRs that apply a rule everywhere are welcome when a rule changes. -- **Action pinning**: pin **every** action - first-party (`actions/*`) and third-party - to a commit SHA with a trailing `# vX.Y.Z` comment, so Renovate / Dependabot can still bump it but a tag swap can't change the executed code. Use `# vX` (major-only) only when the upstream's floating major tag doesn't correspond to a specific patch/minor release SHA - pinning to the floating-tag SHA still gives the SHA guarantee, the version comment just records the major line. Documented exception (no SHA pin at all): [`dotnet/nbgv`](./catalog/snippets/workflows/get-version-task.yml) is consumed via `@master` because the upstream tag stream lags `master` substantially and Dependabot's tag-tracking would propose a downgrade. **This applies to repo-owned build-layer leaves too** - a leaf owning its build specifics is not a reason to use floating tags; Dependabot still bumps SHA pins (updating the SHA + version comment). +- **Action pinning**: pin **every** action - first-party (`actions/*`) and third-party - to a commit SHA with a trailing `# vX.Y.Z` comment, so Renovate / Dependabot can still bump it but a tag swap can't change the executed code. Use `# vX` (major-only) only when the upstream's floating major tag doesn't correspond to a specific patch/minor release SHA - pinning to the floating-tag SHA still gives the SHA guarantee, the version comment just records the major line. Documented exception (no SHA pin at all): `dotnet/nbgv` is consumed via `@master` because the upstream tag stream lags `master` substantially and Dependabot's tag-tracking would propose a downgrade. **This applies to repo-owned build-layer leaves too** - a leaf owning its build specifics is not a reason to use floating tags; Dependabot still bumps SHA pins (updating the SHA + version comment). - **Filename**: reusable workflows (those with `on: workflow_call`) end in `-task.yml`. Entry-point workflows (`on: push` / `pull_request` / `schedule` / `workflow_dispatch`) do NOT use the `-task` suffix; they end with what they do - `-pull-request.yml`, `-release.yml`, etc. The suffix carries semantic meaning: a `-task.yml` file is meant to be `uses:`-d, never triggered directly. - **Workflow `name:`** (the top-level `name:` field): reusable workflow names end in **"task"** (e.g. `Build PyPI library task`); entry-point workflow names end in **"action"** (e.g. `Publish project release action`, `Test pull request action`). The displayed action name in the GitHub Actions UI tells you at a glance whether you're looking at an orchestrator or a callee. - **Job and step `name:` suffixes**: every job's `name:` ends in **"job"**; every step's `name:` ends in **"step"** - including the PR-gate aggregator, whose `name:` is a required-status-check `context:` in a branch ruleset (`Check pull request workflow status job` in `test-pull-request.yml`). A ruleset-bound job's `name:` and its ruleset `context:` are the **same string**: rename them **together** - update the live ruleset and `repo-config/{develop,main}.json` in lockstep with the job `name:`, never one without the other, or required-status-check enforcement silently breaks. There is no un-suffixed exception. @@ -285,7 +285,7 @@ These conventions describe the target state. New and modified workflows must res - **Shells**: every bash surface - a multi-line `run:` block and every committed `.sh` script alike - starts with `set -Eeuo pipefail`: fail fast, fail on undefined vars, fail on a failed pipe segment, and let an `ERR` trap inherit into functions, subshells, and command substitutions (`-E`). The `-E` is defense in depth: the fleet ships no `ERR` trap today, so a script that later adds one inherits the behavior instead of silently losing it. - **Conditionals**: multi-line `if:` uses folded scalar `if: >-` so YAML preserves whitespace correctly. Literal block (`if: |`) is wrong because it embeds newlines inside the boolean expression. - **Boolean inputs**: workflows triggered both via `workflow_call` and `workflow_dispatch` must declare each boolean input in *both* trigger blocks - one definition does not propagate to the other. `workflow_call` delivers booleans as actual booleans; `workflow_dispatch` delivers them as the *strings* `"true"`/`"false"`. Any `if:` consuming a boolean input must compare against both forms - `if: ${{ inputs.foo == true || inputs.foo == 'true' }}`. -- **Validate input/state consistency at entry, fail fast**: when a workflow's inputs must satisfy a cross-input or input-versus-derived-state invariant (e.g. the release branch must match the computed version's prerelease status, or two inputs are mutually exclusive), assert it **once** in a dedicated entry validation step/job that the downstream jobs `needs:`, before any expensive build or publish work - not as partial checks scattered deep in later jobs. One gate that fails fast with a clear `::error::` beats a late or one-directional check. Examples: [`build-release-task.yml`](./catalog/snippets/workflows/build-release-task.yml)'s `validate-release` job (branch-versus-prerelease, both directions) and [`publish-docker-readme-task.yml`](./catalog/snippets/workflows/publish-docker-readme-task.yml)'s "Validate inputs step". +- **Validate input/state consistency at entry, fail fast**: when a workflow's inputs must satisfy a cross-input or input-versus-derived-state invariant (e.g. the release branch must match the computed version's prerelease status, or two inputs are mutually exclusive), assert it **once** in a dedicated entry validation step/job that the downstream jobs `needs:`, before any expensive build or publish work - not as partial checks scattered deep in later jobs. One gate that fails fast with a clear `::error::` beats a late or one-directional check. Examples: `build-release-task.yml`'s `validate-release` job (branch-versus-prerelease, both directions) and `publish-docker-readme-task.yml`'s "Validate inputs step". - **Reusable workflows**: job-level `permissions:` are validated *before* the `if:` evaluates, so even a skipped job needs valid permissions declared. A `release` job with `permissions: contents: write` and `if: ${{ inputs.publish }}` will still cause `startup_failure` on a caller that doesn't grant `contents: write`. Either declare permissions at the call site, or omit the inner block and inherit. - **Allowlist `success` and `skipped` explicitly** when chaining jobs across optional dependencies - `!= 'failure'` lets `cancelled` through (timeout, runner failure, manual cancel). Use `(needs.X.result == 'success' || needs.X.result == 'skipped')`. - **Artifact retention**: workflow artifacts are an intra-run handoff only - durable copies live on the GitHub release, not in workflow artifacts - so they must not survive the run and accumulate against the small account-wide artifact-storage quota. **Clean up each transfer artifact surgically at its point of consumption**: the job that downloads it deletes it by exact name/pattern right after consuming it (the `github-release` job deletes `release-asset--*` after attaching them to the release; `publish-release.yml`'s `publish-pypi` deletes `pypilibrary-build-` after publishing). Deletion needs `actions: write` granted on that job - for a reusable callee (e.g. `github-release` inside `build-release-task.yml`) the **caller** grants it (`publish-release.yml`'s `publish` job does). **Never blanket-delete the run's artifacts** (`gh api .../artifacts --jq '.artifacts[].id'`) - that also destroys diagnostic/log artifacts and the build-records actions emit automatically (`docker/build-push-action`'s `.dockerbuild`), which are exactly what you need to debug a failed run. Set `retention-days: 1` on **every** explicit `upload-artifact`: it is the failure-path backstop - a job that dies before its consumer runs leaves its artifact, reaped within a day - so no separate terminal cleanup job is needed. A repo customizing these jobs must preserve the consume-then-delete shape. @@ -299,8 +299,8 @@ CI runs the full lint set, but run the linters locally before pushing to catch i **Each surface runs the lint with the tool that fits it, all from the same config files** (`.markdownlint-cli2.jsonc`, `cspell.json`, `.editorconfig`): - **CI (authoritative)** runs **markdownlint-cli2**, **cspell**, and **actionlint** as pinned action wrappers (Dependabot bumps them), plus **editorconfig-checker** via Docker `:latest` (its action only installs the CLI, so the Docker one-liner is what actually runs the check). markdownlint covers all `**/*.md`; **cspell is scoped to `README.md` + `HISTORY.md`** (see [CODESTYLE.md](./CODESTYLE.md) "Markdown and Spelling" for why), matching the cspell one-liner below. -- **The [`.husky/pre-commit`](./catalog/snippets/husky/pre-commit) hook** runs **language formatting only** - CSharpier + `dotnet format` (or ruff) via native tooling, no Docker and no doc linters, so it stays fast. -- **The VS Code [Lint tasks](./catalog/snippets/configs/vscode-tasks.json)** run the full doc-lint set via Docker `:latest` on demand, the local surface for Markdown, spelling, workflow, and EditorConfig checks. +- **The `.husky/pre-commit` hook** runs **language formatting only** - CSharpier + `dotnet format` (or ruff) via native tooling, no Docker and no doc linters, so it stays fast. +- **The VS Code Lint tasks** run the full doc-lint set via Docker `:latest` on demand, the local surface for Markdown, spelling, workflow, and EditorConfig checks. The Docker invocations below are the same ones the VS Code tasks use, for ad-hoc or headless (agent) runs. @@ -347,7 +347,7 @@ Contributors commit to this repo with signed commits; the SSH-signing setup live ## Editor and Tasks - **VS Code is the primary IDE, and the experience favors it.** Prefer VS Code tasks and launch configurations for building, running, and testing over ad-hoc shell scripts; a script is the fallback, not the default. -- The `.code-workspace` file carries the shared editor settings and the recommended-extension set. **All VS Code settings and extension recommendations live only here, never in a standalone `.vscode/settings.json` or `.vscode/extensions.json`** (`.vscode/` holds only `tasks.json` and `launch.json`). A **standard set** of extensions applies to every repo (markdownlint, cspell, editorconfig, markdown-all-in-one, better-todo-tree, github-actions, actionlint, shellcheck, claude-code); **language-specific** extensions are added per project (.NET: csdevkit, csharpier; Python: python, pylance, ruff, mypy; Docker: the Docker extension). The catalog holds the full set and per-language additions: [`catalog/snippets/vscode/`](./catalog/snippets/vscode/). +- The `.code-workspace` file carries the shared editor settings and the recommended-extension set. **All VS Code settings and extension recommendations live only here, never in a standalone `.vscode/settings.json` or `.vscode/extensions.json`** (`.vscode/` holds only `tasks.json` and `launch.json`). A **standard set** of extensions applies to every repo (markdownlint, cspell, editorconfig, markdown-all-in-one, better-todo-tree, github-actions, actionlint, shellcheck, claude-code); **language-specific** extensions are added per project (.NET: csdevkit, csharpier; Python: python, pylance, ruff, mypy; Docker: the Docker extension). The catalog holds the full set and per-language additions: `catalog/snippets/vscode/`. - The Table of Contents is maintained by the Markdown All in One extension; `markdown.extension.toc.levels` in the workspace sets which heading levels it includes (see the Markdown rules for the authoring convention and the `` exclusion marker). - **Agents: editing the active `.code-workspace` can reload the VS Code window and drop the agent's session.** Commit all state first, prefer opening the folder rather than the workspace while editing it, or leave workspace edits to the maintainer (a maintainer edit does not reload). diff --git a/spec/audit.py b/spec/audit.py index 90383066..3e23a59b 100644 --- a/spec/audit.py +++ b/spec/audit.py @@ -345,17 +345,23 @@ def check_interface(path, contract, text): # fidelity deviation. Anchored to `uses:`, so a 64-hex docker digest and a tag/branch ref (`@v4`) do not match. # Hex is case-insensitive, and a hand-written note on a pin is not version-shaped, so it survives to be compared. _ACTION_PIN = re.compile(r"(\buses:[ \t]*[^\s@]+)@[0-9a-fA-F]{40}(?:[ \t]+#[ \t]*v?[0-9][\w.\-]*)?") +# A workflow job's `needs:` list names the jobs it sequences after. In a verbatim job region a repo prunes that +# list to the targets it actually vendors - a `needs` entry naming an unvendored job fails the whole workflow to +# load - so the list is owned per repo, not fixed. Mask the inline `[ ... ]` form the canonical uses, same as the +# action pin. The interface contract still checks the required job keys separately. +_JOB_NEEDS = re.compile(r"(^[ \t]*needs:[ \t]*)\[[^\]]*\]", re.MULTILINE) def normalize(text): """Reduce a carried unit to its comparable form: neutralize line endings (EOL variance is governed - separately, not a fidelity deviation) and neutralize a Dependabot-owned action pin - the 40-hex commit a - `uses: @` line pins, together with its trailing ` # vN` version comment - since Dependabot - bumps those per repo and that drift is governed, same category as EOL. This is NOT placeholder masking of - declared per-file tokens; see spec/fidelity-model.md "Normalization". + separately, not a fidelity deviation), a Dependabot-owned action pin (the 40-hex `uses: @` commit + plus its ` # vN` comment, bumped per repo), and a job's owned `needs:` list (pruned per repo to its vendored + targets). All three are governed drift, not a fidelity deviation. This is NOT placeholder masking of declared + per-file tokens; see spec/fidelity-model.md "Normalization". """ text = text.replace("\r\n", "\n").replace("\r", "\n") - return _ACTION_PIN.sub(r"\1@", text) + text = _ACTION_PIN.sub(r"\1@", text) + return _JOB_NEEDS.sub(r"\1", text) @functools.lru_cache(maxsize=1024) # bounded; the keys that recur across repos are the canonical and its history @@ -656,6 +662,17 @@ def audit_repo(entry, spec): for name in sorted(verbatim_needed): findings.extend(check_verbatim(f"{path} section '{name}'", text, path, extract=lambda t, n=name: extract_section(t, n))) + # Undeclared-section advisory (spec/section-model.md): an H2 the manifest does not declare is a + # candidate duplicate of a verbatim section, or repo-specific content to relocate. Advisory only - + # a repo may legitimately carry its own project-specific sections (the AGENTS.md preamble allows + # them) - so it points at the reconciliation, it never fails. AGENTS.md only, where the section + # structure is governed by section-model.md. + if path == "AGENTS.md": + declared = {n.strip().lower() for n in (needed | verbatim_needed)} + declared.add("repository onboarding and conformance") # hub-only section, carried by no repo + h2s = {ln[3:].strip().lower() for ln in text.splitlines() if ln.startswith("## ")} + for h in sorted(h2s - declared): + findings.append(("DRIFT", f"section: '{h}' in {path} is not a declared section - reconcile it (a duplicate of a verbatim section, or repo-specific content that moves to a topical doc), or confirm it is intentional (spec/section-model.md)")) # --- HISTORY.md mirrors the README opening --- # spec/readme-structure.md "HISTORY.md": the changelog opens as the README's twin - same H1 title and the @@ -804,6 +821,20 @@ def _selftest(): else: print(" ok action-pin: version bump normalizes equal, changed action differs, hand-written note survives") + # needs-mask: a verbatim job region whose `needs:` list is pruned to the repo's vendored targets must not + # count as drift (the list is owned), but a structural change to the job's steps must. + needs_full = " github-release:\n needs: [get-version, validate-release, build-nugetlibrary, build-executable]\n steps: []\n" + needs_pruned = " github-release:\n needs: [get-version, validate-release, build-executable]\n steps: []\n" + needs_forked = " github-release:\n needs: [get-version, validate-release]\n steps:\n - run: fork\n" + if content_hash(needs_full) != content_hash(needs_pruned): + ok = False + print(" FAIL needs-mask: a pruned needs list should normalize equal") + elif content_hash(needs_full) == content_hash(needs_forked): + ok = False + print(" FAIL needs-mask: a step change must still hash differently") + else: + print(" ok needs-mask: a pruned needs list normalizes equal, a forked step still differs") + # Region extraction and hashing: a forked github-release block must hash differently from the canonical. region = split_jobs(rel_ok).get("github-release") forked_region = split_jobs(rel_ok.replace(" merge-multiple: true\n", " artifact-ids: 1\n")).get("github-release") diff --git a/spec/fidelity-model.md b/spec/fidelity-model.md index 8143f3b8..d5796e2f 100644 --- a/spec/fidelity-model.md +++ b/spec/fidelity-model.md @@ -12,7 +12,7 @@ Each [`spec/files.json`][files] entry declares one `fidelity`, defaulting to `pr - **presence** - the unit exists (a file, or a markdown section heading). The audit's baseline check. - **intent** - carried faithfully but judged by meaning, not bytes. A downstream copy legitimately differs (a governed divergence or a paraphrase), and equivalence is a human call via `intentRef`. The audit asserts nothing beyond presence. -- **verbatim** - byte-identical to the hub's canonical after line-ending and action-pin normalization. The audit content-hashes the downstream copy against canonical. It applies to a whole file, a workflow job region (a job selected by key), or a markdown section region (a `## heading` block selected by name). The section granularity lets one file be **intent overall while a few of its sections are verbatim** - a universal rule block stays byte-identical fleet-wide even though the rest of the document is a repo-adapted paraphrase, so a stale section or a missing rule is caught while its heading still passes the presence check. +- **verbatim** - byte-identical to the hub's canonical after line-ending, action-pin, and job-needs normalization. The audit content-hashes the downstream copy against canonical. It applies to a whole file, a workflow job region (a job selected by key), or a markdown section region (a `## heading` block selected by name). The section granularity lets one file be **intent overall while a few of its sections are verbatim** - a universal rule block stays byte-identical fleet-wide even though the rest of the document is a repo-adapted paraphrase, so a stale section or a missing rule is caught while its heading still passes the presence check. - **interface** - an overridable body that must honor a named contract. The audit checks the contract by name and wiring, never the body. Fidelity is a declared field defaulting to `presence`, never inferred from `whole`/`placeholders`. `.editorconfig` and `.markdownlint-cli2.jsonc` are both whole with no placeholders yet sit at opposite fidelity, because the discriminator is governance, not field shape. @@ -30,7 +30,7 @@ The fixed interface of a workflow is stated in [`AGENTS.md`][agents] ("Orchestra ## Normalization -A verbatim check compares content by hash after **line-ending and action-pin normalization** - EOL variance is governed by the line-ending rules, and a `uses: @` pin (with its trailing `# vN` comment) is Dependabot-owned and bumped per repo, so both are governed drift rather than a fidelity deviation. This keeps a verbatim workflow job region (the `github-release` job) from flagging on every routine action bump while still catching a real structural fork. It does **not** mask placeholders: a verbatim unit carries none. The files that declare a `placeholders` list (for example `.github/copilot-instructions.md` with ``, ``, ``) are fidelity `intent`, judged by hand and never hashed. Masking could not serve a hash anyway - a downstream copy holds the substituted value (`ptr727`), not the token (``), so masking the token in the canonical alone would guarantee a mismatch. A verbatim unit that ever needed a per-repo substitution would require template-matching (the canonical as a pattern, the copy as an instance), not this content hash. None does today. +A verbatim check compares content by hash after **line-ending and action-pin normalization** - EOL variance is governed by the line-ending rules, a `uses: @` pin (with its trailing `# vN` comment) is Dependabot-owned and bumped per repo, and a job's `needs:` list is pruned per repo to its vendored targets (an unvendored name fails the workflow to load), so all three are governed drift rather than a fidelity deviation. This keeps a verbatim workflow job region (the `github-release` job) from flagging on a routine action bump or a legitimate needs-prune while still catching a real structural fork. It does **not** mask placeholders: a verbatim unit carries none. The files that declare a `placeholders` list (for example `.github/copilot-instructions.md` with ``, ``, ``) are fidelity `intent`, judged by hand and never hashed. Masking could not serve a hash anyway - a downstream copy holds the substituted value (`ptr727`), not the token (``), so masking the token in the canonical alone would guarantee a mismatch. A verbatim unit that ever needed a per-repo substitution would require template-matching (the canonical as a pattern, the copy as an instance), not this content hash. None does today. ## Stale Versus Modified diff --git a/spec/files.json b/spec/files.json index 692e4cb1..07e54e22 100644 --- a/spec/files.json +++ b/spec/files.json @@ -2,7 +2,7 @@ "$schema": "./files.schema.json", "note": "The standardization baseline: files and sections a fleet repo is expected to carry, and their intent authority. The audit mechanically checks presence (letter). Equivalence (intent) is judged by hand, and a section for an absent language or target is N/A. Each entry, and each section, carries an appliesTo selector - see spec/scope-model.md for the scope model and selector vocabulary. Each entry also has a fidelity (presence by default, or intent, verbatim, interface) governing how faithfully the content is checked - see spec/fidelity-model.md. The per-section fidelity of AGENTS.md, which sections are verbatim fleet-law and which are repo-specific, is defined in spec/section-model.md.", "baseline": [ - { "path": "AGENTS.md", "fidelity": "intent", "sections": [{ "name": "Foundational Principles", "fidelity": "verbatim" }, { "name": "Repository Boundaries and Write Safety", "fidelity": "verbatim" }, { "name": "Git and Commit Rules", "fidelity": "verbatim" }, { "name": "Branching Model", "fidelity": "verbatim" }, { "name": "Release Model", "fidelity": "verbatim" }, { "name": "Operational Repositories", "fidelity": "verbatim", "appliesTo": ["operational"] }, { "name": "Repository Onboarding and Conformance", "fidelity": "verbatim" }, { "name": "Pull Request Title and Commit Message Conventions", "fidelity": "verbatim" }, { "name": "Documentation Style Conventions", "fidelity": "verbatim" }, { "name": "Verification Discipline", "fidelity": "verbatim" }, { "name": "PR Review Etiquette", "fidelity": "verbatim" }, { "name": "Communicating with the User", "fidelity": "verbatim" }, { "name": "Workflow YAML Conventions", "fidelity": "verbatim" }, { "name": "Supported Development Platforms", "fidelity": "verbatim" }, { "name": "Devcontainer", "fidelity": "intent" }, { "name": "Editor and Tasks", "fidelity": "verbatim" }, { "name": "Repository Details", "fidelity": "verbatim" }, { "name": "Repository Layout", "fidelity": "intent" }], "intentRef": "AGENTS.md", "appliesTo": "*" }, + { "path": "AGENTS.md", "fidelity": "intent", "sections": [{ "name": "Foundational Principles", "fidelity": "verbatim" }, { "name": "Repository Boundaries and Write Safety", "fidelity": "verbatim" }, { "name": "Git and Commit Rules", "fidelity": "verbatim" }, { "name": "Branching Model", "fidelity": "verbatim" }, { "name": "Release Model", "fidelity": "verbatim" }, { "name": "Operational Repositories", "fidelity": "verbatim" }, { "name": "Pull Request Title and Commit Message Conventions", "fidelity": "verbatim" }, { "name": "Documentation Style Conventions", "fidelity": "verbatim" }, { "name": "Verification Discipline", "fidelity": "verbatim" }, { "name": "PR Review Etiquette", "fidelity": "verbatim" }, { "name": "Communicating with the User", "fidelity": "verbatim" }, { "name": "Workflow YAML Conventions", "fidelity": "verbatim" }, { "name": "Supported Development Platforms", "fidelity": "verbatim" }, { "name": "Devcontainer", "fidelity": "intent" }, { "name": "Editor and Tasks", "fidelity": "verbatim" }, { "name": "Repository Details", "fidelity": "verbatim" }, { "name": "Repository Layout", "fidelity": "intent" }], "intentRef": "AGENTS.md", "appliesTo": "*" }, { "path": "CODESTYLE.md", "fidelity": "intent", "whole": true, "placeholders": ["InternalsVisibleTo project names"], "intentRef": "CODESTYLE.md", "appliesTo": "*" }, { "path": "WORKFLOW.md", "fidelity": "intent", "whole": true, "intentRef": "WORKFLOW.md", "appliesTo": "*" }, { "path": "README.md", "appliesTo": "*" }, diff --git a/spec/section-model.md b/spec/section-model.md index 9b1f2eaf..28282184 100644 --- a/spec/section-model.md +++ b/spec/section-model.md @@ -26,8 +26,7 @@ A section is one of the following. Fidelity is declared in [files.json][files], | Git and Commit Rules | verbatim | universal git law | | Branching Model | verbatim | universal (repo-specific history SHAs removed so it can carry) | | Release Model | verbatim | universal release contract, all target shapes described inline | -| Operational Repositories | verbatim, `appliesTo: operational` | universal for operational repos, N/A for release repos | -| Repository Onboarding and Conformance | verbatim | universal audit and registry obligations | +| Operational Repositories | verbatim | fleet context (the two workflow models); carried by all so the cross-references to it resolve | | Pull Request Title and Commit Message Conventions | verbatim | universal, with generic examples | | Documentation Style Conventions | verbatim | all rule text, including the Line Endings *rule* (a repo's `.editorconfig` value is not here) | | Verification Discipline | verbatim | universal verification law | @@ -40,6 +39,8 @@ A section is one of the following. Fidelity is declared in [files.json][files], | Devcontainer | intent | describes this repo's toolchain and devcontainer, genuinely per-repo | | Repository Layout | intent | describes this repo's directory tree, genuinely per-repo | +**Not carried (hub-only).** `Repository Onboarding and Conformance` lives in the hub's `AGENTS.md` as hub-audit context (reconciling the registry, the STANDUP cold-start, the conformance matrix) but is not a carried section - a downstream agent never runs those. Its one universal rule, that a repo is done when it passes `AUDIT.md` for its type, is carried in `AUDIT.md` itself. Like the model docs and `STANDUP.md`, it is hub machinery, not fleet law. + ## Changing the structure carries review weight The set of sections, and each section's fidelity, is itself governed. @@ -47,7 +48,7 @@ The set of sections, and each section's fidelity, is itself governed. - **Adding a section** is a fleet-wide act - it declares a new rule every repo must carry. Add it to `AGENTS.md`, to `files.json`, and to the table above in the same change, with its fidelity chosen deliberately. - **Changing a verbatim section** re-vendors it across the whole fleet. The audit drift wave that follows is the mechanism working, not a regression. - **Changing a section's fidelity** (intent to verbatim or back) is a governance decision, recorded here with its reason. -- **A downstream repo's extra section** the hub does not declare is drift to reconcile, not a local liberty. Either the rule belongs fleet-wide, so promote it here, or it is repo-specific content that does not belong in a carried section. +- **A downstream repo's extra section** the hub does not declare is drift to reconcile, not a local liberty - including a section whose *heading* differs but whose *content* duplicates a verbatim section (compare by content, not heading name). Either the rule belongs fleet-wide, so promote it here, or its unique part is repo-specific content that moves to the repo's own topical doc (`CODESTYLE`, `ARCHITECTURE`) and the duplicate is deleted. The audit lists a repo's undeclared sections as an advisory so the reconciliation is not missed. ## Enforcement From 664b081990c010016fa3b11fd56cc76d40bac4cb Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Fri, 24 Jul 2026 09:46:32 -0700 Subject: [PATCH 2/5] Address Copilot: needs block/scalar forms, advisory surfaces hub-only, path, lead phrase - normalize() masks the block-list and scalar needs: forms too, not just inline (with a guard test that masking a block list does not eat the next key). - Drop the Repository-Onboarding special-case from the undeclared-section advisory, so a repo still carrying that hub-only section is flagged to remove it. - AGENTS.md: use the full repo-relative path repo-config/operational/develop.json. - fidelity-model.md: the Normalization lead phrase now names all three dimensions. Co-Authored-By: Claude Opus 4.8 --- AGENTS.md | 2 +- spec/audit.py | 26 +++++++++++++++++--------- spec/fidelity-model.md | 2 +- 3 files changed, 19 insertions(+), 11 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0c3f70a6..ad9325d9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,7 +47,7 @@ A state-changing GitHub call is the highest-blast-radius thing an agent does her - *Main:* the check is graph-based - it asks whether main's tip commit is reachable from develop, not whether the two branches have the same content. After any develop -> main release, main's tip is a brand-new merge commit that develop's history doesn't contain. Forward-only develop never adds it (no back-merge of main into develop), so the check would fail on every subsequent release. Other technical workarounds - rebasing develop onto main, or rewriting develop's history - exist but contradict the squash-only develop ruleset and the linearity invariant. - *Develop:* the check stalls bot auto-merge when two bot PRs against develop land within the same window. As soon as the first merges, the second flips to `mergeStateStatus: BEHIND` and GitHub's auto-merge will not fire while strict is on. The merge-bot only *enables* auto-merge on `opened`/`reopened` (see below) and never auto-updates bot branches, and Dependabot's rebase isn't real-time, so the second PR sits OPEN with all checks green indefinitely. Squash mechanics still rebase the diff onto develop's tip on merge, `required_linear_history` still enforces linearity, textual conflicts still block `mergeable: CONFLICTING`, and the required `Check pull request workflow status job` still gates merges - the only thing lost is pre-merge detection of *semantic-but-not-textual* conflicts, which the post-merge develop CI run catches anyway. - See [`repo-config/README.md`](./repo-config/README.md) "Rulesets" for the configured state. -- **Configuring branch protection on a fleet repo: don't hand-build the rules.** Reconstructing the rules by hand is error-prone and has gone wrong on past ports. First delete **all** legacy classic branch-protection rules and any stray rulesets (rulesets are the *only* mechanism used), then create **exactly two rulesets named `develop` and `main`** by importing the committed `repo-config/*.json` ruleset payloads via `gh api -X POST "repos///rulesets"` (`gh ruleset` is read-only). The names are load-bearing - this file and the workflows reference them. Operational repos import `operational/develop.json` as their `develop` ruleset (the `main` ruleset is shared); [`configure.sh`](./repo-config/configure.sh) selects the right develop payload from the registry `workflowModel` automatically. **Brownfield repos** (pre-existing history) need an extra step: `Require signed commits` rejects legacy unsigned commits and the admin bypass does not cover `git push --force`, so re-signing requires temporarily disabling the ruleset. +- **Configuring branch protection on a fleet repo: don't hand-build the rules.** Reconstructing the rules by hand is error-prone and has gone wrong on past ports. First delete **all** legacy classic branch-protection rules and any stray rulesets (rulesets are the *only* mechanism used), then create **exactly two rulesets named `develop` and `main`** by importing the committed `repo-config/*.json` ruleset payloads via `gh api -X POST "repos///rulesets"` (`gh ruleset` is read-only). The names are load-bearing - this file and the workflows reference them. Operational repos import `repo-config/operational/develop.json` as their `develop` ruleset (the `main` ruleset is shared); [`configure.sh`](./repo-config/configure.sh) selects the right develop payload from the registry `workflowModel` automatically. **Brownfield repos** (pre-existing history) need an extra step: `Require signed commits` rejects legacy unsigned commits and the admin bypass does not cover `git push --force`, so re-signing requires temporarily disabling the ruleset. - **Bots (Dependabot and codegen) target both `main` and `develop` in parallel.** [`.github/dependabot.yml`](./.github/dependabot.yml) duplicates every ecosystem entry (one per branch) and `catalog/snippets/workflows/run-codegen-pull-request-task.yml` runs as a matrix over both branches with branch names `codegen-main` and `codegen-develop`. Each branch absorbs its own bot PRs independently, so neither falls behind, and the forward-only rule still holds (nothing is back-merged from main to develop - both branches receive their updates directly). The merge-bot ([`.github/workflows/merge-bot-pull-request.yml`](./.github/workflows/merge-bot-pull-request.yml)) dispatches `--squash` or `--merge` from each PR's base ref via a `case` statement so the form matches the ruleset on either base. Dependabot **security** PRs (CVE-driven) always open against the repo default branch (`main`) regardless of `target-branch` - the same `case` statement covers them. The merge-bot auto-merges **every** Dependabot tier including semver-major (no ecosystem or update-type guard): the required CI checks are the gate, not the bump magnitude, so a major that breaks the build fails its checks and never merges. - **Maintainer-pushed commits on a bot PR auto-disable auto-merge.** The merge-bot's `merge-dependabot` and `merge-codegen` jobs only fire on `opened` / `reopened` events (auto-merge is enabled exactly once per PR). When a maintainer pushes commits to a bot's branch (a `synchronize` event with an actor that isn't the same bot), the merge-bot's `disable-auto-merge-on-maintainer-push` job fires and calls `gh pr merge --disable-auto`. The maintainer's commits stay in the PR but won't auto-merge with the bot's content; re-enable auto-merge manually (`gh pr merge --auto ` or the GitHub UI) when ready. - **Why parallel dual-target rather than develop-only with eventual flow-through:** push-distribution channels (HACS for Home Assistant integrations, Linux distros that vendor from `main`, etc.) consume `main` directly. A develop-only model would leave `main` running stale code during long-running develop features. Codegen content can also be production-critical (live API-derived data, language lists, build catalogs) rather than just sample/demo content, so both branches need fresh codegen on their own cadence. diff --git a/spec/audit.py b/spec/audit.py index 3e23a59b..dceaa590 100644 --- a/spec/audit.py +++ b/spec/audit.py @@ -349,7 +349,12 @@ def check_interface(path, contract, text): # list to the targets it actually vendors - a `needs` entry naming an unvendored job fails the whole workflow to # load - so the list is owned per repo, not fixed. Mask the inline `[ ... ]` form the canonical uses, same as the # action pin. The interface contract still checks the required job keys separately. -_JOB_NEEDS = re.compile(r"(^[ \t]*needs:[ \t]*)\[[^\]]*\]", re.MULTILINE) +_JOB_NEEDS = re.compile( + r"(^[ \t]*)needs:[ \t]*" + r"(?:\[[^\]]*\]" # inline: needs: [a, b] + r"|[A-Za-z0-9_.\-]+[ \t]*(?=\n|$)" # scalar: needs: a + r"|(?:\n[ \t]+-[ \t]*[A-Za-z0-9_.\-]+[ \t]*)+)", # block: needs:\n - a\n - b + re.MULTILINE) def normalize(text): @@ -361,7 +366,7 @@ def normalize(text): """ text = text.replace("\r\n", "\n").replace("\r", "\n") text = _ACTION_PIN.sub(r"\1@", text) - return _JOB_NEEDS.sub(r"\1", text) + return _JOB_NEEDS.sub(r"\1needs: ", text) @functools.lru_cache(maxsize=1024) # bounded; the keys that recur across repos are the canonical and its history @@ -669,7 +674,6 @@ def audit_repo(entry, spec): # structure is governed by section-model.md. if path == "AGENTS.md": declared = {n.strip().lower() for n in (needed | verbatim_needed)} - declared.add("repository onboarding and conformance") # hub-only section, carried by no repo h2s = {ln[3:].strip().lower() for ln in text.splitlines() if ln.startswith("## ")} for h in sorted(h2s - declared): findings.append(("DRIFT", f"section: '{h}' in {path} is not a declared section - reconcile it (a duplicate of a verbatim section, or repo-specific content that moves to a topical doc), or confirm it is intentional (spec/section-model.md)")) @@ -823,17 +827,21 @@ def _selftest(): # needs-mask: a verbatim job region whose `needs:` list is pruned to the repo's vendored targets must not # count as drift (the list is owned), but a structural change to the job's steps must. - needs_full = " github-release:\n needs: [get-version, validate-release, build-nugetlibrary, build-executable]\n steps: []\n" - needs_pruned = " github-release:\n needs: [get-version, validate-release, build-executable]\n steps: []\n" - needs_forked = " github-release:\n needs: [get-version, validate-release]\n steps:\n - run: fork\n" - if content_hash(needs_full) != content_hash(needs_pruned): + needs_full = " github-release:\n needs: [get-version, validate-release, build-nugetlibrary, build-executable]\n runs-on: x\n steps: []\n" + needs_pruned = " github-release:\n needs: [get-version, validate-release, build-executable]\n runs-on: x\n steps: []\n" + needs_block = " github-release:\n needs:\n - get-version\n - build-executable\n runs-on: x\n steps: []\n" + needs_forked = " github-release:\n needs: [get-version, validate-release]\n runs-on: x\n steps:\n - run: fork\n" + if content_hash(needs_full) != content_hash(needs_pruned) or content_hash(needs_full) != content_hash(needs_block): ok = False - print(" FAIL needs-mask: a pruned needs list should normalize equal") + print(" FAIL needs-mask: a pruned needs list (inline or block) should normalize equal") elif content_hash(needs_full) == content_hash(needs_forked): ok = False print(" FAIL needs-mask: a step change must still hash differently") + elif "runs-on: x" not in normalize(needs_block): + ok = False + print(" FAIL needs-mask: masking a block needs list must not consume the next key") else: - print(" ok needs-mask: a pruned needs list normalizes equal, a forked step still differs") + print(" ok needs-mask: pruned needs (inline + block) normalizes equal, forked step differs, next key preserved") # Region extraction and hashing: a forked github-release block must hash differently from the canonical. region = split_jobs(rel_ok).get("github-release") diff --git a/spec/fidelity-model.md b/spec/fidelity-model.md index d5796e2f..093ad59e 100644 --- a/spec/fidelity-model.md +++ b/spec/fidelity-model.md @@ -30,7 +30,7 @@ The fixed interface of a workflow is stated in [`AGENTS.md`][agents] ("Orchestra ## Normalization -A verbatim check compares content by hash after **line-ending and action-pin normalization** - EOL variance is governed by the line-ending rules, a `uses: @` pin (with its trailing `# vN` comment) is Dependabot-owned and bumped per repo, and a job's `needs:` list is pruned per repo to its vendored targets (an unvendored name fails the workflow to load), so all three are governed drift rather than a fidelity deviation. This keeps a verbatim workflow job region (the `github-release` job) from flagging on a routine action bump or a legitimate needs-prune while still catching a real structural fork. It does **not** mask placeholders: a verbatim unit carries none. The files that declare a `placeholders` list (for example `.github/copilot-instructions.md` with ``, ``, ``) are fidelity `intent`, judged by hand and never hashed. Masking could not serve a hash anyway - a downstream copy holds the substituted value (`ptr727`), not the token (``), so masking the token in the canonical alone would guarantee a mismatch. A verbatim unit that ever needed a per-repo substitution would require template-matching (the canonical as a pattern, the copy as an instance), not this content hash. None does today. +A verbatim check compares content by hash after **line-ending, action-pin, and job-needs normalization** - EOL variance is governed by the line-ending rules, a `uses: @` pin (with its trailing `# vN` comment) is Dependabot-owned and bumped per repo, and a job's `needs:` list is pruned per repo to its vendored targets (an unvendored name fails the workflow to load), so all three are governed drift rather than a fidelity deviation. This keeps a verbatim workflow job region (the `github-release` job) from flagging on a routine action bump or a legitimate needs-prune while still catching a real structural fork. It does **not** mask placeholders: a verbatim unit carries none. The files that declare a `placeholders` list (for example `.github/copilot-instructions.md` with ``, ``, ``) are fidelity `intent`, judged by hand and never hashed. Masking could not serve a hash anyway - a downstream copy holds the substituted value (`ptr727`), not the token (``), so masking the token in the canonical alone would guarantee a mismatch. A verbatim unit that ever needed a per-repo substitution would require template-matching (the canonical as a pattern, the copy as an instance), not this content hash. None does today. ## Stale Versus Modified From 59edd61880323bf69ae0983d385b39cbf4a8c444 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Fri, 24 Jul 2026 09:51:03 -0700 Subject: [PATCH 3/5] Address Copilot: constrain the inline needs mask to a single line [^\]]* matched newlines, so an unclosed inline list could swallow across lines to a distant ]. Use [^\]\n]* so the inline branch stays on one line; the block branch already handles a genuine multi-line list. Co-Authored-By: Claude Opus 4.8 --- spec/audit.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/spec/audit.py b/spec/audit.py index dceaa590..f0458cb3 100644 --- a/spec/audit.py +++ b/spec/audit.py @@ -351,7 +351,7 @@ def check_interface(path, contract, text): # action pin. The interface contract still checks the required job keys separately. _JOB_NEEDS = re.compile( r"(^[ \t]*)needs:[ \t]*" - r"(?:\[[^\]]*\]" # inline: needs: [a, b] + r"(?:\[[^\]\n]*\]" # inline: needs: [a, b] (same line only) r"|[A-Za-z0-9_.\-]+[ \t]*(?=\n|$)" # scalar: needs: a r"|(?:\n[ \t]+-[ \t]*[A-Za-z0-9_.\-]+[ \t]*)+)", # block: needs:\n - a\n - b re.MULTILINE) From 18da4e1548c7482763d76df50b93a842f07cb7f7 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Fri, 24 Jul 2026 09:55:59 -0700 Subject: [PATCH 4/5] Address Copilot: needs comment names all 3 forms; selftest adds scalar Co-Authored-By: Claude Opus 4.8 --- spec/audit.py | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/spec/audit.py b/spec/audit.py index f0458cb3..a7175daf 100644 --- a/spec/audit.py +++ b/spec/audit.py @@ -347,8 +347,8 @@ def check_interface(path, contract, text): _ACTION_PIN = re.compile(r"(\buses:[ \t]*[^\s@]+)@[0-9a-fA-F]{40}(?:[ \t]+#[ \t]*v?[0-9][\w.\-]*)?") # A workflow job's `needs:` list names the jobs it sequences after. In a verbatim job region a repo prunes that # list to the targets it actually vendors - a `needs` entry naming an unvendored job fails the whole workflow to -# load - so the list is owned per repo, not fixed. Mask the inline `[ ... ]` form the canonical uses, same as the -# action pin. The interface contract still checks the required job keys separately. +# load - so the list is owned per repo, not fixed. Mask it (the inline `[ ... ]`, scalar, and block-list forms), +# same as the action pin. The interface contract still checks the required job keys separately. _JOB_NEEDS = re.compile( r"(^[ \t]*)needs:[ \t]*" r"(?:\[[^\]\n]*\]" # inline: needs: [a, b] (same line only) @@ -830,10 +830,11 @@ def _selftest(): needs_full = " github-release:\n needs: [get-version, validate-release, build-nugetlibrary, build-executable]\n runs-on: x\n steps: []\n" needs_pruned = " github-release:\n needs: [get-version, validate-release, build-executable]\n runs-on: x\n steps: []\n" needs_block = " github-release:\n needs:\n - get-version\n - build-executable\n runs-on: x\n steps: []\n" + needs_scalar = " github-release:\n needs: build-executable\n runs-on: x\n steps: []\n" needs_forked = " github-release:\n needs: [get-version, validate-release]\n runs-on: x\n steps:\n - run: fork\n" - if content_hash(needs_full) != content_hash(needs_pruned) or content_hash(needs_full) != content_hash(needs_block): + if len({content_hash(needs_full), content_hash(needs_pruned), content_hash(needs_block), content_hash(needs_scalar)}) != 1: ok = False - print(" FAIL needs-mask: a pruned needs list (inline or block) should normalize equal") + print(" FAIL needs-mask: a pruned needs list (inline, block, or scalar) should normalize equal") elif content_hash(needs_full) == content_hash(needs_forked): ok = False print(" FAIL needs-mask: a step change must still hash differently") @@ -841,7 +842,7 @@ def _selftest(): ok = False print(" FAIL needs-mask: masking a block needs list must not consume the next key") else: - print(" ok needs-mask: pruned needs (inline + block) normalizes equal, forked step differs, next key preserved") + print(" ok needs-mask: pruned needs (inline, block, scalar) normalizes equal, forked step differs, next key preserved") # Region extraction and hashing: a forked github-release block must hash differently from the canonical. region = split_jobs(rel_ok).get("github-release") From f36fb126d7329ac2fd11b2fc5c43a2d30987519d Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Fri, 24 Jul 2026 10:03:10 -0700 Subject: [PATCH 5/5] Address Copilot: skip the hub itself in the undeclared-section advisory The hub's own AGENTS.md legitimately carries hub-only sections (Repository Onboarding and Conformance), so auditing ProjectTemplate against itself would false-positive on them. Skip the advisory when the repo is the hub; a downstream repo carrying such a section is still flagged. Co-Authored-By: Claude Opus 4.8 --- spec/audit.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/spec/audit.py b/spec/audit.py index a7175daf..9411112b 100644 --- a/spec/audit.py +++ b/spec/audit.py @@ -672,7 +672,10 @@ def audit_repo(entry, spec): # a repo may legitimately carry its own project-specific sections (the AGENTS.md preamble allows # them) - so it points at the reconciliation, it never fails. AGENTS.md only, where the section # structure is governed by section-model.md. - if path == "AGENTS.md": + # Skip the hub itself: its AGENTS.md is the source and legitimately holds hub-only sections + # (e.g. Repository Onboarding and Conformance) that are deliberately not carried. A downstream + # repo carrying such a section is still flagged, which is the point. + if path == "AGENTS.md" and entry.get("name") != HUB_NAME: declared = {n.strip().lower() for n in (needed | verbatim_needed)} h2s = {ln[3:].strip().lower() for ln in text.splitlines() if ln.startswith("## ")} for h in sorted(h2s - declared):