You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Retrospective + roadmap from the Financial-Modeling onboarding, which exposed that a repo could sit partially onboarded (missing WORKFLOW.md, version.json, rulesets, dependabot.yml) with a stale registry entry (releaseTrigger: none, "no workflows yet") while CI stayed green — and the gaps surfaced reactively through issues (#10, #11, #306) instead of one audit.
Honest root cause: the map was right; nobody was required to walk the territory
The hub already encodes what a conformant repo of each type needs, and running AUDIT.md against Financial-Modeling would have flagged every gap in one report:
spec/files.json lists WORKFLOW.md, version.json, repo-config/develop.json+main.json, .github/dependabot.yml as baseline (appliesTo: "*").
spec/project-types.jsonsource-only requires version.json+NBGV and expect_release_assets:false; branch-model/repo-setup require the rulesets and secrets.
AUDIT.md is a complete §1–10 read-only audit-and-converge procedure composing all of it.
So the failure was operational, not definitional:
The audit was never run, and nothing forced it. No reports/Financial-Modeling/audit.md; onboarding proceeded by reaction.
spec/validate.py only checks the catalog is self-consistent, never that live repos match it — which is exactly why the stale entry passed CI green while being wrong.
There is an AUDIT (verify existing) but no STANDUP (create-to-conformance) counterpart — onboarding had no ordered procedure and no machine-checkable definition of done.
No self-test proving the docs are sufficient for a cold agent to stand a type up.
This aligns with the downward-audit architecture (standard-style repos; the hub audits down; the catalog lists supported types; onboarding a new type is itself a process) — the machinery was there, the obligation to run it was not.
Plan (delivery: sliced Copilot-reviewed PRs; automation: docs + matrix first, live runner next)
A. Enforcement / definition-of-done — AGENTS.md + an AUDIT.md §0: creating or materially touching a fleet repo requires running the audit and reconciling its registry entry; a repo is not "onboarded" until it passes or has a tracked reports/<repo>/audit.md + issue for residual deltas. A half-onboarded repo with no audit report is itself a defect.
B. STANDUP.md — the inverse of AUDIT.md: ordered per-type steps zero→operational, reusing files.json / project-types.json / repo-config / secrets.json, with "AUDIT.md passes" as done.
C. Cold-start self-test / meta-audit — for each supported type, a context-free agent stands the repo up from only the hub docs, then AUDIT.md must pass; iterate the docs/tooling until every type stands up cold. Delivered as a harness doc + a type × cold-standup-passes conformance matrix. Doubles as the acceptance test for the onboarding docs and the on-demand audit for known repos.
D. Live fleet-audit runner (queued next) — extend past validate.py's static check with a python3 + gh runner that executes AUDIT.md §6 (settings/rulesets/secrets diff) + files.json presence across every registry repo and flags drift and stale registry fields. Would have caught this incident automatically.
Each supported repo type can be (1) stood up cold to operational from only the hub docs, and (2) audited to full conformance with an output that lists deviations + repo-specific deltas. The same audit runs on-demand against any known repo.
Retrospective + roadmap from the Financial-Modeling onboarding, which exposed that a repo could sit partially onboarded (missing
WORKFLOW.md,version.json, rulesets,dependabot.yml) with a stale registry entry (releaseTrigger: none, "no workflows yet") while CI stayed green — and the gaps surfaced reactively through issues (#10, #11, #306) instead of one audit.Honest root cause: the map was right; nobody was required to walk the territory
The hub already encodes what a conformant repo of each type needs, and running
AUDIT.mdagainst Financial-Modeling would have flagged every gap in one report:spec/files.jsonlistsWORKFLOW.md,version.json,repo-config/develop.json+main.json,.github/dependabot.ymlas baseline (appliesTo: "*").spec/project-types.jsonsource-onlyrequiresversion.json+NBGV andexpect_release_assets:false;branch-model/repo-setuprequire the rulesets and secrets.AUDIT.mdis a complete §1–10 read-only audit-and-converge procedure composing all of it.So the failure was operational, not definitional:
reports/Financial-Modeling/audit.md; onboarding proceeded by reaction.spec/validate.pyonly checks the catalog is self-consistent, never that live repos match it — which is exactly why the stale entry passed CI green while being wrong.This aligns with the downward-audit architecture (standard-style repos; the hub audits down; the catalog lists supported types; onboarding a new type is itself a process) — the machinery was there, the obligation to run it was not.
Plan (delivery: sliced Copilot-reviewed PRs; automation: docs + matrix first, live runner next)
AGENTS.md+ anAUDIT.md §0: creating or materially touching a fleet repo requires running the audit and reconciling its registry entry; a repo is not "onboarded" until it passes or has a trackedreports/<repo>/audit.md+ issue for residual deltas. A half-onboarded repo with no audit report is itself a defect.STANDUP.md— the inverse ofAUDIT.md: ordered per-type steps zero→operational, reusingfiles.json/project-types.json/repo-config/secrets.json, with "AUDIT.md passes" as done.AUDIT.mdmust pass; iterate the docs/tooling until every type stands up cold. Delivered as a harness doc + atype × cold-standup-passesconformance matrix. Doubles as the acceptance test for the onboarding docs and the on-demand audit for known repos.validate.py's static check with apython3 + ghrunner that executesAUDIT.md §6(settings/rulesets/secrets diff) +files.jsonpresence across every registry repo and flags drift and stale registry fields. Would have caught this incident automatically.files.jsonPython.vscode/tasks.json→vscode-tasks-python.json;python.uvlock.pinnedcheck added.Acceptance criteria
Each supported repo type can be (1) stood up cold to operational from only the hub docs, and (2) audited to full conformance with an output that lists deviations + repo-specific deltas. The same audit runs on-demand against any known repo.
Incident artifacts