Add WORKFLOW.md: workflow style, architecture, behavioral contract, test methodology - #223
Merged
Conversation
…odology A standalone guide for CI/CD workflows, sibling to CODESTYLE.md. Its defining principle is contract-not-implementation: it states required outcomes (inputs -> outputs) and a test methodology, so an agent can audit, test, and assess whether any project's workflows are operational - without dictating how they are built. Mined from the full aggregate of the recent template/porting work (~40 workflow issues/PRs), not just the latest defects. Behaviors are organized into nine domains (D1 fast-feedback, D2 input/state validation, D3 versioning, D4 release, D5 resource cleanup, D6 seam, D7 concurrency/permissions, D8 bots, D9 style), each guarantee stated as the failure-mode it prevents (portable, no issue numbers). Section 5 gives a static-audit checklist, ten end-to-end trace scenarios, and a live-probe procedure; section 6 walks each project type. Scope: new file only; the AGENTS.md "Workflow YAML Conventions"/"Release Model" reconciliation is deferred until this doc is ratified (noted inline). Verification: markdownlint clean; CRLF per .editorconfig; the section 5A checklist run against the template's own workflows reports 20/20 OPERATIONAL, calibrating the methodology. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new repo-root documentation artifact, WORKFLOW.md, intended to be the single reference for GitHub Actions workflow style, architecture, behavioral guarantees (inputs -> outputs), and a verification methodology so workflow reviews can be contract-based rather than implementation-based.
Changes:
- Introduce
WORKFLOW.mdas the consolidated workflow guide (style + architecture + behavioral contract + test methodology). - Define domain-based guarantees (D1-D9) and a structured verification approach (static audit, trace scenarios, live probes).
- Provide per-project-type walkthroughs mapping the contract to common template shapes (console, NuGet, PyPI, Docker, etc.).
Six per-type domain assessors (console, nuget, pypi, docker, data/asset, source-only) reviewed v1 for usability. Cross-cutting fixes: - Define "applicable" / N-A and exclude N-A items from the verdict (near-empty pipelines like source-only are mostly N-A, not failing). - Disambiguate the orchestrator vs build-leaf layers; assert each input in the file that declares it (github/nuget/dockerhub/expect_release_assets are release-task inputs, not leaf inputs). - A no-file-target repo (docker/pypi/source-only) reaches the tag-only shape only by its caller passing expect_release_assets:false; reconcile with the "verbatim orchestration" claim. - D1.2: a type-appropriate validation job (not specifically the .NET unit-test) runs unconditionally; non-.NET repos replace it and keep the aggregator wired. - NBGV/version.json are retained even by no-compiler repos (they own the tag). - Adding/dropping a target edits the orchestrator surface (enable input + job + needs + paths-filter); "verbatim" applies to the github-release job body. Per-type fixes: PyPI version is .dev0 from AssemblyFileVersion on develop only (not generic .devN off SemVer2) + ordering property; PyPI build-artifact delete is unconditional (diverges from release-asset gating on no-op, S9); per-runtime executable intermediates rely on the retention backstop (reconciled with D5.1); D3.5 wrapper external-version is a tracker-only skeleton (leaf must wire the read); NuGet snupkg triple-surface + --skip-duplicate server-dedupe semantics; Docker expect_release_assets/buildcache/secret-on-smoke/readme-validation; new S11 wrapper-bump scenario; per-registry 5C probes; 7z-not-zip; default-branch literal agreement check. Verification: markdownlint clean; CRLF; no issue numbers; the 5A static audit re-run against the template's own workflows still reports 20/20 OPERATIONAL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Round-2 re-review: 4 of 6 assessors returned no actionable feedback; docker and data/asset each raised 2 minors (no blocking/important). Fixes: - Docker: the single-image template buildcache is per-branch, not "per-image" (per-image is the multi-image distinguisher); mark the wrapper external-tag 5A check wrapper-only so it doesn't flag a correct plain Docker repo (records N/A). - Validation-job replacement: a non-.NET repo must re-point BOTH the aggregator and smoke-build (which needs: the validation job by name) to the replacement, else the workflow fails to load (D1.2 generalized; data + source-only noted). - Data/asset S9: qualify "release-create + asset-delete skip" to the scheduled re-run; a workflow_dispatch re-run refreshes and re-runs the asset-delete. Verification: markdownlint clean; CRLF; no issue numbers; dogfood static audit against the template's own workflows stays OPERATIONAL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Reflow to one logical paragraph per line (no hard-wrap), per the repo's
markdown convention (AGENTS.md "Documentation Style"); content byte-identical
modulo whitespace and the collapsed multi-line blockquote.
- Rewrite the canonical-scope note in the current state (not as a future
ratification plan), stating that AGENTS.md is authoritative where section 2 /
release policy overlap and wins on conflict.
- Soften the opening ("The guide", not "The single guide").
- Replace the split/mismatched bold markers in the Docker wrapper 5A note with
plain parentheses.
markdownlint clean; CRLF preserved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Resolves Copilot review: headings follow the title-case-with-short-bind-words rule (AGENTS.md), matching sibling CODESTYLE.md. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ptr727
added a commit
that referenced
this pull request
Jul 4, 2026
Promote the accumulated `develop` work to `main`. Twelve changes since the last promotion: - Add `WORKFLOW.md`: workflow style, architecture, behavioral contract, test methodology (#223) - Use NBGV `IGNORE_GITHUB_REF` instead of the ineffective `GITHUB_REF` override (#222) - Skip validate-release on smoke builds (#220) - Gate asset delete on the release create/refresh condition (#218) - Ship branch rulesets as versioned JSON in the re-sync / drift loop (#212) - Clean up transfer artifacts surgically at consumption, not blanket-delete (#216) - Version each publish leg against its own branch; validate at entry (#215) - Consolidate workspace configurations into a unified ProjectTemplate workspace (#210) - Template convergence barrier: absorb pins, generic release + docker-readme, carry-whole-file (#207) - Key merge-bot concurrency on PR number, not `github.ref` (#206) - Codify orchestrated re-sync personas and full-replacement rule (#205) - Lead action pins; affirm pattern-based artifact handoff (#204) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A week-plus of template work surfaced ~40 workflow issues/PRs and a cascade of behavioral defects (a leg published as stable; cleanup that orphaned the quota then destroyed diagnostics then deleted still-needed assets; a gate that blocked every promotion; an override that did nothing). Each was an outcome failure that cost rounds of PRs to find. There was no single artifact stating what a workflow must do and how to prove it.
What
WORKFLOW.mdat the repo root — a standalone guide, sibling toCODESTYLE.md, mixing code style, architecture, a behavioral contract (expected inputs→outputs), and a test methodology.Defining principle: contract, not implementation. A repo's workflows may be shaped differently; they must satisfy the contract and be defect-free against the expected inputs/outputs. Given the doc, an agent can audit (static), test (trace + probe), and assess (operational / not).
Hardening (multi-agent per-type review, converged)
Six per-domain assessors (console, NuGet, PyPI, Docker, data/asset, source-only) each used the doc to audit/test/assess their project type against the real workflows, over three rounds:
expect_release_assets:false; D1.2 is a type-appropriate validation job (non-.NET repos replace it); NBGV/version.jsonretained even by no-compiler repos; adding/dropping a target edits the orchestrator surface. Plus per-type precision (PyPI.dev0from AssemblyFileVersion; PyPI unconditional post-publish cleanup; executable intermediates on the retention backstop; D3.5 wrapper is a tracker skeleton; NuGet snupkg; new S11 wrapper-bump scenario; 7z-not-zip; …).smoke-build'sneedstoo; S9 schedule-vs-dispatch).Verification
markdownlint-cli2clean; CRLF per.editorconfig; no template issue numbers (portability).Scope
New file only. The
AGENTS.md"Workflow YAML Conventions" / "Release Model" reconciliation (move + pointer, likeCODESTYLE.md) is deferred until this doc is ratified — noted inline at the top of the file.Requesting ratification before the AGENTS.md reconciliation and downstream rollout.
🤖 Generated with Claude Code