From 346930dc05384197dc991d04148e47f9085664bd Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Thu, 9 Jul 2026 16:56:57 -0700 Subject: [PATCH 1/5] Coverage hygiene: gitignore coverage output + codecov.yml standard .gitignore now excludes coverage artifacts (coverage/, *.cobertura.xml, coverage.xml, .coverage, htmlcov/) so a blanket git add -A never commits them. Add a reference codecov.yml snippet (project+patch informational so a coverage delta never gates a PR - distinct from fail_ci_if_error - plus a commented ignore for untested example/benchmark apps), list it in spec/files.json for csharp/python, and extend WORKFLOW.md D1.6. Co-Authored-By: Claude Opus 4.8 (1M context) --- .gitignore | 10 ++++++++++ WORKFLOW.md | 2 +- catalog/snippets/configs/codecov.yml | 19 +++++++++++++++++++ spec/files.json | 1 + 4 files changed, 31 insertions(+), 1 deletion(-) create mode 100644 catalog/snippets/configs/codecov.yml diff --git a/.gitignore b/.gitignore index 0db2f387..3375dd1b 100644 --- a/.gitignore +++ b/.gitignore @@ -18,3 +18,13 @@ dist/ .pytest_cache/ .ruff_cache/ .pyright/ + +# Coverage output (dotnet XPlat/coverlet, Python coverage.py) +coverage/ +[Tt]est[Rr]esults/ +*.cobertura.xml +*.coverage +coverage.xml +.coverage +.coverage.* +htmlcov/ diff --git a/WORKFLOW.md b/WORKFLOW.md index 77536428..eb52fb0a 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -133,7 +133,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as input - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated `!smoke`). *Prevents: a PR publishing; orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter excludes workflow files, so smoke-build skips. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, `needs:` the changes job and the validation job, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). *Prevents: coverage silently going unreported; a stale, unused token.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR - a distinct knob from `fail_ci_if_error` (which only guards the upload step) - and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`; HA-integration repos override this to enforce a threshold. Coverage output is a build artifact - `.gitignore` excludes it (`coverage/`, `*.cobertura.xml`, `coverage.xml`, `.coverage`) so a blanket `git add -A` never commits it. *Prevents: coverage silently going unreported; a stale, unused token; a coverage regression blocking an unrelated PR; a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry diff --git a/catalog/snippets/configs/codecov.yml b/catalog/snippets/configs/codecov.yml new file mode 100644 index 00000000..7299937c --- /dev/null +++ b/catalog/snippets/configs/codecov.yml @@ -0,0 +1,19 @@ +# Codecov configuration. Coverage is reported and trended, never gated: +# informational: true keeps Codecov's project and patch commit statuses +# advisory (always pass) so a coverage delta can never block a PR - a distinct +# knob from the upload step's fail_ci_if_error: false, which only guards upload +# errors. HA-integration repos override this to enforce a coverage threshold. +coverage: + status: + project: + default: + informational: true + patch: + default: + informational: true + +# Exclude code intentionally not unit-tested from the coverage denominator - +# e.g. an example/demo console app or a benchmark project, not the shipped +# library. Add the repo's own paths. +# ignore: +# - "Sandbox/**" diff --git a/spec/files.json b/spec/files.json index 9b59d284..c7e16e21 100644 --- a/spec/files.json +++ b/spec/files.json @@ -16,6 +16,7 @@ { "path": "repo-config/main.json", "intentRef": "repo-config/README.md", "appliesTo": "*" }, { "path": ".github/dependabot.yml", "appliesTo": "*" }, { "path": ".vscode/tasks.json", "sections": ["clean-compile task group"], "reference": "catalog/snippets/configs/vscode-tasks.json", "appliesTo": ["csharp", "python"] }, + { "path": "codecov.yml", "reference": "catalog/snippets/configs/codecov.yml", "intentRef": "WORKFLOW.md", "appliesTo": ["csharp", "python"] }, { "path": ".dockerignore", "appliesTo": ["docker"] }, { "path": "Docker/README.md", "reference": "catalog/snippets/configs/docker-hub-readme.md", "appliesTo": ["docker"] } ] From 2b8841fcc1e2c4d43556fa16f97c56fb1cc1801f Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Thu, 9 Jul 2026 17:00:09 -0700 Subject: [PATCH 2/5] WORKFLOW.md: mark D1.6 coverage-artifact list as illustrative The contract doc points at .gitignore as the source of truth rather than duplicating the full pattern set (avoids two places to maintain). Co-Authored-By: Claude Opus 4.8 (1M context) --- WORKFLOW.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/WORKFLOW.md b/WORKFLOW.md index eb52fb0a..76679dec 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -133,7 +133,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as input - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated `!smoke`). *Prevents: a PR publishing; orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter excludes workflow files, so smoke-build skips. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, `needs:` the changes job and the validation job, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR - a distinct knob from `fail_ci_if_error` (which only guards the upload step) - and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`; HA-integration repos override this to enforce a threshold. Coverage output is a build artifact - `.gitignore` excludes it (`coverage/`, `*.cobertura.xml`, `coverage.xml`, `.coverage`) so a blanket `git add -A` never commits it. *Prevents: coverage silently going unreported; a stale, unused token; a coverage regression blocking an unrelated PR; a coverage artifact committed by a blanket add.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR - a distinct knob from `fail_ci_if_error` (which only guards the upload step) - and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`; HA-integration repos override this to enforce a threshold. Coverage output is a build artifact - `.gitignore` excludes it (e.g. `coverage/`, `*.cobertura.xml`; `.gitignore` is the full source of truth) so a blanket `git add -A` never commits it. *Prevents: coverage silently going unreported; a stale, unused token; a coverage regression blocking an unrelated PR; a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry From 5839f8df63fa0409cd557fdfd3e87728a6b29525 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Thu, 9 Jul 2026 17:03:31 -0700 Subject: [PATCH 3/5] WORKFLOW.md: precise D1.6 wording (gitignore prevents staging untracked coverage output) Co-Authored-By: Claude Opus 4.8 (1M context) --- WORKFLOW.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/WORKFLOW.md b/WORKFLOW.md index 76679dec..34d2b541 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -133,7 +133,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as input - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated `!smoke`). *Prevents: a PR publishing; orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter excludes workflow files, so smoke-build skips. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, `needs:` the changes job and the validation job, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR - a distinct knob from `fail_ci_if_error` (which only guards the upload step) - and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`; HA-integration repos override this to enforce a threshold. Coverage output is a build artifact - `.gitignore` excludes it (e.g. `coverage/`, `*.cobertura.xml`; `.gitignore` is the full source of truth) so a blanket `git add -A` never commits it. *Prevents: coverage silently going unreported; a stale, unused token; a coverage regression blocking an unrelated PR; a coverage artifact committed by a blanket add.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR - a distinct knob from `fail_ci_if_error` (which only guards the upload step) - and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`; HA-integration repos override this to enforce a threshold. Coverage output is a build artifact - `.gitignore` excludes it (e.g. `coverage/`, `*.cobertura.xml`; `.gitignore` is the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported; a stale, unused token; a coverage regression blocking an unrelated PR; a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry From ddc4bfc8fe62ea507a9bf2b9ea6696f3b2a9ea84 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Thu, 9 Jul 2026 17:06:29 -0700 Subject: [PATCH 4/5] WORKFLOW.md: state coverage-threshold override as a portable rule, not HA-specific Co-Authored-By: Claude Opus 4.8 (1M context) --- WORKFLOW.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/WORKFLOW.md b/WORKFLOW.md index 34d2b541..744a8c6e 100644 --- a/WORKFLOW.md +++ b/WORKFLOW.md @@ -133,7 +133,7 @@ The required behaviors, organized by domain. Each is a **MUST**, stated as input - **D1.3 Smoke never publishes and never uploads.** Input: `smoke: true`. Output: full compile/lint/test, but no registry/image push, no release, and **no** artifact uploads (every `upload-artifact`, including any aggregation job, is gated `!smoke`). *Prevents: a PR publishing; orphaned artifacts churning the storage quota.* - **D1.4 Workflow-file changes are not smoke-built.** Input: a PR changing only `.github/workflows/**`. Output: the paths-filter excludes workflow files, so smoke-build skips. *Implication: a workflow-only change is not smoke-built, but actionlint still validates it in CI.* - **D1.5 One required aggregator gates merge.** Input: any PR. Output: a single aggregator job must **succeed**, `needs:` the changes job and the validation job, treat a **skipped** smoke build as pass, and **block** on `failure`/`cancelled`. Its name is ruleset-bound: the job `name:` and the ruleset `context:` are the same string and MUST be renamed together, never independently. *Prevents: a paths-filter error letting a target-changing PR merge unbuilt.* -- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR - a distinct knob from `fail_ci_if_error` (which only guards the upload step) - and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`; HA-integration repos override this to enforce a threshold. Coverage output is a build artifact - `.gitignore` excludes it (e.g. `coverage/`, `*.cobertura.xml`; `.gitignore` is the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported; a stale, unused token; a coverage regression blocking an unrelated PR; a coverage artifact committed by a blanket add.* +- **D1.6 Coverage is reported to Codecov (C# and Python).** Input: a C# or Python repo's validation/test job. Output: tests run with coverage collection (`dotnet test --collect:"XPlat Code Coverage"` or `pytest --cov-report=xml`) and a `codecov/codecov-action` step uploads it, **best-effort** (`continue-on-error` and/or `fail_ci_if_error: false`, so a Codecov outage or an absent token never reds the gate); `CODECOV_TOKEN` lives in the repo's **actions** secret store and reaches the reusable validator via `secrets: inherit`. Required for **every** C# and Python repo that has tests (see `spec/secrets.json` `typeMechanisms`). The repo also ships a **`codecov.yml`** that sets the project and patch statuses to **`informational: true`** so a coverage delta never gates a PR - a distinct knob from `fail_ci_if_error` (which only guards the upload step) - and excludes intentionally-untested, non-shipped code (an example/demo or benchmark project) from the coverage denominator via `ignore`; a repo may override this to enforce a coverage threshold where its quality bar requires it. Coverage output is a build artifact - `.gitignore` excludes it (e.g. `coverage/`, `*.cobertura.xml`; `.gitignore` is the full source of truth) so a blanket `git add -A` won't stage the untracked output. *Prevents: coverage silently going unreported; a stale, unused token; a coverage regression blocking an unrelated PR; a coverage artifact committed by a blanket add.* ### D2 - Input/State Validation at Entry From 86547a60720ffbbb912ca5a7d66c5f868194e346 Mon Sep 17 00:00:00 2001 From: Pieter Viljoen Date: Thu, 9 Jul 2026 17:08:54 -0700 Subject: [PATCH 5/5] codecov snippet: match WORKFLOW.md - portable threshold-override wording Co-Authored-By: Claude Opus 4.8 (1M context) --- catalog/snippets/configs/codecov.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/catalog/snippets/configs/codecov.yml b/catalog/snippets/configs/codecov.yml index 7299937c..eff9a54d 100644 --- a/catalog/snippets/configs/codecov.yml +++ b/catalog/snippets/configs/codecov.yml @@ -2,7 +2,7 @@ # informational: true keeps Codecov's project and patch commit statuses # advisory (always pass) so a coverage delta can never block a PR - a distinct # knob from the upload step's fail_ci_if_error: false, which only guards upload -# errors. HA-integration repos override this to enforce a coverage threshold. +# errors. A repo may override this to enforce a coverage threshold. coverage: status: project: