Skip to content

ci(coverage): gate on patch coverage via Codecov, not global threshold - #763

Merged
JSONbored merged 1 commit into
mainfrom
claude/gifted-tesla-3d8eb6
Jun 16, 2026
Merged

ci(coverage): gate on patch coverage via Codecov, not global threshold#763
JSONbored merged 1 commit into
mainfrom
claude/gifted-tesla-3d8eb6

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Problem

The repo's 97% coverage gate was a global/project-total threshold in vitest.config.ts. A global ratio is non-compositional: each open PR is measured against a moving baseline, so the first PR to merge consumes the buffer above 97% and knocks every other open PR below the gate — even PRs that changed nothing coverage-relevant. The result is constant churn: merge one PR, watch the rest go red, force everyone to re-sync and re-test.

The current numbers prove it — branches and functions sit at exactly 97.00% (13281/13691 and 3566/3676): zero buffer. A single uncovered branch in any merge failed all in-flight PRs.

Fix

Move the 97% requirement to Codecov patch (changed-lines) coverage, which depends only on a PR's own diff and is immune to what else merges. The bar is unchanged for new code — arguably stricter, since every PR must now test its own changes rather than hiding undertested code behind a global buffer.

File Change
vitest.config.ts Emit lcov; drop hard 97% global thresholds to a loose 90% backstop (catastrophe net only — actual coverage is ~97–99%)
codecov.yml (new) patch target 97% (the real gate); project informational (tracked, non-blocking)
.github/workflows/ci.yml Upload coverage to Codecov (SHA-pinned codecov-action@v5.5.5); reword gate guidance
CONTRIBUTING.md Document patch-coverage rule; drop the "aim for 98% to absorb CI variance" band-aid

Verified locally

  • actionlint clean
  • npm run test:coverage passes; coverage/lcov.info emitted (92 files) at the path the upload step reads
  • Global coverage 97–99% — comfortably above the new 90% backstop

Follow-up (manual, after merge)

  1. Add codecov/patch as a required status check in branch protection.
  2. Open PRs adopt the new gate by syncing their branch with main (CI checks out the PR head SHA, so a re-run alone isn't enough) — the same "sync with main" step that previously broke them now fixes them.

Codecov app is installed and CODECOV_TOKEN is set.

The global 97% vitest threshold was non-compositional: each PR was measured
against a moving baseline, so the first PR to merge consumed the buffer above
97% and knocked every other open PR below the gate. Branches and functions were
sitting at exactly 97.00% (zero buffer), so a single uncovered branch in any
merge failed all in-flight PRs and forced everyone to re-sync and re-test.

Move the 97% requirement to Codecov's patch (changed-lines) status, which
depends only on a PR's own diff and is unaffected by what else merges:

- vitest.config.ts: emit lcov; drop the hard 97% global thresholds to a loose
  90% backstop (catastrophe net only, far below actual ~97-99%).
- codecov.yml: patch target 97% (the real gate); project informational only.
- ci.yml: upload coverage to Codecov (SHA-pinned v5.5.5); reword gate guidance.
- CONTRIBUTING.md: document patch coverage; drop the "aim for 98% to absorb
  CI variance" band-aid.

The 97% bar is unchanged for new code and arguably stricter: every PR must now
test its own changes, instead of hiding undertested code behind a global buffer.
@ghost

ghost commented Jun 16, 2026

Copy link
Copy Markdown

Note

Gittensory Gate skipped

PR closed before full evaluation. No late first comment was created.

Signal Result Evidence Action
Gate result ⚠️ Skipped #763 is no longer open. No action.

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

@ghost ghost added the gittensory:reviewed label Jun 16, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jun 16, 2026
@ghost

ghost commented Jun 16, 2026

Copy link
Copy Markdown

gittensory · advisory review

Reviewed 4 changed file(s) — two independent AI reviewers.

Changed files: .github/workflows/ci.yml, CONTRIBUTING.md, codecov.yml, vitest.config.ts

Suggested action:Safe to merge — both reviewers found no blocking issues.

Reviewer A · gpt-oss-120b — recommends ✅ merge
This PR introduces a Codecov patch‑coverage gate, adds a Codecov config, adjusts CI to upload coverage, and lowers the global vitest thresholds to a safe 90% backstop while updating documentation. The changes are straightforward and do not affect runtime code or security boundaries.

Suggestions

  • Verify that the CODECOV_TOKEN secret is configured in the repository settings so the upload step succeeds.
  • Confirm that the npm run test:coverage step runs before the Codecov upload in the CI workflow (the diff only shows the tail of the job).
  • Consider adding a brief comment in vitest.config.ts noting why the thresholds were lowered to 90% for future maintainers.

Worth double-checking

  • If the CI job order is incorrect, the lcov.info file might not exist when Codecov runs, causing a failed upload.
  • Missing or mis‑named CODECOV_TOKEN secret would break the coverage upload step.

Reviewer B · nemotron-3-120b-a12b — recommends ✅ merge
The PR shifts coverage enforcement from a global vitest threshold (97%) to a patch-based gate via Codecov (97% on changed lines), with vitest thresholds lowered to 90% as a catastrophe backstop. This eliminates cross-PR churn while maintaining quality guards. Changes are well-documented, logically sound, and align with the project's testing conventions.

Worth double-checking

  • Ensure CODECOV_TOKEN secret is configured in the repo (required for Codecov action to function).
  • Verify that 90% vitest thresholds are sufficiently below actual coverage to avoid false positives on routine runs.
  • Confirm Codecov's patch coverage calculation correctly ignores excluded paths (src/env.d.ts, apps/, test/, scripts/).

@codecov

codecov Bot commented Jun 16, 2026

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@JSONbored
JSONbored merged commit c8062c4 into main Jun 16, 2026
11 of 12 checks passed
@JSONbored
JSONbored deleted the claude/gifted-tesla-3d8eb6 branch June 16, 2026 21:40
@github-project-automation github-project-automation Bot moved this from Todo to Done in gittensory - v1 roadmap Jun 16, 2026
JSONbored added a commit that referenced this pull request Jun 16, 2026
Two follow-ups from the coverage-gate change (#763) and the duplicate hono
advisory PRs (#760 from Renovate, #761 from Dependabot security updates):

- prBodyNotes referenced the old "97% global coverage gate" that #763 replaced.
  Update it to describe the Codecov patch-coverage gate so Renovate stops
  stamping stale guidance on every dependency PR.
- Record that Renovate is the sole dependency/security bot. GitHub Dependabot
  security updates has been disabled at the repo level (automated-security-fixes)
  so a single advisory no longer produces two PRs; Renovate's vulnerabilityAlerts
  continues to cover security advisories.
JSONbored added a commit that referenced this pull request Jun 16, 2026
Two follow-ups from the coverage-gate change (#763) and the duplicate hono
advisory PRs (#760 from Renovate, #761 from Dependabot security updates):

- prBodyNotes referenced the old "97% global coverage gate" that #763 replaced.
  Update it to describe the Codecov patch-coverage gate so Renovate stops
  stamping stale guidance on every dependency PR.
- Record that Renovate is the sole dependency/security bot. GitHub Dependabot
  security updates has been disabled at the repo level (automated-security-fixes)
  so a single advisory no longer produces two PRs; Renovate's vulnerabilityAlerts
  continues to cover security advisories.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects
Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant