Skip to content

Burn down coverage-based CRAP findings and make the touched-function check blocking #3414

Description

@thymikee

Purpose

#3408 made CRAP come from measured test coverage. The first whole-repo report on main (Coverage job on 5f84627d8, pnpm check:coverage-crap --all) lists 59 functions at or above CRAP 30, 49 of them at 0% coverage. This umbrella burns that list down, then makes the check blocking for the functions a PR touches.

CRAP = CC² × (1 − coverage)³ + CC. A function at CC ≤ 4 can never reach 30, and full coverage reduces the score to CC. So a finding is fixed by testing its uncovered branches through the owning module's real entry point, or by splitting it when CC itself is ≥ 30 (docs/agents/testing.md: no test-only exports or test-only DI).

pnpm test:coverage                           # writes coverage/coverage-istanbul.json
pnpm check:coverage-crap --all               # every function >= 30
pnpm check:coverage-crap --base origin/main  # only functions the branch touches

A child is done when its functions are absent from --all on its PR head, and CI's Report coverage-based CRAP step lists no touched function.

Children

Each child owns disjoint files and ships one PR (Closes #<child>).

Sequencing

Rules for every child

  • Test through public interfaces. No production exports for tests and no test-only DI (docs/agents/testing.md). Test helpers go in *.fixtures.ts.
  • Stay within the owned files. A new fixture or module is named after its owning module, so children don't collide in shared packages.
  • Keep each PR under the 1,000-line gross budget (docs/agents/pull-requests.md). Run pnpm check:affected --run on the pushed head.
  • Don't bulk-regenerate fallow baselines. A split that moves a function can owe a path-keyed health-baseline move.

Activity

  1. thymikee commented on Oct 11, 2026

    @thymikee
    MemberAuthor

    Plan for G, H and I (2026-10-11)

    A–F2 are done. Their PRs are merged, or approved and waiting on merge (#3436, #3440). The remaining work, as decided with Codex:

    • H (CRAP: find why 6 files are unscored (vitest#11541 negative counts) #3425): fix the two upstream bugs with dependency patches.
      • The H2 PR patches @vitest/coverage-v8 so the wrapper offset no longer depends on merge order, and ast-v8-to-istanbul so async implicit-else counts come out right.
      • It doesn't clamp or zero counts and adds no application-code workarounds. The sanitizer keeps dropping a whole file it can't trust, and the vitest#11541 wording is corrected.
      • The upstream issues are drafted for the maintainer to file.
      • The six files are re-scored, and any new findings ≥ 30 get routed as follow-ups.
    • G (CRAP: daemon handlers and server (needs splits) #3424): three PRs, landing before the src/daemon → packages/daemon move.
      • G1: tests for the four low-coverage functions.
      • G2: split session-state.ts. The dispatcher, the boot and shutdown handlers, and the app-state selection, response and query paths each get their own function.
      • G3: split the http-server.ts request-end handler into envelope parsing, authorized request preparation, and invocation/response. One owner keeps request lifetime, cancellation and cleanup.
    • I (CRAP: make the touched-function check blocking #3426): stacked on H2.
      • Every touched function scoring ≥ 30 blocks the PR, using rename-aware changed-line attribution.
      • Missing, stale, partial or corrupt coverage fails with "no trustworthy verdict."
      • The two file-wide exemptions are listed with their scope.
      • main stays report-only.
      • It turns on after G and the re-scored inventory are done.
    • Optional review suggestions on merged child PRs: per-area follow-ups after I. Wall-clock flakiness, shared scenario state and ineffective assertions come first; cosmetic items are dropped.

    Order: H2 + G1, then G2 + G3, then I. At most two workers at a time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions