Skip to content

test(stringified-props): scan the corpus once, outside the timed test - #561

Merged
IgorShevchik merged 3 commits into
mainfrom
claude/flake-stringified-props
Sep 9, 2026
Merged

test(stringified-props): scan the corpus once, outside the timed test#561
IgorShevchik merged 3 commits into
mainfrom
claude/flake-stringified-props

Conversation

@IgorShevchik

Copy link
Copy Markdown
Collaborator

Linked issue

None — this fixes a defect introduced by #545, found by it going red on an unrelated branch.

Type of change

  • Documentation (updates to the documentation or readme)
  • Bug fix (a non-breaking change that fixes an issue)
  • Enhancement (improving an existing functionality)
  • New feature (a non-breaking change that adds functionality)
  • Chore (updates to the build process or auxiliary tools and libraries)
  • Revert (undoing a merged change — retitle this PR revert(Scope): ...)
  • Breaking change (fix or feature that would cause existing functionality to change)

Description

The guard added with #545 read all 218 snapshot files inside its it(), so 28 MB of I/O sat under vitest's 5-second per-test default.

where cost
plain node, standalone ~0.5 s
the nuxt project, file run alone 1.9 s — 38 % of the budget with zero contention
full run, sharing forks with 346 other files past 5 s → timeout
FAIL |nuxt| test/utils/stringified-props.spec.ts > leaks no object-valued prop into the corpus as an attribute
Error: Test timed out in 5000ms.

Not a leak — a direct grep over the corpus returns zero [object Object] either side of this change.

The fix is the pattern already next door

indistinguishable-snapshots.spec.ts does the same 28 MB read and does not flake, because it does it in the describe body:

describe('snapshot entries that are indistinguishable from a sibling', () => {
  const current = collectGroups()   // outside it(), so no per-test timeout

This file now does the same. The corpus is read once instead of once per assertion, and the assertions are free.

It timed out twice; the first time went unnamed

The first occurrence was reported as an unexplained single failure during #549 — one test, one project, and three clean re-runs afterwards. The log had been captured as a tail so the test's identity was lost, and I said at the time I would not call it flaky without a name. This is the name, and the shape matches: one test, one project, intermittent, load-dependent.

Verifying the guard still bites

Appended an [object Object] attribute to Badge.spec.ts.snap; it reds in both projects and names the file and the attribute:

+   "components/__snapshots__/Badge.spec.ts.snap: leaked=\"[object Object]\"",

Worth recording that the first attempt at that mutation stayed green and the guard was briefly under suspicion. The cause was printf escaping the quotes, so the injected text never matched a pattern that was working correctly. A mutation that fails to go red is a claim about the mutation before it is a claim about the test — the mirror of the lesson from #529, where a mutation went red for the wrong reason.

Checks

  • pnpm lint green
  • Full suite green on two consecutive runs: 347 files, 7903 passed, 6 skipped
  • Guard verified red under an injected leak, in both projects; corpus restored from a copy rather than git checkout --

Checklist

  • I have linked an issue or discussion.
  • I have updated the documentation accordingly.

🤖 Generated with Claude Code

https://claude.ai/code/session_012MsMuj8Fic9tjWVjyEyrxc


Generated by Claude Code

The guard added with #545 read all 218 snapshot files inside its `it()`, so
28 MB of I/O sat under vitest's 5s per-test default. Measured: ~0.5s in plain
node, 1.9s in the `nuxt` project on its own — 38% of the budget with no
contention — and past 5s under a full run sharing forks with 346 other files.

It timed out that way twice. The first time was reported as an unexplained
single failure that three clean re-runs could not reproduce and I would not
call flaky without a name; this is the name.

The scan now happens in the describe body, which is how the collision guard
beside it has always done the same read. Describe-body work is not subject to
the per-test timeout, so the assertions are free and the corpus is read once
instead of once per assertion.

No behaviour change to what the guard catches: verified by appending an
`[object Object]` attribute to a snapshot, which reds it in both projects and
names the file and the attribute. Worth recording that the first attempt at
that mutation stayed green and the guard was briefly suspect — `printf` had
escaped the quotes, so the injected text did not match a pattern that was
working correctly. A mutation that fails to go red is a claim about the
mutation before it is a claim about the test.

Full suite green on two consecutive runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012MsMuj8Fic9tjWVjyEyrxc
@IgorShevchik
IgorShevchik merged commit b14f8e9 into main Sep 9, 2026
2 checks passed
@IgorShevchik
IgorShevchik deleted the claude/flake-stringified-props branch September 9, 2026 08:06
IgorShevchik pushed a commit that referenced this pull request Sep 10, 2026
…eading them

Every assertion this repository makes about theme classes is a string
assertion: `test/utils/*.spec.ts` reads the theme sources, the component
snapshots record the rendered `class` attribute, and nothing compiles CSS. An
arbitrary value was therefore only ever proved to be written.

`theme-css-compiles.spec.ts` runs the token entry through the installed
`tailwindcss` engine with the candidate classes read out of `src/theme/*.ts`,
and asserts a `max-height` declaration is emitted for each and that the
`--max-height-popup-*` tokens they reference reach the compiled output.

The incident behind it is #430, which moved the popup caps from literals into
tokens: the mechanism had to be checked by hand during that review, by exactly
this compile. It confirmed the caps worked, and nothing kept them working.

Candidates are extracted rather than restated, so a class that is edited stays
covered and one that is deleted trips the floor assertion instead of thinning
the list in silence. The compile happens once at module scope, not inside an
`it()` — the shape that flaked in #561.

What the guard cannot catch is recorded in the file, measured against the
engine rather than assumed: Tailwind v4 does not validate the meaning of an
arbitrary value. It emits a rule for `max-h-[not a length]` and for an
unbalanced `max-h-[min(var(--x)]`; it refuses an empty value or an unknown
utility. So the first failure mode is caught only in its total form, while the
second — the token dropping out of `:root` — is caught exactly, verified by
deleting `--max-height-popup-list` from `sizes.css`.

A `the guard itself` block pins that `ruleFor` reports absence as well as
presence, so a matcher that always found something could not pass the file
vacuously.

Closes #457

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012MsMuj8Fic9tjWVjyEyrxc
IgorShevchik added a commit that referenced this pull request Sep 10, 2026
…eading them (#564)

Every assertion this repository makes about theme classes is a string
assertion: `test/utils/*.spec.ts` reads the theme sources, the component
snapshots record the rendered `class` attribute, and nothing compiles CSS. An
arbitrary value was therefore only ever proved to be written.

`theme-css-compiles.spec.ts` runs the token entry through the installed
`tailwindcss` engine with the candidate classes read out of `src/theme/*.ts`,
and asserts a `max-height` declaration is emitted for each and that the
`--max-height-popup-*` tokens they reference reach the compiled output.

The incident behind it is #430, which moved the popup caps from literals into
tokens: the mechanism had to be checked by hand during that review, by exactly
this compile. It confirmed the caps worked, and nothing kept them working.

Candidates are extracted rather than restated, so a class that is edited stays
covered and one that is deleted trips the floor assertion instead of thinning
the list in silence. The compile happens once at module scope, not inside an
`it()` — the shape that flaked in #561.

What the guard cannot catch is recorded in the file, measured against the
engine rather than assumed: Tailwind v4 does not validate the meaning of an
arbitrary value. It emits a rule for `max-h-[not a length]` and for an
unbalanced `max-h-[min(var(--x)]`; it refuses an empty value or an unknown
utility. So the first failure mode is caught only in its total form, while the
second — the token dropping out of `:root` — is caught exactly, verified by
deleting `--max-height-popup-list` from `sizes.css`.

A `the guard itself` block pins that `ruleFor` reports absence as well as
presence, so a matcher that always found something could not pass the file
vacuously.

Closes #457

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012MsMuj8Fic9tjWVjyEyrxc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants