Skip to content

test(md_help): expect the guttered console block per syntax-highlighting - #3916

Merged
max-sixty merged 1 commit into
mainfrom
fix/md-help-powerset-snapshot
Aug 26, 2026
Merged

max-sixty merged 1 commit into
mainfrom
fix/md-help-powerset-snapshot

Conversation

@worktrunk-bot

Copy link
Copy Markdown
Collaborator

cargo hack test --feature-powerset fails on the --no-default-features --features cli combination: test_html_comment_inside_a_code_block_is_content baked the highlighted rendering of its trailing ```console block into one inline snapshot, and with syntax-highlighting off that block renders plain. Split the expectation into a #[cfg(feature = "syntax-highlighting")] / #[cfg(not(...))] pair — the shape src/styling/format.rs already uses — so both configurations keep the assertion rather than one of them losing it. Verified by running the test under both feature sets locally.

Detail

The failing diff:

    1     1 │ [107m [0m [2m<!-- wt list -->[0m
    2     2 │
    3       │-[107m [0m [2m[0m[2m[34mwt[0m
          3 │+[107m [0m wt

syntax-highlighting is a default feature, so the required checks (test (linux|macos|windows), fast-checks) build with it on and never see this; only feature-powerset exercises cli without it. That job's own comment in nightly.yaml names this exact case — "a snapshot baking feature-dependent output, only surfaces when tests are built and run per combo".

Introduced by #3908, which landed after main's last nightly run, so no run on main had caught it yet. It surfaced on #3915 (an unrelated Dependabot bump), where the red check was confusing enough to be worth clearing.

The test's subject is that a fenced <!-- wt list --> is content and its marker does not leak into the following block. Both cfg variants assert that; only the trailing block's colouring differs, which is why gating the whole assertion on the feature would have been the wrong trade.

Test plan

cargo test --bin wt md_help                                        # 27 passed
cargo test --no-default-features --features cli --bin wt md_help   # 27 passed

The second command reproduces the failure at 8e40a2b7 and passes here. cargo fmt --check and cargo clippy --no-default-features --features cli are clean.

`test_html_comment_inside_a_code_block_is_content` baked the highlighted
rendering of its trailing `console` block into one inline snapshot. With
`syntax-highlighting` off — the `--no-default-features --features cli`
combination — that block renders plain, so `cargo hack test
--feature-powerset` failed on a test whose subject is marker containment,
not highlighting.

Split the expectation the way `src/styling/format.rs` already does, with a
`#[cfg(feature = ...)]` / `#[cfg(not(...))]` pair, so both configurations
keep the assertion instead of one of them dropping it.
@worktrunk-bot

Copy link
Copy Markdown
Collaborator Author

One thing worth flagging: feature-powerset — the job this fixes — is skipped on this PR. The nightly gate ran it on #3915 (which touches .github/workflows/nightly.yaml) but resolved to skip on a src/-only change, so CI here never exercises the combination the fix is for.

So the verification is the local run in the PR body, not a check on this head: cargo test --no-default-features --features cli --bin wt md_help reproduces the failure at 8e40a2b7 and passes with this commit. Worth a manual nightly dispatch before merging if you want the powerset job's word on it.

@worktrunk-bot

Copy link
Copy Markdown
Collaborator Author

The nightly cron on main has now hit this, which is the confirmation my earlier comment said was missing from this PR's own checks: run 32935569443 on e4238e8, feature-powerset red on cargo test --no-default-features --features cli:

test md_help::tests::test_html_comment_inside_a_code_block_is_content ... FAILED
test result: FAILED. 980 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out
error: process didn't exit successfully: `cargo test --manifest-path Cargo.toml --no-default-features --features cli` (exit status: 101)

Same test, same feature combination as this PR fixes, so no separate fix PR — this one covers it. Adding the nightly label here would get the powerset job to run on this head rather than relying on the local reproduction.

The run's other failure is unrelated

release-target (aarch64-unknown-linux-musl) also went red in the same run, on integration_tests::step_promote::test_promote_bare_repo_with_worktrees — a one-off filesystem anomaly during that test's plain-git setup, unrelated to this change. Diagnosis in #3918.

@worktrunk-bot worktrunk-bot added the nightly Trigger the nightly workflow on this PR label Aug 26, 2026
@max-sixty
max-sixty merged commit 50245a5 into main Aug 26, 2026
53 checks passed
@max-sixty
max-sixty deleted the fix/md-help-powerset-snapshot branch August 26, 2026 09:45
max-sixty pushed a commit that referenced this pull request Aug 26, 2026
…it (#3920)

`tests/CLAUDE.md` tells a test that checks formatted output to use an
inline snapshot, and says nothing about the fact that a snapshot of
*styled* output also bakes the `syntax-highlighting` feature into the
expectation. Following that rule on highlighted output is what took
`main`'s nightly red overnight: #3908's second commit converted three
`contains` checks on `render_markdown_in_help` to inline snapshots, one
of them covering a trailing `console` block, merged green, and
`feature-powerset` failed on the next cron ([run
32935569443](https://github.com/max-sixty/worktrunk/actions/runs/32935569443)).
This adds the caveat to the section that gave the instruction, with the
per-configuration shape #3916 uses to fix that test. Docs only —
`pre-commit run --files tests/CLAUDE.md` passes.

<details><summary>Why here, and the full chain</summary>

## The chain

| When | What |
|---|---|
| 08-25 06:36Z | #3908 opened — `fix(help): treat an HTML comment inside
a code block as content`, test asserted with three `contains` calls |
| 08-25 06:45Z | Self-review cites `tests/CLAUDE.md` → **Inline
snapshots over multi-assert** and pushes `8a141861a`, converting them to
inline snapshots. One captures the trailing `console` fence's rendering,
which under `syntax-highlighting` is `ESC[2mESC[0mESC[2mESC[34mwtESC[0m`
|
| 08-25 16:20Z | Merged |
| 08-26 03:57Z | #3916 opened, having caught the failure on an unrelated
Dependabot PR |
| 08-26 05:49Z | `nightly` cron on `e4238e83`: `feature-powerset` red —
`test md_help::tests::test_html_comment_inside_a_code_block_is_content
... FAILED` |
| 08-26 06:29Z | Diagnosed on #3917; #3918 filed for the same run's
unrelated transient |

The self-review's reasoning was right on its own terms — the `contains`
version proves the comment line survived but says nothing about *how* it
renders, which is what #3908 decides. What it missed is the axis on
which "how it renders" is not a constant.

## Why the required checks can't catch it

`syntax-highlighting` is a default feature, so `test
(linux|macos|windows)`, `fast-checks` and `code-coverage` all build with
it on. `ci`'s `feature-check` job does `cargo check --bin wt
--no-default-features --features cli` on every PR, but `cargo check`
never compiles `#[cfg(test)]` code, so the snapshot is never built
there. The only job that *runs the tests* on that combination is
`feature-powerset` in `nightly`, and `nightly`'s gate runs on a PR only
when the diff touches Cargo/toolchain/nix paths or the PR carries the
`nightly` label — so a `src/`-only PR never sees it. The red is
structurally deferred to the next cron on `main`, i.e. after the merge.

`nightly.yaml`'s own job comment already names the failure mode — "a
snapshot baking feature-dependent output, only surfaces when tests are
built and run per combo" — but that text is in the workflow, not in the
file an author of a unit test reads.

## Why the caveat and not a gate

Gating the whole assertion on `#[cfg(feature = "syntax-highlighting")]`
would compile the test out of the one combination that catches this
class of bug. The pair form keeps the assertion on both sides and is
what the repo already does elsewhere (`src/styling/format.rs` uses
`#[cfg(all(test, feature = "syntax-highlighting"))]` for a module whose
subject *is* the highlighting; here the colouring is incidental, so the
test stays and only the expectation splits).

The last paragraph — apply the `nightly` label to a PR fixing a
`feature-powerset` failure — is the other half of the same incident:
#3916's own `feature-powerset` was skipped until the label went on at
06:32Z, after which it ran green on `2b734bb8`. Nightly's header comment
calls the label "the iteration knob for fixes targeting nightly-only
failures"; nothing under `tests/` said so.

## Gate assessment

- **Evidence level**: Critical by the table's own wording — merged code
took `main` red. 1 occurrence, no prior sighting of this shape in the
tracking log
([#3691](#3691)). Blast
radius was small and it was self-caught within hours, which is why the
proposed change is two paragraphs rather than a process.
- **Structural**: yes. Any author following the snapshot rule on styled
output, reading only that section, gets no warning; the information
lives in `nightly.yaml` and `Cargo.toml`, neither of which a unit-test
author has reason to open.
- **Cost class**: wrong outward action — a red default branch, not
wasted compute.
- **Change type**: targeted fix (a missing caveat on an existing rule),
Gate 2's normal bar.

</details>

---------

Co-authored-by: worktrunk-bot <254187624+worktrunk-bot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

nightly Trigger the nightly workflow on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants