rtk test mislabels a fully-passing suite as [FAIL] when a test name contains the substring FAIL #3741

Bug: rtk test mislabels a fully-passing suite as [FAIL] when a test name contains the substring FAIL

rtk test <cmd> appears to classify pass/fail by substring-matching the word FAIL against the raw output text (which includes test names), instead of parsing the actual pass/fail status glyph (✔/✖) or the reporter's structured summary counters (ℹ pass N / ℹ fail N) that node --test's spec reporter prints.

Any test whose name happens to contain the uppercase substring FAIL (e.g. a regression test that is itself about failure-handling, a very common naming pattern: "...rejected: FAIL...", "...FAILURES section...") gets the entire run wrongly summarized as [FAIL] FAILURES:, even though every individual test passed (✔) and the reporter's own summary says fail 0.

This is dangerous for the exact use case rtk test exists for: an agent/human skimming compacted output would read [FAIL] FAILURES: and believe the suite is broken, when it is 100% green.

Repro (minimal, 2 files)

package.json:

{
  "name": "rtk-bug-repro",
  "version": "1.0.0",
  "private": true,
  "scripts": { "test": "node --test \"*.test.js\"" }
}

repro.test.js:

const { test } = require("node:test");
const assert = require("node:assert");

test("contains uppercase word FAIL in the name", () => {
  assert.ok(true);
});

Commands

$ npm test
> rtk-bug-repro@1.0.0 test
> node --test "*.test.js"

✔ contains uppercase word FAIL in the name (0.45ms)
ℹ tests 1
ℹ suites 0
ℹ pass 1
ℹ fail 0
...

$ rtk test npm test
[FAIL] FAILURES:
  ✔ contains uppercase word FAIL in the name (0.45ms)

OUTPUT (last 5 lines):
  ℹ fail 0
  ℹ cancelled 0
  ℹ skipped 0
  ℹ todo 0
  ℹ duration_ms 57.6223

Note the contradiction in rtk's own output: it prints [FAIL] FAILURES: in the header, lists a test marked ✔ (pass) as the "failure," and then shows ℹ fail 0 in its own OUTPUT tail — three signals in the same block that disagree with each other.

Isolating the trigger

• "...contains lowercase word failure..." → not misclassified (correct).
• "...contains uppercase word FAIL..." → misclassified as [FAIL].

So the match appears to be a case-sensitive substring check for FAIL against the full command output (which includes test names as printed by the reporter), not the actual reporter-emitted status per test or the ℹ fail N summary line.

Real-world trigger (not just a synthetic case)

Found via a regression test whose name is legitimately about testing failure-shape handling:

✔ REGRESSION-FIXTURE: real captured 48-failure selftest shape (cloudflare skills) collapses to fully-passing areas hidden, failing area + every FAIL line kept verbatim (0.45ms)

This test passes; the whole suite (196 other passing tests, 0 failures per node --test's own summary) gets relabeled [FAIL] FAILURES: by rtk test, purely because the test's own name contains FAIL.

Environment

• rtk 0.42.4 (installed via cargo install --git https://github.com/rtk-ai/rtk)
• Windows 11, PowerShell 7
• node --test (Node v24.14.1) as the underlying test runner

Expected

rtk test should classify pass/fail using the reporter's structured summary (ℹ pass N / ℹ fail N) or the actual per-test status glyph, never a substring match against free-text test names — the exact anti-pattern already flagged in #3534 (rtk log misclassifying JSONL records as errors by prose substring match) and #2545 (false positives from text matching), just hitting rtk test this time.

Suggested fix

Parse node --test's TAP/spec summary block (ℹ pass N, ℹ fail N) as the source of truth for the [FAIL]/[PASS] header, and use each line's leading ✔/✖ glyph (not its trailing free-text name) to decide which individual lines belong in the FAILURES: section.