Repository navigation
Conversation
bc97ca4 to
3c68a32
Compare
For a runner rtk has no parser for (`make test`, a custom script), `rtk test` printed the last five lines of its output whatever the exit code, so a failure above a passing tail was hidden: `FAIL: test_login broke` followed by fifty `PASS:` lines showed only the PASS lines. When such a run exits non-zero, show what the output says went wrong: - the error blocks `rtk err` shows (the shared ERROR_PATTERNS, untouched), plus a bare "fail" word, which only test runs read; - with none of those, the exit code and the last ten lines. A run that exits 0 keeps its tail: "warning" or "0 failed" in a green run is not a failure. Runners rtk recognises are unchanged. `run_test_cmd` now goes through `run_filtered_with_exit`, so the summary sees the exit code. Fixes rtk-ai#2420
3c68a32 to
fbdc006
Compare
|
Rebased onto current develop. The code moved to Now only a non-zero exit takes the new path: the Two notes for review: I dropped my third tier ( |
Summary
Fixes #2420
When
rtk test <cmd>doesn't recognise the runner (make test,bazel test, a custom script), it fell back toOUTPUT (last 5 lines):, which hid the failure whenever it wasn't in the tail. #2420's script printsFAIL: test_login brokeamong 50PASS:lines, and only the last fivePASS:lines came out.Now, only when nothing was recognised (no summary, no runner diagnostic block) and the exit code is non-zero:
rtk erruses (ErrorStreamFilter, reused rather than copied), plus a barefail/failsword marker that only test runs read (ErrorStreamFilter::for_test_run(); the sharedERROR_PATTERNSandrtk errare untouched) ->[FAIL] DETECTED FAILURES:capped atCAP_LISTwith... +N more[FAIL] Command failed (exit code: N)plus the last ten non-blank lines (same text asrtk err)A run that exits 0 keeps today's tail, and recognised runners that print a summary are byte-identical (golden tests). The tee recovery hint on failures comes from the existing
print_with_hint, andnever_worsestill applies. A recognised runner that died before its summary (for example acargo testcompile error, exit 101) now shows theerror[E0425]block instead of the last 5 lines.Changes from the first version of this PR, found by re-running #2420's exact script:
FAIL:isn't inERROR_PATTERNS; hence the bare-failmarkerwarning:andSummary: 0 failed, 30 passedgot a false[FAIL] DETECTED FAILURES:banner[ok] All tests passed (no recognized test runner)) is dropped: it claims something rtk doesn't know (cargo test --no-runandpytest --collect-onlyland there too) and it discards the tail that carries things like12 tests collected. Restoring it is a three-line change if you prefer it.The code has moved to
src/core/runner.rssince the first version (run_test_cmd/extract_test_summary), so it is re-applied there.Before/after on #2420's script (
rtk test ./vt.sh, exit 1):Tests
failword boundaries (test_fail_casedoes not match)tests/generic_test_runner_test.rs(unix, 3 e2e tests): rtk test shows only the last 5 output lines (not failures) for unrecognized runners #2420's script, a green run is never reported as failed, and error blockscargo fmt --all -- --check,cargo clippy --all-targets(0 warnings),cargo test --all(all pass)