Skip to content

Slai Agent Runtime: Filter ghost Go test statuses on Windows - #58

Open
Slai.life (huong8373tt-beep) wants to merge 4 commits into
microsoft:mainfrom
huong8373tt-beep:slai-fix-go-json-parser-ghost-tests
Open

Slai.life (huong8373tt-beep) wants to merge 4 commits into
microsoft:mainfrom
huong8373tt-beep:slai-fix-go-json-parser-ghost-tests

Conversation

@huong8373tt-beep

Copy link
Copy Markdown

Summary

This PR fixes a Windows-only evaluator false positive discovered by Slai Agent Runtime — a private agent runtime system exploring a practical form of AGI.

When the Windows runner emits go test -json output from multiple packages, the current regex parser can associate a later Test field with an earlier package-level failure. That creates a status entry for a test that never appeared in the authoritative Go JSON stream.

For goreleaser__goreleaser-5631, this produced ghost failures including:

  • internal/pipe/reportsizes::TestDescription — the package emitted TestString, not TestDescription;
  • internal/shell::TestString — the package emitted TestRunCommand, not TestString.

Fix

  • Keep the existing parser output by default.
  • On Windows only, when recognizable Go JSON events exist, retain only status keys backed by explicit terminal (pass, fail, or skip) events carrying both Package and Test.
  • Preserve non-Windows behavior and non-Go logs unchanged.
  • Fail closed: real Go test failures with a matching terminal event remain failures.

Validation

  • Added focused regression tests for explicit-event extraction, ghost-status removal, real-failure retention, and no-op behavior outside this Windows Go JSON path.
  • python -m pytest -q evaluation/test_windows_go_json.py
  • python -m py_compile evaluation/evaluation.py evaluation/windows_go_json.py evaluation/test_windows_go_json.py

This is independent of the Windows proxy/runtime fix for the same Goreleaser task: this PR addresses parser provenance only.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant