Repository navigation
Slai Agent Runtime:Guard unrepresented Windows .NET test patch regressions - #88
Open
Slai.life (huong8373tt-beep) wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This adds a narrow fail-closed guard for Windows rows where the official C#/.NET
test_patchadds xUnit-style regression methods, but publishedFAIL_TO_PASSmetadata names none of those methods.In that condition, exact-name grading has no stated target for the official regression. A full-suite result, including unrelated GPU, Appium, font, locale, or parser outcomes, must not be converted into a candidate success or failure.
The evaluator writes:
{ "resolved": null, "grading_status": "unavailable", "failure_class": "unrepresented_windows_dotnet_test_patch_regressions" }The guard does not repair test names, infer replacement metadata, modify commands or patches, start a task container, or mark a candidate solved.
Reproduction:
AvaloniaUI__Avalonia-19538The Windows row is based on Avalonia upstream #19538, the FontFamily / FontManager fix for resolving ordinary system font names with a base URI.
Its official
test_patchadds these attributed C# regressions:But the published
FAIL_TO_PASSlist contains six unrelated rendering/parser identities, for example:None names any added FontFamily / FontManager regression. Replaying the exact published test patch against this guard extracts the three methods above and returns unavailable before runtime startup.
The completed Windows run also showed why unrelated full-suite outcomes cannot stand in for the missing contract: Direct2D tests require an unavailable graphics interface, Appium requires a missing service on port 4723, and VSTest output is localized while the published parser recognizes English detailed lines only. Those observations are supporting evidence, not part of this guard's predicate.
Guard boundary
It applies only when all of these hold:
Fact,Theory,Win32Theory,StaFact, orStaTheoryform; andFAIL_TO_PASSidentity ends in any added method name (with optional parameter text).Rows outside that shape continue on the existing evaluation path. Test helpers, production methods, deleted lines, context lines, non-test paths, Linux rows, and rows that represent at least one added method are unchanged.
Relationship to existing PRs
This shares the dataset-integrity principle of #77: an official regression added by a Windows test patch must be represented in published
FAIL_TO_PASSmetadata. It is intentionally separate because #77 parses the distinct Go shape (func Test...in*_test.go), while this PR handles attributed C#/.NET test methods.It is also separate from #74, which rejects visibly malformed expected-test strings. Avalonia's published identities are syntactically well-formed; their defect is that they do not represent the methods introduced by the official patch.
The durable resolution is revalidation and republication of
AvaloniaUI__Avalonia-19538with the actual regression identities and a runnable Windows test environment. This evaluator guard only prevents an unrelated suite outcome from becoming a misleading candidate verdict in the meantime.Validation
The tests cover extraction of added attributed methods; a represented identity; non-Windows behavior; non-test and unattributed no-op behavior; an unrepresented C# regression contract; and the no-runtime-start unavailable report.