Repository navigation
fix(test): the 504 this test looked for was in a random id - #1042
Merged
Merged
Conversation
`expect(sse).not.toContain("504")` searched the whole event stream, response
ids included. About one 32-hex id in 137 contains "504", and the stream
carries two, so this reddened roughly one CI run in 69 while nothing was
wrong. Measured locally: one failure in 37 runs. The failing CI stream
contained "recovered", ended in response.completed, and had no failure frame -
the only "504" anywhere in it was inside resp_612a6504e8ea4cf095d0f02d4f828c80.
The flake is the smaller half. Removing the deadline re-arm this test is named
after left it green, so the regression it exists to prevent was never guarded:
the mock adapter ignored ctx.abortSignal and answered 200 regardless, and a
real header timeout returns an eager HTTP 504 with a JSON body that does not
contain the number anyway.
The mock now honors the signal, and the assertions state the actual contract:
status 200 with an event-stream content type before the body is read, terminal
events by name rather than by substring, and the signal identity that
distinguishes a fresh deadline from the disarmed remains of the old one -
which is the only observable difference, since neither expires.
Verified by mutation: removing the re-arm reddens it, and removing both the
pre-wait clear and the re-arm reddens it. Removing only the pre-wait clear
does not, and that is left honestly unclaimed - it guards a client-cancel race
this test does not exercise.
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe retry regression test now verifies that each fetch attempt receives a distinct, live abort signal. It also validates the SSE response status, headers, successful completion, and absence of a failure event. ChangesRetry deadline validation
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
7 tasks done
agentHits
pushed a commit
to agentHits/opencodex
that referenced
this pull request
Sep 17, 2026
…#1042) `expect(sse).not.toContain("504")` searched the whole event stream, response ids included. About one 32-hex id in 137 contains "504", and the stream carries two, so this reddened roughly one CI run in 69 while nothing was wrong. Measured locally: one failure in 37 runs. The failing CI stream contained "recovered", ended in response.completed, and had no failure frame - the only "504" anywhere in it was inside resp_612a6504e8ea4cf095d0f02d4f828c80. The flake is the smaller half. Removing the deadline re-arm this test is named after left it green, so the regression it exists to prevent was never guarded: the mock adapter ignored ctx.abortSignal and answered 200 regardless, and a real header timeout returns an eager HTTP 504 with a JSON body that does not contain the number anyway. The mock now honors the signal, and the assertions state the actual contract: status 200 with an event-stream content type before the body is read, terminal events by name rather than by substring, and the signal identity that distinguishes a fresh deadline from the disarmed remains of the old one - which is the only observable difference, since neither expires. Verified by mutation: removing the re-arm reddens it, and removing both the pre-wait clear and the re-arm reddens it. Removing only the pre-wait clear does not, and that is left honestly unclaimed - it guards a client-cancel race this test does not exercise.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure
devCI went red on run 30967162304 with a single failing test:The assertion was
expect(sse).not.toContain("504"), and the stream it rejected looked like this:That stream carried
recovered, ended inresponse.completed, and contained no failure frame. The only504anywhere in it was three characters of a random identifier.About one 32-hex id in 137 contains
504, and a stream carries two of them, so this reddened roughly one run in 69 for no reason. Reproduced locally: 1 failure in 37 runs, which matches.The larger half
The flake is not the interesting part. Removing the deadline re-arm that this test is named after left it green:
src/images/loop.tsclear()Two reasons it could not work. The mock adapter ignored
ctx.abortSignaland answered200no matter what the deadline said. And a genuine first-iteration timeout is answered eagerly with an HTTP 504 whose JSON body does not contain the number504at all — so even a real one would have slipped past a substring search.So the regression this test exists to prevent was never actually guarded.
The change
tests/images/loop.test.tsonly. No production code.ctx.abortSignal?.throwIfAborted(), so the deadline means somethingexpect(response.status).toBe(200)plus the content type, checked before the body is consumed, because an eager 504 never produces a stream to inspectevent: response.completedpresent,event: response.failedabsent) instead of a substring hunttests/web-search.test.tshas a test of the same name proving the same thing, and it was already written this way. It passed in the same CI run.What is deliberately not claimed
Dropping only the pre-wait
clear()still passes. That call guards a race between a stale expiry and the client-cancel path, which this test does not exercise. Saying it is covered would be the same kind of overclaim this PR is fixing, so it is left unclaimed and noted in the commit message.Verification
bun x tsc --noEmit: cleanSummary by CodeRabbit