[No QA] Claim the sequential queue read gate for deferred writes - #99815
Conversation
`API.write()` claims the read gate synchronously: `push()` calls `setIsReadyPromisePending()` before its first await, so a READ firing on the next line parks behind the write. `writeWhenReady()` never did, so deferring a write silently dropped that ordering - the queue stayed empty for the length of the deferral, and a destination screen fetching the same data raced ahead and repopulated itself from pre-write server state. `claimReadGateForDeferredWrite()` claims the gate on behalf of a write that is not on the queue yet. `push()` adopts the claim when the write lands, so the handover leaves no gap, and the queue drain resolves it. `flush()` also had to learn about it: its empty-queue branch resolved the gate unconditionally, which during a deferral means "the write hasn't been pushed yet", not "nothing is coming". The follower branch still resolves, since a tab that never processes the queue would otherwise park READs forever. No-op while offline, matching `push()` and `flush()`, neither of which parks READs behind a queue that isn't running.
|
@ikevin127 Please copy/paste the Reviewer Checklist from here into a new comment on this PR and complete it. If you have the K2 extension, you can simply click: [this button] |
Codecov Report✅ Changes either increased or maintained existing code coverage, great job!
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7214f06594
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (!settleOnce()) { | ||
| return; | ||
| } | ||
| // A later write may have opened a new gate since; resolving that one would let READs through |
There was a problem hiding this comment.
❌ CONSISTENCY-16 (docs)
Comments should read as plain, natural sentences. This comment joins two independent clauses with a semicolon, which the style guide asks you to write as two separate sentences instead.
Split the clauses into two sentences:
// A later write may have opened a new gate since. Resolving that one would let READs through
// while somebody else's write is still pending.
if (isReadyPromise !== claimedPromise) {
return;
}Reviewed at: 7214f06 | Please rate this suggestion with 👍 or 👎 to help us improve! Reactions are used to monitor reviewer efficiency.
| waitForIdle: jest.fn(() => Promise.resolve()), | ||
| // Called by the network layer on init; stub so advancing fake timers doesn't hit a missing export. | ||
| flush: jest.fn(), | ||
| // The claim writeWhenReady settles once its write reaches (or fails to reach) the queue; the real |
There was a problem hiding this comment.
❌ CONSISTENCY-16 (docs)
Comments should read as plain, natural sentences. This comment uses a semicolon to join two independent sentences; the style guide asks you to write them as two separate sentences.
Split into two sentences:
// The claim writeWhenReady settles once its write reaches (or fails to reach) the queue. The real
// gate is covered in SequentialQueueReadGateTest.
claimReadGateForDeferredWrite: jest.fn(() => ({handOff: jest.fn(), release: jest.fn()})),Reviewed at: 7214f06 | Please rate this suggestion with 👍 or 👎 to help us improve! Reactions are used to monitor reviewer efficiency.
Nothing imports the type - consumers use the inferred return type of claimReadGateForDeferredWrite - and exporting it added a knip finding.
JakubKorytko
left a comment
There was a problem hiding this comment.
apart from the code, please make the EoC at least a half shorter, more plain english and simple explanations, its hard to read. Do not add "jest", "typecheck" and "eslint" as a part of tests - there are CI checks so no value in them being mentioned in the section. This seems No QA so add [No QA] to the title, and if no tests are available apart from automated just mark N/A on tests, offline tests and qa tests - no need for QA to read so much text just to learn there is nothing for them to test
| if (isReadyPromise !== claimedPromise) { | ||
| return; | ||
| } | ||
| releaseClaimedPromise?.(); |
There was a problem hiding this comment.
two deferred writes share a single gate, because setIsReadyPromisePending() no-ops the second time. Shouldn't this also check "am I the last claim?" so the gate isn't lost for the other ones? Something like bailing unless deferredWriteGateClaims === 0 after settleOnce()
| * flush(), which resolves it: neither parks READs behind a queue that isn't running. | ||
| */ | ||
| function claimReadGateForDeferredWrite(): DeferredWriteReadGateClaim { | ||
| if (isOfflineNetwork()) { |
There was a problem hiding this comment.
this only checks offline once, at claim time. If we're offline then and come back online while the barrier is still waiting, the write never claims the gate at all, so a READ in that window doesn't wait for it. push() re-checks at push time, this doesn't
|
|
||
| // Claimed synchronously, before the barrier is even built: a READ firing on the next line has to | ||
| // see the gate already pending, exactly as it would had `write()` queued the request right here. | ||
| const readGateClaim = claimReadGateForDeferredWrite(); |
There was a problem hiding this comment.
every caller now holds the gate, and it's global: API.read and API.paginate all park on the same waitForIdle(), not scoped to the written key. On the opt-out question, I'd keep it on by default and add the escape hatch instead of making it opt-in. Forgetting to opt out only costs some latency, forgetting to opt in costs a silent stale-read race
|
|
||
| // Then both refer to one gate, so releasing it once is enough - the handover leaves no second | ||
| // gate for a READ to be stranded behind | ||
| first.release(); |
There was a problem hiding this comment.
this locks in the early release: second is still unsettled here, so its write hasn't been pushed, but the assert wants the gate open. If release() only resolves on the last claim, this flips to false
| claim.release(); | ||
| }); | ||
|
|
||
| it('ignores a release once a later write has opened a new gate', async () => { |
There was a problem hiding this comment.
this doesn't reach the stale-promise guard it's named after. stale.release() already settled up in the Given block, so the second call returns early on hasSettled and the isReadyPromise !== claimedPromise check never runs. It goes green on idempotency instead
| /** The write is on its way to `push()`: the queue owns the gate from here, so stop holding `flush()` open. */ | ||
| handOff: () => void; | ||
|
|
||
| /** The write never reached the queue (offline, or `write()` threw): stop holding `flush()` open and resolve the gate. */ |
| /** | ||
| * Claims the read gate for a write that has not been pushed yet, so READs consulting waitForIdle() | ||
| * park behind a deferred write exactly as they park behind a queued one. | ||
| * | ||
| * Without this, deferring a write silently drops the read-after-write ordering that write() gives | ||
| * every caller for free: the queue is empty for as long as the write waits, so a refetch of the same | ||
| * data resolves immediately and can come back with pre-write server state. | ||
| * | ||
| * push() marks the gate pending again once the write lands - idempotently, so it adopts this claim | ||
| * rather than opening a second one - and the queue drain resolves it. The handover therefore has no | ||
| * gap for a READ to slip through. | ||
| * | ||
| * No-op while offline, matching push(), which returns before claiming the gate in that state, and | ||
| * flush(), which resolves it: neither parks READs behind a queue that isn't running. | ||
| */ |
There was a problem hiding this comment.
does it really need to be that long? seems like over-explaining, two first paragraphs could be easily a one sentence imo
| // resolver deleted the only request without pushing a replacement). Resolve here so READs | ||
| // parked on waitForIdle() don't hang until unrelated queue activity releases them. | ||
| resolveIsReadyPromise?.(); | ||
| // |
| // Unless a deferred write is holding the gate: an empty queue then means its write hasn't been | ||
| // pushed *yet*, not that nothing is coming, and resolving would let through the very READs that | ||
| // write is meant to be ordered before. Its own claim resolves the gate if it never reaches the queue. | ||
| // Only this branch is guarded - the follower branch below must still resolve, since a tab that | ||
| // never processes the queue would otherwise park READs forever. |
There was a problem hiding this comment.
why we explain branch below in this comment? The comment itself is also making an assumption that "an empty queue then means its write hasn't been pushed yet, not that nothing is coming" - I guess it's true but when reading I wonder how come we are sure of it
| // `waitForIdle()` hands back a module-level promise, so every test here shares one gate. Each releases what | ||
| // it claims, and this lives apart from SequentialQueueTest so no pushed request has moved the gate first. |
There was a problem hiding this comment.
this comment sounds like it is above the gate but I believe offlineSpy is not the gate?
|
@codex review |
|
@MelvinBot review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 37bf90d766
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
ReviewThe approach is right, but the gate is only opened by the claim — nothing keeps it open. Two paths reopen it while the write is still deferred, so the ordering guarantee doesn't hold in the cases that matter. Details below, then an answer to your design question, 1. An unrelated write draining mid-deferral reopens the gate 🔴
Trace
Fix: apply the same 2.
|
actually, that is something that I thought at first since this is the first flow that has such problems (at least reported, though submit-expense used it for a long time), but assumed safe-by-default is better. Perhaps we indeed should go with opt-in |
|
Awaiting for Jakub's review comments to be addressed before moving forward with C+ review ⏳ |
Piggybacking on isReadyPromise meant every queue path that resolves it
could reopen the gate mid-deferral, so each one needed its own guard.
waitForIdle() now waits out the deferred claims first and reads
isReadyPromise after, which closes the drain-completion and multi-claim
holes without guarding flush() at all.
The network is checked while waiting rather than once at claim time, so
going offline releases parked READs and coming back re-parks them. Add
{claimReadGate: false} to opt out.
|
Restructured rather than patched: deferred writes hold their own gate now, and Offline is checked while waiting instead of once at claim time, so both directions work: going offline releases parked READs, coming back re-parks them. You were right on both tests. The adopt one asserted the early release as correct, and the stale one never reached the guard it was named after. Both replaced. Went with default-on + EoC halved, [No QA] added, tests/offline/QA are N/A. Also went back over the comments themselves and cut them down - the long docblocks are gone. |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 942db7103e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
JakubKorytko
left a comment
There was a problem hiding this comment.
Reads a lot better, a few small ones inline, nothing blocking. @Abdukhamid000 please like or respond briefly to every previous review comment if you have addressed it. I assumed they were all addressed. Also, you don't need to use AI to create messages in response to my review 😅 A simple "Done, re-review please" + a like on review comments or an answer to any comment you haven't addressed, explaining why, is enough. I don't expect perfect English or super long explanations, just for the change to be safe
| } finally { | ||
| // `push()` ran synchronously inside `write()`: the queue holds the gate now, or we were | ||
| // offline and nothing was queued. | ||
| settleReadGateClaim(); |
There was a problem hiding this comment.
I think this only releases the gate at the right time because write() happens to call push() synchronously before its first await. If write()/processRequest() ever grows an early await before reaching push() (a feature flag check, a conflict lookup, anything), this finally block runs first and opens the gate before the write is actually on the queue. A READ could then slip in ahead of a write that looks queued but isn't yet. Nothing here enforces that ordering, it is just implied by the comment above.
| // Wake on a network change too, so going offline releases READs and coming back re-parks them. | ||
| let unsubscribeFromNetworkState = () => {}; | ||
| const networkStateChanged = new Promise<void>((resolve) => { | ||
| unsubscribeFromNetworkState = subscribeToNetworkState(resolve); |
There was a problem hiding this comment.
waitForIdle() runs for every API.read/paginate call, so while a deferred write is pending each read that fires gets its own promise plus its own network-state subscriber here. If several reads happen during one deferral window that's several subscribers all doing the same wait, not one shared wait. Could this be a single module-level promise/subscription that all waiters share instead of one per caller?
| onWriteStarted?: () => void; | ||
|
|
||
| /** Whether READs wait for this write. Defaults to `true`. See the read-gate caveat on `writeWhenReady`. */ | ||
| claimReadGate?: boolean; |
There was a problem hiding this comment.
| claimReadGate?: boolean; | |
| shouldClaimReadGate?: boolean; |
so it reads as a flag at the call site
| * Sole writer of the count, so deferredWritesLanded is pending exactly while claims are outstanding - | ||
| * resolving it early would spin waitForIdle(). Floored: resetQueue() can zero the count mid-claim. | ||
| */ | ||
| function setDeferredWriteClaims(count: number) { |
There was a problem hiding this comment.
Every other state change in this file (pause, unpause, flush, push) logs through Log.info so you can trace what happened from device logs. The new claim counter and the waitForIdle wait loop around it log nothing, so a stuck READ from a lost claim would be invisible in the logs
|
@ikevin127 Done, all 5 addressed — re-review please. Two notes where I did a bit more than asked:
|
ikevin127
left a comment
There was a problem hiding this comment.
🟢 LGTM
cc @JakubKorytko if you wanna sign-off before merge
Reviewer Checklist
|
ikevin127
left a comment
There was a problem hiding this comment.
🟢 LGTM
cc @JakubKorytko if you wanna sign-off before merge
JakubKorytko
left a comment
There was a problem hiding this comment.
LGTM, one small NAB: the description still says {claimReadGate: false} and the option is shouldClaimReadGate now
|
Good catch, description updated to |
|
Please merge main, the failing tests are fixed I think. |
|
@rlinoz done |
| // Drop the listener too, not just the promise, so a suite that parks a READ doesn't leave a subscriber | ||
| // behind on every reset. |
There was a problem hiding this comment.
| // Drop the listener too, not just the promise, so a suite that parks a READ doesn't leave a subscriber | |
| // behind on every reset. |
| /** Fires only after `write()` has been called and returned without throwing. A throwing handler is logged, not thrown. */ | ||
| onWriteStarted?: () => void; | ||
|
|
||
| /** Whether READs wait for this write. Defaults to `true`. See the read-gate caveat on `writeWhenReady`. */ |
There was a problem hiding this comment.
| /** Whether READs wait for this write. Defaults to `true`. See the read-gate caveat on `writeWhenReady`. */ | |
| /** Whether READs wait for this write. Defaults to `true`.*/ |
| // reached synchronously inside write(). Holding the gate a moment longer is safe. Opening | ||
| // it early is not. |
There was a problem hiding this comment.
| // reached synchronously inside write(). Holding the gate a moment longer is safe. Opening | |
| // it early is not. | |
| // reached synchronously inside write(). |
| * | ||
| * Not only READs wait here: the auth-token swaps in `Delegate.connect`/`disconnect`, the account merge in | ||
| * `Session`, and the post-sign-in `openApp` all gate on this too, so a deferred write delays them by up to | ||
| * `SAFETY_TIMEOUT_MS` as well. That wait is intended - swapping the token out from under a pending write is | ||
| * exactly what it prevents - but the `shouldClaimReadGate` opt-out lives on the writer, so a caller adding a |
There was a problem hiding this comment.
| * | |
| * Not only READs wait here: the auth-token swaps in `Delegate.connect`/`disconnect`, the account merge in | |
| * `Session`, and the post-sign-in `openApp` all gate on this too, so a deferred write delays them by up to | |
| * `SAFETY_TIMEOUT_MS` as well. That wait is intended - swapping the token out from under a pending write is | |
| * exactly what it prevents - but the `shouldClaimReadGate` opt-out lives on the writer, so a caller adding a |
| async function waitForIdle(): Promise<unknown> { | ||
| while (deferredWriteClaims > 0 && !isOfflineNetwork()) { | ||
| Log.info('[SequentialQueue] READ is waiting on a deferred write', false, {claims: deferredWriteClaims}); | ||
| // The waits are deliberately sequential, not parallel. Each one re-checks the claim count and the |
There was a problem hiding this comment.
| // The waits are deliberately sequential, not parallel. Each one re-checks the claim count and the | |
| // The waits are deliberately sequential, each one re-checks the claim count and the |
| await Promise.race([deferredWritesLanded, whenNetworkStateChanges()]); | ||
| } | ||
|
|
||
| // Read after the wait, not before: the deferred write's push() has re-closed the gate by now. |
There was a problem hiding this comment.
| // Read after the wait, not before: the deferred write's push() has re-closed the gate by now. | |
| // Read after the wait, the deferred write's push() has re-closed the gate by now. |
|
hmm are the lint errors related? |
|
🚧 rlinoz has triggered a test Expensify/App build. You can view the workflow run here. |
|
✋ This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release. |
|
🧪🧪 Use the links below to test this adhoc build on Android, iOS, and Web. Happy testing! 🧪🧪
|
|
🚀 Deployed to staging by https://github.com/rlinoz in version: 9.4.73-0 🚀
|
|
No help site changes are required for this PR, so I did not create a draft docs PR. This change is internal request-queue plumbing. It adds a read gate that deferred writes hold so a READ can't overtake a
Why this doesn't reach the help site
The PR is also labeled @Abdukhamid000 — there is no help site PR to link, so there is nothing to mark |
|
🚀 Deployed to production by https://github.com/mountiny in version: 9.4.73-3 🚀
Bundle Size Analysis (Sentry): |
cc @JakubKorytko -
writeWhenReadyis yours.Explanation of Change
API.write()closes the sequential queue's read gate the moment it is called, so a READ firing right after it parks behind the write.writeWhenReady()did not: while the write waits on its barrier the queue is empty, sowaitForWrites()resolves straight away and a screen that refetches the same data can read back pre-write server state. That is #99805.Deferred writes now hold a gate of their own, and
waitForIdle()waits on it before it looks at the queue - so a READ cannot overtake a write that has not been pushed yet. It is a separate gate rather than the queue's own, because the queue opens its gate whenever it finds itself empty, which it is for the whole deferral.While offline the gate is ignored: the queue cannot run, and
push()/flush()open theirs for the same reason. The wait re-checks on every network change, so going offline or coming back mid-barrier both end up in the right state.On by default, since forgetting to opt in would cost a silent stale read while forgetting to opt out only costs latency. Pass
{shouldClaimReadGate: false}to opt out.Fixed Issues
$ #99805
PROPOSAL: N/A - root-cause follow-up to the revert in #99814
Tests
N/A - nothing calls
writeWhenReadyin production on this branch, so there is no flow to click through. The behaviour is covered bytests/unit/SequentialQueueReadGateTest.tsandtests/unit/APIWriteWhenReadyTest.ts.Offline steps
N/A
QA steps
N/A
PR Author Checklist
### Fixed Issuessection aboveTestssectionOffline stepssectionQA stepssectionAvatar, I verified the components usingAvatarare working as expected)StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))npm run compress-svg)Avataris modified, I verified thatAvataris working as expected in all cases)Designlabel and/or tagged@Expensify/designso the design team can review the changes.mainbranch was merged into this PR after a review, I tested again and verified the outcome was still expected according to theTeststeps.Screenshots/Videos
Android: Native
N/A
Android: mWeb Chrome
N/A
iOS: Native
N/A
iOS: mWeb Safari
N/A
MacOS: Chrome / Safari
N/A
MacOS: Desktop
N/A