Skip to content

confirm: walk past taken BQ hole; loadq=14 writeq=14 - #137

Merged
reardencode merged 2 commits into
masterfrom
confirm/lookup-taken-walk
Aug 18, 2026
Merged

reardencode merged 2 commits into
masterfrom
confirm/lookup-taken-walk

Conversation

@rearden-grok

@rearden-grok rearden-grok Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Why

After loadq, lookup take_raws the BQ row when it emits a batch. The next block_queue_unresolved_heights(path_lo=tip+1) saw a hole at tip+1 and returned empty, so lookup idled after one wave (~18–25 fat blocks). Scripts sat at scriptq=1 while 200+ later heights sat on the BQ. Host rate at 932–943k dropped ~6.2/s → ~4.4/s.

What

  • Query walk starts at max(path_lo, lookup_taken_hi+1). A missing height above that high-water is still a fetch hole.
  • Pin: unresolved_heights_walks_past_taken_bq_hole (store walker unchanged).
  • Queue caps: loadq=14 · scriptq=4 · writeq=14 (the knobs already on the host).

Test

cargo test -p rbitcoin-query --lib unresolved_heights
cargo test -p rbitcoin-store --lib unresolved_heights_
cargo test -p rbitcoin-net --lib queue_depth_log_and_caps_surface

rearden-grok Bot added 2 commits August 18, 2026 13:17
take_raw removes the BQ row, so unresolved_heights starting at
tip+1 treated the taken prefix as a fetch hole and lookup idled
after one wave. Start the walk after lookup_taken_hi; a missing
height above that high-water still stops.

Queue caps match the running host knobs (14/4/14).
format_info / format_debug / format_sizes pin the live default cap.
@reardencode
reardencode merged commit 930bcf0 into master Aug 18, 2026
13 checks passed
@rearden-grok
rearden-grok Bot deleted the confirm/lookup-taken-walk branch August 19, 2026 18:58
average-gary added a commit to average-gary/rbitcoin that referenced this pull request Oct 7, 2026
Plan D lands on top of sv2/plan-c-fee-push (reardencode reardencode#949, reardencode#951), which
supplies the Arc<Transaction> template bodies and the 64-slot same-tip
retention that validated jobs now rely on instead of a pin. Readers coming
from sv2-spec discussion reardencode#239 also get the pointer to the Core-IPC
implementation of the same messages (sv2-tp reardencode#137).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
average-gary added a commit to average-gary/rbitcoin that referenced this pull request Oct 9, 2026
Plan D lands on top of sv2/plan-c-fee-push (reardencode reardencode#949, reardencode#951), which
supplies the Arc<Transaction> template bodies and the 64-slot same-tip
retention that validated jobs now rely on instead of a pin. Readers coming
from sv2-spec discussion reardencode#239 also get the pointer to the Core-IPC
implementation of the same messages (sv2-tp reardencode#137).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
average-gary added a commit to average-gary/rbitcoin that referenced this pull request Oct 9, 2026
sv2-spec#239 settled on the JDP shape for the missing-transactions leg
(Sjors, 2026-10-08): the TP asks with a proper message and the JDS
answers with one, reusing the 6.4.7 and 6.4.8 layouts under TDP numbers,
and the TP holds the pending proposal. ProposeTemplate drops
transaction_list; ProposeTemplate.MissingTransactions becomes
ProvideMissingTransactions (0x78, TP to client) and
ProvideMissingTransactions.Success (0x7b, client to TP) carries the txs.
A JDS copies both payloads between the TP and its JDC unchanged.

The session keeps a proposal it answered ProvideMissingTransactions
under its request_id (wtxids and coinbase split, no transactions) in a
table of at most MAX_PENDING_PROPOSALS (8) for provide_timeout (30 s,
Sv2TpConfig like setup_timeout); past the bound the oldest is dropped.
A provide for an id it does not hold, already consumed, or expired is
ProposeTemplate.Error unknown-request-id; a proposal under an id still
queued, in flight, or held is duplicate-request-id; a provide short of a
requested position, with a tx the TP did not ask for, or with one that
does not decode is bad-missing-tx and ends that exchange, the first two
before any decode. These edge rules match sv2-tp reardencode#137. A validation that
still lacks a transaction (one left the mempool between rounds) asks
again for every position the mempool does not hold, the supplied ones
included, since the TP holds no transactions across the round trip. The
client frame cap moves the block-sized bound onto
ProvideMissingTransactions.Success (~4.2 MB); ProposeTemplate drops to
wtxids plus coinbase (~2.3 MB).

Red was the compile failure of the journeys against the one-message
shape (no pending table, no 0x7b handler; the old session ignored
0x7b), then a short provide answered ProvideMissingTransactions again
and a ProposeTemplate under an in-flight id validated twice. The
missing-transactions journey walks the round trip, each code, the bound,
and expiry under a 1 s provide_timeout; the D11 journey completes each
copy through its provide and refuses the in-flight duplicate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
average-gary added a commit to average-gary/rbitcoin that referenced this pull request Oct 9, 2026
Plan D lands on top of sv2/plan-c-fee-push (reardencode reardencode#949, reardencode#951), which
supplies the Arc<Transaction> template bodies and the 64-slot same-tip
retention that validated jobs now rely on instead of a pin. Readers coming
from sv2-spec discussion reardencode#239 also get the pointer to the Core-IPC
implementation of the same messages (sv2-tp reardencode#137).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
average-gary added a commit to average-gary/rbitcoin that referenced this pull request Oct 9, 2026
sv2-spec#239 settled on the JDP shape for the missing-transactions leg
(Sjors, 2026-10-08): the TP asks with a proper message and the JDS
answers with one, reusing the 6.4.7 and 6.4.8 layouts under TDP numbers,
and the TP holds the pending proposal. ProposeTemplate drops
transaction_list; ProposeTemplate.MissingTransactions becomes
ProvideMissingTransactions (0x78, TP to client) and
ProvideMissingTransactions.Success (0x7b, client to TP) carries the txs.
A JDS copies both payloads between the TP and its JDC unchanged.

The session keeps a proposal it answered ProvideMissingTransactions
under its request_id (wtxids and coinbase split, no transactions) in a
table of at most MAX_PENDING_PROPOSALS (8) for provide_timeout (30 s,
Sv2TpConfig like setup_timeout); past the bound the oldest is dropped.
A provide for an id it does not hold, already consumed, or expired is
ProposeTemplate.Error unknown-request-id; a proposal under an id still
queued, in flight, or held is duplicate-request-id; a provide short of a
requested position, with a tx the TP did not ask for, or with one that
does not decode is bad-missing-tx and ends that exchange, the first two
before any decode. These edge rules match sv2-tp reardencode#137. A validation that
still lacks a transaction (one left the mempool between rounds) asks
again for every position the mempool does not hold, the supplied ones
included, since the TP holds no transactions across the round trip. The
client frame cap moves the block-sized bound onto
ProvideMissingTransactions.Success (~4.2 MB); ProposeTemplate drops to
wtxids plus coinbase (~2.3 MB).

Red was the compile failure of the journeys against the one-message
shape (no pending table, no 0x7b handler; the old session ignored
0x7b), then a short provide answered ProvideMissingTransactions again
and a ProposeTemplate under an in-flight id validated twice. The
missing-transactions journey walks the round trip, each code, the bound,
and expiry under a 1 s provide_timeout; the D11 journey completes each
copy through its provide and refuses the in-flight duplicate.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant