cl: catch up Gloas alpha11 execution requests

[domiwei]

Summary

Catches Caplin/Gloas up with the consensus-specs v1.7.0-alpha.11 execution-request and builder-request shape, while keeping the broader EL syscall/contract/devnet work in #22093.

• Adds Gloas builder deposit and builder exit request handling through CL state transition, block production, CL execution-request decoding, and SSZREST getPayload encoding.
• Aligns builder deposit signature domain/version handling and execution payload bid validation with alpha11 semantics.
• Keeps alpha11 exited-builder top-up behavior: top-ups reset withdrawable_epoch for exited builders, matching the current repository spectest fixtures.
• Rejects spec-invalid builder deposit transitions for balance overflow and full/unavailable builder registry instead of silently no-oping.
• Uses forkchoice get_shuffling_dependent_root semantics for execution payload bid proposer-preference matching, avoids redundant full-state reads before bid deduplication, and keeps pending bids queued while parent state is temporarily unavailable.
• Fixes Beacon API Gloas edge cases: payload-attestation SSZ request sizing is based on PTC_SIZE, PTC duties POST no longer has a synthetic item cap beyond the existing body bound, and queued execution payload bids return gossip IGNORE while waiting for dependencies/preferences.
• Preserves Electra/Fulu ExecutionRequests SSZ/JSON/hash shape while using the Gloas request shape at Gloas-specific boundaries, including JSON null list handling and zero-value ExecutionRequests fallback behavior.
• Deep-copies ExecutionRequests.Clone() contents, including builder request lists, to avoid mutable-list aliasing.
• Aggregates payload attestations through one shared helper for API and block production, including duplicate PTC positions, out-of-PTC messages, deterministic selection, and malformed aggregate-signature groups.
• Keeps only minimal shared EIP-8282 execution request type constants on the EL protocol side; broader EL syscall/contract/devnet handling remains in execution/protocol: implement EIP-8282 Builder Execution Requests #22093.

Refs #22008.

Spec note

The repository fixture source is currently pinned to consensus-specs v1.7.0-alpha.11 in test-fixtures.json. A later consensus-specs master change adds the swept-only top-up predicate for exited builders (withdrawable_epoch != FAR_FUTURE_EPOCH && balance == 0); this PR intentionally keeps the alpha11 fixture behavior so make -C cl/spectest gloas stays green.

Validation

• GOCACHE=/private/tmp/erigon-gloas-go-cache go test ./cl/beacon/handler -run 'TestPostPayloadAttestations|TestPostPtcDuties|TestAggregatePayloadAttestation' -count=1
• GOCACHE=/private/tmp/erigon-gloas-go-cache go test ./cl/phase1/network/services -run 'TestExecutionPayloadBidService' -count=1
• GOCACHE=/private/tmp/erigon-gloas-go-cache CGO_CFLAGS=-D__BLST_PORTABLE__ go test ./cl/transition/impl/eth2 -run 'Test.*(Attestation|Builder|Gloas|Payment)' -count=1
• GOCACHE=/private/tmp/erigon-gloas-go-cache GOLANGCI_LINT_CACHE=/private/tmp/golangci-gloas-cache-22091-reviewfix make lint

go test ./cl/beacon/handler ./cl/phase1/network/services -count=1 was also attempted inside the sandbox; cl/phase1/network/services passed, while cl/beacon/handler hit the sandbox's httptest port bind restriction in TestGetBlobsFromFrozenSnapshots, unrelated to this PR.

Review

adversarial-code-review was run with the Erigon CL / Gloas specialization and subagents until convergence.

Findings addressed during the latest convergence loop:

• Payload-attestation POST SSZ cap now uses PTC_SIZE, not MAX_PAYLOAD_ATTESTATIONS.
• PTC duties POST no longer rejects valid large validator-index lists solely due to a synthetic item cap.
• Execution payload bids queued for missing dependencies/preferences now return ErrIgnore, and pending bids are retained while parent state is temporarily unavailable.
• Builder pending payment weight updates now accumulate locally and clone once before write-back.
• Block production now delegates payload-attestation aggregation to the shared helper used by the API path.
• PR scope now explicitly targets alpha11 instead of claiming current consensus-specs master semantics.

Final subagent follow-up on c9fbcf4c8a reported no remaining Critical/High/Medium actionable findings for spec/regression/pre-fork compatibility or data-race/performance/error-boundary/corner-case safety.

[Copilot]

Pull request overview

This PR updates Erigon’s CL (Caplin) Gloas support to match the latest consensus-specs execution-request shape (now including builder deposits/exits), tightens execution-payload-envelope request limits via MAX_REQUEST_PAYLOADS, and adds/updates a broad set of regression + spectest coverage (including bumping fixtures to v1.7.0-alpha.11).

Changes:

• Extend ExecutionRequests to include Gloas builder_deposits and builder_exits across SSZ/JSON/hash/clone, plus decoding/encoding from/to the flat EIP-7685 list representation.
• Add builder deposit/exit request processing through state transition and supporting EPBS/bid validation changes.
• Enforce and propagate MAX_REQUEST_PAYLOADS caps for execution payload envelope by-range/by-root (server, client, and chunked retry logic), plus Beacon REST hardening (size bounds, content-type parsing, etc.).

Reviewed changes

Copilot reviewed 37 out of 37 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test-fixtures.json	Bump consensus-spec fixtures to v1.7.0-alpha.11 mainnet tarball.
execution/types/eip7685_requests.go	Add builder request type constants and flat request lengths/types.
execution/engineapi/sszrest_wire.go	Decode execution requests via cltypes.DecodeExecutionRequestsList and ignore requests pre-Electra in SSZREST getPayload.
execution/engineapi/sszrest_test.go	Add tests for builder request decoding and pre-Electra getPayload behavior.
cl/transition/machine/machine.go	Extend block operation processor interface with builder deposit/exit handlers.
cl/transition/machine/block.go	Explicitly reject builder-index voluntary exits.
cl/transition/impl/eth2/operations.go	Wire builder requests into parent payload processing; update bid validation and pending-payment proposer binding; add nil hardening.
cl/transition/impl/eth2/operations_gloas_test.go	Add regression tests for proposer slashing clearing logic and builder request handling.
cl/spectest/consensus_tests/operations.go	Add spectest handlers for builder deposit/exit requests; adjust bid handler input.
cl/spectest/consensus_tests/appendix.go	Register new operations + SSZ static tests; version-aware ExecutionRequests instantiation.
cl/sentinel/handlers/execution_payload_envelopes.go	Apply MAX_REQUEST_PAYLOADS enforcement to by-range/by-root handlers.
cl/sentinel/handlers/execution_payload_envelopes_test.go	Update tests for version-aware requests and new payload caps; add over-limit test.
cl/rpc/rpc.go	Add MaxRequestPayloads() helper and enforce limits for envelope requests.
cl/rpc/rpc_test.go	Add tests for envelope request limit enforcement and fallback behavior.
cl/phase1/stages/gloas_payload_test.go	Use version-aware ExecutionRequests for Gloas fixtures.
cl/phase1/network/services/execution_payload_bid_service.go	Switch proposer-preference matching to forkchoice shuffling-dependent-root semantics; strengthen bid validation; refine pending-bid keying.
cl/phase1/network/services/execution_payload_bid_service_test.go	Expand bid service tests for version gating, dependent root, blob limits, randao checks, and pending-queue key uniqueness.
cl/phase1/network/envelopes.go	Chunk by-root/by-range envelope requests using MAX_REQUEST_PAYLOADS; retain partial responses across failures.
cl/phase1/network/envelopes_test.go	Add tests for filtering unsolicited envelopes / requested-root retention.
cl/phase1/forkchoice/mock_services/forkchoice_mock.go	Respect alwaysCopy by returning a copied state when requested.
cl/phase1/core/state/upgrade.go	Use version-aware empty ExecutionRequests root during Gloas upgrade.
cl/phase1/core/state/epbs.go	Add builder-deposit signature verification; builder registry hardening/limits; overflow guards; implement builder deposit request application.
cl/phase1/core/state/epbs_test.go	Add tests for builder request signature domain behavior, registry limit enforcement, and overflow guards.
cl/cltypes/solid/builder_requests.go	New SSZ/HTR types for BuilderDepositRequest and BuilderExitRequest.
cl/cltypes/execution_requests.go	Expand ExecutionRequests to 5 lists with version-aware SSZ/JSON/HTR/clone + flat-list decoder.
cl/cltypes/epbs_payload.go	Ensure bid/container types support updated request root handling and list sizing.
cl/cltypes/epbs_payload_test.go	Add tests for proposer index inclusion in BuilderPendingPayment SSZ and clone behavior.
cl/cltypes/epbs_builder.go	Add proposer index field to pending payments; deep-copy withdrawal/payment clones; update SSZ/HTR.
cl/cltypes/beacon_block.go	Construct version-aware ExecutionRequests and include builder lists at Gloas boundaries; update flat-list encoding helper.
cl/cltypes/beacon_block_test.go	Add tests for builder request list encoding/decoding and JSON version gating/null handling.
cl/cltypes/beacon_block_blinded.go	Use version-aware ExecutionRequests for blinded body construction.
cl/clparams/config.go	Add MAX_REQUEST_PAYLOADS, builder request limits/types, DomainBuilderDeposit, and PayloadBuilderVersion to config.
cl/beacon/handler/epbs.go	Harden EPBS endpoints (size limits, content-type parsing, PTC duties caps); aggregate payload attestations; adjust bid response shape.
cl/beacon/handler/epbs_test.go	Add tests for new size/cap behaviors, SSZ bid submission, and payload attestation aggregation behavior.
cl/beacon/handler/block_production.go	Decode execution requests via unified decoder; compute Gloas requests root from envelope container; reject blinded blocks at Gloas.
cl/beacon/handler/block_production_test.go	Add test ensuring blinded blocks are rejected at Gloas.
cl/beacon/builder/client.go	Make ExecutionRequests instantiation version-aware for builder API calls.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[yperbasis]

Requesting changes. The per-bid full-state copy (inline on execution_payload_bid_service.go) is the blocker; a payload-attestation aggregation under-count and an error-handling nit follow inline.

Minor (pre-existing line, not inline): block_production_test.go:481 uses a version-0 &cltypes.ExecutionRequests{} in a Gloas FULL-payload fixture — prefer NewExecutionRequestsWithVersion(..., clparams.GloasVersion) to mirror a real envelope. Harmless today.

[Copilot]

Pull request overview

Copilot reviewed 40 out of 40 changed files in this pull request and generated 1 comment.

[domiwei]

Addressed another round of review findings in 4c3d20a8f0.

• Reverted the forced full-state copy per envelope in applyEnvelopeLocked/applyLocalSelfBuildEnvelopeLocked (GetState(root, true) → false). Envelope verification only reads the state — LatestBlockHeader() returns by value and the consume-once PreviousStateRoot is restored with the same correct value on every path — so the copy (plus reinitCaches over all validators, per slot, under the forkchoice write lock) was pure overhead. The stale comment describing the old backfill behavior is replaced.
• Bid service now fetches the parent state only on a validation-state cache miss, inside entry.mu, so repeated bids for the same (parent, slot) no longer take the forkchoice read lock (or a disk load + replay under reorg/lag) per bid, and concurrent fetches for the same key are deduplicated. errBidDependencyUnavailable retry semantics are unchanged.
• Engine SSZ-REST getPayload resolves the beacon config from the running chain (GetConfigsByNetworkName, mainnet fallback for unknown chains) instead of hardcoding mainnet, so request-type bytes and per-payload list limits follow the network.
• maxSignedExecutionPayloadBidSSZSize is derived from EncodingSizeSSZ() instead of magic numbers (value unchanged: 196932).
• The MAX_REQUEST_PAYLOADS → MaxRequestBlocksDeneb fallback duplicated between cl/rpc and the sentinel handlers is now a single BeaconChainConfig.MaxRequestPayloadsLimit() method, so the request and serve sides cannot drift.
• IsValidDepositSignature/IsValidBuilderDepositSignature share one domain-parameterized helper (byte-for-byte equivalent).

One observation from the follow-up adversarial review, for the record: the alwaysCopy revert makes envelope processing a (benign, correct-valued) writer of the shared currentState again, which marginally widens the pre-existing unlocked-Copy race window used by getCheckpointState/GetStateAtBlockRoot(_, true) — the same race class already tolerated for AddChainSegment, and it only triggers when the copied root equals the tip root.

Validation:

• go build ./cl/... ./execution/engineapi/...
• go test ./cl/beacon/handler ./cl/phase1/network/services ./cl/phase1/core/state ./cl/sentinel/handlers ./cl/rpc ./execution/engineapi ./cl/phase1/forkchoice/... -count=1
• go test -race ./cl/phase1/network/services -run 'TestExecutionPayloadBid' -count=1
• make lint twice with a fresh cache: 0 issues both runs

An adversarial subagent review of the commit itself reported no correctness regressions (it specifically traced the PreviousStateRoot consume/restore interleavings across error paths for the shared-state change).

[yperbasis]

High

1. cl/beacon/handler/epbs.go:866 — queued bids get HTTP 400 and are never gossip-published. ProcessMessage now returns ErrIgnore-wrapped errors when it queues a bid (proposer preferences / parent state not yet available), but the handler maps any error to 400 and returns before the Publish at line 879. Other pool handlers treat errors.Is(err, services.ErrIgnore) as success (e.g. pool.go:312), and processPendingBids never publishes either — so a valid bid POSTed just before preferences arrive is rejected and never propagates from this node. Add the ErrIgnore carve-out, and consider publishing queued bids once they validate.

2. cl/phase1/network/services/execution_payload_bid_service.go:369 — unsynchronized copy of a possibly-shared live state, now cached. For head-parent bids (the common case) GetStateAtBlockRoot(root, false) returns the shared f.currentState pointer and releases the RLock on return; the subsequent Slot()/Copy() race with OnBlock's in-place TransitionState, and the possibly-torn copy is persisted in validationStateCache and reused for randao/builder/signature checks. (Pattern pre-exists, e.g. proposer_preferences_service.go:114, but caching makes it worse.) Also, for non-head parents the call already returns a freshly replayed caller-owned state, so the unconditional Copy() materializes the full state twice per miss — alwaysCopy=true and dropping the explicit Copy removes that; the head-root copy additionally needs to happen under the forkchoice lock.

3. cl/phase1/network/services/execution_payload_bid_service.go:118 — validationStateCache pins up to 4 full CachingBeaconStates with no TTL or invalidation. Hundreds of MB each at mainnet scale, and entries are dead ~2 slots after creation (bids are only valid for the current/next slot) yet survive until LRU displacement. No other gossip service pins full states. Use lru.NewWithTTL / slot-tick pruning, or cache something slimmer.

Medium

4. cl/phase1/network/services/execution_payload_bid_service.go:318 — highest-bid check runs after state fetch and BLS verify. It is a field-only LRU lookup and an IGNORE condition with no spec-mandated order. With up to ~256 builders bidding per slot, every losing bid pays a full BLS verification before being discarded. Hoist the value check ahead of the expensive work (keeping the pre-Add re-check).

5. cl/phase1/core/state/epbs.go:378 — builder-registry logic duplicated across state, transition and gossip. Four copies of the builder-by-pubkey scan (epbs.go:233/335/381, operations.go:1794); ApplyBuilderDepositRequest duplicates ~80–85% of ApplyDepositForBuilder; and the usable-builder predicate is spelled out twice (validateBuilderAvailability bid service :410 vs ProcessExecutionPayloadBid operations.go:532–543) — where the nil/bounds guard at operations.go:536 is dead code (IsActiveBuilder just performed those exact checks) and the same builder entry is fetched four times. Centralize state-layer helpers (BuilderIndexByPubkey, GetPayloadBuilder/ValidateBuilderForBid), preserving ErrIgnore-only-for-cover-bid on the gossip side and the self-build bypass.

6. cl/cltypes/execution_requests.go:83 — the Electra-vs-Gloas schema decision is hand-maintained in ~7 places. The effectiveVersion() < GloasVersion branch is repeated in EncodingSizeSSZ/EncodeSSZ/DecodeSSZ/HashSSZ plus both JSON methods, and the ordered type-byte mapping lives in two files (GetExecutionRequestsList if-chain, beacon_block.go:763, vs the DecodeExecutionRequestsList switch that enforces strictly-ascending order). Missing one site at the next fork means silent hash/encoding divergence. A version-dispatched schema() (exact BeaconBody.getSchema precedent) plus one canonical {typeByte, minVersion, list} table driving both encode and decode; Clone()'s five identical copy loops can share a small helper.

7. cl/clparams/config.go:703 — EIP-8282 type bytes added as yaml-configurable, but the spec defines them as Constants. A bogus yaml key silently diverges CL from the EL's hard constants (execution/types/eip7685_requests.go) at the SSZ-REST decode boundary. Since this extends the pre-existing 0x00–0x02 pattern, the proportionate fix is a startup cross-check asserting the cfg values match the execution/types constants (all five types).

Low

8. cl/beacon/handler/epbs.go:271 — on ViewHeadState/aggregation failure the pool GET returns an empty 200 with a Debug-only log; sibling endpoints surface 503 (e.g. :917).
9. Nits: verbatim-duplicated queue-and-ignore blocks in ProcessMessage (bid service :184/:201) and GetHeader running twice per accepted bid; executionRequestsFromList (sszrest_wire.go:261) is a one-line pass-through — inline it; maxExecutionPayloadEnvelopeRequestSize = MaxRlpBlockSize*4 borrows an EL RLP constant with an undocumented ×4 while sibling caps derive from SSZ sizes.

[domiwei]

Addressed the latest review feedback in a200ffe:\n\n- Split queued bid handling from hard ErrIgnore with a new ErrBidQueued sentinel. REST now returns 200 and republishes only for actually queued dependency/preference cases; hard ignore cases return 400 and are not gossiped.\n- Moved the highest-bid check ahead of proposer-preference/state dependency work, while keeping the final re-check before storing. This rejects lower bids before pending queue/state fetch work.\n- Added TTL to the bid validation-state cache so full copied states are bounded by both size and slot-time expiry.\n- Added CL/EL execution request type constant validation for scheduled Electra and Gloas request types, including the Gloas-only edge where base request types are still present in the Gloas schema.\n\nValidation:\n- make lint\n- go test ./cl/beacon/handler -run 'TestPostExecutionPayloadBid'\n- go test ./cl/phase1/network/services -run 'TestExecutionPayloadBidService(HighestBid|RejectsLowerBidBeforeStateFetch|WaitsForProposerPreferences|WaitsForParentState)'\n- go test ./cl/clparams -run 'TestCustomConfig'\n- git diff --check\n\nI also reran the adversarial subagent review loop after the fixes; both the safety/perf and spec/regression reviewers reported convergence with no new actionable findings.

[yperbasis]

CI is red

[Copilot]

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 1 comment.

[yperbasis]

Reviewed head 7d5541ba92 against consensus-specs v1.7.0-alpha.11 (beacon-chain/fork/p2p/validator), plus local build, affected test packages, and lint — no spec deviations found. Approving; findings below are non-blocking.

Findings

• PostEthV1BeaconExecutionPayloadBid still publishes the bid to gossip when ProcessMessage returns ErrBidQueued, i.e. before signature/builder/randao validation; peers will REJECT an invalid bid and down-score us. Not a regression (the old nil return published too), but since this PR introduces the queued distinction, consider deferring the publish until the pending bid validates.
• Low: the final validateHighestBid + seenCache.Add + HighestBids.Add in validateAndStoreBid is check-then-store with no lock spanning it — two concurrently validated bids that both beat the old max can land lower-last, and two concurrent first bids from one builder can both do full BLS. A small mutex around the store would close it.
• Note: EngineServer.beaconChainConfig() falls back chain-name → mainnet config, so an external CL on a custom-genesis devnet gets mainnet list caps in SSZ getPayload decoding. Only observable on minimal-preset networks; fine to leave.
• Note: until #22093 lands, any builder-deposit-contract traffic on devnet-6 splits erigon's EL from other clients (requests_hash over types 0x03/0x04) — the green kurtosis job just means assertoor doesn't exercise it. Relatedly, KnownRequestTypes now includes 0x03/0x04 unconditionally (currently unreferenced); when #22093 wires it into newPayload validation, the pre-Gloas fork-gating has to happen there.

Nits

• GetEthV1BeaconPoolPayloadAttestations returns 200-with-empty-list when ViewHeadState/aggregation fails, masking "not synced" as "no attestations"; the four identical returns could also collapse to one exit point.
• requestEnvelopesByRange derives count from the full block span before chunking, so widely spaced blocks sweep empty slot ranges — bounded, and it's a fallback path.
• maxSignedExecutionPayloadBidSSZSize() is recomputed per request; a package-level var would do.

Re the Copilot comment on DecodeExecutionRequestsList rejecting 1-byte entries (#22091 (comment)): false positive. Both the engine API ("has a length of 1-byte or shorter … MUST return -32602"; "Elements MUST be longer than 1-byte") and consensus-specs get_execution_requests (assert len(request_data) != 0, kept in the gloas alpha11 version) mandate the rejection, and TestDecodeExecutionRequestsListRejectsInvalidShape already pins it.

[domiwei]

Addressed the latest adversarial-review findings in 5e14b0c7b1:\n\n- Pending execution payload bids now key by signed bid root in addition to builder/slot, so distinct same-builder same-slot bids no longer overwrite or suppress each other; exact duplicates still dedup and the global pending cap/expiry still bound the queue.\n- Final bid storage now serializes the seen check plus highest-bid compare/write under bidStoreMu, keeping expensive validation outside the lock.\n- Missing-block payload attestations now return ErrAttestationQueued; REST accepts queued attestations without pooling or gossiping them.\n- Pending payload attestations now key by signed attestation root in addition to block/validator, so distinct same-validator same-block messages can coexist until validation.\n\nValidation run locally:\n- go test ./cl/phase1/network/services\n- go test ./cl/beacon/handler -run 'TestPostPayloadAttestations|TestPostExecutionPayloadBid'\n- go test ./cl/beacon/handler (non-sandbox; httptest needs local port bind)\n- git diff --check\n- make lint\n\nI also reran the two adversarial subagent reviewers after the fixes; both converged with no new Critical/High/Medium actionable findings. Residual queue DoS is bounded by the existing global caps and expiry.