Skip to content

Make BeaconChain::kzg field mandatory#6267

Merged
mergify[bot] merged 27 commits into
sigp:unstablefrom
eserilev:beacon-chain-kzg-field-required
Sep 23, 2024
Merged

Make BeaconChain::kzg field mandatory#6267
mergify[bot] merged 27 commits into
sigp:unstablefrom
eserilev:beacon-chain-kzg-field-required

Conversation

@eserilev

Copy link
Copy Markdown
Member

Issue Addressed

Closes #6260

Proposed Changes

Make the kzg field in BeaconChain and BeaconChainBuilder mandatory

Comment thread beacon_node/client/src/builder.rs Outdated
@eserilev eserilev added code-quality work-in-progress PR is a work-in-progress labels Aug 16, 2024
@eserilev eserilev changed the title Make BeaconChain kzg field required Make BeaconChain::kzg field mandatory Aug 16, 2024
@dapplion

Copy link
Copy Markdown
Collaborator

This PR adds a new trusted setup, is that expected?

@eserilev

eserilev commented Aug 29, 2024

Copy link
Copy Markdown
Member Author

removed the extraneous trusted setup. It looks like loading KZG has added like 5 min to a few CI test groups. I'm going to dig in there and make sure were not doing any KZG precomputation when possible. Kevs recent PR makes this easy to do

@eserilev

Copy link
Copy Markdown
Member Author

There no longer seems to be any noticeable difference in CI times between this branch and unstable

@eserilev eserilev added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Aug 29, 2024
@michaelsproul michaelsproul added the v6.0.0 New major release for hierarchical state diffs label Sep 16, 2024
Comment thread beacon_node/beacon_chain/src/beacon_chain.rs
Comment thread beacon_node/beacon_chain/benches/benches.rs Outdated
Comment thread beacon_node/beacon_chain/src/builder.rs Outdated
Comment thread beacon_node/beacon_chain/src/data_column_verification.rs Outdated
Comment thread beacon_node/beacon_chain/src/test_utils.rs Outdated
Comment thread beacon_node/beacon_chain/tests/store_tests.rs Outdated
Comment thread beacon_node/beacon_chain/tests/store_tests.rs Outdated
Comment thread beacon_node/network/src/subnet_service/tests/mod.rs Outdated
Comment thread crypto/kzg/src/lib.rs
pub struct Kzg {
trusted_setup: KzgSettings,
context: Option<DASContext>,
context: DASContext,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could keep this DASContext optional for the pre-peerDAS case, but then again if there's no perf impact maybe it's OK to keep it initialized

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was no noticeable performance impact that I noticed (test-wise at least)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was changed to optional because it adds 550ms to startup time and 110 mb extra memory usage for an unused feature on mainnet:
#6212 (comment)

It may also impact our CI time as I imagine Kzg is loaded many times.

A profile run on our basic-sim test show that 88% of CPU time is spent on loading DASContext during startup
image

I think we should make this optional until PeerDAS is scheduled.

Comment thread crypto/kzg/src/trusted_setup.rs Outdated
@michaelsproul michaelsproul added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Sep 17, 2024
@eserilev eserilev added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Sep 19, 2024
@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Sep 23, 2024
@michaelsproul

Copy link
Copy Markdown
Member

@mergify queue

@mergify

mergify Bot commented Sep 23, 2024

Copy link
Copy Markdown

queue

✅ The pull request has been merged automatically

Details

The pull request has been merged automatically at b619f1a

mergify Bot added a commit that referenced this pull request Sep 23, 2024
@mergify mergify Bot merged commit b619f1a into sigp:unstable Sep 23, 2024
Comment on lines +107 to +109
} else {
KZG_NO_PRECOMP.clone()
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The KZG precomputation is only used for PeerDAS.
I think we can remove this branch and default to KZG if PeerDAS is not enabled?

} else if spec.deneb_fork_epoch.is_some() {
Kzg::new_from_trusted_setup(trusted_setup).map_err(kzg_err_msg)?
} else {
Kzg::new_from_trusted_setup_no_precomp(trusted_setup).map_err(kzg_err_msg)?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can get rid of this new_from_trusted_setup_no_precomp branch too?

Comment thread crypto/kzg/src/lib.rs

/// Load the kzg trusted setup parameters from a vec of G1 and G2 points.
pub fn new_from_trusted_setup(trusted_setup: TrustedSetup) -> Result<Self, Error> {
let peerdas_trusted_setup = PeerDASTrustedSetup::from(&trusted_setup);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is now identical to new_from_trusted_setup_das_enabled.
I think maybe what we want is

  • if PeerDAS scheduled, load it with DASContext
  • Else load with no DASContext OR no precomputation (although I prefer the former as initilisation of PeerDASTrustedSetup still require cloning trusted setup bytes.

chong-he pushed a commit to chong-he/lighthouse that referenced this pull request Nov 26, 2024
* make kzg field required

* update todo

* always load trusted setup WIP

* fmt

* use new rust_eth_kzg version

* merge conlficts

* add kzg fn with trusted setup disabled

* as_slice

* add kzg with no precomp

* ignore udep for kzg

* refactor kzg init

* fix peerdas kzg schedule

* fix

* udeps

* uuuudeps

* merge conflict resolved

* merge conflict

* merge conflicts

* resolve TODO

* update

* move kzg to a test util fn

* remove trusted setup default impl

* lint fmt

* fix failing test

* lint

* fix test

* Merge branch 'unstable' into beacon-chain-kzg-field-required
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

code-quality ready-for-merge This PR is ready to merge. v6.0.0 New major release for hierarchical state diffs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants