Skip to content

Add DataColumnSidecar gossip topic and message handling#6147

Merged
mergify[bot] merged 9 commits into
sigp:unstablefrom
jimmygchen:data-column-gossip
Jul 25, 2024
Merged

Add DataColumnSidecar gossip topic and message handling#6147
mergify[bot] merged 9 commits into
sigp:unstablefrom
jimmygchen:data-column-gossip

Conversation

@jimmygchen

Copy link
Copy Markdown
Member

Issue Addressed

Part of #6072.

This PR contains:

  • Add gossip data column topic and message handling (excludes gossip validation)

Upstream PRs:

@jimmygchen jimmygchen added work-in-progress PR is a work-in-progress das Data Availability Sampling das-unstable-merge labels Jul 22, 2024
# Conflicts:
#	beacon_node/beacon_chain/src/data_column_verification.rs
#	beacon_node/beacon_chain/src/lib.rs
@jimmygchen jimmygchen marked this pull request as draft July 22, 2024 14:42
@jimmygchen jimmygchen changed the title Add DataColumnSidecar gossip topic and verification (#5050 and #5783) Add DataColumnSidecar gossip topic and verification Jul 22, 2024
@jimmygchen jimmygchen added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Jul 22, 2024
@jimmygchen jimmygchen marked this pull request as ready for review July 22, 2024 14:57
@jimmygchen jimmygchen changed the title Add DataColumnSidecar gossip topic and verification Add DataColumnSidecar gossip topic and message handling Jul 22, 2024

@pawanjay176 pawanjay176 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good. The das logic seems sane, but I have mainly focused on ensuring that we don't trigger any of the das related code if this is run on mainnet.

Just to be extra safe, I would prefer to remove all instances of panic macros and just let the TODO(das) comments guide us to incomplete code.

Also confirmed that we are not subscribing to any data column subnet topics in this PR so any of the upstream code wouldn't get triggered

Comment thread beacon_node/lighthouse_network/src/types/pubsub.rs
Comment thread beacon_node/network/src/sync/manager.rs Outdated
Comment thread consensus/types/src/data_column_subnet_id.rs
Comment thread consensus/types/src/data_column_subnet_id.rs Outdated
Comment thread consensus/types/src/data_column_subnet_id.rs
@jimmygchen jimmygchen requested a review from pawanjay176 July 24, 2024 00:58
@jimmygchen jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Jul 24, 2024
Comment thread beacon_node/lighthouse_network/src/types/pubsub.rs
Comment thread beacon_node/network/src/network_beacon_processor/gossip_methods.rs
UnknownParentBlob(PeerId, Arc<BlobSidecar<E>>),

/// A data column with an unknown parent has been received.
UnknownParentDataColumn(PeerId, Arc<DataColumnSidecar<E>>),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's actually useless to have to forward the payload in this error :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're referring to GossipDataColumnError::ParentUnknown instead of SyncMessage here? we do need to keep the data column in the BlockComponent when doing a parent lookup.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the unused error types in GossipDataColumnError and leaving this line unchanged. Let me know if you still want the payload removed for this error?

@jimmygchen jimmygchen requested a review from dapplion July 24, 2024 12:28
@jimmygchen jimmygchen added ready-for-review The code is ready for review and removed ready-for-merge This PR is ready to merge. labels Jul 24, 2024
E: EthSpec,
{
let log_clone = log.clone();
let spec_clone = spec.clone();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can do this later, but we should probably avoid cloning the ChainSpec so frequently by Arcing it

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To do it properly we should probably Arc it everywhere, but we could probably do it in a separate PR as it will be quite a big change. I've Arc'd the spec only in Discovery and NetworkService to avoid the performance regression for now.

@michaelsproul michaelsproul left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved modulo the ChainSpec cloning in subnet_predicate which could represent a performance regression on mainnet

@michaelsproul

Copy link
Copy Markdown
Member

@mergify queue

@mergify

mergify Bot commented Jul 25, 2024

Copy link
Copy Markdown

queue

✅ The pull request has been merged automatically

Details

The pull request has been merged automatically at 4e5a363

mergify Bot added a commit that referenced this pull request Jul 25, 2024
@mergify mergify Bot merged commit 4e5a363 into sigp:unstable Jul 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

das Data Availability Sampling das-unstable-merge ready-for-review The code is ready for review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants