Event Listener Upgrade + Wrapper Types + sdk::main macro by tbraun96 · Pull Request #333 · tangle-network/blueprint

tbraun96 · 2024-10-02T21:01:28Z

No description provided.

Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com>

drewstone · 2024-10-03T15:16:25Z

Points of feedback from meeting:

Event listener is required, move existing defaults out and into the SDK as options to choose from out of the box.
Aim for as little extra code as needed on Blueprint developer for specifying their event listeners. Less is more.
Docs for each section of event listeners: EVM Contract listeners, Tangle listener, Periodic Listeners, and ways to extend and build Custom Listeners on top of this.
Docs inside tangle-docs.

shekohex

There is still more work to be done here for sure, but I believe we are deviating from the original goal of this PR which is to support other even watchers other than the default ones (Substrate, EVM), and other types of Event Handlers, so why not adopting what we currently have and add more events watcher, like the common ones we see would be good for other blueprints?

tbraun96 · 2024-10-04T23:17:59Z

There is still more work to be done here for sure, but I believe we are deviating from the original goal of this PR which is to support other even watchers other than the default ones (Substrate, EVM), and other types of Event Handlers, so why not adopting what we currently have and add more events watcher, like the common ones we see would be good for other blueprints?

My latest commit after your comment addresses this concern.

evm event_watcher notation needs to be moved into the event_listener block

* chore: merge main and clean up merge conflicts * fix: resolve more merge conflicts * fix: comments * feat: fix more merge conflicts relating to changes on main * fix: lib * fix: attempt to resolve build

It was (mistakenly?) removed in #333. Went ahead and added a default path for blueprint manager as well.

* chore: bump rustdoc-types (#348) Also pin to the latest nightly * fix: load substrate signer from env correctly (#349) * fix: add `data_dir` back to `GadgetConfiguration` (#350) It was (mistakenly?) removed in #333. Went ahead and added a default path for blueprint manager as well. * Multi job runner + SDK main macro (#346) * feat: use aggregator and separate out from the blueprint job * fix: uncomment generate_json (#358) * fix: updates using aggregator * fix: solve registry error in aggregator * fix: prevent duplicates * fix: got test working * fix: fmt and clippy cleanup * feat(cli): support custom repo and path sources (#360) * feat: added test in eigen IS blueprint, cleaned up testing code for it * Event Workflows (phase 1: Custom listeners) (#359) * Add wrapper types + trait structure to enforce structure for event flows * Decoupling/refactor and ensure integration test passes * periodic web poller working and further refactor of macro code * Everything compiling, integration test passes * fix(sdk)!: downgrade substrate dependencies for now It's currently impossible to use some APIs downstream, since they expose `sp_core` (v0.34.0) types instead of the `sdk::keystore::sp_core_subxt` (v0.31.0) types. Cargo will refuse to build blueprints using them. I just got rid of the `sdk::keystore::sp_core_subxt` hack and downgraded all of the dependencies necessary (I think). This won't be an issue once #318 is taken care of. * feat(sdk): re-export `libp2p` --------- Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com> Co-authored-by: Thomas Braun <38082993+tbraun96@users.noreply.github.com> Co-authored-by: Tjemmmic <mdtjemmes@proton.me>

* feat: add eigenlayer context * merge: combine context and proc macro tests * fix: removing unused code and fixing test * fix: move static vars to the lib * fix: move static vars to the lib * chore: debug * fix: lint and fmt to keep things clean * chore: add logging for debugging * fix: identified issue * fix: start on proc macro for abi grabbing * fix: compiles and deploys tasks, event listener not firing * chore: separate event listener files out * fix: spelling and blocks to mine * fix: more logging * chore: remove logs, evm event listener is broke * fix: remove one loop from event listening flow, polling seems to work, event catching not * Fix: Got back to original signing bug * Fix: Fixed address change - back to signature invalid revert * Fix: Correctly pulls task index from events * fix: use EL convert to g1/g2 point * Use incredible squaring aggregator (#364) * chore: bump rustdoc-types (#348) Also pin to the latest nightly * fix: load substrate signer from env correctly (#349) * fix: add `data_dir` back to `GadgetConfiguration` (#350) It was (mistakenly?) removed in #333. Went ahead and added a default path for blueprint manager as well. * Multi job runner + SDK main macro (#346) * feat: use aggregator and separate out from the blueprint job * fix: uncomment generate_json (#358) * fix: updates using aggregator * fix: solve registry error in aggregator * fix: prevent duplicates * fix: got test working * fix: fmt and clippy cleanup * feat(cli): support custom repo and path sources (#360) * feat: added test in eigen IS blueprint, cleaned up testing code for it * Event Workflows (phase 1: Custom listeners) (#359) * Add wrapper types + trait structure to enforce structure for event flows * Decoupling/refactor and ensure integration test passes * periodic web poller working and further refactor of macro code * Everything compiling, integration test passes * fix(sdk)!: downgrade substrate dependencies for now It's currently impossible to use some APIs downstream, since they expose `sp_core` (v0.34.0) types instead of the `sdk::keystore::sp_core_subxt` (v0.31.0) types. Cargo will refuse to build blueprints using them. I just got rid of the `sdk::keystore::sp_core_subxt` hack and downgraded all of the dependencies necessary (I think). This won't be an issue once #318 is taken care of. * feat(sdk): re-export `libp2p` --------- Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com> Co-authored-by: Thomas Braun <38082993+tbraun96@users.noreply.github.com> Co-authored-by: Tjemmmic <mdtjemmes@proton.me> * fix: remappings out of date * fix: remove eigenlayer-middleware from tangle blueprint git modules * chore: close handles once test completes, rpc server still not shutting down * fix: remove broken invalid test cases * fix: proc macro doc fixes to avoid more maintenance --------- Co-authored-by: Tjemmmic <mdtjemmes@proton.me> Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com> Co-authored-by: Thomas Braun <38082993+tbraun96@users.noreply.github.com>

* feat: add eigenlayer context * merge: combine context and proc macro tests * fix: removing unused code and fixing test * fix: move static vars to the lib * fix: move static vars to the lib * chore: debug * fix: lint and fmt to keep things clean * chore: add logging for debugging * fix: identified issue * fix: start on proc macro for abi grabbing * fix: compiles and deploys tasks, event listener not firing * chore: separate event listener files out * fix: spelling and blocks to mine * fix: more logging * chore: remove logs, evm event listener is broke * fix: remove one loop from event listening flow, polling seems to work, event catching not * Fix: Got back to original signing bug * Fix: Fixed address change - back to signature invalid revert * Fix: Correctly pulls task index from events * fix: use EL convert to g1/g2 point * feat: use aggregator and separate out from the blueprint job * fix: updates using aggregator * fix: solve registry error in aggregator * fix: prevent duplicates * fix: got test working * fix: fmt and clippy cleanup * feat: added test in eigen IS blueprint, cleaned up testing code for it * feat: convert aggregator process_new_jobs to blueprint * Use incredible squaring aggregator (#364) * chore: bump rustdoc-types (#348) Also pin to the latest nightly * fix: load substrate signer from env correctly (#349) * fix: add `data_dir` back to `GadgetConfiguration` (#350) It was (mistakenly?) removed in #333. Went ahead and added a default path for blueprint manager as well. * Multi job runner + SDK main macro (#346) * feat: use aggregator and separate out from the blueprint job * fix: uncomment generate_json (#358) * fix: updates using aggregator * fix: solve registry error in aggregator * fix: prevent duplicates * fix: got test working * fix: fmt and clippy cleanup * feat(cli): support custom repo and path sources (#360) * feat: added test in eigen IS blueprint, cleaned up testing code for it * Event Workflows (phase 1: Custom listeners) (#359) * Add wrapper types + trait structure to enforce structure for event flows * Decoupling/refactor and ensure integration test passes * periodic web poller working and further refactor of macro code * Everything compiling, integration test passes * fix(sdk)!: downgrade substrate dependencies for now It's currently impossible to use some APIs downstream, since they expose `sp_core` (v0.34.0) types instead of the `sdk::keystore::sp_core_subxt` (v0.31.0) types. Cargo will refuse to build blueprints using them. I just got rid of the `sdk::keystore::sp_core_subxt` hack and downgraded all of the dependencies necessary (I think). This won't be an issue once #318 is taken care of. * feat(sdk): re-export `libp2p` --------- Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com> Co-authored-by: Thomas Braun <38082993+tbraun96@users.noreply.github.com> Co-authored-by: Tjemmmic <mdtjemmes@proton.me> * fix: merge fix-el branch * feat: use aggregator blueprint * fix: remappings out of date * fix: remove eigenlayer-middleware from tangle blueprint git modules * chore: close handles once test completes, rpc server still not shutting down * fix: remove broken invalid test cases * fix: proc macro doc fixes to avoid more maintenance * chore: merge main * fix: move helpers to separate file * fix: moving helpers along w/ eigenlayer context * fix: get both blueprints running * fix: expose addresses better * feat: test exits successfully and all pieces work seamlessly * fix: add helper * fix: clippy and fmt * fix: context derive test * fix: context derive test --------- Co-authored-by: Tjemmmic <mdtjemmes@proton.me> Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com> Co-authored-by: Thomas Braun <38082993+tbraun96@users.noreply.github.com>

tbraun96 added 2 commits October 2, 2024 15:37

chore: add docs, cleanup blueprint for incredible squaring

6eec395

feat: add SubstrateEventListenerContext

3c66a16

Serial-ATA reviewed Oct 2, 2024

View reviewed changes

Comment thread docs/event_listeners.md Outdated

Serial-ATA reviewed Oct 2, 2024

View reviewed changes

Comment thread docs/event_listeners.md Outdated

Comment thread docs/event_listeners.md

Comment thread docs/event_listeners.md

Comment thread docs/event_listeners.md Outdated

drewstone and others added 3 commits October 2, 2024 23:32

Update event_listeners.md

2a12ceb

Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com>

Update event_listeners.md

400c463

Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com>

Update event_listeners.md

a2636f9

Co-authored-by: Alex <69764315+Serial-ATA@users.noreply.github.com>

shekohex requested changes Oct 3, 2024

View reviewed changes

Comment thread sdk/src/event_listener/substrate.rs Outdated

tbraun96 added 2 commits October 3, 2024 09:43

Cleanup

07cf128

EVT -> EVENT type param name change on struct

2b6a37b

tbraun96 added 4 commits October 3, 2024 17:50

Begin moving watchers into event listeners

f40cb78

wip

c2fae28

wip

7a658d8

Blueprint compiling

252d95c

shekohex requested changes Oct 4, 2024

View reviewed changes

Comment thread macros/blueprint-proc-macro/src/job.rs Outdated

Comment thread macros/blueprint-proc-macro/src/job.rs Outdated

Get multiple event listeners working w/tangle

cf2d581

drewstone changed the base branch from main to drew/gadget-runner October 7, 2024 15:45

drewstone changed the base branch from drew/gadget-runner to main October 7, 2024 15:45

tbraun96 added 4 commits October 7, 2024 15:49

work on reports macro

68495ab

QOS and report macros working

efecf37

All lints, fmt, passing

569251d

wip

6333b5c

tbraun96 changed the title ~~Event listener Docs + Wrapper Types~~ Event Listener Upgrade + Wrapper Types + sdk::main macro Oct 8, 2024

tbraun96 added 5 commits October 8, 2024 18:37

All minus 1 job-macro-related compile error in evm event listener impl

9fafad2

Get eigen mostly compiling

5e746b9

reduce complexity for next debugging session

c20da09

tangle compiling and working, evm compiling

d0f3759

evm event_watcher notation needs to be moved into the event_listener block

Remove event_watcher

83dbd1c

drewstone and others added 11 commits October 9, 2024 18:51

Merge main to reorg against eigenlayer refactoring (#344)

c3bdccc

* chore: merge main and clean up merge conflicts * fix: resolve more merge conflicts * fix: comments * feat: fix more merge conflicts relating to changes on main * fix: lib * fix: attempt to resolve build

wip

2b863d4

EvmEventListener -> EvmContractEventListener

f6ee821

Add relevant git submodules

7a973ce

1 compile error

97ef90e

Get tangle integration test passing

99ffdab

fix: simplify joinfill

0eb01fb

fix: fmt

73000a3

fix: fix compilation with joinfill issue

3b5c3b1

fix: cleanup and push testing forward

34279f0

Resolve conflicts

77f34e5

drewstone merged commit 8ee8790 into main Oct 10, 2024

drewstone deleted the event-listener-docs branch October 13, 2024 18:07

Serial-ATA added a commit that referenced this pull request Oct 15, 2024

fix: add data_dir back to GadgetConfiguration

5189686

It was (mistakenly?) removed in #333. Went ahead and added a default path for blueprint manager as well.

Serial-ATA mentioned this pull request Oct 15, 2024

fix: add data_dir back to GadgetConfiguration #350

Merged

drewstone pushed a commit that referenced this pull request Oct 15, 2024

fix: add data_dir back to GadgetConfiguration (#350)

ff53746

It was (mistakenly?) removed in #333. Went ahead and added a default path for blueprint manager as well.

This was referenced Oct 23, 2024

chore: release #378

Merged

chore: release #379

Merged

webb-spider Bot mentioned this pull request Nov 8, 2024

chore: release #445

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Event Listener Upgrade + Wrapper Types + sdk::main macro#333

Event Listener Upgrade + Wrapper Types + sdk::main macro#333
drewstone merged 32 commits into
mainfrom
event-listener-docs

tbraun96 commented Oct 2, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

drewstone commented Oct 3, 2024

Uh oh!

shekohex left a comment

Uh oh!

Uh oh!

Uh oh!

tbraun96 commented Oct 4, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

tbraun96 commented Oct 2, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

drewstone commented Oct 3, 2024

Uh oh!

shekohex left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tbraun96 commented Oct 4, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants