Add assertoor tests to CI#6192
Conversation
01ced96 to
c6b10ce
Compare
c6b10ce to
766dbd0
Compare
|
I don't think we need
|
…d-assertoor-github-action
…d-assertoor-github-action
|
the |
|
Can you upload the logs to the CI run artifacts? To debug failures |
|
logs should now be uploaded with a 3 day retention window |
| path: | | ||
| scripts/local_testnet/logs | ||
| retention-days: 3 |
There was a problem hiding this comment.
Looks like these keys are invalid for this github action. (see warnings on the run)
The step below defines the artifact path and retention, I think this can be removed?
There was a problem hiding this comment.
We already upload the local testnet logs, did you mean to upload the assertoor logs? They could be included in the enclave logs too but i haven't checked
| run_block_proposal_check: true | ||
| run_blob_transaction_test: true | ||
| tests: | ||
| - https://raw.githubusercontent.com/ethpandaops/assertoor-test/master/assertoor-tests/big-calldata-tx-test.yaml No newline at end of file |
There was a problem hiding this comment.
Looks like the big-calldata-tx-test.yaml is the one taking the most time (>20m) and doesn't run concurrently with the other 3 tests. Maybe we can exclude this one from the CI job and have it as a recurring test on GitHub action?
Add assertoor tests to CI
This PR adds a new step to the local_testnet CI. This new step spins up a kurtosis testnet (defined in
assertoor_network_params.yml) and runs assertoor tests as a github action viakurtosis-assertoor-github-actionIssues w/ assertoor github action
Note that the repository ppa:rmescandon/yq is no longer maintained and does not contain the most up to date version of
yqmikefarah/yq#1308
Installation recommendations for
yqcan be found here:https://github.com/mikefarah/yq?tab=readme-ov-file#install
I've updated the
local_testnet.ymlscript to installyqviasnapChoosing which tests to run
The list of assertoor tests can be found here: https://github.com/ethpandaops/assertoor-test/tree/master/assertoor-tests
Theres a few tests that are broken at the momemnt (mev-block-proposal as an example) and a few that take 48 hours to complete. I've picked the most relevant tests that finish running at a reasonable time.
Theres some electra tests that we may want to add in the future