try-runtime: fix proof size

[kianenigma]

as spoken with Alan from moonbeam, the current proof size that's reported in the on-runtime-upgrade is not correct. IIRC, it had something to do with reading/writing the code, which is not on-par with reality.

All in all, make sure that the proof size reported by try-runtime is 100% accurate.

[crystalin]

Yes, the difference is because during the runtime upgrade of a parachain you have 2 parts.

1. Sending the runtime wasm
2. Getting it enacted and running the runtime upgrade.

In the try-runtime both are done at the same block it seems so the PoV is the sum of both action, while in reality those 2 happen in a different block

[kianenigma]

Yes, the difference is because during the runtime upgrade of a parachain you have 2 parts.

1. Sending the runtime wasm
2. Getting it enacted and running the runtime upgrade.

In the try-runtime both are done at the same block it seems so the PoV is the sum of both action, while in reality those 2 happen in a different block

Thinking about this again, I am not sure if the :CODE: is the cause of the issue actually. Any discrepancy must have been due to pre_upgrade and post_upgrade being executed in the call as well, which I will address in #12537.

:CODE: is actually never read from within the runtime in a try-runtime call, so it should on-par.

All in all, after #12537, the PoV that try-runtime reports for on-runtime-upgrade is exactly the PoV of the entire OnRuntimeUpgrade calls within the runtime. This is probably slightly less than a real block being produced, because the real block will have other stuff in it as well, but nothing more than a few on_initialize if the weight is tracked correctly.

[crystalin]

@kianenigma now that I think about it, It is porbably another case of the state_version: 1
@bkchr told me that the current behavior with state_version 0 is to include the blob in all the blocks until it is applied (which is what we observe) but this behavior is fixed with state_version 1.

[bkchr]

behavior with state_version 0 is to include the blob in all the blocks until it is applied

You mean the code? This should currently also not happening. At least for your Parachain in production. With try-runtime it could happen.

[crystalin]

@bkchr from our previous conversation on Element:

Last point also, it seems that (in substrate v0.9.26 at least) the :code is still in the pov for every block between the ValidationFunction stored (7) and the apply (10):

proposing at 11 (23 ms), Compressed PoV size: 16.77kb { header: 0.341796875kb, extrinsics: 3.013671875kb, storage_proof: 17.2353515625kb }
proposing at 10 (17 ms), Compressed PoV size: 2665.kb { header: 0.341796875kb, extrinsics: 3.02734375kb, storage_proof: 2662.1298828125kb }
proposing at 10 (21 ms), Compressed PoV size: 2665.kb { header: 0.341796875kb, extrinsics: 3.02734375kb, storage_proof: 2662.1298828125kb }
proposing at 9 (11 ms), Compressed PoV size: 1353.kb { header: 0.341796875kb, extrinsics: 2.9697265625kb, storage_proof: 1352.7060546875kb }
proposing at 8 (18 ms), Compressed PoV size: 1353.kb { header: 0.341796875kb, extrinsics: 2.9697265625kb, storage_proof: 1352.71875kb }
proposing at 7 (19 ms), Compressed PoV size: 1353.kb { header: 0.341796875kb, extrinsics: 2.9697265625kb, storage_proof: 2689.5361328125kb }
proposing at 6 (68 ms), Compressed PoV size: 1355.kb { header: 0.341796875kb, extrinsics: 1339.515625kb, storage_proof: 18.7841796875kb }
proposing at 5 (13 ms), Compressed PoV size: 16.38kb { header: 0.341796875kb, extrinsics: 2.91796875kb, storage_proof: 16.4365234375kb }
proposing at 4 (13 ms), Compressed PoV size: 18.46kb { header: 0.341796875kb, extrinsics: 2.91796875kb, storage_proof: 19.6455078125kb }

Even worse, the block when it is stored (7) and applied (10) contains 2 times the PoV of the wasm blob (and the 10 is produced 2 times as I mentioned before)
(I'll test that on v0.9.29 tomorrow)

Basti.await:
The only proper solution to this is the new trie format. Aka state_version=1

Did I misunderstood this ?

[bkchr]

I remember the discussion, but not why I said this xD I mean, for the PoV containing the wasm blob twice should probably be solved by state_version=1, because the deletion still pulls the data into the PoV. For the other case of always including the code in every PoV, it mays happens because PendingValidationCode for whatever reason is somehow a branch node. To find this out we would need to look into the trie/proof.

Now after looking again at your output, it is weird that already at block 6 the code appears in the PoV? For the block after registering the upgrade on the relay chain we call this code, which could also pull the code again into the PoV.

[kianenigma]

@crystalin have you had the chance to try any try-runtime with the correct state version and see if that gives a value that is aligned? Or even better, if you have the resources, you can try the new branch in #12537.

[crystalin]

I've been redoing the test and it seems to be fixed in current version of substrate

try-runtime =>

// storage-version = 0
main try-runtime::cli: proof size: 1.39 MB (1420.474609375 KB) (1454566 bytes)    
main try-runtime::cli: compact proof size: 1.41 MB (1447.4248046875 KB) (1482163 bytes)    
main try-runtime::cli: zstd-compressed compact proof 390.11 KB (399477 bytes)    
main try-runtime::cli: TryRuntime_on_runtime_upgrade executed without errors. Consumed weight = 17750000000, total weight = 500000000000 (0.0355)

// storage-version = 1
main try-runtime::cli: proof size: 1.36 MB (1392.947265625 KB) (1426378 bytes)    
main try-runtime::cli: compact proof size: 1.39 MB (1420.1552734375 KB) (1454239 bytes)    
main try-runtime::cli: zstd-compressed compact proof 389.56 KB (398906 bytes)    
main try-runtime::cli: TryRuntime_on_runtime_upgrade executed without errors. Consumed weight = 17750000000, total weight = 500000000000 (0.0355) 

Real runtime upgrade:

proposing at 65 ( 5 ms), Compressed PoV size: 15.29kb { header: 0.3427734375kb, extrinsics: 3.015625kb, storage_proof: 15.4638671875kb }
proposing at 66 ( 6 ms), Compressed PoV size: 15.31kb { header: 0.3427734375kb, extrinsics: 3.015625kb, storage_proof: 15.501953125kb }
proposing at 67 (27 ms), Compressed PoV size: 1380.kb { header: 0.3427734375kb, extrinsics: 1364.5458984375kb, storage_proof: 18.6103515625kb }
proposing at 68 ( 6 ms), Compressed PoV size: 1377.kb { header: 0.3427734375kb, extrinsics: 3.0673828125kb, storage_proof: 2738.76953125kb }
proposing at 69 ( 6 ms), Compressed PoV size: 1377.kb { header: 0.3427734375kb, extrinsics: 3.0673828125kb, storage_proof: 1377.181640625kb }
proposing at 70 ( 5 ms), Compressed PoV size: 1377.kb { header: 0.3427734375kb, extrinsics: 3.0673828125kb, storage_proof: 1377.0908203125kb }
proposing at 71 ( 9 ms), Compressed PoV size: 2690.kb { header: 0.3427734375kb, extrinsics: 3.125kb, storage_proof: 2687.310546875kb }
proposing at 72 (11 ms), Compressed PoV size: 408.3kb { header: 0.3427734375kb, extrinsics: 3.111328125kb, storage_proof: 1466.287109375kb }
proposing at 72 (11 ms), Compressed PoV size: 408.3kb { header: 0.3427734375kb, extrinsics: 3.111328125kb, storage_proof: 1466.287109375kb }
proposing at 73 ( 6 ms), Compressed PoV size: 15.80kb { header: 0.3427734375kb, extrinsics: 3.111328125kb, storage_proof: 16.056640625kb }
proposing at 73 ( 5 ms), Compressed PoV size: 16.16kb { header: 0.3427734375kb, extrinsics: 3.111328125kb, storage_proof: 16.056640625kb }
proposing at 74 ( 6 ms), Compressed PoV size: 15.58kb { header: 0.3427734375kb, extrinsics: 3.111328125kb, storage_proof: 15.7041015625kb }

(Block 72 is the one doing the migration)

[kianenigma]

The small difference is also because try-runtime only executes OnRutnimeUpgrade, while in reality, even if the migration block is empty, it called into OnInitialize as well.