feat(blst): use lodestar-z blst - #8900
Conversation
blst with blst-z where possible
Summary of ChangesHello @spiral-ladder, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request updates the beacon node's cryptographic backend by replacing the existing Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
blst with blst-z where possibleblst with blst-z where possible
There was a problem hiding this comment.
Code Review
This pull request replaces the @chainsafe/blst dependency with @chainsafe/blst-z in most of the codebase. The changes are mostly import replacements, but also include an update to the aggregateSignatures function calls, adding a false argument to likely disable signature verification for performance. In two spec test files, aggregateVerify is still imported from the old @chainsafe/blst library, which seems intentional. My review includes suggestions to add comments explaining this mixed dependency for better maintainability.
5604001 to
59688b7
Compare
|
You should be able to either |
ec6bb4d to
6c95d69
Compare
blst with blst-z where possibleblst with lodestarz/blst
blst with lodestarz/blstblst with lodestar-z/blst
|
| Benchmark suite | Current: 351f758 | Previous: 713b218 | Ratio |
|---|---|---|---|
| proposeBlockBody type=full, size=empty | 3.7098 ms/op | 705.63 us/op | 5.26 |
| phase0 getAttestationDeltas - 250000 normalcase | 20.212 ms/op | 5.7549 ms/op | 3.51 |
Full benchmark results
| Benchmark suite | Current: 351f758 | Previous: 713b218 | Ratio |
|---|---|---|---|
| getPubkeys - index2pubkey - req 1000 vs - 250000 vc | 881.16 us/op | 1.3970 ms/op | 0.63 |
| getPubkeys - validatorsArr - req 1000 vs - 250000 vc | 41.127 us/op | 40.437 us/op | 1.02 |
| BLS verify - blst | 904.87 us/op | 761.81 us/op | 1.19 |
| BLS verifyMultipleSignatures 3 - blst | 1.3766 ms/op | 1.3520 ms/op | 1.02 |
| BLS verifyMultipleSignatures 8 - blst | 2.2631 ms/op | 2.1523 ms/op | 1.05 |
| BLS verifyMultipleSignatures 32 - blst | 7.4355 ms/op | 6.8202 ms/op | 1.09 |
| BLS verifyMultipleSignatures 64 - blst | 13.984 ms/op | 13.336 ms/op | 1.05 |
| BLS verifyMultipleSignatures 128 - blst | 26.491 ms/op | 25.281 ms/op | 1.05 |
| BLS deserializing 10000 signatures | 647.81 ms/op | 644.34 ms/op | 1.01 |
| BLS deserializing 100000 signatures | 6.4482 s/op | 6.5224 s/op | 0.99 |
| BLS verifyMultipleSignatures - same message - 3 - blst | 948.72 us/op | 795.30 us/op | 1.19 |
| BLS verifyMultipleSignatures - same message - 8 - blst | 1.0774 ms/op | 963.53 us/op | 1.12 |
| BLS verifyMultipleSignatures - same message - 32 - blst | 1.6936 ms/op | 1.5599 ms/op | 1.09 |
| BLS verifyMultipleSignatures - same message - 64 - blst | 2.5090 ms/op | 2.4625 ms/op | 1.02 |
| BLS verifyMultipleSignatures - same message - 128 - blst | 4.1409 ms/op | 4.0952 ms/op | 1.01 |
| BLS aggregatePubkeys 32 - blst | 19.342 us/op | 17.866 us/op | 1.08 |
| BLS aggregatePubkeys 128 - blst | 68.767 us/op | 63.509 us/op | 1.08 |
| getSlashingsAndExits - default max | 50.335 us/op | 51.498 us/op | 0.98 |
| getSlashingsAndExits - 2k | 363.67 us/op | 394.05 us/op | 0.92 |
| proposeBlockBody type=full, size=empty | 3.7098 ms/op | 705.63 us/op | 5.26 |
| isKnown best case - 1 super set check | 163.00 ns/op | 182.00 ns/op | 0.90 |
| isKnown normal case - 2 super set checks | 164.00 ns/op | 166.00 ns/op | 0.99 |
| isKnown worse case - 16 super set checks | 165.00 ns/op | 167.00 ns/op | 0.99 |
| validate api signedAggregateAndProof - struct | 1.5163 ms/op | 1.5007 ms/op | 1.01 |
| validate gossip signedAggregateAndProof - struct | 1.5219 ms/op | 1.4828 ms/op | 1.03 |
| batch validate gossip attestation - vc 640000 - chunk 32 | 111.50 us/op | 109.65 us/op | 1.02 |
| batch validate gossip attestation - vc 640000 - chunk 64 | 98.002 us/op | 100.51 us/op | 0.98 |
| batch validate gossip attestation - vc 640000 - chunk 128 | 90.160 us/op | 95.316 us/op | 0.95 |
| batch validate gossip attestation - vc 640000 - chunk 256 | 88.530 us/op | 91.341 us/op | 0.97 |
| bytes32 toHexString | 299.00 ns/op | 300.00 ns/op | 1.00 |
| bytes32 Buffer.toString(hex) | 173.00 ns/op | 168.00 ns/op | 1.03 |
| bytes32 Buffer.toString(hex) from Uint8Array | 237.00 ns/op | 239.00 ns/op | 0.99 |
| bytes32 Buffer.toString(hex) + 0x | 165.00 ns/op | 175.00 ns/op | 0.94 |
| Return object 10000 times | 0.21140 ns/op | 0.21620 ns/op | 0.98 |
| Throw Error 10000 times | 3.1764 us/op | 3.3616 us/op | 0.94 |
| toHex | 106.92 ns/op | 102.21 ns/op | 1.05 |
| Buffer.from | 83.559 ns/op | 92.518 ns/op | 0.90 |
| shared Buffer | 56.098 ns/op | 58.101 ns/op | 0.97 |
| fastMsgIdFn sha256 / 200 bytes | 1.4220 us/op | 1.4680 us/op | 0.97 |
| fastMsgIdFn h32 xxhash / 200 bytes | 163.00 ns/op | 153.00 ns/op | 1.07 |
| fastMsgIdFn h64 xxhash / 200 bytes | 204.00 ns/op | 203.00 ns/op | 1.00 |
| fastMsgIdFn sha256 / 1000 bytes | 4.6090 us/op | 4.7830 us/op | 0.96 |
| fastMsgIdFn h32 xxhash / 1000 bytes | 254.00 ns/op | 249.00 ns/op | 1.02 |
| fastMsgIdFn h64 xxhash / 1000 bytes | 260.00 ns/op | 255.00 ns/op | 1.02 |
| fastMsgIdFn sha256 / 10000 bytes | 42.156 us/op | 41.764 us/op | 1.01 |
| fastMsgIdFn h32 xxhash / 10000 bytes | 1.2880 us/op | 1.2930 us/op | 1.00 |
| fastMsgIdFn h64 xxhash / 10000 bytes | 840.00 ns/op | 832.00 ns/op | 1.01 |
| send data - 1000 256B messages | 4.2361 ms/op | 4.9202 ms/op | 0.86 |
| send data - 1000 512B messages | 5.8632 ms/op | 7.4359 ms/op | 0.79 |
| send data - 1000 1024B messages | 5.7120 ms/op | 7.0452 ms/op | 0.81 |
| send data - 1000 1200B messages | 6.9480 ms/op | 6.9359 ms/op | 1.00 |
| send data - 1000 2048B messages | 20.801 ms/op | 11.859 ms/op | 1.75 |
| send data - 1000 4096B messages | 77.695 ms/op | 55.240 ms/op | 1.41 |
| send data - 1000 16384B messages | 340.91 ms/op | 234.86 ms/op | 1.45 |
| send data - 1000 65536B messages | 1.7191 s/op | 1.1756 s/op | 1.46 |
| enrSubnets - fastDeserialize 64 bits | 714.00 ns/op | 751.00 ns/op | 0.95 |
| enrSubnets - ssz BitVector 64 bits | 256.00 ns/op | 259.00 ns/op | 0.99 |
| enrSubnets - fastDeserialize 4 bits | 100.00 ns/op | 108.00 ns/op | 0.93 |
| enrSubnets - ssz BitVector 4 bits | 261.00 ns/op | 261.00 ns/op | 1.00 |
| prioritizePeers score -10:0 att 32-0.1 sync 2-0 | 202.17 us/op | 199.00 us/op | 1.02 |
| prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 | 233.68 us/op | 230.30 us/op | 1.01 |
| prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 | 328.25 us/op | 338.10 us/op | 0.97 |
| prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 | 607.99 us/op | 595.23 us/op | 1.02 |
| prioritizePeers score 0:0 att 64-1 sync 4-1 | 695.19 us/op | 701.85 us/op | 0.99 |
| array of 16000 items push then shift | 1.2455 us/op | 1.2852 us/op | 0.97 |
| LinkedList of 16000 items push then shift | 7.8670 ns/op | 7.6300 ns/op | 1.03 |
| array of 16000 items push then pop | 67.100 ns/op | 80.742 ns/op | 0.83 |
| LinkedList of 16000 items push then pop | 6.1730 ns/op | 6.1020 ns/op | 1.01 |
| array of 24000 items push then shift | 1.8454 us/op | 1.8883 us/op | 0.98 |
| LinkedList of 24000 items push then shift | 9.1560 ns/op | 7.2620 ns/op | 1.26 |
| array of 24000 items push then pop | 101.43 ns/op | 114.04 ns/op | 0.89 |
| LinkedList of 24000 items push then pop | 6.8770 ns/op | 6.1100 ns/op | 1.13 |
| intersect bitArray bitLen 8 | 3.8930 ns/op | 3.9750 ns/op | 0.98 |
| intersect array and set length 8 | 29.972 ns/op | 29.163 ns/op | 1.03 |
| intersect bitArray bitLen 128 | 23.337 ns/op | 23.149 ns/op | 1.01 |
| intersect array and set length 128 | 510.24 ns/op | 496.66 ns/op | 1.03 |
| bitArray.getTrueBitIndexes() bitLen 128 | 998.00 ns/op | 963.00 ns/op | 1.04 |
| bitArray.getTrueBitIndexes() bitLen 248 | 1.7540 us/op | 1.7080 us/op | 1.03 |
| bitArray.getTrueBitIndexes() bitLen 512 | 3.7990 us/op | 3.5860 us/op | 1.06 |
| Full columns - reconstruct all 6 blobs | 175.32 us/op | 158.14 us/op | 1.11 |
| Full columns - reconstruct half of the blobs out of 6 | 86.634 us/op | 103.47 us/op | 0.84 |
| Full columns - reconstruct single blob out of 6 | 34.126 us/op | 32.903 us/op | 1.04 |
| Half columns - reconstruct all 6 blobs | 415.25 ms/op | 391.52 ms/op | 1.06 |
| Half columns - reconstruct half of the blobs out of 6 | 205.94 ms/op | 197.90 ms/op | 1.04 |
| Half columns - reconstruct single blob out of 6 | 74.848 ms/op | 70.924 ms/op | 1.06 |
| Set add up to 64 items then delete first | 1.6951 us/op | 1.6405 us/op | 1.03 |
| OrderedSet add up to 64 items then delete first | 2.5634 us/op | 2.5345 us/op | 1.01 |
| Set add up to 64 items then delete last | 1.8892 us/op | 1.8577 us/op | 1.02 |
| OrderedSet add up to 64 items then delete last | 2.8257 us/op | 3.0439 us/op | 0.93 |
| Set add up to 64 items then delete middle | 1.8955 us/op | 1.8815 us/op | 1.01 |
| OrderedSet add up to 64 items then delete middle | 4.3165 us/op | 4.5716 us/op | 0.94 |
| Set add up to 128 items then delete first | 3.8102 us/op | 3.6749 us/op | 1.04 |
| OrderedSet add up to 128 items then delete first | 5.7808 us/op | 5.7287 us/op | 1.01 |
| Set add up to 128 items then delete last | 3.6273 us/op | 3.6183 us/op | 1.00 |
| OrderedSet add up to 128 items then delete last | 5.4738 us/op | 6.2908 us/op | 0.87 |
| Set add up to 128 items then delete middle | 3.7061 us/op | 3.6457 us/op | 1.02 |
| OrderedSet add up to 128 items then delete middle | 11.463 us/op | 11.932 us/op | 0.96 |
| Set add up to 256 items then delete first | 7.6856 us/op | 7.3754 us/op | 1.04 |
| OrderedSet add up to 256 items then delete first | 12.829 us/op | 12.158 us/op | 1.06 |
| Set add up to 256 items then delete last | 7.3212 us/op | 7.6209 us/op | 0.96 |
| OrderedSet add up to 256 items then delete last | 12.083 us/op | 11.679 us/op | 1.03 |
| Set add up to 256 items then delete middle | 7.9538 us/op | 7.1280 us/op | 1.12 |
| OrderedSet add up to 256 items then delete middle | 35.724 us/op | 35.847 us/op | 1.00 |
| runFastConfirmationRules vc:100000 bc:96 eq:0 | 4.9276 ms/op | 4.4471 ms/op | 1.11 |
| runFastConfirmationRules vc:600000 bc:96 eq:0 | 37.363 ms/op | 35.616 ms/op | 1.05 |
| runFastConfirmationRules vc:1000000 bc:96 eq:0 | 61.902 ms/op | 59.280 ms/op | 1.04 |
| runFastConfirmationRules vc:600000 bc:320 eq:0 | 38.099 ms/op | 35.643 ms/op | 1.07 |
| runFastConfirmationRules vc:100000 bc:96 eq:1000 | 1.4042 s/op | 1.2424 s/op | 1.13 |
| pass gossip attestations to forkchoice per slot | 2.6341 ms/op | 2.5867 ms/op | 1.02 |
| forkChoice updateHead vc 100000 bc 64 eq 0 | 420.40 us/op | 408.16 us/op | 1.03 |
| forkChoice updateHead vc 600000 bc 64 eq 0 | 2.5229 ms/op | 2.4250 ms/op | 1.04 |
| forkChoice updateHead vc 1000000 bc 64 eq 0 | 4.1022 ms/op | 4.0761 ms/op | 1.01 |
| forkChoice updateHead vc 600000 bc 320 eq 0 | 2.5135 ms/op | 2.4433 ms/op | 1.03 |
| forkChoice updateHead vc 600000 bc 1200 eq 0 | 2.5983 ms/op | 2.4974 ms/op | 1.04 |
| forkChoice updateHead vc 600000 bc 7200 eq 0 | 4.2362 ms/op | 3.4264 ms/op | 1.24 |
| forkChoice updateHead vc 600000 bc 64 eq 1000 | 2.4354 ms/op | 2.4801 ms/op | 0.98 |
| forkChoice updateHead vc 600000 bc 64 eq 10000 | 2.5440 ms/op | 2.5797 ms/op | 0.99 |
| forkChoice updateHead vc 600000 bc 64 eq 300000 | 6.7934 ms/op | 6.8921 ms/op | 0.99 |
| computeDeltas 1400000 validators 0% inactive | 12.175 ms/op | 12.387 ms/op | 0.98 |
| computeDeltas 1400000 validators 10% inactive | 11.577 ms/op | 11.612 ms/op | 1.00 |
| computeDeltas 1400000 validators 20% inactive | 10.922 ms/op | 11.002 ms/op | 0.99 |
| computeDeltas 1400000 validators 50% inactive | 8.9325 ms/op | 9.0199 ms/op | 0.99 |
| computeDeltas 2100000 validators 0% inactive | 18.165 ms/op | 18.585 ms/op | 0.98 |
| computeDeltas 2100000 validators 10% inactive | 17.452 ms/op | 17.457 ms/op | 1.00 |
| computeDeltas 2100000 validators 20% inactive | 16.149 ms/op | 16.626 ms/op | 0.97 |
| computeDeltas 2100000 validators 50% inactive | 10.781 ms/op | 11.003 ms/op | 0.98 |
| altair processAttestation - 250000 vs - 7PWei normalcase | 2.8637 ms/op | 2.5063 ms/op | 1.14 |
| altair processAttestation - 250000 vs - 7PWei worstcase | 3.8966 ms/op | 3.2823 ms/op | 1.19 |
| altair processAttestation - setStatus - 1/6 committees join | 122.40 us/op | 109.42 us/op | 1.12 |
| altair processAttestation - setStatus - 1/3 committees join | 224.90 us/op | 222.04 us/op | 1.01 |
| altair processAttestation - setStatus - 1/2 committees join | 335.15 us/op | 293.15 us/op | 1.14 |
| altair processAttestation - setStatus - 2/3 committees join | 392.90 us/op | 370.51 us/op | 1.06 |
| altair processAttestation - setStatus - 4/5 committees join | 557.95 us/op | 533.60 us/op | 1.05 |
| altair processAttestation - setStatus - 100% committees join | 656.10 us/op | 633.73 us/op | 1.04 |
| altair processBlock - 250000 vs - 7PWei normalcase | 4.2494 ms/op | 4.2202 ms/op | 1.01 |
| altair processBlock - 250000 vs - 7PWei normalcase hashState | 13.383 ms/op | 15.443 ms/op | 0.87 |
| altair processBlock - 250000 vs - 7PWei worstcase | 24.980 ms/op | 23.497 ms/op | 1.06 |
| altair processBlock - 250000 vs - 7PWei worstcase hashState | 48.879 ms/op | 48.111 ms/op | 1.02 |
| phase0 processBlock - 250000 vs - 7PWei normalcase | 1.4315 ms/op | 1.7747 ms/op | 0.81 |
| phase0 processBlock - 250000 vs - 7PWei worstcase | 20.085 ms/op | 20.666 ms/op | 0.97 |
| altair processEth1Data - 250000 vs - 7PWei normalcase | 301.76 us/op | 297.61 us/op | 1.01 |
| getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:16 | 3.4470 us/op | 3.6550 us/op | 0.94 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:220 | 20.839 us/op | 22.064 us/op | 0.94 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:43 | 6.6520 us/op | 7.4360 us/op | 0.89 |
| getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:19 | 4.3930 us/op | 4.6480 us/op | 0.95 |
| getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1021 | 95.434 us/op | 94.756 us/op | 1.01 |
| getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11778 | 1.4097 ms/op | 1.5336 ms/op | 0.92 |
| getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 | 1.8548 ms/op | 1.8805 ms/op | 0.99 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 | 1.8802 ms/op | 2.1043 ms/op | 0.89 |
| getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 | 3.7588 ms/op | 4.8168 ms/op | 0.78 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 | 2.1439 ms/op | 2.2977 ms/op | 0.93 |
| getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 | 5.3246 ms/op | 5.3602 ms/op | 0.99 |
| Tree 40 250000 create | 338.24 ms/op | 366.78 ms/op | 0.92 |
| Tree 40 250000 get(125000) | 96.590 ns/op | 94.699 ns/op | 1.02 |
| Tree 40 250000 set(125000) | 1.0319 us/op | 1.0410 us/op | 0.99 |
| Tree 40 250000 toArray() | 15.186 ms/op | 17.615 ms/op | 0.86 |
| Tree 40 250000 iterate all - toArray() + loop | 15.280 ms/op | 17.365 ms/op | 0.88 |
| Tree 40 250000 iterate all - get(i) | 40.740 ms/op | 41.743 ms/op | 0.98 |
| Array 250000 create | 2.1919 ms/op | 2.6666 ms/op | 0.82 |
| Array 250000 clone - spread | 669.36 us/op | 800.82 us/op | 0.84 |
| Array 250000 get(125000) | 0.30400 ns/op | 0.30400 ns/op | 1.00 |
| Array 250000 set(125000) | 0.30500 ns/op | 0.30500 ns/op | 1.00 |
| Array 250000 iterate all - loop | 58.825 us/op | 58.941 us/op | 1.00 |
| phase0 afterProcessEpoch - 250000 vs - 7PWei | 42.039 ms/op | 41.654 ms/op | 1.01 |
| Array.fill - length 1000000 | 2.3306 ms/op | 2.1171 ms/op | 1.10 |
| Array push - length 1000000 | 8.8181 ms/op | 10.640 ms/op | 0.83 |
| Array.get | 0.21250 ns/op | 0.21252 ns/op | 1.00 |
| Uint8Array.get | 0.23895 ns/op | 0.24944 ns/op | 0.96 |
| phase0 beforeProcessEpoch - 250000 vs - 7PWei | 22.510 ms/op | 25.629 ms/op | 0.88 |
| altair processEpoch - mainnet_e81889 | 357.60 ms/op | 288.76 ms/op | 1.24 |
| mainnet_e81889 - altair beforeProcessEpoch | 25.280 ms/op | 38.158 ms/op | 0.66 |
| mainnet_e81889 - altair processJustificationAndFinalization | 9.6720 us/op | 7.1910 us/op | 1.35 |
| mainnet_e81889 - altair processInactivityUpdates | 6.5077 ms/op | 7.6580 ms/op | 0.85 |
| mainnet_e81889 - altair processRewardsAndPenalties | 21.189 ms/op | 20.838 ms/op | 1.02 |
| mainnet_e81889 - altair processRegistryUpdates | 603.00 ns/op | 594.00 ns/op | 1.02 |
| mainnet_e81889 - altair processSlashings | 162.00 ns/op | 159.00 ns/op | 1.02 |
| mainnet_e81889 - altair processEth1DataReset | 165.00 ns/op | 154.00 ns/op | 1.07 |
| mainnet_e81889 - altair processEffectiveBalanceUpdates | 1.7740 ms/op | 6.1162 ms/op | 0.29 |
| mainnet_e81889 - altair processSlashingsReset | 724.00 ns/op | 735.00 ns/op | 0.99 |
| mainnet_e81889 - altair processRandaoMixesReset | 1.4520 us/op | 1.4390 us/op | 1.01 |
| mainnet_e81889 - altair processHistoricalRootsUpdate | 161.00 ns/op | 158.00 ns/op | 1.02 |
| mainnet_e81889 - altair processParticipationFlagUpdates | 489.00 ns/op | 487.00 ns/op | 1.00 |
| mainnet_e81889 - altair processSyncCommitteeUpdates | 133.00 ns/op | 131.00 ns/op | 1.02 |
| mainnet_e81889 - altair afterProcessEpoch | 43.818 ms/op | 42.946 ms/op | 1.02 |
| capella processEpoch - mainnet_e217614 | 1.2845 s/op | 947.24 ms/op | 1.36 |
| mainnet_e217614 - capella beforeProcessEpoch | 88.918 ms/op | 65.263 ms/op | 1.36 |
| mainnet_e217614 - capella processJustificationAndFinalization | 32.523 ms/op | 7.2910 us/op | 4460.74 |
| mainnet_e217614 - capella processInactivityUpdates | 14.781 ms/op | 18.037 ms/op | 0.82 |
| mainnet_e217614 - capella processRewardsAndPenalties | 101.96 ms/op | 102.63 ms/op | 0.99 |
| mainnet_e217614 - capella processRegistryUpdates | 4.5800 us/op | 4.6970 us/op | 0.98 |
| mainnet_e217614 - capella processSlashings | 144.00 ns/op | 158.00 ns/op | 0.91 |
| mainnet_e217614 - capella processEth1DataReset | 141.00 ns/op | 159.00 ns/op | 0.89 |
| mainnet_e217614 - capella processEffectiveBalanceUpdates | 19.427 ms/op | 20.347 ms/op | 0.95 |
| mainnet_e217614 - capella processSlashingsReset | 701.00 ns/op | 720.00 ns/op | 0.97 |
| mainnet_e217614 - capella processRandaoMixesReset | 1.6050 us/op | 1.4000 us/op | 1.15 |
| mainnet_e217614 - capella processHistoricalRootsUpdate | 141.00 ns/op | 154.00 ns/op | 0.92 |
| mainnet_e217614 - capella processParticipationFlagUpdates | 465.00 ns/op | 491.00 ns/op | 0.95 |
| mainnet_e217614 - capella afterProcessEpoch | 108.81 ms/op | 110.87 ms/op | 0.98 |
| phase0 processEpoch - mainnet_e58758 | 387.31 ms/op | 399.42 ms/op | 0.97 |
| mainnet_e58758 - phase0 beforeProcessEpoch | 80.006 ms/op | 91.114 ms/op | 0.88 |
| mainnet_e58758 - phase0 processJustificationAndFinalization | 7.9310 us/op | 7.3350 us/op | 1.08 |
| mainnet_e58758 - phase0 processRewardsAndPenalties | 18.138 ms/op | 19.248 ms/op | 0.94 |
| mainnet_e58758 - phase0 processRegistryUpdates | 2.3250 us/op | 2.4190 us/op | 0.96 |
| mainnet_e58758 - phase0 processSlashings | 144.00 ns/op | 154.00 ns/op | 0.94 |
| mainnet_e58758 - phase0 processEth1DataReset | 365.00 ns/op | 149.00 ns/op | 2.45 |
| mainnet_e58758 - phase0 processEffectiveBalanceUpdates | 831.27 us/op | 855.80 us/op | 0.97 |
| mainnet_e58758 - phase0 processSlashingsReset | 953.00 ns/op | 1.0100 us/op | 0.94 |
| mainnet_e58758 - phase0 processRandaoMixesReset | 1.7210 us/op | 1.6530 us/op | 1.04 |
| mainnet_e58758 - phase0 processHistoricalRootsUpdate | 145.00 ns/op | 175.00 ns/op | 0.83 |
| mainnet_e58758 - phase0 processParticipationRecordUpdates | 1.6730 us/op | 1.8450 us/op | 0.91 |
| mainnet_e58758 - phase0 afterProcessEpoch | 34.728 ms/op | 37.429 ms/op | 0.93 |
| phase0 processEffectiveBalanceUpdates - 250000 normalcase | 1.0328 ms/op | 1.0448 ms/op | 0.99 |
| phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 | 1.6750 ms/op | 1.6707 ms/op | 1.00 |
| altair processInactivityUpdates - 250000 normalcase | 13.755 ms/op | 13.379 ms/op | 1.03 |
| altair processInactivityUpdates - 250000 worstcase | 12.881 ms/op | 13.847 ms/op | 0.93 |
| phase0 processRegistryUpdates - 250000 normalcase | 4.2140 us/op | 2.4810 us/op | 1.70 |
| phase0 processRegistryUpdates - 250000 badcase_full_deposits | 241.10 us/op | 145.85 us/op | 1.65 |
| phase0 processRegistryUpdates - 250000 worstcase 0.5 | 98.247 ms/op | 63.431 ms/op | 1.55 |
| altair processRewardsAndPenalties - 250000 normalcase | 21.230 ms/op | 17.196 ms/op | 1.23 |
| altair processRewardsAndPenalties - 250000 worstcase | 17.924 ms/op | 17.169 ms/op | 1.04 |
| phase0 getAttestationDeltas - 250000 normalcase | 20.212 ms/op | 5.7549 ms/op | 3.51 |
| phase0 getAttestationDeltas - 250000 worstcase | 5.2904 ms/op | 5.7291 ms/op | 0.92 |
| phase0 processSlashings - 250000 worstcase | 77.035 us/op | 67.095 us/op | 1.15 |
| altair processSyncCommitteeUpdates - 250000 | 12.794 ms/op | 10.243 ms/op | 1.25 |
| BeaconState.hashTreeRoot - No change | 184.00 ns/op | 187.00 ns/op | 0.98 |
| BeaconState.hashTreeRoot - 1 full validator | 98.892 us/op | 79.966 us/op | 1.24 |
| BeaconState.hashTreeRoot - 32 full validator | 2.0790 ms/op | 821.64 us/op | 2.53 |
| BeaconState.hashTreeRoot - 512 full validator | 16.336 ms/op | 8.5153 ms/op | 1.92 |
| BeaconState.hashTreeRoot - 1 validator.effectiveBalance | 159.81 us/op | 109.54 us/op | 1.46 |
| BeaconState.hashTreeRoot - 32 validator.effectiveBalance | 3.1125 ms/op | 1.5105 ms/op | 2.06 |
| BeaconState.hashTreeRoot - 512 validator.effectiveBalance | 24.239 ms/op | 21.397 ms/op | 1.13 |
| BeaconState.hashTreeRoot - 1 balances | 136.96 us/op | 82.864 us/op | 1.65 |
| BeaconState.hashTreeRoot - 32 balances | 1.5785 ms/op | 756.15 us/op | 2.09 |
| BeaconState.hashTreeRoot - 512 balances | 7.1770 ms/op | 6.3482 ms/op | 1.13 |
| BeaconState.hashTreeRoot - 250000 balances | 207.30 ms/op | 112.19 ms/op | 1.85 |
| aggregationBits - 2048 els - zipIndexesInBitList | 21.980 us/op | 20.632 us/op | 1.07 |
| regular array get 100000 times | 23.573 us/op | 23.271 us/op | 1.01 |
| wrappedArray get 100000 times | 23.761 us/op | 23.384 us/op | 1.02 |
| arrayWithProxy get 100000 times | 9.8209 ms/op | 9.5841 ms/op | 1.02 |
| ssz.Root.equals | 22.282 ns/op | 21.688 ns/op | 1.03 |
| byteArrayEquals | 21.952 ns/op | 21.436 ns/op | 1.02 |
| Buffer.compare | 9.0690 ns/op | 8.8330 ns/op | 1.03 |
| processSlot - 1 slots | 10.824 us/op | 10.286 us/op | 1.05 |
| processSlot - 32 slots | 2.3850 ms/op | 2.1852 ms/op | 1.09 |
| getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei | 8.9043 ms/op | 4.0210 ms/op | 2.21 |
| getCommitteeAssignments - req 1 vs - 250000 vc | 1.7155 ms/op | 1.6574 ms/op | 1.04 |
| getCommitteeAssignments - req 100 vs - 250000 vc | 3.4920 ms/op | 3.4038 ms/op | 1.03 |
| getCommitteeAssignments - req 1000 vs - 250000 vc | 3.7742 ms/op | 3.6703 ms/op | 1.03 |
| findModifiedValidators - 10000 modified validators | 827.13 ms/op | 756.80 ms/op | 1.09 |
| findModifiedValidators - 1000 modified validators | 534.75 ms/op | 662.99 ms/op | 0.81 |
| findModifiedValidators - 100 modified validators | 326.44 ms/op | 357.93 ms/op | 0.91 |
| findModifiedValidators - 10 modified validators | 204.08 ms/op | 298.82 ms/op | 0.68 |
| findModifiedValidators - 1 modified validators | 237.75 ms/op | 257.37 ms/op | 0.92 |
| findModifiedValidators - no difference | 145.04 ms/op | 270.73 ms/op | 0.54 |
| migrate state 1500000 validators, 3400 modified, 2000 new | 4.4660 s/op | 3.2307 s/op | 1.38 |
| RootCache.getBlockRootAtSlot - 250000 vs - 7PWei | 3.8300 ns/op | 3.7500 ns/op | 1.02 |
| state getBlockRootAtSlot - 250000 vs - 7PWei | 487.91 ns/op | 407.44 ns/op | 1.20 |
| computeProposerIndex 100000 validators | 1.4000 ms/op | 1.3343 ms/op | 1.05 |
| getNextSyncCommitteeIndices 1000 validators | 2.9693 ms/op | 2.8841 ms/op | 1.03 |
| getNextSyncCommitteeIndices 10000 validators | 25.867 ms/op | 25.284 ms/op | 1.02 |
| getNextSyncCommitteeIndices 100000 validators | 91.604 ms/op | 83.057 ms/op | 1.10 |
| computeProposers - vc 250000 | 571.64 us/op | 543.80 us/op | 1.05 |
| computeEpochShuffling - vc 250000 | 41.069 ms/op | 38.872 ms/op | 1.06 |
| getNextSyncCommittee - vc 250000 | 10.147 ms/op | 9.5443 ms/op | 1.06 |
| nodejs block root to RootHex using toHex | 104.24 ns/op | 93.728 ns/op | 1.11 |
| nodejs block root to RootHex using toRootHex | 65.858 ns/op | 61.457 ns/op | 1.07 |
| nodejs fromHex(blob) | 1.1416 ms/op | 737.64 us/op | 1.55 |
| nodejs fromHexInto(blob) | 651.12 us/op | 628.91 us/op | 1.04 |
| nodejs block root to RootHex using the deprecated toHexString | 508.22 ns/op | 535.77 ns/op | 0.95 |
| nodejs byteArrayEquals 32 bytes (block root) | 26.819 ns/op | 26.096 ns/op | 1.03 |
| nodejs byteArrayEquals 48 bytes (pubkey) | 38.549 ns/op | 37.115 ns/op | 1.04 |
| nodejs byteArrayEquals 96 bytes (signature) | 36.835 ns/op | 33.594 ns/op | 1.10 |
| nodejs byteArrayEquals 1024 bytes | 44.052 ns/op | 41.517 ns/op | 1.06 |
| nodejs byteArrayEquals 131072 bytes (blob) | 1.8193 us/op | 1.7364 us/op | 1.05 |
| browser block root to RootHex using toHex | 148.55 ns/op | 142.62 ns/op | 1.04 |
| browser block root to RootHex using toRootHex | 136.65 ns/op | 166.17 ns/op | 0.82 |
| browser fromHex(blob) | 2.2144 ms/op | 1.5671 ms/op | 1.41 |
| browser fromHexInto(blob) | 658.84 us/op | 622.00 us/op | 1.06 |
| browser block root to RootHex using the deprecated toHexString | 361.19 ns/op | 365.22 ns/op | 0.99 |
| browser byteArrayEquals 32 bytes (block root) | 29.102 ns/op | 27.210 ns/op | 1.07 |
| browser byteArrayEquals 48 bytes (pubkey) | 40.955 ns/op | 38.437 ns/op | 1.07 |
| browser byteArrayEquals 96 bytes (signature) | 77.960 ns/op | 72.716 ns/op | 1.07 |
| browser byteArrayEquals 1024 bytes | 781.12 ns/op | 749.95 ns/op | 1.04 |
| browser byteArrayEquals 131072 bytes (blob) | 98.292 us/op | 94.714 us/op | 1.04 |
by benchmarkbot/action
## Description During finalized sync, blocks are imported in a tight loop without yielding to the event loop. When BLS verification is async (not awaiting the execution engine's `newPayload`), this prevents the checkpoint state cache's `processState()` cleanup from running, causing unbounded state accumulation and OOM on memory-constrained hosts. This adds `nextEventLoop()` after each `importBlock` to allow pending async cleanup (state serialization, cache eviction) to execute between block imports. ### Root Cause In `processBlocks()`, the import loop previously had a natural yield point when `importBlock` awaited the execution engine. With async BLS via `@chainsafe/blst` (PR #8900), `importBlock` no longer blocks on the EL call, so blocks blast through without yielding. The checkpoint state cache's `processState()` — which runs as fire-and-forget async — never gets a chance to serialize and evict old states, causing memory to climb until OOM. ### Fix ```typescript for (const fullyVerifiedBlock of fullyVerifiedBlocks) { await importBlock.call(this, fullyVerifiedBlock, opts); await nextEventLoop(); // Allow processState() cleanup to run } ``` `nextEventLoop()` is an existing utility (`sleep(0)`) that yields to the event loop's timers phase, giving pending microtasks and macrotasks a chance to execute. ### Testing Deployed as commit 983d923 on feat3 infrastructure nodes — resolved OOM crash loops on nodes with sufficient memory. Memory-constrained hosts (~16GB) may need additional tuning (reduced `maxBlockStates`, queue depth limits). Co-authored-by: Cayman <caymannava@gmail.com> --- > [!NOTE] > This PR was authored with AI assistance (Claude). All code has been reviewed and validated. Co-authored-by: lodekeeper <lodekeeper@users.noreply.github.com> Co-authored-by: Cayman <caymannava@gmail.com>
549bae6 to
9152e82
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9152e822a4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
b50b073 to
80473b8
Compare
blst with lodestar-z/blstblst and pubkeys with lodestar-z
Scope this PR down to the blst -> lodestar-z/blst swap per review feedback: - Restore StandardPubkeyCache (PubkeyIndexMap + PublicKey[]) in @lodestar/state-transition, now holding lodestar-z/blst PublicKey objects; one cache instance per registry, no process-global state - Drop all usage of the native @chainsafe/lodestar-z/pubkeys cache; its migration moves to a follow-up PR (see #9647) - Revert pubkeyCache.aggregate(indices) to aggregatePublicKeys(indices.map(getOrThrow)) - Re-export PubkeyCache/createPubkeyCache/syncPubkeys from @lodestar/state-transition (no breaking export change) - Remove native-cache-specific unit tests and global cache resets in tests/testUtils
…pto only" This reverts commit 2b6e03b.
| nodeLinker: isolated | ||
|
|
||
| catalog: | ||
| "@chainsafe/lodestar-z": "https://github.com/chainsafe/lodestar-z.git#2f5f281ccc6710630fcf0cb8e23c2eda3629974c" |
There was a problem hiding this comment.
this should be changed to an official release before merging
|
I'm changing this to consume only
|
|
also converting to draft first since this is not ready for proper re-review yet |
|
This PR is now BLS-only, #9728 is built on top of this + native pk cache
For standalone benchmarks, see this comment #9728 (comment) |
There was a problem hiding this comment.
💡 Codex Review
lodestar/.github/actions/setup-and-build/action.yml
Lines 32 to 36 in 6e5dc41
This adds Zig only to the shared setup action, but the release-tag workflow .github/workflows/docs-version.yml still runs pnpm install --frozen-lockfile directly at lines 40-50 without installing Zig. Since the newly allowed @chainsafe/lodestar-z build explicitly requires Zig, each release tag reaches its prepare script on a stock ubuntu-latest runner and fails before generating versioned docs; the direct install in .github/workflows/docs-backfill.yml has the same problem when processing a release containing this dependency.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Depends on ChainSafe/lodestar-z#306
Motivation
blstwith alodestar-zbindings toblst-zfor usage in the beacon node