Skip to content

feat(blst): use lodestar-z blst - #8900

Open
spiral-ladder wants to merge 141 commits into
unstablefrom
bing/blst-z
Open

feat(blst): use lodestar-z blst#8900
spiral-ladder wants to merge 141 commits into
unstablefrom
bing/blst-z

Conversation

@spiral-ladder

@spiral-ladder spiral-ladder commented Feb 13, 2026

Copy link
Copy Markdown
Member

Depends on ChainSafe/lodestar-z#306

Motivation

  • Replacing usages of blst with a lodestar-z bindings to blst-z for usage in the beacon node
  • Replacing usages of the pubkey cache with a global singleton native pubkey cache. We now completely remove the ts impl and only rely on the zig impl

@spiral-ladder spiral-ladder changed the title Bing/blst z wip: replace blst with blst-z where possible Feb 13, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @spiral-ladder, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request updates the beacon node's cryptographic backend by replacing the existing @chainsafe/blst library with new lodestar-z bindings to @chainsafe/blst-z. This change is intended to leverage an alternative or optimized implementation for BLS signature operations, which are critical for the aggregation of attestations and sync committee messages within the Ethereum consensus layer. The update involves modifying dependencies, adjusting function calls, and updating build configurations to support the new library.

Highlights

  • BLS Library Migration: Replaced usages of the @chainsafe/blst library with lodestar-z bindings to @chainsafe/blst-z across the beacon node for BLS cryptographic operations.
  • Dependency Updates: Added @chainsafe/blst-z as a local file dependency and @chainsafe/zapi to the project's yarn.lock file, reflecting the new BLS implementation.
  • Signature Aggregation Parameter Change: Modified calls to aggregateSignatures in several operational pools to include a false parameter, likely related to specific validation or optimization settings within the new BLS library.
  • Partial BLS Function Import: Maintained the import of aggregateVerify from the original @chainsafe/blst library in specific test files, while other BLS functions were migrated to @chainsafe/blst-z.
  • Build System Integration: Updated the .gitignore file to exclude zig-deps/, accommodating the new Zig-based dependencies.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • .gitignore
    • Added zig-deps/ to the ignore list.
  • packages/beacon-node/package.json
    • Introduced @chainsafe/blst-z as a local file dependency.
  • packages/beacon-node/src/chain/opPools/aggregatedAttestationPool.ts
    • Updated BLS import to @chainsafe/blst-z.
    • Modified aggregateSignatures calls to include a false parameter.
  • packages/beacon-node/src/chain/opPools/attestationPool.ts
    • Updated BLS import to @chainsafe/blst-z.
    • Modified aggregateSignatures calls to include a false parameter.
  • packages/beacon-node/src/chain/opPools/syncCommitteeMessagePool.ts
    • Updated BLS import to @chainsafe/blst-z.
    • Modified aggregateSignatureInto call to include a false parameter.
  • packages/beacon-node/src/chain/opPools/syncContributionAndProofPool.ts
    • Updated BLS import to @chainsafe/blst-z.
    • Modified aggregate function call to include a false parameter.
  • packages/beacon-node/src/chain/opPools/utils.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/e2e/api/impl/lightclient/endpoint.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/e2e/chain/bls/multithread.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/mocks/mockedBls.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/perf/bls/bls.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/spec/bls/bls.ts
    • Updated BLS imports, specifically re-importing aggregateVerify from @chainsafe/blst.
  • packages/beacon-node/test/spec/general/bls.ts
    • Updated BLS imports, specifically re-importing aggregateVerify from @chainsafe/blst.
  • packages/beacon-node/test/unit-minimal/chain/genesis/genesis.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/unit-minimal/chain/opPools/aggregatedAttestationPool.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/unit/chain/bls/bls.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/unit/chain/opPools/syncCommittee.test.ts
    • Updated BLS import to @chainsafe/bls-z.
  • packages/beacon-node/test/unit/chain/opPools/syncCommitteeContribution.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/unit/chain/validation/attestation/validateGossipAttestationsSameAttData.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/unit/chain/validation/blsToExecutionChange.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/unit/chain/validation/voluntaryExit.test.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/utils/cache.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/utils/node/validator.ts
    • Updated BLS import to @chainsafe/blst-z.
  • packages/beacon-node/test/utils/state.ts
    • Updated BLS import to @chainsafe/blst-z.
  • yarn.lock
    • Added new entries for @chainsafe/blst-z and @chainsafe/zapi dependencies.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@spiral-ladder spiral-ladder changed the title wip: replace blst with blst-z where possible feat(blst): replace blst with blst-z where possible Feb 13, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces the @chainsafe/blst dependency with @chainsafe/blst-z in most of the codebase. The changes are mostly import replacements, but also include an update to the aggregateSignatures function calls, adding a false argument to likely disable signature verification for performance. In two spec test files, aggregateVerify is still imported from the old @chainsafe/blst library, which seems intentional. My review includes suggestions to add comments explaining this mixed dependency for better maintainability.

Comment thread packages/beacon-node/test/spec/bls/bls.ts Outdated
Comment thread packages/beacon-node/test/spec/general/bls.ts Outdated
@spiral-ladder
spiral-ladder force-pushed the bing/blst-z branch 2 times, most recently from 5604001 to 59688b7 Compare February 13, 2026 14:57
@wemeetagain

Copy link
Copy Markdown
Member

You should be able to either pnpm link or use a url version of the dependency instead of vendoring the whole lodestar-z codebase.

@spiral-ladder
spiral-ladder force-pushed the bing/blst-z branch 4 times, most recently from ec6bb4d to 6c95d69 Compare February 14, 2026 02:46
@wemeetagain wemeetagain changed the title feat(blst): replace blst with blst-z where possible feat(blst): replace blst with lodestarz/blst Feb 16, 2026
@wemeetagain wemeetagain changed the title feat(blst): replace blst with lodestarz/blst feat(blst): replace blst with lodestar-z/blst Feb 17, 2026
@github-actions

github-actions Bot commented Feb 17, 2026

Copy link
Copy Markdown
Contributor

⚠️ Performance Alert ⚠️

Possible performance regression was detected for some benchmarks.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold.

Benchmark suite Current: 351f758 Previous: 713b218 Ratio
proposeBlockBody type=full, size=empty 3.7098 ms/op 705.63 us/op 5.26
phase0 getAttestationDeltas - 250000 normalcase 20.212 ms/op 5.7549 ms/op 3.51
Full benchmark results
Benchmark suite Current: 351f758 Previous: 713b218 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 881.16 us/op 1.3970 ms/op 0.63
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 41.127 us/op 40.437 us/op 1.02
BLS verify - blst 904.87 us/op 761.81 us/op 1.19
BLS verifyMultipleSignatures 3 - blst 1.3766 ms/op 1.3520 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst 2.2631 ms/op 2.1523 ms/op 1.05
BLS verifyMultipleSignatures 32 - blst 7.4355 ms/op 6.8202 ms/op 1.09
BLS verifyMultipleSignatures 64 - blst 13.984 ms/op 13.336 ms/op 1.05
BLS verifyMultipleSignatures 128 - blst 26.491 ms/op 25.281 ms/op 1.05
BLS deserializing 10000 signatures 647.81 ms/op 644.34 ms/op 1.01
BLS deserializing 100000 signatures 6.4482 s/op 6.5224 s/op 0.99
BLS verifyMultipleSignatures - same message - 3 - blst 948.72 us/op 795.30 us/op 1.19
BLS verifyMultipleSignatures - same message - 8 - blst 1.0774 ms/op 963.53 us/op 1.12
BLS verifyMultipleSignatures - same message - 32 - blst 1.6936 ms/op 1.5599 ms/op 1.09
BLS verifyMultipleSignatures - same message - 64 - blst 2.5090 ms/op 2.4625 ms/op 1.02
BLS verifyMultipleSignatures - same message - 128 - blst 4.1409 ms/op 4.0952 ms/op 1.01
BLS aggregatePubkeys 32 - blst 19.342 us/op 17.866 us/op 1.08
BLS aggregatePubkeys 128 - blst 68.767 us/op 63.509 us/op 1.08
getSlashingsAndExits - default max 50.335 us/op 51.498 us/op 0.98
getSlashingsAndExits - 2k 363.67 us/op 394.05 us/op 0.92
proposeBlockBody type=full, size=empty 3.7098 ms/op 705.63 us/op 5.26
isKnown best case - 1 super set check 163.00 ns/op 182.00 ns/op 0.90
isKnown normal case - 2 super set checks 164.00 ns/op 166.00 ns/op 0.99
isKnown worse case - 16 super set checks 165.00 ns/op 167.00 ns/op 0.99
validate api signedAggregateAndProof - struct 1.5163 ms/op 1.5007 ms/op 1.01
validate gossip signedAggregateAndProof - struct 1.5219 ms/op 1.4828 ms/op 1.03
batch validate gossip attestation - vc 640000 - chunk 32 111.50 us/op 109.65 us/op 1.02
batch validate gossip attestation - vc 640000 - chunk 64 98.002 us/op 100.51 us/op 0.98
batch validate gossip attestation - vc 640000 - chunk 128 90.160 us/op 95.316 us/op 0.95
batch validate gossip attestation - vc 640000 - chunk 256 88.530 us/op 91.341 us/op 0.97
bytes32 toHexString 299.00 ns/op 300.00 ns/op 1.00
bytes32 Buffer.toString(hex) 173.00 ns/op 168.00 ns/op 1.03
bytes32 Buffer.toString(hex) from Uint8Array 237.00 ns/op 239.00 ns/op 0.99
bytes32 Buffer.toString(hex) + 0x 165.00 ns/op 175.00 ns/op 0.94
Return object 10000 times 0.21140 ns/op 0.21620 ns/op 0.98
Throw Error 10000 times 3.1764 us/op 3.3616 us/op 0.94
toHex 106.92 ns/op 102.21 ns/op 1.05
Buffer.from 83.559 ns/op 92.518 ns/op 0.90
shared Buffer 56.098 ns/op 58.101 ns/op 0.97
fastMsgIdFn sha256 / 200 bytes 1.4220 us/op 1.4680 us/op 0.97
fastMsgIdFn h32 xxhash / 200 bytes 163.00 ns/op 153.00 ns/op 1.07
fastMsgIdFn h64 xxhash / 200 bytes 204.00 ns/op 203.00 ns/op 1.00
fastMsgIdFn sha256 / 1000 bytes 4.6090 us/op 4.7830 us/op 0.96
fastMsgIdFn h32 xxhash / 1000 bytes 254.00 ns/op 249.00 ns/op 1.02
fastMsgIdFn h64 xxhash / 1000 bytes 260.00 ns/op 255.00 ns/op 1.02
fastMsgIdFn sha256 / 10000 bytes 42.156 us/op 41.764 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 1.2880 us/op 1.2930 us/op 1.00
fastMsgIdFn h64 xxhash / 10000 bytes 840.00 ns/op 832.00 ns/op 1.01
send data - 1000 256B messages 4.2361 ms/op 4.9202 ms/op 0.86
send data - 1000 512B messages 5.8632 ms/op 7.4359 ms/op 0.79
send data - 1000 1024B messages 5.7120 ms/op 7.0452 ms/op 0.81
send data - 1000 1200B messages 6.9480 ms/op 6.9359 ms/op 1.00
send data - 1000 2048B messages 20.801 ms/op 11.859 ms/op 1.75
send data - 1000 4096B messages 77.695 ms/op 55.240 ms/op 1.41
send data - 1000 16384B messages 340.91 ms/op 234.86 ms/op 1.45
send data - 1000 65536B messages 1.7191 s/op 1.1756 s/op 1.46
enrSubnets - fastDeserialize 64 bits 714.00 ns/op 751.00 ns/op 0.95
enrSubnets - ssz BitVector 64 bits 256.00 ns/op 259.00 ns/op 0.99
enrSubnets - fastDeserialize 4 bits 100.00 ns/op 108.00 ns/op 0.93
enrSubnets - ssz BitVector 4 bits 261.00 ns/op 261.00 ns/op 1.00
prioritizePeers score -10:0 att 32-0.1 sync 2-0 202.17 us/op 199.00 us/op 1.02
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 233.68 us/op 230.30 us/op 1.01
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 328.25 us/op 338.10 us/op 0.97
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 607.99 us/op 595.23 us/op 1.02
prioritizePeers score 0:0 att 64-1 sync 4-1 695.19 us/op 701.85 us/op 0.99
array of 16000 items push then shift 1.2455 us/op 1.2852 us/op 0.97
LinkedList of 16000 items push then shift 7.8670 ns/op 7.6300 ns/op 1.03
array of 16000 items push then pop 67.100 ns/op 80.742 ns/op 0.83
LinkedList of 16000 items push then pop 6.1730 ns/op 6.1020 ns/op 1.01
array of 24000 items push then shift 1.8454 us/op 1.8883 us/op 0.98
LinkedList of 24000 items push then shift 9.1560 ns/op 7.2620 ns/op 1.26
array of 24000 items push then pop 101.43 ns/op 114.04 ns/op 0.89
LinkedList of 24000 items push then pop 6.8770 ns/op 6.1100 ns/op 1.13
intersect bitArray bitLen 8 3.8930 ns/op 3.9750 ns/op 0.98
intersect array and set length 8 29.972 ns/op 29.163 ns/op 1.03
intersect bitArray bitLen 128 23.337 ns/op 23.149 ns/op 1.01
intersect array and set length 128 510.24 ns/op 496.66 ns/op 1.03
bitArray.getTrueBitIndexes() bitLen 128 998.00 ns/op 963.00 ns/op 1.04
bitArray.getTrueBitIndexes() bitLen 248 1.7540 us/op 1.7080 us/op 1.03
bitArray.getTrueBitIndexes() bitLen 512 3.7990 us/op 3.5860 us/op 1.06
Full columns - reconstruct all 6 blobs 175.32 us/op 158.14 us/op 1.11
Full columns - reconstruct half of the blobs out of 6 86.634 us/op 103.47 us/op 0.84
Full columns - reconstruct single blob out of 6 34.126 us/op 32.903 us/op 1.04
Half columns - reconstruct all 6 blobs 415.25 ms/op 391.52 ms/op 1.06
Half columns - reconstruct half of the blobs out of 6 205.94 ms/op 197.90 ms/op 1.04
Half columns - reconstruct single blob out of 6 74.848 ms/op 70.924 ms/op 1.06
Set add up to 64 items then delete first 1.6951 us/op 1.6405 us/op 1.03
OrderedSet add up to 64 items then delete first 2.5634 us/op 2.5345 us/op 1.01
Set add up to 64 items then delete last 1.8892 us/op 1.8577 us/op 1.02
OrderedSet add up to 64 items then delete last 2.8257 us/op 3.0439 us/op 0.93
Set add up to 64 items then delete middle 1.8955 us/op 1.8815 us/op 1.01
OrderedSet add up to 64 items then delete middle 4.3165 us/op 4.5716 us/op 0.94
Set add up to 128 items then delete first 3.8102 us/op 3.6749 us/op 1.04
OrderedSet add up to 128 items then delete first 5.7808 us/op 5.7287 us/op 1.01
Set add up to 128 items then delete last 3.6273 us/op 3.6183 us/op 1.00
OrderedSet add up to 128 items then delete last 5.4738 us/op 6.2908 us/op 0.87
Set add up to 128 items then delete middle 3.7061 us/op 3.6457 us/op 1.02
OrderedSet add up to 128 items then delete middle 11.463 us/op 11.932 us/op 0.96
Set add up to 256 items then delete first 7.6856 us/op 7.3754 us/op 1.04
OrderedSet add up to 256 items then delete first 12.829 us/op 12.158 us/op 1.06
Set add up to 256 items then delete last 7.3212 us/op 7.6209 us/op 0.96
OrderedSet add up to 256 items then delete last 12.083 us/op 11.679 us/op 1.03
Set add up to 256 items then delete middle 7.9538 us/op 7.1280 us/op 1.12
OrderedSet add up to 256 items then delete middle 35.724 us/op 35.847 us/op 1.00
runFastConfirmationRules vc:100000 bc:96 eq:0 4.9276 ms/op 4.4471 ms/op 1.11
runFastConfirmationRules vc:600000 bc:96 eq:0 37.363 ms/op 35.616 ms/op 1.05
runFastConfirmationRules vc:1000000 bc:96 eq:0 61.902 ms/op 59.280 ms/op 1.04
runFastConfirmationRules vc:600000 bc:320 eq:0 38.099 ms/op 35.643 ms/op 1.07
runFastConfirmationRules vc:100000 bc:96 eq:1000 1.4042 s/op 1.2424 s/op 1.13
pass gossip attestations to forkchoice per slot 2.6341 ms/op 2.5867 ms/op 1.02
forkChoice updateHead vc 100000 bc 64 eq 0 420.40 us/op 408.16 us/op 1.03
forkChoice updateHead vc 600000 bc 64 eq 0 2.5229 ms/op 2.4250 ms/op 1.04
forkChoice updateHead vc 1000000 bc 64 eq 0 4.1022 ms/op 4.0761 ms/op 1.01
forkChoice updateHead vc 600000 bc 320 eq 0 2.5135 ms/op 2.4433 ms/op 1.03
forkChoice updateHead vc 600000 bc 1200 eq 0 2.5983 ms/op 2.4974 ms/op 1.04
forkChoice updateHead vc 600000 bc 7200 eq 0 4.2362 ms/op 3.4264 ms/op 1.24
forkChoice updateHead vc 600000 bc 64 eq 1000 2.4354 ms/op 2.4801 ms/op 0.98
forkChoice updateHead vc 600000 bc 64 eq 10000 2.5440 ms/op 2.5797 ms/op 0.99
forkChoice updateHead vc 600000 bc 64 eq 300000 6.7934 ms/op 6.8921 ms/op 0.99
computeDeltas 1400000 validators 0% inactive 12.175 ms/op 12.387 ms/op 0.98
computeDeltas 1400000 validators 10% inactive 11.577 ms/op 11.612 ms/op 1.00
computeDeltas 1400000 validators 20% inactive 10.922 ms/op 11.002 ms/op 0.99
computeDeltas 1400000 validators 50% inactive 8.9325 ms/op 9.0199 ms/op 0.99
computeDeltas 2100000 validators 0% inactive 18.165 ms/op 18.585 ms/op 0.98
computeDeltas 2100000 validators 10% inactive 17.452 ms/op 17.457 ms/op 1.00
computeDeltas 2100000 validators 20% inactive 16.149 ms/op 16.626 ms/op 0.97
computeDeltas 2100000 validators 50% inactive 10.781 ms/op 11.003 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 2.8637 ms/op 2.5063 ms/op 1.14
altair processAttestation - 250000 vs - 7PWei worstcase 3.8966 ms/op 3.2823 ms/op 1.19
altair processAttestation - setStatus - 1/6 committees join 122.40 us/op 109.42 us/op 1.12
altair processAttestation - setStatus - 1/3 committees join 224.90 us/op 222.04 us/op 1.01
altair processAttestation - setStatus - 1/2 committees join 335.15 us/op 293.15 us/op 1.14
altair processAttestation - setStatus - 2/3 committees join 392.90 us/op 370.51 us/op 1.06
altair processAttestation - setStatus - 4/5 committees join 557.95 us/op 533.60 us/op 1.05
altair processAttestation - setStatus - 100% committees join 656.10 us/op 633.73 us/op 1.04
altair processBlock - 250000 vs - 7PWei normalcase 4.2494 ms/op 4.2202 ms/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase hashState 13.383 ms/op 15.443 ms/op 0.87
altair processBlock - 250000 vs - 7PWei worstcase 24.980 ms/op 23.497 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase hashState 48.879 ms/op 48.111 ms/op 1.02
phase0 processBlock - 250000 vs - 7PWei normalcase 1.4315 ms/op 1.7747 ms/op 0.81
phase0 processBlock - 250000 vs - 7PWei worstcase 20.085 ms/op 20.666 ms/op 0.97
altair processEth1Data - 250000 vs - 7PWei normalcase 301.76 us/op 297.61 us/op 1.01
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:16 3.4470 us/op 3.6550 us/op 0.94
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:220 20.839 us/op 22.064 us/op 0.94
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:43 6.6520 us/op 7.4360 us/op 0.89
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:19 4.3930 us/op 4.6480 us/op 0.95
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1021 95.434 us/op 94.756 us/op 1.01
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11778 1.4097 ms/op 1.5336 ms/op 0.92
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.8548 ms/op 1.8805 ms/op 0.99
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.8802 ms/op 2.1043 ms/op 0.89
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.7588 ms/op 4.8168 ms/op 0.78
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.1439 ms/op 2.2977 ms/op 0.93
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 5.3246 ms/op 5.3602 ms/op 0.99
Tree 40 250000 create 338.24 ms/op 366.78 ms/op 0.92
Tree 40 250000 get(125000) 96.590 ns/op 94.699 ns/op 1.02
Tree 40 250000 set(125000) 1.0319 us/op 1.0410 us/op 0.99
Tree 40 250000 toArray() 15.186 ms/op 17.615 ms/op 0.86
Tree 40 250000 iterate all - toArray() + loop 15.280 ms/op 17.365 ms/op 0.88
Tree 40 250000 iterate all - get(i) 40.740 ms/op 41.743 ms/op 0.98
Array 250000 create 2.1919 ms/op 2.6666 ms/op 0.82
Array 250000 clone - spread 669.36 us/op 800.82 us/op 0.84
Array 250000 get(125000) 0.30400 ns/op 0.30400 ns/op 1.00
Array 250000 set(125000) 0.30500 ns/op 0.30500 ns/op 1.00
Array 250000 iterate all - loop 58.825 us/op 58.941 us/op 1.00
phase0 afterProcessEpoch - 250000 vs - 7PWei 42.039 ms/op 41.654 ms/op 1.01
Array.fill - length 1000000 2.3306 ms/op 2.1171 ms/op 1.10
Array push - length 1000000 8.8181 ms/op 10.640 ms/op 0.83
Array.get 0.21250 ns/op 0.21252 ns/op 1.00
Uint8Array.get 0.23895 ns/op 0.24944 ns/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 22.510 ms/op 25.629 ms/op 0.88
altair processEpoch - mainnet_e81889 357.60 ms/op 288.76 ms/op 1.24
mainnet_e81889 - altair beforeProcessEpoch 25.280 ms/op 38.158 ms/op 0.66
mainnet_e81889 - altair processJustificationAndFinalization 9.6720 us/op 7.1910 us/op 1.35
mainnet_e81889 - altair processInactivityUpdates 6.5077 ms/op 7.6580 ms/op 0.85
mainnet_e81889 - altair processRewardsAndPenalties 21.189 ms/op 20.838 ms/op 1.02
mainnet_e81889 - altair processRegistryUpdates 603.00 ns/op 594.00 ns/op 1.02
mainnet_e81889 - altair processSlashings 162.00 ns/op 159.00 ns/op 1.02
mainnet_e81889 - altair processEth1DataReset 165.00 ns/op 154.00 ns/op 1.07
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.7740 ms/op 6.1162 ms/op 0.29
mainnet_e81889 - altair processSlashingsReset 724.00 ns/op 735.00 ns/op 0.99
mainnet_e81889 - altair processRandaoMixesReset 1.4520 us/op 1.4390 us/op 1.01
mainnet_e81889 - altair processHistoricalRootsUpdate 161.00 ns/op 158.00 ns/op 1.02
mainnet_e81889 - altair processParticipationFlagUpdates 489.00 ns/op 487.00 ns/op 1.00
mainnet_e81889 - altair processSyncCommitteeUpdates 133.00 ns/op 131.00 ns/op 1.02
mainnet_e81889 - altair afterProcessEpoch 43.818 ms/op 42.946 ms/op 1.02
capella processEpoch - mainnet_e217614 1.2845 s/op 947.24 ms/op 1.36
mainnet_e217614 - capella beforeProcessEpoch 88.918 ms/op 65.263 ms/op 1.36
mainnet_e217614 - capella processJustificationAndFinalization 32.523 ms/op 7.2910 us/op 4460.74
mainnet_e217614 - capella processInactivityUpdates 14.781 ms/op 18.037 ms/op 0.82
mainnet_e217614 - capella processRewardsAndPenalties 101.96 ms/op 102.63 ms/op 0.99
mainnet_e217614 - capella processRegistryUpdates 4.5800 us/op 4.6970 us/op 0.98
mainnet_e217614 - capella processSlashings 144.00 ns/op 158.00 ns/op 0.91
mainnet_e217614 - capella processEth1DataReset 141.00 ns/op 159.00 ns/op 0.89
mainnet_e217614 - capella processEffectiveBalanceUpdates 19.427 ms/op 20.347 ms/op 0.95
mainnet_e217614 - capella processSlashingsReset 701.00 ns/op 720.00 ns/op 0.97
mainnet_e217614 - capella processRandaoMixesReset 1.6050 us/op 1.4000 us/op 1.15
mainnet_e217614 - capella processHistoricalRootsUpdate 141.00 ns/op 154.00 ns/op 0.92
mainnet_e217614 - capella processParticipationFlagUpdates 465.00 ns/op 491.00 ns/op 0.95
mainnet_e217614 - capella afterProcessEpoch 108.81 ms/op 110.87 ms/op 0.98
phase0 processEpoch - mainnet_e58758 387.31 ms/op 399.42 ms/op 0.97
mainnet_e58758 - phase0 beforeProcessEpoch 80.006 ms/op 91.114 ms/op 0.88
mainnet_e58758 - phase0 processJustificationAndFinalization 7.9310 us/op 7.3350 us/op 1.08
mainnet_e58758 - phase0 processRewardsAndPenalties 18.138 ms/op 19.248 ms/op 0.94
mainnet_e58758 - phase0 processRegistryUpdates 2.3250 us/op 2.4190 us/op 0.96
mainnet_e58758 - phase0 processSlashings 144.00 ns/op 154.00 ns/op 0.94
mainnet_e58758 - phase0 processEth1DataReset 365.00 ns/op 149.00 ns/op 2.45
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 831.27 us/op 855.80 us/op 0.97
mainnet_e58758 - phase0 processSlashingsReset 953.00 ns/op 1.0100 us/op 0.94
mainnet_e58758 - phase0 processRandaoMixesReset 1.7210 us/op 1.6530 us/op 1.04
mainnet_e58758 - phase0 processHistoricalRootsUpdate 145.00 ns/op 175.00 ns/op 0.83
mainnet_e58758 - phase0 processParticipationRecordUpdates 1.6730 us/op 1.8450 us/op 0.91
mainnet_e58758 - phase0 afterProcessEpoch 34.728 ms/op 37.429 ms/op 0.93
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.0328 ms/op 1.0448 ms/op 0.99
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.6750 ms/op 1.6707 ms/op 1.00
altair processInactivityUpdates - 250000 normalcase 13.755 ms/op 13.379 ms/op 1.03
altair processInactivityUpdates - 250000 worstcase 12.881 ms/op 13.847 ms/op 0.93
phase0 processRegistryUpdates - 250000 normalcase 4.2140 us/op 2.4810 us/op 1.70
phase0 processRegistryUpdates - 250000 badcase_full_deposits 241.10 us/op 145.85 us/op 1.65
phase0 processRegistryUpdates - 250000 worstcase 0.5 98.247 ms/op 63.431 ms/op 1.55
altair processRewardsAndPenalties - 250000 normalcase 21.230 ms/op 17.196 ms/op 1.23
altair processRewardsAndPenalties - 250000 worstcase 17.924 ms/op 17.169 ms/op 1.04
phase0 getAttestationDeltas - 250000 normalcase 20.212 ms/op 5.7549 ms/op 3.51
phase0 getAttestationDeltas - 250000 worstcase 5.2904 ms/op 5.7291 ms/op 0.92
phase0 processSlashings - 250000 worstcase 77.035 us/op 67.095 us/op 1.15
altair processSyncCommitteeUpdates - 250000 12.794 ms/op 10.243 ms/op 1.25
BeaconState.hashTreeRoot - No change 184.00 ns/op 187.00 ns/op 0.98
BeaconState.hashTreeRoot - 1 full validator 98.892 us/op 79.966 us/op 1.24
BeaconState.hashTreeRoot - 32 full validator 2.0790 ms/op 821.64 us/op 2.53
BeaconState.hashTreeRoot - 512 full validator 16.336 ms/op 8.5153 ms/op 1.92
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 159.81 us/op 109.54 us/op 1.46
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 3.1125 ms/op 1.5105 ms/op 2.06
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 24.239 ms/op 21.397 ms/op 1.13
BeaconState.hashTreeRoot - 1 balances 136.96 us/op 82.864 us/op 1.65
BeaconState.hashTreeRoot - 32 balances 1.5785 ms/op 756.15 us/op 2.09
BeaconState.hashTreeRoot - 512 balances 7.1770 ms/op 6.3482 ms/op 1.13
BeaconState.hashTreeRoot - 250000 balances 207.30 ms/op 112.19 ms/op 1.85
aggregationBits - 2048 els - zipIndexesInBitList 21.980 us/op 20.632 us/op 1.07
regular array get 100000 times 23.573 us/op 23.271 us/op 1.01
wrappedArray get 100000 times 23.761 us/op 23.384 us/op 1.02
arrayWithProxy get 100000 times 9.8209 ms/op 9.5841 ms/op 1.02
ssz.Root.equals 22.282 ns/op 21.688 ns/op 1.03
byteArrayEquals 21.952 ns/op 21.436 ns/op 1.02
Buffer.compare 9.0690 ns/op 8.8330 ns/op 1.03
processSlot - 1 slots 10.824 us/op 10.286 us/op 1.05
processSlot - 32 slots 2.3850 ms/op 2.1852 ms/op 1.09
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 8.9043 ms/op 4.0210 ms/op 2.21
getCommitteeAssignments - req 1 vs - 250000 vc 1.7155 ms/op 1.6574 ms/op 1.04
getCommitteeAssignments - req 100 vs - 250000 vc 3.4920 ms/op 3.4038 ms/op 1.03
getCommitteeAssignments - req 1000 vs - 250000 vc 3.7742 ms/op 3.6703 ms/op 1.03
findModifiedValidators - 10000 modified validators 827.13 ms/op 756.80 ms/op 1.09
findModifiedValidators - 1000 modified validators 534.75 ms/op 662.99 ms/op 0.81
findModifiedValidators - 100 modified validators 326.44 ms/op 357.93 ms/op 0.91
findModifiedValidators - 10 modified validators 204.08 ms/op 298.82 ms/op 0.68
findModifiedValidators - 1 modified validators 237.75 ms/op 257.37 ms/op 0.92
findModifiedValidators - no difference 145.04 ms/op 270.73 ms/op 0.54
migrate state 1500000 validators, 3400 modified, 2000 new 4.4660 s/op 3.2307 s/op 1.38
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 3.8300 ns/op 3.7500 ns/op 1.02
state getBlockRootAtSlot - 250000 vs - 7PWei 487.91 ns/op 407.44 ns/op 1.20
computeProposerIndex 100000 validators 1.4000 ms/op 1.3343 ms/op 1.05
getNextSyncCommitteeIndices 1000 validators 2.9693 ms/op 2.8841 ms/op 1.03
getNextSyncCommitteeIndices 10000 validators 25.867 ms/op 25.284 ms/op 1.02
getNextSyncCommitteeIndices 100000 validators 91.604 ms/op 83.057 ms/op 1.10
computeProposers - vc 250000 571.64 us/op 543.80 us/op 1.05
computeEpochShuffling - vc 250000 41.069 ms/op 38.872 ms/op 1.06
getNextSyncCommittee - vc 250000 10.147 ms/op 9.5443 ms/op 1.06
nodejs block root to RootHex using toHex 104.24 ns/op 93.728 ns/op 1.11
nodejs block root to RootHex using toRootHex 65.858 ns/op 61.457 ns/op 1.07
nodejs fromHex(blob) 1.1416 ms/op 737.64 us/op 1.55
nodejs fromHexInto(blob) 651.12 us/op 628.91 us/op 1.04
nodejs block root to RootHex using the deprecated toHexString 508.22 ns/op 535.77 ns/op 0.95
nodejs byteArrayEquals 32 bytes (block root) 26.819 ns/op 26.096 ns/op 1.03
nodejs byteArrayEquals 48 bytes (pubkey) 38.549 ns/op 37.115 ns/op 1.04
nodejs byteArrayEquals 96 bytes (signature) 36.835 ns/op 33.594 ns/op 1.10
nodejs byteArrayEquals 1024 bytes 44.052 ns/op 41.517 ns/op 1.06
nodejs byteArrayEquals 131072 bytes (blob) 1.8193 us/op 1.7364 us/op 1.05
browser block root to RootHex using toHex 148.55 ns/op 142.62 ns/op 1.04
browser block root to RootHex using toRootHex 136.65 ns/op 166.17 ns/op 0.82
browser fromHex(blob) 2.2144 ms/op 1.5671 ms/op 1.41
browser fromHexInto(blob) 658.84 us/op 622.00 us/op 1.06
browser block root to RootHex using the deprecated toHexString 361.19 ns/op 365.22 ns/op 0.99
browser byteArrayEquals 32 bytes (block root) 29.102 ns/op 27.210 ns/op 1.07
browser byteArrayEquals 48 bytes (pubkey) 40.955 ns/op 38.437 ns/op 1.07
browser byteArrayEquals 96 bytes (signature) 77.960 ns/op 72.716 ns/op 1.07
browser byteArrayEquals 1024 bytes 781.12 ns/op 749.95 ns/op 1.04
browser byteArrayEquals 131072 bytes (blob) 98.292 us/op 94.714 us/op 1.04

by benchmarkbot/action

nflaig pushed a commit that referenced this pull request Feb 17, 2026
## Description

During finalized sync, blocks are imported in a tight loop without
yielding to the event loop. When BLS verification is async (not awaiting
the execution engine's `newPayload`), this prevents the checkpoint state
cache's `processState()` cleanup from running, causing unbounded state
accumulation and OOM on memory-constrained hosts.

This adds `nextEventLoop()` after each `importBlock` to allow pending
async cleanup (state serialization, cache eviction) to execute between
block imports.

### Root Cause

In `processBlocks()`, the import loop previously had a natural yield
point when `importBlock` awaited the execution engine. With async BLS
via `@chainsafe/blst` (PR #8900), `importBlock` no longer blocks on the
EL call, so blocks blast through without yielding. The checkpoint state
cache's `processState()` — which runs as fire-and-forget async — never
gets a chance to serialize and evict old states, causing memory to climb
until OOM.

### Fix

```typescript
for (const fullyVerifiedBlock of fullyVerifiedBlocks) {
  await importBlock.call(this, fullyVerifiedBlock, opts);
  await nextEventLoop(); // Allow processState() cleanup to run
}
```

`nextEventLoop()` is an existing utility (`sleep(0)`) that yields to the
event loop's timers phase, giving pending microtasks and macrotasks a
chance to execute.

### Testing

Deployed as commit 983d923 on feat3 infrastructure nodes — resolved OOM
crash loops on nodes with sufficient memory. Memory-constrained hosts
(~16GB) may need additional tuning (reduced `maxBlockStates`, queue
depth limits).

Co-authored-by: Cayman <caymannava@gmail.com>

---
> [!NOTE]
> This PR was authored with AI assistance (Claude). All code has been
reviewed and validated.

Co-authored-by: lodekeeper <lodekeeper@users.noreply.github.com>
Co-authored-by: Cayman <caymannava@gmail.com>
@wemeetagain
wemeetagain marked this pull request as ready for review February 18, 2026 16:43
@wemeetagain
wemeetagain requested a review from a team as a code owner February 18, 2026 16:43

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9152e822a4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/test-bun.yml Outdated
Comment thread .github/workflows/benchmark.yml Outdated
Comment thread .github/workflows/test-bun.yml Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
Comment thread packages/beacon-node/test/utils/blocksAndData.ts Outdated
Comment thread packages/beacon-node/src/chain/bls/blsVerifier.ts Outdated
@wemeetagain wemeetagain changed the title feat(blst): replace blst with lodestar-z/blst feat(blst): replace blst and pubkeys with lodestar-z Feb 23, 2026
bing and others added 4 commits July 16, 2026 12:14
Scope this PR down to the blst -> lodestar-z/blst swap per review feedback:

- Restore StandardPubkeyCache (PubkeyIndexMap + PublicKey[]) in
  @lodestar/state-transition, now holding lodestar-z/blst PublicKey
  objects; one cache instance per registry, no process-global state
- Drop all usage of the native @chainsafe/lodestar-z/pubkeys cache;
  its migration moves to a follow-up PR (see #9647)
- Revert pubkeyCache.aggregate(indices) to
  aggregatePublicKeys(indices.map(getOrThrow))
- Re-export PubkeyCache/createPubkeyCache/syncPubkeys from
  @lodestar/state-transition (no breaking export change)
- Remove native-cache-specific unit tests and global cache resets in
  tests/testUtils
Comment thread pnpm-workspace.yaml Outdated
nodeLinker: isolated

catalog:
"@chainsafe/lodestar-z": "https://github.com/chainsafe/lodestar-z.git#2f5f281ccc6710630fcf0cb8e23c2eda3629974c"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be changed to an official release before merging

@spiral-ladder

spiral-ladder commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

had a botched unsigned commit 2b6e03b which i reverted in 626f9ab. Re-applied the same changes in a new commit d614a9e

@spiral-ladder

spiral-ladder commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

I'm changing this to consume only lodestar-z/blst. We had some discussions on whether we should ship bls or bls+pubkeys but didn't come to a conclusion, and given the following reasons i'm sticking to just bls changes for now, unless metrics show that this is a regression:

  1. native pk cache still has some unresolved concerns. Most importantly, the race condition previously discussed, amongst other issues: see Tracking: blst-z (lodestar-z BLS + pubkey migration) — release-candidate readiness #9647 for details
  2. it's better to introduce changes one at a time to spot for any potential regressions. Bundling them together makes this opaque, we're not sure if bls or the cache implementation regressed. Credits to @twoeths for this point
  3. we're missing pk cache benchmarks as noted by @twoeths

@spiral-ladder

spiral-ladder commented Jul 16, 2026

Copy link
Copy Markdown
Member Author

also converting to draft first since this is not ready for proper re-review yet

@spiral-ladder

Copy link
Copy Markdown
Member Author

This PR is now BLS-only, #9728 is built on top of this + native pk cache

  1. we're missing pk cache benchmarks as noted by @twoeths

For standalone benchmarks, see this comment #9728 (comment)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

# Required by @chainsafe/lodestar-z's `prepare` script in dev builds
- name: Setup Zig
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2.2.1
with:
version: 0.16.0

P2 Badge Install Zig in direct-install workflows

This adds Zig only to the shared setup action, but the release-tag workflow .github/workflows/docs-version.yml still runs pnpm install --frozen-lockfile directly at lines 40-50 without installing Zig. Since the newly allowed @chainsafe/lodestar-z build explicitly requires Zig, each release tag reaches its prepare script on a stock ubuntu-latest runner and fails before generating versioned docs; the direct install in .github/workflows/docs-backfill.yml has the same problem when processing a release containing this dependency.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

7 participants