refactor(blst): use lodestar-z pubkey cache by spiral-ladder · Pull Request #9728 · ChainSafe/lodestar

spiral-ladder · 2026-07-30T07:14:45Z

Motivation

Replaces the pk cache implementation with calls to the native centralized cache.

Description

Reverts 'refactor: restore TS pubkey cache, use lodestar-z for BLS crypto only' (d614a9e) and adapts call sites to the refactored native pubkey cache API from ChainSafe/lodestar-z#522

Sample output

Generating deterministic valid BLS public keys...
{
  node: 'v24.16.0',
  platform: 'linux-x64',
  cacheSize: 16384,
  benchmarkTime: 1000,
  warmupTime: 250
}
┌─────────┬──────────────┬──────────┬─────────────┐
│ (index) │ cache        │ totalMs  │ nsPerLookup │
├─────────┼──────────────┼──────────┼─────────────┤
│ 0       │ 'TypeScript' │ '2.042'  │ '124.6'     │
│ 1       │ 'Zig native' │ '22.418' │ '1368.3'    │
└─────────┴──────────────┴──────────┴─────────────┘
    ✔ BLS indexed getOrThrow - TypeScript                                  5263158 ops/s    190.0000 ns/op        -     815160 runs   1.00 s
    ✔ BLS indexed getOrThrow - Zig native                                  4255319 ops/s    235.0000 ns/op        -     791887 runs   1.00 s
    ✔ BLS indexed getOrThrow + toBytes - TypeScript                       474608.4 ops/s    2.107000 us/op        -     273294 runs   1.00 s
    ✔ BLS indexed getOrThrow + toBytes - Zig native                       458715.6 ops/s    2.180000 us/op        -     269953 runs   1.00 s
    ✔ BLS indexed getPubkeyBytes - Zig native                             554631.2 ops/s    1.803000 us/op        -     314441 runs   1.00 s
    ✔ BLS aggregatePublicKeys 1 - TypeScript cache                        469483.6 ops/s    2.130000 us/op        -     258249 runs   1.00 s
    ✔ BLS nativeCache.aggregate 1 - Zig native cache                       4761905 ops/s    210.0000 ns/op        -     845962 runs   1.00 s
    ✔ BLS aggregatePublicKeys 32 - TypeScript cache                       37808.61 ops/s    26.44900 us/op        -      27792 runs   1.00 s
    ✔ BLS nativeCache.aggregate 32 - Zig native cache                     46242.77 ops/s    21.62500 us/op        -      33270 runs   1.00 s
    ✔ BLS aggregatePublicKeys 128 - TypeScript cache                      10596.14 ops/s    94.37400 us/op        -       7898 runs   1.00 s
    ✔ BLS nativeCache.aggregate 128 - Zig native cache                    13808.53 ops/s    72.41900 us/op        -      10279 runs   1.00 s
    ✔ BLS aggregatePublicKeys 512 - TypeScript cache                      2708.214 ops/s    369.2470 us/op        -       2028 runs   1.00 s
    ✔ BLS nativeCache.aggregate 512 - Zig native cache                    3621.063 ops/s    276.1620 us/op        -       2709 runs   1.00 s
    ✔ BLS mixed 32 signature sets - TypeScript                            2088.485 ops/s    478.8160 us/op        -       1563 runs   1.00 s
    ✔ BLS mixed 32 signature sets - Zig native                            2664.492 ops/s    375.3060 us/op        -       1994 runs   1.00 s


  15 passing
  0 failed
  0 pending

AI Assistance Disclosure

codex-gpt-sol was used to generate this code

Reverts 'refactor: restore TS pubkey cache, use lodestar-z for BLS crypto only' (d614a9e) and adapts call sites to the refactored native pubkey cache API from lodestar-z cayman/refactor-pubkey-cache (PR #522): set() -> append(), load(filepath, maxCapacity).

spiral-ladder · 2026-08-05T08:03:20Z

https://github.com/spiral-ladder/pubkey-cache-benchmarks

This is a standalone personal repo benchmarking the original pk cache vs the new one, instructions to reproduce, results and more details in the README. TLDR: pubkey aggregation is about ~15% faster across different set sizes, lookups are slower (since lookups have to cross napi boundary)

cc @twoeths you might be interested in this

twoeths · 2026-08-10T08:31:31Z

https://github.com/spiral-ladder/pubkey-cache-benchmarks

This is a standalone personal repo benchmarking the original pk cache vs the new one, instructions to reproduce, results and more details in the README. TLDR: pubkey aggregation is about ~15% faster across different set sizes, lookups are slower (since lookups have to cross napi boundary)

cc @twoeths you might be interested in this

thanks for the statistic @spiral-ladder

there is a performance regression on getOrThrown() api (70ns with native pubkey cache vs 32ns on nodeJS pubkey cache) which is mainly used for SingleAttestation gossip validation. As validators are supposed to always submit SingleAttestation per epoch, this flow is quite heavy
I think we can write a new getPubkeyBytes() and use it instead to avoid that. The bls worker needs the Uint8Array pubkey, not the BLSPubkey.
- if we go with this direction, need a new benchmark to compare today's getOrThrow() + serialize() vs the new getPubkeyBytes()
on the benchmark:
- please run on one of our common infrastructure node, we may have a different result there
- given 1M active validators, we have 1M/32/64 ~ 500 members per committee, so need to also benchmark pubkey aggregation with this number (or 512)

Address review feedback from ChainSafe/lodestar#9728 (comment): - Aggregate sizes are now 1, 32, 128, and 512. Size 512 covers a full committee at 1M active validators (1M / 32 slots / 64 committees). - New indexed cases compare getOrThrow() + toBytes() with the native getPubkeyBytes() from lodestar-z d479f339, matching the BLS worker job path that consumes Uint8Array pubkeys. getPubkeyBytes had 26% lower latency than getOrThrow() + toBytes(). - Refresh README results from a new run on the same machine.

spiral-ladder · 2026-08-11T03:57:59Z

https://github.com/spiral-ladder/pubkey-cache-benchmarks
This is a standalone personal repo benchmarking the original pk cache vs the new one, instructions to reproduce, results and more details in the README. TLDR: pubkey aggregation is about ~15% faster across different set sizes, lookups are slower (since lookups have to cross napi boundary)
cc @twoeths you might be interested in this

thanks for the statistic @spiral-ladder

there is a performance regression on getOrThrown() api (70ns with native pubkey cache vs 32ns on nodeJS pubkey cache) which is mainly used for SingleAttestation gossip validation. As validators are supposed to always submit SingleAttestation per epoch, this flow is quite heavy

I think we can write a new getPubkeyBytes() and use it instead to avoid that. The bls worker needs the Uint8Array pubkey, not the BLSPubkey.

if we go with this direction, need a new benchmark to compare today's getOrThrow() + serialize() vs the new getPubkeyBytes()

on the benchmark:

please run on one of our common infrastructure node, we may have a different result there

given 1M active validators, we have 1M/32/64 ~ 500 members per committee, so need to also benchmark pubkey aggregation with this number (or 512)

updated the benchmarks with new sample output.

This uses:

getPubkeyBytes, a new API (PR here)
@ChainSafe/benchmark for the benchmarking
benchmarks 512 key aggregation

Expose the cached 48-byte compressed pubkey bytes by validator index without materializing a PublicKey wrapper or serializing in JS. The BLS worker job path in Lodestar needs Uint8Array pubkeys, so this avoids the getOrThrow() + toBytes() round trip on the SingleAttestation gossip validation hot path. Refs ChainSafe/lodestar#9728 (comment)

twoeths

I was supposed to post this asyncAggregateWithRandomness() improvement but @wemeetagain already implemented it here
bing/blst-pk-cache...cayman/blst-pk-cache

spiral-ladder · 2026-08-13T07:49:16Z

right now with same message verification, we do:

get pubkey object from native given validator index (when validating gossip attestation)

asyncAggregateWithRandomness(){pubkey, Uint8Array}[N]

we can improve asyncAggregateWithRandomness by:

not to get pubkey from native

asyncAggregateWithRandomness(){validatorIndex, Uint8Array}[N]

(maybe as a follow up, not in this PR)

i kinda think this might be small enough that we can make this part of this PR? I would be comfortable letting this soak for a few more days before next release if we change this

spiral-ladder · 2026-08-13T07:50:07Z

I guess one caveat is this would be mostly replaced by the new branch that @wemeetagain is working on

edit: lol u edited too

codecov · 2026-08-13T08:10:23Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 52.58%. Comparing base (4f2f64b) to head (dab776d).

Additional details and impacted files

@@               Coverage Diff               @@
##           bing/blst-z    #9728      +/-   ##
===============================================
- Coverage        52.58%   52.58%   -0.01%     
===============================================
  Files              848      848              
  Lines            59977    59975       -2     
  Branches          4418     4418              
===============================================
- Hits             31540    31538       -2     
  Misses           28378    28378              
  Partials            59       59

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dab776d058

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-13T08:14:22Z


  addPubkey(index: ValidatorIndex, pubkey: Uint8Array): void {
-    this.pubkeyCache.set(index, pubkey);
+    this.pubkeyCache.append(index, pubkey);


Preserve pubkeys for competing states

When two valid competing blocks extend the same pre-state with different deposits, each can assign a different pubkey to the same new validator index. The process-wide native cache is append-only and cannot remap an existing index, so after the first branch calls append, processing the second branch either fails or continues using the first branch's pubkey, causing valid signatures on the second branch to be rejected. The cache must support branch-specific mappings or avoid publishing unfinalized registry additions globally.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-13T08:14:22Z

+    const headroomEpochs = (90 * 24 * 60 * 60) / (config.SECONDS_PER_SLOT * SLOTS_PER_EPOCH);
+    const pubkeyCacheHeadroom = MAX_PENDING_DEPOSITS_PER_EPOCH * Math.ceil(headroomEpochs);
+    pubkeyCache.ensureCapacity(anchorState.validators.length + pubkeyCacheHeadroom);


Account for pre-Electra registry growth

For a node started before Electra, new validators can be added by up to MAX_DEPOSITS per slot, but this reservation assumes the post-Electra limit of MAX_PENDING_DEPOSITS_PER_EPOCH per epoch. At the maximum pre-Electra rate, the advertised 90-day headroom is exhausted in roughly 90 / SLOTS_PER_EPOCH days, after which append reallocates while BLS worker threads may be reading, despite the preceding comment identifying that operation as unsafe. Size the reservation according to the active fork or make growth synchronized.

AGENTS.md reference: AGENTS.md:L187-L197

Useful? React with 👍 / 👎.

asyncAggregateWithRandomness now resolves public keys from the native pubkey cache by validator index, so same-message signature sets carry {index, signature} and the per-attestation getOrThrow + PublicKey object crossing on gossip validation disappears.

spiral-ladder · 2026-08-14T09:31:37Z

my claude force pushed while i was working on something oops

spiral-ladder self-assigned this Jul 30, 2026

github-project-automation Bot added this to Lodestar Team Coordination Jul 30, 2026

spiral-ladder added 2 commits July 30, 2026 15:21

Merge branch 'bing/blst-z' into bing/blst-pk-cache

f6d4e30

test: reset native pubkey cache per unit file

9667784

matthewkeil moved this to In Progress in Lodestar Team Coordination Jul 30, 2026

matthewkeil added this to the mainnet-blst milestone Jul 30, 2026

spiral-ladder added 9 commits August 1, 2026 04:38

refactor(blst-z): use native pubkey cache sync

d21a10f

test: use valid validator registries with native cache

1eb93a5

test: generate unique validator pubkeys

7d5c714

test: adapt assertions for unique pubkeys

66d15ec

test: undo unnecessary refactor

35e11db

test: undo change generateValidators -> generateValidator

8382487

refactor: use native pubkey sync directly

6a106df

Merge branch 'bing/blst-z' into bing/blst-pk-cache

2631765

test: use native pubkey cache in cached state tests

c7bd622

spiral-ladder mentioned this pull request Aug 4, 2026

Tracking: blst-z (lodestar-z BLS + pubkey migration) — release-candidate readiness #9647

Open

22 tasks

spiral-ladder mentioned this pull request Aug 5, 2026

feat(blst): use lodestar-z blst #8900

Open

wemeetagain changed the title ~~refactor(blst-z): use native lodestar-z pubkey cache everywhere~~ refactor(blst-z): use lodestar-z pubkey cache Aug 6, 2026

wemeetagain changed the title ~~refactor(blst-z): use lodestar-z pubkey cache~~ refactor(blst): use lodestar-z pubkey cache Aug 6, 2026

spiral-ladder added 4 commits August 7, 2026 19:26

Merge branch 'bing/blst-z' into bing/blst-pk-cache

64d9b26

bump lodestar-z

587dadd

test: reuse interop pubkeys via pkix snapshots

56cacc2

fix: update pubkeyCacheWorker to pubkeyCache.syncPubkeys method API

54abef0

spiral-ladder mentioned this pull request Aug 11, 2026

feat(pubkeys): add getPubkeyBytes binding ChainSafe/lodestar-z#555

Merged

Merge branch 'bing/blst-z' into bing/blst-pk-cache

b08c1d5

spiral-ladder added 2 commits August 11, 2026 17:39

Merge branch 'bing/blst-z' into bing/blst-pk-cache

4a83477

fix: pubkeyCache usage in new files

dab776d

twoeths reviewed Aug 13, 2026

View reviewed changes

spiral-ladder marked this pull request as ready for review August 13, 2026 08:10

spiral-ladder requested a review from a team as a code owner August 13, 2026 08:10

chatgpt-codex-connector Bot reviewed Aug 13, 2026

View reviewed changes

twoeths approved these changes Aug 13, 2026

View reviewed changes

spiral-ladder force-pushed the bing/blst-pk-cache branch from cb43ace to dab776d Compare August 14, 2026 09:13

fix usage

4838a5e

Navigation Menu

Uh oh!

Uh oh!

refactor(blst): use lodestar-z pubkey cache - #9728

refactor(blst): use lodestar-z pubkey cache#9728
spiral-ladder wants to merge 21 commits into
bing/blst-zfrom
bing/blst-pk-cache

spiral-ladder commented Jul 30, 2026 •

edited

Loading

Uh oh!

spiral-ladder commented Aug 5, 2026

Uh oh!

twoeths commented Aug 10, 2026

Uh oh!

spiral-ladder commented Aug 11, 2026

Uh oh!

twoeths left a comment •

edited

Loading

Uh oh!

spiral-ladder commented Aug 13, 2026

Uh oh!

spiral-ladder commented Aug 13, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Aug 13, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Aug 13, 2026

Uh oh!

chatgpt-codex-connector Bot Aug 13, 2026

Uh oh!

spiral-ladder commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

spiral-ladder commented Jul 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

spiral-ladder commented Aug 5, 2026

Uh oh!

twoeths commented Aug 10, 2026

Uh oh!

spiral-ladder commented Aug 11, 2026

Uh oh!

twoeths left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

spiral-ladder commented Aug 13, 2026

Uh oh!

spiral-ladder commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Aug 13, 2026

Codecov Report

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

spiral-ladder commented Aug 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

spiral-ladder commented Jul 30, 2026 •

edited

Loading

twoeths left a comment •

edited

Loading

spiral-ladder commented Aug 13, 2026 •

edited

Loading