fix: skip sync committee participation while node is optimistic by lodekeeper · Pull Request #9752 · ChainSafe/lodestar

lodekeeper · 2026-08-03T09:27:36Z

Motivation

While debugging a lodestar-besu node on glamsterdam-devnet-7 that was following the chain optimistically (CL at head, EL still backfilling), the validator client was observed publishing SyncCommitteeMessages even though /eth/v1/node/syncing reported is_optimistic=true:

info: Published SyncCommitteeMessage slot=143118, count=17

The optimistic sync spec says an optimistic validator MUST NOT participate in sync committees:

An optimistic validator MUST NOT participate in sync committees (i.e., sign across the DOMAIN_SYNC_COMMITTEE, DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF or DOMAIN_CONTRIBUTION_AND_PROOF domains).

Attestations (produceAttestationData) are already gated on optimistic execution status in the beacon-node API. Sync committee participation is asymmetric: the base sync committee message is built in the validator client from the head root (ChainHeaderTracker / getBlockRoot) and submitted directly, so the validator client must honor the beacon node's optimistic status before signing that root.

Description

Expose the node's optimistic status via SyncingStatusTracker.isNodeOptimistic(), derived from the last successful /eth/v1/node/syncing poll the tracker already performs every slot. Returns undefined when the status is unknown, so callers don't over-suppress.
In SyncCommitteeService.runSyncCommitteeTasks, skip sync committee duties before fetching duties when the node is already optimistic, so DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF is not signed while optimistic.
After waiting for the block, recheck the actual head/root and skip DOMAIN_SYNC_COMMITTEE signing if that head is optimistic.
Keep sync committee contribution aggregation/signing scoped to the same already-validated beaconBlockRoot; the beacon-node produceSyncCommitteeContribution endpoint remains responsible for rejecting optimistic block roots before the VC signs DOMAIN_CONTRIBUTION_AND_PROOF.
Replace the now-resolved TODO/PENDING comment in validator/index.ts with a pointer to the new VC-side gate.

Steps to test

Added unit coverage in packages/validator/test/unit/services/syncCommittee.test.ts asserting that optimistic status suppresses selection-proof signing and sync committee message signing, while contribution aggregation still uses the previously validated block root. Existing non-optimistic sync committee behaviour is unchanged.

An optimistic validator must not sign sync committee messages, since it has not fully verified the execution payload of the head block it would sign over. Attestations and sync committee contributions are already gated on optimistic execution status, but the base sync committee message — built in the validator client from the head root, with no beacon-node produce endpoint to gate — was not. Expose the node's optimistic status from SyncingStatusTracker and skip sync committee duties in SyncCommitteeService when the node is optimistic. Spec: https://github.com/ethereum/consensus-specs/blob/v1.6.1/sync/optimistic.md#participating-in-sync-committees 🤖 Generated with AI assistance Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

nflaig · 2026-08-03T09:29:37Z

+      // An optimistic validator MUST NOT participate in sync committees, since it has not
+      // fully verified the execution payload of the head block it would sign over.
+      // https://github.com/ethereum/consensus-specs/blob/v1.6.1/sync/optimistic.md#participating-in-sync-committees
+      if (this.syncingStatusTracker.isNodeOptimistic() === true) {


do we need to modify the validator client for this? are we doing this for attestations too?

Double-checked the attestation path — the gate is entirely BN-side: produceAttestationData → notOnOptimisticBlockRoot(targetRoot) throws NodeIsSyncing (503), and the VC just surfaces that error (no VC-side optimistic check). That works because attestations must fetch their data from a BN produce endpoint, so there's a natural place to 503.

Sync committee is asymmetric:

Contributions already have that chokepoint — produceSyncCommitteeContribution calls the same notOnOptimisticBlockRoot and 503s. Already handled BN-side, no VC change needed.

The base SyncCommitteeMessage has no produce endpoint — the VC builds it itself from the head root (ChainHeaderTracker.getCurrentChainHead / getBlockRoot) and only calls the BN to submit (POST /pool/sync_committees). There's no produce call in that path to return 503 from. A 503 at submit-time is after we've already signed, and would also reject relayed messages; adding a dedicated produce endpoint just to gate feels heavier than the check itself.

So unless you'd rather add a produce endpoint, the base message has to be gated VC-side — though note the BN already reports the head's optimistic status (getBlockRoot's execution_optimistic meta + the head SSE event), so it's really "make the VC honor what the BN already tells it," not new consensus logic.

That said, you're right that the current spot is wrong, and Codex caught two real bugs with it:

Selection proofs (DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF) are signed inside getDutiesAtSlot before this gate, so an optimistic validator still signs them.

Node-level is_optimistic from the periodic poll isn't the head actually being signed (can flip optimistic during the block wait).

I'll revise to gate per-head-root (checked after the block wait) and cover selection-proof signing too, dropping the node-level check. Good to keep it VC-side, or would you prefer a BN produce endpoint?

Done in bfeee38. I kept this VC-side because the base SyncCommitteeMessage path still has no BN produce endpoint: the VC now checks the cached node optimistic state before getDutiesAtSlot so selection proofs are not signed while the node is already optimistic, and it rechecks the actual head/root after waitForBlockSlot before signing or publishing messages/contributions. Attestations and sync committee contributions remain BN-gated through their produce endpoints.

nflaig

@lodekeeper not sure why we fix this on vc side, isn't it better to have the beacon node return 503? please double check how we do this for attestations

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6cdb222e51

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-08-03T09:31:15Z

+      // An optimistic validator MUST NOT participate in sync committees, since it has not
+      // fully verified the execution payload of the head block it would sign over.
+      // https://github.com/ethereum/consensus-specs/blob/v1.6.1/sync/optimistic.md#participating-in-sync-committees
+      if (this.syncingStatusTracker.isNodeOptimistic() === true) {


Gate optimism before signing selection proofs

When isNodeOptimistic() is true, this guard runs only after getDutiesAtSlot(slot) has already called signSyncCommitteeSelectionProof for each subnet, so an optimistic validator still signs over DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF. That violates the same optimistic-sync rule this check is meant to enforce, and in distributed aggregation mode those partial proofs can also be submitted before this return is reached. Move the optimism gate ahead of duty proof generation, or make duty lookup avoid producing selection proofs while optimistic.

Useful? React with 👍 / 👎.

Confirmed — getDutiesAtSlot → getSelectionProofs signs over DOMAIN_SYNC_COMMITTEE_SELECTION_PROOF (and, in distributed-aggregation mode, submits the partial selections) before this gate runs, so an optimistic validator still signs the selection proofs. Good catch. Will move the optimistic gate ahead of duty/selection-proof generation so nothing is signed while optimistic.

Fixed in bfeee38. The optimistic-node check now runs before getDutiesAtSlot, so we do not sign sync committee selection proofs or submit distributed selections when the node is already known optimistic.

chatgpt-codex-connector · 2026-08-03T09:31:15Z

+      // An optimistic validator MUST NOT participate in sync committees, since it has not
+      // fully verified the execution payload of the head block it would sign over.
+      // https://github.com/ethereum/consensus-specs/blob/v1.6.1/sync/optimistic.md#participating-in-sync-committees
+      if (this.syncingStatusTracker.isNodeOptimistic() === true) {


Recheck optimism after waiting for the block

If /eth/v1/node/syncing was non-optimistic at slot start but the node imports the slot block optimistically before waitForBlockSlot(slot) resolves, this early check still allows produceAndPublishSyncCommittees to sign and publish the new optimistic head from ChainHeaderTracker. The optimistic status needs to be checked against the head that is actually about to be signed, or rechecked after the block wait and before signing.

Useful? React with 👍 / 👎.

Valid — the node-level is_optimistic from the periodic /eth/v1/node/syncing poll isn't necessarily the head that gets signed after waitForBlockSlot, so there's a window where a freshly-imported optimistic head slips through. Will switch to checking the optimistic status of the actual head root being signed, after the block wait, rather than the early node-level check.

Fixed in bfeee38. After waitForBlockSlot, the VC resolves the head/root it is about to sign and skips the sync committee message/contribution flow if that head is optimistic, closing the slot-start status race.

nflaig

in principle this seems correct to me, we should not cast these messages while the EL is syncing/optimistic, I haven't reviewed the code in detail

wemeetagain

This makes sense to me.
This can't be done on the BN side since there is no data beyond a block root needed to sign over that can be gated by the BN.

lodekeeper · 2026-08-04T20:50:18Z

Thanks, acknowledged. This resolves the BN-vs-VC placement question for me: the base SyncCommitteeMessage path only has the block root to sign and no BN produce endpoint where optimism can be rejected before signing, so the VC-side gate is the right place for that path.

The follow-up in bfeee38fc5 also handles the two signing-order issues from the inline threads: it gates before getDutiesAtSlot so selection proofs are not signed while already optimistic, and it rechecks the actual head/root after waitForBlockSlot before signing/publishing. No further code change from this approval.

spiral-ladder

one Q

spiral-ladder · 2026-08-05T09:44:18Z

      // Step 2. If an attestation was produced, make an aggregate.
      // First, wait until the `CONTRIBUTION_DUE_BPS` of the slot
      const syncContributionDueMs = this.config.getSyncContributionDueMs(fork);
      await sleep(syncContributionDueMs - this.clock.msFromSlot(slot), signal);


@lodekeeper shouldn't we also skip signing DOMAIN_CONTRIBUTION_AND_PROOF like done in produceAndPublishAggregates if an optimistic head arrives during this period?

Good catch, yes. produceSyncCommitteeContribution gates the contribution block root, but it does not cover the case where a new optimistic head arrives during the wait between DOMAIN_SYNC_COMMITTEE signing and DOMAIN_CONTRIBUTION_AND_PROOF signing.

Fixed in 75383738c6: after the contribution wait, the VC now resolves the current head again and returns before producing/signing contributions if that head is optimistic. I also added a unit test for that race and updated the PR body so the contribution-and-proof path is explicitly covered.

nflaig · 2026-08-05T11:55:58Z

+      const contributionHead = await this.getHeadForSyncCommitteeMessage(slot);
+      if (contributionHead.executionOptimistic) {
+        this.logger.debug("Skipping sync committee contributions while head is optimistic", {slot});
+        return;
+      }


this is about aggregated messages isn't it? so not sure it's needed here, we can still aggregate even if node is optimistic, what we want to avoid is signing messages that we haven't validated properly ourselves due to node being optimistic

@lodekeeper can you check how other clients handle this

Checked other clients and I agree with this distinction.

Lighthouse fetches a non-optimistic head root once, signs sync committee messages for that root, then later produces/signs aggregates for the same root. It does not recheck the current head before aggregate signing.

Nimbus has the same shape: getHeadBlockRoot rejects optimistic responses before message signing, then contribution production uses that saved root after the contribution delay.

Teku also carries lastSignatureBlockRoot from the message duty into the aggregation duty.

Prysm is a bit stricter in its REST VC path because it asks for the current head root when producing the contribution, and that helper rejects optimistic heads, but its BN produce endpoint also rejects optimistic mode before returning contribution data.

So the extra check I added was over-broad for Lodestar's flow: if we already signed DOMAIN_SYNC_COMMITTEE for a non-optimistic beaconBlockRoot, we can still ask the BN to aggregate for that same root even if the current head later becomes optimistic. The BN produceSyncCommitteeContribution path remains the right place to reject an optimistic requested root before the VC signs DOMAIN_CONTRIBUTION_AND_PROOF.

Adjusted in 43907237ef: removed the second current-head check before contribution production, changed the unit test to assert aggregation still uses the previously validated root, and updated the PR body.

thanks for the correction @nflaig, seems right

wemeetagain · 2026-08-12T08:33:31Z

🎉 This PR is included in v1.46.0 🎉

lodekeeper requested a review from a team as a code owner August 3, 2026 09:27

github-project-automation Bot added this to Lodestar Team Coordination Aug 3, 2026

nflaig reviewed Aug 3, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Aug 3, 2026

View reviewed changes

fix: gate sync committee signing on optimistic head

bfeee38

nflaig reviewed Aug 3, 2026

View reviewed changes

wemeetagain previously approved these changes Aug 4, 2026

View reviewed changes

spiral-ladder reviewed Aug 5, 2026

View reviewed changes

fix: recheck optimism before sync contributions

7538373

lodekeeper dismissed wemeetagain’s stale review via 7538373 August 5, 2026 09:58

Merge branch 'unstable' into fix/optimistic-sync-committee

0eae928

spiral-ladder previously approved these changes Aug 5, 2026

View reviewed changes

nflaig reviewed Aug 5, 2026

View reviewed changes

fix: keep sync contributions on validated root

4390723

lodekeeper dismissed spiral-ladder’s stale review via 4390723 August 5, 2026 12:11

nflaig approved these changes Aug 5, 2026

View reviewed changes

spiral-ladder approved these changes Aug 5, 2026

View reviewed changes

nflaig merged commit 5e77d15 into ChainSafe:unstable Aug 5, 2026
24 of 25 checks passed

github-project-automation Bot moved this to Done in Lodestar Team Coordination Aug 5, 2026

Uh oh!

Uh oh!

Conversation

lodekeeper commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Description

Steps to test

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nflaig left a comment

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nflaig left a comment

Choose a reason for hiding this comment

Uh oh!

wemeetagain left a comment

Choose a reason for hiding this comment

Uh oh!

lodekeeper commented Aug 4, 2026

Uh oh!

spiral-ladder left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

wemeetagain commented Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

lodekeeper commented Aug 3, 2026 •

edited

Loading