Skip to content

feat(consume): Consume tests with runloop monad#32

Merged
pdobacz merged 22 commits into
forks/monad_ninefrom
execute-with-eestnet
Jul 17, 2026
Merged

feat(consume): Consume tests with runloop monad#32
pdobacz merged 22 commits into
forks/monad_ninefrom
execute-with-eestnet

Conversation

@pdobacz

@pdobacz pdobacz commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Adds a consume direct test consumer enabling execution of spec test fixtures against a more production-like environment of monad execution client.

Currently there are 3 main use cases for this setup:

  1. Run all spec tests in a slower but more production-like environment (than monad-ethereum-test executable)
  2. Enable running fork transition tests (for MIP-8 specifically, but generally too)
  3. Enable running preliminary performance regression tests for MIP-8

Requires https://github.com/monad-exp/monad-eest-rust-harness/pull/1

Greptile Summary

This PR wires a new MonadFixtureConsumer (consume direct) into the EEST framework so blockchain fixtures can be executed against the production monad runloop via the eest-runner harness. It also adds a --monad-runloop filler flag that stamps consensus-derived header fields (prev_randao, gas_limit, extra_data, requests_hash) so filled block hashes and EIP-2935 history storage match what the runloop produces.

  • MonadFixtureConsumer (clis/monad.py): digests fixtures into an eest-runner input document, provisions a fresh triedb (slot-encoded, page-encoded, or dual-timeline for transition forks), spawns the harness with container-lifecycle cleanup via PR_SET_PDEATHSIG + docker kill, and compares every post-state account and the final state root against the fixture.
  • --monad-runloop filler option: stamps monad-specific header fields during fill and propagates a monad_runloop marker so tests that rely on a test-controlled gas limit (e.g. test_tx_gas_limit) can skip themselves gracefully.
  • Fork cleanup: removes the unused PragueToMONAD_EIGHTAtTime15k transition fork and narrows the mip4 test to only the MONAD_NINE transition boundary.

Confidence Score: 5/5

Safe to merge; the new consumer, filler option, and workflow are self-contained and don't touch shared production paths.

The core MonadFixtureConsumer logic is carefully scoped: it only activates on MONAD_EIGHT+ fixture networks, cleans up Docker containers on every exit path, and compares results field-by-field. The --monad-runloop flag is opt-in and isolated from existing fill flows. The one non-blocking concern (no per-test harness timeout) is a reliability nit with no correctness impact.

packages/testing/src/execution_testing/client_clis/clis/monad.py — the communicate() call in _run_harness has no per-invocation timeout.

Important Files Changed

Filename Overview
packages/testing/src/execution_testing/client_clis/clis/monad.py New MonadFixtureConsumer: digests blockchain fixtures into eest-runner input, provisions triedb (with dual-timeline for transition forks), runs the harness in a subprocess with container-lifecycle cleanup, and compares post-state and state-root. Logic is correct; communicate() has no per-invocation timeout.
.github/workflows/consume_release.yaml New workflow for consuming a fixtures release against a pinned monad stack; all user inputs are threaded through env vars rather than inline shell interpolation, and optional submodule overrides are handled via actions/checkout.
packages/testing/src/execution_testing/test_types/block_types.py Adds MonadRunloopDefaults dataclass (consistent with EnvironmentDefaults pattern) with consensus-derived header constants: prev_randao (baked BLS-signature hash), gas_limit, and an enabled flag set at configure time.
packages/testing/src/execution_testing/specs/blockchain.py Stamps monad blocks with runloop-conformant header fields (prev_randao, gas_limit, extra_data, requests_hash) when --monad-runloop is active; correctly resolves per-block fork so transition fixtures are covered.
packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py Adds --monad-runloop pytest option and corresponding monad_runloop marker propagation in collection; also gates gas_limit override on the new option.
packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/releases.py Adds monad-developers/execution-specs to SUPPORTED_REPOS and strips the tests- prefix from Monad release tag names when deriving the fixture asset filename; backward-compatible for Ethereum repos.
packages/testing/src/execution_testing/forks/forks/transition.py Removes the unused PragueToMONAD_EIGHTAtTime15k transition fork; the test_p256verify_before_fork.py comment explains the downstream impact.
tests/frontier/validation/test_transaction.py Adds a monad_runloop skip marker to test_tx_gas_limit because the monad runloop fixes every block to the proposal gas limit, making the over-limit tx scenario untestable.
tests/monad_nine/mip4_checkreservebalance/test_fork_transition.py Removes subsequent_forks=True from valid_at_transition_to marker, scoping the test only to the exact MONAD_NINE transition rather than all subsequent forks.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant pytest
    participant MonadFixtureConsumer
    participant monad_mpt
    participant eest_runner
    participant Docker

    pytest->>MonadFixtureConsumer: consume_fixture(fixture_path)
    MonadFixtureConsumer->>MonadFixtureConsumer: load + validate fixture JSON
    MonadFixtureConsumer->>MonadFixtureConsumer: skip if any expectException block
    MonadFixtureConsumer->>monad_mpt: --create (slot/page/dual triedb)
    alt transition fixture (slot+page)
        MonadFixtureConsumer->>monad_mpt: --activate-secondary
    end
    MonadFixtureConsumer->>MonadFixtureConsumer: write input.json (genesis alloc + blocks)
    MonadFixtureConsumer->>eest_runner: Popen(--input, --output, --ledger-dir, --db)
    eest_runner->>Docker: start runloop container (EEST_RUNNER_CONTAINER_NAME)
    eest_runner->>Docker: propose + finalize blocks via ledger sim
    Docker-->>eest_runner: execution results
    eest_runner-->>MonadFixtureConsumer: output.json (post_state, state_root)
    MonadFixtureConsumer->>MonadFixtureConsumer: compare post_state accounts
    MonadFixtureConsumer->>MonadFixtureConsumer: compare state_root vs last blockHeader
    MonadFixtureConsumer-->>pytest: pass / raise Exception(mismatches)
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant pytest
    participant MonadFixtureConsumer
    participant monad_mpt
    participant eest_runner
    participant Docker

    pytest->>MonadFixtureConsumer: consume_fixture(fixture_path)
    MonadFixtureConsumer->>MonadFixtureConsumer: load + validate fixture JSON
    MonadFixtureConsumer->>MonadFixtureConsumer: skip if any expectException block
    MonadFixtureConsumer->>monad_mpt: --create (slot/page/dual triedb)
    alt transition fixture (slot+page)
        MonadFixtureConsumer->>monad_mpt: --activate-secondary
    end
    MonadFixtureConsumer->>MonadFixtureConsumer: write input.json (genesis alloc + blocks)
    MonadFixtureConsumer->>eest_runner: Popen(--input, --output, --ledger-dir, --db)
    eest_runner->>Docker: start runloop container (EEST_RUNNER_CONTAINER_NAME)
    eest_runner->>Docker: propose + finalize blocks via ledger sim
    Docker-->>eest_runner: execution results
    eest_runner-->>MonadFixtureConsumer: output.json (post_state, state_root)
    MonadFixtureConsumer->>MonadFixtureConsumer: compare post_state accounts
    MonadFixtureConsumer->>MonadFixtureConsumer: compare state_root vs last blockHeader
    MonadFixtureConsumer-->>pytest: pass / raise Exception(mismatches)
Loading

Reviews (5): Last reviewed commit: "chore(ci): Use monad-eest-rust-harness a..." | Re-trigger Greptile

pdobacz added 14 commits July 15, 2026 11:49
Co-Authored-By: Claude <claude-opus-4-8>
(cherry picked from commit 68a3578)
Co-Authored-By: Claude <claude-opus-4-8[1m]>
(cherry picked from commit 84ae0fc)
Cuts per-test runtime ~70x (112s to ~1.6s) with identical post-state.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit 2561c49)
Names the container and sets PR_SET_PDEATHSIG so a killed docker client
no longer leaves the runloop spinning past pytest.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit 428fc1f)
Co-Authored-By: Claude <claude-opus-4-8[1m]>
(cherry picked from commit e62599e)
Stamps monad consensus header fields so filled block hashes + EIP-2935
storage match the runloop; consumer now checks them and the state root.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit de3b3cb)
Adds MONAD_NEXT/transition revision schedules and --state-machine monad for MIP-8.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit c5b6f2e)
Slot/page/dual-db init from the fixture revision schedule.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit cad717d)
Co-Authored-By: Claude Opus 4.8 (1M context)
(cherry picked from commit 702b90a)
200M default gas limit under --monad-runloop, monad_runloop skip marker, mip4 transition fork-scope fix.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit c037989)
Fills with --monad-runloop on eestnet chain through MONAD_NEXT for the eest-runner consumer.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit 64b5770)
Removes the pre-monad entry transition; its only consumer (p256verify before-fork test) now resolves to no forks.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit 6321736)
Run a fixtures release against a pinned monad stack via consume direct + eest-runner.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit c3b7aff)
@pdobacz
pdobacz force-pushed the execute-with-eestnet branch from 1ab8995 to 01cd8ee Compare July 15, 2026 12:18
pdobacz added 3 commits July 15, 2026 12:24
Add the monad fork's release repo to SUPPORTED_REPOS so monad_runloop@... etc. resolve. Releases live in execution-specs now that EEST mirroring is disabled.

Co-Authored-By: Claude <claude-opus-4-8>
Summary now sources effective release/filter and resolved stack SHAs from the steps that ran (push events leave inputs empty); artifact keeps consume.log only.

Co-Authored-By: Claude Opus 4.8 (1M context)
(cherry picked from commit 5e20ed4)
Co-Authored-By: Claude Opus 4.8 (1M context)
(cherry picked from commit 6d1fa4d)
@pdobacz
pdobacz force-pushed the execute-with-eestnet branch from 01cd8ee to d415ab4 Compare July 15, 2026 12:24
Co-Authored-By: Claude <claude-opus-4-8>
@pdobacz
pdobacz marked this pull request as ready for review July 15, 2026 13:10
@pdobacz
pdobacz requested review from QEDK and mijovic as code owners July 15, 2026 13:10
Comment thread .github/workflows/consume_release.yaml
Comment thread packages/testing/src/execution_testing/client_clis/clis/monad.py
Comment thread .github/workflows/consume_release.yaml
Prevents shell/output injection via inputs.release and inputs.filter.

Co-Authored-By: Claude <claude-fable-5>
pdobacz added 2 commits July 16, 2026 07:07
Co-Authored-By: Claude <claude-opus-4-8>
Strip the tests- tag prefix when deriving fixtures_<feature>.tar.gz.

Co-Authored-By: Claude <claude-opus-4-8>

@QEDK QEDK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@pdobacz

pdobacz commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator Author

Tacked on last commit to not point to execute-with-eestnet branches here and in monad-eest-rust-harness, as they will no longer exist after the merges.

Will merge once CI clears

@pdobacz
pdobacz merged commit 8d738ae into forks/monad_nine Jul 17, 2026
5 checks passed
@pdobacz
pdobacz deleted the execute-with-eestnet branch July 17, 2026 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants