Skip to content

feat(consume): Consume tests with runloop monad#32

Open
pdobacz wants to merge 21 commits into
forks/monad_ninefrom
execute-with-eestnet
Open

feat(consume): Consume tests with runloop monad#32
pdobacz wants to merge 21 commits into
forks/monad_ninefrom
execute-with-eestnet

Conversation

@pdobacz

@pdobacz pdobacz commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Adds a consume direct test consumer enabling execution of spec test fixtures against a more production-like environment of monad execution client.

Currently there are 3 main use cases for this setup:

  1. Run all spec tests in a slower but more production-like environment (than monad-ethereum-test executable)
  2. Enable running fork transition tests (for MIP-8 specifically, but generally too)
  3. Enable running preliminary performance regression tests for MIP-8

Requires https://github.com/monad-exp/monad-eest-rust-harness/pull/1

Greptile Summary

This PR wires up a new consume direct test consumer — MonadFixtureConsumer — that runs blockchain spec fixtures against the production monad runloop via the eest-runner harness. It also adds a --monad-runloop fill flag that stamps consensus-derived header fields (prev_randao, gas_limit, extra_data, requests_hash) onto filled blocks so hashes and EIP-2935 storage match the harness output.

  • New consumer (clis/monad.py): digests fixtures into a genesis + per-block input document, provisions a fresh triedb (slot/page/dual-encoded based on the fork schedule), runs eest-runner in a named Docker container with parent-death signal cleanup, and asserts per-account post-state and final state root.
  • New workflow (consume_release.yaml): manual dispatch to run a pinned monad-stack release; inputs routed through env vars, optional monad-bft/monad-execution overrides with validation, and a Markdown summary step that always runs.
  • Fork cleanup: PragueToMONAD_EIGHTAtTime15k removed as unused; test_fork_transition.py narrowed to the MONAD_NINE boundary only.

Confidence Score: 5/5

Safe to merge; the new consumer and workflow are well-structured and the changes to existing files are small and correct.

The consumer logic is sound: triedb provisioning correctly handles slot/page/dual-encoded schedules, container teardown covers both graceful and SIGKILL paths, and the state-root + per-account comparisons are complete. The workflow correctly routes all user inputs through env vars and handles missing stack SHAs with fallbacks. The two findings are quality-of-life improvements with no correctness impact on the happy path.

packages/testing/src/execution_testing/client_clis/clis/monad.py — the new consumer is the most complex new file and the one to re-read if future harness changes alter exit behavior.

Important Files Changed

Filename Overview
packages/testing/src/execution_testing/client_clis/clis/monad.py New MonadFixtureConsumer that digests blockchain fixtures and drives eest-runner; logic is sound but communicate() has no timeout and the success-path output file check is missing (see comments).
.github/workflows/consume_release.yaml New manual workflow to consume a pinned monad-runloop fixture release; inputs correctly routed through env vars, fallbacks cover missing stack SHAs, previous injection issue is fixed.
packages/testing/src/execution_testing/specs/blockchain.py Stamps monad runloop header fields (gas_limit, prev_randao, extra_data, requests_hash) on blocks when --monad-runloop is active; correctly scoped to MONAD_EIGHT+ forks via issubclass check.
packages/testing/src/execution_testing/cli/pytest_commands/plugins/consume/releases.py asset_name now strips the tests- prefix from monad release tags; backward-compatible for all upstream repos whose tags don't carry that prefix.
packages/testing/src/execution_testing/test_types/block_types.py Adds MonadRunloopDefaults dataclass as session-level global config; consistent with existing EnvironmentDefaults pattern.
packages/testing/src/execution_testing/forks/forks/transition.py Removes unused PragueToMONAD_EIGHTAtTime15k transition fork; the test file that targeted it is left with a note that it now generates no fixtures.
packages/testing/src/execution_testing/cli/pytest_commands/plugins/filler/filler.py Adds --monad-runloop CLI option and wires it into pytest_configure and pytest_collection_modifyitems; straightforward option plumbing.
tests/monad_nine/mip4_checkreservebalance/test_fork_transition.py Removes subsequent_forks=True, scoping the test to only the MONAD_NINE transition; intentional for MIP-8 fork-transition coverage.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant pytest as pytest / consume direct
    participant consumer as MonadFixtureConsumer
    participant mpt as monad-mpt
    participant harness as eest-runner (Docker)
    participant checker as post-state checker

    pytest->>consumer: consume_fixture(fixture_path, fixture_name)
    consumer->>consumer: load JSON, resolve FORK_REVISION_SCHEDULES
    consumer->>consumer: skip if any expectException block
    consumer->>consumer: _digest_genesis_alloc + _digest_blocks to input.json
    consumer->>mpt: "--create --state-machine ethereum|monad"
    mpt-->>consumer: triedb formatted
    opt transition fixture (slot + page)
        consumer->>mpt: --activate-secondary --state-machine monad
        mpt-->>consumer: secondary timeline active
    end
    consumer->>harness: eest-runner --input --output --ledger-dir --db
    Note over harness: Ledger sim proposes/finalizes blocks,<br/>drives runloop FFI, writes output.json
    harness-->>consumer: stdout, stderr, returncode
    alt "returncode != 0"
        consumer->>pytest: raise Exception (with stdout/stderr)
    else success
        consumer->>checker: load output.json post_state + state_root
        checker->>checker: _compare_account for each postState entry
        checker->>checker: compare final stateRoot
        alt mismatches
            checker->>pytest: raise Exception (mismatch list)
        else pass
            checker->>pytest: return (test passes)
        end
    end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant pytest as pytest / consume direct
    participant consumer as MonadFixtureConsumer
    participant mpt as monad-mpt
    participant harness as eest-runner (Docker)
    participant checker as post-state checker

    pytest->>consumer: consume_fixture(fixture_path, fixture_name)
    consumer->>consumer: load JSON, resolve FORK_REVISION_SCHEDULES
    consumer->>consumer: skip if any expectException block
    consumer->>consumer: _digest_genesis_alloc + _digest_blocks to input.json
    consumer->>mpt: "--create --state-machine ethereum|monad"
    mpt-->>consumer: triedb formatted
    opt transition fixture (slot + page)
        consumer->>mpt: --activate-secondary --state-machine monad
        mpt-->>consumer: secondary timeline active
    end
    consumer->>harness: eest-runner --input --output --ledger-dir --db
    Note over harness: Ledger sim proposes/finalizes blocks,<br/>drives runloop FFI, writes output.json
    harness-->>consumer: stdout, stderr, returncode
    alt "returncode != 0"
        consumer->>pytest: raise Exception (with stdout/stderr)
    else success
        consumer->>checker: load output.json post_state + state_root
        checker->>checker: _compare_account for each postState entry
        checker->>checker: compare final stateRoot
        alt mismatches
            checker->>pytest: raise Exception (mismatch list)
        else pass
            checker->>pytest: return (test passes)
        end
    end
Loading

Reviews (4): Last reviewed commit: "fix(consume): resolve release asset for ..." | Re-trigger Greptile

pdobacz added 14 commits July 15, 2026 11:49
Co-Authored-By: Claude <claude-opus-4-8>
(cherry picked from commit 68a3578)
Co-Authored-By: Claude <claude-opus-4-8[1m]>
(cherry picked from commit 84ae0fc)
Cuts per-test runtime ~70x (112s to ~1.6s) with identical post-state.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit 2561c49)
Names the container and sets PR_SET_PDEATHSIG so a killed docker client
no longer leaves the runloop spinning past pytest.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit 428fc1f)
Co-Authored-By: Claude <claude-opus-4-8[1m]>
(cherry picked from commit e62599e)
Stamps monad consensus header fields so filled block hashes + EIP-2935
storage match the runloop; consumer now checks them and the state root.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit de3b3cb)
Adds MONAD_NEXT/transition revision schedules and --state-machine monad for MIP-8.
Co-Authored-By: Claude <claude-opus-4-8[1m]>

(cherry picked from commit c5b6f2e)
Slot/page/dual-db init from the fixture revision schedule.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit cad717d)
Co-Authored-By: Claude Opus 4.8 (1M context)
(cherry picked from commit 702b90a)
200M default gas limit under --monad-runloop, monad_runloop skip marker, mip4 transition fork-scope fix.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit c037989)
Fills with --monad-runloop on eestnet chain through MONAD_NEXT for the eest-runner consumer.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit 64b5770)
Removes the pre-monad entry transition; its only consumer (p256verify before-fork test) now resolves to no forks.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit 6321736)
Run a fixtures release against a pinned monad stack via consume direct + eest-runner.
Co-Authored-By: Claude Opus 4.8 (1M context)

(cherry picked from commit c3b7aff)
@pdobacz pdobacz force-pushed the execute-with-eestnet branch from 1ab8995 to 01cd8ee Compare July 15, 2026 12:18
pdobacz added 3 commits July 15, 2026 12:24
Add the monad fork's release repo to SUPPORTED_REPOS so monad_runloop@... etc. resolve. Releases live in execution-specs now that EEST mirroring is disabled.

Co-Authored-By: Claude <claude-opus-4-8>
Summary now sources effective release/filter and resolved stack SHAs from the steps that ran (push events leave inputs empty); artifact keeps consume.log only.

Co-Authored-By: Claude Opus 4.8 (1M context)
(cherry picked from commit 5e20ed4)
Co-Authored-By: Claude Opus 4.8 (1M context)
(cherry picked from commit 6d1fa4d)
@pdobacz pdobacz force-pushed the execute-with-eestnet branch from 01cd8ee to d415ab4 Compare July 15, 2026 12:24
Co-Authored-By: Claude <claude-opus-4-8>
@pdobacz pdobacz marked this pull request as ready for review July 15, 2026 13:10
@pdobacz pdobacz requested review from QEDK and mijovic as code owners July 15, 2026 13:10
Comment thread .github/workflows/consume_release.yaml
Comment thread packages/testing/src/execution_testing/client_clis/clis/monad.py
Comment thread .github/workflows/consume_release.yaml
Prevents shell/output injection via inputs.release and inputs.filter.

Co-Authored-By: Claude <claude-fable-5>
pdobacz added 2 commits July 16, 2026 07:07
Co-Authored-By: Claude <claude-opus-4-8>
Strip the tests- tag prefix when deriving fixtures_<feature>.tar.gz.

Co-Authored-By: Claude <claude-opus-4-8>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant