Skip to content

chore(tests-benchmark, ci): mark test_keccak_max_permutations as slow#3058

Closed
danceratopz wants to merge 1 commit into
ethereum:forks/amsterdamfrom
danceratopz:mark-test_keccak_max_permutations-slow
Closed

chore(tests-benchmark, ci): mark test_keccak_max_permutations as slow#3058
danceratopz wants to merge 1 commit into
ethereum:forks/amsterdamfrom
danceratopz:mark-test_keccak_max_permutations-slow

Conversation

@danceratopz

Copy link
Copy Markdown
Member

🗒️ Description

test_keccak_max_permutations is by far the most expensive test in the bench-gas smoke check: It fills in ~465s per format, while the next slowest benchmark fills in ~12s. The test searches for the input size that maximizes keccak permutations per block and then fills a maximal block, and that work is repeated per fixture format.

Marking it @pytest.mark.slow excludes it from the smoke check (just bench-gas and the benchmark sanity checks run -m "not slow"), consistent with the existing slow benchmarks test_blockhash and test_bls12_g2_msm. The release benchmark matrix (.github/configs/feature.yaml) does not filter slow, so the test still ships in the released fixtures.

🔗 Related Issues or PRs

Addresses #2673. Complements #3057 (fill only the blockchain_test format in bench-gas): Even with that format reduction the test still fills once at ~465s and sets the smoke check's floor, and this marker removes it.

✅ Checklist

  • All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    just static
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).

`test_keccak_max_permutations` dominates the `bench-gas` smoke check: It fills
in ~465s per format, while the next slowest benchmark is ~12s, because it
searches for the input size that maximizes keccak permutations per block and
then fills a maximal block. Marking it `slow` excludes it from the smoke check
(`just bench-gas` runs `-m "not slow"`), matching `test_blockhash` and
`test_bls12_g2_msm`.

The release benchmark matrix (`.github/configs/feature.yaml`) does not filter
`slow`, so the test still ships in the released fixtures.
@danceratopz danceratopz marked this pull request as draft June 29, 2026 11:24
@codecov

codecov Bot commented Jun 29, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.22%. Comparing base (3f888bc) to head (93dd354).

Additional details and impacted files
@@               Coverage Diff                @@
##           forks/amsterdam    #3058   +/-   ##
================================================
  Coverage            93.22%   93.22%           
================================================
  Files                  624      624           
  Lines                36926    36926           
  Branches              3377     3377           
================================================
  Hits                 34424    34424           
  Misses                1708     1708           
  Partials               794      794           
Flag Coverage Δ
unittests 93.22% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@danceratopz danceratopz changed the title chore(tests): mark test_keccak_max_permutations as slow chore(tests-benchmark, ci): mark test_keccak_max_permutations as slow Jun 29, 2026
@danceratopz danceratopz added C-chore Category: chore A-test-benchmark Area: execution_testing.benchmark and tests/benchmark A-ci Area: Continuous Integration labels Jun 29, 2026
@danceratopz

Copy link
Copy Markdown
Member Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ci Area: Continuous Integration A-test-benchmark Area: execution_testing.benchmark and tests/benchmark C-chore Category: chore

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant