chore(ci): run benchmark sanity checks on size-xl-x64 runner

[danceratopz]

🗒️ Description

Run the benchmark sanity-checks jobs (Benchmark Gas Values, Fixed Opcode Count CLI, Fixed Opcode Count Config) on the size-xl-x64 self-hosted runner instead of ubuntu-latest.

The standard ubuntu-latest runner only exposes 2 cores, so fill/pytest with -n auto --maxprocesses 10 was capped at 2 workers, making Benchmark Gas Values take roughly 28 minutes. The XL runner has enough cores for -n auto to scale toward the existing --maxprocesses 10 cap, matching the runner already used by test.yaml.

🔗 Related Issues or PRs

N/A.

✅ Checklist

• All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:

  just static

• All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
• All: Considered updating the online docs in the ./docs/ directory.
• All: Set appropriate labels for the changes (only maintainers can apply labels).

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.20%. Comparing base (5f8c109) to head (d14ee17).
⚠️ Report is 18 commits behind head on forks/amsterdam.

Additional details and impacted files

@@               Coverage Diff                @@
##           forks/amsterdam    #3012   +/-   ##
================================================
  Coverage            93.20%   93.20%           
================================================
  Files                  620      620           
  Lines                38777    38777           
  Branches              3342     3342           
================================================
  Hits                 36144    36144           
  Misses                1773     1773           
  Partials               860      860           

Flag	Coverage Δ
unittests	93.20% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[danceratopz]

Thanks for jumping in with a review @LouisTsai-Csie. It didn't lead to an improvement (which is curious). Will revisit when I get the chance, I think we need #2693

[danceratopz]

Thanks for jumping in with a review @LouisTsai-Csie. It didn't lead to an improvement (which is curious). Will revisit when I get the chance, I think we need #2693

My friend told me that:

... the Phase 2 "slowest durations" has a cliff:

465.28s  test_keccak_max_permutations[...blockchain_test...]@t8n-cache-bc5796d4
464.94s  test_keccak_max_permutations[...blockchain_test_engine_x...]@t8n-cache-bc5796d4
408.27s  test_keccak_max_permutations[...blockchain_test_engine...]@t8n-cache-bc5796d4
 12.29s  test_point_evaluation_uncachable   ← next slowest

One test eats ~465s per format; everything else is ≤12s. And note the shared @t8n-cache-bc5796d4 suffix: that's the --dist=loadgroup key, so all three format-variants are pinned to a single xdist worker and run serially → ~1338s (~22 min) on one core. The other 15 cores finish the entire rest of the suite and then sit idle.

We only ignore these test cases so far:

An easy quick win would be to mark this test slow:

• chore(tests-benchmark, ci): mark test_keccak_max_permutations as slow #3058

But tbh, I think there's an easy optimization to be had here that would help releases; will work on that. Perhaps it will even supersede #3058.

[danceratopz]

#3057 and #3060 have brought the run times down to ~5 min (for now); let's leave this on the Github runners. We can always bump later if need be.