Skip to content

Exclude benchmarks and fuzzing from default builds#8364

Open
myrrc wants to merge 1 commit into
developfrom
myrrc/faster-ci-builds
Open

Exclude benchmarks and fuzzing from default builds#8364
myrrc wants to merge 1 commit into
developfrom
myrrc/faster-ci-builds

Conversation

@myrrc

@myrrc myrrc commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

This PR adds default-members in Cargo.toml which exclude benchmarks and fuzzing
from the default build. This should have an impact on CI as well.

@myrrc myrrc requested a review from a team June 11, 2026 15:13
@myrrc myrrc force-pushed the myrrc/faster-ci-builds branch from f3852c9 to 2837817 Compare June 11, 2026 15:13
@myrrc myrrc requested a review from AdamGS June 11, 2026 15:13
Comment thread Cargo.toml Outdated
@myrrc myrrc marked this pull request as draft June 11, 2026 15:21
@AdamGS

AdamGS commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

I'm not sure I agree with the first point, but the latter seems very reasonable, no reason to treat auxiliary code the same as the code crates.

@codspeed-hq

codspeed-hq Bot commented Jun 11, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 17.99%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 6 regressed benchmarks
✅ 1520 untouched benchmarks
⏩ 10 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation chunked_bool_canonical_into[(1000, 10)] 20.4 µs 35.4 µs -42.42%
Simulation chunked_varbinview_into_canonical[(1000, 10)] 176.8 µs 212.9 µs -16.98%
Simulation chunked_varbinview_canonical_into[(100, 100)] 272.7 µs 307.9 µs -11.41%
Simulation encode_varbin[(1000, 8)] 143.6 µs 160.6 µs -10.61%
Simulation encode_varbin[(1000, 4)] 142.9 µs 159.9 µs -10.61%
Simulation encode_varbin[(1000, 32)] 148.5 µs 165.2 µs -10.09%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing myrrc/faster-ci-builds (862eff4) with develop (c6a0b7b)

Open in CodSpeed

Footnotes

  1. 10 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@AdamGS AdamGS left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug info is a feature of dev builds. If you don't want it, you can build the specific crates you care about or use ci profile.

@myrrc myrrc marked this pull request as ready for review June 11, 2026 15:33
robert3005
robert3005 previously approved these changes Jun 11, 2026
@robert3005 robert3005 dismissed their stale review June 11, 2026 17:26

Adam is right

@myrrc myrrc force-pushed the myrrc/faster-ci-builds branch from 2837817 to fee14ce Compare June 12, 2026 09:01
@myrrc myrrc requested a review from AdamGS June 12, 2026 09:01
@myrrc myrrc force-pushed the myrrc/faster-ci-builds branch from fee14ce to 862eff4 Compare June 12, 2026 09:03
@AdamGS

AdamGS commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

you have a conflict in CI

Comment thread .github/workflows/sql-benchmarks.yml Outdated
Comment thread Cargo.toml Outdated
Comment thread .github/workflows/bench.yml Outdated
@myrrc myrrc force-pushed the myrrc/faster-ci-builds branch from 862eff4 to 8be14a7 Compare June 12, 2026 10:54
Signed-off-by: Mikhail Kot <mikhail@spiraldb.com>
@myrrc myrrc force-pushed the myrrc/faster-ci-builds branch from 8be14a7 to df4edba Compare June 12, 2026 10:55
@myrrc myrrc requested a review from AdamGS June 12, 2026 10:55
@myrrc myrrc enabled auto-merge (squash) June 12, 2026 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants