Skip to content

Add ingredient index benchmarks for collision-heavy and plain storage shapes - #1722

Merged
rubensworks merged 3 commits into
master-26-ltsfrom
test/index-perf-component-collisions
Sep 7, 2026
Merged

Add ingredient index benchmarks for collision-heavy and plain storage shapes#1722
rubensworks merged 3 commits into
master-26-ltsfrom
test/index-perf-component-collisions

Conversation

@rubensworks

@rubensworks rubensworks commented Sep 6, 2026

Copy link
Copy Markdown
Member

The existing GameTestsPerformanceIngredientIndex benchmarks build their fixture by spreading instances over every registered item, adding component variants only once the item list is exhausted. One shape, and a middling one: collision chains stay short enough to hide the shapes that hurt, and it carries enough components to hide the shape that dominates real storages.

Four shapes are added:

  • SINGLE_ITEM: every instance on one item, distinct only by components. A storage full of enchanted books, damaged tools, or written books.
  • FEW_ITEMS: spread over 50 item types with component variants within each.
  • HEAVY_COMPONENTS: spread as before, but every stack carries lore and container contents, so the cost of hashing a component map is visible. This guards the opposite risk, a hash that is more discriminating but more expensive.
  • PLAIN: instances distinct by item and count only, so none carries a component patch. This is what a large modpack storage mostly looks like, many unique items and few component variants, and no benchmark covered it.
  • MIXED: one instance in ten carries a component and the rest are plain. PLAIN and SPREAD bracket a real storage network rather than describing one; this sits between them.

Item-only lookups also only had a benchmark on the spread shape, although the number of variants of one item is exactly what decides their cost. They are now covered on the plain and single-item shapes too.

Eighteen benchmarks in total. The existing SPREAD shape and its benchmark names are unchanged, so existing result history stays comparable.

This is measurement only. It does not depend on any CyclopsCore change and can be merged on its own.

One correction to the existing fixture

BuiltInRegistries.ITEM.stream() includes minecraft:air. A stack of air is empty, and an empty stack carries no components, so any fixture entry built on it indexes nothing. That was harmless while air came up once per pass over the registry, but it makes a single-item shape index nothing at all. Air is now filtered out of the pool.

Numbers

PERFORMANCE_BENCHMARK_ENABLED=true ./gradlew runGameTestServer, ms per operation, medians of six runs, against CyclopsCore 1.30.4-DEV as it is on master-26-lts today.

benchmark ms/op
index_lookup_exact 0.004670
index_lookup_item 0.268168
index_lookup_nonempty_first 0.000243
index_lookup_nonempty_all 0.012368
index_modification 0.000744
index_lookup_exact_plain 0.001227
index_lookup_item_plain 0.231160
index_modification_plain 0.000540
index_lookup_exact_mixed 0.001618
index_lookup_item_mixed 0.220683
index_modification_mixed 0.000501
index_lookup_exact_single_item 3.796415
index_modification_single_item 3.291132
index_lookup_item_single_item 0.432068
index_lookup_exact_few_items 0.070593
index_modification_few_items 0.156185
index_lookup_exact_heavy_components 0.003482
index_modification_heavy_components 0.000423

Two things stand out, and both were invisible before.

The single-item shape is roughly 800 times slower per operation than the spread shape, because a component-blind ItemStack hash puts every variant of one item in a single bucket.

Item-only lookups cost 0.22 to 0.27 ms per operation on every shape, which is 200 to 500 times the cost of an exact lookup. That is not hashing at all: IngredientMapSingleClassified falls through to a full scan whenever the queried classifier holds nothing, which for a per-priority index is most lookups.

CyclopsMC/CyclopsCore#238 addresses both. With it these become 0.001252 and 0.001053 respectively.

./gradlew build passes.

@coveralls

coveralls commented Sep 6, 2026

Copy link
Copy Markdown

Coverage Status

coverage: 55.602% (-0.06%) from 55.657% — test/index-perf-component-collisions into master-26-lts

@rubensworks rubensworks changed the title Add ingredient index benchmarks for collision-heavy storage shapes Add ingredient index benchmarks for collision-heavy and plain storage shapes Sep 6, 2026
@rubensworks
rubensworks force-pushed the test/index-perf-component-collisions branch from 5791877 to c884155 Compare September 6, 2026 18:15
@rubensworks
rubensworks marked this pull request as ready for review September 6, 2026 18:20

Copy link
Copy Markdown
Member Author

A master-1.21-lts version of this now exists as #1727, since the plan is to upmerge rather than maintain both.

The port is not quite identical: on 1.21 the benchmark class lives under src/integrationtest rather than src/main, and uses batch rather than environment on @GameTest. The shapes and the fixture are the same.

The 1.21 baseline these benchmarks expose is much worse than this one. index_lookup_item is 9.51 ms/op there against 0.27 here, and an exact lookup on the single-item shape 5.15 against 3.79. I have not isolated why; the likeliest explanation is that 1.21.1 registers fewer items, so 5000 instances spread across them put more variants on each item and lengthen every collision chain.

This PR stays open. Merge whichever branch suits your upmerge direction.


Generated by Claude Code

The existing index benchmarks spread instances over every registered item, so
collision chains in the hash-based ingredient collections stay short and the
cost of a component-blind ItemStack hash stays invisible. Three shapes are
added that make it visible:

* SINGLE_ITEM: every instance on one item, distinct only by data components,
  which is what a storage full of enchanted books or damaged tools looks like.
* FEW_ITEMS: spread over 50 item types with component variants within each.
* HEAVY_COMPONENTS: spread as before, but each stack carries a large component
  payload, so the cost of hashing components themselves is measurable. This
  guards the opposite risk of a more expensive hash.

Air is now excluded from the item pool. A stack of it is empty and carries no
components, so a fixture built on it would index nothing.

These are measurement only, independent of any hash change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v
The collision-heavy shapes cover storages full of component variants. They say
nothing about the opposite and far more common shape: a large modpack storage
holding many unique items and few component variants. A PLAIN shape covers it,
with instances distinct by item and count so that none carries a component
patch.

Item-only lookups also only had a benchmark on the spread shape. They are now
covered on the plain and single-item shapes too, since how many variants of one
item exist is exactly what decides their cost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v
PLAIN and SPREAD bracket the realistic case rather than covering it. A real
storage network is mostly bulk material with a tail of enchanted, damaged or
named items, so MIXED gives one in ten instances a component and leaves the
rest plain.

This is the shape that decides whether hashing data components is worth what it
costs, since the cost falls only on the instances that carry them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v
@rubensworks
rubensworks force-pushed the test/index-perf-component-collisions branch from c884155 to 7adddff Compare September 7, 2026 17:18
@rubensworks
rubensworks merged commit d55dc68 into master-26-lts Sep 7, 2026
7 checks passed
@rubensworks
rubensworks deleted the test/index-perf-component-collisions branch September 7, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants