Fix ItemStack hashing and classified ingredient lookups - #240
Merged
Conversation
getItemStackHashCode hashed only count and item. Equality compares data components, so the hash was strictly less discriminating than equality: every stack of one item landed in the same bucket of any hash-based ingredient collection. Collapsed collections normalise the count to 1 before using a stack as a key, so the count term is constant there and the hash degenerated to a function of the item alone. With enough component variants per item the buckets treeify and every lookup turns into a tree walk doing full data component comparisons, which is what made large Integrated Dynamics storage networks scale quadratically. Profiling a 50k stack storage terminal open showed 61% of server thread samples inside treeified HashMap buckets and 65% inside IngredientInstanceWrapper.equals. Including components takes that open from 4686 ms to 168 ms of server thread time. The exclusion comment dated from NBT tags, which were expensive to hash. Component maps are not, and vanilla hashes them the same way in ItemStack.hashItemAndComponents. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v (cherry picked from commit d710f4f)
ItemStackHelpersCommon is abstract on more than getItemStackHashCode, so an anonymous subclass does not compile. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v (cherry picked from commit 8589bd8)
A single-classified collection partitions its instances by a category type. When
a query's match condition covers that category, every match has to share the
query's classifier, so an absent classifier means an empty result. contains and
iterator already returned one directly, but getAll, keySet, containsKey,
countKey and count fell through to the unclassified path, which scans every
instance to produce a result that was already known to be empty.
That fallback made the scan the common case rather than the exception. An
IntegratedDynamics storage index keeps one classified map per priority level, so
looking up an item touches every level, and every level that does not happen to
hold that item scanned all of its entries.
Measured on the IntegratedDynamics index benchmarks, item-only lookups over
5000 instances spread across 200 positions and 4 priority levels:
index_lookup_item 0.240831 ms/op before
0.001157 ms/op after
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v
(cherry picked from commit 855d78a)
Stacks carrying no component patch, and stacks whose only component was set back to its default, are the cases most of a storage network consists of. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v (cherry picked from commit b052104)
A component map hashes its prototype alongside its patch, and the prototype is the item's default components. The item is already in the hash, so hashing its defaults again distinguishes nothing while walking the whole default map. For a plain stack, which carries no patch at all, that walk was the entire cost of the hash, and plain stacks are what most of a storage network consists of. This stays consistent with equality because PatchedDataComponentMap keeps its patch sanitized: setting a component to its default removes it from the patch rather than storing it. Two stacks of one item therefore have equal components exactly when they have equal patches. Vanilla can only report an empty patch by building one, which is free for exactly the stacks this catches, so the common implementation does that and NeoForge overrides it with the direct check. Measured on the IntegratedDynamics index benchmarks, over 5000 plain stacks distinct by item and count: index_lookup_exact 0.002014 -> 0.000988 ms/op index_modification 0.001276 -> 0.000582 ms/op Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v (cherry picked from commit c676887)
This branch uses JUnit 4 and bootstraps items through intrusive holders rather than binding components directly, so the two new test classes follow the pattern the existing item tests here use. The assertions are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v
Member
Author
|
Correcting a placeholder in the description: the benchmark shapes referenced there are added in CyclopsMC/IntegratedDynamics#1727, which is the 1.21 counterpart of the benchmark PR. Generated by Claude Code |
This was referenced Sep 7, 2026
rubensworks
commented
Sep 7, 2026
…mStackHelpersCommon.java
…temStackHelpersNeoForge.java
…mStackHelpersCommon.java
|
rubensworks
pushed a commit
that referenced
this pull request
Sep 7, 2026
Matches how this landed on master-1.21-lts in #240. The reasoning belongs in the PR, not repeated in the file. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mxjin31W1Lmq5XK1CCe84v
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



The 1.21 version of #238. Same three changes, measured on 1.21.1, where the effect is larger than on 26.1.
1. The ItemStack hash ignored data components
IItemStackHelpers.getItemStackHashCodehashed only the count and the item, while equality compares components in full. Every stack of the same item therefore hashed alike, so the hash-based ingredient collections collapsed into one bucket per item type and each lookup in such a bucket became a linear scan doing full component comparisons.The method still carried the commented-out NBT hashing it inherited from before data components, with a note that tags were too expensive to hash. Component maps are not.
2. Classified lookups scanned everything when the classifier was empty
A single-classified collection partitions instances by a category type, and ItemStack is classified by its item. When a query's match condition covers that category, every match has to share the query's classifier, so an absent classifier means an empty result.
containsanditeratoralready returned one directly.getAll,keySet,containsKey,countKeyandcountfell through to the unclassified path, which scans every instance to produce a result already known to be empty.An IntegratedDynamics storage index keeps one classified map per priority level, so an item lookup visits every level, and every level not holding that item scanned all of its entries. This is a pre-existing bug, independent of change 1, and it is why item-only lookups were slow before any of this.
3. Plain stacks were hashing their item's default components
A component map hashes its prototype alongside its patch, and the prototype is the item's defaults, which the item already in the hash stands for. For a stack carrying no patch that walk was the entire cost of the hash, and plain stacks are what most of a storage network consists of.
This is safe on 1.21.1 for the same reason as on 26.1:
PatchedDataComponentMapkeeps its patch sanitized. I checked the 1.21.1 sources directly, bothsetandapplyPatchdrop an entry whose value equals the prototype's default rather than storing it, andhashCodeisprototype.hashCode() + patch.hashCode() * 31. Two stacks of one item therefore have equal components exactly when they have equal patches.ItemStack.isComponentsPatchEmpty()exists in NeoForge 21.1.2, so the NeoForge override works here too, and vanillagetComponentsPatch().isEmpty()backs the common implementation.Numbers
IntegratedDynamics index benchmarks on 1.21.1,
PERFORMANCE_BENCHMARK_ENABLED=true ./gradlew runGameTestServer, ms per operation, medians of three whole runs each side. The shapes are added in CyclopsMC/IntegratedDynamics#PENDING_ID_BENCH.How this differs from the 26.1 result, and what I cannot explain
On 26.1 the modification benchmarks regressed, roughly 2x on the spread shape and 4.9x with a heavy component payload. On 1.21 they improve instead, 5.9x and 2.2x. The reason is that the 1.21 baseline is far worse to begin with:
index_modificationstarts at 0.009933 here against 0.000744 on 26.1, andindex_lookup_itemat 9.51 against 0.27. The added hashing cost is real on both branches, but on 1.21 it is swamped by how much worse the collisions were.I have not isolated why the 1.21 baseline collides so much harder. The most likely explanation is that 1.21.1 registers fewer items, so spreading 5000 instances over them puts more component variants on each item and lengthens every collision chain. That is a hypothesis I did not test, not a measured finding.
Three rows still regress, all small and all with overlapping distributions at n=3:
index_modification_plain1.26x (before 0.000438 to 0.000544, after 0.000463 to 0.000963)index_lookup_exact_mixed1.25xindex_lookup_item_single_item1.09x, which is consistent across runs. That query returns every variant of an item that has 5000 of them, so classification cannot narrow it and each result costs a dearer hash.Tests
TestItemStackHelpersHashCode: equal stacks hash equal, different items and counts hash differently, components affect the hash, 1000 component variants of one item produce over 99% distinct hashes, plain stacks still spread over items and counts, a component set back to its default hashes as plain again, and the hash agrees withItemStack.isSameItemSameComponentsplus count over 100 samples.TestSingleClassifiedAbsentClassifier: the five affected methods return empty for an absent classifier and correct results for a present one, and a counting inner collection asserts that nothing is scanned in the absent case, so the optimization is pinned and not just its answer.Both are written against this branch's JUnit 4 setup and bootstrap items through intrusive holders, matching the existing item tests here. A separate commit makes that adjustment, so the port is visible on its own.
./gradlew buildpasses on all loaders.Relationship to the 26.1 branch
#238 is the same change against
master-26-ltsand stays open. The five ingredient collection files are byte identical between the two branches, so onlyItemStackHelpersCommonand the tests needed adjusting. Merge whichever suits your upmerge direction; they are not meant to both land independently.Notes for review
What was not verified
hasComponentPatchimplementation, which builds a patch instance for stacks that carry one. Only NeoForge was benchmarked.Generated by Claude Code