Threading of post pre - #7201
Draft
hnil wants to merge 2 commits into
Draft
Conversation
prepareLocalCellData's per-cell extractor walk is now parallelised over ElementChunks with a per-thread ElementContext; the block-data extractors write through pre-allocated per-cell slots so the walk stays race-free and bit-identical. The region-average density extractor mutates shared region-indexed storage, so it falls back to a single (serial) chunk via GenericOutputBlackoilModule::requiresSerialExtraction(). calc_inplace's region sums now compute the independent per-quantity rank-local sweeps in parallel (new regionSumLocal), keeping the MPI reduction and the Inplace updates serial so the result is bit-identical across thread counts. Verified bit-identical UNRST/INIT on spe9 and norne across 1/4/8 threads. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ElementContext tracer assembly recomputes intensive/extensive quantities per element; that recompute heap-allocates and serialises on the allocator under threading, so it anti-scales (norne: 6.6 -> 12.4 s from 1 -> 8 threads) while the matrix writes stay flat. assembleTracerEquationsTpfa_ instead reuses cached intensive quantities plus the linearizer's neighbour/transmissibility table and the ElementContext-free LocalResidual::computeFlux, mirroring the reservoir tpfa assembly, and drops updateAllIntensive/ExtensiveQuantities(). Selected by --tracer-tpfa-assembly (default true); the ElementContext assembly is kept and used automatically when the linearizer has no getNeighborInfo(). norne pre/post 15.2 -> 18.4 s (old, anti-scaling) becomes 12.9 -> 5.9 s (new). Bit-identical UNRST on norne (standard + MSW); matches to last mantissa bits on the vaporized-oil tracer decks. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
hnil
added
the
manual:irrelevant
hnil
marked this pull request as draft
June 24, 2026 11:39
Member
|
Thanks. Speeding up is always good.
Can you explain? I didn't see that from the code
Why do you want to keep the old implementation and add a parameter? |
Member
Author
|
At least on mac the was an anti calling with threading of the tracer code i.e. the code slowed down with opm process. I kept the old code since it may at some point or for some model be need to use the elemtctx (I do not have the full overview). It also in principle is more genneral. |
Member
Author
|
Combing this with the linear solver part give. This the missing part is threading of amg. amgcl threads but is about the same final speed since original code is slower pga more iterations. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This should thread the post pre at least to 2x. In norne it fixes a scaling problem + give better small serial improvement due to better tracer assembly.