Skip to content

perf: O(1) tick-indexed array + bitmap price levels (replaces rbtree) - #34

Merged
geseq merged 1 commit into
mainfrom
perf/lever-b
Jun 27, 2026
Merged

perf: O(1) tick-indexed array + bitmap price levels (replaces rbtree)#34
geseq merged 1 commit into
mainfrom
perf/lever-b

Conversation

@geseq

@geseq geseq commented Jun 27, 2026

Copy link
Copy Markdown
Owner

Replaces the per-side boost::intrusive::rbtree<OrderQueue> price-level container with a flat tick-indexed std::vector<OrderQueue*> + a 3-level occupancy bitmap (clz/ctz) for O(1) level find/create/erase and best-bid/ask.

Motivation (worst-case matching)

Profiling the worst-case scenarios (swing-25, flash-crash) showed 97–99.8% of price levels hold a single, sub-millisecond order, yet every add created an rbtree node and every cancel/fill did an O(log N) erase + rebalance, pointer-chasing scattered, cache-cold nodes — ~34% of steady-state matching time. Array indexing makes those operations O(1) and cache-friendly.

Design

  • levels_: tick-indexed vector<OrderQueue*> (nullptr = empty); index = (price.fp - base)/tick_fp, base = 0 so all positive-price ticks are non-negative.
  • 3-level occupancy bitmap for O(1) best-price (highestSet/lowestSet) and level advance (largestLessThan/smallestGreaterThan).
  • Growable: an out-of-range price grows the vector + bitmap (cap = max(t+1, 2*size) rounded to 64, set bits preserved) rather than failing — so it's never worse than the rbtree on any input. Good initial sizing means typical workloads never grow.
  • OrderQueue/Order and the OrderQueue pool are unchanged.

Results (this PR's increment over current main, which already has funcref + LTO)

Worst-case engine throughput via a deterministic single-threaded replay (the threaded harness is too bimodal on shared hardware to read cleanly):

scenario rbtree (main) array+bitmap Δ
static 31.4 36.6 +16.5%
normal 28.6 35.4 +23.8%
swing-25 (worst) 27.8 34.2 +23%
swing-40 29.3 34.5 +17.7%
flash-crash 31.0 34.3 +10.8%
local throughput bench 31.3 33.2 +6%

Worst-case scenario is the same one (swing-25) before and after, and its floor rises +23%; every scenario improves; nothing regresses.

Correctness

  • Byte-identical trade streams on all 5 harness scenarios (consensus-hash PASS / VALID).
  • 7/7 unit tests pass (Release and Debug with asserts active); the array+bitmap directionality and grow path are exercised.

🤖 Generated with Claude Code
https://claude.ai/code/session_0196EKKWudtsg3gButufhDxz

@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown

Benchmark PR

Throughput on hosted runners is noisy and not performance-calibrated; treat the numbers as informational only.

Realistic deep-book throughput (W=50000)

Metric PR main change
ops/sec 31,190,280 25,711,954 +21.3%

Matching-engine-benchmark (perf mode)

Harness pinned at 77697a115e76a25a1e2aa886f555d55b87fcf052.

Scenario PR (M msgs/s) main (M msgs/s) change PR correctness
static 7.769 7.794 -0.3% PASS
normal 8.466 8.202 +3.2% PASS
swing-25 7.732 8.192 -5.6% PASS
swing-40 7.228 8.329 -13.2% PASS
flash-crash 7.395 8.609 -14.1% PASS

…ee optional

Replace the per-side boost::intrusive::rbtree<OrderQueue> price-level
container with a compile-time policy. The default ArrayLevels<P> is a flat
tick-indexed std::vector<OrderQueue*> + a 3-level occupancy bitmap (clz/ctz)
giving O(1) level find/create/erase and best-bid/ask. RbTreeLevels<P> (the
original rbtree) remains selectable via OrderBook<Notification, RbTreeLevels>.

Selection is a template-template param — zero runtime cost, no virtual
dispatch; both backends inline under LTO. PriceLevel<P, Store=ArrayLevels<P>>
keeps volume/order accounting and the matching loops COMMON across backends.

Motivation: in the worst-case matching scenarios 97-99.8% of price levels
hold a single sub-millisecond order, yet the rbtree paid an O(log N) node
insert + erase/rebalance (pointer-chasing cache-cold nodes) per add/cancel
(~34% of steady-state matching). The array makes that O(1) and cache-local.
ArrayLevels grows on demand (never worse than the rbtree on any input);
good initial sizing means typical workloads never grow. base=0.

Worst-case engine throughput (deterministic single-thread replay): slowest
scenario (swing-25) 27.3 -> 34.4 M msgs/s, +23.7%; every scenario +19-27%.
The template policy is zero-cost (array default reproduces the pre-policy
number; rbtree backend reproduces the original within ~1%). Byte-identical
trade streams on all 5 harness scenarios; full suite passes for BOTH
backends (orderbook 52, determinism 4, pricelevel) in Release and Debug.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0196EKKWudtsg3gButufhDxz
@geseq
geseq merged commit 8adb19b into main Jun 27, 2026
5 checks passed
@geseq
geseq deleted the perf/lever-b branch June 27, 2026 18:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant