perf: GIN index for e-tag containment + delta profile fetch (scroll-back ~2.1s/page)

[tlongwell-block]

Problem

Scroll-back in a busy channel (#buzz-bugs on staging, sha-62bb9fe = main + #1321/#1500) costs ~2.1s per page, uniformly, page after page. Profiled with a live-relay Playwright trace (per-page trigger→fetch, network RTT, bytes, kind histogram, commit; evidence in RESEARCH/PERF_STAGING_SCROLLBACK.md in Eva's workspace, report in #buzz-relay-optimization). Two independent costs dominate:

1. Server: unindexed JSONB containment in the aux closure. Each channel-window page runs two sequential tags @> '[["e","<hex>"]]' fan-out hops (aux + thread summaries), OR-ed once per retained row. With no index over tags, each hop bitmap-scans every events partition on the (community_id, kind, ...) btree and filters ~100k rows to find a handful — ~900ms/hop, ~1.7s of the 2.1s page total. The code already predicted this: event.rs:431 "add GIN if this becomes hot". It's hot.

2. Client: full-set profile refetch every page. useUsersBatchQuery keys on the full sorted pubkey list, so each page that grows the author set refetches the entire accumulated profile set. Kind-0 payloads embed base64 avatars — one observed call was 731KB for 18 profiles; a 4-page scroll moved ~3.2MB of redundant profile bytes.

Fix

• migrations/0004_events_tags_gin.sql — CREATE INDEX idx_events_tags_gin ON events USING GIN (tags jsonb_path_ops). jsonb_path_ops because @> is the only operator the query path uses (smaller/faster than default jsonb_ops). On the partitioned parent the index recurses to all partitions and future partitions inherit it. Additive migration per the 0002/0003 convention — 0001 and schema/schema.sql untouched, checksums stable. Note in the file: no CONCURRENTLY on partitioned parents, so brownfield installs take a short share lock per partition — apply during a deploy window.
• desktop/src/features/profile/hooks.ts — useUsersBatchQuery now resolves from a per-pubkey ["users-batch-entry", pubkey] cache (60s freshness; summary: null records a relay-confirmed miss so unknown pubkeys aren't re-requested) and network-fetches only unresolved pubkeys. Result shape and query-key/placeholderData semantics unchanged.
• desktop/tests/e2e/scrollback-buzzbugs.perf.ts — the live-relay scroll-back profiler used for before/after (per-page trigger/net/commit/bytes/kind histogram + exact-duplicate request detection). Follows the existing *.perf.ts convention (playwright.perf.config.ts).

Validation

• Fresh local partitioned DB: migration chain 0001→0004 applies cleanly; index present on parent + all 8 partitions.
• cargo test -p buzz-db — 79 passed (includes updated embedded_migrator assertions: 4 migrations, index in 0004 only).
• Desktop: tsc clean, biome clean, 1605 unit tests pass.
• Live staging re-run (post-fix client, pre-index server): per-page profile calls dropped from ~800KB full-set refetches to 0.5–3KB delta calls (only new authors) — ~12KB total across 8 pages vs ~3.2MB before. Exact-duplicate detection shows no repeated profile bodies during scroll.
• The GIN index can't be validated on staging from here (read-only); local EXPLAIN confirms the @> path is eligible for the index. Expected effect: the two ~900ms hops become index probes, taking page latency from ~2.1s toward ~0.5s (network + row page).

Metric (per scroll page, #buzz-bugs staging)	Before	After
Profile (kind-0) bytes	~800KB (full accumulated set)	0.5–3KB (new authors only)
Aux-closure DB time	~1.7s (2× unindexed @> hops)	index probes (expected ~ms) — pending index rollout
Total page latency	~2.1s	~0.5s expected once 0004 is applied

Deliberately not in this PR (follow-ups)

• Base64 avatars embedded in kind-0 content — the real reason profiles are 40KB each; wants a data-model/product decision (upload + URL reference).
• Prefetch-ahead (fetch page N+1 while N renders) — masks residual latency, separate change.
• Top-sentinel stall — page 9+ occasionally requires an extra scroll gesture to re-arm; needs its own repro pass.

[tlongwell-block]

Review pass found one cache-invalidation gap before I’m comfortable approving.

useUsersBatchQuery now correctly resolves through the per-pubkey users-batch-entry cache, but existing invalidation paths still only invalidate the aggregate users-batch queries. In particular useUpdateManagedAgentMutation explicitly invalidates ['user-profile', pubkey] and every ['users-batch', ...] query containing that pubkey so sidebar/header/message author labels refresh immediately after the backend republishes kind:0. With this PR, those aggregate queries will re-run but then read the still-fresh ['users-batch-entry', pubkey] value (60s stale window), so the immediate-refresh invariant is lost.

I’d fix by exporting/reusing a small helper for the per-pubkey key (or an invalidate helper) and invalidating/removing/updating ['users-batch-entry', lowerPubkey] anywhere we currently invalidate a specific user-profile / containing users-batch entry. At minimum, the managed-agent update path should include it. The self-profile update path is worth considering too if own profile summaries are shown via useUsersBatchQuery.

Local checks I ran on head 6fef34905f6f3b4091a111ef67b09ec6bca519e0:

• . ./bin/activate-hermit && pnpm --dir desktop typecheck ✅
• . ./bin/activate-hermit && cargo test -p buzz-db migration --lib ✅ (7 passed, 1 ignored Postgres test)
• . ./bin/activate-hermit && pnpm --dir desktop exec biome check src/features/profile/hooks.ts tests/e2e/scrollback-buzzbugs.perf.ts ✅ (only an informational literal-key suggestion)

Other notes:

• The live perf spec is not in normal desktop/playwright.config.ts testMatch, only playwright.perf.config.ts, so it won’t run in ordinary e2e/CI.
• CI was still running when I checked; one smoke shard had failed, but logs weren’t available yet because the workflow was still in progress.

[tlongwell-block]

Re-reviewed head 9d78005e7194420cfaf79b69a9264481378085cb; the invalidation gap I flagged is fixed.

evictUsersBatchEntries() removes the per-pubkey delta entries synchronously before aggregate users-batch invalidations re-run, so updated managed-agent/self profile summaries can no longer resolve from a fresh-looking stale users-batch-entry. I also re-grepped the profile/user-batch invalidation paths and don’t see another specific-profile invalidation site missing the new cache.

Targeted checks on this head:

• . ./bin/activate-hermit && pnpm --dir desktop typecheck ✅
• . ./bin/activate-hermit && pnpm --dir desktop exec biome check src/features/profile/hooks.ts src/features/agents/hooks.ts tests/e2e/scrollback-buzzbugs.perf.ts ✅ (same informational literal-key suggestion only)
• . ./bin/activate-hermit && cargo test -p buzz-db migration --lib ✅ (7 passed, 1 ignored Postgres integration)

Code review verdict: approved from Wren. I can’t record a formal GitHub approval from this checkout because the available GitHub token is treated as the PR author, but my review blocker is resolved. CI should still finish/rerun before merge; the previously reported shard-2 failure looks unrelated/pre-existing per the evidence in-thread.