fix(ai): deterministic stream recovery for global assistant on mobile - #2065
Conversation
📝 WalkthroughWalkthroughApp resume recovery now dynamically gates execution, resolves a resume action, protects asynchronous message loading, conditionally evicts stale stream messages, and rejoins agent or global streams across both assistant views. ChangesApp resume recovery
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant AppStateRecovery
participant ResumeResolver
participant TryRecover
participant MessageStore
AppStateRecovery->>ResumeResolver: Resolve native and streaming state
ResumeResolver-->>AppStateRecovery: Return resume action
AppStateRecovery->>TryRecover: Stop locally and recover
TryRecover->>MessageStore: Probe live stream and conditionally evict stale message
TryRecover-->>AppStateRecovery: Rejoin stream or load persisted messages
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bb8b15ac02
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| // Block recovery if streaming OR pending send OR any editing active | ||
| enabled: !isStreaming && currentConversationId !== null && !useEditingStore.getState().isAnyEditing(), | ||
| onResume: useCallback(async () => { | ||
| const action = resolveResumeAction({ native: isCapacitorApp(), isStreaming: displayIsStreaming }); |
There was a problem hiding this comment.
Import the resume helpers in SidebarChatTab
This new resume handler calls resolveResumeAction and isCapacitorApp, but SidebarChatTab.tsx does not import either helper. Any build or render path that reaches this component now fails on undefined identifiers instead of mounting the sidebar/recovery hook; add the same imports used by the other chat views before using them here.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed and fixed in c2cd36d — both imports were genuinely missing. Added to SidebarChatTab.tsx:
import { isCapacitorApp } from '@/hooks/useCapacitor';
import { resolveResumeAction } from '@/lib/ai/streams/resolveResumeAction';Same two imports AiChatView.tsx uses. This was the other half of the web#build failure.
While confirming the fix I also checked the stop this handler calls: it comes from usePageAgentSidebarChat, which resolves to the raw mode-selected useChat stop (selectedAgent ? agentStop : globalStop) — i.e. local-only, matching the chatStop semantics in AiChatView. It clears local useChat state so the rejoin attaches cleanly and does not signal the server, so the server-owned run keeps generating and is rejoined rather than killed. Documented that in a comment at the call site, since it is the non-obvious part.
Verified locally: tsc --noEmit clean, bun run build exits 0, lint clean.
| effectiveIsStreaming, | ||
| selectedAgent, | ||
| agentStop, | ||
| rejoinAgentStream, |
There was a problem hiding this comment.
Declare the rejoin callback before the resume hook
This dependency array is evaluated during render, but rejoinAgentStream is only initialized later by useAgentChannelMultiplayer in the same component. Rendering GlobalAssistantView therefore hits the temporal dead zone before the later hook runs; move the multiplayer hook above this useAppStateRecovery call or use the existing ref when wiring the resume callback.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed and fixed in c2cd36d — this was a real build break, not a nit: CI failed web#build with Type error: Block-scoped variable 'rejoinAgentStream' used before its declaration.
Fixed by routing the agent rejoin through the existing rejoinAgentStreamRef — declared near the top of the component and assigned immediately after useAgentChannelMultiplayer runs. That ref exists for exactly this hook-ordering problem and is already the in-file idiom: tryRecover calls rejoinAgentStreamRef.current() the same way. I preferred it over reordering the hooks, since hoisting useAgentChannelMultiplayer would drag its dependencies (setAgentMessages, loadConversation, …) up with it for no benefit.
Also collapsed the agent/global stop branch to the already-mode-selected rawStop.
Verified locally: tsc --noEmit clean and bun run build exits 0 (both previously failed).
82fd84b to
710eb7f
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
apps/web/src/components/layout/middle-content/page-views/dashboard/GlobalAssistantView.tsx (1)
710-751: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy liftResume-recovery logic is duplicated with
SidebarChatTab.tsx.This entire block (comments,
resumeEnabled,onResumeaction resolution and rejoin/refresh sequencing) is near-identical toSidebarChatTab.tsxLines 622-662. The two already diverged once (this file needed a ref to fix a TDZ that the sidebar never had), which is exactly the kind of drift risk duplicated logic creates. Consider extracting a shareduseResumeStreamRecoveryhook (native check, action resolution, stop/rejoin/refresh sequencing) parameterized bystop,rejoinAgent,rejoinGlobal,refresh,isStreaming,selectedAgent.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/web/src/components/layout/middle-content/page-views/dashboard/GlobalAssistantView.tsx` around lines 710 - 751, The resume-recovery implementation around resumeEnabled and the useAppStateRecovery onResume callback duplicates SidebarChatTab.tsx and should be centralized. Extract a shared useResumeStreamRecovery hook that owns native action resolution, enabled gating, stop, agent/global rejoin, and refresh sequencing, parameterized by stop, rejoinAgent, rejoinGlobal, refresh, isStreaming, and selectedAgent; then replace both component-local blocks with the hook while preserving the ref-based agent rejoin behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@apps/web/src/components/layout/middle-content/page-views/dashboard/GlobalAssistantView.tsx`:
- Around line 710-751: The resume-recovery implementation around resumeEnabled
and the useAppStateRecovery onResume callback duplicates SidebarChatTab.tsx and
should be centralized. Extract a shared useResumeStreamRecovery hook that owns
native action resolution, enabled gating, stop, agent/global rejoin, and refresh
sequencing, parameterized by stop, rejoinAgent, rejoinGlobal, refresh,
isStreaming, and selectedAgent; then replace both component-local blocks with
the hook while preserving the ref-based agent rejoin behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 6926b1ce-08db-4c3f-b511-f486e6395731
📒 Files selected for processing (2)
apps/web/src/components/layout/middle-content/page-views/dashboard/GlobalAssistantView.tsxapps/web/src/components/layout/right-sidebar/ai-assistant/SidebarChatTab.tsx
Self-review found a blocker the PR itself introduced — fixed in 496e0a7Running an adversarial pass over my own diff surfaced that this PR was fixing only half of its own stated "Defect 2". It added the stop+rejoin, but left the DB refresh as the same dumb, unguarded fetch — and then wired that fetch to fire on the rejoin path, where it is at its most dangerous. The reply is not persisted until the run completes, so a refresh issued while a stream is live returns a snapshot with no assistant message. Two distinct problems fell out of that: 1. The write was unguarded.
Both paths now funnel through guarded, reconciled loaders. 2. The order was racy. Concretely (agent mode, the worst variant): resume at T issues the GET; the rejoined stream completes at T+150ms and renders the reply; the GET from T resolves at T+800ms (first request after a mobile resume is routinely slow) and wipes it. Global mode self-heals via the completion→ The order is now stop → await refresh → rejoin, so no DB request is ever outstanding when the stream completes: the refresh catches a run that finished while backgrounded, and the rejoin attaches to one that is still live. Also
Known follow-up (not this PR)
Validation
|
Rewrote the fix: resume now goes through
|
Round 3 found the PR didn't actually work — fixed in 5fefc3cThird adversarial pass. The rejoin was attaching correctly and then rendering nothing. Two findings, both real. 1. Dedup collision — the blockerThe server mints one id and uses it for both the assistant UI message and the stream registry row: const serverAssistantMessageId = createId(); // app/api/ai/global/[id]/messages/route.ts:1027
…
messageId: serverAssistantMessageId, // :1089 → aiStreamSessions row
generateId: () => serverAssistantMessageId, // :1119 → the UI messageSo the id useChat holds for the half-streamed bubble is the live stream's The rejoin then re-adds that same stream to the pending store under the same id — and both surfaces drop a pending stream whose Net effect: we stopped the fetch, rejoined the server stream, and then displayed zero of its tokens. The user would sit in front of a frozen partial reply until completion. The PR was a no-op on its own headline path.
2. The DB fallback fired in exactly the cases where a DB write is unsafe
The native path now ends at TestsAdded coverage pinning the eviction (the dedup collision) and asserting the native path never contains a Validation
Known follow-ups (deliberately not in this PR)
|
Round 4: the eviction was being undone, and could leave the screen empty — fixed in 13391511.
|
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@apps/web/src/components/layout/middle-content/page-views/dashboard/GlobalAssistantView.tsx`:
- Around line 780-833: Extract shared resumeEnabled and stale-partial eviction
helpers, then replace all local mirrors with imports so production code and
tests exercise the same implementations. In
apps/web/src/components/layout/middle-content/page-views/dashboard/GlobalAssistantView.tsx:780-833
and
apps/web/src/components/layout/right-sidebar/ai-assistant/SidebarChatTab.tsx:1009-1067,
import and use the shared resumeEnabled while preserving the callback’s
conversation/editing checks. In
apps/web/src/components/layout/middle-content/page-views/dashboard/__tests__/GlobalAssistantView.test.tsx:115-162
and 334-360, and
apps/web/src/components/layout/right-sidebar/ai-assistant/__tests__/SidebarChatTab.test.tsx:133-180
and 613-639, remove the resumeEnabled/ResumeEffect/planResume and
evictStalePartial mirrors and import the shared implementations used by the
views and tryRecover.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 3756dac7-a648-4a24-b4c2-14990119204c
📒 Files selected for processing (4)
apps/web/src/components/layout/middle-content/page-views/dashboard/GlobalAssistantView.tsxapps/web/src/components/layout/middle-content/page-views/dashboard/__tests__/GlobalAssistantView.test.tsxapps/web/src/components/layout/right-sidebar/ai-assistant/SidebarChatTab.tsxapps/web/src/components/layout/right-sidebar/ai-assistant/__tests__/SidebarChatTab.test.tsx
Round 5 — happy path verified clean; two fixes in e6ac419Fifth adversarial pass traced the full native path end-to-end (background mid-stream → foreground → stop → probe → evict → rejoin → bootstrap → SSE replay → render → completion) and verified it correct, including:
Two things still needed fixing. 1. The stop suppressed the fallback — a regression against master
if (!error || status !== 'error' || isRetryingRef.current) return;So the Failure: iOS backgrounds during the TTFB window and the POST never reaches the server (a radio drop right on the background transition is common). On resume: stop aborts the dead POST → Master handled this: the dead fetch raised an error →
I'd also written that the reconnect/ 2. The eviction gate counted parts differently from the seederThe gate used raw Validation
|
Round 6 — the regenerate fallback could destroy a healthy run. Fixed in 25746fb (+1e13c8d15)Two fixes to the fallback I added last round. 1. Silence is not an answer (ship-blocker)
I was treating the second as if it were the first. And on this path the second is common, not exotic — the first request after an iOS foreground is the one most likely to fail, with the radio still coming up. It matters because every generation start calls
On persistent silence it does nothing, which is safe: the 2. The regenerate gate was not conversation-scoped
Both files already carry the conversation-scoped flag for exactly this hazard ( Verified sound (not defects)
Tests
Validation
|
Round 7 — the probe signal was leaky three ways. Fixed in 9bb092fAll three landed on the same failure: regenerating over a healthy run, which
All three are gone by construction: Also fixed: Verified sound this round
Tests
Validation
|
Round 8 — the fallback could DELETE a completed reply. Fixed in 264055dThree defects; two of them combine into the worst thing this PR could do. 1. Silence from the DB check was read as "nothing is persisted"Round 7 taught the If the messages GET throws or returns non-ok, we learned nothing about what is persisted. The code treated that as "no reply exists" and regenerated. But So a run that finished while backgrounded would have its reply deleted from the DB and regenerated from scratch: content loss, plus a second bill.
2. The "was the user's turn persisted?" guard was broken by paginationIt compared user-message counts ( Past one page that guard is permanently false. Step 2 could never recover on a long conversation, and every interrupted turn there fell through to the regenerate in (1) — deleting the very reply it should have refetched. Now asked by identity: does the DB contain the id of our last local user message? That message is by definition the newest, so it is always inside the returned window. Correct and pagination-proof. 3.
|
ac332b3 to
80a8db1
Compare
Rebased onto master, and the last of the duplication is goneRebase (master had landed in these same files)Master picked up two commits touching
Duplication (CodeRabbit's thread, and every self-review round before it)The resume gate and the eviction rule existed in six copies — once per component, then hand-mirrored again in each test file — and the mirrors could not fail. Both are now single pure modules the components and the tests share: One thing the extraction got wrong on the first pass, caught on review: it swapped the eviction from a Validation
|
Final review pass — one real gap closed (7a6c2c8)A fresh end-to-end read of the final state. The core design verified sound — the eviction has real server data behind it ( 1.
|
The double-regenerate fix was incomplete — real mutex now (e2268d6)Verifying the previous commit surfaced that the
Closed with an actual lock: one The status guard stays as the belt to this mutex's braces: the lock releases when It's a mutex, not a one-shot latch — released in a Also fixed a stale comment referencing Validation
|
Both GlobalAssistantView.tsx and SidebarChatTab.tsx had two defects in their useAppStateRecovery wiring that orphaned AI streams on mobile: 1. `enabled` was a render-time boolean `!isStreaming` — iOS freezes JS on background, so the captured value was `false` (streaming). Recovery was gated off in exactly the case it was written for. 2. `onResume` was a dumb DB fetch — no local stop, no server rejoin. Even if recovery fired, it would fetch a stale snapshot (reply not persisted yet) and clobber the in-progress bubble. The fix matches AiChatView.tsx's proven pattern exactly: - `enabled` -> callback form (evaluated at fire time, gates on editing only) - `onResume` -> resolveResumeAction: on native always returns 'rejoin-and-refresh', which stops local useChat + rejoins server stream + refreshes from DB
Two build-breaking defects from the previous commit, both flagged by Codex:
1. GlobalAssistantView referenced `rejoinAgentStream` in the resume callback,
but that binding is only initialized further down by useAgentChannelMultiplayer
— a temporal dead zone that failed the type-check ("Block-scoped variable
'rejoinAgentStream' used before its declaration") and broke `web#build`.
Use the existing `rejoinAgentStreamRef`, which exists for exactly this
hook-ordering problem and is already the in-file idiom in tryRecover.
2. SidebarChatTab called `resolveResumeAction` and `isCapacitorApp` without
importing either. Added both imports.
Also collapse the agent/global stop branch in GlobalAssistantView to the
already-mode-selected `rawStop`, matching the local-only stop semantics
documented in AiChatView.
The resume wiring had zero coverage on either surface — which is how the render-time `enabled` boolean survived, and how it regressed a second time. Follows the file's established pattern (pure mirrors of the hook-heavy component's logic), but the decision itself is NOT mirrored: planResume calls the real resolveResumeAction, so these tests pin the wiring against the real policy rather than a copy of it. Covers: - resumeEnabled takes NO streaming argument — the regression is now inexpressible in the signature; it gates on user-editing only. - native mid-stream => stop + rejoin (agent or global) + refresh - native idle => still stops and rejoins (deterministic, not flag-gated) - web mid-stream => noop (a live fetch survives a tab switch) - web idle => refresh only
Found by adversarial self-review, not by a reviewer. The PR fixed only HALF of its own stated "Defect 2": it added the stop+rejoin, but left the DB refresh as the same dumb, unguarded fetch — and then wired it to fire on the rejoin path, where it is at its most dangerous. The reply is not persisted until the run completes, so a refresh issued while a stream is live returns a snapshot with NO assistant message. Two consequences: 1. Unguarded write. GlobalAssistantView.handlePullUpRefresh did a raw fetch -> setMessages, and SidebarChatTab.handleAppResume bypassed loadGlobalMessages — the documented "single writer for the global-mode server->view path" sitting directly above it. Neither carried the stale-response check or mergeServerAndPending, so the pre-reply snapshot went straight over the live assistant bubble. Both now funnel through guarded, reconciled loaders. 2. Racy order. The refresh was fired AFTER the rejoin, so its request could still be in flight when the rejoined stream completed — landing last and overwriting the just-completed reply. In agent mode nothing heals that (there is no completion-triggered refetch), leaving the reply invisible until the user reselected the conversation. The order is now stop -> await refresh -> rejoin, so no DB request is ever outstanding when the stream completes: the refresh catches a run that finished while backgrounded, the rejoin attaches to one still live. Tests updated to pin the ordering (refresh strictly before rejoin), and the gate's regression test now asserts on arity so an `isStreaming` parameter cannot be reintroduced — the previous assertion was vacuous.
…ually holds The previous commit ordered the resume as stop -> await refresh -> rejoin, but in the sidebar's global path the "await" was a no-op: loadGlobalMessages returned void and fired its fetch as a floating promise, so handleAppResume resolved immediately and the rejoin ran with the DB request still outstanding — exactly the race the reordering was meant to close. loadGlobalMessages now returns the promise chain, and handleAppResume awaits it. The other four call sites (load-on-select, refreshSignal, retry) still ignore the result; they only need to kick a refresh off, not sequence against it.
…tream lookup Two defects in my own previous commit, both found by self-review. 1. Vacuous stale guard. I copied the "id the load was requested for" ref pattern from AiChatView/SidebarChatTab, but that pattern only works there because every load path funnels through the single loader that advances the ref — so switching conversation advances it and the stale response is dropped. GlobalAssistantView does NOT load on select through handlePullUpRefresh (it uses the globalInitialMessages / agent-load-signal effects), so nothing else ever moved the ref. It always equalled the id its own in-flight fetch was issued for, the guard always passed, and a response for the conversation the user just left would still be applied to the one they switched to. Now guards against the LIVE conversation (currentConversationIdRef, mirrored every render), which is the invariant that actually holds on this surface. 2. Unscoped pending-stream lookup. The merge matched on `isOwn && conversationId` alone. A conversation id is only unique within its channel, and this surface switches between the global channel and per-agent channels, so that could splice a different agent's in-flight bubble into this conversation. Now scoped by channel via the store's own getOwnStreams(channelId), mirroring AiChatView.
Round-2 self-review found the previous approach was still wrong, and in a way that made the primary symptom worse. An OWN stream is deliberately never in the pending-streams store while its POST body is being consumed (markChannelConsuming -> shouldAttachStream returns false, so chat:stream_start is skipped). So on the resume path the store is EMPTY for the stream we are recovering, which means mergeServerAndPending had nothing to merge and was inert. The "guarded" refresh therefore still wrote a pre-reply DB snapshot straight over the half-streamed assistant bubble — the partial reply vanished for a full round-trip until the bootstrap re-seeded it. The very bug the ordering was supposed to prevent, just moved. The mistake was reaching for a DB read at all. Both surfaces already have `tryRecover` — the rejoin-first probe useStreamRecovery uses on a network error — and a background/foreground cycle IS a network error on iOS, just one we are told about. It asks /active-streams first (the server's authoritative answer) and only touches the DB when nothing is live: live stream -> rejoin it, no DB read at all already persisted -> refetch the completed reply neither -> fall through to the plain refresh onResume now stops the local fetch (which also releases the channel's `consuming` mark, without which the rejoin's bootstrap would skip attaching) and delegates to tryRecover. No bespoke ordering, no inert merge, and the DB is never read while a run is still generating. SidebarChatTab's useAppStateRecovery moves below tryRecover, which it now closes over. handlePullUpRefresh keeps its stale-guard + reconciliation: it is still reachable from the pull-up and refreshSignal paths, where a late-landing load can clobber a conversation the user has since switched away from. Tests reworked to pin the real invariant — on the rejoin path the DB is NOT read unless tryRecover comes back empty. Dropped the vacuous arity assertion.
…an unsafe DB refresh
Round-3 self-review found the rejoin was attaching and then rendering NOTHING.
1. Dedup collision (blocker — the PR did not actually work).
The server mints ONE id and uses it for both the assistant UI message
(`generateId: () => serverAssistantMessageId`) and the stream registry row, so
the id useChat holds for the half-streamed bubble IS the live stream's
messageId. `Chat.stop()` documents that it "keeps the generated tokens", so
that bubble survives the stop — and the rejoin then re-adds the same stream to
the pending store. Both surfaces drop a pending stream whose messageId already
appears in `messages` (dedupRemoteStreams / ChatMessagesArea), so the rejoined
stream was filtered straight back out: not one token would render, and the user
would sit in front of a frozen partial until completion.
tryRecover's rejoin branch now reads the live stream's messageId off
/active-streams and evicts the matching local message before rejoining. Nothing
is lost: the bootstrap seeds the stream from the server's registry buffer, which
holds every part pushed so far — strictly more than the partial we froze with.
This also fixes the same latent bug on the network-error rejoin path.
2. Unsafe DB fallback (clobber).
onResume fell through to a blind DB refresh whenever tryRecover returned false —
which is exactly the two cases where a DB write is UNSAFE:
- the /active-streams probe FAILED (the first request after a foreground is
the likeliest to, radio still coming up). A stream may well be live, and the
DB snapshot cannot contain an unpersisted reply, so the refresh erased the
in-progress bubble.
- the DB is BEHIND local state (step 2's dbUserCount >= localUserCount guard
rejected it) — e.g. a send whose POST never reached the server. The refresh
erased the user's own prompt.
The native path now ends at tryRecover, which already refetches when the run
finished while we were away. Doing nothing in the other cases is correct: local
state is newer than anything we could fetch, and the reconnect path heals it.
Also corrected two comments that still described the previous (abandoned) ordering.
…t the raw useChat one Self-review of the previous commit: GlobalAssistantView's eviction called the raw setAgentMessages/setGlobalLocalMessages, which only touch useChat. The component has agentSetMessages/globalSetMessages for exactly this — agent mode must also drop the message from the dashboard store (setConversationMessages), or the store keeps the stale partial and can serve it back to a co-mounted surface. Matches how tryRecover's step 2 already writes both.
…inst an empty checkpoint Round-4 self-review found the previous commit's eviction was being undone, and could also leave the user with nothing at all. 1. mergeServerAndPending re-inserted the very id the eviction removed. It synthesizes an assistant message under the pending stream's messageId when that id is absent from the DB snapshot — which is exactly the id both renderers dedup on. So after a rejoin, the next refreshSignal (a socket reconnect is near-certain right after an iOS foreground) ran handlePullUpRefresh, merged a FROZEN snapshot of the stream's parts into `messages` under that id, and the live pending stream was deduped away again. The bubble stopped updating until completion — the exact symptom the eviction exists to prevent. On these surfaces the pending store IS the bubble: an in-flight stream renders from `remoteStreams`. Merging a static copy of it into `messages` is not needed for visibility and actively kills the live render. The merge is removed — which also restores master's behaviour here, since master never had it. (I had added it two commits ago while trying to make a mid-stream DB read safe; the read is gone now, so the guard it needed is gone too.) 2. Evicting against an empty checkpoint could leave NOTHING on screen. /active-streams `parts` is the registry's DEBOUNCED checkpoint (persisted every N parts), so it is empty for a stream only a few parts old. If we evicted the local partial and the SSE join then failed — the documented multi-instance case, where the multicast lives in another process — the bootstrap removes the stream and the user is left with nothing, strictly worse than the frozen partial we started with. Eviction is now gated on the server actually having parts to render in its place; otherwise we keep the partial and still attempt the rejoin. Tests pin both: eviction only when the checkpoint is non-empty, and never touching the user's turn.
…ch the seeder's part count Round-5 self-review. The happy path traced clean end-to-end, but two things remained. 1. The native resume path swallowed the "nothing to recover" case (regression vs master). `Chat.stop()` aborts the fetch, so useChat settles at `ready` with NO `error` — and useStreamRecovery only fires on `status === 'error'`. So the stop we added destroys the very signal that used to drive the fallback, and tryRecover's false return was being discarded. A turn whose POST died on the background transition (a radio drop right then is common) would find no live stream, no persisted reply and no error: the user's prompt sat unanswered forever, with no retry and no banner. Master recovered it — the dead fetch raised an error, useStreamRecovery probed, came up empty, and regenerated. onResume now falls through to handleRetry() when tryRecover comes up empty — the same fallback useStreamRecovery applies on the network-error path. Still NOT a DB refresh, which remains unsafe here for the reasons documented at the call site. Gated on a turn actually having been in flight when we backgrounded, so an ordinary resume on an idle conversation can never fire a spurious generation. (The frozen render-time streaming flag is a faithful record of that, which is all it is used for — deciding whether the TRANSPORT is alive is still resolveResumeAction's job.) 2. The eviction gate counted parts differently from the bootstrap that seeds them. The gate used the raw `parts.length`, but the bootstrap seeds `(stream.parts ?? []).filter(isValidPartFrame)` and it is THAT count which becomes skipReplayCount — and a skipReplayCount of 0 is what makes a failed join drop the stream. A checkpoint of malformed frames would therefore read as "safe to evict" while seeding nothing, and a failed SSE join would leave the user with an empty screen: exactly what the gate exists to prevent. Both now use isValidPartFrame. Also corrected the comment that claimed the reconnect/refreshSignal path heals every skipped case — it does not heal a run that never started server-side.
Proactive fix, ahead of review. The new handleRetry() fallback was gated on effectiveIsStreaming/displayIsStreaming, which is NOT "a turn of ours was in flight for the conversation on screen". Both flags fold in a stream that is still running against a conversation the user has since navigated away from — the useChat instance has a stable id, so it keeps reporting streaming across a conversation switch, for the OLD conversation's in-flight request. Regenerating on the strength of that would fire a generation for the turn the user is now LOOKING at rather than the one that was actually interrupted: a spurious reply, and a spurious charge, on an untouched conversation. Both files already carry the conversation-scoped flag for exactly this hazard (isOwnAgentStream/isOwnGlobalStreamForCurrentConversation on the dashboard, isOwnStreamForCurrentConversation in the sidebar — each latching the conversation the stream actually started in). The gate now uses it.
…answer Round-6 self-review found the regenerate fallback could destroy a healthy run. tryRecover returns false for two OPPOSITE outcomes: "the server says nothing is live" -> the run died; regenerating is the recovery "the probe never reached the server" -> we know NOTHING; a run may still be live and the resume path was treating the second as if it were the first. The first request after a foreground is the one most likely to fail (cold radio), so on this path that case is common, not exotic. It matters because every generation start calls takeOverConversationStreams. A regenerate issued while the run is in fact still live does NOT race it — it ABORTS it. So the fallback would have: killed a healthy, possibly nearly-finished generation; re-run any write tools it had already executed (a turn that created a page creates it twice — the side effects are not rolled back); billed the discarded tokens; and stranded its partial in the DB, since onFinish persists the aborted response AFTER handleRetry's delete of the trailing assistant. tryRecover now records reachability separately from its verdict (only a 2xx means the server actually told us what is live). onResume re-probes twice, backing off, before concluding — the radio returns well inside that — and regenerates ONLY on an answered probe. On persistent silence it does nothing, which is safe: the stop released this channel's `consuming` mark, so a live run is picked up by the socket-reconnect bootstrap once the network returns, and a dead one leaves the user their prompt and the retry action on it. Tests: planResume now models ownTurnInFlight and probeAnswered as inputs distinct from the broad streaming flag, so neither the conversation-scoped gate nor the probe-reachability gate can silently regress. Both were previously unfalsifiable.
… ref
Round-7 self-review found the probe-reachability signal was leaky in three ways,
all landing on the same failure — regenerating over a healthy run, which
takeOverConversationStreams turns into an abort of it.
1. `probeAnsweredRef.current = res.ok` was set BEFORE the body was parsed. res.json()
can still throw on a body that dies mid-read — which is exactly the
cold-radio-after-foreground case this path exists for — and the catch swallowed it
while we went on believing we had an answer. Now set only after a body we actually
parsed.
2. One shared mutable slot, two callers. tryRecover is called by BOTH this surface's
resume handler and useStreamRecovery's network-error retry, and they can be in
flight together — so one caller's probe could answer the other caller's question.
3. The reset sat after tryRecover's early returns, so a bail-out (`!channelId`, e.g.
`user` transiently null while auth rehydrates on foreground — the token very likely
expired while backgrounded) left the ref holding a PREVIOUS call's `true`, and the
resume path would regenerate on the strength of a probe it never made.
All three are gone: tryRecover now RETURNS `{ recovered, probeAnswered }`
(RecoveryAttempt), so each caller reads its own probe and there is no shared state to
race or go stale. useStreamRecovery takes a thin adapter reading only `.recovered` —
the reachability half of the answer is for the resume path, which is the one that
would otherwise regenerate over a live run.
Also: step 2 read defensively (`Array.isArray(data) ? data : data.messages`) but wrote
`data.messages` raw, which would blank the surface outright if the route ever answered
with a bare array. It now writes the same normalized array the guards were computed
from.
Tests: planResume takes a SEQUENCE of attempts, so the re-probe loop is modelled — a
probe that only answers on a later attempt still regenerates (a cold radio must not
strand the turn), and one that never answers never does.
Round-8 self-review. Three defects, two of which combine into the worst outcome this PR could produce: DELETING a completed reply and paying to generate it again. 1. Silence from the DB check was read as "nothing is persisted". Round 7 taught the probe to distinguish "the server says nothing is live" from "the probe got no answer". Step 2 had exactly the same hole and it is the more destructive one: if the messages GET throws or returns non-ok, we learned nothing about what is persisted — but the code treated that as "no reply exists" and regenerated. handleRetry DELETEs the trailing assistant BY ID, and that id is the one the server persisted the reply under (serverAssistantMessageId names the registry row, the UI message and the DB row alike). So a run that FINISHED while backgrounded would have its reply deleted from the DB and regenerated from scratch: content loss plus a second bill. RecoveryAttempt now carries dbAnswered alongside probeAnswered, and canConcludeTurnIsLost requires BOTH. The resume loop re-asks until both come back. 2. The "was the user's turn persisted?" guard was broken by pagination. It compared user-message COUNTS (dbUserCount >= localUserCount). tryRecover's GET is unpaginated, so the route applies its default limit of 50 and returns only the newest 50 rows — while local `messages` is the initial 50 PLUS every turn since. Past one page the guard is PERMANENTLY false, so step 2 could never recover on a long conversation, and every interrupted turn there fell through to the regenerate in (1) — deleting the very reply it should have refetched. Now asked by identity: does the DB contain the id of our last local user message. The last user message is by definition the newest, so it is always inside the returned window. Correct and pagination-proof. 3. tryRecover's writes had no stale-conversation guard. It writes after two awaits into a useChat instance whose id is constant across conversation switches, so a recovery for the conversation the user just left could land in the one they moved to. Every sibling loader guards this (handlePullUpRefresh, loadGlobalMessages); tryRecover now does too, against the live conversation.
…d-recovery Proactive, ahead of review. tryRecover now refuses to WRITE into a conversation the user has navigated away from — but the regenerate that follows it had no such guard, and handleRetry always acts on the LIVE conversation. The recovery spans up to a few seconds of network (three bounded probes), and the user can switch conversation inside that window. The turn we set out to recover belongs to the conversation we started from; regenerating after a switch would fire a generation for the turn they moved TO instead — a spurious reply, and a spurious charge, on an untouched conversation, and it would delete that conversation's trailing assistant message on the way in. onResume now latches the conversation the interrupted turn belongs to and re-checks it against the live one before retrying.
… the real ones CodeRabbit, and every self-review round before it, kept landing on the same thing: the resume gate and the eviction rule existed in six copies — once per component, then hand-mirrored again in each test file — and the mirrors could not fail. A component regression would have left the suite green. Both are now single pure modules that the components and the tests share: - canResumeRecovery(conversationId, isAnyEditing) — the useAppStateRecovery gate. It takes no streaming argument BY CONSTRUCTION, which is the whole point: the original bug was a render-time boolean folding in `!isStreaming`, captured before iOS froze JS and therefore always reporting the streaming state we went away in. - evictStalePartial(messages, liveMessageId, serverParts) — owns both halves of the rule: drop the frozen bubble whose id collides with the rejoined stream (or the dedup filters that stream straight back out and it renders nothing), but ONLY when the server's debounced checkpoint has frames the bootstrap can actually seed from, counted with the same isValidPartFrame predicate the bootstrap uses. It returns the same reference when it declines, so callers skip the write entirely. The tests now import both, so they assert against the shipped implementation rather than a copy of it. Added direct coverage for the case only the real helper can have: a checkpoint of malformed frames counts as empty, because that is what the bootstrap would seed from.
The extraction in the previous commit swapped the eviction from a setMessages UPDATER to a value computed from currentMessagesRef, so it could compare references and skip a no-op write. That was the wrong trade in the sidebar. In GlobalAssistantView the two are equivalent — agentSetMessages/globalSetMessages resolve an updater against their OWN refs and always call the setters with a value, so nothing was reading React's `prev` there anyway. But SidebarChatTab's setters are the raw useChat ones, and passing them an updater resolves it against the AI SDK Chat store's LIVE messages array, which the transport and useAgentChannelMultiplayer write synchronously without waiting for a render. Reading a render-time ref instead opened a lost-update window: a store write landing after the last render and before the fetch continuation would be clobbered by our value write. Split the rule instead of weakening it. `canEvictStalePartial(serverParts)` is now exported, so the call sites can ask BEFORE writing — an unsafe checkpoint still costs no state write — and the eviction itself goes back through the updater form, so the filter runs against the freshest list. evictStalePartial stays self-guarding on the same predicate, so the rule holds even for a caller that forgets to ask, and a test pins the two against each other so they cannot drift.
Final review pass. Three findings, one of them a real gap.
1. onResume and useStreamRecovery could BOTH regenerate the same turn.
They watch the same failure from opposite sides — useStreamRecovery fires on
`status === 'error'`, onResume on the app-resume event — and share no lock. If
iOS delivers the dead fetch's rejection as an error before the resume listener
runs, useStreamRecovery may already have retried. rawStop() cannot have cancelled
it: Chat.stop() returns early unless the status is streaming/submitted, so on an
errored chat it is a no-op. onResume would then regenerate on top — exactly the
double destruction its own comments warn about, since takeOverConversationStreams
aborts the run that just started and handleRetry deletes its assistant message on
the way in.
The gate now also requires that nothing has restarted the turn, read through a ref
because the closure's copy of `isStreaming` is the value captured before we were
frozen.
2. A comment that lied, and contradicted a test in the same PR. It still described
"neither → falls through to handlePullUpRefresh", which is the DB-refresh fallback
deliberately removed two commits earlier — and which the tests explicitly assert
never happens ("never a DB refresh").
3. Dead expression. Step 1's guard called decideRecovery with hasLiveStream hardcoded
true, so it could only ever answer 'rejoin' — decoration dressed as a decision.
The rejoin-first priority is expressed by the ORDER of the two steps; step 2 is
where decideRecovery genuinely decides.
Verification of the previous commit showed its `nothingHasRestarted` guard was an incomplete fix. useStreamRecovery calls clearError() BEFORE running its own probes, so for the entire window in which it is deciding whether to regenerate, the chat status reads `ready` and looks idle to us — the status check passes and both paths can still call handleRetry for the same turn. The status guard only catches the sub-case where the other path has already reached a running generation. Close it with an actual lock: a single regenerationInFlightRef wraps handleRetry as regenerateTurnOnce, and BOTH callers go through it — useStreamRecovery (via its handleRetry prop) and the resume handler. Held across the whole of handleRetry, whose message DELETEs are a network round-trip. The status guard stays as the belt to this mutex's braces: the lock releases when handleRetry returns, but the generation it started is still running, and that generation's own `submitted` status is what keeps the other path out afterwards. The mutex is a mutex, not a one-shot latch — it releases in a finally, so a later genuine failure can still recover, and a throwing handleRetry does not wedge it shut. Tested with a real concurrent-invocation harness (both paths firing while the first is still awaiting): exactly one regenerate. Also fixed a stale comment referencing `rawStop()` in SidebarChatTab, which has no such binding — it calls the (equally local-only) `stop()`.
Rebase onto master picked up #2073, which moves the stream registry checkpoint from a part-count cadence to a time-based one. The eviction logic is unaffected — it gates on whether the checkpoint has isValidPartFrame frames, whatever the cadence — but the comments describing it as "persisted every N parts" / "debounced" are now stale. Reworded to the durable truth: the checkpoint lags the live stream, so it is empty in a stream's first moments.
e2268d6 to
a268a6a
Compare
…uck-streaming gaps Addresses two CodeRabbit findings: 1. Global route had no execute-end durable-persist block at all (only chat/route.ts did). If the client disconnected after generation (mobile backgrounding — the exact scenario #2065 just fixed client-side for this same route), onFinish might never fire and the placeholder stayed 'streaming' forever. Added an execute-end block to the global route mirroring chat/route.ts's: unconditional, using whatever lifecycle.getBufferedParts() has, status aborted ? 'interrupted' : 'complete', plus the same conversations.lastMessageAt bump. onFinish's own no-responseMessage branch is now a no-op (execute-end has already terminalized the row by the time it would run) — replaced the duplicate persist logic there with a debug log. 2. Both routes' execute-end persist was still gated on `bufferedParts.length > 0 || aborted`. A run that exhausted its retries without ever aborting or producing a responseMessage (sustained provider outage) fell through that guard AND onFinish's `if (responseMessage)` guard, leaving the placeholder stuck at 'streaming' forever — same failure class as the abort gap already fixed, just for a third path (clean exit, no content) neither prior fix covered. Both routes' execute-end write is now unconditional. Updated/added tests in both routes' stream-socket-events.test.ts for: non-aborted empty-buffer persists as 'complete' (chat), aborted-with-content persists via execute-end (global, migrated off the now-inert onFinish path), lastMessageAt bump via execute-end (global), and non-aborted-with-content persists as 'complete' via execute-end (global). Fixed global credit-gate.test.ts's lifecycle mock (missing getBufferedParts, now called unconditionally). Verified each new/changed assertion against the pre-fix code before committing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QQTjjDoYto6GHcBBfWEnB
…aming|complete|interrupted) (#2076) * feat(ai): assistant message row at stream start + status column (streaming|complete|interrupted) Adds status text NOT NULL DEFAULT 'complete' (streaming|complete|interrupted) to chat_messages and messages, and inserts an empty assistant placeholder row the moment generation starts (inside startGenerationExclusive's run closure, same critical section as takeover+lifecycle-create per the PR 4 seam note). Reader inventory (~30 sites) updated to exclude 'streaming' placeholders: - Model-context/compaction loads: page + global history loads, chatMessageRepository (v1/completions, page-payload-service), ask_agent, consult route (+ fixed a pre-existing missing isActive filter on its fallback branch), page-read-tools, ask-user-resume's fetchById/fetchLastAssistant (both page + global adapters). - Previews: conversation-repository's raw-SQL CTEs (last-message preview + count). - Mutations: edit/delete of a streaming row now 409s (chat + global [messageId] routes). Undo: the undo-vs-in-flight-stream UX call (409 whole undo / allow+hidden write / allow+abort) is a product decision, left open — ai-undo-service now excludes streaming rows from its sweep entirely (preview count + both soft-delete updates) as the safe interim default, so undo can never soft-delete a live stream's row. - Converters: convertDbMessageToUIMessage / convertGlobalAssistantMessageToUIMessage (all branches, including a reconstruction-success branch that previously dropped extra fields entirely) now propagate status. - Read APIs: chat/messages, global/[id]/messages, page-agents conversation messages, and v1/conversations/[id] GET all gate streaming rows behind includeStreaming=1. Client dedup: mergeServerAndPending (shared by AiChatView and SidebarChatTab) now replaces a server-loaded streaming placeholder with the richer live stream version instead of losing to the empty DB row. - Exports: tenant-export's column lists and gdpr-export's collectUserMessages both carry status now. Terminal writes: saveMessageToDatabase / saveGlobalAssistantMessageToDatabase's onConflictDoUpdate sets status: 'complete' (execute-end + onFinish, matching the existing durable-persist / best-effort-refine split). 'interrupted' via abort/ materialization is PR 3's wiring, per the epic's own "contract fixed here" framing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QQTjjDoYto6GHcBBfWEnB * fix(ai): terminalize aborted/stopped streams as interrupted, not complete Addresses two Codex review findings on #2076: 1. saveMessageToDatabase / saveGlobalAssistantMessageToDatabase's onConflictDoUpdate always set status:'complete', even for a run the user (or credit gate) stopped — contradicting the status contract ('interrupted' = terminal, real partial output). Both now accept an explicit status param; execute-end and onFinish in both chat routes pass 'interrupted' whenever agentRun.terminalReason === 'aborted' or the route's abortSignal is aborted. 2. If a stream was stopped before any buffered part or responseMessage existed, no saveMessageToDatabase call ever touched the placeholder row — since streaming rows are hidden from every reader by default and edit/delete now 409 on them, that left an invisible, permanently-locked ghost row. Fixed by writing a (possibly empty-content) 'interrupted' row whenever the run was aborted, regardless of buffered content: - chat/route.ts: execute-end's guard widened from `bufferedParts.length > 0` to `bufferedParts.length > 0 || aborted`. - global/[id]/messages/route.ts: onFinish gained a fallback branch for `!responseMessage && aborted`, using lifecycle.getBufferedParts() the same way chat/route.ts's execute-end block does (this route had no execute-end persist block at all — a pre-existing architectural difference from chat/route.ts, not something this fix introduces). Also closes the same failure class for the rarer case where createUIMessageStream itself throws before execute()/onFinish ever run: both routes' outer catch now does a best-effort terminal write (status:'interrupted'), guarded by a new assistantMessagePersisted flag so it can never downgrade a row a successful execute-end/onFinish write already settled. New tests in both routes' stream-socket-events.test.ts covering: aborted-with-content → interrupted, aborted-with-zero-content → still persisted as interrupted, non-aborted → complete (regression guard), and outer-catch cleanup. Verified each new assertion fails against the pre-fix code (temporarily reverted, confirmed RED, restored) before committing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QQTjjDoYto6GHcBBfWEnB * fix(ai): preserve buffered content in abort cleanup; bump lastMessageAt; dedup aborted check Self-review (8-angle) on the previous fix commit surfaced two real bugs and one worthwhile simplification, independently confirmed by 3 separate finder angles: 1. CRITICAL: the outer-catch cleanup wrote in the wrong order — it called lifecycle.finish(true) BEFORE reading lifecycle.getBufferedParts(). finish() deletes the multicast registry entry getBufferedParts() reads from, so the cleanup write always persisted EMPTY content, silently discarding any real partial content the fix's own comment claimed to preserve. Fixed in both routes by capturing getBufferedParts() before finish(). chat/route.ts has a SEPARATE inner catch (createUIMessageStream construction failure) that also calls finish() before rethrowing to the outer catch — that inner catch now captures the buffer too (bufferedPartsAtStreamError), and the outer catch prefers that capture when set, since a fresh call by the time it runs would already see the inner catch's cleared buffer. 2. The global route's new onFinish fallback branch (no responseMessage, but aborted) didn't bump conversations.lastMessageAt the way the sibling responseMessage branch does — a conversation whose only new activity was an interrupted, no-responseMessage assistant row wouldn't surface in a lastMessageAt-sorted conversation list. Fixed to match the sibling branch. 3. `agentRun?.terminalReason === 'aborted' || abortSignal.aborted` was duplicated 3x across the two routes (with a comment on one copy claiming it was "computed once and reused" — true within its own closure, false across the file). Extracted to `isRunAborted()` in run-agent-with-retry.ts, which already owns the terminalReason vocabulary. New tests: both routes now have a regression test that reproduces the real finish()-then-getBufferedParts() ordering (mocked to return [] only after finish fires) and asserts the persisted uiMessage.parts still contain the pre-crash content; a lastMessageAt bump assertion for the fallback branch (call-count delta across the onFinish boundary, to avoid a false-pass from other db.update calls elsewhere in the same request); and a unit test suite for isRunAborted. All three new/changed assertions verified RED against the prior code before committing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QQTjjDoYto6GHcBBfWEnB * fix(db): regenerate status-column migration against latest master master gained two new migrations (0208_cloudy_ozymandias, 0209_sprite_reclaim_triggers) since this branch was last rebased, colliding with this PR's own 0208 migration number. Rebased onto origin/master, resolved the migration collision by keeping master's 0208/0209 journal entries and dropping this PR's stale 0208 migration + snapshot, then regenerated via `bun run db:generate` — the status column migration is now 0210_next_mariko_yashida.sql, journal-clean on top of master's latest. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QQTjjDoYto6GHcBBfWEnB * fix(ai): unconditional execute-end terminal write closes remaining stuck-streaming gaps Addresses two CodeRabbit findings: 1. Global route had no execute-end durable-persist block at all (only chat/route.ts did). If the client disconnected after generation (mobile backgrounding — the exact scenario #2065 just fixed client-side for this same route), onFinish might never fire and the placeholder stayed 'streaming' forever. Added an execute-end block to the global route mirroring chat/route.ts's: unconditional, using whatever lifecycle.getBufferedParts() has, status aborted ? 'interrupted' : 'complete', plus the same conversations.lastMessageAt bump. onFinish's own no-responseMessage branch is now a no-op (execute-end has already terminalized the row by the time it would run) — replaced the duplicate persist logic there with a debug log. 2. Both routes' execute-end persist was still gated on `bufferedParts.length > 0 || aborted`. A run that exhausted its retries without ever aborting or producing a responseMessage (sustained provider outage) fell through that guard AND onFinish's `if (responseMessage)` guard, leaving the placeholder stuck at 'streaming' forever — same failure class as the abort gap already fixed, just for a third path (clean exit, no content) neither prior fix covered. Both routes' execute-end write is now unconditional. Updated/added tests in both routes' stream-socket-events.test.ts for: non-aborted empty-buffer persists as 'complete' (chat), aborted-with-content persists via execute-end (global, migrated off the now-inert onFinish path), lastMessageAt bump via execute-end (global), and non-aborted-with-content persists as 'complete' via execute-end (global). Fixed global credit-gate.test.ts's lifecycle mock (missing getBufferedParts, now called unconditionally). Verified each new/changed assertion against the pre-fix code before committing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QQTjjDoYto6GHcBBfWEnB * fix(ai): 4 self-review findings from 8-angle diff review - ask-user-resume.ts's persist closures never passed status, silently resetting an 'interrupted' row back to 'complete' on every ask_user merge/dismiss write. Both page and global adapters now preserve the fetched row's status. - onFinish in both routes wrote a phantom empty 'interrupted' row for pre-aborted streams: the AI SDK always calls onFinish with a non-null responseMessage (an empty shell) even when execute() wrote nothing, but the placeholder INSERT is deliberately skipped when preAborted. Guarded on lifecycle.preAborted. - Outer-catch cleanup in both routes could fabricate a phantom row for an exception thrown before `lifecycle` was ever assigned (inside startGenerationExclusive's callback, before the placeholder INSERT ran) — now requires `lifecycle` itself, not just !preAborted. - Chat route's credit-gate abort was invisible to runAgentWithRetry's classification: the combined AbortSignal.any([...]) was only passed to the nested streamText call, not to runAgentWithRetry's own abortSignal param, so a mid-stream credit exhaustion was misclassified and persisted as 'complete' instead of 'interrupted'. - search-tools.ts and discovery-service.ts read chat_messages/messages without excluding status='streaming', a gap in the PR's own reader inventory. All fixes verified RED->GREEN. Full typecheck+build clean. * test(ai): strengthen lastMessageAt assertion to check target table, not just call count CodeRabbit round-4 review nitpick: the assertion only checked that db.update's call count increased, not that any of the new calls actually targeted conversations. Any unrelated update in the same code path would have made the test pass without exercising the lastMessageAt bump. Now asserts one of the new update calls targets the conversations table specifically. RED-verified by temporarily removing the lastMessageAt bump from apps/web/src/app/api/ai/global/[id]/messages/route.ts and confirming the assertion fails; restored and confirmed GREEN. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QQTjjDoYto6GHcBBfWEnB --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Summary
On mobile, when an AI stream is in progress and the app backgrounds, the stream keeps generating server-side (disconnect-immune by design). But the client couldn't rejoin it on foreground — both
GlobalAssistantView.tsxandSidebarChatTab.tsxhad defects in theiruseAppStateRecoverywiring that orphaned the stream.This is the bug behind the "invisible AI stream on mobile" issue. PR #2061 fixed the flashing/clobbering on send/reload/switch (client-side state machine). This PR fixes the recovery path that never fired.
Root cause
Defect 1:
enabledwas a render-time booleaniOS freezes JS the moment the app backgrounds. The boolean is captured at render — so when the app went away mid-stream,
!isStreamingwasfalse. The recovery hook was gated off in exactly the case it was written for.The
useAppStateRecoverydocstring documents this exact failure:That comment was written when
AiChatView.tsxwas fixed.GlobalAssistantViewandSidebarChatTabwere never updated.Defect 2:
onResumewas a dumb DB fetchBoth handlers just fetch messages from the DB and
setMessages. But the reply isn't persisted until the run completes — so mid-stream the fetch returns a snapshot with no assistant message, and writing it clobbers the in-progress bubble. No local stop, no server rejoin.The fix
enabled→ callback form — evaluated at fire time, gating on user-editing only, never on streaming.onResume→ stop, thentryRecover()— and critically, no blind DB read.Why
tryRecover, and not a refreshBoth components already have
tryRecover— the rejoin-first probeuseStreamRecoveryruns on a network error. A background/foreground cycle is a network error on iOS; it's just one we get told about. It asks/active-streamsfirst — the server's authoritative answer — and only touches the DB when nothing is live:/active-streamssaysEarlier iterations of this PR tried to make a mid-stream DB refresh safe (stale-guards,
mergeServerAndPending, careful ordering). That was the wrong tree to bark up: an own stream is deliberately never in the pending-streams store while its POST body is being consumed (markChannelConsuming→shouldAttachStreamreturns false →chat:stream_startis skipped). So on the resume path the store is empty for the very stream being recovered, the merge had nothing to merge, and the "guarded" refresh still wrote a pre-reply snapshot over the half-streamed bubble. The right answer is to not read the DB while a run is generating.Why the rejoin must evict the local partial
The server mints one id and uses it for both the assistant UI message (
generateId: () => serverAssistantMessageId) and the stream registry row. So the id useChat holds for the half-streamed bubble is the live stream'smessageId.Chat.stop()is explicit that it "keeps the generated tokens", so that bubble survives the stop — and the rejoin then re-adds the same stream to the pending store under the same id. Both surfaces drop a pending stream whosemessageIdalready appears inmessages(dedupRemoteStreams,ChatMessagesArea.visibleRemoteStreams), so the rejoined stream would be filtered straight back out and not one token of it would render.tryRecover's rejoin branch therefore reads the livemessageIdoff/active-streamsand evicts the matching local message before rejoining. (This also fixes the same latent bug on the network-error rejoin path.)But only when the server has something to put in its place.
/active-streamsalso returnsparts— the registry's debounced checkpoint, persisted every 20 parts, so it is empty for a stream only a few parts old. (Counted withisValidPartFrame, the same predicate the bootstrap seeds with, since it is that post-filter count which becomesskipReplayCount.) Evicting against an empty checkpoint and then failing the SSE join (the documented multi-instance case — the multicast registry is per-process) makes the bootstrap remove the stream, leaving the user with nothing: strictly worse than the frozen partial. So eviction is gated onparts.length > 0; otherwise we keep the partial and still attempt the rejoin.Why
handlePullUpRefreshmust NOT reconcile with the pending storeOn these two surfaces the pending store is the bubble — an in-flight stream renders from
remoteStreams, not frommessages.mergeServerAndPendingsynthesizes a message under the pending stream'smessageId, i.e. exactly the id the renderers dedup on. Merging here would freeze the live bubble at a static snapshot and stop it updating until completion. So this loader deliberately writes the DB snapshot as-is; the id stays absent frommessagesand the stream keeps rendering live.Why silence is not an answer
"We recovered nothing" is not the same claim as "there was nothing to recover".
tryRecoverasks two questions, and either can come back unanswered — the first request after a foreground is the one most likely to fail, radio still coming up:/active-streamssilenttakeOverConversationStreams, so the regenerate does not race it — it aborts it: re-running write tools it already executed (a turn that created a page creates it twice), billing the discarded tokens, stranding its partialhandleRetrydeletes the trailing assistant by id — the same id the server persisted the reply under — so it deletes the finished reply and pays for it againSo
RecoveryAttemptcarriesprobeAnsweredanddbAnsweredalongsiderecovered, each set only after a body we actually parsed, andcanConcludeTurnIsLostrequires both. The resume loop re-asks (bounded) until both come back. On persistent silence it does nothing — which is safe: thestop()released theconsumingmark, so a live run is picked up by the socket-reconnect bootstrap once the network returns, a persisted reply is picked up by the next load, and a genuinely dead turn leaves the user their prompt and its retry action.Was the user's turn persisted? Asked by identity, not by counting
The old guard compared user-message counts (
dbUserCount >= localUserCount). That silently breaks past one page: this GET is unpaginated, so the route applies its defaultlimit: 50and returns only the newest 50 rows, while localmessagesis the initial 50 plus every turn since. Beyond that boundary the guard is permanently false — step 2 could never recover on a long conversation, and every interrupted turn there fell through to a regenerate that deleted the reply it should have refetched.It now checks whether the DB contains the id of the last local user message. That message is by definition the newest, so it is always inside the returned window; and both routes persist the user turn under the client's id (
userMessage.id || createId()), so the identity round-trips. Correct and pagination-proof.Why the fallback is a regenerate, not a DB refresh
tryRecoverreturning false means one of exactly two things, and a DB write is unsafe in both:dbUserCount >= localUserCountguard rejected it, e.g. a send whose POST never reached the server. A refresh would erase the user's own prompt.But doing nothing is wrong too, and this is subtle:
Chat.stop()aborts the fetch, so useChat settles atreadywith noerror— anduseStreamRecoveryonly fires onstatus === 'error'. The stop we added therefore destroys the very signal that used to drive recovery. A turn whose POST died on the background transition would find no stream, no reply, and no error, and the user's prompt would sit unanswered forever. (Master recovered it: the dead fetch raised an error,useStreamRecoveryprobed, came up empty, regenerated.)So we regenerate — the same fallback
useStreamRecoveryapplies — but gated four ways:effectiveIsStreamingstays true for a stream still running against a conversation the user has since left);tryRecover's questions came back (above);handleRetryalways acts on the live conversation, so a switch mid-recovery would otherwise fire a generation for the turn the user moved to;useStreamRecoverywatches the same failure from the other side and regenerates too — and it callsclearError()before its own probes, so throughout its decision window the status readsreadyand looks idle. A singleregenerationInFlightRefmutex wrapshandleRetry(regenerateTurnOnce), and both paths go through it, so the same turn can never be regenerated twice; a!isStreamingRef.currentbelt then stops us regenerating on top of a turn that has already restarted and moved on (the mutex releases whenhandleRetry's DELETEs finish, but the generation it kicked off is still running).Why the stop must come first
rawStop()/stop()is the local-only useChat stop. It does not signal the server (that'sabortActiveStreamByMessageId), so the run keeps generating and stays rejoinable. But it also ends the dead response body, which releases the channel'sconsumingmark — and without that, the rejoin's bootstrap classifies the stream as one this tab is already reading off its POST body and skips attaching it. The rejoin would silently do nothing.Ordering is safe because both
tryRecoverand the bootstrapawaittheir/active-streamsfetch before consultingisChannelConsuming, so the abort-triggered unmark (a microtask) always lands first.Changes
GlobalAssistantView.tsxenabled→ callback;onResume→ stop +tryRecover(no DB fallback);tryRecoverevicts the stale partial on rejoin (gated on the server checkpoint being non-empty, and written through the store-syncing setter);handlePullUpRefreshgains a live-conversation stale-guardSidebarChatTab.tsxtryRecover, which it now closes over); same eviction;handleAppResumefunnels throughloadGlobalMessages(the documented single writer) instead of its own raw fetchlib/ai/streams/canResumeRecovery.tslib/ai/streams/evictStalePartial.tsisValidPartFrame)lib/ai/streams/recoveryAttempt.tsRecoveryAttempt+canConcludeTurnIsLost__tests__Tests
The resume wiring previously had zero coverage on either surface — which is how the render-time
enabledboolean survived. Following each file's established pattern (pure mirrors of the hook-heavy component's logic), the decision is not mirrored:planResumecalls the realresolveResumeAction, so the tests pin the wiring against the real policy.stop→tryRecover, and the DB is never readtryRecoverreturns false are exactly the two where a DB write is unsafe)stopalways precedes the probe (it's what releases theconsumingmark)messageId— and only that one message, never the user's turnValidation
bun run typecheck— cleanbun run build(apps/web) — exits 0bunx eslinton all touched files — cleanvitest— 815 passing acrosscomponents/layout+lib/ai/streamsBehaviour change worth calling out
Replacing the native DB refresh with a regenerate means two narrow cases no longer refresh on resume: a conversation whose last DB message is a user message where no turn was in flight, and another device having deleted the trailing assistant message. Both are healed by the socket-reconnect
refreshSignalthat fires on resume anyway.On the tests
The resume wiring previously had zero coverage on either surface — which is how the render-time
enabledboolean survived in the first place.The three load-bearing rules are now shared pure modules, so the tests assert against the shipped implementation rather than a copy of it:
canResumeRecovery,evictStalePartial, andcanConcludeTurnIsLost.planResumeremains a mirror of the sequencing only (theonResumebody is genuinely imperative — stop → probe → bounded re-probe → regenerate), but its two decisions call the realresolveResumeActionand the realcanConcludeTurnIsLost.Known follow-ups (deliberately not in this PR)
resolveResumeActiontreats all Capacitor platforms as "the fetch is always dead" —isCapacitorApp()is true on Android too, so a healthy Android stream gets stopped and rejoined on every foreground ≥5s. It renders correctly now that the eviction is in place, but it is needless churn. Pre-existing shared policy (AiChatViewalready does this); worth gating onplatform === 'ios'separately.AiChatViewstill doesstop → rejoin → refresh, and carries the same dedup collision on its rejoin path. The same fix applies. Left alone rather than widening this PR's blast radius.Refs: #2061, #2018