Skip to content

fix: preserve response continuity on physical WebSockets - #694

Merged
icebear0828 merged 6 commits into
icebear0828:devfrom
shiitin:dev
Jul 30, 2026
Merged

fix: preserve response continuity on physical WebSockets#694
icebear0828 merged 6 commits into
icebear0828:devfrom
shiitin:dev

Conversation

@shiitin

@shiitin shiitin commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR fixes response continuation handling for pooled WebSocket connections.

When store=false is used, a previous_response_id is only valid on the physical WebSocket that created it. Previously, a continuation could be routed through another pooled connection or a newly established one, which could result in upstream previous_response_not_found errors and break implicit conversation continuity.

This change tracks response ownership at the physical WebSocket level and ensures continuations are routed only to the connection that owns the parent response. When that connection is no longer available, implicit continuations recover safely by replaying the full input instead of attempting an invalid continuation.

Along the way, this PR also improves WebSocket timeout handling, tightens per-account connection limits under concurrent load, and makes the container health check more robust in proxy environments.

What changed

Physical WebSocket response ownership

Responses are now associated with the physical WebSocket that produced them.

Requests containing previous_response_id are routed only to the owning connection after verifying that it:

  • belongs to the expected account;
  • is still alive;
  • has not expired; and
  • is not already processing another response.

Ownership information is removed whenever a connection is closed, replaced, evicted, or when the upstream rejects the response ID. Each physical WebSocket retains only its most recent response anchor.

Explicit continuations no longer fall back to another pooled connection or a newly created connection, since those continuations are not valid upstream.

Implicit continuation recovery

Implicit continuations are now recovered more safely when the original connection is no longer usable.

Generation-based chain advancement prevents concurrent sibling requests from advancing the implicit chain out of order.

If the owning connection has disappeared, recovery establishes a new continuation root by replaying the full input on an isolated pool key.

If the owning connection is merely busy, it remains the active parent so sibling requests can still complete without unnecessarily resetting the continuation chain.

turnState is no longer restored from cross-turn affinity. Only the value explicitly supplied by the current request is forwarded.

WebSocket lifecycle

Persistent and one-shot WebSocket requests now use bounded response-start waits.

Requests that connect successfully but never receive a response-start event now fail instead of waiting indefinitely.

Timeout errors are propagated correctly, and timers, streams, and connection resources are cleaned up before returning.

The existing one-shot fallback is preserved only for full-input requests that do not include previous_response_id.

Connection pool

Connection factories now count toward maxPerAccount while they are being created.

Capacity is reserved before asynchronous connection creation begins and released regardless of whether creation succeeds or fails, preventing concurrent acquisitions from temporarily exceeding the configured limit.

Container health check

The container health check now ignores inherited HTTP_PROXY, HTTPS_PROXY, and ALL_PROXY variables when probing the local endpoint.

It connects directly to 127.0.0.1, applies a bounded timeout, prefers the runtime PORT environment variable, and falls back to the configured port or 8080.

Tests now use an ephemeral port instead of reserving port 8080.

Why

When response storage is disabled (store=false), response continuations are tied to the physical WebSocket that created the parent response.

Pooling by logical session alone is therefore insufficient. A continuation must be sent through the same physical connection; otherwise, the upstream may reject it with previous_response_not_found, leaving implicit conversation state unrecoverable.

This change makes that ownership explicit, prevents unsafe cross-connection continuations, and provides a safe recovery path when the original connection is no longer available.

Test coverage

Added or expanded coverage for:

  • physical WebSocket response ownership;
  • explicit and implicit continuations;
  • missing, dead, expired, busy, and account-mismatched owners;
  • concurrent sibling chain advancement;
  • stale connection recovery;
  • persistent and one-shot response-start timeouts;
  • timeout resource cleanup;
  • concurrent per-account connection creation;
  • failed factory capacity release;
  • proxy-independent health checks;
  • runtime health-check port overrides; and
  • ephemeral health-check test ports.

Validation

Validated on Ubuntu 24.04 x86_64 with Node.js 22.23.1.

  • Production build
  • Application TypeScript typecheck
  • Script TypeScript typecheck
  • Shell syntax check
  • Root test suite: 261/261 files passed
  • Root tests: 2587 passed, 1 skipped
  • Web test suite: 7/7 files passed
  • Web tests: 24/24 passed
  • git diff --check
Root:
Test Files  261 passed (261)
Tests       2587 passed | 1 skipped (2588)

Web:
Test Files  7 passed (7)
Tests       24 passed (24)

Compatibility

  • No configuration migration is required.
  • Full-input requests retain the existing one-shot fallback behavior.
  • Explicit previous_response_id requests now fail instead of being replayed across a different physical WebSocket.
  • The health check remains backward compatible with the configured port and the default port 8080.

Fixes #691

Thank you for taking the time to review this PR. :)

@icebear0828 icebear0828 reopened this Jul 30, 2026
@icebear0828
icebear0828 merged commit 4db59c4 into icebear0828:dev Jul 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[question] Problems about Codex API error (400)

2 participants