Skip to content

feat(gateway): support Responses Lite and GPT-6 Astra - #508

Open
Longtianmu wants to merge 36 commits into
Menci:mainfrom
Longtianmu:codex/responses-lite
Open

feat(gateway): support Responses Lite and GPT-6 Astra#508
Longtianmu wants to merge 36 commits into
Menci:mainfrom
Longtianmu:codex/responses-lite

Conversation

@Longtianmu

Copy link
Copy Markdown

Codex can select Responses Lite, where tool declarations and base instructions live in ordered input items. Treating those requests as standard Responses changes prompt identities and can lose tool configuration or history. This PR adds a metadata-selected Lite transport and extends GPT-6 Astra compatibility while keeping native requests and existing gateway state handling intact.

Responses Lite

  • Carry the transport through endpoint metadata, model configuration/cache, provider dispatch, and the localized Playground. Select it from upstream capabilities or operator configuration, without guessing from model names.
  • Convert standard requests into stable Lite prefixes. Native Lite requests retain client IDs, chronological declarations, image references, async function/custom-tool history, configuration updates, and open-string fields.
  • Preserve Lite semantics through role compatibility, hosted-tool emulation, compaction, and stored-response continuation. Keep transport markers out of unrelated protocols and preserve Codex stream options and compact access selection.
  • Exercise actual serialized generate/compact requests for Codex, Custom, Azure, and Ollama, plus gateway HTTP/SSE and non-streaming responses. Preserve forwarded upstream errors' status, headers, and bytes; the Codex SSE compatibility workaround no longer overwrites error media types.

GPT-6 Astra and WebSocket compatibility

  • Update Codex compatibility metadata and the bundled Astra entry from Codex 0.154.0. Advertise Lite through the model catalog and invalidate older derived catalog rows.
  • Project Codex's client-local reasoning aliases into public inference metadata using its effort resolver. Native Codex discovery retains Ultra orchestration; explicitly supplied request values remain untouched.
  • Implement generate: false prewarm as chainable gateway state without an inference call. Handle WebSocket-only fields and connection limits at the socket boundary, including named stream_id lanes, per-lane FIFO ordering, and concurrent lanes. Isolate per-turn authentication and prevent in-flight snapshot refreshes from undoing an eviction while retaining durable fallback.
  • Keep asynchronous tool results and between-response configuration_update items in native history. True mid-turn response.steer is outside this PR: it requires a provider duplex session, automatic successor responses, and independent endpoint capability evidence. Local prewarm does not promise upstream cache prefill.

Protocol conversion and boundaries

  • Cover Chat Completions, Anthropic, and Gemini sources routed through Lite Responses, and Lite sources routed to standard Responses, Chat Completions, and Anthropic.
  • Preserve namespace function/custom-tool identity through declarations, forced choices, call history, and returned events. Bound generated aliases to the target protocol's limit and accept omitted/null function schemas consistently.
  • Reject async, programmatic, or configuration-update semantics that a target cannot represent before making an upstream call. Native values remain upstream-owned.
  • Successful responses retain Floway's existing ID, affinity, state, and output-envelope handling; this is not a new byte-transparent success proxy. Successful compact responses retain the existing provider envelope contract.

Test Plan

  • Reproduce and fix the Lite marker, native-prefix, Codex dispatch, namespace, and reasoning-metadata regressions.
  • Run the complete checked-in Linux Verify workflow on final commit ea1a6ca5: all eight jobs passed, including type generation, lint, full typecheck, all 5,905 tests in 553 files, installers, the three repository invariant checks, and the web production build with all four output assertions.
  • Read the installer harness results: 109 passed, 0 failed, 1 skipped because the pinned Codex 0.144.5 binary was absent.
  • Exercise concurrent WebSocket lanes and snapshot eviction with deterministic barriers, including a mutation check proving the 16-response cap assertion detects an incorrect limit.
  • Open the current production build in Chrome with isolated API fixtures in English and Simplified Chinese. Verify Standard/Lite switching and hints in Playground, upstream defaults, and per-model configuration; confirm the two configuration controls remain independent and no browser errors occur.
  • Live authenticated Astra/Codex upstream smoke test: no configured Floway instance or upstream test account was available. Wire and gateway integration tests use controlled upstream fixtures.

# Conflicts:
#	packages/gateway/src/repo/models-cache-contract.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant