Skip to content

fix: Emit OpenAI-compatible SSE event headers - #167

Open
maralbahari wants to merge 23 commits into
vllm-project:mainfrom
EmbeddedLLM:fix-gateway-sse-event-header
Open

fix: Emit OpenAI-compatible SSE event headers#167
maralbahari wants to merge 23 commits into
vllm-project:mainfrom
EmbeddedLLM:fix-gateway-sse-event-header

Conversation

@maralbahari

@maralbahari maralbahari commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Fixes #168

Make gateway streaming responses emit OpenAI-compatible named SSE frames:

event: response.output_text.delta
data: {"type":"response.output_text.delta", ...}

Previously, the gateway emitted only the data: line.

Changes

  • Emit an event: header for each JSON SSE event.
  • Derive the header value from the event payload’s type.
  • Keep data: [DONE] as an unnamed terminal marker.
  • Apply named events consistently to:
    • model response events
    • gateway tool lifecycle events
    • terminal response events
    • error events
  • Reject invalid event names containing newline characters to prevent SSE header injection.
  • Update WebSocket forwarding to consume complete named SSE frames while continuing to send only the JSON payload to WebSocket clients.
  • Re-record gateway web-search, MCP, and custom-tool cassettes with the corrected wire format.

Testing

  • Added assertions that every JSON data: event has exactly one preceding event: header.
  • Asserted that each SSE header matches the corresponding payload type.
  • Updated gateway, integration, stateful-response, and WebSocket tests for complete SSE frames.
  • Verified recorded gateway event-header and data occurrences against OpenAI behavior.

Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
Signed-off-by: maral <maralbahari.98@gmail.com>
…-header

Signed-off-by: maral <maralbahari.98@gmail.com>
@maralbahari
maralbahari marked this pull request as ready for review August 6, 2026 14:54
@maralbahari

Copy link
Copy Markdown
Collaborator Author

@franciscojavierarceo ready for review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gateway streaming responses omit SSE event headers

1 participant