fix: normalize null tool_calls fields in streaming chat deltas

[Ranoobaba]

Have you read the Contributing Guidelines? Yes.

Issue #160

Describe your changes

What and why. Issue #160 reports that the API returns an explicit null where the OpenAI streaming format omits the field entirely, in three places:

1. choices[n].delta.tool_calls = null on text only chunks.
2. choices[n].delta.tool_calls[n].function.arguments = null on the first tool call chunk, where the name is given.
3. choices[n].delta.tool_calls[n].function.name = null on argument continuation chunks.

The root cause is on the API side, but the SDK today amplifies it instead of absorbing it, because the streaming delta model (DeltaContent) only declares content. Everything else rides on extra="allow", so on current main:

1. delta.tool_calls is None when the API sends null, but it raises AttributeError when the API omits the field. Null and missing are observably different to SDK users, which is the exact problem the issue describes, one level up.
2. Tool call fragments surface as raw dicts, not the ToolCalls and FunctionCall models that the non streaming ChatCompletionMessage uses.
3. model_dump(exclude_none=True) does not strip the nulls nested inside those raw dicts, so {"function": {"arguments": None}} and {"function": {"name": None}} leak into OpenAI compatible consumers.

The fix. Add ChatCompletionDeltaContent(DeltaContent) in src/together/types/chat_completions.py declaring tool_calls: List[ToolCalls] | None = None, and use it for ChatCompletionChoicesChunk.delta. That single typed field means:

1. null and missing both parse to None, indistinguishable, matching OpenAI semantics for all three reported instances. FunctionCall.name and .arguments were already str | None, so the nested nulls normalize the same way once the items are validated.
2. Fragments validate into the same ToolCalls models used by the non streaming path, so typing is consistent across streaming and non streaming, and identical to how openai-python types its deltas (ChoiceDeltaToolCallFunction.name and .arguments are Optional[str]).
3. model_dump(exclude_none=True) now recursively omits the API provided nulls, producing OpenAI shaped deltas with no Together special cases.

Deliberate non decisions:

1. No coercion of None into an empty string for arguments. openai-python also leaves absent fields as None; coercing would diverge from it and silently change is None checks.
2. Text completion deltas untouched. DeltaContent is shared with CompletionChunk, so the new field is added on a chat specific subclass rather than the shared class.
3. No wire or request changes. Request serialization (ChatCompletionRequest.model_dump(exclude_none=True)) is unaffected.
4. index on tool call fragments stays undeclared, kept intentionally out of scope. It still flows through via extra="allow" and is asserted in tests; declaring it would add an index: None key to non streaming ChatCompletionMessage.tool_calls dumps.

Sync and async clients share these models (ChatCompletionChunk(**line.data) at both call sites in resources/chat/completions.py), so both are covered.

Known limitations

1. The root cause is on the API side; this PR is client side hardening. Raw HTTP consumers and other SDKs still see the nulls on the wire. As of today the wire behavior appears unchanged since 2024: the issue is still open, and downstream parsers still carry Together specific .nullable() overrides that cite this exact issue (for example big-AGI's openai.wiretypes.ts, which marks tool_calls, function.name, and function.arguments nullable with a "[TogetherAI]" note linking to Non-standard OpenAI-like response format on tool_calls. #160). I did not have an API key available to re-probe the live wire, so that evidence is corroborative rather than a fresh capture. The SDK level behavior fixed here reproduces deterministically from the issue's own payloads either way.
2. Behavior differences, quantified. All three are the normalization the issue asks for, but they are listed so reviewers can see them: (1) reading delta.tool_calls on a chunk with no tool calls changes from AttributeError to None, which is strictly fewer crashes; (2) tool call fragments change from raw dict to typed ToolCalls models, so delta.tool_calls[0]["function"] becomes delta.tool_calls[0].function, which matches the long standing typed non streaming path, and anyone who wants dicts gets the same shape back from model_dump(); (3) a plain model_dump() without exclude_none=True of a text only delta now includes tool_calls: None even when the API omitted the key, which is standard Pydantic optional field behavior and matches ChatCompletionMessage and openai-python.
3. V1 is in maintenance mode (V2 lives in together-py), so this is scoped as a small maintenance bug fix. V2's Stainless generated ChoiceDelta.tool_calls: Optional[List[ToolChoice]] already types the field, but V2's Function.name and arguments are declared as required str, so if the API still emits the nulls, V2's lenient response parsing will surface None there too. The underlying API cleanup tracked by this issue is still worth doing server side.

Testing

New file tests/unit/test_chat_completion_stream_types.py, 6 tests using chunk fixtures shaped exactly like the issue's three instances (text only tool_calls: null, first fragment arguments: null, continuation name: null), plus an omitted field control, an extra field passthrough check (delta.role, tool_calls[n].index), an exclude_none recursive dump assertion, and a non streaming regression control.

Fail then pass, demonstrated by restoring the un-fixed file with git restore --source=main -- src/together/types/chat_completions.py:

On un-fixed main (5 failed, 1 passed, where the pass is the non streaming control):
  FAILED test_null_tool_calls_parses_like_omitted_tool_calls   AttributeError: 'DeltaContent' object has no attribute 'tool_calls'
  FAILED test_tool_call_delta_items_are_typed_models           AssertionError (raw dicts, not ToolCalls)
  FAILED test_null_function_name_on_continuation_chunks        AttributeError: 'dict' object has no attribute 'function'
  FAILED test_exclude_none_dump_produces_openai_shaped_deltas  AssertionError: None survived in the dump
  FAILED test_extra_wire_fields_are_preserved                  AttributeError: 'dict' object has no attribute 'index'
  PASSED test_non_streaming_tool_calls_unchanged               (control: non streaming was already typed)

With the fix: 6 passed. Full unit suite: 212 passed.

Lint, format, and typing: ruff check clean, black --check clean, and mypy (using the repo's mypy.ini) reports no errors in the changed files. The 4 pre-existing errors elsewhere are identical on main.

Precedent

This is the same category of change as #401 ("fix: Avoid crashing when models return the incorrect object const") and #239 (accept an empty string finish_reason): small type level hardening of the response models against API format quirks, with no wire changes.

[broly-code-security-scanner]

Broly Security Scan

Note

✅ Clean scan
No vulnerabilities detected in this PR.

Note

Re-scan this PR anytime with /broly scan — useful after /broly undismiss, or to refresh findings without a new push.

Broly — SAST (zai-org/GLM-5.2) · Secrets · SCA · GH Actions (zizmor) · Containers · SBOM · Powered by Together AI