fix(models): correct DeepSeek V4 max_output_tokens to the documented 384K - #758
Merged
Conversation
…384K
Both V4 rows carried max_output_tokens=8_192 — a placeholder sitting beside
a 1_000_000 context window, under-reporting the model 47x in the token
warning. DeepSeek documents 384_000 for V4 Pro (including the max reasoning
modes).
Corrected, and pinned in the test, with a note that it is ADVISORY on this
wire so the next reader does not mistake the inertness for a bug:
* OpenAICompatibleProvider sends NO ``max_tokens`` field at all, so
DeepSeek applies its own server-side default either way. Contrast
AnthropicProvider (``_default_max_tokens`` computes and sends one on
every request) and Gemini (``config_kwargs["max_output_tokens"]``) —
on those wires the value is load-bearing; here it is not.
* The one live consumer is the auto-compact reservation, which clamps to
MAX_OUTPUT_TOKENS_FOR_SUMMARY (20_000). Effective input therefore moves
991_808 -> 980_000: 1.2%, in the more conservative direction.
Why the disclaimer is worth the lines: this field was suspected of
truncating long ``effort=max`` responses during the terminal-bench 2.1
DeepSeek triage (2026-07-27). It cannot — it never reaches the request.
Per-trial durations located the real cause elsewhere: on the 50 trials both
models completed, DeepSeek runs at 0.92x opus-5's wall-clock (it is
marginally FASTER), while a handful of tasks stall outright —
crack-7z-hash burns its full 30-minute budget where opus-5 finishes in 18
seconds, a 120x spread against a 0.92x median. That is a loop, not a token
ceiling, and is tracked separately.
No score change expected from this commit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Both V4 rows carried
max_output_tokens=8_192— a placeholder sitting beside a1_000_000context window, under-reporting the model 47× in the token warning. DeepSeek documents 384,000 for V4 Pro (including the max reasoning modes).It is advisory on this wire
Corrected and pinned, with a comment so the inertness isn't mistaken for a bug:
max_tokens?_default_max_tokenscomputes and sends one per requestconfig_kwargs["max_output_tokens"]The only live consumer is the auto-compact reservation, which clamps to
MAX_OUTPUT_TOKENS_FOR_SUMMARY(20,000). Effective input moves 991,808 → 980,000 — 1.2%, in the more conservative direction.Why the disclaimer earns its lines
This field was suspected of truncating long
effort=maxresponses during the terminal-bench 2.1 DeepSeek triage. It cannot — it never reaches the request. Per-trial durations located the real cause elsewhere:effort=maxlatency is not the problem.crack-7z-hashburns its full 30-minute budget where opus-5 finishes in 18 seconds — a 120× spread against a 0.92× median. That is a loop, not a token ceiling, and is tracked separately.No score change expected from this commit. It fixes a displayed number and removes a false lead.
Tests: 389 passing in the affected area; the pinning test updated with the reasoning.
🤖 Generated with Claude Code