fix(query): send reasoning effort on the OpenAI-compatible wire

eval/harbor/RUN_LUNA_TB21.md

@@ -0,0 +1,120 @@
+# Running `openai/gpt-5.6-luna` on terminal-bench 2.1
+
+OpenRouter-proxied GPT-5.6 Luna, at reasoning effort `max`, through the
+clawcodex Harbor adapter.
+
+## Model facts (OpenRouter `/models`, fetched 2026-07-31)
+
+| | |
+|---|---|
+| id | `openai/gpt-5.6-luna` (a `-pro` variant exists with identical specs) |
+| context | 1,050,000 tokens (registered as 1,048,576 = 2^20, matching the sibling gpt-5.6 rows; under-reading is the safe direction) |
+| max output | 128,000 tokens |
+| price | $0.10/M in, $0.60/M out — doubling to $0.20 / $0.90 above 272K prompt tokens |
+| reasoning | `reasoning_effort` supported; verified honored, not just accepted — reasoning tokens rise low 148 → medium 154 → high 266 → max 516 on a fixed prompt |
+| tools | `tools` + `tool_choice` supported; returns `finish_reason=tool_calls` |
+| vision | text + image + file → text |
+
+## Prerequisite: a build that actually sends the effort
+
+Until 2026-07-31, clawcodex emitted reasoning effort **only** on the
+Anthropic wire. On the headless `--print` path the adapter drives,
+`--effort` was a silent no-op for every OpenAI-compatible provider —
+OpenRouter, OpenAI, DeepSeek, Z.AI. The job config would record
+`effort=max` and the request body would carry no effort field at all.
+
+Two consequences:
+
+1. Containers must install a clawcodex build dated 2026-07-31 or later, so
+   pin `--ak source=git+…@<branch-or-main>` at a commit that includes the
+   fix. The PyPI package (1.2.1) does not.
+2. Earlier non-Anthropic runs labelled `effort=max` — including the
+   `tb21-deepseek-max` job in `jobs/` — actually ran at the provider's
+   default effort. Do not compare a new Luna run against those as if the
+   effort setting matched.
+
+## Commands
+
+Run from the repo root. `PYTHONPATH` must point at `eval/harbor` so Harbor
+can import the adapter in the host process.
+
+```bash
+export OPENROUTER_API_KEY=$(python3 -c \
+  "import json,os;print(json.load(open(os.path.expanduser('~/.clawcodex/config.json')))['providers']['openrouter']['api_key'])")
+
+export CX_SOURCE=git+https://github.com/agentforce314/clawcodex@main   # must contain the effort fix
+```
+
+### Smoke first — two tasks, ~5 minutes
+
+Never start an 89-task run without this; a broken install or a missing key
+fails identically on every task and costs an hour to find out.
+
+```bash
+PYTHONPATH=$PWD/eval/harbor harbor run \
+  --dataset terminal-bench/terminal-bench-2-1 \
+  --agent clawcodex_agent:Clawcodex \
+  --model openrouter/openai/gpt-5.6-luna \
+  --ak effort=max \
+  --ak source=$CX_SOURCE \
+  -i 'terminal-bench/fix-git' \
+  -i 'terminal-bench/openssl-selfsigned-cert' \
+  --job-name smoke-tb21-luna-max \
+  --jobs-dir eval/harbor/jobs \
+  --n-concurrent 2
+```
+
+Expect reward 1.0 on both and zero infra errors. Then confirm the effort
+really shipped — the whole point of the pin:
+
+```bash
+grep -ro '"reasoning_effort": *"[a-z]*"' eval/harbor/jobs/smoke-tb21-luna-max | head
+```
+
+### Full run — all 89 tasks
+
+```bash
+PYTHONPATH=$PWD/eval/harbor harbor run \
+  --dataset terminal-bench/terminal-bench-2-1 \
+  --agent clawcodex_agent:Clawcodex \
+  --model openrouter/openai/gpt-5.6-luna \
+  --ak effort=max \
+  --ak source=$CX_SOURCE \
+  --job-name tb21-luna-max \
+  --jobs-dir eval/harbor/jobs \
+  --n-concurrent 4
+```
+
+`-k 5` matches the official leaderboard's pass@5 methodology (the published
+Claude Code 0.79 is k=5); the in-repo 3-way comparison numbers — clawcodex
+0.58, openclaude 0.551, claude-code 0.719 — are all k=1, so keep k=1 to
+compare against those.
+
+### Results
+
+```bash
+harbor view eval/harbor/jobs                       # browse trajectories
+python3 eval/harbor/compare_trajectories.py …      # NOT ad-hoc counting:
+                                                   # a denominator mistake
+                                                   # once inverted 4 metrics
+```
+
+## Gotchas
+
+- **Model string is doubly-qualified.** Harbor splits `--model` on the FIRST
+  slash only, so `openrouter/openai/gpt-5.6-luna` → `--provider openrouter
+  --model openai/gpt-5.6-luna`. That is intended; do not "fix" it to a
+  single slash.
+- **Hub datasets namespace task names.** Include filters need the prefix:
+  `-i 'terminal-bench/fix-git'`. A bare `-i fix-git` matches nothing.
+- **The key must be in the host environment.** The adapter forwards
+  `OPENROUTER_API_KEY` from the host env into the container; it does not
+  read `~/.clawcodex/config.json` for the provider key. Hence the `export`
+  above. Alternatively pass `--ae OPENROUTER_API_KEY="$OPENROUTER_API_KEY"`.
+- **Docker credential helper.** If image pulls hang forever, check
+  `~/.docker/config.json` for `credsStore: "desktop"` — a Docker Desktop
+  update can rewrite it, and it wedges even anonymous pulls of public
+  images. It should be `osxkeychain`.
+- **Cost.** Luna is cheap (~$0.10/M in). A full 89-task run at k=1 lands in
+  the low single-digit dollars, versus roughly two orders of magnitude more
+  for an Opus run.

eval/harbor/clawcodex_agent.py

@@ -26,6 +26,20 @@
 ``--provider deepseek --model deepseek-v4-flash``). A bare name is passed
 through as ``--model`` alone, falling back to clawcodex's own routing.
 
+The split is on the FIRST slash only, so an OpenRouter id — itself
+``vendor/model`` — round-trips correctly by stacking the two conventions:
+``openrouter/openai/gpt-5.6-luna`` → ``--provider openrouter --model
+openai/gpt-5.6-luna``. For example, terminal-bench 2.1 at maximum
+reasoning effort::
+
+    export OPENROUTER_API_KEY=sk-or-v1-...
+    PYTHONPATH=eval/harbor harbor run \
+        --dataset terminal-bench/terminal-bench-2-1 \
+        --agent clawcodex_agent:Clawcodex \
+        --model openrouter/openai/gpt-5.6-luna \
+        --ak effort=max \
+        --jobs-dir eval/harbor/jobs --n-concurrent 4
+
 API keys reach the container two ways, either is sufficient:
 
 * exported in the host environment (the running provider's key vars are
@@ -38,13 +52,35 @@
 * ``max_turns`` — clawcodex ``--max-turns`` (default 300 here; the CLI's
   own default of 50 is too low for terminal-bench tasks). The
   ``CLAWCODEX_MAX_TURNS`` host env var works as a fallback.
-* ``effort`` — clawcodex ``--effort`` (low|medium|high|xhigh|max) for
-  models that support ``output_config.effort`` (Opus 5, Opus 4.6/4.8,
-  Sonnet 4.6, Fable 5). ``xhigh`` is model-dependent (opus-5/opus-4-8 yes;
-  sonnet-4-6/opus-4-6 no) — clawcodex degrades it to ``high`` where
-  rejected. A model missing from clawcodex's effort allowlist drops the
-  flag SILENTLY, so a new model needs a clawcodex build that registers it
-  (see ``source`` below) before an effort number means anything.
+* ``effort`` — clawcodex ``--effort`` (low|medium|high|xhigh|max). How it
+  reaches the wire depends on the provider family, and the two behave
+  differently:
+
+  - **Anthropic wire** — sent as ``output_config.effort``, and gated on
+    clawcodex's effort allowlist (Opus 5, Opus 4.6/4.8, Sonnet 4.6,
+    Fable 5). A model missing from that allowlist drops the flag SILENTLY,
+    so a new Anthropic model needs a clawcodex build that registers it
+    (see ``source`` below) before an effort number means anything.
+    ``xhigh`` is model-dependent (opus-5/opus-4-8 yes; sonnet-4-6/opus-4-6
+    no) and degrades to ``high`` where rejected.
+  - **OpenAI-compatible wire** (openrouter, openai, deepseek, zai, …) —
+    sent as the top-level ``reasoning_effort`` body field, with no model
+    allowlist and no ``xhigh`` clamp; the level passes through verbatim.
+    ONE exception: the ChatGPT-subscription path (``subscription=true``
+    with ``--model openai/…``) clamps ``xhigh`` and ``max`` down to
+    ``high`` before sending, because that backend advertises only
+    low/medium/high and rejects higher tiers. So ``--ak effort=max`` plus
+    subscription auth really runs at ``high`` — an API-key or OpenRouter
+    run of the same model does not.
+
+  REQUIRES a clawcodex build from 2026-07-31 or later. Before that, effort
+  was emitted ONLY on the Anthropic branch, so ``--effort`` was a silent
+  no-op for every OpenAI-compatible provider — an eval run against, say,
+  ``openrouter/openai/gpt-5.6-luna`` would report ``effort=max`` in its
+  config and send nothing. Pin ``source`` accordingly when benchmarking a
+  non-Anthropic model at a specific effort, and confirm the level is really
+  on the wire rather than trusting the job config.
+
   ``CLAWCODEX_EFFORT`` host env var works as a fallback.
 * ``version`` — pin a ``clawcodex-cli`` PyPI version (default: latest).
 * ``source`` — full pip-installable spec overriding the PyPI package, e.g.

src/models/configs.py

@@ -416,6 +416,31 @@ class ModelConfig:
         context_window=1_048_576,
         max_output_tokens=128_000,
     ),
+    # The same model as the row above, under its OpenRouter id. This table is
+    # keyed by BARE name and ``get_model_config`` deliberately does not strip
+    # a ``<vendor>/`` prefix (see its docstring), so without this row the id
+    # that actually reaches the provider on the OpenRouter path matches
+    # nothing and silently falls back to DEFAULT_CONTEXT_WINDOW (200K) — a 5x
+    # under-read that makes auto-compact fire at a fifth of the real window.
+    # Added for the terminal-bench harness, which drives this model as
+    # ``--model openrouter/openai/gpt-5.6-luna``; Harbor splits on the FIRST
+    # slash, so clawcodex receives ``--model openai/gpt-5.6-luna``.
+    #
+    # Only Luna is duplicated, not the whole Sol/Terra family: a qualified row
+    # is the narrow, per-model answer to a general gap, and adding rows
+    # nobody routes yet is speculative duplication. The general fix (teaching
+    # the resolver the vendor prefix, as ``get_pricing`` already does) is
+    # "decision #1" and stays out of scope — see the docstring.
+    #
+    # Base for the prefix fallback is "openai/gpt-5.6", which nothing else
+    # claims, so ``openai/gpt-5.6-luna-pro`` resolves here too — matching how
+    # the bare rows above let ``gpt-5.6-sol-pro`` through.
+    "openai/gpt-5.6-luna": ModelConfig(
+        model_id="openai/gpt-5.6-luna",
+        display_name="GPT-5.6 Luna",
+        context_window=1_048_576,
+        max_output_tokens=128_000,
+    ),
     "gpt-5.5": ModelConfig(
         model_id="gpt-5.5",
         display_name="GPT-5.5",
@@ -486,7 +511,25 @@ class ModelConfig:
 
 
 def get_model_config(model_id: str) -> ModelConfig | None:
-    """Get config for a model, or None if unknown."""
+    """Get config for a model, or None if unknown.
+
+    Exact match, then a prefix fallback for date-variant ids (a row's claimed
+    prefix is its key minus the last ``-``-segment).
+
+    NOT attempted: stripping a leading ``<vendor>/`` segment so OpenRouter ids
+    resolve to their bare row. ``get_pricing`` (services/pricing.py) does
+    exactly that, and mirroring it here is tempting — but it is deliberately
+    out of scope, the same call ``tests/test_deepseek_prefix_cache.py`` pins
+    as "decision #1" (``deepseek/deepseek-v4-pro`` keeps the 200K default).
+    Two reasons it is not a free win: it would silently outrank a user's own
+    ``modelLimits`` override, which ``get_context_window_for_model`` consults
+    only when this returns ``None``; and it would resolve ids whose bare name
+    prefix-matches an unrelated row, in the window-WIDENING direction, which
+    overflows the request instead of merely compacting early. Reversing that
+    decision is its own change with its own test sweep. A vendor-qualified
+    model that needs a real window gets an explicit row instead — see
+    ``openai/gpt-5.6-luna``.
+    """
     if model_id in MODEL_CONFIGS:
         return MODEL_CONFIGS[model_id]
     # Try prefix match (for date-variant models)

src/providers/openai_provider.py

@@ -55,12 +55,23 @@ def _subscription_reasoning_effort(requested: str | None = None) -> str:
     """Reasoning effort for subscription requests.
 
     Precedence: the session's ``/effort`` setting (arrives as
-    ``extra_body.reasoning_effort`` via the agent-server's
-    ``_EffortProvider`` wrapper) → ``CLAWCODEX_OPENAI_REASONING_EFFORT``
-    → ``medium`` (OpenCode's default, transform.ts:1176, and the
-    backend's own default_reasoning_level). ``xhigh``/``max`` clamp to
-    ``high`` — the general gpt-5.x models advertise low/medium/high and
-    reject higher tiers.
+    ``extra_body.reasoning_effort``, injected at the wire boundary by
+    ``query.py::_call_model_sync`` for every OpenAI-compatible provider)
+    → ``CLAWCODEX_OPENAI_REASONING_EFFORT`` → ``medium`` (OpenCode's
+    default, transform.ts:1176, and the backend's own
+    default_reasoning_level).
+
+    ``xhigh``/``max`` clamp to ``high`` HERE, and only here: this is the
+    ChatGPT-subscription backend (chatgpt.com/backend-api/codex), whose
+    general gpt-5.x models advertise low/medium/high and reject higher
+    tiers (probed 2026-07-25). That is narrower than the public API —
+    developers.openai.com/api/docs/guides/reasoning lists none | minimal |
+    low | medium | high | xhigh | max and notes support varies by model —
+    and narrower than what a gateway may accept (``openai/gpt-5.6-luna``
+    via OpenRouter takes both ``xhigh`` and ``max``, probed 2026-07-31,
+    with reasoning-token counts rising monotonically across the ladder).
+    So the clamp is a property of THIS backend, not of the level names;
+    the generic OpenAI-compatible path deliberately does not clamp.
     """
     for candidate in (requested, os.environ.get("CLAWCODEX_OPENAI_REASONING_EFFORT")):
         effort = (candidate or "").strip().lower()
@@ -275,8 +286,8 @@ def _subscription_request_body(
             "stream": True,
             "include": list(INCLUDE_ENCRYPTED_REASONING),
             "reasoning": {
-                # /effort arrives as extra_body.reasoning_effort via the
-                # agent-server's _EffortProvider wrapper (agent_server.py).
+                # /effort arrives as extra_body.reasoning_effort, injected
+                # at the wire boundary by query.py::_call_model_sync.
                 "effort": _subscription_reasoning_effort(
                     (kwargs.get("extra_body") or {}).get("reasoning_effort")
                 ),

src/query/agent_loop_compat.py

@@ -403,13 +403,17 @@ async def _maybe_recall_memories(
     query_text = _last_user_text(messages)
     if not query_text.strip():
         return None
-    # R5 (ch11 N1) — unwrap /effort's _EffortProvider so the recall SELECTOR
-    # runs on the raw provider: (a) _resolve_recall_model's
-    # isinstance(AnthropicProvider) check sees through the wrapper → the
-    # small_fast_model cost pin applies in effort mode too (it was bypassed —
-    # a bare wrapper class isn't an AnthropicProvider); (b) the wrapper's
-    # reasoning_effort injection doesn't leak into the cheap selector call.
-    # Safe: _inner is _EffortProvider-exclusive, so this is a no-op otherwise.
+    # R5 (ch11 N1) — unwrap any provider decorator so the recall SELECTOR runs
+    # on the raw provider: _resolve_recall_model's isinstance(AnthropicProvider)
+    # check has to see through it, or the small_fast_model cost pin is bypassed
+    # (a bare wrapper class isn't an AnthropicProvider).
+    #
+    # This was introduced for /effort's ``_EffortProvider``, which has since
+    # been deleted — reasoning effort is now applied at the wire boundary in
+    # query.py for both provider families rather than by wrapping. Kept as a
+    # cheap general guard: ``_inner`` is not an attribute any real provider
+    # defines, so this is a no-op unless some future decorator reintroduces
+    # the same shape.
     provider = getattr(provider, "_inner", provider)
     try:
         from src.memdir import get_auto_mem_path