Skip to content

feat: support allowlisted Pi models and primary effort routing - #112

Open
0xroylee wants to merge 2 commits into
mainfrom
codex/pi-multi-model-routing
Open

feat: support allowlisted Pi models and primary effort routing#112
0xroylee wants to merge 2 commits into
mainfrom
codex/pi-multi-model-routing

Conversation

@0xroylee

Copy link
Copy Markdown
Contributor

What Changed

  • Add optional models.allowlist and models.implementPrimaryEffort settings so Pi can use an explicitly selected Kimi or GLM implementer with a supported effort.
  • Enforce exact model admission during startup, dispatch, retries, and unfinished recovery. Preserve recorded attempts and independent Review of the trusted implementation commit.
  • Keep the advisor deterministic and preserve upstream per-role efforts configuration. Include the implementation plan, architecture diagram, execution graph, and operator instructions.

Why

Roc previously requested medium effort for every implementation attempt, which cannot select Pi's Kimi K3 entries. Profile mappings also did not enforce an exact model allowlist. This change enables controlled provider selection without adding another runtime or a paid LLM selector call.

The integration preserves the per-role effort feature merged in #101. The specific primary override applies to Terra; efforts.implement applies to Sol high-risk/escalated work, defaulting to medium when omitted. Existing per-role fallback behavior remains when the new model policy is absent.

Implementation plan · Architecture source · Execution graph source

How to Test

  1. Run bun run check for lint, typecheck, and the full repository suite.
  2. Run the focused settings, advisor, Pi backend/harness, backend-session, and onboarding tests. The fixtures include a Codex Scout -> Kimi Implement -> independent Codex Review flow.
  3. Inspect the operator configuration example in README.details.md. An unsupported or denied effective mapping must fail startup before task admission; denial of an unfinished recovered attempt must preserve history and require replanning.

Verified on rebased head daa5b58d064dad340bca4fc1d85517d3ff056b55, based on 420c754eec556d0ceb9d7ba0d121971e3b1287b5:

  • Full bun run check: lint and typecheck passed; 368 tests passed, 0 failed.
  • Focused six-file suite: 66 tests passed, 0 failed.
  • Independent combined-policy and standards/spec reviews: no remaining findings.
  • git diff --check: passed.
  • Both diagrams: 9/9 showcase checks, zero errors/warnings; desktop containment and light/dark inspection passed.
  • Real Pi RPC startup, get_state, get_available_models, and shutdown passed with zero model prompts.

The native evidence was checked against content fingerprint sha256:62ef4e397b55ddb5c87bbd9a0e5c89fff69bcb23315653ae8365ff1dde4f3481. This records local verification, not GitHub CI status.

Acceptance checklist

Checked items mean automated verification passed on the head above; human acceptance is separate.

  • Old settings still parse and keep their existing model choices and effort behavior.
    • Result: passed.
    • Evidence: settings round-trip tests and legacy per-role routing/fallback cases in test/settings.test.ts and test/scheduler/model-routing.test.ts.
  • The allowlist governs effective defaults, primary routes, escalation, and new execution from recovered attempts.
    • Result: passed.
    • Evidence: exact admission tests in the advisor, Pi backend, and Pi harness; all are in the passing focused suite.
  • Kimi K3 at medium is rejected before a prompt; a catalog-qualified high route is admitted.
    • Result: passed with scripted catalog evidence.
    • Evidence: test/agents/pi/backend.test.ts rejects unsupported medium before starting a role client and runs the high-effort Kimi vertical fixture.
  • A primary high route retains high on a same-profile retry and uses medium after Codex Sol escalation.
    • Result: passed for the default role-effort configuration.
    • Evidence: test/scheduler/model-routing.test.ts verifies retained Terra effort and default Sol medium. Additional combined-policy tests preserve an explicit upstream efforts.implement override on Sol.
  • A denied model or model/effort readback mismatch produces no role prompt.
    • Result: passed.
    • Evidence: Pi backend admission and harness model/readback mismatch tests.
  • Recovery preserves the recorded model, effort, cursor, and work. It never silently substitutes the latest configured model.
    • Result: passed.
    • Evidence: harness tests cover denied recovery with and without a cursor, plus recorded-output replay without another model call. The cursor-less admission gap found during review was fixed.
  • Independent Review validates the exact trusted commit in a separate session.
    • Result: passed.
    • Evidence: the scripted Codex Scout -> Kimi Implement -> Codex Review flow in test/agents/pi/backend.test.ts and exact-target Review tests in test/agents/pi/harness.test.ts.
  • Onboarding preserves the routing fields; logs retain sanitized errors.
    • Result: passed.
    • Evidence: test/integration/onboard-skill-allowlist.test.ts and the settings diagnostic tests.
  • No new paid router call, extra reviewer, or automatic provider-switch loop is introduced.
    • Result: passed by source review.
    • Evidence: the registry remains Pi-only; the change extends the existing deterministic advisor and bounded retry flow.
  • Human acceptance after checking the criteria, evidence, and limitations.

Additional Notes

  • Benchmarking is deferred. No claim of lower cost or superior model quality is made.
  • The real Pi startup/state/catalog check sent no model prompt and confirmed clean shutdown. That environment exposes Codex only; live Kimi/GLM access remains unverified.
  • Generated diagram viewers and visual-check artifacts are included. Review the compact JSON sources in docs/design/ for their authored content.

Default role execution graph

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant