Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions AI_SETUP_LANES.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,16 @@ This is **guidance, not a hard rule**. Maintainer override is always allowed.
| **Advisor** | Fable 5 (via `advisorModel: "fable"`) | `high` (server-side) |
| **Driver** | Sonnet 5 (`claude-sonnet-5`) | `medium` default, escalate `high` → `xhigh` for hard tasks |
| **Reviewer** | Codex (GPT-5.6 Sol) xhigh | — |
| **Escalation** | Opus 4.8 xhigh or Fable 5 review | When stuck or high-stakes |
| **Escalation** | Opus 4.8 xhigh takes over as driver (or run a Fable 5 review pass) | When stuck (2 failed attempts / LOW confidence) or high-stakes |

The new standard. Sonnet 5 beats Opus 4.6 on every coding benchmark (SWE-bench Verified 85.2% vs 80.8%, Terminal-Bench 80.4% vs 65.4%) and generally uses less Max quota — but the savings are not a fixed ratio: Sonnet 5's newer tokenizer produces ~30% more tokens for the same text than Opus 4.6's (per Anthropic's pricing docs), and community cost reports suggest the advantage narrows at `high`/`xhigh`. No controlled Sonnet-5-vs-Opus-4.6 quota measurement exists — check your own burn with `/usage`. Fable 5 advises at key decision points via native `advisorModel` (v2.1.170+). GPT-5.6 Sol xhigh reviews cross-family. Escalate to Opus 4.8 xhigh for the hardest debugging or architecture decisions — don't run Opus as the daily driver.

**Effort escalation ladder:** Start at `medium` — CodeRabbit's testing found it captures most of Sonnet 5's upside at the lowest cost. Raise to `high` when medium struggles, `xhigh` for hard debugging, multi-file migrations, or long agent runs. `max` is rarely worth it — doubles cost for marginal gains per the same CodeRabbit testing.

**Two escalation axes — don't conflate them.** The effort ladder above raises reasoning *within* the Sonnet 5 driver. Model escalation is a different move: it *swaps the driver* — after 2 failed attempts, LOW confidence, or on a high-stakes change, Opus 4.8 xhigh takes over as driver (or run a Fable 5 review pass on the diff). Cranking Sonnet's effort past `xhigh` is not the escalation path.

**Advisor failure has a fallback, not a shrug.** `advisor()` is a server-side tool and does fail during API incidents. When it errors, spawn a Fable subagent as the fallback reviewer — the same rule the `/sdlc` skill carries ("if down, spawn Fable subagent"). The advisor check is never skipped; only its transport changes.

**Requires:** Claude Code v2.1.197+ (Sonnet 5 alias resolution), Fable 5 access for advisor.

## Setup B — Opus 4.6 Stability (Legacy Flagship)
Expand Down Expand Up @@ -142,10 +146,10 @@ If the advisor returns "Advisor unavailable," the server-side harness failed to

**Step 1 — restart the session.** Exit and run `claude` (not `--resume`). A fresh process re-initializes the server handshake. This resolves most advisor failures.

**Step 2 — if the API incident persists:**
**Step 2 — if the API incident persists, swap the transport, not the check:**

- Continue with your driver model and no advisor`/model sonnet` for Setup A, `/model claude-opus-4-6` for Setup B. Interactive — stays on your Max subscription.
- Or proceed without the advisor and let the Codex xhigh PR gate catch issues.
- Spawn a Fable subagent as the fallback reviewerthe `/sdlc` skill's standing rule ("if down, spawn Fable subagent"). Batch your plan or open questions into one subagent consult at each point where you'd have called `advisor()`. Runs interactively on your Max subscription like any other agent.
- Keep driving with your lane's model (`/model sonnet` for Setup A, `/model claude-opus-4-6` for Setup B). The subagent replaces the advisor's transport; the Codex xhigh PR gate remains the separate final backstop, not a substitute for the advisor check.

**Last resort (scripted/CI only):**

Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,14 @@ The wizard defines four AI coding setups in [`AI_SETUP_LANES.md`](AI_SETUP_LANES

Setup D's whole point: **the discipline of knowing when NOT to use discipline.** When blast radius is low and you just need fast cheap hands, skip the SDLC overhead.

### Reading Setup A precisely

Clarified 2026-07-13 after these exact points kept getting re-confused; each rule states its why:

- **Effort escalation stays inside the driver.** Sonnet 5 starts at `medium`; `/effort high` when it struggles, `xhigh` for hard debugging or long agent runs. Why `medium` and not `high`: the old "start at `high`" guidance was removed in ROADMAP #440 — it had no measurement behind it, and CodeRabbit's testing (the source this repo already cites) found `medium` captures most of Sonnet 5's upside without paying for the top effort tiers, while community cost reports suggest the quota advantage erodes at `high`/`xhigh`.
- **Model escalation swaps the driver.** After 2 failed attempts, LOW confidence, or on high-stakes changes, Opus 4.8 xhigh takes over as driver (or run a Fable 5 review pass on the diff). Why a swap and not more effort: the lane's policy treats repeated failure as a sign the *approach* needs different eyes, not deeper reasoning on the same track — so once the effort ladder is exhausted, the next rung is a different model, not a bigger bill.
- **Advisor failure has a fallback, not a shrug.** Fable 5 advises via `advisorModel: "fable"`; when `advisor()` errors (it's a server-side tool — API incidents happen), spawn a Fable subagent as the fallback reviewer, exactly as the `/sdlc` skill prescribes. Why: the advisor's job is catching wrong approaches *before* they're built, so a transport failure changes how the advice is obtained — not whether the check happens.

**A note on `[1m]` and billing.** Sonnet 5 always runs at its native 1M context — no `[1m]` suffix needed, no separate billing tier. For Opus, the `[1m]` suffix is the 1M-context alias; as of [March 2026](https://claude.com/blog/1m-context-ga), 1M context is GA at standard pricing — **no long-context surcharge, no premium tier, no API-only restriction.** Interactive Claude Code sessions on Max / Team / Enterprise plans include 1M context automatically. (Pro users need "Enable usage credits" turned on once.) The [June 15, 2026 billing split](https://codersera.com/blog/anthropic-june-2026-billing-change-claude-code/) moved *headless* surfaces — `claude -p`, Agent SDK, GitHub Actions, third-party apps — off the Max subscription onto a separate metered credit pool. Interactive Claude Code in your terminal stays on Max. Full details in [`AI_SETUP_LANES.md` § How Billing Works](AI_SETUP_LANES.md#how-billing-works--1m-context-max-plan-and-the-june-15-split).

## How It Works
Expand Down
3 changes: 2 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ Add entries when a candidate is mid-evidence (one signal exists but doesn't yet
| Advisor auto-fallback in SDLC skill | **TRIGGER FIRED 2026-06-11.** When `advisor()` fails, the SDLC skill should automatically spawn a Fable subagent as fallback — proved in v1.82.0 triage (batch-reviewed 4 issues, caught #395 already fixed). Currently this pattern lives only in private memory; needs to be in `skills/sdlc/SKILL.md` so every project gets it. Also: auto-try advisor before every plan, not just when user asks. | Confidence ramp trial ships clean | 2026-07-11 | v1.82.0 triage session 2026-06-11 |
| Detect process-rule memories in /feedback or /setup | Users saving memories like "always run tests" are patching /sdlc gaps with private memory. /feedback or /setup could detect `type: feedback` entries with process-rule patterns and suggest contributing back to /sdlc instead. | Maintainer runs memory audit and finds ≥3 process-rule memories | 2026-08-11 | v1.83.0 session 2026-06-12 |
| Copilot Cowork SDLC port | GitHub Copilot desktop app launched in technical preview (June 2, 2026) with parallel multi-agent in git worktrees. Could port SDLC wizard as a Copilot extension. Different plugin format from Claude — would be a new sibling repo like `codex-sdlc-wizard`. Low priority: no demand signal, Copilot extension ecosystem is immature. | Second external user asks OR Copilot extension format stabilizes with hooks equivalent | 2026-08-20 | v1.84.0 triage session 2026-06-20 |
| Run /insights with Fable on this repo | Native CC `/insights` generates session usage report (friction patterns, satisfaction, outcome summaries). Run once with Fable model at max effort to get a high-quality analysis of how the wizard is actually being used. Review the HTML report for patterns that could improve /sdlc or hooks. | Next Fable session | 2026-07-22 | v1.83.0 session 2026-06-12, re-dated 2026-06-22 triage |
| ~~Run /insights with Fable on this repo~~ | **TRIGGER FIRED 2026-07-13.** Ran on Fable at max effort; global report generated + all 22 session facets analyzed per-repo. Findings: `/insights` is global-only (per-repo auditing stays in the filed GH issues — it can't do that job); the 30-day `cleanupPeriodDays` default had already deleted all history for ~15 project dirs, so **retention, not analysis, is the coverage bottleneck** (fix: raise `cleanupPeriodDays`); 8/11 surviving real sessions fully achieved, every failure environmental (Apple sync, npm perms, sandbox) not comprehension; the one recurring Claude-side friction family (overconfidence vs user-presented evidence, 3 incidents) already has 2 of 3 memory rules institutionalized. No /sdlc or hook changes warranted from this pass. | ~~Next Fable session~~ **FIRED** | — | v1.83.0 session 2026-06-12, re-dated 2026-06-22 triage |
| Rename research: "sdlc-wizard" → "SDLC harness" / "agentic harness"? | Maintainer signal 2026-07-13: "SDLC-wizard might not work [as a name] — maybe SDLC harness or agentic harness following SDLC — we can workshop it later." The product has arguably outgrown "wizard": it's an enforcement harness (blocking hooks, codex gate, cross-model review loops, drift tests), not a one-shot setup dialog. Tension to resolve: the xdlc ecosystem preamble explicitly says sdlc-wizard "earned the wizard form" — a rename must engage that history, not ignore it. Blast radius is large (repo name, npm package, sibling repos' inherited docs, README/ARCHITECTURE, xdlc references), so this needs a naming workshop + migration plan, not a drive-by edit. Workshop only — no rename before candidates are compared and consumer impact is mapped. | Maintainer schedules the naming workshop OR a second person independently reports name confusion OR next major-version planning starts | 2026-09-11 | This session 2026-07-13 (post-#440 wrap-up) |
| Memory audit + repo efficiency pass with Fable | Run Memory Audit Protocol on private memory files — promote portable lessons, delete stale entries. Then: Fable batch-review the full repo (SKILL.md files, hooks, wizard doc) for efficiency, dead code, stale references, and consolidation opportunities. Goal: build a 95% confidence task list of cleanup items that can be worked in parallel. | Next Fable session | 2026-07-22 | v1.82.0 triage session 2026-06-11, re-dated 2026-06-22 triage |
| ~~Sync AI Setup Lanes to Claude-family sibling wizards~~ | **DONE 2026-06-11.** gdlc v0.3.0, rdlc v0.7.0 shipped with AI Setup Lanes v2. Cowork plugin port in PR #410. Originally: `claude-gdlc-wizard` (v0.2.2) and `claude-rdlc-wizard` (v0.6.1) have stale 69-70 line AI_SETUP_LANES.md vs sdlc-wizard's 217 lines. Missing: 3-lane structure (Premium/Saver/Lite), advisor fallback escalation, Fable effort guidance, usage signals, autocompact cross-ref. Each needs a tailored port — gdlc is game-dev domain, rdlc is research domain. Codex/xdlc/ldlc out of scope (different ecosystem). When people run `/update` in those repos they should get the same Premium lane experience. | Next release of either sibling | 2026-07-11 | v1.82.0 triage session 2026-06-11 |

Expand Down
47 changes: 44 additions & 3 deletions tests/test-doc-consistency.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1255,12 +1255,13 @@ test_ai_setup_lanes_reviewer_is_gpt56() {
local bad=""
# "5\.6" AND "Sol" (not just one or the other) so a Sol->Terra swap, or a
# future GPT-5.7 Sol rename, both fail.
for n in 13 16 28 41 43 119 164 168 206 207 210; do
# (Line numbers re-pinned +4 after the 2026-07-13 Setup A clarity insertion.)
for n in 13 16 32 45 47 123 168 172 210 211 214; do
bad="$bad$(_check_line_has_and_lacks "$F" "$n" "5\.6,Sol" "5\.5")"
done
# L123 is the fallback-chain line: must name "5\.6" AND BOTH Sol (primary)
# L127 is the fallback-chain line: must name "5\.6" AND BOTH Sol (primary)
# and Terra (fallback target) so a Terra->Luna swap also fails.
bad="$bad$(_check_line_has_and_lacks "$F" 123 "5\.6,Sol,Terra" "5\.5" "5\.4")"
bad="$bad$(_check_line_has_and_lacks "$F" 127 "5\.6,Sol,Terra" "5\.5" "5\.4")"
if [ -z "$bad" ]; then
pass "AI_SETUP_LANES.md: all reviewer-model lines reference GPT-5.6 Sol/Terra, none reference stale GPT-5.5/5.4"
else
Expand Down Expand Up @@ -1474,6 +1475,46 @@ test_sonnet5_default_effort_is_medium
test_no_unsupported_sonnet5_sweet_spot
test_opus46_max_sweet_spot_guard

# #440 follow-up (maintainer ask 2026-07-13): Setup A's two escalation axes and
# the advisor-failure fallback kept getting re-confused ("sonnet 5 xhigh? or
# high?" / "sometimes advisor fails then we need a sub agent remember").
# Both README and AI_SETUP_LANES must state, explicitly:
# (a) model escalation SWAPS THE DRIVER (Opus 4.8 xhigh takes over) — it is
# not a higher rung on Sonnet's effort ladder;
# (b) when advisor() errors, the fallback is spawning a Fable subagent — the
# check is never skipped (same rule the /sdlc skill already carries).
test_setup_a_escalation_and_advisor_fallback_explicit() {
local bad=""
grep -q 'takes over as driver' "$REPO_ROOT/AI_SETUP_LANES.md" \
|| bad="$bad AI_SETUP_LANES.md:driver-swap"
grep -q 'spawn a Fable subagent' "$REPO_ROOT/AI_SETUP_LANES.md" \
|| bad="$bad AI_SETUP_LANES.md:advisor-fallback"
grep -q 'takes over as driver' "$REPO_ROOT/README.md" \
|| bad="$bad README.md:driver-swap"
grep -q 'spawn a Fable subagent' "$REPO_ROOT/README.md" \
|| bad="$bad README.md:advisor-fallback"
# Negative half (Codex round-1 P1-3): positive assertions alone false-green
# while the advisor-outage procedure still offers a "no advisor" path. The
# outage section must route to the subagent fallback, never to skipping.
local outage
outage="$(sed -n '/^## When the Advisor Is Unavailable/,/^## [^W]/p' "$REPO_ROOT/AI_SETUP_LANES.md")"
if [ -z "$outage" ]; then
bad="$bad AI_SETUP_LANES.md:outage-section-missing"
else
printf '%s' "$outage" | grep -qiE 'no advisor|without the advisor' \
&& bad="$bad AI_SETUP_LANES.md:outage-still-offers-skip-path"
printf '%s' "$outage" | grep -q 'Fable subagent' \
|| bad="$bad AI_SETUP_LANES.md:outage-missing-subagent-fallback"
fi
if [ -z "$bad" ]; then
pass "Setup A: driver-swap escalation + advisor subagent fallback explicit in README and AI_SETUP_LANES"
else
fail "Setup A clarity missing:$bad"
fi
}

test_setup_a_escalation_and_advisor_fallback_explicit

# ────────────────────────────────────────────
# Summary
# ────────────────────────────────────────────
Expand Down