BaseInfinity · BaseInfinity · Jul 20, 2026 · Jul 20, 2026
diff --git a/ROADMAP.md b/ROADMAP.md
@@ -104,5 +104,7 @@ Living tracker of projects shipped using this wizard. **Rule:** only list projec
 | 439 | GPT-5.6 family (Sol/Terra/Luna) supersedes GPT-5.5 — reviewer-model doc sweep | **Found 2026-07-10:** OpenAI shipped a new flagship family — `gpt-5.6-sol` (flagship, "strongest for complex coding, computer use, research, cybersecurity"), `gpt-5.6-terra` (balanced, "competitive with GPT-5.5 at lower cost"), `gpt-5.6-luna` (high-volume/cheap) — verified against primary OpenAI docs (`developers.openai.com/api/docs/guides/latest-model`, `learn.chatgpt.com/docs/models`, `developers.openai.com/api/docs/pricing`), not inferred. This repo names GPT-5.5 as the cross-model reviewer in 6 live-guidance files inherited by every consumer repo — all updated to GPT-5.6 Sol: `AI_SETUP_LANES.md`, `README.md`, `CLAUDE_CODE_SDLC_WIZARD.md`, `skills/sdlc/SKILL.md` + `cowork/skills/sdlc/SKILL.md` (both copies, moved together per the doc-duplication-drift lesson from v1.85.0), `AGENTS.md`, and this repo's own `CLAUDE.md`. **Explicitly untouched (historical record, not live guidance):** `CHANGELOG.md`, `ROADMAP_ARCHIVE.md`, `ROADMAP.md`'s #302 citation, README.md's Andon Labs vending-bench citation (L151, names the model an actual past eval ran on), and this file's own L113 E2E-audit citation — rewriting any of these would falsify what model was actually used at the time; regression tests explicitly guard both untouched citations. **Design decision — auto-detect preserved, no script/hook changes:** a Fable-subagent scoping pass recommended pinning an explicit `-c model=gpt-5.6-sol` in `scripts/codex-review-with-progress.sh` and `CI_CD.md`'s invocation. Direct file reads found this was already a deliberate, documented design (self-updating without maintenance at each model bump — proven live: the CLI's own default already resolved to Sol with zero config change) independently stated in 3 places; pinning would trade that property away for marginal audit-trail precision. Kept auto-detect; this is a pure prose fix, not a behavior change. **Reasoning-tier question resolved:** user asked whether an "ultra" tier exists above `max` for the final review gate. Sourced research (run against Sol itself once the local Codex CLI was upgraded past its `gpt-5.6-sol`-incompatible v0.142.3) confirmed no — the effort axis is still `none/low/medium/high/xhigh/max`; Codex's "Ultra" is a separate multi-agent orchestration mode, not deeper single-model reasoning. No OpenAI-published data compares `xhigh` vs `max` for PR-style review specifically. Kept `xhigh` as the default (preserves the prior baseline per OpenAI's own migration guidance), documented Pro/`max` as an escalation option for unusually risky PRs in `AI_SETUP_LANES.md`'s Final Review Policy, mirroring the existing Sonnet-5→Opus-4.8 escalation pattern. **Incident during verification:** the pre-edit full-suite baseline run corrupted `README.md` and `tests/e2e/score-history.jsonl` — `tests/test-persist-score-history.sh`'s git-fixture setup silently fell back to running real git operations (bogus `"initial"`/`"seed history"` commits, a `feature/x` branch) against the actual repo when its tmpdir creation failed under a sandboxed shell, rather than hard-failing. Root-caused via `git reflog`, both files restored from the known-good commit, garbage branches removed (explicit user confirmation obtained before any force-delete), `main`/`origin/main` were never touched. Flagged as a separate follow-up bug in `test-persist-score-history.sh` (needs to hard-fail on tmpdir-creation failure instead of silently using cwd) — not fixed here, out of scope; the same test independently reproduced a second symptom of this fragility during this task's own final regression sweep (its fixture push got blocked by the afterhours hook, unrelated to model-name content — confirmed no repo damage this time). **Second gap caught before review:** re-grepping all 7 target files from scratch (rather than trusting prior session state) found `CLAUDE.md:75` still said "GPT-5.5" — dropped during the corruption/recovery above and never redone. Caught RED via its own regression test, fixed, GREEN. **Fable review (fresh instance):** clean — independently re-grepped every spelling of "5.5"/"5.4" repo-wide (including files outside the original scope list, e.g. `RESEARCH_58_CLAW_OMO_OMX.md`) and confirmed every remaining hit is a genuine historical citation. Found and fixed one real nit: a stale comment claiming the test helper used a bash nameref (it doesn't — bash 3.x/macOS has none; it echoes for command-substitution capture). **Codex xhigh (Sol) cross-model review — 3 rounds to CERTIFIED, not rubber-stamped:** round 1 found 2 P1s — (a) the new escalation-to-`max`/Pro paragraph in `AI_SETUP_LANES.md` made 2 pre-existing lines (`README.md:178`, `CLAUDE_CODE_SDLC_WIZARD.md:260`) stale, since both still called `xhigh` "the highest" tier; fixed both to describe `xhigh` as the default with escalation, outside the original 6-file scope but directly caused by this diff; (b) the 8 new regression tests only checked for the `5.6` substring, so they passed under a demonstrated Sol→Terra (primary reviewer) or Terra→Luna (fallback target) swap — fixed by requiring `Sol`/`Terra` by name, mutation-verified against the reviewer's own 3 example mutations. Round 2 found a further gap in that same fix: requiring `Sol` alone (without also requiring `5.6`) still passed a demonstrated GPT-5.6 Sol → GPT-5.7 Sol mutation — fixed by refactoring the test helper to require a comma-separated list of ALL of `5.6` + `Sol`/`Terra` together per line, re-verified against all 4 mutations (the 3 from round 1 plus the new one). Round 3: CERTIFIED, no further findings. **Entry gate:** this repo is the canonical source other wizard siblings inherit "GPT-5.5 xhigh reviewer" guidance from — stale model guidance here propagates everywhere. **Verified:** 8 new per-location regression tests in `tests/test-doc-consistency.sh` (RED confirmed against pre-edit content, GREEN after; mutation-tested against 4 distinct swap scenarios, not just 2), full test suite before/after comparison, Fable review, 3-round Codex xhigh (Sol) cross-model review to CERTIFIED. |
 | 448 | Copilot CLI compatibility watch — second strict consumer of our SKILL.md format | **Demand signal 2026-07-14 (external-user pain event, #444):** GitHub Copilot CLI ≥1.0.65 tightened `argument-hint` frontmatter validation to string-only and silently dropped all 7 wizard skills for an external user — thejesh23, the repo's **first outside contributor** (fix merged as PR #445 under his authorship; regression test added in PR #447 so the bug class can't return). We learned about the breaking change from his bug report, not from monitoring: `weekly-update.yml` and `cc-version-drift.yml` watch only `anthropics/claude-code`. Copilot CLI ships multiple releases per week (v1.0.69 Jul 7 → v1.0.70 Jul 10 → v1.0.71-2 Jul 15, checked directly against the GitHub releases API this session) and is actively building out its skills/plugins surface per its own release notes (`copilot skill list`/`add`, skill pickers, plugin marketplaces) — every release is a fresh chance for another validation tightening. **Scope:** (a) extend release watching to `github/copilot-cli` — a cheap no-LLM check in the style of `cc-version-drift.yml`, flagging releases whose notes mention skill/frontmatter/slash-command/validation changes; (b) treat SKILL.md frontmatter as a cross-vendor surface going forward: any NEW frontmatter key added to wizard skills gets checked against both Claude Code and Copilot CLI semantics before shipping (the #447 test guards `argument-hint` specifically; new keys need the same scrutiny at introduction); (c) explicitly distinct from the parked "Copilot Cowork SDLC port" row — this row protects EXISTING consumers of the current skills, the port would be a new product with its own Prove-It gate. **Entry gate:** external-user pain event, demonstrated live — a consumer's install silently lost every skill. |
 | 449 | Release-drift gate — main must not silently outrun the published npm package | **Incident 2026-07-14 (found while shipping #444's fix):** npm `latest` was still v1.86.0 while main sat 7 consumer-affecting commits ahead (GPT-5.6 reviewer sweep #441, Sonnet-5-medium + hook floor #443, the #445 Copilot fix itself, #446, #447) — `npx agentic-sdlc-wizard@latest init` was still installing skills Copilot CLI silently drops even after the fix was merged, thanked, and the issue closed, and nothing in the process would ever have prompted a release (the six earlier commits had been accumulating unreleased since July 7, with the last published tag dating to July 5). **Root causes:** (a) publish requires a manual `git tag v* && git push origin v*` that nothing prompts for — the rule lived only in the maintainer's private memory (`feedback_tag_for_npm_release`), the exact "process rule as private memory = /sdlc gap" anti-pattern the Memory Audit Protocol names; (b) the "small commits stay local until a release" habit plus "docs don't feel release-worthy" let guidance changes pile up with no release decision ever being made; (c) `cc-version-drift.yml` watches CLAUDE CODE's npm drift but nothing watches OUR OWN package's drift vs main — the machinery existed, pointed outward. **Scope:** (a) extend the drift workflow (or add a sibling job) to compare `npm view agentic-sdlc-wizard version` against main's `package.json` AND count commits since the last `v*` tag touching consumer-distributed paths (`skills/`, `hooks/`, `cli/`, `CLAUDE_CODE_SDLC_WIZARD.md`, `cowork/`); past a threshold (e.g. any consumer-path change older than 7 days, or ≥5 commits), open/refresh a "release due" issue listing the stranded commits — no LLM, same style as the existing drift check; (b) promote the tag-to-publish rule from private memory into `SDLC.md` Lessons Learned + the wizard doc's release section (Memory Audit Protocol promotion path); (c) explicitly NOT auto-publish-on-merge — the human release gate and Release Review Focus checklist stay; the fix targets the forgetting, not the gate. Fixed-in-part by v1.87.0 (the catch-up release itself, same day). **Entry gate:** maintainer pain event, demonstrated live — the fix was merged and celebrated while production consumers kept installing the bug, with no mechanism anywhere that would have changed that. |
+| 450 | WebFetch summarization is unreliable for verification-grade doc citations — caught mid-review costing a full Codex round | **Maintainer pain event 2026-07-20 (#456 arc):** while fixing the Cowork hook-enforcement bug, Claude cited Anthropic's `prompt`-hook response schema as `{"yes": true/false}` via `WebFetch`, used that citation as the stated root cause in a preflight doc, and shipped a fix built on it. Codex xhigh round-1 review disputed the citation with its own doc fetch, correctly stating the real schema is `{"ok": true, "reason": "..."}`. Re-fetching the SAME URL with WebFetch a second time produced a THIRD different, still-incomplete summary ("field name not specified in this excerpt") — confirming the tool's built-in summarization model is genuinely inconsistent across calls, not a one-off fluke. Only a raw `curl` fetch of the `.md` version of the doc produced the actual, verbatim, load-bearing schema. Cost: a full extra Codex review round (round 1's finding 1) that a correct citation the first time would have avoided. **Root cause:** `WebFetch` runs fetched content through "a small, fast model" (per its own tool description) before returning it — fine for exploratory reading, unsafe when the fetched fact will be asserted as ground truth in a review artifact (preflight docs, root-cause claims, handoff.json) that a cross-model reviewer or future session will trust without re-checking. **Scope (Actionable now, process-only, zero-API):** (a) add a line to `skills/sdlc/SKILL.md`'s Cross-Model Review section: when a preflight/handoff document cites an external doc as the stated root cause or as evidence a reviewer is told to trust, prefer a raw fetch (`curl <url>.md` or equivalent, when the host allows it) over `WebFetch` for that specific citation, or explicitly quote the verbatim retrieved text rather than a paraphrase; (b) note in `SDLC.md`'s gotchas that `WebFetch` summaries are not guaranteed byte-stable across repeated calls to the same URL — don't treat a first WebFetch result as verified without a second independent check when it's about to become a claim other reviewers rely on. **Entry gate:** maintainer pain event, demonstrated live twice in the same session (two different WebFetch calls to the identical URL, two different wrong/incomplete answers) — not theoretical. |
+| 451 | Post-session discipline (postmortem capture, ROADMAP logging) should run proactively, not only when the maintainer asks | **Maintainer pain event 2026-07-20:** "i just hate how i have ot always remind you thats not good /sdlc." This is the second time this exact complaint has been raised almost verbatim — #436 records the first instance (2026-07-04, "I hate having to remind 'don't forget to cross-model review with codex'"), which was fixed mechanically (hooks that printed warnings but never actually blocked). Tonight's instance is different in kind: `skills/sdlc/SKILL.md`'s own "After Session (Capture Learnings)" and "Post-Mortem: Process Failures Become Rules" sections already say learnings should be captured and turned into rules — but nothing about the SDLC skill's own structure makes Claude actually run that step at natural session-close points (after a merge, before declaring a multi-PR session done) without being explicitly asked. Verified same session: Claude also gave a wrong answer once (claiming a cross-model reviewer discrepancy existed, based on a single miscalibrated ad-hoc test) before the maintainer's pushback forced a proper re-check against real evidence — a related but distinct failure (jumping to a conclusion instead of verifying against primary transcripts first). **Root cause:** unlike TDD RED/GREEN or the cross-model review gate, there is no hook-level or skill-level trigger for "session end, multi-PR arc complete, or maintainer signals wrap-up → offer/run a postmortem pass automatically." The capture step exists in the doc but is opt-in-by-memory, the same "/sdlc gap living as an unenforced doc section" pattern #436 and #449 both independently found in other places. **Scope (Actionable now, process-only, zero-API):** (a) add an explicit trigger condition to `skills/sdlc/SKILL.md`'s After Session section: when a session ships 2+ merged PRs, or the maintainer uses language like "good session"/"that's it for tonight"/"what's next" after a merge, proactively offer (not silently skip) a short postmortem — what went well, what cost extra rounds, any new process rule worth logging — before considering the session's SDLC checklist complete; (b) fold this pattern explicitly into the checklist item "Capture learnings (after session — TESTING.md, CLAUDE.md, or feature docs)" so it's not just a description but a triggerable step in the TodoWrite list itself. **Entry gate:** maintainer pain event, second occurrence of the same complaint class (#436, #451) — the specific mechanism differs each time (hooks that don't block vs. a doc section nobody auto-triggers) but the pattern ("I have to keep asking for good /sdlc") is now recurring, which is itself the signal this needs a structural fix, not another one-off apology. |
 
 _67 fully-resolved rows archived → [ROADMAP_ARCHIVE.md](ROADMAP_ARCHIVE.md), including #231 (shepherd migration, the single largest row in the pre-cut file)._