Skip to content

feat(decision-framing): add bounded research priors - #727

Merged
atomchung merged 1 commit into
mainfrom
codex/issue-715-generic-parity
Aug 1, 2026
Merged

feat(decision-framing): add bounded research priors#727
atomchung merged 1 commit into
mainfrom
codex/issue-715-generic-parity

Conversation

@atomchung

@atomchung atomchung commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Scope

  • Adds four audited, bounded research priors for the no-book route.
  • Adds the single SKILL.md → decision-framing.md → research-priors.md loading path.
  • Defines research-aware strategy framing as baseline → applicable strategy map → zero or one discriminating question last.
  • Keeps ordinary no-book single-trade framing separately bounded at three questions.

Boundaries

  • No engine arithmetic, schema, state, persistence, runtime web client, fund recommendation, allocation, suitability conclusion, forecast, or trade verdict.
  • No Generic Parity runner, case/witness bank, receipt platform, judge plumbing, or evaluator documentation.

Validation

  • python3.12 tests/test_research_priors.py
  • python3.12 tests/test_doc_language.py
  • python3.12 tests/test_output_voice.py
  • python3.12 tests/run_all.py (full offline suite)
  • Independent review, including a re-review after two scope/order corrections.

Evidence status

This is deterministic implementation evidence only. Owner-live, cross-model Generic Parity, and Antigravity testing are explicitly deferred.

@atomchung atomchung left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCK] Independent review found a private-boundary escape: --output-dir is caller-controlled and the runner writes exact raw model outputs under it without rejecting the effective coach-state root. python3 evals/run_generic_parity.py --output-dir ~/.trade-coach/parity would place evaluator output inside coach state, contradicting #715's receipt boundary. Please normalize and reject equal/nested output paths (including symlink resolution), add a regression test, then rerun review. Python 3.11 and 3.12 CI are green, but this remains a merge blocker.

Comment thread evals/run_generic_parity.py Outdated
import anthropic as anthropic_module
anthropic = anthropic_module
client = anthropic.Anthropic()
run_dir = args.output_dir / f"{dt.datetime.now(dt.timezone.utc).strftime('%Y%m%dT%H%M%SZ')}-{uuid.uuid4().hex[:8]}"

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[BLOCK] args.output_dir is not checked against the effective coach-state root before creating a directory and later writing A*.raw.txt and receipt.json. This permits raw evaluator output inside ~/.trade-coach when supplied by the caller, violating the stated outside-coach-state receipt boundary. Reject equal/nested paths after resolving symlinks (and test it).

Copy link
Copy Markdown
Owner Author

Review — blocked; do not merge or mark ready yet

Reviewed draft head 8e2ba87d3c81086554253ffbb3da6d37982af960 against #715 owner comment 5151689090, #27, and the already-open #726.

P1 — this creates a second owner for the same #715 product contract

#715 explicitly authorized one small draft/reviewable PR for the A01 loop. PR #726 already changes references/decision-framing.md, adds A01/A07/A10 witnesses, and owns the same visible outcome. This PR changes the same authority again while also adding a second evaluation implementation. Keeping both open makes it unclear which branch owns the product wording and guarantees drift/conflict.

Pick one surviving product PR. My recommendation is to keep the smaller #726 product leaf, rebase it onto main@5ed29d9, and treat any probe runner as local iteration tooling or a separately owned follow-up only if evidence proves it must be committed. Do not merge both.

P1 — the route order contradicts the accepted A01 outcome

decision-framing.md says the order is baseline → exception gate → strategy map → at most one question, but then says: “First ask the material exception: is the money needed…”. That restores question-first intake before the user receives the complete strategy-class map and can also create a second question after the map. The owner disposition requires available value and the bounded map first, followed by at most one discriminating question.

State liquidity/narrow-exposure as a material limitation/branch condition in the answer. Do not ask it before the map. Add a deterministic witness/mutation that fails if any question precedes the strategy-class completion.

P1 — the evaluator can report candidate_pass with no semantic evidence

generic_parity.receipt_disposition([], {}) returns candidate_pass: the classification list is empty, so neither ambiguity nor disagreement fires. This is a fail-open release-evidence bug. An empty report, a missing axis, or a partial axis set must be ambiguous/failure, never a pass. Add tests for empty, missing, and unknown classifications, and require the complete AXES set before candidate_pass.

P2 — the PR exceeds the authorized first loop

The owner allowed one A01 product correction and one A01/A07/A10 rerun; it explicitly did not authorize the full #716 catalogue or broad evaluator work. This PR adds four shared research priors, 11 changed files, a new case/witness bank, a runner, receipt format, docs, and judge plumbing. Those may be useful, but they are separate durable owners from the smallest #715 product cut and should not be bundled merely because quota prevented the live rerun.

Evidence status

Standard CI is green on Python 3.11/3.12, which proves the branch is mechanically stable. The required live evidence is still ambiguous due provider quota, and CI does not resolve the ownership/order/fail-open findings above.

Required before re-review:

  1. resolve feat(no-book): map strategy class before exit #726/feat(decision-framing): add bounded research priors #727 into one owning product PR;
  2. preserve value/map-before-question ordering;
  3. make semantic receipt aggregation fail closed on absent/partial axes;
  4. narrow or re-home the [research·product] Cross-route evidence-backed decision priors — 建立不低於通用模型的回答基準 #716 research authority and evaluator expansion;
  5. rerun the frozen A01/A07/A10 probe when independent-family capacity is available.

#713 remains the merge blocker; no owner-live acceptance is supplied by this PR.

Copy link
Copy Markdown
Owner Author

Owner disposition — surviving PR, but narrow to the product/reference slice

The owner now explicitly wants the next test candidate to have basic research-aware no-book answers. This supersedes the earlier recommendation to keep #726: PR #727 becomes the one surviving implementation owner, provided it is narrowed before re-review.

Required changes tonight:

  1. Keep the product slice only

    • retain references/research-priors.md with at most the four current priors;
    • retain the thin decision-framing.md consumption path;
    • add/verify the smallest guaranteed route pointer needed for the real no-book host to load the catalogue;
    • retain only deterministic witnesses needed to prove the product contract.
  2. Remove the evaluator expansion from this PR

    • remove evals/run_generic_parity.py, evals/generic_parity.py, new case/witness banks, judge plumbing and evaluator-design documentation unless an existing repository test genuinely requires a tiny reusable helper;
    • no new receipt format or committed cross-model harness in this slice;
    • tomorrow's live A01/A07/A10 probe may use existing/local tooling and can motivate a separate owner only if needed.
  3. Fix the visible interaction order

    • state the research-backed baseline and applicable limitation/exception conditions;
    • provide the bounded strategy-class map;
    • only then ask at most one discriminating question;
    • do not ask the liquidity/material-exception question before the user receives the available baseline and choices.
  4. Make the route actually consume the prior

    • a new reference file alone is dead documentation;
    • trace the supported no-book entry from SKILL.md/route authority to decision-framing.md and the new catalogue, and add a structural regression that fails if the pointer or catalogue disappears;
    • do not duplicate the full priors across prompt surfaces.
  5. Preserve strict boundaries

    • no named product/fund recommendation, ranking, geography choice, allocation percentage, cash-reserve amount, moving average, stop percentage, timing window, expected return, suitability claim, persistence, engine/schema/domain object or runtime web client;
    • thematic/sector ETF is not broad diversification;
    • tactical governance cannot leak into a long-horizon market policy;
    • recurring future savings remains distinct from staging already-available cash.

No owner-live or billable/cross-model fast probe tonight. Run focused deterministic tests, full offline suite and standard Python 3.11/3.12 CI. Update #716/#715 with exact head and evidence, then stop at ready-for-independent-review. Do not merge without a separate owner merge authorization.

@atomchung
atomchung force-pushed the codex/issue-715-generic-parity branch from 8e2ba87 to c5b79a3 Compare August 1, 2026 15:46
@atomchung atomchung changed the title feat(decision-framing): add generic parity probe feat(decision-framing): add bounded research priors Aug 1, 2026
@atomchung

Copy link
Copy Markdown
Owner Author

Independent re-review of c5b79a31bb77c1d96ef620bd931e05394fc7fac7: no findings.

The previous Generic Parity thread is outdated because its runner and raw-output path were removed from this PR. The current diff contains only the bounded reference slice; it keeps the baseline/map-before-question order, caps the research-aware path at one final question, and has deterministic loading/order mutation coverage. Full offline tests passed locally on Python 3.12; GitHub Actions passed Python 3.11 and 3.12 (run 30706651485). No owner-live, cross-model, or Antigravity acceptance is claimed.

@atomchung
atomchung marked this pull request as ready for review August 1, 2026 15:54
@atomchung
atomchung merged commit ee50914 into main Aug 1, 2026
3 checks passed
@atomchung
atomchung deleted the codex/issue-715-generic-parity branch August 1, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant