Skip to content

feat(release): AI-agentic release skill with multi-model consensus - #662

Draft
martintmk wants to merge 22 commits into
mainfrom
user/martintomka/20260812-release-agentic-skill
Draft

feat(release): AI-agentic release skill with multi-model consensus#662
martintmk wants to merge 22 commits into
mainfrom
user/martintomka/20260812-release-agentic-skill

Conversation

@martintmk

@martintmk martintmk commented Aug 12, 2026

Copy link
Copy Markdown
Member

What

Repackages the release workflow as a unified repository skill at .github/skills/release-packages/, with model judgment limited to source compatibility and proc-macro contracts while deterministic scripts own version arithmetic, cascades, ordering, and atomic application.

Architecture

  • SKILL.md orchestrates targeted, changed, and all-package releases plus multi-model consensus.
  • release-facts.ps1 emits schema-v2 workspace facts: dependency/type exposure edges, proc-macro publication edges, implementation closures, runtime façades, baselines, and modification state.
  • resolve-plan.ps1 owns token parsing, Cargo-aware SemVer arithmetic, exact pins, fixed-point cascades, macro-contract attestations, ambiguity blocking, warnings, and dependency-first ordering.
  • apply-plan.ps1 rejects blocked or stale plans and performs manifest edits, changelog/README generation, validation, and rollback atomically.
  • scripts/ci/semver-report.ps1 remains the CI entry point over the shared release library.

Important behavior

  • Ordinary Rust dependency breaks propagate through public type-exposure edges; every direct released dependency still gives published dependents a patch pickup floor.
  • Proc-macro implementation dependency versions do not automatically imply a breaking consumer contract. Macro releases require reviewed attestations covering exports, accepted syntax, compile behavior, generated API/runtime paths, and hygiene.
  • Missing, incomplete, contradictory, or ambiguous macro reviews block resolution instead of guessing.
  • Public macro façades are inferred from concrete allowlist roots. Wildcards and unpublished consumers are not treated as publication evidence; optional macro_runtime metadata is reserved for exceptional generated-runtime relationships.
  • Unpublished implementation helpers participate in macro review scope without becoming release candidates.
  • Exact proc-macro pins and cascade-reached non-patch macro classifications still require contract review.
  • First releases, pins, force warnings, downstream effective types, dependency ordering, stale-plan checks, and rollback remain deterministic.

Validation

  • 541 Pester tests pass, covering stable, 0.x, and 0.0.x arithmetic; pins; first releases; fixed-point chains and diamonds; ordinary exposure; compatible/breaking proc-macro contracts; public/private macro use; renamed and wildcard edges; generated-runtime ambiguity; unpublished helpers; stale schemas; malformed attestations; changelogs; and atomic rollback.
  • Workspace formatting, README generation, spelling, and Clippy pass.
  • The live templated_uri oracle keeps templated_uri_macros_impl and templated_uri_macros at patch for a compatible macro contract while preserving templated_uri's independent breaking ohno type exposure.
  • The last 10 distinct historical release commits were replayed in isolated worktrees, with the audit posted at feat(release): AI-agentic release skill with multi-model consensus #662 (comment).
  • Independent Claude and GPT reviews were applied, including fail-closed fixes for stale facts, exact pins, cascade-reached macro classifications, wildcard/renamed publication evidence, registry dependencies, and unpublished implementation helpers.

The retired prompt and top-level helper locations are removed; release documentation now points to the unified skill bundle.

martintmk and others added 2 commits August 12, 2026 18:51
…helpers

Introduce two small, thin-shell PowerShell helpers that the forthcoming
AI-agentic release skill forwards its mechanical sub-tasks to, so that
different reasoning models start from an identical, machine-checked basis
instead of hand-parsing cargo metadata / git and possibly diverging:

- scripts/release-facts.ps1: emits deterministic workspace release facts as
  JSON (folder/name/version/published/proc-macro/deps, baseline commit sha,
  and the unreleased-modification set), reusing Get-WorkspacePackages,
  Get-PreviousVersionBumpCommit and Get-PackagesWithUnreleasedChanges.
- scripts/release-changelog.ps1: regenerates one package CHANGELOG.md via the
  existing tested Write-Changelog (conventional-commit sections, PR links,
  Unreleased folding, cascade "Now requires X of Y" bullets).

Both are thin shells over the existing scripts/lib release library; they add
no new release logic. Covered by new Pester tests reusing the synthetic
workspace harness. CI's scripts/ci/semver-report.ps1 is untouched.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59264f5d-db1f-4379-9cc3-6663f2914b90
Add .github/prompts/release-packages.prompt.md, an AI-agentic replacement
for the interactive release-packages.ps1 driver. It encodes the release
planning algorithm (token parsing, cargo-semver-checks classification,
proc-macro manual review, cascade toward dependents with patch floor +
stronger-of, elevation invariants A/B, explicit-pin rules, and the
version-bump table) as precise, model-independent instructions so the
affected-package set and version-bump sequence are reproducible across
reasoning models.

Mechanical sub-tasks are forwarded to the deterministic helpers
(release-facts.ps1, release-changelog.ps1, cargo semver-checks, just readme)
rather than reimplemented in prose. Before any write, a multi-model
consensus gate requires independent models to agree on the plan. The skill
also corrects current-script shortcomings: atomic apply gated by cargo check
with rollback, conservative handling of an unresolved baseline for an
already-released crate, validated dependency-requirement edits, and explicit
proc-macro review flags. Intentional Cargo 0.x/0.0.x conservatism preserved.

docs/releasing.md now documents the AI-agentic path alongside the existing
driver, which (and scripts/ci/semver-report.ps1) remains in place for CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59264f5d-db1f-4379-9cc3-6663f2914b90
@martintmk martintmk added the agency-rocket Touched by a rocket skill label Aug 12, 2026
@codecov

codecov Bot commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (ddacce5) to head (b58d192).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main     #662    +/-   ##
========================================
  Coverage   100.0%   100.0%            
========================================
  Files         503      503            
  Lines       57407    57585   +178     
========================================
+ Hits        57407    57585   +178     
Flag Coverage Δ
linux 77.2% <ø> (?)
linux-arm 76.7% <ø> (?)
windows 78.4% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

martintmk and others added 2 commits August 12, 2026 19:28
Multi-model code review (GPT-5.5, Sonnet-4.6) findings:

- Skill Step 4 (cascade): the "one level per user-source seed" wording
  contradicted "transitive" and could let a model produce an incomplete plan
  for chains of 3+ packages (miss `top` in `bottom -> middle -> top`). Rewrote
  it as an explicit fixed-point transitive closure that also expands
  cascade-added entries.
- Skill Step 5 (review queue): the hard "published AND modified" gate conflicted
  with -All mode, which must walk every publishable package. Made the initial
  review roots mode-specific (targeted/changed = published+modified;
  all = every published) while keeping the modified gate for later findings.
- release-facts.ps1: an unresolvable/un-fetched BaseRef was swallowed and
  disguised as "no baseline", silently dropping each crate's semver floor.
  Validate BaseRef once up front with Test-GitRef and fail loudly; genuine
  new-crate cases still yield a null baseline. Added a regression test.
- ReleaseFacts test: the publish=false assertion passed vacuously (priv_pkg was
  never modified). It now modifies the unpublished package, so the assertion
  actually exercises the published filter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59264f5d-db1f-4379-9cc3-6663f2914b90
Opus review (High): Step 5 deterministically defaults to the objective
cargo-semver-checks floor, so the skill cannot and should not reproduce golden
scenarios whose Expect.Released encodes a subjective operator choice not derivable
from the diff (S16 elevating a comment-only change to non-breaking; S22/S23 -All
force-releasing unchanged packages). The blanket "MUST match Expect.Released"
claim overreached and the S16 example was misleading.

- Step 5: decide change type from diff evidence -- default to the tool floor
  (which already reflects additions as non-breaking and removals as breaking),
  elevate only on citable evidence, and leave OUT any package with no
  consumer-visible change (including unmodified packages surfaced by -All). Forcing
  an unchanged release or a level above the evidence now requires an explicit token,
  never a model guess. This makes -All-with-no-changes deterministically release
  nothing.
- Determinism test cases: split into FACT-DETERMINED scenarios (S13, S17, S18, 0.x
  cascade, 0.0.z) that MUST match, and JUDGMENT-DEPENDENT scenarios (S16, S22, S23)
  where the consensus gate requires model agreement but the historical human choice
  is not an oracle. Corrected the S16 example honestly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59264f5d-db1f-4379-9cc3-6663f2914b90
Comment thread .github/prompts/release-packages.prompt.md Outdated
Comment thread .github/prompts/release-packages.prompt.md Outdated
Address PR review feedback (martintmk):

- Change spec is now optional in targeted mode. A bare `name` token lets the
  skill determine the change type objectively (cargo-semver-checks for libraries,
  diff review for proc-macros) exactly as it does for cascade dependents, so the
  caller no longer has to state it. `name@type` remains a lower-bound override and
  `name@version` an explicit pin. Updated Inputs, Step 1, and Step 2 accordingly.
- On multi-model consensus divergence the gate now emits a structured ambiguity
  report: which package/rule diverged, each model's decision, a quote of the
  under-specified sentence, and a concrete proposed edit to this skill that would
  make the case deterministic -- so every divergence improves the skill instead of
  only blocking.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59264f5d-db1f-4379-9cc3-6663f2914b90
@martintmk

Copy link
Copy Markdown
Member Author

can you simulate a release of tick crate according to change history and output the affected versions according to new prompt?

@martintmk

Copy link
Copy Markdown
Member Author

review this PR

Comment thread .github/prompts/release-packages.prompt.md Outdated

@martintmk martintmk left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Copilot speaking]

Reviewed the AI-agentic release skill and its two helpers against the existing tested release library, and verified the claims below against the real workspace (53 packages) on the PR head. The helpers are clean thin shells and the new Pester tests are green (13/13), but the split of "what gets forwarded to deterministic code" is inverted, and one of the headline corrections guards a branch that cannot fire.

Verdict: changes requested. (REQUEST_CHANGES is not accepted on your own PR, so this is posted as a comment.)

Design note, not attached to a single line

The determinism argument leans on the consensus gate, but a 3-sample vote is evidence of agreement, not of determinism: the same model can diverge run-to-run, so unanimity on four cases does not establish the property the PR claims. It also cannot run in CI and multiplies the cost and latency of every release. That is acceptable for the genuinely subjective part — the Step 5 diff judgments that replace the operator's TTY call. It is the wrong tool for the cascade closure and the version arithmetic, which are already deterministic code in scripts/lib/. Forwarding those (finding 1) would shrink the gate to the part that actually needs it and let the version-bump table, the elevation ranks and the pin rules stop being restated in prose where they can drift from Get-NextVersion / Get-StrongerChangeType.

Verified while reviewing: scripts/release-facts.ps1 runs clean on the real workspace (exit 0, ~34s, 53 packages); scripts/release-changelog.ps1 produces the expected 🔧 Maintenance / ⚠️ Breaking sections; Invoke-Pester on both new files passes 13/13. I also checked whether the breaking-cascade test was coverage-shaped and it is not — a non-breaking cascade renders no literal Breaking, so the assertion does discriminate.

Comment thread .github/prompts/release-packages.prompt.md Outdated
Comment thread .github/skills/release-packages/scripts/release-facts.ps1 Outdated
Comment thread .github/prompts/release-packages.prompt.md Outdated
Comment thread .github/prompts/release-packages.prompt.md Outdated
Comment thread .github/prompts/release-packages.prompt.md Outdated
Comment thread scripts/release-facts.ps1 Outdated
Comment thread docs/releasing.md Outdated
martintmk and others added 2 commits August 13, 2026 10:25
Per review, go back to a simpler prompt-based release process and delete the
script pieces whose logic now lives in the release prompt. The prompt remains the
single source of truth for planning (which packages, which version bumps); only
small, mechanical, tested helpers remain.

Deleted (logic now in .github/prompts/release-packages.prompt.md):
- scripts/release-packages.ps1 (interactive driver)
- scripts/lib/release-flow.ps1 (plan resolution, cascade, elevation, apply,
  interactive UI) -- its only survivors, the changelog generator, were extracted
- the plan-engine tests: scenarios/*, Scenarios.Tests, PromptFlow.Tests,
  Parse/Resolve token tests, Topology-Presets, Releasing-Integration,
  DirectDependencyChangelogReasons.Tests, and the scenario harness Invoke-Scenario

Kept / simplified:
- scripts/lib/changelog.ps1 -- extracted Write-Changelog + its 3 helpers and 4
  module-scope maps so scripts/release-changelog.ps1 keeps working
- scripts/lib/releasing.ps1 -- trimmed to the 21 helpers the survivors need
  (release-facts.ps1, release-changelog.ps1, ci/semver-report.ps1); 730 lines gone
- deletion set verified via an AST call-graph closure; all 145 remaining Pester
  tests pass and ci/semver-report.ps1 still runs green

Review fixes folded in:
- change-spec is optional (bare `name`); consensus divergence emits an ambiguity
  report proposing a concrete skill edit
- release-facts.ps1 emits an `everReleased` fact (release tags); Step 3 branches on
  it so a never-released crate ships its first release at its declared version
  instead of being misclassified breaking (hasBaseline can't distinguish this)
- apply step edits only the version value inside the [workspace.dependencies]
  inline table (never drops default-features)
- `just readme` note corrected (workspace-wide; package= does not narrow)
- BaseRef-validation comment softened; docs updated to the prompt-based process

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59264f5d-db1f-4379-9cc3-6663f2914b90
main advanced with #661 (show planned version in release decision prompt), which
modified release-flow.ps1, PromptFlow.Tests.ps1 and two releasing.ps1 functions
(New-ResolvedReleaseSetFromBaseRef, Get-UnreleasedModifiedDependencies). All of
those belong to the plan engine this branch retires, so the deletions win and the
trimmed releasing.ps1 is kept. Full Pester suite (146) green post-merge.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59264f5d-db1f-4379-9cc3-6663f2914b90
@martintmk

Copy link
Copy Markdown
Member Author

PR description is broken, try to update it

Comment thread .github/prompts/release-packages.prompt.md Outdated
@martintmk

Copy link
Copy Markdown
Member Author

PR description is wrong, update and refresh it according to latest changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Comment thread README.md Outdated
@martintmk

Copy link
Copy Markdown
Member Author

Note

🤖 Historical release audit generated by GitHub Copilot using the release skill in this PR, cargo-semver-checks 0.48.0, direct source review, and independent GPT/Claude model-family reviews.

Historical validation: last 10 distinct release commits

I replayed the last 10 distinct commits referenced by package release tags in a separate clone. A single release commit can own many tags, so auditing the last 10 tag names would only sample part of the latest batch.

For each commit the replay:

  1. checked out the released source in an isolated worktree;
  2. restored only package and workspace dependency versions to their parent values;
  3. rebuilt release facts as of the parent, including historically reachable tags;
  4. ran objective SemVer checks where a prior published baseline existed;
  5. resolved bare package tokens with the current deterministic resolver;
  6. reviewed source, public API, behavior, proc-macro contracts, and cascade edges independently.

The sample covers 10 release commits, 65 package tags, first releases, patch/nonbreaking/breaking changes, proc-macro bundles, stable 0.x cascades, and staged batch releases. The synthetic validation suite remains 476 passing tests, including the 304-case resolver oracle.

Release-by-release result

# Release Actual packages Result
1 7c185b44 — bytesbuf 0.8 batch 28 Most direct and exposed-cascade breaking bumps were correct. Found a real plurality under-bump and two omitted dependency-floor releases.
2 aae9c5eb — templated-uri macro boundary 3 Macro 0.3 elevations were intentional compatibility-boundary repair. Strict skill policy would also patch-release five dependents; existing Cargo ranges remained compatible.
3 b6509226 — rest-over-grpc 0.2 1 First recorded publication. The 0.2 choice was defensible because the diff contained breaking behavioral/generated-contract changes and there were no published dependents.
4 0de29c65 — internity bundle 3 Correct first releases at 0.1.0, including proc-macro review boundaries.
5 f9fcf506 — fetch-winhttp 1 Correct first release; design-only crate with no dependent cascade.
6 94229b87 — bytesbuf 0.7 batch 22 Released bumps were consistent. The skill would additionally require multitude 0.8.0 and rest_over_grpc 0.2.1; multitude is the material omitted exposed-breaking cascade.
7 78aa4470 — multitude streaming/stats 1 Under-versioned: shipped 0.7.1, minimum valid target 0.8.0.
8 fa49a453 — arena pinning/deserialization 2 Both direct breaking bumps were correct. Resolver-proposed consumers were staged and then released at the same predicted targets in later commits.
9 958d222a — plurality 0.1.1 1 Valid first release; skipping 0.1.0 cannot be reconstructed from a bare token but is not a SemVer defect. Deferred consumers were later released at the resolver's exact targets.
10 8ae7af65 — multitude Serde/Cow 3 Numeric versions were valid. multitude should have been classified nonbreaking rather than patch, but both classifications produce 0.6.2 under repository rules. Proc-macro first releases were correct.

Confirmed historical defects

1. plurality 0.2.1 should have been 0.3.0

The latest batch added manual UnwindSafe/RefUnwindSafe implementations for public Box, Rc, and Arc types. They replaced structural auto-trait derivation and narrowed allocator bounds.

A cross-review compiled the real source on both sides and reproduced:

plurality 0.2.0: Box<u64, InteriorMutableAllocator>: UnwindSafe
plurality 0.2.1: Box<u64, InteriorMutableAllocator>: !UnwindSafe

The same regression exists for Rc and Arc. This is a compiler-visible public compatibility break, not only a classification-label issue. The correct transition was 0.2.0 -> 0.3.0. cargo-semver-checks did not detect the narrowed generic auto-trait bounds.

Evidence: boxed.rs, rc.rs, sync.rs.

2. multitude 0.7.1 should have been 0.8.0

Release 78aa4470 added feature-accessible public API:

  • Arena::try_deserialize_json_each;
  • Arena::try_deserialize_json_each_with_limits;
  • public, non-exhaustive JsonEachError.

These are backward-compatible public additions, so the source-review elevation rule requires nonbreaking. For a 0.7.x crate, the exact target is 0.8.0, not the shipped 0.7.1. Two independent reviews reached the same result.

Evidence: json.rs, json_each_error.rs.

3. Latest batch omitted layered 0.3.7 and consequently rest_over_grpc 0.2.1

The latest batch released plurality 0.2.1 but left previously released direct dependent layered at 0.3.6. Under the skill's dependency-floor invariant, every released dependency gives a publishable, previously released direct dependent a patch floor even when the dependency transition is compatible and private.

The missing layered 0.3.7 then creates a patch floor for exposed dependent rest_over_grpc, requiring 0.2.1. This is one primary omission plus one derivative omission.

Evidence: layered/Cargo.toml, rest_over_grpc/Cargo.toml.

4. Bytesbuf 0.7 batch omitted the exposed multitude 0.8.0 cascade

Release 94229b87 moved bytesbuf from 0.6.0 to 0.7.0. multitude publicly exposes bytesbuf types, so the breaking dependency transition required a breaking dependent release. The deterministic resolver produces multitude 0.8.0. The historical batch omitted it.

Existing crates.io artifacts were not retroactively broken because multitude 0.7.1 retained its old published requirement; this is a release-set consistency defect under the repository's new cascade rules rather than proof that an already downloaded artifact stopped compiling.

The same strict policy also produces rest_over_grpc 0.2.1 after layered 0.3.6, but that omission is only a patch-floor alignment issue.

Policy deviations without demonstrated consumer breakage

  • Release 2's templated_uri 0.3.6 would, under the new dependency-floor rule, also release http_extensions, seatbelt_http, fetch_hyper, fetch, and fetch_azure at patch targets. Existing caret requirements already admitted 0.3.6, so this is historical process drift rather than a breaking-version defect.
  • Releases 8 and 9 staged broad dependent batches for several days. The later bytesbuf 0.7 batch published those crates at the resolver's exact predicted targets, so no lasting omission remained.
  • Release 10's public Cow and Serde additions were labeled patch instead of nonbreaking, but repository arithmetic maps both to 0.6.2. This is signaling/changelog fidelity only.
  • The macro compatibility repair in release 2 intentionally elevated the macro crates to 0.3.0 despite patch-only source changes; that elevation prevented incompatible resolver combinations and was justified.

Skill changes resulting from the audit

  • Source review now explicitly treats narrowed generic or auto-trait implementation bounds as breaking.
  • The documented cargo-semver-checks blind-spot guidance now calls out manual auto-trait implementations replacing structural derivation.
  • Bare package tokens remain the recommended interface; exact version pins are reserved for deliberate first-version or exceptional historical choices.

Conclusion

The replay validates the resolver's arithmetic, fixed-point ordering, first-release behavior, proc-macro handling, and exposure-aware cascades across real history. It also demonstrates why the skill cannot delegate classification entirely to cargo-semver-checks: the two strongest defects were a missed public addition and a missed generic auto-trait-bound narrowing.

The audited skill would have prevented both numeric under-bumps and would have made the omitted dependency-floor releases explicit before application.

Expand deterministic resolver and atomic apply coverage, document source-review blind spots, and use bare crate tokens as the default skill interface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Comment thread .github/skills/release-packages/SKILL.md
Treat exposed major dependency upgrades as breaking and distinguish packaged documentation repairs from generated README churn.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Resolve the release-tooling overlap by porting exposed-dependency cascade fixes into the unified facts and resolver architecture, including renamed crate roots and indirect public re-exports.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Detect cargo-semver-checks without executing a missing cargo subcommand and make live cascade tests independent of tags omitted by shallow CI checkouts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
@martintmk martintmk self-assigned this Aug 14, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR repackages the repository’s release workflow into a unified .github/skills/release-packages/ skill, backed by deterministic PowerShell helpers for fact gathering, plan resolution, atomic application, and changelog generation, and updates tests/docs accordingly.

Changes:

  • Adds the release-packages skill bundle (SKILL.md + references) and deterministic scripts: release-facts.ps1, resolve-plan.ps1, apply-plan.ps1, and release-changelog.ps1.
  • Extracts deterministic changelog generation into a reusable library (scripts/lib/changelog.ps1) and retargets tests to the new structure.
  • Reworks Pester coverage: adds new unit/integration suites for the skill and removes legacy scenario infrastructure and the old top-level release driver script.

Reviewed changes

Copilot reviewed 61 out of 62 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
scripts/tests/Pester/unit/releasing/WriteChangelog.Tests.ps1 Updates unit tests to source the new changelog library instead of the retired release-flow helper.
scripts/tests/Pester/unit/releasing/ReleaseFacts.Tests.ps1 Adds hermetic unit tests for release-facts.ps1 fact emission using synthetic workspaces.
scripts/tests/Pester/unit/releasing/ReleaseChangelog.Tests.ps1 Adds tests for the thin release-changelog.ps1 wrapper over Write-Changelog.
scripts/tests/Pester/unit/releasing/PureFunctions.Tests.ps1 Removes tests for functions migrated/retired with the old flow; updates sourcing to changelog.ps1 where needed.
scripts/tests/Pester/unit/releasing/GitFs.Tests.ps1 Removes legacy flow tests tied to retired functions and scenario machinery.
scripts/tests/Pester/unit/releasing/DirectDependencyChangelogReasons.Tests.ps1 Removes legacy unit tests for direct-dependency changelog reasons (covered elsewhere in the new structure).
scripts/tests/Pester/unit/release-packages/ParseReleaseTokens.Tests.ps1 Removes legacy token parsing tests from the retired release-flow implementation.
scripts/tests/Pester/scenarios/Scenarios.Tests.ps1 Removes the legacy end-to-end scenario test runner suite.
scripts/tests/Pester/scenarios/S28-proc-macro-no-material-then-cascade.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S27-proc-macro-recursive-review.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S26-proc-macro-cascade-review.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S25-proc-macro-user-review.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S24-pin-with-cascade-conflict-force.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S23-all-force-release-unchanged.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S22-all-no-modifications-ignore-all.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S21-changed-no-modifications.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S20-changed-accept-with-cascade.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S19-changed-ignore-everything.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S18-explicit-pin-rejected-when-not-greater.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S17-explicit-version-pin.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S16-stable-cascade-elevation.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S15-auto-upgrade-of-user-source.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S14-pin-with-cascade-conflict.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S13-pin-with-cascade-satisfied.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S12-multi-package-single-invocation.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S11-stable-version-minor-distinct.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S10-stable-version-patch-distinct.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S09-ignore-then-cascade.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S08-invalid-then-valid-input.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S07-view-diff-then-decide.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S06-invariant-b-elevation-review.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S05-diamond4-aggregation.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S03-decline-all.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S02-accept-and-decline.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S01-clean-dependency-no-prompts.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/scenarios/S00-smoke-fresh-release.scenario.psd1 Removes legacy scenario fixture.
scripts/tests/Pester/integration/Topology-Presets.Tests.ps1 Removes legacy integration coverage for topology presets (replaced by new suites).
scripts/tests/Pester/integration/ReleaseSkillSemver.Tests.ps1 Adds integration coverage validating cargo semver-checks classifications on synthetic workspaces (skippable if tool absent).
scripts/tests/Pester/integration/ReleaseSkillApply.Tests.ps1 Adds integration coverage for atomic apply/rollback behavior in apply-plan.ps1.
scripts/tests/Pester/integration/ExposureCascade-RealWorkspace.Tests.ps1 Refactors live-workspace cascade tests to use release-facts.ps1 + resolve-plan.ps1 inputs/outputs.
scripts/tests/Pester/integration/DependencyRename.Tests.ps1 Refactors rename/lib-target exposure tests to validate both Get-WorkspacePackages extraction and release-facts.ps1 exposure edges.
scripts/tests/Pester/_common/TestHelpers.ps1 Updates helper documentation to reflect the new library sourcing conventions.
scripts/tests/Pester/_common/Invoke-Scenario.ps1 Removes the legacy scenario runner implementation.
scripts/release-packages.ps1 Removes the retired interactive release driver entrypoint (superseded by the skill).
scripts/lib/changelog.ps1 Introduces extracted, deterministic changelog generation logic (Write-Changelog, conventional commit grouping, unreleased folding, cascade bullets).
README.md Updates release/changelog documentation to reference the new release-packages skill instead of the removed script.
AGENTS.md Updates agent guidance to reference the release-packages skill and new workflow location.
.github/skills/release-packages/SKILL.md Adds the skill definition and workflow orchestration guidance for deterministic release planning/apply + consensus.
.github/skills/release-packages/scripts/resolve-plan.ps1 Adds the deterministic resolver responsible for token parsing, version arithmetic, cascades, pins, ordering, and warnings.
.github/skills/release-packages/scripts/release-facts.ps1 Adds deterministic fact gathering (graph, baselines, modified set, exposure edges) as JSON.
.github/skills/release-packages/scripts/release-changelog.ps1 Adds deterministic wrapper for generating a single package changelog via Write-Changelog.
.github/skills/release-packages/scripts/apply-plan.ps1 Adds atomic plan application with manifest edits, changelog/README generation, validation, and rollback.
.github/skills/release-packages/references/version-rules.md Adds canonical version arithmetic/rules reference used by the skill.
.github/skills/release-packages/references/scenarios.md Documents where executable scenario coverage lives after scenario fixture removal.
.github/skills/release-packages/references/planning.md Adds the model-owned planning reference (facts, classification, resolver contract, consensus gate, apply rules).
Suppressed comments (1)

.github/skills/release-packages/SKILL.md:77

  • The apply step references scripts/apply-plan.ps1, but the apply script is introduced under .github/skills/release-packages/scripts/apply-plan.ps1. Update this path so users can run the documented command successfully.
6. **Apply atomically**
   - Run `scripts/apply-plan.ps1 -PlanPath <plan.json>`.
   - Never reproduce its writes or rollback behavior by hand.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/skills/release-packages/SKILL.md Outdated
Comment thread .github/skills/release-packages/references/planning.md Outdated
martintmk and others added 9 commits August 14, 2026 10:46
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Require complete evidenced selection decisions, validate exact token alignment, and define baseline compile evidence for proc-macro compatibility.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Freeze exact changed paths, reject test-only first releases, derive manual-review flags, and require implemented API evidence for compatibility classification.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Emit schema-v4 manifest change facts, enforce canonical dependency-scope decisions, and cover Cargo table edge cases that caused cross-model seed drift.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
…stic

Advance the release tooling from schema v4 to v5 with four mechanical
guards so independent reasoning models resolve to one release plan.

- Macro compile-fixture evidence: proc-macro contracts must carry typed
  baseline/current compile results for every changed tests/ui fixture in
  their review scope; the verdict floor is derived (pass->fail breaking,
  fail->pass nonbreaking) and a declared verdict below it blocks.
- Behavior-fix regression evidence: the behavior-fix selection reason
  requires a same-probe baseline-fail/current-pass measurement; a
  pass/pass adaptation must be internal-only.
- External exposure lane: an effective non-dev external dependency
  requirement change (including workspace-inherited) that is breaking and
  named in a crate's cargo_check_external_types allowlist forces a
  breaking floor; proc-macro-only and private dependencies do not.
- Own-diff classification floor: a previously released library whose own
  packaged Rust source did not change beyond doc comments, with no
  exposed breaking external dependency change, cannot be classified
  breaking or nonbreaking on its own; cascades still apply.

The floors are enforced on both explicit tokens and cascade dependents.
Full release Pester suite: 671 passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Close the two lower-order nondeterminism sources a ten-run isolated
simulation batch surfaced. Version and release-tuple output were already
identical across all ten runs; these normalize the remaining metadata:

- The resolver derives the canonical decline reason for a metadata/
  generated diff instead of trusting the model's wording. generated-
  artifact-only is valid only when the sole changed files are this crate's
  generated README.md or CHANGELOG.md; when a Cargo.toml also changed the
  reason is release-metadata-only, and a pure generated diff cannot be
  release-metadata-only. The two reasons are now mutually exclusive.
- The emitted macro `reviewed` scope is the resolver-computed scope (self
  plus modified implementation-closure members and modified runtime
  partners), not the model-supplied list. A contract may still review more,
  but unmodified extras such as an unmodified runtime facade no longer vary
  the output.

Re-resolving all ten batch requests now yields identical selection and
macro output for every correctly-labelled request and rejects the
mislabelled ones. Full release Pester suite: 676 passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
A ten-run isolated batch left one residual: a package whose own diff is a
doc-comment repair (ohno) was declined internal-only by one model and
accepted authored-doc-fix by nine, differing only in release source. Make
the outcome mechanical.

- release-facts emits a positive `docCommentChanged` fact: an added or
  removed rustdoc-visible doc comment (`///`, exactly three slashes, or
  `//!`) in packaged library source. Candidate source is restricted to
  `src/**` plus `build.rs`, so non-shipped `scripts/`/`xtask/` sources and
  test/bench/example files never drive a classification. `build.rs` raises
  implementation but never the doc signal.
- The resolver requires `authored-doc-fix` only when `docCommentChanged` is
  true and `rustImplementationChanged` is false, so a plain `//` comment or
  whitespace edit stays eligible for `internal-only`. A published
  normal/build/features manifest change takes precedence, so a package with
  both a doc tweak and a dependency change uses `runtime-manifest-change`.

Re-resolving all ten batch requests now yields one identical normalized
plan (version, release tuples with source, selection, macro) for the nine
correctly-labelled requests and rejects the mislabelled one. Full release
Pester suite: 680 passing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Behavior-preserving simplification of the release tooling. Facts output is
byte-identical and the resolved plan is unchanged (680 Pester tests pass).

- resolve-plan.ps1: extract the two identical plan-output emitters
  (Format-SelectionDecisionOutput, Format-MacroContractOutput) used by both
  the blocked and resolved paths; extract the repeated
  clean-string-list and normalized-modified-files pipelines into helpers;
  drop a redundant duplicate-token check, a redundant string-verdict throw,
  a redundant prefix assignment, and factor a doubled boolean.
- release-facts.ps1: resolve the per-package modifiedFiles lookup once and
  reuse it, keeping the established JSON null-for-empty shape.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f77d7bd4-9f62-45e2-8805-5ead9000f286
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agency-rocket Touched by a rocket skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants