fix(scripts): build semver-checks baselines under a short target directory - #702
Conversation
…ctory Pointing the baseline build at rust-lld cleared LNK1104, but it only moved the MAX_PATH ceiling one step earlier in the toolchain. The C compilers that -sys crates drive through the cc crate are still bound by it: MSVC cl.exe resolves its -Fo argument against MAX_PATH and fails with C1083, which is how aws-lc-sys now aborts a `fetch` baseline build from a deep worktree. Unlike the linker there is no long-path-aware drop-in to switch to, since clang-cl is not part of a default Rust or Visual Studio install, so the remaining lever is the path itself. cargo-semver-checks nests baseline builds under the workspace target directory and exposes no flag to move them, but it derives that location from cargo metadata, so CARGO_TARGET_DIR does reach it. Invoke-SemverChecksCli now scopes CARGO_TARGET_DIR to <volume>\oxi-sc\<digest of repository root> around the cargo call and restores or removes it afterwards, alongside the existing linker override. That is a fixed 18 characters in place of a repository path that is unbounded; the observed failure nested 216 characters of cargo-semver-checks and aws-lc-sys output beneath the target directory, leaving no room for a 56-character root. The volume is the repository's own, so the build stays on the filesystem the developer chose, and the digest keeps sibling clones apart. The path is deterministic rather than unique so consecutive runs reuse the baseline rustdoc they just built; concurrent runs are safe because cargo locks that directory exactly as it does target/. As with the linker, an explicit CARGO_TARGET_DIR is respected rather than overridden, and a directory that cannot be created degrades to a warning rather than aborting a release. Non-Windows platforms are unchanged. The error hint that previously told the caller to shorten the repository path now names C1083 as well and points at the remaining lever, since shortening the path is no longer the caller's job. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR addresses Windows MAX_PATH failures in cargo semver-checks baseline builds by relocating the semver-checks target directory to a short, per-clone path via CARGO_TARGET_DIR, complementing the existing rust-lld linker override. It also adds unit + integration coverage to ensure the environment-variable lifecycle and the cl.exe failure mode (C1083) are exercised and documented.
Changes:
- Add
Get-SemverChecksTargetDirPathand teachInvoke-SemverChecksClito scopeCARGO_TARGET_DIRto a short, deterministic per-repo directory on Windows (fail-open with warning if it can’t be created). - Update failure hinting to mention
C1083and point atCARGO_TARGET_DIRas the remaining lever. - Add extensive Pester unit tests for target-dir selection/lifecycle and an integration test that reproduces cl.exe’s long-path failure and validates the relocation fix.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| scripts/lib/releasing.ps1 | Implements short per-clone target dir selection and applies/restores CARGO_TARGET_DIR around cargo semver-checks; updates Windows failure hint text. |
| scripts/tests/Pester/unit/releasing/SemverChecksLinker.Tests.ps1 | Adds unit tests for target-dir path shape and env-var lifecycle (create/apply/restore/fail-open warning). |
| scripts/tests/Pester/unit/releasing/PureFunctions.Tests.ps1 | Updates assertions for the Windows path-length hint to reference CARGO_TARGET_DIR and include C1083. |
| scripts/tests/Pester/integration/SemverChecksLinker.Tests.ps1 | Adds an integration repro: deep-path compile fails with C1083, then succeeds under the relocated root. |
| docs/releasing.md | Documents the Windows short target directory behavior, rationale, and cleanup implications. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #702 +/- ##
=======================================
Coverage 100.0% 100.0%
=======================================
Files 560 560
Lines 60859 60859
=======================================
Hits 60859 60859 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
… applied Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
scripts/tests/Pester/integration/SemverChecksLinker.Tests.ps1:124
New-Itemis creating an extended-length path using the\\?\prefix, but it uses-Pathrather than-LiteralPath. Because?is a wildcard character in PowerShell paths, using-Pathcan trigger wildcard expansion/interpretation and break the intended device-path behavior. Use-LiteralPathfor\\?\...paths.
$deep = $root
while ($deep.Length -lt 240) {
$deep = Join-Path $deep 'nested-directory-segment'
}
$null = New-Item -ItemType Directory -Path "\\?\$deep" -Force
Sander Saares (sandersaares)
left a comment
There was a problem hiding this comment.
It's a nasty one but I guess we have no good choice. Feels dirty.
Yea, I hate it. |
…nconditional Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
docs/releasing.md:339
- The docs state that the relocated baseline build stays on "the repository's own" volume, but
Get-SemverChecksTargetDirPathexplicitly falls back to$env:SystemDrivefor UNC roots (since there’s no drive letter to anchor to). Calling out that exception here would avoid surprising anyone running releases from a UNC path.
call — a fixed 18 characters in place of a repository path that is
unbounded. The volume is the repository's own, so the build stays on the
filesystem you chose, and the digest keeps sibling clones apart. The
path is deterministic rather than unique so that consecutive runs reuse
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…drive-root paths Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
Previously missed (1) — in code that hasn't changed since the last review.
scripts/lib/releasing.ps1:1121
- The per-clone directory token only uses the first 4 bytes of SHA-256 (8 hex chars). That leaves a non-trivial chance of collision across many clones, which would cause two unrelated repositories on the same volume to share one
CARGO_TARGET_DIR(locking contention and potential artifact cross-talk), contradicting the intent that the digest "keeps sibling clones apart".
$sha = [System.Security.Cryptography.SHA256]::Create()
try {
$digest = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($full.ToLowerInvariant()))
} finally {
$sha.Dispose()
}
$token = [System.BitConverter]::ToString($digest[0..3]).Replace('-', '').ToLowerInvariant()
return (Join-Path $volume (Join-Path $script:SemverChecksTargetDirName $token))
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.
Suppressed comments (1)
scripts/lib/releasing.ps1:1302
- This hint says the scripts relocate "only when the repository path and the package name together project past" the MAX_PATH limit, but
Get-SemverChecksTargetDirPathalso relocates when the repo root can’t be projected (UNC roots and paths that can’t be normalised). Rewording avoids an objectively incorrect description of when relocation can apply.
" If the output contains LNK1104, C1083 or a path-length error, a MAX_PATH-bound tool was reached. Build artifacts can nest over 200 characters below the target directory, and these scripts relocate the build to a short path only when the repository path and the package name together project past that limit. Set CARGO_TARGET_DIR to a short path (for example ${env:SystemDrive}\$script:SemverChecksTargetDirName) to force the relocation, or move the repository closer to the volume root."
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
9cee105
into
microsoft:main
Follow-up to #642 (AB#7790786 is still open).
Pointing the baseline build at
rust-lldclearedLNK1104, but it only moved theMAX_PATHceiling one step earlier in the toolchain. The work item filed four days after that merge is a different failure: sixC1083errors fromcl.exe, with no realLNK1104in the log.Why the linker fix doesn't cover this
-syscrates compile C through thecccrate, and MSVCcl.exeresolves its-Foargument againstMAX_PATH. Unlike the linker there is no long-path-aware drop-in guaranteed to be present —clang-clships with neither a default Rust nor a default Visual Studio install.Measured on this machine with a 296-character output path:
cl.exe14.44.35207fatal error C1083: Cannot open compiler generated file: '': Invalid argumentclang-cl(LLVM)cc's own 8.3 short-name fallback does not rescue it either; the field log shows it triedOXIDIZ~1.WOR\...\SEMVER~1\GIT-7C~1\and still failed.The lever that remains
cargo semver-checksnests baseline builds under the workspace target directory and exposes no flag to move them, but it derives that location from cargo metadata — soCARGO_TARGET_DIRdoes reach it.Invoke-SemverChecksClinow scopesCARGO_TARGET_DIRto<volume>\oxi-sc\<digest of repository root>around the cargo call, alongside the existing linker override. The path budget:The 216-character suffix is set by
cargo-semver-checksandaws-lc-sysand cannot be shortened from here, which is why the repository path itself cannot be relied on to leave room.The volume is the repository's own, so the build stays on the filesystem the developer chose, and the digest keeps sibling clones apart. The path is deterministic rather than unique so consecutive runs reuse the baseline rustdoc they just built; concurrent runs are safe because cargo locks that directory exactly as it does
target/.As with the linker, an explicit
CARGO_TARGET_DIRis respected rather than overridden, and a directory that cannot be created degrades to a warning rather than aborting a release. Non-Windows platforms are unchanged.The error hint that previously told the caller to shorten the repository path now names
C1083as well and points at the remaining lever, since shortening the path is no longer the caller's job.Scope
Only the release library is changed, which covers
release-packages.ps1(where the failure was reported) andscripts/ci/semver-report.ps1.anvil-semver-checkcallscargo semver-checksdirectly and iscargo-anvil-managed, so it is untouched here; its Windowspr-fastleg has the same latent exposure and needs a change inox-tools. It has not failed because GitHub runner workspaces (D:\a\oxidizer\oxidizer, 22 chars) leave about 22 characters of headroom.Testing
New integration test compiles a C file with the real MSVC toolchain at a >260-character output path, asserts
C1083specifically so the premise cannot be satisfied by an unrelated failure, then repeats the compile under the relocated root and asserts it succeeds. It skips cleanly when no MSVC toolchain is present.Twelve new unit tests cover the path shape (volume, length, determinism, clone separation, equivalent spellings, UNC fallback, non-Windows) and the environment-variable lifecycle (applied, created, restored, restored on throw, explicit value respected, unwritable directory warns).
Full Pester suite: 544 passed, 0 failed (unit 427, integration 89, scenarios 28).