Skip to content

fix(scripts): build semver-checks baselines under a short target directory - #702

Merged
Kateřina Churanová (kate-shine) merged 8 commits into
microsoft:mainfrom
kate-shine:u/kchuranov/semver-checks-compile-maxpath
Aug 28, 2026
Merged

fix(scripts): build semver-checks baselines under a short target directory#702
Kateřina Churanová (kate-shine) merged 8 commits into
microsoft:mainfrom
kate-shine:u/kchuranov/semver-checks-compile-maxpath

Conversation

@kate-shine

Copy link
Copy Markdown
Contributor

Follow-up to #642 (AB#7790786 is still open).

Pointing the baseline build at rust-lld cleared LNK1104, but it only moved the MAX_PATH ceiling one step earlier in the toolchain. The work item filed four days after that merge is a different failure: six C1083 errors from cl.exe, with no real LNK1104 in the log.

Why the linker fix doesn't cover this

-sys crates compile C through the cc crate, and MSVC cl.exe resolves its -Fo argument against MAX_PATH. Unlike the linker there is no long-path-aware drop-in guaranteed to be present — clang-cl ships with neither a default Rust nor a default Visual Studio install.

Measured on this machine with a 296-character output path:

Compiler Result
cl.exe 14.44.35207 fatal error C1083: Cannot open compiler generated file: '': Invalid argument
clang-cl (LLVM) wrote the object

cc's own 8.3 short-name fallback does not rescue it either; the field log shows it tried OXIDIZ~1.WOR\...\SEMVER~1\GIT-7C~1\ and still failed.

The lever that remains

cargo semver-checks nests baseline builds under the workspace target directory and exposes no flag to move them, but it derives that location from cargo metadata — so CARGO_TARGET_DIR does reach it.

Invoke-SemverChecksCli now scopes CARGO_TARGET_DIR to <volume>\oxi-sc\<digest of repository root> around the cargo call, alongside the existing linker override. The path budget:

C:\Source\oxidizer.worktrees\release-threadaware-fallout   56  (repository root)
  \target\semver-checks\git-<40 sha>\local-fetch-...\...   216 (fixed by the tools)
                                                          = 272  fails
C:\oxi-sc\1a2b3c4d                                         18
                                                          = 234  fits

The 216-character suffix is set by cargo-semver-checks and aws-lc-sys and cannot be shortened from here, which is why the repository path itself cannot be relied on to leave room.

The volume is the repository's own, so the build stays on the filesystem the developer chose, and the digest keeps sibling clones apart. The path is deterministic rather than unique so consecutive runs reuse the baseline rustdoc they just built; concurrent runs are safe because cargo locks that directory exactly as it does target/.

As with the linker, an explicit CARGO_TARGET_DIR is respected rather than overridden, and a directory that cannot be created degrades to a warning rather than aborting a release. Non-Windows platforms are unchanged.

The error hint that previously told the caller to shorten the repository path now names C1083 as well and points at the remaining lever, since shortening the path is no longer the caller's job.

Scope

Only the release library is changed, which covers release-packages.ps1 (where the failure was reported) and scripts/ci/semver-report.ps1. anvil-semver-check calls cargo semver-checks directly and is cargo-anvil-managed, so it is untouched here; its Windows pr-fast leg has the same latent exposure and needs a change in ox-tools. It has not failed because GitHub runner workspaces (D:\a\oxidizer\oxidizer, 22 chars) leave about 22 characters of headroom.

Testing

New integration test compiles a C file with the real MSVC toolchain at a >260-character output path, asserts C1083 specifically so the premise cannot be satisfied by an unrelated failure, then repeats the compile under the relocated root and asserts it succeeds. It skips cleanly when no MSVC toolchain is present.

Twelve new unit tests cover the path shape (volume, length, determinism, clone separation, equivalent spellings, UNC fallback, non-Windows) and the environment-variable lifecycle (applied, created, restored, restored on throw, explicit value respected, unwritable directory warns).

Full Pester suite: 544 passed, 0 failed (unit 427, integration 89, scenarios 28).

…ctory

Pointing the baseline build at rust-lld cleared LNK1104, but it only
moved the MAX_PATH ceiling one step earlier in the toolchain. The C
compilers that -sys crates drive through the cc crate are still bound by
it: MSVC cl.exe resolves its -Fo argument against MAX_PATH and fails
with C1083, which is how aws-lc-sys now aborts a `fetch` baseline build
from a deep worktree. Unlike the linker there is no long-path-aware
drop-in to switch to, since clang-cl is not part of a default Rust or
Visual Studio install, so the remaining lever is the path itself.

cargo-semver-checks nests baseline builds under the workspace target
directory and exposes no flag to move them, but it derives that location
from cargo metadata, so CARGO_TARGET_DIR does reach it.
Invoke-SemverChecksCli now scopes CARGO_TARGET_DIR to
<volume>\oxi-sc\<digest of repository root> around the cargo call and
restores or removes it afterwards, alongside the existing linker
override. That is a fixed 18 characters in place of a repository path
that is unbounded; the observed failure nested 216 characters of
cargo-semver-checks and aws-lc-sys output beneath the target directory,
leaving no room for a 56-character root.

The volume is the repository's own, so the build stays on the filesystem
the developer chose, and the digest keeps sibling clones apart. The path
is deterministic rather than unique so consecutive runs reuse the
baseline rustdoc they just built; concurrent runs are safe because cargo
locks that directory exactly as it does target/.

As with the linker, an explicit CARGO_TARGET_DIR is respected rather
than overridden, and a directory that cannot be created degrades to a
warning rather than aborting a release. Non-Windows platforms are
unchanged.

The error hint that previously told the caller to shorten the repository
path now names C1083 as well and points at the remaining lever, since
shortening the path is no longer the caller's job.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI lite review requested due to automatic review settings August 27, 2026 10:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses Windows MAX_PATH failures in cargo semver-checks baseline builds by relocating the semver-checks target directory to a short, per-clone path via CARGO_TARGET_DIR, complementing the existing rust-lld linker override. It also adds unit + integration coverage to ensure the environment-variable lifecycle and the cl.exe failure mode (C1083) are exercised and documented.

Changes:

  • Add Get-SemverChecksTargetDirPath and teach Invoke-SemverChecksCli to scope CARGO_TARGET_DIR to a short, deterministic per-repo directory on Windows (fail-open with warning if it can’t be created).
  • Update failure hinting to mention C1083 and point at CARGO_TARGET_DIR as the remaining lever.
  • Add extensive Pester unit tests for target-dir selection/lifecycle and an integration test that reproduces cl.exe’s long-path failure and validates the relocation fix.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
scripts/lib/releasing.ps1 Implements short per-clone target dir selection and applies/restores CARGO_TARGET_DIR around cargo semver-checks; updates Windows failure hint text.
scripts/tests/Pester/unit/releasing/SemverChecksLinker.Tests.ps1 Adds unit tests for target-dir path shape and env-var lifecycle (create/apply/restore/fail-open warning).
scripts/tests/Pester/unit/releasing/PureFunctions.Tests.ps1 Updates assertions for the Windows path-length hint to reference CARGO_TARGET_DIR and include C1083.
scripts/tests/Pester/integration/SemverChecksLinker.Tests.ps1 Adds an integration repro: deep-path compile fails with C1083, then succeeds under the relocated root.
docs/releasing.md Documents the Windows short target directory behavior, rationale, and cleanup implications.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/lib/releasing.ps1 Outdated
@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.0%. Comparing base (edae40d) to head (c684eb6).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #702   +/-   ##
=======================================
  Coverage   100.0%   100.0%           
=======================================
  Files         560      560           
  Lines       60859    60859           
=======================================
  Hits        60859    60859           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

… applied

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 27, 2026 11:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

scripts/tests/Pester/integration/SemverChecksLinker.Tests.ps1:124

  • New-Item is creating an extended-length path using the \\?\ prefix, but it uses -Path rather than -LiteralPath. Because ? is a wildcard character in PowerShell paths, using -Path can trigger wildcard expansion/interpretation and break the intended device-path behavior. Use -LiteralPath for \\?\... paths.
            $deep = $root
            while ($deep.Length -lt 240) {
                $deep = Join-Path $deep 'nested-directory-segment'
            }
            $null = New-Item -ItemType Directory -Path "\\?\$deep" -Force

Comment thread scripts/lib/releasing.ps1

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a nasty one but I guess we have no good choice. Feels dirty.

Comment thread docs/releasing.md
Comment thread docs/releasing.md
Comment thread docs/releasing.md
@kate-shine

Copy link
Copy Markdown
Contributor Author

It's a nasty one but I guess we have no good choice. Feels dirty.

Yea, I hate it.

…nconditional

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 27, 2026 12:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

docs/releasing.md:339

  • The docs state that the relocated baseline build stays on "the repository's own" volume, but Get-SemverChecksTargetDirPath explicitly falls back to $env:SystemDrive for UNC roots (since there’s no drive letter to anchor to). Calling out that exception here would avoid surprising anyone running releases from a UNC path.
call — a fixed 18 characters in place of a repository path that is
unbounded. The volume is the repository's own, so the build stays on the
filesystem you chose, and the digest keeps sibling clones apart. The
path is deterministic rather than unique so that consecutive runs reuse

Comment thread scripts/tests/Pester/unit/releasing/SemverChecksLinker.Tests.ps1 Outdated
Comment thread scripts/tests/Pester/integration/SemverChecksLinker.Tests.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 27, 2026 13:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Comment thread scripts/lib/releasing.ps1 Outdated
Comment thread scripts/lib/releasing.ps1 Outdated
…drive-root paths

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 27, 2026 13:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

Previously missed (1) — in code that hasn't changed since the last review.

scripts/lib/releasing.ps1:1121

  • The per-clone directory token only uses the first 4 bytes of SHA-256 (8 hex chars). That leaves a non-trivial chance of collision across many clones, which would cause two unrelated repositories on the same volume to share one CARGO_TARGET_DIR (locking contention and potential artifact cross-talk), contradicting the intent that the digest "keeps sibling clones apart".
    $sha = [System.Security.Cryptography.SHA256]::Create()
    try {
        $digest = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($full.ToLowerInvariant()))
    } finally {
        $sha.Dispose()
    }
    $token = [System.BitConverter]::ToString($digest[0..3]).Replace('-', '').ToLowerInvariant()

    return (Join-Path $volume (Join-Path $script:SemverChecksTargetDirName $token))

Comment thread scripts/lib/releasing.ps1
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 27, 2026 13:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings August 27, 2026 14:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Suppressed comments (1)

scripts/lib/releasing.ps1:1302

  • This hint says the scripts relocate "only when the repository path and the package name together project past" the MAX_PATH limit, but Get-SemverChecksTargetDirPath also relocates when the repo root can’t be projected (UNC roots and paths that can’t be normalised). Rewording avoids an objectively incorrect description of when relocation can apply.
        " If the output contains LNK1104, C1083 or a path-length error, a MAX_PATH-bound tool was reached. Build artifacts can nest over 200 characters below the target directory, and these scripts relocate the build to a short path only when the repository path and the package name together project past that limit. Set CARGO_TARGET_DIR to a short path (for example ${env:SystemDrive}\$script:SemverChecksTargetDirName) to force the relocation, or move the repository closer to the volume root."

Comment thread scripts/lib/releasing.ps1 Outdated
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 27, 2026 16:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comment thread scripts/lib/releasing.ps1
Comment thread scripts/lib/releasing.ps1
Comment thread docs/releasing.md

@martin-kolinek martin-kolinek left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖: Approved.

Comment thread scripts/lib/releasing.ps1
@kate-shine
Kateřina Churanová (kate-shine) merged commit 9cee105 into microsoft:main Aug 28, 2026
51 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants