fix(scripts): build semver-checks baselines under a short target directory

docs/releasing.md

@@ -318,6 +318,85 @@ be read from `rustc -vV` the default linker stands (with a warning).
 Other hosts are unaffected — `*-windows-gnu` and non-Windows targets
 already default to linkers that handle long paths.
 
+#### Windows: baseline builds run under a short target directory
+
+Redirecting the linker clears `LNK1104`, but it does not help the C
+compilers that `-sys` crates drive through the `cc` crate. MSVC `cl.exe`
+resolves its `-Fo` argument against `MAX_PATH` and fails with `C1083`
+("Cannot open compiler generated file"), and unlike the linker there is
+no long-path-aware drop-in guaranteed to be installed — `clang-cl` is
+not part of a default Rust or Visual Studio install. So the remaining
+lever is the length of the path itself.
+
+`cargo semver-checks` nests baseline builds under the workspace target
+directory and offers no flag to move them, but it derives that location
+from cargo metadata, so `CARGO_TARGET_DIR` does reach it. On Windows the
+release scripts therefore point it at
+`<volume>\oxi-sc\<digest-of-repository-root>` for the duration of the
+call — a fixed 18 characters in place of a repository path that is
+unbounded. The volume is the repository's own, so the build stays on the
+filesystem you chose, and the digest keeps sibling clones apart. The
+path is deterministic rather than unique so that consecutive runs reuse
+the baseline rustdoc they just built; concurrent runs are safe because
+cargo locks that directory exactly as it does `target/`.
+
+The observed worst case nests 216 characters of cargo-semver-checks and
+`aws-lc-sys` build output beneath the repository root, which is why the
+repository path itself cannot be relied on to leave enough room. The
+report in AB#7790786 breaks down as:
+
+| Segment | Chars |
+| --- | ---: |
+| `\target` | 7 |
+| `\semver-checks` | 14 |
+| `\git-<40-char commit sha>` | 45 |
+| `\local-fetch-0_15_0-x86_64_pc_windows_msvc-<16 hex>` | 59 |
+| `\target\debug\build` | 19 |
+| `\aws-lc-sys-<16 hex>` | 28 |
+| `\out` | 4 |
+| `\<16 hex>-jitterentropy-health.o` | 40 |
+| **Total below the repository root** | **216** |
+
+None of those segments are ours to shorten: the nesting is chosen by
+cargo-semver-checks, and the leaf by `aws-lc-sys` and the `cc` crate. In
+the field the repository root was 56 characters
+(`C:\Source\oxidizer.worktrees\release-threadaware-fallout`), for 272
+in total — past the 260-character limit. Replacing the root and its
+`\target` with an 18-character `C:\oxi-sc\<8 hex>` brings the same build
+to 227, leaving about 32 characters of headroom.
+
+That headroom is what the relocation buys, and it is worth noting how
+little of it there is by default. A GitHub-hosted runner checks out at
+`D:\a\oxidizer\oxidizer`, which is only 22 characters, so the same build
+lands at 238 and passes today with roughly 21 characters to spare — a
+longer crate name or a dependency version bump would be enough to
+consume it.
+
+As with the linker, an explicit `CARGO_TARGET_DIR` is respected rather
+than overridden. If the directory cannot be created the build proceeds
+in place with a warning, since a probe must never abort a release.
+
+A checkout that already has room is left where it is. Before relocating,
+the scripts project the build path from the repository root, the 216
+above, and the name of the package being checked — the nesting embeds
+that name once, so `http_client_api_with_templated_uri` reaches 29
+characters further than `fetch` did. If the projection fits within
+`MAX_PATH` with a margin to spare, the default target directory stands.
+A Dev Drive checkout such as `D:\oxidizer` therefore keeps its build
+output inside the repository, where `cargo clean` and `git clean` can
+still reach it, and only the checkouts that need the relocation get it.
+
+The margin exists because the 216 is one measurement rather than a
+bound. The package name is projected for explicitly, but the version
+string and the depth of a dependency's own build output are not ours to
+predict, and a dependency upgrade can lengthen either. If the projection
+is wrong the build still fails the way it did before, and the error
+names `CARGO_TARGET_DIR` as the lever to set by hand.
+
+Note that when the relocation does apply, those artifacts live outside
+the repository, so `cargo clean` and `git clean` do not remove them;
+delete `<volume>\oxi-sc` to reclaim the space.
+
 #### Proc-macro-only packages require manual SemVer review
 
 `cargo semver-checks` deliberately supports ordinary library targets,

scripts/lib/releasing.ps1

@@ -1001,8 +1001,132 @@ function Get-SemverChecksLinkerEnvName {
     return "CARGO_TARGET_${triple}_LINKER"
 }
 
-# Runs cargo semver-checks, linking with rust-lld where link.exe would overflow
-# MAX_PATH.
+# Directory name for relocated semver-checks builds, placed at a volume root.
+# Kept terse on purpose: every character here is one the MAX_PATH budget loses.
+$script:SemverChecksTargetDirName = 'oxi-sc'
+
+# Longest path a baseline build has been observed to nest below the repository
+# root: 216 characters, from the failure in AB#7790786. docs/releasing.md
+# breaks it down segment by segment.
+$script:SemverChecksNestingLength = 216
+
+# The package that measurement was taken for. The nesting embeds the package
+# name once, so checking a longer-named package shifts the projection by the
+# difference in name length.
+$script:SemverChecksNestingPackage = 'fetch'
+
+# Longest path Windows accepts without the \\?\ prefix, which the tools in
+# question do not use.
+$script:MaxPathLength = 259
+
+# Held back from the budget because the 216 is one measurement, not a bound.
+# The package name is projected for explicitly, but the version string and the
+# depth of a dependency's own build output are not ours to predict, and a
+# dependency upgrade can lengthen either. Sized to absorb that drift while
+# still leaving a Dev Drive or other short checkout on its default target
+# directory.
+$script:SemverChecksNestingSlack = 24
+
+# Returns a short, per-clone target directory for baseline builds on Windows,
+# or $null where the default target directory should stand.
+#
+# The linker override above keeps link.exe out of the way, but it cannot help
+# the C compilers that -sys crates drive through the cc crate. MSVC cl.exe
+# resolves its -Fo argument against MAX_PATH and fails with C1083 ("Cannot open
+# compiler generated file"), and unlike the linker there is no long-path-aware
+# drop-in to switch to: clang-cl is not guaranteed to be installed. The
+# remaining lever is the length of the path itself.
+#
+# cargo-semver-checks nests baseline builds under the workspace target
+# directory and offers no flag to move them, but it derives that location from
+# cargo metadata, so CARGO_TARGET_DIR does reach it. Rooting the build at the
+# repository's own volume keeps it on the filesystem the developer chose, and
+# the digest of the repository root keeps sibling clones from sharing one
+# directory. The result is a fixed 18 characters, in place of a repository path
+# that is unbounded.
+#
+# A checkout that already has room is left alone: relocating puts build output
+# outside the repository, where neither cargo clean nor git clean will reach
+# it, and that is not a cost worth imposing on a short path such as a Dev Drive
+# root. The projection below decides, and it is deliberately pessimistic --
+# guessing wrong means an opaque C1083 partway through a release.
+#
+# The path is deterministic rather than unique so that consecutive runs reuse
+# the baseline rustdoc they just built; concurrent runs are safe because cargo
+# locks the target directory exactly as it does for target/.
+function Get-SemverChecksTargetDirPath {
+    [CmdletBinding()]
+    param(
+        [Parameter(Mandatory = $true)][string]$RepoRoot,
+        [Parameter(Mandatory = $true)][string]$PackageName
+    )
+
+    if (-not $IsWindows) {
+        return $null
+    }
+
+    # Normalise first: the digest has to be stable across callers that spell the
+    # same repository root differently (trailing separator, relative segments,
+    # or casing, none of which Windows treats as distinct). GetFullPath also
+    # rewrites forward slashes, so only backslashes remain to trim.
+    #
+    # A malformed or over-long root makes GetFullPath throw. Rather than abort
+    # -- which would strand exactly the long paths this exists for -- fall back
+    # to the raw string and let the relocation proceed unprojected.
+    $normalised = $true
+    try {
+        $full = [System.IO.Path]::GetFullPath($RepoRoot)
+    } catch {
+        Write-Warning "Could not normalise '$RepoRoot' ($($_.Exception.Message)); relocating the baseline build on the strength of the path as given."
+        # Everything downstream assumes the backslashes GetFullPath would have
+        # produced, so stand in for the one part of it that still applies.
+        # Without this, 'C:/repo' and 'C:\repo' would digest differently and
+        # break the same-clone-same-directory guarantee on this path alone.
+        $full = $RepoRoot -replace '/', '\'
+        $normalised = $false
+    }
+
+    # Trimming the separator keeps 'C:\repo\' and 'C:\repo' on one digest, but a
+    # drive root is all separator: trimming it would leave 'C:', which names the
+    # current directory on that drive rather than its root.
+    $trimmed = $full.TrimEnd('\')
+    if ($trimmed -notmatch '^[A-Za-z]:$') {
+        $full = $trimmed
+    }
+
+    # A UNC root has no drive letter to anchor to and would keep the very
+    # length this function exists to shed, so fall back to the system drive.
+    $volume = try { [System.IO.Path]::GetPathRoot($full) } catch { '' }
+    $isUnc = $volume.StartsWith('\\')
+    if ([string]::IsNullOrWhiteSpace($volume) -or $isUnc) {
+        $volume = "$env:SystemDrive\"
+    }
+
+    # UNC has already lost the argument -- its own prefix is long enough that
+    # the projection below is not worth consulting -- and an unnormalised path
+    # is not a length worth trusting.
+    if (-not $isUnc -and $normalised) {
+        $nameShift = $PackageName.Length - $script:SemverChecksNestingPackage.Length
+        $projected = $full.Length + $script:SemverChecksNestingLength + $nameShift
+        if (($projected + $script:SemverChecksNestingSlack) -le $script:MaxPathLength) {
+            Write-Verbose "Baseline build projects to about $projected characters; leaving it under the repository's own target directory."
+            return $null
+        }
+    }
+
+    $sha = [System.Security.Cryptography.SHA256]::Create()
+    try {
+        $digest = $sha.ComputeHash([System.Text.Encoding]::UTF8.GetBytes($full.ToLowerInvariant()))
+    } finally {
+        $sha.Dispose()
+    }
+    $token = [System.BitConverter]::ToString($digest[0..3]).Replace('-', '').ToLowerInvariant()
+
+    return (Join-Path $volume (Join-Path $script:SemverChecksTargetDirName $token))
+}
+
+# Runs cargo semver-checks, linking with rust-lld and building under a short
+# target directory where MSVC tooling would otherwise overflow MAX_PATH.
 #
 # The setting travels by environment variable because that is the only channel
 # that reaches the cargo invocation which matters. cargo-semver-checks exposes
@@ -1046,6 +1170,26 @@ function Invoke-SemverChecksCli {
             }
         }
 
+        $targetDirApplied = $false
+        $targetDir = Get-SemverChecksTargetDirPath -RepoRoot $RepoRoot -PackageName $PackageName
+        if ($targetDir) {
+            if (Test-Path 'Env:\CARGO_TARGET_DIR') {
+                # As with the linker, an explicit choice wins: whoever set this
+                # has already decided where the build should land.
+                Write-Verbose 'CARGO_TARGET_DIR is already set; building where it points.'
+            } else {
+                try {
+                    $null = New-Item -ItemType Directory -Path $targetDir -Force -ErrorAction Stop
+                    $env:CARGO_TARGET_DIR = $targetDir
+                    $targetDirApplied = $true
+                } catch {
+                    # Fail open -- a probe must never break a release. Name the
+                    # symptom, though, so a later C1083 is not a mystery.
+                    Write-Warning "Could not create '$targetDir'; building under the default target directory, which was projected to be too long for MSVC and so may fail with C1083. ($($_.Exception.Message))"
+                }
+            }
+        }
+
         try {
             # A required version bump produces an expected non-zero exit code.
             $PSNativeCommandUseErrorActionPreference = $false
@@ -1055,6 +1199,9 @@ function Invoke-SemverChecksCli {
             if ($applied) {
                 Remove-Item -Path "Env:\$linkerVar" -ErrorAction SilentlyContinue
             }
+            if ($targetDirApplied) {
+                Remove-Item -Path 'Env:\CARGO_TARGET_DIR' -ErrorAction SilentlyContinue
+            }
         }
     } finally {
         Pop-Location
@@ -1152,7 +1299,7 @@ function ConvertFrom-SemverChecksOutput {
     }
 
     $pathHint = if ($IsWindows) {
-        ' If the output contains LNK1104 or a path-length error, a MAX_PATH-bound tool was reached; shorten the repository path.'
+        " If the output contains LNK1104, C1083 or a path-length error, a MAX_PATH-bound tool was reached. Build artifacts can nest over 200 characters below the target directory. These scripts relocate the build to a short path when they judge the default one too long, but that judgement is a projection and can fall short. Set CARGO_TARGET_DIR to a short path (for example ${env:SystemDrive}\$script:SemverChecksTargetDirName) to force the relocation, or move the repository closer to the volume root."
     } else {
         ''
     }

scripts/tests/Pester/integration/SemverChecksLinker.Tests.ps1

@@ -84,3 +84,99 @@ edition = "2021"
         }
     }
 }
+
+Describe 'relocated target directory' {
+    # The linker override cannot help the C compilers that -sys crates drive
+    # through the cc crate: MSVC cl.exe resolves its -Fo argument against
+    # MAX_PATH, and there is no long-path-aware drop-in guaranteed to be
+    # installed. The unit tests pin the environment-variable lifecycle; this
+    # exercises the claim underneath it, that the ceiling is real and that
+    # shortening the root is what clears it.
+    It 'compiles at a depth that defeats cl.exe once the root is shortened' -Skip:(-not $IsWindows) {
+        $vswhere = Join-Path ${env:ProgramFiles(x86)} 'Microsoft Visual Studio\Installer\vswhere.exe'
+        if (-not (Test-Path -LiteralPath $vswhere)) {
+            Set-ItResult -Skipped -Because 'vswhere is absent, so no MSVC toolchain can be located'
+            return
+        }
+
+        $vsPath = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath |
+            Select-Object -First 1
+        $vcvars = if ($vsPath) { Join-Path $vsPath 'VC\Auxiliary\Build\vcvars64.bat' } else { $null }
+        if (-not $vcvars -or -not (Test-Path -LiteralPath $vcvars)) {
+            Set-ItResult -Skipped -Because 'no MSVC C toolchain is installed'
+            return
+        }
+
+        $root = Join-Path ([System.IO.Path]::GetTempPath()) ('oxi-cl-' + [guid]::NewGuid().ToString('n').Substring(0, 8))
+        $null = New-Item -ItemType Directory -Path $root -Force
+
+        try {
+            # No #include, so the compile needs nothing from INCLUDE and the only
+            # variable under test is the length of the output path.
+            $source = Join-Path $root 'probe.c'
+            Set-Content -LiteralPath $source -Value 'int probe(void){return 42;}' -Encoding ascii
+
+            $deep = $root
+            while ($deep.Length -lt 240) {
+                $deep = Join-Path $deep 'nested-directory-segment'
+            }
+            # CreateDirectory rather than New-Item: the \\?\ prefix is what lets
+            # this exceed MAX_PATH, and `?` is a wildcard to PowerShell's -Path,
+            # for which New-Item offers no -LiteralPath counterpart.
+            $null = [System.IO.Directory]::CreateDirectory("\\?\$deep")
+
+            # Named after the object that failed in the field, so the reproduction
+            # keeps the same shape as the report.
+            $leaf = 'a9466447ad5a187b-jitterentropy-health.o'
+            $deepObj = Join-Path $deep $leaf
+            $deepObj.Length | Should -BeGreaterThan 260 -Because 'the premise is a path MSVC cannot open'
+
+            $compile = {
+                param($outPath)
+                # A failing compile is the premise of the first half of this
+                # test, not an error: the suite runner turns non-zero native
+                # exits into terminating errors, so opt out for these calls.
+                $PSNativeCommandUseErrorActionPreference = $false
+                $line = "call `"$vcvars`" >nul && cl.exe -nologo -c `"$source`" `"-Fo$outPath`""
+                $text = & $env:ComSpec /c $line 2>&1 | Out-String
+                [pscustomobject]@{ Output = $text; ExitCode = $LASTEXITCODE }
+            }
+
+            $deepResult = & $compile $deepObj
+            if ($deepResult.ExitCode -eq 0) {
+                Set-ItResult -Skipped -Because 'this toolchain opens long output paths, so the relocation has nothing to prove here'
+                return
+            }
+
+            # C1083 specifically: a laxer pattern would let an unrelated failure
+            # satisfy the premise and make the assertion below vacuous.
+            $deepResult.Output | Should -Match 'C1083'
+
+            # Now the same compile beneath the directory the release scripts pick.
+            $shortRoot = Get-SemverChecksTargetDirPath -RepoRoot $root -PackageName 'fetch'
+            $shortRoot | Should -Not -BeNullOrEmpty -Because 'the temp root is long enough to call for relocation'
+            $shortDir = Join-Path $shortRoot 'probe'
+
+            # Creating a directory at the volume root can be denied by policy on
+            # a locked-down machine. Production warns and builds in place when
+            # that happens, so the test skips rather than reporting a failure it
+            # is not evidence of.
+            try {
+                $null = New-Item -ItemType Directory -Path $shortDir -Force -ErrorAction Stop
+            } catch {
+                Set-ItResult -Skipped -Because "the volume root is not writable here: $($_.Exception.Message)"
+                return
+            }
+            try {
+                $shortResult = & $compile (Join-Path $shortDir $leaf)
+
+                $shortResult.ExitCode | Should -Be 0 -Because "cl.exe should compile under the short root, but said: $($shortResult.Output)"
+                $shortResult.Output | Should -Not -Match 'C1083'
+            } finally {
+                Remove-Item -LiteralPath $shortRoot -Recurse -Force -ErrorAction SilentlyContinue
+            }
+        } finally {
+            Remove-Item -LiteralPath "\\?\$root" -Recurse -Force -ErrorAction SilentlyContinue
+        }
+    }
+}

scripts/tests/Pester/unit/releasing/PureFunctions.Tests.ps1

@@ -313,7 +313,14 @@ Describe 'ConvertFrom-SemverChecksOutput' {
 
     It 'includes the Windows path-length hint on build failures' -Skip:(-not $IsWindows) {
         { ConvertFrom-SemverChecksOutput -Output 'LINK : fatal error LNK1104' -PackageName 'foo' } |
-            Should -Throw -ExpectedMessage '*shorten the repository path*'
+            Should -Throw -ExpectedMessage '*CARGO_TARGET_DIR*'
+    }
+
+    It 'names the compiler failure alongside the linker one in the hint' -Skip:(-not $IsWindows) {
+        # The relocated build directory removed LNK1104, which moved the ceiling
+        # onto cl.exe; a hint that named only the linker would misdirect.
+        { ConvertFrom-SemverChecksOutput -Output 'fatal error C1083' -PackageName 'foo' } |
+            Should -Throw -ExpectedMessage '*C1083*'
     }
 
 }