fix(tests): exclude blue-named .pth files from green skip-download property - #263
Open
MrAnkleManic wants to merge 1 commit into
Open
Conversation
…operty Since the CorridorKeyBlue filename convention (nikopueringer#241), _discover_checkpoint classifies any checkpoint whose basename contains 'blue' (case-insensitive) as a blue checkpoint. Property 2's strategy could generate such names (e.g. 'blue.pth'), so green discovery correctly found no match and entered the auto-download path the test forbids. The failure surfaced as OSError: [Errno 9] Bad file descriptor rather than an assertion: the bare hf_hub_download mock returned a MagicMock, which shutil.copy2 -> open() treated as file descriptor 1 (MagicMock.__index__() returns 1), reading from and then closing pytest's captured stdout. - Restrict Property 2's green strategy to names without the blue token, mirroring the backend's case-insensitive check. - Add a companion property: blue-named .pth files skip the download when screen_color='blue'. - Give the must-not-download mocks (Properties 2 and 3) a loud AssertionError side effect so any regression fails with a clear message instead of fd corruption. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tests/test_pbt_auto_download.py::TestExistingCheckpointSkipsDownload::test_existing_pth_skips_downloadfails once Hypothesis generates a.pthfilename containing the blue token, e.g.blue.pth(after which the falsifying example is cached in.hypothesis/and the failure becomes deterministic):Root cause
Two layers:
The property is stale since feat: dedicated blue-screen model (CorridorKeyBlue) with auto-detection #241.
_discover_checkpointnow classifies any checkpoint whose basename containsblue(case-insensitive) as a blue checkpoint. The test calls_discover_checkpoint(TORCH_EXT)with the defaultscreen_color="green", so a generated name likeblue.pthis (correctly) filtered out of green discovery and the auto-download path runs — which the test forbids. The property "any existing.pthskips download" stopped being true when the filename convention landed.Why it surfaces as
EBADFinstead of a clean assertion: the test patcheshf_hub_downloadwith a baremock.patch(noreturn_value), so the "downloaded path" is aMagicMock. That flows intoshutil.copy2()→open(src, 'rb'), and sinceMagicMock.__index__()returns1,FileIOtreats the mock as file descriptor 1 — opening pytest's captured stdout for reading and closing it on exit, which raisesOSError: [Errno 9] Bad file descriptorand poisons subsequent examples in the run.Fix (test-only)
BLUE_FILENAME_TOKEN, matched case-insensitively exactly as the backend does)..pthskips the download whenscreen_color="blue".side_effect=AssertionError(...)so any future regression fails with a clear message instead of fd corruption.Verification
All 5 tests in the file pass, including replaying the previously cached
blue.pthfalsifying example from.hypothesis/.ruff checkandruff format --checkare clean.uv.lockuntouched.🤖 Generated with Claude Code