Fall back to nogds at runtime when cuFile handle registration fails - #87
Merged
takeshi-yoshimura merged 1 commit intoJul 7, 2026
Conversation
Collaborator
|
Thanks for the contribution. The change looks good to me, but please add signed-off-by line in your commit. Then, I will merge this after the change #81. |
gitbisector
force-pushed
the
gds-runtime-fallback
branch
2 times, most recently
from
July 3, 2026 05:32
96ff25f to
7775b5e
Compare
Collaborator
|
@gitbisector |
cuFile can probe as available yet fail cuFileHandleRegister at I/O time (compat-mode hosts without nvidia-fs, checkpoints on overlayfs, CI runners). Catch the failure in submit_io, warn once, and transparently delegate the copier to the nogds bounce path -- so every consumer stops carrying its own gds->nogds retry wrapper. The fallback (and its bounce-buffer reader) lives only for the file's submit/wait cycle. Signed-off-by: git bisector <gitbisector@gmail.com> Co-authored-by: Claude <noreply@anthropic.com>
gitbisector
force-pushed
the
gds-runtime-fallback
branch
from
July 6, 2026 22:04
7775b5e to
a82f67e
Compare
Contributor
Author
|
Rebased onto main and resolved the conflict — both |
takeshi-yoshimura
merged commit Jul 7, 2026
02dd37f
into
foundation-model-stack:main
13 checks passed
Collaborator
|
Merged. thanks! |
takeshi-yoshimura
added a commit
to takeshi-yoshimura/fastsafetensors
that referenced
this pull request
Jul 7, 2026
- README: ROCm is no longer nogds-only, drop "without GDS". - overview.md: document hipFile-based direct loading on ROCm >= 7.2 (added in foundation-model-stack#85) instead of claiming ROCm has no GDS equivalent, and reflect that the gds copier now falls back to the nogds path at runtime with a warning (foundation-model-stack#87) rather than failing to open files. - development.md: the Makefile target is test-vllm, not vllm. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Signed-off-by: Takeshi Yoshimura <tyos@jp.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
cuFile can probe as available (
is_gds_supported()/is_cufile_found()pass) yet failcuFileHandleRegisterat I/O time — compat-mode hosts without the nvidia-fs kernel module, checkpoints on filesystems cuFile can't register (e.g. overlayfs on CI runners), etc. Today that surfaces as a hardRuntimeError: raw_gds_file_handle: cuFileHandleRegister returned an error = 5027fromGdsFileCopier.submit_io, and every consumer ends up carrying its own gds→nogds retry wrapper — e.g. vllm-project/vllm#40183 needed exactly that fix when its CI runner hit 5027 (weights on overlayfs, probe says GDS is fine).This catches the registration failure inside
GdsFileCopier.submit_io, warns once, and transparently delegates the copier to the nogds bounce path. The fallback copier (and its bounce-buffer reader) lives only for that file's submit/wait cycle, so no pinned memory outlives the load.Test: monkeypatched
gds_file_handleraising the 5027 error → load completes via the fallback, tensors byte-identical tosafetensors.load_file, and the bounce buffer is released afterwards.make lintclean; unit suite green on CPU and on a CUDA host where cuFile genuinely fails registration (the fallback turns that previously-failing environment green).