Skip to content

Decouple host drivers and shutdown - #2702

Open
Arc676 wants to merge 1 commit into
NVIDIA:mainfrom
Arc676:decouple-driver-host-shutdown
Open

Decouple host drivers and shutdown#2702
Arc676 wants to merge 1 commit into
NVIDIA:mainfrom
Arc676:decouple-driver-host-shutdown

Conversation

@Arc676

@Arc676 Arc676 commented Jul 31, 2026

Copy link
Copy Markdown

Description

Decouple the host shutdown flag from the host driver flag. This adds support for systems that have the NVIDIA drivers installed locally but do not run systemd, for example Talos. The user can install the drivers via system extension but set WITH_SHUTDOWN_HOST_GPU_CLIENTS=false via Helm values to disable all code paths in the MIG manager that depend on systemd.

This requires a change to the chart because the entrypoint for the MIG manager is managed by the operator and the variable is set here. The user can't override the value or modify the ConfigMap; the former approach would have no effect and the latter approach would be immediately reverted by the operator.

See discussions in NVIDIA/mig-parted#382 and the parent issue NVIDIA/mig-parted#356.

Checklist

  • No secrets, sensitive information, or unrelated changes

  • Lint checks passing (make lint)

    • Skipped since no Go changes
  • Generated assets in-sync (make validate-generated-assets)

    • Didn't work, although not sure why
       - Generating CRDs from the codebase
       $PATH_TO_REPO/bin/controller-gen rbac:roleName=gpu-operator-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases
       -: build constraints exclude all Go files in $PATH_TO_REPO/vendor/github.com/cyphar/filepath-securejoin/pathrs-lite
       Error: not all generators ran successfully
       run `controller-gen rbac:roleName=gpu-operator-role crd webhook paths=./... output:crd:artifacts:config=config/crd/bases -w` to see all available markers, or `controller-gen rbac:roleName=gpu-operator-role crd webhook paths=./... output:crd:artifacts:config=config/crd/bases -h` for usage
       make: *** [manifests] Error 1
      
  • Go mod artifacts in-sync (make validate-modules)

    • Again, no code changes
  • Test cases are added for new code paths

    • No new code paths

Testing

Tested manually on a Kubernetes cluster running the aforementioned patch.

@copy-pr-bot

copy-pr-bot Bot commented Jul 31, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Don't assume that systemd clients must be shut down if the host has
drivers

Signed-off-by: Alessandro Vinciguerra <alessandro.vinciguerra@postfinance.ch>
@Arc676
Arc676 force-pushed the decouple-driver-host-shutdown branch from ac78991 to 5aa78c1 Compare July 31, 2026 07:34
@myeolenv

myeolenv commented Aug 1, 2026

Copy link
Copy Markdown

Thanks for the contribution. We will review this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants