Koboldcpp should index more then the first 4 GPU

[frenzybiscuit]

See attached screenshot.

[wbruna]

I think it should have worked. Apart from the launcher drop-downs, I can't find anything on the SD code actually limiting the max device index.

A quick test:

$ python koboldcpp.py --usevulkan --sdmodel dreamshaper_8.safetensors --sdclipdevice 9
(...)
WARNING: device 9 doesn't exist, falling back to default for CLIP

That warning is already at the C++ layer, a bit above the sd.cpp backend, when we translate indexes to device names. That code doesn't care about the number, as long as it points to a valid ggml device.

[LostRuins]

It was weird, for some reason --sdclipdevice 4 didn't get pass argparse for them, but it worked fine for me.

[frenzybiscuit]

Yeah no, it does appear to work.

Hmmmmmmm

[NintendoManiac64]

5 GPU setups technically exist if you count an iGPU i.e. 4 dGPU + 1 iGPU, but last I checked koboldcpp refuses to use dGPU+iGPU together, insisting on using either dGPU(s) only or iGPU only even if the dGPU(s) and iGPU are from the same vendor (AMD/Intel/Nvidia).

(also the platforms with enough PCIe lanes to properly feed 4 dGPUs tend to not have an iGPU in the first place unless you're doing PCIe 5.0 bifurcation...which I suppose is totally a thing)

[LostRuins]

It's now possible. You can set GPU to all, then override the ones you want with --device e..g --device Vulkan0,Vulkan1 which can use both