[ENHANCEMENT]: Enable PDL for all cuco kernels

[sleeepyjack]

The Programmatic Dependent Launch (PDL) mechanism allows for a dependent secondary kernel to launch before the primary kernel it depends on in the same CUDA stream has finished executing. Available starting with devices of compute capability 9.0, this technique can provide performance benefits when the secondary kernel can complete significant work that does not depend on the results of the primary kernel.

Cuco kernels should all support PDL.

[KaranPradhan266]

Hi @sleeepyjack, This looks very interesting. I don’t have prior hands on experience with CUDA or cuCollections yet, but I’m comfortable with modern C++ and would love to take this on as a learning opportunity.

Please let me know if there’s a preferred starting point or a specific kernel you’d recommend looking at first.

Also, I’m developing on macOS without local CUDA access, so I’d be relying on CI and reviewer feedback for runtime validation. Please let me know if that works.

[sleeepyjack]

Hi @KaranPradhan266,
Thanks for your interest!

Sure, we can figure this out together. Relying on CI should be fine I think. It would be great though if can get your hands on a Linux box just for compiling the code and catching compilation errors before going to CI. Note that each CI run might take an hour or so. You don't want to wait on CI for every little syntax error.

As for building/developing cuco, we have a correponding section in the project readme. We're happy to help with any questions.

I reached out to our core libraries team as they added PDL support a couple of months ago and have likely delevoped some utility macros we can make use of. I'll let you know once they got back to me.

In the meanwhile, could you read into the docs I shared in the issue description and familiarize with the PDL feature? That would be great.

[sleeepyjack]

Hi @KaranPradhan266 ,
Sorry for the delay - I was out on vacation.

I've managed to locate the corresponding CCCL macros for PDL.
They are defined in cuda/std/__cccl/cuda_capabilities.h.

Placing the _CCCL_PDL_GRID_DEPENDENCY_SYNC() correctly before the first read in the kernel is of primary importance. Then turn on PDL at the kernel launch.

Placing _CCCL_PDL_TRIGGER_NEXT_LAUNCH() can sometimes result in speedups and sometimes cause slowdowns as well. I have also seen cases, where it depends on the problem size whether the trigger is beneficial or a slowdown. It's fine to not trigger the next launch (since it's implicit at the end of the kernel). We can evaluate this with benchmarks.

Someone on Reddit posted a nice tutorial on how to use cuco on Google Colab. Would that work for you?

[KaranPradhan266]

Thanks so much for the pointers, @sleeepyjack !
I’ll check out that video and get started with the PDL macros you mentioned.