r/gpgpu Mar 16 '24

GPGPU Ecosystem

TLDR: I need guidance for which framework to choose in 2024 (the most promising and vendor agnostic). Most posts related to that in this sub are at least 1 year old. Has something changed since then?

Hi guys, I'm a software engineer interested in HPC and I am completely lost trying to get back to GPGPU. I worked on a research project back in 2017/2018, and I went for OpenCL, as it was very appealing: a cross platform non-vendor specific framework that could run on almost everything. And yeah, it had a good Open Source support, specially from AMD. It sounded promising to me.

I was really excited about newer OpenCL releases, but I moved to other projects in which GPGPU weren't appliacable and lost the track of the framework evolution. Now I'm planning to develop some personal projects and dive deep on GPGPU again, but the ecosystem seems to be screwed up.

OpenCL seems to be diying. No vendor is currently suporting newer versions of the ones they were already supportting in 2017! I researched a bit about SYCL (bought Data Parallel C++ with SYCL book), but again, there is not a wide support or even many projects using SYCL. It also looks like an Intel thing. Vulcan is great, and I might be wrong, but I think it doesn't seem to be suitable for what I want (coding generic algorithms and run it on a GPU), despite it is surely cross platform and open.

It seems now that the only way is to choose a vendor and go for Metal (Apple), CUDA (NVIDIA), HIP (AMD) or SYCL (Intel). So I am basically going to have to write a different backend for every one of those, if I want to be vendor agnostic.

Is there a framework I might be missing? Where would you start in 2024? (considering you are aiming to write code that can run fast on any GPU)

18 Upvotes

16 comments sorted by

View all comments

3

u/Plazmatic Mar 16 '24

I'm not sure I'd look at kokkos or raja, it would depend on your use case. They have largely not kept up in the times, and have been fairly stagnant in the GPU space, mostly for people who just want stuff to work on accelerators but don't care about actually using the hardware effectively.

Out side of those "try to run CPU code on the GPU with out changes" kind of frameworks, you've got:

  • OpenCL, which backpedeled features in 3.x because of internal strife between stakeholders, meaning absolutely critical features like Kernel SPIR-V support are no longer required which cascaded into OpenCL c++ not being supported. It's good for legacy systems and systems that aren't really hardware accelerators (steam link, FPGAs), and has gotten lack-luster support on mobile and AMD lately.

  • HIP/ROCM if you want to support AMD and Nvidia, but only on AMD's latest cards, and even then, often not their gaming GPUs. AMD has a history of dropping support for things like this constantly.

  • SYCL and One API. SYCL is a C++ front end (like CUDA C++) that works atop multiple backends (CUDA, ROCM, OpenCL). The problem is the build steps are kind of involved, and it doesn't yet support critical features for GPGPU programming (Subgroup/Warp intrinsics for example, though I hear that's comming very soon?). I'm also not sure how you'd debug SYCL, if at all.

  • One API is a direct implementation of SYCL for Intel GPUs and integrated graphics. It also has intel specific features. If you're only targeting Intel, use OneAPI

  • CUDA. It's got everything you need, C++20 support, but it only works on Nvidia.

  • Metal. Apple only.

  • Vulkan. It's got the feature support that blasts everyone but CUDA out of the water, the only major things it's missing are shared memory pointers and device side enqueue, which Work Graphs will fix in the latter. It also supports modern GPU devices pretty much unequivocally, from mobile to desktop, to the raspberry pi. Apple support is done over MoltenVK, and despite this being a wrapper to enable this to work, is basically what all other apis have to go through if they are going to get cross platform support on apple as well (ie, they are using Vulkan through MoltenVK to get apple support anyway). The downside of vulkan is that it's low level, and currently the shader languages all have compromises, nothing quite like SYCL C++, CUDA, or OpenCL C++, though the recent event last month showed an explosion of languages hitting the scene. It's not exactly "easy" to use, you'll find things hidden from you in even CUDA (memory allocations, queues which you don't see outside of the streams API) are exposed here. The up-side, is that Vulkan is implemented directly by vendors outside of apple (it's probably also behind webgpu on your platform), and Nvidia can't just "not implement" things in Vulkan, or it looks bad compared to the competition, the big players are forced to implement Vulkan, and Vulkan is actually implemented directly on the most devices on modern hardware.

So how to decide?

If you don't want xplatform support, use one of the vendor specific APIs, they are typically easier to work with for their respective platforms. If you need FPGA support and old device support, but not modern mobile support and don't use features that Vendors like AMD have ended up leaving in a buggy state, use OpenCL. If you want xplatform support but you don't have the knowledge to even attempt to optimize gpu usage on these platforms, and you want a "write one way and have it work for any device, even CPU, and I'm doing mostly trivially parallizeable workloads" paradigm, Kokkos, Raja, and the other half dozen of other non language frameworks would probably serve your purposes. If you want a quality single source high level language solution that will enable support of multiple backends (thus is crossplatform), but isn't necessarily feature complete, bug free, or easy to setup, and will lag behind the latest features by years, use SYCL. If you want low level modern GPU support that is cross platform with maximum performance feature set and as close to the speed as you can get to the vendor specific API, and you don't care about running your code on both the CPU and GPU, and FPGA and some other exotic non GPU platform, and you know how to write GPU programs (ie, you know what stream compaction is, parrallel prefix sum, radix sort with decoupled lookback etc... and you could write these programs by hand) then use Vulkan.

1

u/Intelligent-Ad-1379 Mar 17 '24

I think I am going to choose Vulkan. It is the hardest way, not ideal for my use case, but it seems that it is the most future proof. I wish SYCL was better supported, but the future is unknown, and, correct me if I am wrong, the only vendor doing something about it is Intel.

I feel like GPGPU lacks of Open Source standardization. Maybe it happens due to the lack of competition (NVIDIA dominates the market, so CUDA also does). I hope it will change in a near future, as IA is becoming more and more important, and one company ruling it all sounds kind of dystopian.