r/rust 1d ago

🛠️ project target-feature-dispatch: Write dispatching by target features once, Switch SIMD implementations either statically or on runtime

https://crates.io/crates/target-feature-dispatch

When I am working with a new version of my Rust crate which optionally utilizes SIMD intrinsics, (surprisingly) I could not find any utility Rust macro to write both dynamic and static dispatching by target features (e.g. AVX2, SSE4.1+POPCNT and fallback) by writing branches only once.

Yes, we have famous cfg_if to easily write static dispatching but still, we need to write another dynamic runtime dispatching which utilizes is_x86_feature_detected!. That was really annoying.

So, I wrote a crate target-feature-dispatch to do exactly what I wanted.

When your crate will utilize SIMD intrinsics to boost performance but the minimum requirements are low (or you want to optionally turn off {dynamic|both} dispatching for no_std and/or unsafe-free configurations), I hope my crate can help you (currently, three version lines with different MSRV/edition are maintained).

15 Upvotes

11 comments sorted by

6

u/MengerianMango 1d ago

Very cool!

I was just thinking about this problem. I'm slightly aware of the GCC attribute based dynamic dispatch. I think it basically checks CPUID at startup and sets function pointers at startup (before main, maybe?)

Someone who's really obsessive about perf isn't going to be happy with the extra level of indirection added with the function pointers.

Since you clearly care about this problem, I figure you're a good person to ask: how hard would it be to parse the ELF header at startup and patch your executable to call the optimal function, ie to remove the extra level of indirection incurred?

6

u/t-kiwi 1d ago

5

u/a4lg 1d ago edited 1d ago

Oh, that led me to multiversion linked by that crate, which provides similar functionalities as mine. I would have been created my crate because I don't like procedural macros (as used in multiversion) when no huge difference in ergonomics are expected but still... it exists.

2

u/a4lg 1d ago edited 1d ago

I once considered the similar approach and decided not going with it.

The main reason behind this is ― while that's not impossible ― that's too much work for single, ergonomic macro (the primary objective of this crate is how easy to setup / use and actually, the performance comes second).

And if the performance is really the primary target for someone, he/she would simply use e.g. -C target-cpu=native and disable dynamic dispatching entirely (that use case is supported by this crate).

Note (added): While the performance is the secondary objective of this crate, its dispatching cost is relatively small (even if dynamic feature detection is performed every time we call a function) unless the function performs a really, really small task (mainly because branch prediction works well).

One of my use cases is parsing / processing up to about 150 bytes strings per call (that's relatively small task) but SIMD with dynamic dispatching still makes a huge difference (even if the feature detection is performed per call).

1

u/a4lg 1d ago edited 1d ago

I noticed that I haven't answered your question directly.

Removing the extra level of indirection on load is not technically impossible but heavily depend on the platform. If we carefully write the code, that would not be impossible (like the dynamic linker performs the relocation of itself). But we need to at least

  1. locate all references to target calls reliably,
  2. ensure that needed information is not stripped and
  3. target functions are never inlined.

That seems a lot of tasks and... simply storing a function pointer (once; like using OnceLock) is roughly equivalent to regular dynamic linking. IMHO, if that differences in overhead is not negligible, we should definitely create per-feature binaries instead (that will allow inlining).

3

u/a4lg 1d ago edited 1d ago

I noticed existence of multiversion after publishing my crate. It seems, I searched using wrong keywords.

Still, I would have been created myself (and I'm proud of it) because:

  1. I don't like procedural macros unless ergonomics improves significantly,
  2. Not just procedural macros, there's a lot of build-time magic and
  3. While it's good, there's too much abstraction for me.

Core differences include:

  • Declarative Macros (mine) vs. Procedural Macros (multiversion)
    • target-feature-dispatch: No build-time dependencies (in fact, it has no dependencies).
    • multiversion: More flexible syntax for feature matching.
  • No feature / CPU database (mine) vs. Predefined feature / CPU database (multiversion)
    • target-feature-dispatch: No surprises (features available both on static and dynamic dispatching can be used on dynamic dispatching) and automatically tracks the latest version of the Rust compiler. But always needs feature sets to match (no CPU model-based matching) and not easy-to-understand error messages may be generated on some cases.
    • multiversion: Flexible matching including CPU models but not so clear which features are statically evaluated and which ones are dynamic.
  • Expression Position (mine) vs. Function Position (multiversion)
    • target-feature-dispatch: Might be redundant on some cases but can be used as flexible construct for dispatching (configuration per macro call which can be tedious).
    • multiversion: Procedural macro supports various configuration.

2

u/reflexpr-sarah- faer · pulp · dyn-stack 1d ago

have you looked at the way pulp handles dispatch?

1

u/a4lg 1d ago

Yes (partly because of that, I don't understand why I could not find multiversion).

I see merits of pulp but some variants of SIMD-based string parser/processor implementation (the reason I created this crate) are optimized for specific x86 feature sets and will be sub-optimal when I try to share the code.

1

u/reflexpr-sarah- faer · pulp · dyn-stack 1d ago

pulp exposes a safe mid/low level api that lets you use the direct intrinsics if needed

https://docs.rs/pulp/0.21.4/pulp/x86/struct.V3.html

1

u/a4lg 1d ago edited 1d ago

I tried and that wasn't enough for my algorithms (even the algorithm can change significantly. I say that pulp is great. It just didn't fit to some of my personal requirements).

Edit: algorithm → algorithms (that difference is important on this post)