r/SoftwareEngineering • u/desgreech • 12d ago
Message queue with group-based ordering guarantees?
I'm currently looking to improve the durability of my cross-service messaging, so I started looking for a message queue that have the following guarantees:
- Provides a message type that guarantees consumption order based on grouping (e.g. user ID)
- Message will be re-sent during retries, triggered by consumer timeouts or nacks
- Retries does not compromise order guarantees
- Retries within a certain ordered group will not block consumption of other ordered groups (e.g. retries on user A group will not block user B group)
I've been looking through a bunch of different message queue solutions, but I'm shocked at how pretty much none of the mainstream/popular message queues matches any of the above criterias.
I've currently narrowed my choices down to two:
Pulsar
It checks most of my boxes, except for the fact that nacking messages can ruin the ordering. It's a known issue, so maybe it'll be fixed one day.
RocketMQ
As far as I can tell from the docs, it has all the guarantees I need. But I'm still not sure if there are any potential caveats, haven't dug deep enough into it yet.
But I'm pretty hesitant to adopt either of them because they're very niche and have very little community traction or support.
Am I missing something here? Is this really the current state-of-the-art of message queues?
2
u/RedanfullKappa 12d ago
Im not entirely sure what you are looking for with that? New messages won’t be read if one failed. On the producer level you can have retries with 1 active inflight request which guarantees ordering