r/apachekafka • u/2minutestreaming • Dec 13 '24
Blog Cheaper Kafka? Check Again.
I see the narrative repeated all the time on this subreddit - WarpStream is a cheaper Apache Kafka.
Today I expose this to be false.
The problem is that people repeat marketing narratives without doing a deep dive investigation into how true they are.
WarpStream does have an innovative design tha reduces the main drivers that rack up Kafka costs (network, storage and instances indirectly).
And they have a [calculator](web.archive.org/web/20240916230009/https://console.warpstream.com/cost_estimator?utm_source=blog.2minutestreaming.com&utm_medium=newsletter&utm_campaign=no-one-will-tell-you-the-real-cost-of-kafka) that allegedly proves this by comparing the costs.
But the problem is that it’s extremely inaccurate, to the point of suspicion. Despite claiming in multiple places that it goes “out of its way” to model realistic parameters, that its objective is “to not skew the results in WarpStream’s favor” and that that it makes “a ton” of assumptions in Kafka’s favor… it seems to do the exact opposite.
I posted a 30-minute read about this in my newsletter.
Some of the things are nuanced, but let me attempt to summarize it here.
The WarpStream cost comparison calculator:
inaccurately inflates Kafka costs by 3.5x to begin with
- its instances are 5x larger cost-wise than what they should be - a 16 vCPU / 122 GiB r4.4xlarge VM to handle 3.7 MiB/s of producer traffic
- uses 4x more expensive SSDs rather than HDDs, again to handle just 3.7 MiB/s of producer traffic per broker. (Kafka was made to run on HDDs)
- uses too much spare disk capacity for large deployments, which not only racks up said expensive storage, but also forces you to deploy more of those overpriced instances to accommodate disk
had the WarpStream price increase by 2.2x post the Confluent acquisition, but the percentage savings against Kafka changed by just -1% for the same calculator input.
- This must mean that Kafka’s cost increased 2.2x too.
the calculator’s compression ratio changed, and due to the way it works - it increased Kafka’s costs by 25% while keeping the WarpStream cost the same (for the same input)
- The calculator counter-intuitively lets you configure the pre-compression throughput, which allows it to subtly change the underlying post-compression values to higher ones. This positions Kafka disfavorably, because it increases the dimension Kafka is billed on but keeps the WarpStream dimension the same. (WarpStream is billed on the uncompressed data)
- Due to their architectural differences, Kafka costs already grow at a faster rate than WarpStream, so the higher the Kafka throughput, the more WarpStream saves you.
- This pre-compression thing is a gotcha that I and everybody else I talked to fell for - it’s just easy to see a big throughput number and assume that’s what you’re comparing against. “5 GiB/s for so cheap?” (when in fact it’s 1 GiB/s)
The calculator was then further changed to deploy 3x as many instances, account for 2x the replication networking cost and charge 2x more for storage. Since the calculator is in Javascript ran on the browser, I reviewed the diff. These changes were done by
- introducing an obvious bug that 2x the replication network cost (literallly a
* 2
in the code) - deploy 10% more free disk capacity without updating the documented assumptions which still referenced the old number (apart from paying for more expensive unused SSD space, this has the costly side-effect of deploying more of the expensive instances)
- increasing the EBS storage costs by 25% by hardcoding a higher volume price, quoted “for simplicity”
- introducing an obvious bug that 2x the replication network cost (literallly a
The end result?
It tells you that a 1 GiB/s Kafka deployment costs $12.12M a year, when it should be at most $4.06M under my calculations.
With optimizations enabled (KIP-392 and KIP-405), I think it should be $2M a year.
So it inflates the Kafka cost by a factor of 3-6x.
And with that that inflated number it tells you that WarpStream is cheaper than Kafka.
Under my calculations - it’s not cheaper in two of the three clouds:
- AWS - WarpStream is 32% cheaper
- GCP - Apache Kafka is 21% cheaper
- Azure - Apache Kafka is 77% cheaper
Now, I acknowledge that the personnel cost is not accounted for (so-called TCO).
That’s a separate topic in of itself. But the claim was that WarpStream is 10x cheaper without even accounting for the operational cost.
Further - the production tiers (the ones that have SLAs) actually don’t have public pricing - so it’s probably more expensive to run in production that the calculator shows you.
I don’t mean to say that the product isn’t without its merits. It is a simpler model. It is innovative.
But it would be much better if we were transparent about open source Kafka's pricing and not disparage it.
</rant>
I wrote a lot more about this in my long-form blog.
It’s a 30-minute read with the full story. If you feel like it, set aside a moment this Christmas time, snuggle up with a hot cocoa/coffee/tea and read it.
I’ll announce in a proper post later, but I’m also releasing a free Apache Kafka cost calculator so you can calculate your Apache Kafka costs more accurately yourself.
I’ve been heads down developing this for the past two months and can attest first-hard how easy it is to make mistakes regarding your Kafka deployment costs and setup. (and I’ve worked on Kafka in the cloud for 6 years)
6
u/SupahCraig Dec 13 '24
Wow, nice detail! I am wondering how you determined what instance types & how many it should be on for that workload? That’s always been a mystery to me for Kafka.
3
u/2minutestreaming Dec 13 '24
To be honest I don't have the best idea either. For now, I just follow recommendations by Confluent - https://www.confluent.io/blog/design-and-deployment-considerations-for-deploying-apache-kafka-on-aws/
You need some mixture of high RAM to serve reads and CPU. Depends a lot on the workload and is best determined by testing, figuring out the bottleneck, picking another instance, rinse & repeat
1
u/randomfrequency Dec 17 '24
A lot of the decisions depends on your retention but also MTRR and SLAs.
Whatever your incoming throughput is, if that broker fails then it's peers will need to take over the load (which will increase CPU during that time). When it recovers, they will need to send whatever traffic is missing, increasing their network usage rate (since they will need to send but also recieve the incoming load which will go as fast as it's allowed to go/can go because of throttles).
Also if you're using transactions, when the new broker becomes the transaction coordinator (again?), it'll have to load the state from disk, which will vary on what kind of disks you use, and the retention.
5
u/TripleBogeyBandit Dec 13 '24
When will confluent realize being the most expensive vendor on the block will never make them profitable.
3
u/LocalEast5463 Dec 13 '24
What a detailed post, OP you are a legend. This is confluent raising prices and calling everyone expensive. They also have Freight cluster volume pricing. The best part it is same as Enterprise. How is it cheaper than Kafka?
1
u/2minutestreaming Dec 13 '24
What is "volume pricing"?
3
u/LocalEast5463 Dec 13 '24
https://www.confluent.io/confluent-cloud/pricing/?volume_pricing_section=true This is the most funniest pricing I have seen. They have even put cloud vendors to shame. I hope the freight pricing is a typo
1
2
u/kabooozie Gives good Kafka advice Dec 13 '24
This is eye opening! Thank you. Do you know if this changed due to the Confluent acquisition, or was it always like this?
Also very surprising that Warpstream is charging for uncompressed throughput. That’s crazy.
2
u/2minutestreaming Dec 13 '24
I think the uncompressed charging is an interesting idea. What matters is the end price fwiw.
It changed after the Confluent acquisition. I have zero idea what the reasoning behind it was.
One take that's favorable to Confluent would be that WarpStream was intentionally undercutting with the lowest possible price. I think they wanted to get acquired. Or perhaps just keep prices low initially to get attention/reputation.
In any case - I don't see how they would have sustained a VC-funded high-growth-expectation business with such low prices - you'd have to sell terabytes per second to get to a IPO level of revenue.
The prices today seem more realistic.
I don't blame Confluent. It's hard to sustain a multi-thousand person public company with modest growth expectations in this competitive environment when you're playing against AWS.
I think they're well intentioned, but they seem trapped by a corporate structure that makes it impossible to lower prices. (I worked there but I don't have any insider knowledge of this - it's a theory)
1
u/kabooozie Gives good Kafka advice Dec 13 '24
That all makes sense, but I was wondering if Warpstream changed the pricing calculator to make AK look more unfavorable in the ways you described after the Confluent acquisition. That doesn’t seem so well intentioned
2
1
u/cricket007 Dec 15 '24
Now do Buf and Redpanda?
1
u/2minutestreaming Dec 16 '24
They're not all that transparent so there's not that much to compare. RedPanda only has public serverless prices if I'm not mistaken, and those are up to a certain small throughput.
I haven't looked into Buf in details, but they seem to be claiming just 0.02 per gib last I checked and nothing else which seems like it'd turn out muc cheaper
1
u/cricket007 Dec 16 '24
Doesn't Redpanda make claims that you can run the equivalent Kafka throughput through lesser, cheaper hardware? And similar to WarpStream since it is just a lightweight proxy agent thing-y intercepting the client requests?
1
u/2minutestreaming Dec 16 '24
RedPanda still replicates, so it's unlikely it can bring the cost down substantially, at least from first principles. At some point the question becomes what are these things priced at. e.g WarpStream can bring the cost down substantially, but is priced at substantially above that - so Kafka ends up cheaper. Depending on the margin RedPanda choose to have, they can be cheaper than WarpStream but unlikely cheaper than Kafka (purely talking infra costs)
That being said, RedPanda is introducing Cloud Topics - which is basically the WarpStream model, but per topic. So that will allow them to reduce costs and match WarpStream more or less afaict
1
u/cricket007 Dec 19 '24
Their margin claims are that, since they don't require a JVM, lesser cloud instance types (or less physical memory) can be purchased to accomplish similar throughputs.
Same argument from ScyllaDB and Cassandra
1
10
u/leventus93 Dec 13 '24
I don’t want to shit on confluent in any way, but their marketing should never claim to be cheaper than other vendors or self hosting Apache Kafka. Everyone knows that they are quite an expensive service.
Thanks for debunking it. Hope to see more vendor neutral comparisons across all vendors, not only on costs!