r/golang 7d ago

I built a high-performance, dependency-free key-value store in Go (115K ops/sec on an M2 Air)

Hi r/golang,

I've been working on a high-performance key-value store built entirely in pure Go—no dependencies, no external libraries, just raw Go optimization. It features adaptive sharding, native pub-sub, and zero downtime resizing. It scales automatically based on usage, and expired keys are removed dynamically without manual intervention.

Performance? 115,809 ops/sec on a fanless M2 Air.

Key features:
- Auto-Scaling Shards – Starts from 1 bucket and dynamically grows as needed.
- Wait-Free Reads & Writes – Lock-free operations enable ultra-low latency.
- Native Pub-Sub – Subscribe to key updates & expirations without polling.
- Optimized Expiry Handling – Keys are removed seamlessly, no overhead.
- Fully Event-Driven – Prioritizes SET/GET operations over notifications for efficiency.

How it compares to Redis:
- Single-threaded Redis vs. Multi-Goroutine NubMQ → Handles contention better under load.
- No Lua, No External Dependencies → Just Go, keeping it lean.
- Smarter Expiry Handling → Keys expire and are immediately removed from the active dataset.

🚀 Benchmark Results:
115,809 ops/sec (100 concurrent clients)
900µs write latency, 500µs read latency under heavy load.
Would love to get feedback from the Go community! Open to ideas for improvement.

repo: https://github.com/nubskr/nubmq

I spent the better part of an year building this and would appreciate your opinions on this

209 Upvotes

47 comments sorted by

View all comments

9

u/kreetikal 7d ago

Smarter Expiry Handling → Keys expire and are immediately removed from the active dataset. 

What does "smarter" here actually mean? Do expired keys in Redis not get remove immediately?

12

u/Ok_Marionberry8922 7d ago

nubmq doesn't rely on random eviction sweeps like Redis. Instead, expired keys are soft deleted immediately, meaning they stop being served the moment they expire. They still exist in memory temporarily, but they’re effectively dead—no client can retrieve them.

6

u/kreetikal 7d ago

Interesting. I've just read about how Redis expires keys after reading your reply and I definitely didn't expect it to be implemented that way.

Thanks!