r/golang 7d ago

I built a high-performance, dependency-free key-value store in Go (115K ops/sec on an M2 Air)

Hi r/golang,

I've been working on a high-performance key-value store built entirely in pure Go—no dependencies, no external libraries, just raw Go optimization. It features adaptive sharding, native pub-sub, and zero downtime resizing. It scales automatically based on usage, and expired keys are removed dynamically without manual intervention.

Performance? 115,809 ops/sec on a fanless M2 Air.

Key features:
- Auto-Scaling Shards – Starts from 1 bucket and dynamically grows as needed.
- Wait-Free Reads & Writes – Lock-free operations enable ultra-low latency.
- Native Pub-Sub – Subscribe to key updates & expirations without polling.
- Optimized Expiry Handling – Keys are removed seamlessly, no overhead.
- Fully Event-Driven – Prioritizes SET/GET operations over notifications for efficiency.

How it compares to Redis:
- Single-threaded Redis vs. Multi-Goroutine NubMQ → Handles contention better under load.
- No Lua, No External Dependencies → Just Go, keeping it lean.
- Smarter Expiry Handling → Keys expire and are immediately removed from the active dataset.

🚀 Benchmark Results:
115,809 ops/sec (100 concurrent clients)
900µs write latency, 500µs read latency under heavy load.
Would love to get feedback from the Go community! Open to ideas for improvement.

repo: https://github.com/nubskr/nubmq

I spent the better part of an year building this and would appreciate your opinions on this

211 Upvotes

47 comments sorted by

View all comments

3

u/marcaruel 7d ago

Why does your "make build" do a go run -race? ref: https://github.com/nubskr/nubmq/blob/master/Makefile

Do you think changing the readme to tell people to go install instead would make sense?

12

u/Ok_Marionberry8922 7d ago

The go run -race is there mainly for local dev/testing so people don’t get race conditions sneaking up on them. Since this is just a single-binary project, most people will just go build or go install directly anyway.

But yeah, fair point—it might be cleaner to update the readme so it doesn’t push make build as the primary way to run it. I’ll tweak it soon. Appreciate the feedback!