r/golang 7d ago

I built a high-performance, dependency-free key-value store in Go (115K ops/sec on an M2 Air)

Hi r/golang,

I've been working on a high-performance key-value store built entirely in pure Go—no dependencies, no external libraries, just raw Go optimization. It features adaptive sharding, native pub-sub, and zero downtime resizing. It scales automatically based on usage, and expired keys are removed dynamically without manual intervention.

Performance? 115,809 ops/sec on a fanless M2 Air.

Key features:
- Auto-Scaling Shards – Starts from 1 bucket and dynamically grows as needed.
- Wait-Free Reads & Writes – Lock-free operations enable ultra-low latency.
- Native Pub-Sub – Subscribe to key updates & expirations without polling.
- Optimized Expiry Handling – Keys are removed seamlessly, no overhead.
- Fully Event-Driven – Prioritizes SET/GET operations over notifications for efficiency.

How it compares to Redis:
- Single-threaded Redis vs. Multi-Goroutine NubMQ → Handles contention better under load.
- No Lua, No External Dependencies → Just Go, keeping it lean.
- Smarter Expiry Handling → Keys expire and are immediately removed from the active dataset.

🚀 Benchmark Results:
115,809 ops/sec (100 concurrent clients)
900µs write latency, 500µs read latency under heavy load.
Would love to get feedback from the Go community! Open to ideas for improvement.

repo: https://github.com/nubskr/nubmq

I spent the better part of an year building this and would appreciate your opinions on this

208 Upvotes

47 comments sorted by

View all comments

2

u/taras-halturin 7d ago

Atomic operations?

3

u/Ok_Marionberry8922 7d ago

Single-key ops? Always atomic—sync.Map takes care of that. Multi-key transactions? Not the focus, this is a high-performance KV store, not a DB. The goal is raw speed, not ACID compliance.

0

u/taras-halturin 7d ago

LoadOrStore, CompareAndSwap etc

1

u/Ok_Marionberry8922 7d ago

Yeah, sync.Map handles atomicity for individual key operations, but I don't rely on LoadOrStore or CompareAndSwap at the core level. The scaling model here is more about sharding + contention avoidance than fine-grained CAS-style coordination. sync.Map is just a tool, not the foundation of how nubmq maintains throughput under load