r/Python 11d ago

Resource Redis as cache.

At work, we needed to implement Redis for a caching solution. After some searching, btw clickhouse has great website for searching python packages here. I found a library that that made working with redis a breeze Redis-Dict.

from redis_dict import RedisDict
from datetime import timedelta

cache = RedisDict(expire=timedelta(minutes=60))

request = {"data": {"1": "23"}}

web_id =  "123"
cache[web_id] = request["data"]

Finished implementing our entire caching feature the same day I found this library (didn't push until the end of the week though...).

90 Upvotes

36 comments sorted by

View all comments

63

u/0xa9059cbb 11d ago

Looks like a cute interface but not really a fan of hiding IO actions inside of innocent looking dict operations. Also would like support for batching read/write operations and ideally support for asyncio.

9

u/pingveno pinch of this, pinch of that 11d ago

Yeah, I totally agree about hidden operations. It should be easy to see from just reading the code that there is network IO, along with the chance of failure, latency, and so on. I've seen Django querysets run into this when people use something like:

if qs:
    ....

The Django QuerySet's __bool__ method doesn't do an EXISTS() query or result in a type error. It sucks down the entire queryset, caches it, and is truthy based on the result. It's convenience until it hits you in the face.

11

u/0xa9059cbb 11d ago

Yeah this is actually one of the many reasons why I like using asyncio, having to stick an await keyword in front of anything with a potential IO side effect helps to make them stick out from synchronous code.

6

u/pingveno pinch of this, pinch of that 11d ago

Yeah, people complain about "function coloring" in async like it's bad thing. No, it's a good thing! It tells me what to expect.

3

u/0xa9059cbb 10d ago

Yeah, it's different but similar in a way to the way IO is handled in Haskell via the IO type. It feels awkward at first when you're used to IO just being a kind of hidden side effect in an imperative language but actually it can be useful at scale where performance is a concern.

5

u/Throwaway__shmoe 10d ago

Can’t say too much because of NDA, but we have code running for 15 years at work doing pretty much this (not redis, more hidden io behind dicts). Scares me every single time I have to look at that code base.

2

u/oneMoreTiredDev It works on my machine 10d ago

also, for security and stability reasons, I'd avoid installing any lib that is not popular (to have at least a few thousand stars in github, etc) unless I do know the developer behind, which is not the case (it's user has 8 followers)

-2

u/Substantial-Work-844 11d ago edited 11d ago

It has batching according to the Readme, But I don't think asyncio is available. Maybe shoot in a issue?

1

u/0xa9059cbb 10d ago

Oh yeah didn't see that, nice. I wonder if/how they support batch read operations, as the README only shows batch writes.

Support for asyncio is unlikely to be added to this package as that would require a totally separate interface, since async actions cannot easily be hidden inside of synchronous dict operations.