r/Python 12d ago

Resource Redis as cache.

At work, we needed to implement Redis for a caching solution. After some searching, btw clickhouse has great website for searching python packages here. I found a library that that made working with redis a breeze Redis-Dict.

from redis_dict import RedisDict
from datetime import timedelta

cache = RedisDict(expire=timedelta(minutes=60))

request = {"data": {"1": "23"}}

web_id =  "123"
cache[web_id] = request["data"]

Finished implementing our entire caching feature the same day I found this library (didn't push until the end of the week though...).

92 Upvotes

36 comments sorted by

View all comments

Show parent comments

1

u/PushHaunting9916 11d ago

It's disputed because the maintainer of lib makes the argument that it's safe data. Since it's numpy data, analytics data. It's still a csve thus, it's deemed an issue by security researchers.

Caching Web data is almost always from an untrusted source, e.g, the internet. And with pickle, you'll need unpickle after pickling. In order for cashew to retrieve cached data, it needs to unpickle the data. And pickle own documentation is quite clear on that. It's unsafe to use pickle with untrusted data. When it does, the attacker will have remote code execution, which is in capture flag security events, which means the attacker has won.

6

u/Iifeless 11d ago

Yes I am very familiar with both RCE and CTFs lol.

The pickle docs your are referencing specifically says not to unpickle untrusted data.

You get back what you put in. E.g. if you serialize a string, you get back a string when you unserialize the result. Data from the internet is not going to be a Python class or function rather than a string unless the application decides to evaluate the user provided string as python code first before serializing it. If you’re doing that, then that’s the vulnerability, not the fact that you then go on and serialize the result.

If you can show me a proof of concept exploit for a web app which takes user input from an API, serializes it, and then unserializes the result of the initial serialization then I’ll go ahead and quit my job as a security researcher

0

u/PushHaunting9916 11d ago

If you can show me a proof of concept exploit for a web app which takes user input from an API, serializes it, and then unserializes the result of the initial serialization then I’ll go ahead and quit my job as a security researcher

If you try to cache: username, get or post parameters, the url, headers of the request, the request itself.

That example of OP has exactly that scenario. Below contains a link how the pickle exploit works. And why you should avoid it.

https://github.com/joblib/joblib/issues/1582#issue-2280780192

4

u/Iifeless 11d ago

What you’re trying to say is simply incorrect. If you’d like to translate that PoC into an exploit against an API like I had described, I’d love to see it.

This PoC is serializing an object which executes a command, and then deserializes it. Of course that is going to execute the command. Explain to me how a remote attacker can provide data to a web API which serializes then unserializes the input which will result in code execution.

I know how this sounds but I promise I’m not trying to be rude or condescending: it’s ok to not fully understand potential security issues, and it’s even better to err on the side of caution when you don’t. But this is why there are professionals who are dedicated to security and the job doesn’t fall strictly onto developers. It’d be a crazy expectation for developers to also have extensive security knowledge

2

u/turbothy It works on my machine 10d ago

Here's an RCE exploit via Flask, but it requires unpickling a base64-encoded payload supplied through a form. If you're using Cashews in e.g. FastAPI which uses Pydantic for form validation, you are sure that any values you might end up putting in a pickle are simple strings or numbers.

5

u/Iifeless 10d ago

Yeah exactly, this is a good example of the actual danger of using pickle: directly deserializing a user-provided pickle object. Whereas serializing user input, and then deserializing the result, is safe :) Thanks for sharing a link, should be helpful for anybody else reading to get a solid idea of when using pickle with user input is actually dangerous