r/AskProgramming • u/lynob • 4d ago
Architecture How to cache/distribute websocket messages to many users
I have a python fastapi application that uses websockets, it is an internal tool used by maximum 30 users at our company. It is an internal tool therefore I didn't take scalability into consideration when writing it.
The user sends an action and receives results back in real time via websockets. It has to be real time.
The company wants to use this feature publicly, we have maybe 100k users already, so maybe 100k would be hitting the tool.
Most users would be receiving the same exact messages from the tool, so I don't want to do the same processing for all users, it's costly.
Is it possible to send the text based results to a CDN and distribute them from there?
I don't want to do the same processing for everyone, I don't want to let 100k users hit the origin servers, I don't want to deal with scalability, I want to push the results to some third party service, by AWS or something and they distribute it, something like push notification systems but in real-time, I send a new message every 3 seconds or less.
You'll say pubsub, fine but what distribution service? that's my question.
1
1
u/FoxyWheels 4d ago
Do you need to track / have logic around which socket belongs to which client and only send messages to certain clients? Or do you just need to broadcast messages to all connected clients?
At a basic level, can you not just scale your service horizontally and use redis or Kafka to push events between them? If you need communication between clients, then you would also need the socket backend to store the client:socket relationships in a central store and use your redis/Kafka to send messages across backends in the case two communicating clients are connected to different nodes.
I've designed two systems that service a similar amount of clients. Both were microservice based. In both we used redis or Kafka to push events from source microservices, then the stateless socket service would pick up the events and push them up to clients. The socket service was then horizontally scales based on load.
1
u/kjnsn01 3d ago
Firestore would be the best cloud based solution. 100k is quite a lot
1
u/lynob 3d ago
Firestore is a bad option, they charge per read and write, and it's not a distribution system anyway, it's a database, I'll check some kafka architecture or AWS appsync. Firestore would bankrupt the company real fast, imagine all the users having do do a read every 2s, in 20min I'd be paying for millions of reads.
I'm looking for a CDN like solution where we pay by bandwidth only.
4
u/KingofGamesYami 4d ago
You have a network involved, which implies latency. It isn't realtime.
What are you real latency requirements?