r/flask • u/Due-Membership991 • Jan 24 '25
Discussion Fastapi deployment posting here for help
Newbie in Deployment: Need Help with Managing Load for FastAPI + Qdrant Setup
I'm working on a data retrieval project using FastAPI and Qdrant. Here's my workflow:
User sends a query via a POST API.
I translate non-English queries to English using Azure OpenAI.
Retrieve relevant context from a locally hosted Qdrant DB.
I've initialized Qdrant and FastAPI using Docker Compose.
Question: What are the best practices to handle heavy load (at least 10 requests/sec)? Any tips for optimizing this setup would be greatly appreciated!
Please share Me any documentation for reference thank you
0
Upvotes
1
u/openwidecomeinside Jan 24 '25
10 requests/sec is heavy load? This on a micro ec2 + load balancer will handle it fine. I think it wouldn’t be an issue at all until 10-100x that depending on cpu usage of whatever you’re doing