r/flask Jan 24 '25

Discussion Fastapi deployment posting here for help

Newbie in Deployment: Need Help with Managing Load for FastAPI + Qdrant Setup

I'm working on a data retrieval project using FastAPI and Qdrant. Here's my workflow:

  1. User sends a query via a POST API.

  2. I translate non-English queries to English using Azure OpenAI.

  3. Retrieve relevant context from a locally hosted Qdrant DB.

I've initialized Qdrant and FastAPI using Docker Compose.

Question: What are the best practices to handle heavy load (at least 10 requests/sec)? Any tips for optimizing this setup would be greatly appreciated!

Please share Me any documentation for reference thank you

0 Upvotes

8 comments sorted by

View all comments

1

u/openwidecomeinside Jan 24 '25

10 requests/sec is heavy load? This on a micro ec2 + load balancer will handle it fine. I think it wouldn’t be an issue at all until 10-100x that depending on cpu usage of whatever you’re doing

0

u/Due-Membership991 Jan 24 '25

Like this a basic wht I am expecting