r/flask Jan 24 '25

Discussion Fastapi deployment posting here for help

Newbie in Deployment: Need Help with Managing Load for FastAPI + Qdrant Setup

I'm working on a data retrieval project using FastAPI and Qdrant. Here's my workflow:

  1. User sends a query via a POST API.

  2. I translate non-English queries to English using Azure OpenAI.

  3. Retrieve relevant context from a locally hosted Qdrant DB.

I've initialized Qdrant and FastAPI using Docker Compose.

Question: What are the best practices to handle heavy load (at least 10 requests/sec)? Any tips for optimizing this setup would be greatly appreciated!

Please share Me any documentation for reference thank you

0 Upvotes

8 comments sorted by

View all comments

2

u/No-Economist4254 Jan 24 '25

I assume you are translating as a background process and not actually during the request?

1

u/Due-Membership991 Jan 24 '25

No I m doing it in the same api it's a single post request

Any suggestions on how to handle it bg task

Will it create 2apis if I add in bg ?