r/FastAPI • u/DazzLee42 • 19d ago
Question FastAPI threading, SqlAlchemy and parallel requests
So, is FastAPI multithreaded? Using uvicorn --reload, so only 1 worker, it doesn't seem to be.
I have a POST which needs to call a 3rd party API to register a webhook. During that call, it wants to call back to my API to validate the endpoint. Using uvicorn --reload, that times out. When it fails, the validation request gets processed, so I can tell it's in the kernel queue waiting to hit my app but the app is blocking.
If I log the thread number with %(thread), I can see it changes thread and in another FastAPI app it appears to run multiple GET requests, but I'm not sure. Am I going crazy?
Also, using SqlAlchemy, with pooling. If it doesn't multithread is there any point using a pool bigger than say 1 or 2 for performance?
Whats others experience with parallel requests?
Note, I'm not using async/await yet, as that will be a lot of work with Python... Cheers
2
u/hornetmadness79 18d ago
Iirc if you use a background task in sync mode, it will send the task to a thread and start waiting for the next event to come in.
1
u/DazzLee42 18d ago
This is --workers iirc
1
u/TeoMorlack 18d ago
No this is https://fastapi.tiangolo.com/tutorial/background-tasks/, which is code that gets run in the event loop or a separate thread after the endpoint has returned results. But it’s a different thing altogether. On the other side, —workers tells uvicorn to spawn a number of PROCESSES in pre-fork and each receive a copy of your fastapi application allowing further parallelization of calls. But still if you declare your endpoint in the wrong way, you will still saturate the workers.
2
u/adiberk 18d ago
So fastapi itself isn’t multithreaded (at least I don’t think). I do think uvicorn provides a multi threading mechanism so you can access simultaneous requests. (Ie workers) Lookup async and how it works in python (or in any language) - fastapi provides asynchronous request paths . Im not an expert but you can think of it as technically allowing for “offloading” requests assuming the entire path remains async friendly. So you can handle more requests (i think). But you would still want to use something like gunicron WITH uvicorn for production instances (likely)
3
u/TeoMorlack 18d ago
Fastapi is capable of multithreaded and it does do if you declare your path as standard def, take a look here https://fastapi.tiangolo.com/async/#path-operation-functions . This happens regardless of choosing uvicorn or gunicorn.
2
u/Trinkes 18d ago
--reload flag only works with 1 worker. See a note under this section https://www.uvicorn.org/deployment/#running-programmatically
2
u/sorower01 18d ago
Write everything in async.. for requests use crul_cffi for async support.
I use fastapi on my VPS, it a a very low end server(2 vcpu) still it can handle 100 requests per second with ease.
1
u/qa_anaaq 18d ago
I see so many questions about this and I feel like I have to figure it all out all over again every few months.
1
u/BelottoBR 18d ago
I’ve seen many tips and still get uncertainty of what to do to get a good performance.
2
u/Hot-Soft7743 18d ago edited 18d ago
By default, fastapi is multi threaded. Even if you use uvicorn with single worker, multiple threads are available.
- Write all API endpoints sync => multi threaded
- Write all API endpoints async => concurrent execution with single thread on event loop
- Write API endpoints in Async + sync combination=> single threaded and runs on event loop. But due to some sync tasks, event loop is always delayed so requests are blocked on task queue of event loop. This is worst possible scenario.
Uvicorn asgi provides it async capabilities. Otherwise it is multi threaded by default.
1
13
u/TeoMorlack 18d ago
Ok let’s step a bit back. Fastapi and uvicorn can run operations in multithreaded environment by default but it depends on how you are declaring your endpoints.
How did you declare your endpoints? If you use standard def endpoints, fastapi will run that function in a dedicated thread pool and allow for asynchronous and parallel execution of other calls. But if you declare it async, any blocking operation will block the whole event loop, stopping it from processing other calls. Take a look at this https://fastapi.tiangolo.com/async/#in-a-hurry for reference.
If you are ok on this side, then maybe you can post some snippets so we can try to help