Lately, with all that AI buzz, langchain and all, I had to do a lot of Python work, where each service runs as a REST API end-point. Obviously, I wanted to go the “dynamic routing” way, as I did on Foxx Builder (https://github.com/skitsanos/foxx-builder) and other projects, so I needed Flask to handle my route handlers exactly the same way. I went with Gunicorn initially; you can even see it here: https://github.com/skitsanos/flask-dynamic-routes. But it all was fine until its multithreading backfired when I wanted to run it in containers. Azure libs that I used didn't like how it worked, and I needed a bunch of containers behind the load balancer. I had to migrate to Waitress, and my headaches were gone.
Waitress is single-threaded, which means it serves requests in a sequential manner, one after another. This makes it easy to reason about how your application handles concurrent requests.
Gunicorn, on the other hand, uses multiple worker processes (usually multiple threads within each process) to handle concurrent requests. While this can improve concurrency and performance, it can also introduce synchronization and thread safety complexities in your application code.
Waitress is designed to be a simple, production-ready WSGI (Web Server Gateway Interface) server that's easy to set up and use. It's a pure-Python server that doesn't require any additional configuration or management.
Gunicorn, while powerful and feature-rich, can be more complex to set up and configure, especially if you need to manage multiple worker processes. In my case, I didn’t need all that fancy stuff. Gunicorn sounded to me like a swiss-knife, when, in my case, I just needed a simple butter knife.
Waitress is a lightweight server with minimal overhead. It doesn't have as many features as Gunicorn, but it's often sufficient for smaller applications and can be a better choice when you want to keep your server footprint minimal.
Gunicorn is a more heavyweight server with more features and options. This can be advantageous for larger applications with high traffic, but it may be overkill for simpler projects.
Some of my things are offloaded to OpenResty directly, so Lua handles things even before it reaches the Python backend. My main criteria here were to have super-fast deployment, super simple configuration when running in a container, and the ability to handle a big load.
