Getting 503 Server Error during load tests

I'm load testing a simple fastapi backend running on railway. Using locust for load testing with 100 users with around 10 http requests per second. I observe that around 20% of http requests are returning 503 Server Error at locust client. Same setup works fine locally. Is railway throttling/limitting these requests or am I doing something wrong?

Any help will be appreciated. Thanks.

Sample Error:
POST /api/v1/auth/login: HTTPError('503 Server Error: Service Unavailable for url: /api/v1/auth/login')
Project Id: a5a6f516-4f3a-4f38-979b-b0689c7e5ab7
whats your start command on railway?
Didn't configure it, using the defaults
I don't think there is a default built into nixpacks for fastapi, would you happen to have a Procfile?
Nope, my assumption is nixpacks is picking up main.py from project root, here is how main.py looks like

import os
from app import app
import uvicorn

if name == "main":
uvicorn.run(app, host="", port=int(os.getenv("PORT", 8000)))
yep that would do it, I was thinking about running the uvicorn command directly
either way, you may want to play around with more workers and more threads
Yep, have some plans for that, but for now I wanted to squeeze out performance from simple single worker config and see what it can deliver at its best ...
it looks like you've already done that
Maybe, but getting the 503 errors as mentioned above 😅
yeah seems like uvicorn can't keep up
I would be interested to see how hypercorn does
Not sure if uvicorn is having trouble with handling 10 requests per second, I seen it deliver much more. I'm looking more deep into the cause. However I strongly feel 503 Server Error type errors are something a proxy or a load balancer would throw to rate limit requests. Wanted to check if railway has some mechanism/service which is blocking these request by certain criteria or is it more to do with fastapi/uvicorn?
I'm positive railways envoy proxy can handle plenty more then 10 r/s
there would be no rate limiting, there's only metering
Thanks for the info, will now dig deeper into fastapi/uvicorn then ...
give hypercorn a try too!
Yep, on my todo list 🙂
So I carried few more experiments to find the root cause of 503 Server Error type errors:

Experiment 1: Original Setup (Default uvicorn config)
I found following response header and text
{'content-type': 'text/html', 'x-railway-fallback': 'true', 'content-length': '2942', 'date': 'Mon, 04 Sep 2023 10:01:25 GMT', 'server': 'railway'}

Text: A html page which says
<h1 class="error-404">Nothing here... yet</h1>
<h1 class="error-503">Application failed to respond</h1>

<a href="https://railway.app" target="_blank"> Go to Railway </a>

Experiment 2: Changed uvicorn's limit_concurrency to 10 in order to force induce 503 errors.
{'date': 'Mon, 04 Sep 2023 10:17:40 GMT', 'server': 'railway', 'content-type': 'text/plain; charset=utf-8', 'transfer-encoding': 'chunked'}
Service Unavailable

From above experiments I see a difference between response headers mainly 'x-railway-fallback': 'true' being set when the 503 error is not thrown directly by uvicorn/fastapi in Experiment 1, plus I don't see 503 errors in the railway's Observability section in the same Experiment.

Whereas 'x-railway-fallback': 'true' is missing from headers in Experiment 2 and I see 503 errors in the railway's Observability section in this experiment.

Any ideas what this could mean?
I did try hypercorn and it was quite effective in reducing error rate. However it does produce similar 503 Server Error type errors though in significantly less amount as compared to uvicorn.
interesting results, the endpoint you are stress testing, what does it return?
You are asking about expected success result or the erroneous result?
the expected response? like what does the endpoint you are testing return? or am I thinking about this incorrectly?
I'm stress testing complete user journey so I'm hitting around 13 endpoints which return response which vary in shape of empty response to a complex json response.
ah yes i see now

Looking for more? Join the community!

Recommended Posts
Ideal way to compose and manage a full stack app?Railway doesn't really seem to be exposing an example of how one could/should compose a full stack aUsage estimatesSo im trying to make a simple project using a MySQL database and im trying to figure out how much itMy website is not running due to SSL supportThe browser is showing following message: alfajr.edu.pk uses an unsupported protocol. ERR_SSL_VERSIPricing/resource questionHere's a silly question, how should I decide on what plan to use for hosting? I'm pretty dumb at thiAny solution for Typebot 2.17? Still showing server errorAPI response timeHello! I am looking for the best place to deploy my API and I currently have the "Pro Plan" package.Added a template as CRON service and it shows "Your project has no deploys"I added this template to take automatic DB backup - https://railway.app/template/UGKaB8 I added a Cservice in only one environment?Is it possible to have certain service(s) only in specific environment? I am making a new cron job tdeploy not building uptried it already three times now canceled teh ones before at around 5 mins the third is currently atError connecting to Redis database.As shown in the picture, I cannot connect to the database. I do not know the reason, but I think thaLove Railway but confused on pricingHello! I’m currently using the free trial of Railway for a service, and I’m loving it. As my free trWhat I thought was a simple python upgrade broke my deployed flask service. Need some help.Project ID: b06af9f3-d9c9-4d4c-a74f-c5521f149984/service/872870dd-f98a-486c-9f14-09fcd5bc43e5Question regarding pausing subscriptionHi there! I’m relatively new to railway, I’ve registered before the pricing change and got verifiedError with discord bot using go and pythonI am new to asking questions in a place like this, so please forgive me if there are any problems ISelenium driver unable to fetch page in Selenium deploymentI am facing an issue while deploying my app via railway.app. 

Here is the description of the app: I up loaded Tesseract-OCR Folder on gitup but on railway cant't findpytesseract.pytesseract.TesseractNotFoundError: tesseract is not installed or it's not in your PATH.How to enable Email on CloudflareI have a React/NodeJS app running on Railway using a custom domain. Cloudflare DNS records have beeUnused DatabaseHey, I have a backend talking to my express server. Both are hosted on Railway. The thing is I am noRun a command in production (rake)Hi there, newcomer from Heroku here — I need to run a command in my production environment (in this don't remember how to log in into my accountI need to access to my account to suspend something, but I faced this notification (in the picture)