R
Railway•12mo ago
Harry12345

Hey Guys👋Getting Request Timeout Issue on Load Test

I deployed a node app with express under railway, everything using railway managed services postgres and redis,. After some time on load test, i started seeing this issue {"remote_ip":"34.87.74.158","remote_port":443,"url":" https://onebet-dev.up.railway.app/event/bet/place","status":0,"status_text":"","proto":"","headers":%7B%7D,"cookies":%7B%7D,"body":null,"timings":%7B"duration":60000.783,"blocked":0.001,"looking_up":0,"connecting":0,"tls_handshaking":0,"sending":0.125,"waiting":60000.658,"receiving":0%7D,"tls_version":"","tls_cipher_suite":"","ocsp":%7B"produced_at":0,"this_update":0,"next_update":0,"revoked_at":0,"revocation_reason":"","status":""%7D,"error":"request timeout","error_code":1050,"request": How to solve this, is this issue due to ratelimiting on railway side or issue on node app side?
23 Replies
Percy
Percy•12mo ago
Project ID: 2cb48114-4aaa-4153-b45f-277a43de1723
Harry12345
Harry12345•12mo ago
2cb48114-4aaa-4153-b45f-277a43de1723
Brody
Brody•12mo ago
railway has no hard set rate limits tell us more about how you're load testing, how many connections are you able to I'm currently achieve before you get timeouts?
Harry12345
Harry12345•12mo ago
i was trying arounf 100 concurrent connection and total of 2000 requests sometimes at 50 concurrent connections it is happening
Brody
Brody•12mo ago
2000 requests per ?? can you give me more metrics?
Harry12345
Harry12345•12mo ago
2000 req in span of 3-5 mins breaking at 50 or 100 rps
Brody
Brody•12mo ago
what kind of data does this endpoint you where testing return?
Harry12345
Harry12345•12mo ago
it will return a json, takes input of a payload the processing will take around 2 secs, it has 3 db calls from prisima and then grpc request the load on db and server running the grpc is fine i am not seeing any spikes on railway instance also still it is getting timeout
Brody
Brody•12mo ago
are you able to do more than 100 concurrent connections and more than 5000 req/5mins when hosting your app elsewhere?
Harry12345
Harry12345•12mo ago
that we havent tried we will try that and update back, but we wanted to understand how railway handle requests for more than 100 rps usually 100-500 rps
Adam
Adam•12mo ago
Given that you’re on the hobby plan, you only have access to 8vCPU and 8GB of RAM. Can you please share your metrics for the period of time you were load testing? it’s possible you have maxed out the CPU
Harry12345
Harry12345•12mo ago
do we directly deploy node app and expose with railway, or is it better if we add some load balancer on top
Adam
Adam•12mo ago
Railway can load balance with replicas, that should be a non issue
Brody
Brody•12mo ago
have you ran the same test locally and achieved more throughput? are you sure this isn't a bottleneck in your code?
Harry12345
Harry12345•12mo ago
sure, ill share the metrics, is the project on hobby plan? my account is not creator of project
Brody
Brody•12mo ago
what plan is the project owner on?
Adam
Adam•12mo ago
I assumed that the project was on your account, your discord account doesn’t have the pro plan role
Harry12345
Harry12345•12mo ago
ill check the owners project plan, and also make load test locally and get back on this, thanks for the insights
Harry12345
Harry12345•12mo ago
No description
Harry12345
Harry12345•12mo ago
these are the metrics in past 6 hrs
Brody
Brody•12mo ago
metrics look perfectly normal, looking forward to the local load testing results
Adam
Adam•12mo ago
agreed. Looks like it’s crashing after a spike, but that spike is nowhere near your max
Harry12345
Harry12345•12mo ago
yes, its better if i load test locally and elimate the possibilty of code level issues, thanks
Want results from more Discord servers?
Add your server