File Upload Timeout

Hi!
I'm running a Django app on a Starter Plan. I want to upload a larger file (70MB) but I get a timeout. I increased timeout for gunicorn to 7 minutes. But, get timeout after 5 - I guess its the the Apache timeout. How would I go about changing Apache timeout on Railway platform? Also, why is it that it is taking so much time to upload? Would a switch to Developer Plan help with the speed? Or, if you guys have any other solution that could help me speed up the upload speed that would be great!

Thanks in advance 🙂

3b89960c-3369-4735-a08b-a5a6c2349a83

@Brody Tagging you as advised in other thread. Thx 🙂

railway does not use Apache, they use something called envoy, envoy is a proxy that forwards all external traffic to your app, and nothing about envoy is configurable by the user, this 5 minute limit is set by railway on all post requests

but I'm sure you have fast internet and it likely wouldn't normally take you 5 minutes to upload a 70mb file. this is where the second problem comes in to play, railways envoy proxy was slightly misconfigured and this resulted in the unintentional speed throttling of upload speeds to a maximum of 10 megabits/s when making requests with http 2, this accidental speed limit only effects http 2 based requests

there is no way to change the time limit for single post requests what so ever. but you can side step around the accidental throttling by using cloudflare in front of your app, cloudflare will use http 1.1 to connect to your app, and there is no throttling issues with http 1.1 thus no throttling

and to answer another question, all this applies the same for the developer plan too

@babadudi - if you need me to clarify anything, just ask

Thanks for the detailed answer. Yeah, I already switched to Developer Plan and can see that its the same. And yes, could you please provide more info on how to do the Cloudflare setup?

do you own a custom domain?

yes

who did you buy it from

its a Czech provider

okay, you will have to change over to using cloudflare's nameservers (you do not have to transfer the domain)

let me send you a guide on how to do that, one sec

can you send me your custom domain, just so I can track your progress

But, Im a bit sceptical if this is the issue. Even if it would be uploading with 10mbs it should be uploaded in 1 minute

okay so what's your upload speed?

like from speedtest.net

9MBs

do you have insomnia on your computer?

no

should I download it?

do you think you could download it for this, I have a utility hosted on railway that will give you your real upload speed to railway

ok

let me know when you got that downloaded

got it

So what should I do? 🙂

one sec

sure

just had to hop on my computer
open insomnia
application -> preferences -> request / response -> request timeout, set it to 0

dont change anything else

got it

do you have a 20-30mb file on hand? doesnt matter what it is at all

Yes

send that file in a post request to https://utilities.up.railway.app/upload as a binary file, then show me what the response says when it completes

uploaded file size 29.07 Megabytes
server received file in 79.70 seconds
average upload speed 0.36 Megabytes/s or 2.92 Megabits/s

note: this endpoint has a throttler set to 50.00 Megabits/s

yep 2.92 megabits /s

yeah but upload to my app is definitely slower

what tech stack

I tried to upload various file sizes and I could upload maximum 60MB it the 5 minute timeout

django, gunicorn

oh i figured i prob should ask, what are you doing with these uploaded files

loading them to a Buffer and then parsing data. Its an xml file

memory buffer, i wonder where the slowdown is coming from

in this case, you are more limited by your own code than railway

since you can upload to my endpoint faster than you can upload to yours

Not sure, because I added some logging for debugging and I dont even hit the code that handles what happens after a POST request is sent.

But yeah, that is weird 😄

And I dont even see '1' printed out and I get a timeout

well yeah the connection would be terminated at 5 mins and your code would bail before it does anything with the data

hold on just a moment

where are you uploading these files from

Either from an url or a file through a form

show me the url that these requests are being sent to

okay you are using https, all good there

so yeah you are running into the limitations of your own code before you are running into the throttling limits of railway, might be worth it to search around for your problem on stack overflow and alike, since at the moment its not a railway issue

Ok. Thanks a lot for help 🙂

no problem!