503 Service Unavailable
Getting 503 Service Unavailable even though #🚨｜incidents says the issue is resolved
49 Replies
Project ID:
3b58a837-a508-4969-a318-7c37050d177e3b58a837-a508-4969-a318-7c37050d177e
are you still running into this issue?
yes
wasnt seeing any new logs on railway dashboard since around 8hrs ago.
the dashboard isnt loading anymore rn either
have you restarted your service?
restarting the service worked for us, but sucks we had to wake up this morning and manually restart it
no. trying now
restarting fixed production env
staging still 503s
seeing this error on staging
trying using external db url instead of internal
yeah that fixed it for now. not sure about the internal url
wanna open another help thread for the failed lookup issue?
would love to, but gonna be hard for us to test it out because our build+deploy somehow takes around 10mins 💀
what kind of app is it?
koajs (express) backend api
how big is the image?
2.642GB
that is a very large image, do you have any idea on what makes it so big?
yeah thats something in our backlogs to improve the build. havent gotten around to look into it.
we use a monorepo. Not sure if the whole repo is being built in the image
isolated or shared?
wdym?
is it an isolated or shared monorepo?
not entirely sure. looking at the docs rn
its shared
can you link me to the docs?
i meant the railway docs: https://docs.railway.app/guides/monorepo#deploying-an-isolated-monorepo
there are packages being shared bw backend & frontend subrepos. so its a shared monorepo
nx, turbo, workspaces. what kind of shared monorepo?
pnpm workspaces + turbo
are you making sure to only install the deps that one particular app needs?
not entirely sure. havent gotten around to debugging it yet
what kind of deps does the backend require? anything heavy like chrome?
nah nothing too big like that. typical backend deps like nodemon, test runners, ts, bullmq, koa, typeorm etc.
how long does railway say it takes to build your app? this would be printed in the build logs
4 min 5.9 s
but in total i think i've seen around 8mins from build trigger
so publishing the image is whats taking the longest
yeah seems like it
make sure you arent installing unnecessary packages, and it will likely cut down both times
hmm got it.
is there any way we could replicate the image build in local?
i assume you are using nixpacks on railway right?
yea
you can use nixpacks locally too then! https://nixpacks.com/docs/install
or if thats not something you are interested in, write up a dockerfile then you can test building an image from that locally and railway will use it too.
tbh you probably should go the Dockerfile route, it will give you by far the most control over your build size
hmm. yeah that makes sense. will try out the Dockerfile route. Thats easier / familiar too ig
Thanks! 🙏
no problem! and when you do have the build and deploy times down, feel free to open a thread for that lookup error!
yup. will do
prolly spin up another instance and test on there
sounds good
Hey 👋
what happens if i provide a dockerfile and a buildCommand as well?
wanted to confirm whether the build environment also takes in the same configured environment variables?
is it okay to run test suites during the build?
or is railway considered a deployment only system?
the build command in the service settings is unavailable when building with a dockerfile.
the build environment does make all your service variables available.
test suites would best be done in github, and then you can have railway wait for those to pass / fail before deploying.
Makes sense. Thanks.
The thing about running the tests in GitHub is this:
The build happens once on GitHub for running the tests. Then it happens again at railway
both dont charge for build
Wondering More about the total time taken from push to deploy, not so much about charging
Guess no way around it tho
it indeed would take longer
Yup