R
Railway10mo ago
Mrxbox98

Database private url not working.

Previously my project used the database public url and I just tried to switch to the private url and this is the issue I encounter: https://i.mrxbox98.me/file/2024/02/chrome_mqtsKLgRWC.png
100 Replies
Percy
Percy10mo ago
Project ID: b820c80e-50e9-475a-85c6-89832fccdd8c
Mrxbox98
Mrxbox9810mo ago
b820c80e-50e9-475a-85c6-89832fccdd8c
Fragly
Fragly10mo ago
Could you try adding a 3 second sleep in your start command? reason for this is because private networking takes a bit to initialize, so adding a 3 second sleep will allow it to initialize fully before connecting
Brody
Brody10mo ago
but that kinda looks like the build logs? if so, the private network is not yet available to the builder
Fragly
Fragly10mo ago
oh true, didn't notice that
Mrxbox98
Mrxbox9810mo ago
So I should us the database_url for build then the private for runtime?
Brody
Brody10mo ago
correct, or depending on what you are doing with the database during build, you can move it to runtime before your app starts (but use a health check so that railway knows when your app is ready to handle traffic)
Mrxbox98
Mrxbox9810mo ago
Ty, also will using my private url reduce the egress charge from my database if the query is from a deployment in the same project?
Brody
Brody10mo ago
if you only ever use the private url to connect to the database, there will be zero egress costs for database <--> service data transfer
Mrxbox98
Mrxbox9810mo ago
ty
Mrxbox98
Mrxbox989mo ago
Im getting this issue even when I have a 10 second sleep in the start of my app
No description
Brody
Brody9mo ago
are you using alpine?
Mrxbox98
Mrxbox989mo ago
Not sure, I dont use a docker container im just using the default nodejs
Brody
Brody9mo ago
you dont use a dockerfile?
Mrxbox98
Mrxbox989mo ago
No i just use the nixpack thing
Brody
Brody9mo ago
show me how you have implemented the sleep
Mrxbox98
Mrxbox989mo ago
I use sveltekit so I put a 10 second sleep in the server.hooks.ts. I tested in a dev env to make sure it actually delays app startup.
Brody
Brody9mo ago
did you await the sleep?
Mrxbox98
Mrxbox989mo ago
No description
Mrxbox98
Mrxbox989mo ago
yep
Brody
Brody9mo ago
can you add the sleep to your start command instead also, the database is in the same project right?
Mrxbox98
Mrxbox989mo ago
Yep would it be something like sleep 10 && npm start
Brody
Brody9mo ago
yeah
Mrxbox98
Mrxbox989mo ago
ok ill try that
Mrxbox98
Mrxbox989mo ago
ill check in my postgres logs to see if there are any errors there i can match up
2024-02-07 20:00:20.408 UTC [13553] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13552] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13548] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13546] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13551] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13549] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13547] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13550] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13447] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13544] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13543] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13556] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13558] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13560] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13557] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13545] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13540] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13555] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13542] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13541] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13448] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13444] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13564] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13561] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13539] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13559] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13563] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13562] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.414 UTC [13554] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.416 UTC [13565] LOG: could not receive data from client: Connection reset by peer
2024-02-07 20:00:20.408 UTC [13553] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13552] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13548] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13546] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13551] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13549] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13547] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.408 UTC [13550] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13447] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13544] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13543] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13556] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13558] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13560] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13557] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13545] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13540] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13555] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13542] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13541] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.410 UTC [13448] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13444] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13564] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13561] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13539] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13559] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13563] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.411 UTC [13562] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.414 UTC [13554] LOG: could not receive data from client: Connection reset by peer

2024-02-07 20:00:20.416 UTC [13565] LOG: could not receive data from client: Connection reset by peer
Im getting that in my postgres logs a lot
Limas
Limas9mo ago
Tagging on this I had a really similar issue yesteday what worked for me was changing postgres.railway.internal to just postgres as the hostname.
Mrxbox98
Mrxbox989mo ago
Ty, ill try to see if that works Quick question, did you hardcode the variable in? I use prisma so im unable to do a replace
Limas
Limas9mo ago
You should have something like this in your prisma schema file, the connection string should come from a env variable it shouldn't be hard coded, from there you should have your reference variable from the database service something like: REDIS_URL=${{Redis.REDIS_URL}} And you can modify is as you wish or you could just grab it directly and put it there.
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
}
Mrxbox98
Mrxbox989mo ago
If I do that though the url ends up having postgres.railway.internal in it
Mrxbox98
Mrxbox989mo ago
Still facing the issue (sorry I’m on phone)
No description
Brody
Brody9mo ago
can you connect to the database publicly?
Mrxbox98
Mrxbox989mo ago
Ill try the public url and see if that fixes it Havent faced the issue yet with the public url but Ill give it a day to check. I would rather use the private url though to reduce egress costs Been about 8 hours and havent gotten any issues
Adam
Adam9mo ago
@Brody ^ for visibility
Mrxbox98
Mrxbox989mo ago
Still no issues, hopefully this will get fixed soon so I won't go broke from egress charges
Mrxbox98
Mrxbox989mo ago
I removed some lint warnings since it showed some irrelevant source code
Brody
Brody9mo ago
now deploy logs please
Mrxbox98
Mrxbox989mo ago
From when I use the private url or the public one?
Brody
Brody9mo ago
private
Brody
Brody9mo ago
why run install twice?
No description
Mrxbox98
Mrxbox989mo ago
didnt realize that, ill remove it
Brody
Brody9mo ago
can you show me your code of the postgres client?
Mrxbox98
Mrxbox989mo ago
I use prisma
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
directUrl = env("DIRECT_DATABASE_URL")
}
datasource db {
provider = "postgresql"
url = env("DATABASE_URL")
directUrl = env("DIRECT_DATABASE_URL")
}
Brody
Brody9mo ago
whats the difference between those two urls?
Mrxbox98
Mrxbox989mo ago
I use the public url for directUrl because thats what the prisma migrate uses
Brody
Brody9mo ago
gotcha, good to know what is DATABASE_URL set to on your service?
Mrxbox98
Mrxbox989mo ago
DATABASE_URL is the private url in this deployment
Brody
Brody9mo ago
did you hardcode it?
Mrxbox98
Mrxbox989mo ago
yes
Brody
Brody9mo ago
please use reference variables
Mrxbox98
Mrxbox989mo ago
But I can undo the hardcoding if needed I can send a deployment logs of a deployment where it used reference variables
Mrxbox98
Mrxbox989mo ago
That is where it used reference variables
Brody
Brody9mo ago
show me the reference variable you used for database_url please
Brody
Brody9mo ago
deploy logs for the database please
Brody
Brody9mo ago
have you tried redeploying the database
Mrxbox98
Mrxbox989mo ago
ill try Same issue on redeployment, I can send latest logs if you wish
Brody
Brody9mo ago
would you be open to adding me to your project?
Mrxbox98
Mrxbox989mo ago
Sure Whats your github?
Brody
Brody9mo ago
im talking railway project
Mrxbox98
Mrxbox989mo ago
How do I add?
Brody
Brody9mo ago
from the project settings fair warning i would be able to see your variables since id need to be an editor to be of any help
Mrxbox98
Mrxbox989mo ago
No worries, what email should I invite?
Mrxbox98
Mrxbox989mo ago
invited
Brody
Brody9mo ago
one min so what do we have going on, are these services the same?
Mrxbox98
Mrxbox989mo ago
1 service is a discord bot that uses hte postgres data but that hasnt ecountered any issues the other is the web application that is facing problems
Brody
Brody9mo ago
both look like they have lots of errors
Mrxbox98
Mrxbox989mo ago
The discord bot has a lot of errors but none are the postgres connection error
Brody
Brody9mo ago
so i can ignore the bot
Mrxbox98
Mrxbox989mo ago
Yeah
Brody
Brody9mo ago
i may have asked already but whats the tech stack of the site? prisma and?
Mrxbox98
Mrxbox989mo ago
sveltekit and prisma Maybe the issue occurs after the web server crahes once due to an unrelated reason then on the automatic restart the issues start occuring
Brody
Brody9mo ago
I'd recommend moving the sleep into the start script
Mrxbox98
Mrxbox989mo ago
I had done that before (via the hooks.ts) method but it also didn’t work https://discord.com/channels/713503345364697088/1136319185882984598 this person seems to have had the same issue
Brody
Brody9mo ago
please add it to your start script
Brody
Brody9mo ago
start script please, that's the start command
Mrxbox98
Mrxbox989mo ago
Ok added
Brody
Brody9mo ago
stop your app from crashing 🤣
Mrxbox98
Mrxbox989mo ago
:legostare: idk why the internal network dns goes down if my deployment crashes then automatically restarts
Brody
Brody9mo ago
i don't know either, but.. I'm not joking
Mrxbox98
Mrxbox989mo ago
ik its a working temporary solution, but i hope railway team fixes the issue soon because i dont want my entire site to go down until i redeploy if something else causes a exit & reboot
Brody
Brody9mo ago
I wouldn't go pinning this on railway just yet
Mrxbox98
Mrxbox989mo ago
it worked well with the public url and i think the only difference in the private one is the dns
Brody
Brody9mo ago
heres a log of my database test service crash looping, i have modified the code to exit with an error and have railway restart on crashes, the deployment now sits in a crashed state yet no tests failed
Brody
Brody9mo ago
so yes while having to put the 3 second sleep there in the first place is an issue with railway (and will be fixed) with the 3 second sleep there is no issues, the issue would likely lie in your code
Mrxbox98
Mrxbox989mo ago
Maybe since my application is crashing the db connection is being unexpectedly closed causing something wierd to happen?
Mrxbox98
Mrxbox989mo ago
because it looks like the deployment is resolving the dns properly
Brody
Brody9mo ago
yep so where back to this, railway may be able to fix the need for the sleep, but doesn't help if your app is misbehaving, only you can fix your code
Mrxbox98
Mrxbox989mo ago
wierd that it only happens on the private url tho. wouldnt the private and public basically act the same since the dns resolved properly
Brody
Brody9mo ago
I'm sorry I have no clue what is happening with your code, but I can't reduce this, my code closes connections before exiting with a non zero code
Mrxbox98
Mrxbox989mo ago
No worries, ill try to debug by changing aroud some prisma settings etc.. I was able to fix the initial crash. Basically I used a library that created a connection on an api call to an external server, but this connection was never closed. At one point the sockets run out so the nodejs runtime crashes on a fetch request. I still can't be 100% sure why the postgres wasn't able to connection because of this, maybe because even after the node proccess restarted the sockets were still used.
Brody
Brody9mo ago
awsome!
Want results from more Discord servers?
Add your server