Error: connect ETIMEDOUT/ECONNREFUSED on all db writes
Project ID: 316aa21f-dc55-4e9f-be8f-13129d27d7c0
I'm using the legacy postgres. About 2 hours ago, all db write calls from my application started hanging and resulting in an ETIMEDOUT error.
It's now resulting in a ECONNREFUSED error.
I have tried redeploying an older version of my application, with no difference.
If I build and run the app locally pointed at the prod db, it works normally -- so I believe the issue is with the deployed railway service.
current error:
9 Replies
Project ID:
316aa21f-dc55-4e9f-be8f-13129d27d7c0
#🚨|incidents - possibly related, looking into it.
So it appears that your production Redis in on Stacker 50.
Is the data on Redis requiring persistence, or do you have a depency on that Redis?
(I would also migrate your Postgres DB while you can.)
redis data does require persistence
I will migrate postgres before the cutoff, but can't do it at this moment because of the associated downtime :/
No worries here, can you check if the workload is back online?
Solution
We just restored service for the Redis.
Migration doesn't kill the old service until the new one is healthy.
@Angelo yes, it's working again. Thank you for your help!
Unfortunately I have been recently informed that we stopped the services while the handover is going on, but I would encourage you to do it sooner vs. later, V2 Databases weren't affected by this outage.
noted, we will prioritize it ASAP. ty again for the help