All services down in Singapore

Hi team, I've messages my business class team but just for more visibility, our databases are down across all our projects by the looks of things. They've crashed and don't appear to be up and running yet (it's been 17 mins) Project 3e218ec0-7c31-4a5e-ba63-d0f7439aa07d Service 8ec6131b-e177-40ec-824e-9fa9f3960add
8 Replies
Percy
Percy8mo ago
Project ID: 3e218ec0-7c31-4a5e-ba63-d0f7439aa07d,8ec6131b-e177-40ec-824e-9fa9f3960add
nickmacavoy
nickmacavoy8mo ago
One project is live as of right now, checking the others
nickmacavoy
nickmacavoy8mo ago
A single host in asian-southeast1 encountered a hardware failure. W...
A single host in asian-southeast1 encountered a hardware failure. We've failed over to a new host. - Incident details - Railway Status
Brody
Brody8mo ago
please go ahead and redeploy your crashed postgres databases
nickmacavoy
nickmacavoy8mo ago
Hey Brody. Thanks. They redeployed on their own, but the services that relies on them have had to be redeploying to get everything working. Restarting didn't seem to work I'm bringing them up one by one
Brody
Brody8mo ago
let me know if you run into issues during that process
nickmacavoy
nickmacavoy8mo ago
Thanks Brody. All looking good now. This can be closed. Thanks team for the fairly quick resolution – 18m 28s downtime in the end at a very fortunate time of the day for our business
Brody
Brody8mo ago
sorry for the inconvenience, hopefully the team can implement a fix so that this particular error doesn't happen again
Want results from more Discord servers?
Add your server