Getting upstream errors, ALL requests failing outside of US

Hi team, we're unfortunately very close to moving off Railway due to a barrage of (likely) K8s errors happening across all our services:
upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: immediate connect error: Cannot assign requested address
upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: immediate connect error: Cannot assign requested address
To make matters worse, our non-American customers aren't able to access our service at all, and we have no gates on our end that would cause that to happen. It's difficult for us because we have no observability on our end to detect why this is happening, and it's literally making us lose customers in realtime. 1. Can someone look into why this is happening across our services? (team name: Toma). 2. Can we get a dedicated server so this doesn't happen again? 3. If this is resolved, we expect our usage to increase dramatically. How can we get a dedicated customer support person in case shit hits the fan again?
No description
5 Replies
Percy
Percy10mo ago
Project ID: 2a4b7826-55df-465c-91e0-218d8e17a85f
Percy
Percy10mo ago
Project ID: 2a4b7826-55df-465c-91e0-218d8e17a85f
blandthony
blandthony10mo ago
2a4b7826-55df-465c-91e0-218d8e17a85f
kylegill (kg)
kylegill (kg)10mo ago
Hey @blandthony , for what it's worth, we've got others in another open thread (https://discord.com/channels/713503345364697088/1197584373227200703) with the same issue. Hoping to get things escalated. We suspect it's related to this incident: https://discord.com/channels/713503345364697088/846875565357006878/1197588980653375588
blandthony
blandthony10mo ago
Thanks @kylegill (kg) 🙏
Want results from more Discord servers?
Add your server