R
Railway•5mo ago
stephan

Cloudflare timeouts some times

Hi there, I have a project with muliple services, sometime cloudlfare cant connect ot railway, I have the SSL mode is set to Full, everything is working fine, it's just that sometimes the server does not respond. There are 8 instances active right now and it's still happening e0d0ed17-eedc-4575-8b89-1b314af7f950
49 Replies
Percy
Percy•5mo ago
Project ID: e0d0ed17-eedc-4575-8b89-1b314af7f950
Brody
Brody•5mo ago
there's an ongoing incident right now that's likely related to your issue #🚨|incidents
stephan
stephan•5mo ago
We have this issues since 2-3 days Also the Incident is for US-WEST, but we are on EU WEST
Brody
Brody•5mo ago
hmm theres no been no incidents for eu-west in the last few days, what error do you get from cloudflare?
stephan
stephan•5mo ago
No description
stephan
stephan•5mo ago
I just deployed to US East, to see if this error still shows
Brody
Brody•5mo ago
are there any errors in the deploy logs?
stephan
stephan•5mo ago
No errors at all
Brody
Brody•5mo ago
what kind of app is it?
stephan
stephan•5mo ago
Node + NestJS It's working fine on US East I guess but it's very weird because I have other services on that project, it's the same monorepo they are deployed in EU West and they are working fine
Brody
Brody•5mo ago
very odd, but no one else has reported timeout issues until today
stephan
stephan•5mo ago
I will leave it in US east for today, and tomorrow I will move it again to EU, working fine on US east
Brody
Brody•5mo ago
okay sounds good, keep me updated!
stephan
stephan•5mo ago
I still have the same issue @Brody I have tried everything. I created another service with the same configuration, if this doesn't fix it, I will have to move more than 15 services to another provider
stephan
stephan•5mo ago
No description
Brody
Brody•5mo ago
when you visit the cloudflare protected domain and get that timeout message, what would visiting the railway domain directly show you?
stephan
stephan•5mo ago
visiting the railway domain would not work, same as visiting my domain it just stays on loading
Brody
Brody•5mo ago
that would have to timeout at some point too right? when it does, what would you see?
stephan
stephan•5mo ago
it would respond after sometime, I havent seen a railway error
Brody
Brody•5mo ago
this is starting to look a lot like another users issue, they thought it was a railway outage, but it turned out they where getting ddos'ed and their app was soft locking
stephan
stephan•5mo ago
No description
Brody
Brody•5mo ago
right but you can still access the app through the railway provided domain either way, we can't rule anything out
stephan
stephan•5mo ago
No, I cant, it's the same thing as going through my domain; the ap will respond after 30s And then the app automatically becomes responsive again
Brody
Brody•5mo ago
sorry I meant that theoretically you could access it via the railway domain, and that could be where the ddos is getting through and not showing on cloudflare
stephan
stephan•5mo ago
I was using a easypanel server before and we had no isses at all, I moved all our services on Railway and this started happening I have removed the railway domain only using my domains
Brody
Brody•5mo ago
that technically doesn't close off access
stephan
stephan•5mo ago
But a ddos attack would use resources on Railway right?
Brody
Brody•5mo ago
did you say all 15 of your services are experiencing the same thing?
stephan
stephan•5mo ago
Noo, its only one
Brody
Brody•5mo ago
yeah, your memory is pretty high for a node/nest app right?
stephan
stephan•5mo ago
there are 4 instances active
Brody
Brody•5mo ago
replicas?
stephan
stephan•5mo ago
yes sorry
Brody
Brody•5mo ago
okay going to ask the team
thomas
thomas•5mo ago
I will be looking into this inside of an hour
stephan
stephan•5mo ago
@thomas were you able to check
Brody
Brody•5mo ago
friendly request, please dont ping the team #🛂|readme #5
thomas
thomas•5mo ago
Hey I am sorry, but we have a possible incident we are investigating that pushed back my estimate you are very much still on my list, could you post the service ID?
stephan
stephan•5mo ago
9e89d04a-cc90-42a3-ac7f-89b05dcd314e
stephan
stephan•5mo ago
@Brody I think you are right about the DDoS attack
No description
No description
thomas
thomas•5mo ago
just an idea, you can sometimes tell based on billing for egress unless cloudflair is protecting you sorry commenting without all the context yet
stephan
stephan•5mo ago
No description
Brody
Brody•5mo ago
you can always talk directly to the railway service with host header manipulation, bypassing cloudflare
stephan
stephan•5mo ago
what would that solve?
Brody
Brody•5mo ago
I'm talking about how you could be getting ddos'ed without cloudflare noticing what does your network metrics look like?
stephan
stephan•5mo ago
No description
stephan
stephan•5mo ago
All the metrics look normal to me I only noticed that on the logs
thomas
thomas•5mo ago
So this is a US east only issue? Can you explain that part? I see the service is in EU West at the moment. To be clear, are you still experiencing this in that region?
stephan
stephan•5mo ago
No this is happening on all regions