RTT to nearby datacenter has gone way up in the past few days

IWIan Woodfill9/17/2023
Regions have been amazing and are super helpful for our use case. We host our database in AWS us-east-1, which is in the Washington DC area. We've moved our Railway servers to GCP us-east-4 as that is in the same local area. I was experimenting with this last week and was getting single digit to low double digit millisecond RTT/latency from Railway to AWS. This weekend I redeployed the same service and found that the latency has gone back up to >70 ms round-trip time.

I'm scratching my head here, because when I can double check the IP of my Railway service and AWS database and see they are both in the Washington DC area, yet this latency is really bad for that proximity.

My only thought is perhaps in your development work you all have added some configuration that adds some sort of an additional layer to this networking that adds additional latency.

What could be going on here?
IWIan Woodfill9/17/2023
These datacenters should have very low latency between them: https://medium.com/@sachinkagarwal/public-cloud-inter-region-network-latency-as-heat-maps-134e22a5ff19
For context, my local instance has RTT of ~20ms, and I'm in NY. Theres definitely some funky networking going on here, given gcp-us-east4 (railway) <-> aws-us-east-1 is over 3x slower than NY <-> aws-is-east-1, and given that last week it was very low latency
actually, resolving the IP address for my services shows that they're still in us-west, despite having RAILWAY_REGION=us-east4
oh, it seems like RAILWAY_REGION does not work utilizing a shared variable!
This is resolved
for future reference, the region in use is printed in the first line in the deployment logs
IWIan Woodfill9/17/2023
yes it is! good to know
IWIan Woodfill9/17/2023
do you know how volumes work with regions? what happens if you change the region of a service that has a volume?
while your question is a little different than the wording used in the docs, I think they are similar enough to not risk it without first testing it in a test project
IWIan Woodfill9/17/2023
I'd also be interested to know the results of that test!
We’re discussing this next week 🙂
Right now they don’t work. If you set a region on a svc with volume attached, it’ll be stuck at the deploy phase
Primarily because your volume can be on a different box than what it was on when switching regions

Looking for more? Join the community!

Recommended Posts
Publishing .net 7 app - project file not foundHi, I'm new in railway and i'm faced with file structure problem. I heared that way to publish is sIssue with staging: The value needs to be expressed as a valid JSON string.i'm trying to make a dev and product env's, but now i created a service on the dev env and trying toIssue with staging: The value needs to be expressed as a valid JSON string.i'm trying to make a dev and product env's, but now i created a service on the dev env and trying toUse Volume with grafana will permission deniedI was configed volume and data path, but when i start my grafana, it will crash. This is log: Error:if i disconect an account of github ?whats going on with the repository are linking with this account ? fb928ae2-62f0-4f8f-be68-9f7902efCommunication between projects without requestsHey everyone, a question. Is it possible to establish communication between projects? For example, Ihow to setup webhooks?Hey guys I'm looking for how to setup a deploy webhook similar to vercel for my website with sanity.Template services in forked environment not deployingHi! When I create a new environment from the base one the two services, that I created from a templaHow can I update the file permissions inside a railway development instance?I receive this error: "PermissionError: [Errno 13] Permission denied: './src/engines/komodo-14/LinudockerIs it possible to use commands or access through a terminal a docker image? I have a Docker image ofDeployment failingI am trying to deploy some updates to my app and I am getting errors during the build process. DockPayload CMS app 500 errorURL: https://edit.gravybo.at/ ID: 9c654a5e-96b8-42bd-a2a0-d82a9e0b8d11 App builds/runs locally, howConfiguring Redis TemplateHi, I was wondering how we can configure the new Redis from the templates. In particular setting theNew deployments maxing out RAM?I have no idea why this started happening, but any new deployments made for one of my services now mPlease help, we are down during a live eventI am not sure what is wrong, everything was working fine but now we cannot access the site. I lookedError installing Python Packages via requirements.txt inside Docker buildProject ID: 1b6776e2-c07c-4f4f-8d54-cd1c9aee3de0 I am trying to deploy an application via DockerHow can I set usage limits?Hello! Following the latest changelog, I wanted to configure usage limits and reading the documentatWhat Postgres Version is railway usingWhat Postgres Version is railway usingcredit missingHello, before the railway plan rework, I purchased $5 credits. Now after the rework, I only have theSomebody got a working PostHog Template?The current template in the marketplace is not deploying: https://railway.app/template/16rDub