Issues are still ongoing, appears to affect Digitalocean, Vultr and other providers hosting at Equin

Issues are still ongoing, appears to affect Digitalocean, Vultr and other providers hosting at Equinix AM3 routed via AMS-IX. I have asked the others to report the issue here: https://community.cloudflare.com/t/increased-latency-from-equinix-to-cloudflare-via-ams-ix/549391

Sadly haven't received a reply from support yet, once the others replied, I hope it can be escalated

Cloudflare Community

Increased latency from Equinix to Cloudflare via AMS-IX

Hello, Myself and a couple of other Cloudflare users, hosting services at Digitalocean, Vultr and other parties using Equinix (AM3?) in Amsterdam, NL, are seeing elevated and fluctuating response times to Cloudflare services (proxied websites, Pages projects etc.) routed via AMS-IX. These issues started last night for all of the people I am in...

BBjarn Issues are still ongoing, appears to affect Digitalocean, Vultr and other provid...

zegevlier•8/26/23, 1:10 PM

I just tried spinning up a VM on Vultr in the Amsterdam region, and I can't reproduce this. Once the pages project was in cache, all requests took <0.5s. I also tried a site that's just proxied, and files hosted on R2, and they all have normal response times

zegevlier•8/26/23, 1:11 PM

Can you perhaps share a URL that you're seeing this on? I was using curlcurl for testing, do you suggest another method?

zegevlier•8/26/23, 1:23 PM

CF caches things near the visitors

Zzegevlier I just tried spinning up a VM on Vultr in the Amsterdam region, and I can't repr...

BjarnOP•8/26/23, 1:28 PM

It happens 1-2x times per hour on average, then everything is fine. It's very hard to reproduce.

I'll put them in a thread, one sec

BjarnOP•8/26/23, 2:24 PM

Also saw increased errors to R2 (failing/slow requests) from my home ISP. So I don't think it's only isolated to Equinix. I'm from NL tho - so maybe the same route?

Andy•8/26/23, 3:37 PM

wow i did a domain transfer from namecheap and it was FAST

Andy•8/26/23, 3:38 PM

namecheap sent a confirmation email and i approved and it was done less then a minute later

Original message was deleted

Chaika•8/26/23, 8:20 PM

yea it looks like it was just really short, that survey banner was only from August 9th to August 11th, I thought they'd keep it around longer

Bbrad Anyone else having issues logging into the dashboard at the moment?

Rhaegal•8/27/23, 12:33 AM

https://discord.com/channels/595317990191398933/895794943182909470/1145153750013595729

Andy•8/27/23, 2:15 AM

huh nice 404 on the dashboard home page

dax•8/27/23, 3:08 AM

According to https://developers.cloudflare.com/fundamentals/api/reference/limits/
Cloudflare API is 1200 requests per five minutesCloudflare API is 1200 requests per five minutes I'm using cloudflare images. Does this limit apply to images as well?

If so - does that mean that our Application built on cloudflare images can only serve on average 4 qps?

Dubz•8/27/23, 3:09 AM

Hello. Having some issues with tunnels/WARP. I have a tunnel running on two RasPis at home in the same subnet as my UPSes. However I cannot access the web GUI of the UPSes when I connect to WARP on my phone.
I can, however, access the UPS web interface via public hostname (so I know it's not blocked from RasPi to UPS) as well as other network devices on other subnets. I can also access the GUI from my desktop, on a third subnet from the others. Just can't through WARP for some reason. Browser says connection refused when I try. Not sure what else I can check here...

Dubz•8/27/23, 3:23 AM

Also trying to update cloudflared but repos saying no updates sooo

Ddax According to https://developers.cloudflare.com/fundamentals/api/reference/limits...

Cyb3r-Jak3•8/27/23, 3:27 AM

I believe the API is only used for uploads and not serving images

CCyb3r-Jak3 I believe the API is only used for uploads and not serving images

dax•8/27/23, 3:29 AM

I'm hoping that's the case :/ Although there's also the scaling concern on uploads on peak hours. 1200 requests per 5 minutes isn't a lot. Not to mention this limit being in the same bucket as all other Cloudflare API calls

dax•8/27/23, 3:32 AM

I see a previous post saying if you upgrade to enterprise (spend 2k+ a month), you can raise this to 9999 per 5 minutes, which is still very low for modern day applications with >50K monthly active users

dax•8/27/23, 4:06 AM

Created a post on the forums in case anyone has a similar question later on: https://community.cloudflare.com/t/cloudflare-global-rate-limit-concerns/549583

Seems a bit silly if user generated traffic (uploads) can prevent us from even touching our data plane since they share the same API limits.

Unsmart•8/27/23, 4:11 AM

You can technically use multiple users each with their own auth tokens to increase the limit that way but yeah generally avoiding the API is best. Using R2 to store and serve images would probably be better

dax•8/27/23, 4:12 AM

Does r2 uploads get aggregated into a separate rate limit?

Unsmart•8/27/23, 4:13 AM

theres no rate limits for r2 uploads

dax•8/27/23, 4:13 AM

multiple users each with their own auth tokensmultiple users each with their own auth tokens

Ah I thought the 1200 req per 5 minutes was global for the account

dax•8/27/23, 4:13 AM

so it's per/user?

Unsmart•8/27/23, 4:13 AM

I mean at least in theory theres no rate limit on r2 eventually you might hit bucket scaling problems but

Unsmart•8/27/23, 4:14 AM

Its per cloudflare user. API tokens are tied to a specific cloudflare user and each user has their own limit

dax•8/27/23, 4:14 AM

Interesting. We might just shard our workers into using a pool of accounts upon startup to overcome that limit lol

Unsmart•8/27/23, 4:14 AM

you can invite as many users to your cf account as you want not sure if they have any limits on that (if they do it should be pretty high afaik)

dax•8/27/23, 4:14 AM

good to know - thanks!

Daniel Sincere•8/27/23, 4:18 AM

If I have a CNAMECNAME pointing to a Cloudflare tunnel (Tunnel A), when I update the CNAMECNAME to point to a second Cloudflare tunnel (Tunnel B), how long will it take for traffic to go to Tunnel B?

Daniel Sincere•8/27/23, 4:19 AM

Or to ask the question a different way, after updating the CNAMECNAME to point to Tunnel B, how soon can I delete Tunnel A?

Daniel Sincere•8/27/23, 4:20 AM

I hope it's fairly quickly - less than a minute?

DDaniel Sincere If I have a `CNAME` pointing to a Cloudflare tunnel (Tunnel A), when I update th...

Dubz•8/27/23, 4:30 AM

I think a minute is Cloudflare's TTL for DNS, but it may be different (faster) for internal routing to update. I wouldn't rely on it heavily for automation purposes, but for an outage/recovery or migration, should go unnoticed by most.

Dubz•8/27/23, 4:31 AM

TTL for public DNS I mean, not internal routing

Daniel Sincere•8/27/23, 4:31 AM

yeah, doing a migration -

DDubz TTL for public DNS I mean, not internal routing

Daniel Sincere•8/27/23, 4:31 AM

not sure how to parse your first message in light of this correction?

Dubz•8/27/23, 4:32 AM

what?

Dubz•8/27/23, 4:33 AM

migration, you shouldn't really notice. It should be maybe a minute tops to propogate if I had to guess. In other words, unless you're doing this via automation, it'll probably be fast enough to stay ahead of you (unless you're trying to speed through it for some reason).

Dubz•8/27/23, 4:34 AM

Like a minute is not long at all

Daniel Sincere•8/27/23, 4:34 AM

ah gotcha

Daniel Sincere•8/27/23, 4:34 AM

thanks so much @Dubz (pls ping/reply) !

DDubz I think a minute is Cloudflare's TTL for DNS, but it may be different (faster) f...

Daniel Sincere•8/27/23, 5:32 AM

Can a Cloudflare tunnel have multiple endpoints? Can I run cloudflaredcloudflared on two separate clusters?

DDaniel Sincere Can a Cloudflare tunnel have multiple endpoints? Can I run `cloudflared` on two ...

Dubz•8/27/23, 5:35 AM

Yes, that's what you're supposed to do. redundancy

DDubz Yes, that's what you're supposed to do. redundancy

Daniel Sincere•8/27/23, 5:36 AM

Oh, so that might be another way to migrate traffic from one cluster to another!

Dubz•8/27/23, 5:37 AM

if the tunnel host is changing absolutely. If the target is changing, not really how that works

Dubz•8/27/23, 5:38 AM

it's for load balancing/redundancy basically. So if one tunnel breaks, the other can still serve. The service itself is either ran locally on that machine (like a CDN maybe) or all points to one main server they can access across either

DDubz it's for load balancing/redundancy basically. So if one tunnel breaks, the other...

Daniel Sincere•8/27/23, 5:51 AM

if one tunnel breaks

Just want to make sure I understand

In Cloudflare DNS, I'll have CNAME api.example.com dead-beef.cfargotunnel.comCNAME api.example.com dead-beef.cfargotunnel.com and then I'll have cloudflaredcloudflared running on cluster 1, and cloudflaredcloudflared running on cluster 2, using the same tunnel credentials. So the tunnel is load-balanced between cluster 1 and cluster 2?

Dubz•8/27/23, 5:56 AM

yes...

Issues are still ongoing, appears to affect Digitalocean, Vultr and other providers hosting at Equin

Similar Threads

Similar Threads

Similar Threads