I’m having trouble connecting to a website that’s blocking me with a 1xxx error for website protection. Is there a reason I am unable to connect to it?
You have to contact the website owner, it may be a Cloudflare block message, but it’s at the behest of the website owner and therefore you’ll always need to speak with them
Hiya, We are creating our caching system at the moment, I'm wondering, is there a way to tell cloudflare; "go fetch all the urls in this path" in order for to cache it prior to any user visiting it? If so, would it work for updates? I don't mind creating a microservice that does it, just wondering if anyone can lead me the right direction for that and if there's any API support for that
There isn't, and I would keep in mind too Cloudflare cache by default is heavily partitioned, there's one cache per colo/datacenter. That would be a lot of requests. Enterprise has something called Prefetch URLs (https://developers.cloudflare.com/speed/optimization/content/prefetch-urls/), which uses link response headers with prefetch that it'll bring into cache if responded with. (i.e user requests document html, it responds with link headers for assets, CF pre-fetches those before the user does)
You have Tiered Cache as well which for free you get the Smart Topology, which selects a CF location closest to your origin to check for cache, i.e User -> Local CF Datacenter/colo -> Smart Tiered Caching Selected -> Real Origin. If your goal is high cache % at the cost of some latency and you're not using an anycast service/something that lives in more then one location, Smart Tiered Caching can help.
(Edit: Enterprise has other topologies for tiered caching if you are ENT, that can work with even anycast origins in multiple locations and a middle tier option to further increase cache % with less latency)
As for APIs there is a Purge API: https://developers.cloudflare.com/api/operations/zone-purge#purge-cached-content-by-url You could have really long Cache TTL (keeping in mind it's a maximum and not guaranteed to be in cache for that long), and then call the purge API when updating content to try to increase cache % with still getting instant updates
So lets try it with an example. Our servers are based at NVirginia (nothing new :)). 1. We have around 4,000 static pages that barely (if ever) change. 2. We have our main page which gets exposure to many users and as there are more cache hits it increases cache time
I want to update my cache on my main page every 3mins for example (We are using the workers method to serve cached content with real-time fetched data). Which means every 3 mins I want to kind of "push" and update for the cached data.
With the static pages, lets say one of my writers makes an update, I want to purge it obviously, but then I want to create a new cache that sets a proper
age
age
for it, so incase google comes and tries to check it, it will be fetched from cache, and not access the server.
Hopefully this can provide more info on what I'm looking for, I would say google/client would be the same, since I want it to be as fast as possible obviously. Wouldn't it be possible to send a list of URLs to get cached? Like 4K URLs, and set a time (1 day) for example for caching. Does age even play a role here?
Send where? There's no API you can send items to for Cloudflare to fetch & cache for you. You could do a local fetch, but keeping in mind what I said above, without tiered cache on that would just populate the local cf datacenter cache. There's no way to fill up all of the caches globally without having requests to all of those
There is a min. edge cache ttl too. https://developers.cloudflare.com/cache/how-to/edge-browser-cache-ttl/ For Free, 2 hours, Pro, 1 hour, Biz and Ent, 1 second. Anything less then that just wouldn't be cached at edge. You cache it for longer though (let's say 24 hours) and call the cache purge api at any time though, might be easier to control it that way if you want to adjust how long it'll be cached for, like with the main page
There are other potential options as well, could leverage KV and local cache in a Worker, could be kind of pricey though. Which plan are you on? There's a feedback loop there where the more requests you get, the more places your assets will be in cache, the lower latency, etc, but can't force it. Cloudflare wants only frequent assets in cache, there's a finite amount of space
As I was saying above Cache is per colo/datacenter. Cloudflare has points of presence (PoPs), identified by airport codes like ORD (Chicago), EWR (New York). Some of those PoPs are multi-colo (MCPs), and each with their own cache. There's some 200-300 PoPs or something and around 550 colos. If there was a global cache pull api that would be a ton of requests and 550 copies of each asset lol, that's why there are tools like tiered caching and such. Generally speaking though your visitors are not going to be that evenly spread, just going to be a few popular points most likely
You can use the CDN-Cache-Control response header to control Cloudflare's cache ttl and such, and then Cache-Control would just be for the browser: https://developers.cloudflare.com/cache/concepts/cdn-cache-control/ ex, cache main page and all other pages for 24 hours, purge via API whenever needed. Cloudflare has an extension for Wordpress (APO) which does exactly that, not a bad pattern
Analytics and lots of data Could move to off-topic to talk about that Cloudflare also uses Clickhouse for all of its analytics, traffic graphs, Cloudflare Radar and such, it's really powerful
Is there an HTTP API that lets me clear the cache for a specific route?
My content for this endpoint changes randomly - sometimes once a week, sometimes multiple times per day. I want to clear the cache for that page after my site re-deploys
if you have Always use HTTPS it would redirect everyone first/you could just assume everything -> https. If you needed to keep it, you could just have two rules, one matching ssl and another without
yea I don't think there's any easier way for that, if it was relative (i.e /contact-us/ to /contacts/), you could just have it be static and relative, but if it's to a different domain I think you're forced to put scheme.
Is https://blog.cloudflare.com/secrets-store in beta yet, or any news on it? As the post was 7 months ago, and I've not seen anything mentioned since (but maybe I missed news)
Introducing Secrets Store by Cloudflare - the ultimate solution for managing your application secrets securely. Safeguard sensitive information, track access, and maintain ease of use.