The internet is littered with bots and your pages.dev domain is public, and things like DNS are too,
The internet is littered with bots and your pages.dev domain is public, and things like DNS are too, so bots end up finding it
Cthat's scary..how do they find it, is cloudflare publicizing it?
Cor is there a mouse in the house and cloudflare dev ops need to wake up
Swelcome to the internet.. its normal
AThis is just how the internet works, anything public can be found and bots are everywhere - it’s nothing to worry about though, assuming what you have there isn’t private data or anything
SWhat im more confused about now is how i pass the ctx.waitUntil to a function 
i ran into the classic
do i really need apply/call in that situtation?
i ran into the classic
TypeError: Illegal invocation: function called with incorrect this reference first. now when i pass down the entire platform into the function waitUntil ends up undefined (its available where im trying to pass it to the function)do i really need apply/call in that situtation?
CEvery pages.dev gets a certificate, certificates are all logged as part of Certificate Transparency (CT) Logs the second they are issued, bots listen to CT Logs
Cthat seems to be a security risk the platform in users...why is that public?
IIt's just how the internet works. It's not specific to Cloudflare, every platform works this way.
IAny time a certificate is issued, anywhere (that is trusted by browsers), it's logged publicly.
Copposite, Certificates Issuance are public to increase security. So no Certificate Authority can issue Certificates without it being known
I
Cokay so...would it not be smarter to set these tempory pages.dev things via: page.dev/subfolder instead a a subdomain?
HSome projects deployed to Pages may expect a (sub)domain to themselves
IThe issue with this is that they have no way to know what you have on the production domain.
IIf you put it at
/foo then what if I have a file/folder in prod that's called /foo?IIt would overwrite it.
CNo, for security again, as pages.dev uses subdomains and is part of the public suffix list, browsers treat each subdomain as its own website, separated in terms of cookies, cors, etc
CIf you were using subpaths, browsers would assume each pages website are part of the same security context/get around some limits of CORS, allow access to cookies, etc
Coh, even if you mean just for preview URLs it'd still be more way messy for the reasons Isaac said
Cthis is all true, but also true is that these bots are a problem. perhaps on this domain cloudflare could think of a way to be more pro-active against the bots and honey pot them to immediately get them on a block list. Or maybe they are and this is just one that got through
IEven for preview URLs this would apply ^
IImagine if you set a cookie on the preview path to log a user in, and then it applies on the prod path too
CBots just aren't really an issue, and it's very difficult to block only the bad ones without blocking intentional ones
IYou can always put Access in front if you're worried about security.
Cif some ip keeps making requests for non existant files accross multiple sub domains...say 20 times..or some threshold...thats not a signal that could be automated to trigger it as a bad actor?
Cor it could just be a crawler following bad links?
Cthis would be specific to pages.dev
Cdoesn't change that
Cpage.dev isn't meant for seo crawling scenarios
Cor to be indexed
CStatic Requests are super cheap to Cloudflare and get even cheaper the more cache gets hit
IHow do you propose tracking that, though? Imagine if 32 users (Starlink) share an IP, and they hit 404s. If you block that IP you might be blocking 1 bad user but 31 good users who hit a 404.
CIt'd most likely be more compute intensive to try to store/do all that, then just serve the static requests
Ccool down period?
CDoing all that across tons of machines is a ton of cost of sync, and a ton of code to go wrong, etc
Cat some point workers-and-pages-discussions
CAny anti abuse stuff like that can go wrong too and block legit use cases, just not worth it
Cbecause cloudflare has so much traffic going through it..at some point its the only answer to crush the bot delema that soaks up so much computer and bandwith on the internet
CDenial of service attacks only exists because there is no communication or effort amongst hosts to do anything
Conly cloudlfare has the ability to centralize an effort
CBandwidth is pretty cheap to CF in most cases with their vast network, and compute is more expensive to do anti-bot stuff across thousands of machines then to just serve static assets which can be cached everywhere
Cof my VPS servers that run behind cloudflare, 80% of the cpu is just bots going for non existant files
Cand occassionally i'll drop in, block an ASN and then it will be another ASN 1 month later
ISure but at that point it's very user-specific and Cloudflare wouldn't take action to block things on your own domain when it might be blocking things for legitimate users.
Csending any abuse type email is generally never read
IIf you want to reduce that kind of thing you can deploy your own rules that are tailored to your specific domain
II.e rate limiting rules.
Ci mean i do it for my domains already, i'm suggesting something just for pages.dev, but also my 2ndary conversation is the concept of cloudflare being the hub to lower bot traffic in the world in some way. It would be a form of community service in line with how it pushed back against patent trolls.
Cjust an idea
Canyways...carry on
