this is all true, but also true is that these bots are a problem. perhaps on this domain cloudflare

this is all true, but also true is that these bots are a problem. perhaps on this domain cloudflare could think of a way to be more pro-active against the bots and honey pot them to immediately get them on a block list. Or maybe they are and this is just one that got through

CChaika If you were using subpaths, browsers would assume each pages website are part of...

Isaac McFadyen•3/4/25, 7:06 PM

Even for preview URLs this would apply ^

Isaac McFadyen•3/4/25, 7:06 PM

Imagine if you set a cookie on the preview path to log a user in, and then it applies on the prod path too

Ccaptain this is all true, but also true is that these bots are a problem. perhaps on thi...

Chaika•3/4/25, 7:06 PM

Bots just aren't really an issue, and it's very difficult to block only the bad ones without blocking intentional ones

Isaac McFadyen•3/4/25, 7:06 PM

You can always put Access in front if you're worried about security.

CChaika Bots just aren't really an issue, and it's very difficult to block only the bad ...

captainOP•3/4/25, 7:08 PM

if some ip keeps making requests for non existant files accross multiple sub domains...say 20 times..or some threshold...thats not a signal that could be automated to trigger it as a bad actor?

Ccaptain if some ip keeps making requests for non existant files accross multiple sub dom...

Chaika•3/4/25, 7:08 PM

or it could just be a crawler following bad links?

CChaika or it could just be a crawler following bad links?

captainOP•3/4/25, 7:09 PM

this would be specific to pages.dev

Chaika•3/4/25, 7:09 PM

doesn't change that

captainOP•3/4/25, 7:09 PM

page.dev isn't meant for seo crawling scenarios

captainOP•3/4/25, 7:09 PM

or to be indexed

Chaika•3/4/25, 7:10 PM

Static Requests are super cheap to Cloudflare and get even cheaper the more cache gets hit

Ccaptain if some ip keeps making requests for non existant files accross multiple sub dom...

Isaac McFadyen•3/4/25, 7:10 PM

How do you propose tracking that, though? Imagine if 32 users (Starlink) share an IP, and they hit 404s. If you block that IP you might be blocking 1 bad user but 31 good users who hit a 404.

Chaika•3/4/25, 7:10 PM

It'd most likely be more compute intensive to try to store/do all that, then just serve the static requests

IIsaac McFadyen How do you propose tracking that, though? Imagine if 32 users (Starlink) share a...

captainOP•3/4/25, 7:10 PM

cool down period?

Chaika•3/4/25, 7:10 PM

Doing all that across tons of machines is a ton of cost of sync, and a ton of code to go wrong, etc

Chaika•3/4/25, 7:11 PM

at some point workers-and-pages-discussions

Chaika•3/4/25, 7:11 PM

Any anti abuse stuff like that can go wrong too and block legit use cases, just not worth it

captainOP•3/4/25, 7:11 PM

because cloudflare has so much traffic going through it..at some point its the only answer to crush the bot delema that soaks up so much computer and bandwith on the internet

captainOP•3/4/25, 7:12 PM

Denial of service attacks only exists because there is no communication or effort amongst hosts to do anything

captainOP•3/4/25, 7:13 PM

only cloudlfare has the ability to centralize an effort

Chaika•3/4/25, 7:13 PM

Bandwidth is pretty cheap to CF in most cases with their vast network, and compute is more expensive to do anti-bot stuff across thousands of machines then to just serve static assets which can be cached everywhere

captainOP•3/4/25, 7:13 PM

of my VPS servers that run behind cloudflare, 80% of the cpu is just bots going for non existant files

captainOP•3/4/25, 7:14 PM

and occassionally i'll drop in, block an ASN and then it will be another ASN 1 month later

Ccaptain of my VPS servers that run behind cloudflare, 80% of the cpu is just bots going ...

Isaac McFadyen•3/4/25, 7:14 PM

Sure but at that point it's very user-specific and Cloudflare wouldn't take action to block things on your own domain when it might be blocking things for legitimate users.

captainOP•3/4/25, 7:14 PM

sending any abuse type email is generally never read

Isaac McFadyen•3/4/25, 7:14 PM

If you want to reduce that kind of thing you can deploy your own rules that are tailored to your specific domain

Isaac McFadyen•3/4/25, 7:15 PM

I.e rate limiting rules.

captainOP•3/4/25, 7:16 PM

i mean i do it for my domains already, i'm suggesting something just for pages.dev, but also my 2ndary conversation is the concept of cloudflare being the hub to lower bot traffic in the world in some way. It would be a form of community service in line with how it pushed back against patent trolls.

captainOP•3/4/25, 7:16 PM

just an idea

captainOP•3/4/25, 7:17 PM

anyways...carry on

captainOP•3/4/25, 7:43 PM

now getting a ton of 404's for /sito/wp-includes/wlwmanifest.xml I HATE BOTS...just flooding me.

Silvan•3/4/25, 10:38 PM

i get thousand of bot requests a day :Shrug:

Silvan•3/4/25, 10:39 PM

we are smart and dont put credentials or anything on the client tho so we are fine

Silvan•3/4/25, 10:43 PM

god i love how many services cf offers i have to hold back to not implement everything for simple apps

but maybe rate limiting.. just 1 more

1984 Ford Laser•3/5/25, 12:16 AM

Is there somewhere I can track the changes to CF docs?

11984 Ford Laser Is there somewhere I can track the changes to CF docs?

Chaika•3/5/25, 12:17 AM

https://github.com/cloudflare/cloudflare-docs/commits/production/ ?

GitHub

Commits · cloudflare/cloudflare-docs

Cloudflare’s documentation. Contribute to cloudflare/cloudflare-docs development by creating an account on GitHub.

CChaika https://github.com/cloudflare/cloudflare-docs/commits/production/ ? 🤔

1984 Ford Laser•3/5/25, 12:17 AM

Thank you wasnt sure of the exact repo

1984 Ford Laser•3/5/25, 12:20 AM

This was the change I was trying to figure out manually

no wonder I couldnt find it lol

Silvan•3/5/25, 12:20 AM

i updated some lib and now i get that build error even tho im using

"compatibility_flags": ["nodejs_als"]

"compatibility_flags": ["nodejs_als"]

any clues why this would error now?

and yes

nodejs_compat

nodejs_compat

would fix this im curious tho

Silvan•3/5/25, 12:24 AM

could this be a change from

node:async_hooks

node:async_hooks

to just

async_hooks

async_hooks

kunal•3/5/25, 1:53 AM

Does hyperdrive cache queries that return 0 rows?

Kkunal Does hyperdrive cache queries that return 0 rows?

Mitya•3/5/25, 9:21 AM

#hyperdrive

Mitya•3/5/25, 9:25 AM

Am I right that

waitUntil()

waitUntil()

can be used to ensure an asynchronous process will complete even after a response is sent? So if my worker has this:

waitUntil(new Promise(res => setTimeout(() => {
    console.log('hi!');
    res();
}, 5000)));
return new Response('foo');

waitUntil(new Promise(res => setTimeout(() => {
    console.log('hi!');
    res();
}, 5000)));
return new Response('foo');

...will I see "hi!" in the console 5 seconds after the worker has sent a response? (And presumably I don't need to use

await

await

with the above, as that would defeat the point and delay the response?)

Murder Chicken•3/5/25, 2:33 PM

If I'm getting build errors for workers where the only error in the build log is "

Failed: error occurred while updating repository submodules

Failed: error occurred while updating repository submodules

" is there any way to get more information about the error? I've exhausted my ability to troubleshoot past this.

piffie•3/5/25, 2:47 PM

hmmm i have an issue that cloudflare/pages-action@v1 still uses node 20 - but it seems i need node 23 to deploy the latest nuxt to it. Is the pages-action outdated? or will there be an update for it?

Scott•3/5/25, 4:30 PM

Anyone know why there is suddenly a 1,000 worker route limit per account?

Scott•3/5/25, 4:30 PM

I have more than 1,000 but users are now reporting this error.

Scott•3/5/25, 4:31 PM

There is nothing documented about it, and I can't even tell if the Business plan ups the limit.

SuperHelpflare•3/5/25, 4:38 PM

Please do not post your question in multiple channels/post it multiple times per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

theodm•3/5/25, 5:24 PM

Hi, does anyone knows how CPU is shared between isolates (workers) ?
I am running an LZ based compression algo in Rust compiled to WASM.
For the same file compressed I see high discrepancies in CPU time: most of the time it consumes about 1500ms of CPU time but sometime 2500 or 3000ms.
The worker is just calling that wasm instance, no I/O calls.

So it seems vCPU allocated to an isolate is not fixed. This seems unfair as we are billed by CPU time and sometimes we can be billed 2x,3x for the exact same compute (no I/O involved) depending on the vCPU allocated to our isolate.

Plus the fact that it adds latency to get the response, so we pay more for slower execution..

this is all true, but also true is that these bots are a problem. perhaps on this domain cloudflare

Similar Threads