Cloudflare Developers•10mo ago

or it could just be a crawler following bad links?

CChaika or it could just be a crawler following bad links?

captain•3/4/25, 7:09 PM

this would be specific to pages.dev

ChaikaOP•3/4/25, 7:09 PM

doesn't change that

captain•3/4/25, 7:09 PM

page.dev isn't meant for seo crawling scenarios

captain•3/4/25, 7:09 PM

or to be indexed

ChaikaOP•3/4/25, 7:10 PM

Static Requests are super cheap to Cloudflare and get even cheaper the more cache gets hit

Ccaptain if some ip keeps making requests for non existant files accross multiple sub dom...

Isaac McFadyen•3/4/25, 7:10 PM

How do you propose tracking that, though? Imagine if 32 users (Starlink) share an IP, and they hit 404s. If you block that IP you might be blocking 1 bad user but 31 good users who hit a 404.

ChaikaOP•3/4/25, 7:10 PM

It'd most likely be more compute intensive to try to store/do all that, then just serve the static requests

IIsaac McFadyen How do you propose tracking that, though? Imagine if 32 users (Starlink) share a...

captain•3/4/25, 7:10 PM

cool down period?

ChaikaOP•3/4/25, 7:10 PM

Doing all that across tons of machines is a ton of cost of sync, and a ton of code to go wrong, etc

ChaikaOP•3/4/25, 7:11 PM

at some point workers-and-pages-discussions

ChaikaOP•3/4/25, 7:11 PM

Any anti abuse stuff like that can go wrong too and block legit use cases, just not worth it

captain•3/4/25, 7:11 PM

because cloudflare has so much traffic going through it..at some point its the only answer to crush the bot delema that soaks up so much computer and bandwith on the internet

captain•3/4/25, 7:12 PM

Denial of service attacks only exists because there is no communication or effort amongst hosts to do anything

captain•3/4/25, 7:13 PM

only cloudlfare has the ability to centralize an effort

ChaikaOP•3/4/25, 7:13 PM

Bandwidth is pretty cheap to CF in most cases with their vast network, and compute is more expensive to do anti-bot stuff across thousands of machines then to just serve static assets which can be cached everywhere

captain•3/4/25, 7:13 PM

of my VPS servers that run behind cloudflare, 80% of the cpu is just bots going for non existant files

captain•3/4/25, 7:14 PM

and occassionally i'll drop in, block an ASN and then it will be another ASN 1 month later

Ccaptain of my VPS servers that run behind cloudflare, 80% of the cpu is just bots going ...

Isaac McFadyen•3/4/25, 7:14 PM

Sure but at that point it's very user-specific and Cloudflare wouldn't take action to block things on your own domain when it might be blocking things for legitimate users.

captain•3/4/25, 7:14 PM

sending any abuse type email is generally never read

Isaac McFadyen•3/4/25, 7:14 PM

If you want to reduce that kind of thing you can deploy your own rules that are tailored to your specific domain

Isaac McFadyen•3/4/25, 7:15 PM

I.e rate limiting rules.

captain•3/4/25, 7:16 PM

i mean i do it for my domains already, i'm suggesting something just for pages.dev, but also my 2ndary conversation is the concept of cloudflare being the hub to lower bot traffic in the world in some way. It would be a form of community service in line with how it pushed back against patent trolls.

captain•3/4/25, 7:16 PM

just an idea

captain•3/4/25, 7:17 PM

anyways...carry on

captain•3/4/25, 7:43 PM

now getting a ton of 404's for /sito/wp-includes/wlwmanifest.xml I HATE BOTS...just flooding me.

Silvan•3/4/25, 10:38 PM

i get thousand of bot requests a day :Shrug:

Silvan•3/4/25, 10:39 PM

we are smart and dont put credentials or anything on the client tho so we are fine

Silvan•3/4/25, 10:43 PM

god i love how many services cf offers i have to hold back to not implement everything for simple apps

but maybe rate limiting.. just 1 more

1984 Ford Laser•3/5/25, 12:16 AM

Is there somewhere I can track the changes to CF docs?

11984 Ford Laser Is there somewhere I can track the changes to CF docs?

ChaikaOP•3/5/25, 12:17 AM

https://github.com/cloudflare/cloudflare-docs/commits/production/ ?

GitHub

Commits · cloudflare/cloudflare-docs

Cloudflare’s documentation. Contribute to cloudflare/cloudflare-docs development by creating an account on GitHub.

CChaika https://github.com/cloudflare/cloudflare-docs/commits/production/ ? 🤔

1984 Ford Laser•3/5/25, 12:17 AM

Thank you wasnt sure of the exact repo

1984 Ford Laser•3/5/25, 12:20 AM

This was the change I was trying to figure out manually

no wonder I couldnt find it lol

Silvan•3/5/25, 12:20 AM

i updated some lib and now i get that build error even tho im using

"compatibility_flags": ["nodejs_als"]

"compatibility_flags": ["nodejs_als"]

any clues why this would error now?

and yes

nodejs_compat

nodejs_compat

would fix this im curious tho

Silvan•3/5/25, 12:24 AM

could this be a change from

node:async_hooks

node:async_hooks

to just

async_hooks

async_hooks

kunal•3/5/25, 1:53 AM

Does hyperdrive cache queries that return 0 rows?

Kkunal Does hyperdrive cache queries that return 0 rows?

Mitya•3/5/25, 9:21 AM

#hyperdrive

Mitya•3/5/25, 9:25 AM

Am I right that

waitUntil()

waitUntil()

can be used to ensure an asynchronous process will complete even after a response is sent? So if my worker has this:

waitUntil(new Promise(res => setTimeout(() => {
    console.log('hi!');
    res();
}, 5000)));
return new Response('foo');

waitUntil(new Promise(res => setTimeout(() => {
    console.log('hi!');
    res();
}, 5000)));
return new Response('foo');

...will I see "hi!" in the console 5 seconds after the worker has sent a response? (And presumably I don't need to use

await

await

with the above, as that would defeat the point and delay the response?)

Murder Chicken•3/5/25, 2:33 PM

If I'm getting build errors for workers where the only error in the build log is "

Failed: error occurred while updating repository submodules

Failed: error occurred while updating repository submodules

" is there any way to get more information about the error? I've exhausted my ability to troubleshoot past this.

piffie•3/5/25, 2:47 PM

hmmm i have an issue that cloudflare/pages-action@v1 still uses node 20 - but it seems i need node 23 to deploy the latest nuxt to it. Is the pages-action outdated? or will there be an update for it?

Scott•3/5/25, 4:30 PM

Anyone know why there is suddenly a 1,000 worker route limit per account?

Scott•3/5/25, 4:30 PM

I have more than 1,000 but users are now reporting this error.

Scott•3/5/25, 4:31 PM

There is nothing documented about it, and I can't even tell if the Business plan ups the limit.

SuperHelpflare•3/5/25, 4:38 PM

Please do not post your question in multiple channels/post it multiple times per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

theodm•3/5/25, 5:24 PM

Hi, does anyone knows how CPU is shared between isolates (workers) ?
I am running an LZ based compression algo in Rust compiled to WASM.
For the same file compressed I see high discrepancies in CPU time: most of the time it consumes about 1500ms of CPU time but sometime 2500 or 3000ms.
The worker is just calling that wasm instance, no I/O calls.

So it seems vCPU allocated to an isolate is not fixed. This seems unfair as we are billed by CPU time and sometimes we can be billed 2x,3x for the exact same compute (no I/O involved) depending on the vCPU allocated to our isolate.

Plus the fact that it adds latency to get the response, so we pay more for slower execution..

Mitya•3/5/25, 5:28 PM

CF devs may be interested to know that Cursor, the AI assistant, is giving 404 links to the docs. This is because the URLs it outputs are relative, not absolute, yet lack the prefix

, so they are served related to the base

https://developers.cloudflrare.com/workers/ai/

https://developers.cloudflrare.com/workers/ai/

. This results in URLs like the below being generated:

https://developers.cloudflare.com/workers/ai/queues/configuration/javascript-apis/

https://developers.cloudflare.com/workers/ai/queues/configuration/javascript-apis/

Simon Colby•3/5/25, 7:49 PM

Hey guys, I have a problem that my production worker is much slower then staging one even though the only difference is the traffic (the code and configuration is exactly the same). Do you know how can I fix it?

Ttheodm Hi, does anyone knows how CPU is shared between isolates (workers) ? I am runni...

Keebs•3/5/25, 8:29 PM

feels like that is something that would be easier done external? :D

Keebs•3/5/25, 8:30 PM

like a server that is set up for compression - lz compiled with cpu flags and then its probably 10ms, you could transfer the data from worker to worker and compress in way less time then

Ttheodm Hi, does anyone knows how CPU is shared between isolates (workers) ? I am runni...

Isaac McFadyen•3/5/25, 8:33 PM

By definition CPU time is invariant of number of vCPUs. Even if two cores are allocated/working at the same time then there will be the same number of CPU (milli)seconds... but that can't happen in Workers because they're single-threaded anyway. Not sure what the reason for the variance you're seeing there is. How are you getting the specific CPU time metrics? GraphQL API?

KKeebs like a server that is set up for compression - lz compiled with cpu flags and th...

theodm•3/5/25, 8:37 PM

That what I currently do, those CPU heavy algos are running from GCP cloud run with allocated CPU. Waiting for cloudflare containers to see what they have to offer

or it could just be a crawler following bad links?

Similar Threads