no idea, could be non-cloudflare bots, but i don't know how they would know our website exists. it'

no idea, could be non-cloudflare bots, but i don't know how they would know our website exists. it's not attached to a domiain name, just using the default pages domain name

MattIPv4•10/17/23, 11:40 PM

Certificate transparency logs

AviOP•10/17/23, 11:40 PM

i'm not familiar

AviOP•10/17/23, 11:40 PM

what's that?

MattIPv4•10/17/23, 11:40 PM

https://crt.sh/?q=pages.dev

AviOP•10/17/23, 11:40 PM

ohhh

MattIPv4•10/17/23, 11:41 PM

https://certificate.transparency.dev/

AviOP•10/17/23, 11:41 PM

duh, you're right, i guess since each of these has a certificate they're part of the public record

MattIPv4•10/17/23, 11:41 PM

indeed

AviOP•10/17/23, 11:42 PM

so, that's totally possible someone's scraping this data. but in order to trigger the behavior i'm seeing, you'd need JS enabled AND a mobile useragent. i guess it's possible for such a bot to exist, but seems pretty unlikely

AviOP•10/17/23, 11:42 PM

maybe i need a more colorful imagination

MattIPv4•10/17/23, 11:42 PM

That being said, I thought pages had changed how they issue certs to stop this happening

AviOP•10/17/23, 11:43 PM

yeah this list stops at 2021-06-17

MattIPv4•10/17/23, 11:43 PM

But generally speaking, if something is on the internet, something will find it

AviOP•10/17/23, 11:43 PM

yeah... i just want to know who. are there logs for traffic to my pages project somewhere?

MattIPv4•10/17/23, 11:44 PM

Someone else here might be more knowledgeable on what exactly enables discovery of pages projects quickly after deployment

MMattIPv4 That being said, I thought pages had changed how they issue certs to stop this h...

Chaika•10/17/23, 11:45 PM

There's a wildcard cert *.pages.dev*.pages.dev which covers the production branch before the project specific one gets issued, and then each Project gets issued a wildcard *.pages-project.pages.dev, pages-project.pages.dev*.pages-project.pages.dev, pages-project.pages.dev for any previews/branches

MattIPv4•10/17/23, 11:45 PM

As for logs, iirc you'll either want to implement your own middleware to do that, or attach a custom domain and use Cloudflare's standard web anayltics

Chaika•10/17/23, 11:46 PM

Can use bulk redirects to redirect people from your pages.dev to your Custom Domain, and then Web Traffic Analytics if Pro or higher to see specific IPs and such

AviOP•10/17/23, 11:46 PM

this? https://dash.cloudflare.com/?to=/:account/web-analytics

AAvi this? https://dash.cloudflare.com/?to=/:account/web-analytics

Chaika•10/17/23, 11:48 PM

Pages does have one click support for that, but that won't give you user IPs or anything, just generic country, device/browser/os, paths, hosts, referrars

AviOP•10/17/23, 11:48 PM

would it give UAs?

AviOP•10/17/23, 11:49 PM

also, our setup is a bit odd — we have a single domain attached to a worker that acts as a transparent reverse proxy across multiple pages projects

AviOP•10/17/23, 11:49 PM

every time we push to main we spin up a whole new pages project

MattIPv4•10/17/23, 11:49 PM

I don't think web analytics gives UA details

MattIPv4•10/17/23, 11:50 PM

web traffic analytics will though for a zone that's pro

AAvi would it give UAs?

Chaika•10/17/23, 11:50 PM

It only gives info parsed from the UA like browser/OS, not the raw UA

CChaika It only gives info parsed from the UA like browser/OS, not the raw UA

AviOP•10/17/23, 11:52 PM

and this is only aggregate data right?

MattIPv4•10/17/23, 11:53 PM

Both web analytics and web traffic analytics are aggregated

MattIPv4•10/17/23, 11:53 PM

If you want raw logs, you'll either want instant logs (business) or logpush (ent)

MattIPv4•10/17/23, 11:53 PM

Or again, write middleware in a function on your pages project and do logging yourself

AAvi also, our setup is a bit odd — we have a single domain attached to a worker that...

Chaika•10/17/23, 11:54 PM

You could leverage your worker too, to ship logs off somewhere, Analytics Engine or Queues, or just an http endpoint

AviOP•10/17/23, 11:58 PM

thanks. in this particular case, the weird thing is that requests seem to be coming to the pages project itself, bypassing the router

AviOP•10/17/23, 11:59 PM

our site isn't set up for this, since we expect all traffic to come from the router worker

AviOP•10/17/23, 11:59 PM

that's why i'm curious who is making these requests

Chaika•10/18/23, 12:00 AM

ah yea probably bots off CT Logs. You can use a Bulk Redirect rule to redirect any requests coming to the Pages project itself, and Custom domains would still work. Although if your router is fetching the pages.dev directly, would break

AviOP•10/18/23, 12:01 AM

right yeah that's exactly what happens. the router transparently reverse proxies the

of the pages.dev directly. the pages.dev is configured to use all absolute URLs for CSS and JS, so all of those are made directly to the pages.dev project, so we don't have to add extra hops through the router

AviOP•10/18/23, 12:06 AM

does instant logs have any retention period, or is it only live?

James•10/18/23, 12:07 AM

Only live

patfien•10/18/23, 4:05 AM

Hey guys, I've never worked with cloudflare before but a client of mine is using it to deliver their videos to their website.

They've been having issues passing Core Web Vitals on a lot of their pages, specifically LCP. I'm still trying to confirm it's the cloudflare video but based on deduction it's the most liekly culprit.

They're embedding via the stream player: https://developers.cloudflare.com/stream/viewing-videos/using-the-stream-player/

Is this the most efficient way to embed from a page speed optimization perspective?

Use the Stream Player · Cloudflare Stream docs

Cloudflare provides a customizable web player that can play both on-demand and live video, and requires zero additional engineering work.

Ppatfien Hey guys, I've never worked with cloudflare before but a client of mine is using...

patfien•10/18/23, 4:05 AM

Here's a peak of the GTMetrix waterfall

AAvi no idea, could be non-cloudflare bots, but i don't know how they would know our...

AviOP•10/18/23, 6:22 AM

aha! i finally found one of these mysterious crawlers. here is the useragent. it has to be spoofed.

Mozilla/5.0 (iPhone; CPU iPhone OS 16_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/99.0.4844.47 Mobile/15E148 Safari/604.1

Mozilla/5.0 (iPhone; CPU iPhone OS 16_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/99.0.4844.47 Mobile/15E148 Safari/604.1

christianwilliams•10/18/23, 8:13 AM

We get an increased abuse in our website with VPNs, and Cloudflare doesn't do much help; is there anything that we can configure in the IP rules?

christianwilliams•10/18/23, 8:13 AM

Like rep

Adel Asaad•10/18/23, 8:55 AM

Adel Asaad•10/18/23, 8:56 AM

I need to backup my Cloudflare pages site that I have setup using direct upload

Adel Asaad•10/18/23, 8:56 AM

Can someone help me

Pterodactyl Support Bot•10/18/23, 9:20 AM

well, this is first time i've seen someone using CF tunnels for a botnet command and control

Pterodactyl Support Bot•10/18/23, 9:20 AM

they are even using an open source wireguard lib to connect instead of cloudflared

Pterodactyl Support Bot•10/18/23, 9:21 AM

gonna report it and hope the domain/account gets shut down

zegevlier•10/18/23, 9:28 AM

?abuse

no idea, could be non-cloudflare bots, but i don't know how they would know our website exists. it'

Similar Threads