Any way to check blocked user agents?
I've been asked by a Partner whether my sites are blocking any particular user agents (crawlers), is there any way to check which (if any) user agents are being blocked? I haven't configured any WAF rules or anything.
4 Replies
Support AI
According to the Cloudflare documentation, User Agent Blocking rules are applied after Zone Lockdown rules. If you haven't configured any WAF rules or Zone Lockdown rules, it's unlikely that any user agents are being blocked. However, to confirm, you can check the User Agent Blocking rules in your Cloudflare account.
As the documentation doesn't provide a direct way to check which user agents are being blocked, you can try checking the User Agent Blocking rules in your Cloudflare dashboard or using the Cloudflare API to retrieve the rules.
You can refer to the Cloudflare documentation on User Agent Blocking for more information: https://developers.cloudflare.com/waf/tools/user-agent-blocking/
Feedback
Feedback has been submitted! Thank you :)
This is an AI generated response, it may not be 100% accurate. Please validate the output. You can ask followups by replying to this message or mentioning me.
Is it possible to see all the crawlers that have tried to view my site?
You can view the rule that caused a block or challenge page to be presented at https://dash.cloudflare.com/?to=/:account/:zone/security/events
Thanks @Leo !