Google blocked by Cloudflare Edge despite whitelist? Can't make sense of logs!
I'm having an issue where Google's crawler is being blocked by Cloudflare's edge, even though we specifically have a whitelisting rule that skips all security rules.
Mainly, I can't make sense of the log explorer entry. The log explorer says "CacheStatus=unknown" which would suggest it's not being served from cache. However, the origin is not delivering any response, so the 403 response was clearly sent by Cloudflare.
Second, the security rule triggered is our whitelisting rule. The rule basically makes it so that all custom rules are skipped for verified search engines. I've attached a screenshot of the rule, and below is an example of a block event in our logs.
I'd really appreciate help with figuring out what is causing the 403 response to occur. Thank you so much!
log
CacheCacheStatus
unknown
CacheReserveUsed
false
CacheResponseStatus
403
CacheTieredFill
false
ClientCountry
us
ClientIP
66.249.68.35
ClientRequestHost
www.worldhistory.org
ClientRequestMethod
GET
ClientRequestPath
/Kublai_Khan/
ClientRequestURI
/Kublai_Khan/
ClientXRequestedWith
-
Date
2025-07-29
EdgeEndTimestamp
2025-07-29T04:24:39Z
EdgeRequestHost
www.worldhistory.org
EdgeResponseStatus
403
EdgeStartTimestamp
2025-07-29T04:24:39Z
OriginResponseStatus
0
OriginResponseTime
0
RayID
9669d91038c3d44d
RequestHeaders
{}
SecurityRuleDescription
Whitelist: Search Engines
SecurityRuleID
bc909ecfc0dd4018827b43a8872d5e66
SecurityRuleIDs
bc909ecfc0dd4018827b43a8872d5e66
WorkerStatus
unknown
WorkerSubrequest
false
7 Replies
Also, the events show up in the firewall logs. They all contain Action=skip and EdgeResponseStatus=403 which again doesn't make any sense to me.
You can view the rule that caused a block or challenge page to be presented at https://dash.cloudflare.com/?to=/:account/:zone/security/events
Well that's the thing! No firewall rule was triggered.
Here's an example:
When I click on "view related security events" there are none.
Nope. This is what I'm seeing in the log explorer:
How can I tell whether a worker was involved? I am under the impression that WorkerStatus and WorkerSubrequest are the indicators.
We have no workers delivering content, no.
Just checking if anyone else might have insights? 🙂