GPTBot in nginx logs but should be blocked
In my free account I have AI Audit set to block GPT Bot - OpenAI, I've also got a custom security rule first in the list that blocks various User Agents one of which is defined as (http.user_agent contains "GPTBot").
With both these in place I'm still seeing in my nginx logs
[09/Aug/2025:15:27:41 +0000] "GET aURL HTTP/1.0" 200 56651 "-" "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.2; +https://openai.com/gptbot)"
I'm pretty new to Cloudflare so maybe this is user error or something else. Cheers.
3 Replies
So bypassing Cloudflare entirely and going to the instance
misread, my bad
I manually added a robots.txt file blocking GPTBot and only that rule, after that file was accessed again buy GPTBot the requests stopped soon after. So I guess they have the IP because Cloudflare IS sending a robots.txt file with the block as well but it was not working. Thanks.