AI blocking interefering with OIDC server query
Hi,
Cloudflare's commendable new feature to block AI scrapers unfortunately appears to be interefering with my application.
I am a solo developer who has deployed my application behind a free Cloudflare service using Zero Trust. I run my application and OIDC server behind a Cloudflare tunnel. My application queries the OIDC server via user browser redirects but also directly from my application server to obtain the certfiacte used to sign a user's OIDC token. The server then uses this certificate to validate the token.
This has been working fine so far but in the last 3 days the application server started seeing 403 responses when querying the OIDC server for the certificate. The same query works fine from my browser. The 403 is due to a challenge being presented to my application.
Is it possible to specifically allow my application to get through without a challenge?
Many thanks!
9 Replies
I am curious to understand if "Zero Trust > Access > Service authentication" can help with this.
You can view the rule that caused a block or challenge page to be presented at https://dash.cloudflare.com/?to=/:account/:zone/security/events
Thanks @SuperHelpflare ! I can confirm that is indeed the case including by running curl from my server's host
I am now going to try using Service authentication - including the token in the request header - I will report results here
Support AI
I'm glad you're exploring alternative solutions. According to the Cloudflare documentation, using a service token can help authenticate your application server's requests to the OIDC server. You can find more information on service tokens in the Cloudflare API documentation, specifically in the section on Access API Examples: Service Token.
Please let me know if you have any further questions or if there's anything else I can help you with.
Sources
Feedback
Feedback has been submitted! Thank you :)
This is an AI generated response, it may not be 100% accurate. Please validate the output. You can ask followups by replying to this message or mentioning me.
@Leo Do you mean the Discord bot?
I pinged the Discord bot because I realized too late that it was a bot
I did suspect the cause - I confirmed by running curl and decoding the returned output that proved to be a robot challenge
I am planning to use Service auth to bypass the robot challenge
I could be wrong
I am using Zero Trust
I am using a tunnel in Zero Trust
So what do you recommend? My application and OIDC server are deployed at the end of a tunnel configured using ZT
It seems Service auth under "Zero Trust > Access > Service authentication" should help.
It is not a serice blocking the request - it is CF's bot challenge
CF's rules for determining what to challenge - I dont know much else
Hmmm.... would you mind sharing the link again?
I am new to CF and dont quite know how to navigate to the WAF log
Support AI
I'm unable to provide an answer to that at the moment. Please rephrase your query and I'll try again.
This is an AI generated response, it may not be 100% accurate. Please validate the output. You can ask followups by replying to this message or mentioning me.
Matched service
Service: Bot fight mode
Action taken: Managed Challenge
Bot Fight Mode cannot be skipped by WAF Rules or other "Bypass" actions.
Bot Fight Mode is a very aggressive solution that should only be enabled if you are actively under attack.
Remedies:
- Upgrade to a Paid plan and use Super Bot Fight Mode
- Disable Bot Fight Mode
Yeah Service auth may not work...
Oh well I will try anyways
Why are you so sure? Does it not allow CF to conclude that the originator of the request is a trusted entity?
And hence the bot challenge need not be applied?
Got it
Hey man - I never claimed to be an expert - just trying something
Thanks for your help