How to access sites with bot detection?
I'm trying to access https://www.thechronicle.com.au/ (News Corp Australia) but getting blocked.
Error: "You might have been detected and blocked as a crawler bot!"
Tried:
• web_fetch - Cookie restrictions
• agent-browser - Bot detection triggered
Setup: OpenClaw 2026.2.17, Agent Browser skill, headless mode
Questions:
1. Browser profiles with real cookies/UA?
2. Residential proxy support?
3. CAPTCHA integration?
4. Cloudflare bypass options?
Goal: News article monitoring
Error: "You might have been detected and blocked as a crawler bot!"
Tried:
• web_fetch - Cookie restrictions
• agent-browser - Bot detection triggered
Setup: OpenClaw 2026.2.17, Agent Browser skill, headless mode
Questions:
1. Browser profiles with real cookies/UA?
2. Residential proxy support?
3. CAPTCHA integration?
4. Cloudflare bypass options?
Goal: News article monitoring
The AI that actually does things. Emails, calendar, home automation - all from your favorite chat app. New shell, ready to help. The lobster way.
130,529Members
Resources
Recent Announcements
Similar Threads
Was this page helpful?
