A site that shows cloudflare captcha ALWAYS
I immediately get captcha on every URL.
Accessing it in a normal GUI browser typing site homepage URL: captcha.
Searching this site in google, clicking on the link in google results: browser shows site address and...: captcha.
(by the way, they changed it, few months ago this site was not that restrictive)
Well... what is our best solution for sites always showing cloudflare captchas ?
3 Replies
View post on community site
This post has been pushed to the community knowledgebase. Any replies in this thread will be synced to the community site.
Apify Community
Depends on the website, they may require it even for regular users or they use captchas only for suspicious IPs, fingerprints and so on.
sunny-green•6mo ago
Truly often they always check, even for regular users.
However, you can tweak your scraper's configuration to try bypassing it. Generally, using Playwright + firefox with a residential proxy tends to work best, although it’s also the most expensive option.
btw, Here are some articles that might help:
https://help.apify.com/en/articles/7029488-how-to-bypass-cloudflare
https://docs.apify.com/academy/anti-scraping/mitigation/cloudflare-challenge.md
How to bypass Cloudflare | Apify Help & Support
Apify's default scrapers are a great choice for Cloudflare protected sites
Bypassing Cloudflare browser check | Academy | Apify Documentation
Learn how to bypass Cloudflare browser challenge with Crawlee.