How to solve example.com needs to review the security of your connection before proceeding?
I am trying to scrape a site that has a page that check if the connection is secure and I get this error:
WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. Request blocked - received 403 status code.
5 Replies
foreign-sapphireOP•3y ago
I think it needs to wait longer, instead of to shut down after 1 second and to try again, because it always get the error, how can this be done?
quickest-silver•3y ago
This is a Cloudflare check for a real browser usually. Try to wait longer, use headless: false and Firefox
foreign-sapphireOP•3y ago
Is there a guide on how to use Firefox?
other-emerald•3y ago
@pelsec check out this example: https://crawlee.dev/docs/examples/playwright-crawler-firefox
foreign-sapphireOP•3y ago
Thank you