Crawlee & Apify•2y ago

How to make Puppeteer crawler ignore errors on page?

I receive some 401 code errors while loading the page, but these requests are not important for me. The problem is crawling is crashing, because of that error and I cannot continue to interact with page.

2 Replies

correct-apricotOP•2y ago

Or how can I block specific API request on page load?

Pepa J•2y ago

Hi @Nikita Gankin : You should be also to set specific http statuses that will blocked in crawler options:

AnyCrawler({
    sessionPoolOptions: {
        blockedStatusCodes: [404]
    }
});

AnyCrawler({
    sessionPoolOptions: {
        blockedStatusCodes: [404]
    }
});

Or set it to empty array if you want to handle everything by your self 🙂

Gaming

Programming

How to make Puppeteer crawler ignore errors on page?

Did you find this page helpful?