Crawlee & Apify•3y ago

New page middleware

I need to use adblock from here https://www.npmjs.com/package/@cliqz/adblocker-playwright, but not quite sure what is the best way to integrate it with Crawlee. For now, I create page with dummy url, enable adblocker on the page and then navigate to the target website. I am curious is there an API to add some kind of middleware to the page creation process, to add adblocker to every newly created page before target url navigation? Couldn't find anything suitable in neither Crawlee or Playwright docs 😦 Thank you in advance!

3 Replies

stormy-gold•3y ago

You can run this function is preNavigationHooks where you have access to the page.

PlaywrightBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
  blocker.enableBlockingInPage(page);
});

PlaywrightBlocker.fromPrebuiltAdsAndTracking(fetch).then((blocker) => {
  blocker.enableBlockingInPage(page);
});

deep-jadeOP•3y ago

Thank you very much!

provincial-silver•3y ago

docs: https://crawlee.dev/api/puppeteer-crawler/interface/PuppeteerCrawlerOptions#preNavigationHooks - pls be aware it will be called on retry as well

Gaming

Programming

New page middleware

Did you find this page helpful?