CA
Crawlee & Apify•3y ago
xenial-black

Override browser permission on PuppeteerCrawler

Hi, a quick question: how do I override a certain permission on a page when using PuppeteerCrawler? Something like so: 
...,
preNavigationHooks: [
  async (crawlingContext, gotoOptions) => {
    const { page, browserController: { browser } } = crawlingContext;

    const context = await browser.defaultBrowserContext();
    context.clearPermissionOverrides();
    context.overridePermissions('some-domain', ['some-feature']);
  }
],
...
...,
preNavigationHooks: [
  async (crawlingContext, gotoOptions) => {
    const { page, browserController: { browser } } = crawlingContext;

    const context = await browser.defaultBrowserContext();
    context.clearPermissionOverrides();
    context.overridePermissions('some-domain', ['some-feature']);
  }
],
...
When I run it, I get a warning message: WARN PuppeteerCrawler: Reclaiming failed request back to the list or queue. Cannot read private member from an object whose class did not declare it I understand that I'm accessing private prop but how/where do I override such permissions on PuppeteerCrawler? 🙂 Thanks! 🙏
6 Replies
xenial-black
xenial-blackOP•3y ago
Hi @vladdy, can you please assist with this? 🙏
metropolitan-bronze
metropolitan-bronze•3y ago
Oh..interesting Let me take a look
xenial-black
xenial-blackOP•3y ago
Thank you!
metropolitan-bronze
metropolitan-bronze•3y ago
do you happen to have a stacktrace too? also, can you try getting the context via page.browserContext() instead (followed by the same context.overridePermissions calls)?
xenial-black
xenial-blackOP•3y ago
@stacktrace: 
ERROR PuppeteerCrawler: Request failed and reached maximum retries. TypeError: Cannot read private member from an object whose class did not declare it
    at __classPrivateFieldGet (/path/node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:25:94)
    at Proxy.defaultBrowserContext (/path/node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:213:16)
    at PuppeteerCrawler.preNavigationHooks (file:///path/src/index.js:46:39)
    at PuppeteerCrawler._executeHooks (/path/node_modules/@crawlee/basic/internals/basic-crawler.js:794:23)
    at PuppeteerCrawler._handleNavigation (/path/node_modules/@crawlee/browser/internals/browser-crawler.js:263:20)
    at PuppeteerCrawler._runRequestHandler (/path/node_modules/@crawlee/browser/internals/browser-crawler.js:215:28)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async PuppeteerCrawler._runRequestHandler (/path/node_modules/@crawlee/puppeteer/internals/puppeteer-crawler.js:102:9)
    at async wrap (/path/node_modules/@apify/timeout/index.js:52:21)
ERROR PuppeteerCrawler: Request failed and reached maximum retries. TypeError: Cannot read private member from an object whose class did not declare it
    at __classPrivateFieldGet (/path/node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:25:94)
    at Proxy.defaultBrowserContext (/path/node_modules/puppeteer/lib/cjs/puppeteer/common/Browser.js:213:16)
    at PuppeteerCrawler.preNavigationHooks (file:///path/src/index.js:46:39)
    at PuppeteerCrawler._executeHooks (/path/node_modules/@crawlee/basic/internals/basic-crawler.js:794:23)
    at PuppeteerCrawler._handleNavigation (/path/node_modules/@crawlee/browser/internals/browser-crawler.js:263:20)
    at PuppeteerCrawler._runRequestHandler (/path/node_modules/@crawlee/browser/internals/browser-crawler.js:215:28)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async PuppeteerCrawler._runRequestHandler (/path/node_modules/@crawlee/puppeteer/internals/puppeteer-crawler.js:102:9)
    at async wrap (/path/node_modules/@apify/timeout/index.js:52:21)
Cool, testing... Perfect! That works. Thanks! ✨
metropolitan-bronze
metropolitan-bronze•3y ago
oh, sweet No worries

Did you find this page helpful?