Error Starting Crawler

Occasionally, I saw this error when starting my actor,

.....
2022-11-13T17:05:48.132Z INFO  System info {"apifyVersion":"3.1.1","apifyClientVersion":"2.6.0","crawleeVersion":"3.1.1","osType":"Linux","nodeVersion":"v16.17.0"}
2022-11-13T17:05:50.750Z INFO  PuppeteerCrawler: Starting the crawl
2022-11-13T17:05:50.841Z ERROR PuppeteerCrawler: Request failed and reached maximum retries. TypeError: Cannot convert undefined or null to object
2022-11-13T17:05:50.843Z     at Function.keys (<anonymous>)
2022-11-13T17:05:50.843Z     at BayesianNode.sample (/home/myuser/node_modules/generative-bayesian-network/bayesian-node.js:79:39)
2022-11-13T17:05:50.844Z     at BayesianNetwork.generateSample (/home/myuser/node_modules/generative-bayesian-network/bayesian-network.js:41:42)
2022-11-13T17:05:50.845Z     at FingerprintGenerator.getFingerprint (/home/myuser/node_modules/fingerprint-generator/fingerprint-generator.js:35:62)
2022-11-13T17:05:50.845Z     at /home/myuser/node_modules/@crawlee/browser-pool/fingerprinting/hooks.js:23:48
2022-11-13T17:05:50.846Z     at BrowserPool._executeHooks (/home/myuser/node_modules/@crawlee/browser-pool/browser-pool.js:550:19)
2022-11-13T17:05:50.847Z     at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-11-13T17:05:50.848Z     at async BrowserPool._launchBrowser (/home/myuser/node_modules/@crawlee/browser-pool/browser-pool.js:463:13)
2022-11-13T17:05:50.848Z     at async /home/myuser/node_modules/@crawlee/browser-pool/browser-pool.js:274:37 {"id":"KzpBnJWAKJGFlVD","url":"...","method":"GET","uniqueKey":"..."}
....

.....
2022-11-13T17:05:48.132Z INFO  System info {"apifyVersion":"3.1.1","apifyClientVersion":"2.6.0","crawleeVersion":"3.1.1","osType":"Linux","nodeVersion":"v16.17.0"}
2022-11-13T17:05:50.750Z INFO  PuppeteerCrawler: Starting the crawl
2022-11-13T17:05:50.841Z ERROR PuppeteerCrawler: Request failed and reached maximum retries. TypeError: Cannot convert undefined or null to object
2022-11-13T17:05:50.843Z     at Function.keys (<anonymous>)
2022-11-13T17:05:50.843Z     at BayesianNode.sample (/home/myuser/node_modules/generative-bayesian-network/bayesian-node.js:79:39)
2022-11-13T17:05:50.844Z     at BayesianNetwork.generateSample (/home/myuser/node_modules/generative-bayesian-network/bayesian-network.js:41:42)
2022-11-13T17:05:50.845Z     at FingerprintGenerator.getFingerprint (/home/myuser/node_modules/fingerprint-generator/fingerprint-generator.js:35:62)
2022-11-13T17:05:50.845Z     at /home/myuser/node_modules/@crawlee/browser-pool/fingerprinting/hooks.js:23:48
2022-11-13T17:05:50.846Z     at BrowserPool._executeHooks (/home/myuser/node_modules/@crawlee/browser-pool/browser-pool.js:550:19)
2022-11-13T17:05:50.847Z     at processTicksAndRejections (node:internal/process/task_queues:96:5)
2022-11-13T17:05:50.848Z     at async BrowserPool._launchBrowser (/home/myuser/node_modules/@crawlee/browser-pool/browser-pool.js:463:13)
2022-11-13T17:05:50.848Z     at async /home/myuser/node_modules/@crawlee/browser-pool/browser-pool.js:274:37 {"id":"KzpBnJWAKJGFlVD","url":"...","method":"GET","uniqueKey":"..."}
....

It seems caused by fingerprint generator. Is there anything I can improve to prevent this error? thanks

2 Replies

conscious-sapphire•3y ago

Thanks, would you be so kind and report to https://github.com/apify/crawlee/issues with extra context? thanks

GitHub

Issues · apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast. - Issues · apify/crawlee

conscious-sapphire•3y ago

To work around it now, you can disable fingerprints via browserPoolOptions: { useFingerprints: false }

Gaming

Programming

Error Starting Crawler

Did you find this page helpful?