CA
Crawlee & Apify•3y ago
fair-rose

PlaywrightCrawler in AWS Lambda

Hi guys, trying to run PlaywrightCrawler in a lambda but having some issues. 
browserType.launchPersistentContext: Executable doesn't exist at /home/sbx_user1051/.cache/ms-playwright/chromium-1060/chrome-linux/chrome
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝
    at async PlaywrightPlugin._launch (/opt/nodejs/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:99:40)
    at async BrowserPool._launchBrowser (/opt/nodejs/node_modules/@crawlee/browser-pool/browser-pool.js:465:29)
    at async /opt/nodejs/node_modules/@crawlee/browser-pool/browser-pool.js:274:37
browserType.launchPersistentContext: Executable doesn't exist at /home/sbx_user1051/.cache/ms-playwright/chromium-1060/chrome-linux/chrome
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝
    at async PlaywrightPlugin._launch (/opt/nodejs/node_modules/@crawlee/browser-pool/playwright/playwright-plugin.js:99:40)
    at async BrowserPool._launchBrowser (/opt/nodejs/node_modules/@crawlee/browser-pool/browser-pool.js:465:29)
    at async /opt/nodejs/node_modules/@crawlee/browser-pool/browser-pool.js:274:37
Unfortunately I can't run npx playwright install since it is a lambda function. I am also trying to upload and use a separate chromium executable (https://github.com/Sparticuz/chromium), but get a different error: 
WARN  PlaywrightCrawler: Reclaiming failed request back to the list or queue. browserType.launchPersistentContext: spawn Unknown system error -8
WARN  PlaywrightCrawler: Reclaiming failed request back to the list or queue. browserType.launchPersistentContext: spawn Unknown system error -8
I'm guessing this error could be because of a version mismatch with playwright. Where can I find the chromium version for playwright that I should use? Any other ideas?
2 Replies
MEE6
MEE6•2y ago
@FranMan just advanced to level 1! Thanks for your contributions! 🎉
sunny-green
sunny-green•2y ago
Are you using the correct Docker image? https://crawlee.dev/docs/guides/docker-images#actor-node-playwright-chrome
Running in Docker | Crawlee
Example Docker images to run your crawlers

Did you find this page helpful?