Why does Chromium close it self after each request even though the queued links aren't completed?
I use PlaywrightCrawler with headless chromium in crawlee, and I have many links in the enqueLinks but when I check my activity Monitor (task manager) it shows me a Chromium and when the requestHandler has done the task for the current link the chromium closes and restarts again to continue with the next link.
This costs so much time and makes the whole scraping really slow.
How do I fix it?
2 Replies
fair-rose•2y ago
when page handled its closed and reopening new page in the same tab will not speed up anything, if you want better performance check if Cheerio works for your needs
fair-rose•2y ago
Try keepAlive