New Tab
When I start a crawl using PlaywrightCrawler it always opens the URL in a new tab, leaving an empty tab open. This affects performance. Is there a way to turn this off and not open a new tab but use the empty start tab instead?
3 Replies
passive-yellowOP•2y ago
I also believe one major sportsbetting site I'm trying to scrape is using this to detect scraping,
harsh-harlequin•2y ago
Check https://playwright.dev/ - crawlee is a wrapper around it so there might be defaults or settings to change the way how browser started
extended-yellow•2y ago
@Moby I think this happens if you do addRequests and then run with empty args.
Try to directly specify your first url inside .run method.