CA
ugly-tan
How to run Website Content Crawler asynchronously
Hi all, I'm new to Apify and was wondering how to get the Website Content Crawler to crawl pages asynchronously.
I used the python ApifyClient and tried passing 50 url's to the "startUrls" input field. I would have expected the actor to run these asynchronously on its own, but the run took 25 min so I'm assuming it didn't.
Another option I guess is to use the ApifyClientAsync and run 50 individual async runs and then collect their results.
What's the recommended way to do this? I'm planning on crawling around 100 000 urls with depth 0.
0 Replies