CA
adverse-sapphire
Crawlee with multiple Crawlers?
Does the python crawlee allow for multiple crawlers to be run using one router?
Just asking as a coleague asked me if it would be possible because curl requests are a lot faster than playwright, so if we can use curl for half the requests and only load the browser for the other portion where it's needed, it could significantly speed up some processes
2 Replies
Someone will reply to you shortly. In the meantime, this might help:
-# This post was marked as solved by TGTGamer. View answer.
adverse-sapphireOP•5mo ago
Found answer via hall. https://github.com/apify/crawlee-python/discussions/573
GitHub
Running different requests with different crawlers? · apify crawlee...
I'm trying to solve a situation where I want to make the initial request with a plain crawler (because it's an API or something), but continue with subsequent requests to detail pages with ...