How to set depth-breadth first crawl?
I suspect that crawlee will perform a breadth first crawl by default with
enqueueLinks(...)
Does exist somme option to perform a depth first?3 Replies
stormy-gold•3y ago
There unfortunately isn't a way to do this, but the idea is great! Please make a feature request on Crawlee's Github here: https://github.com/apify/crawlee/issues
GitHub
Issues · apify/crawlee
Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast. - Issues · apify/crawlee
stormy-goldOP•3y ago
GitHub
Option to perform a depth first crawl · Issue #1595 · apify/crawlee
Which package is the feature request for @crawlee/core Feature Crawlee will perform a breadth first crawl by default with enqueueLinks(...). Make some option to perform a depth first. Ref Wikipedia...
stormy-gold•3y ago
Looks like the
forefront feature has now been added to enqueueLinks