Stoping Crawler when done in scraping
Good day everyone, How can I make the crawler stop? When its done to scraper/request a certain url?
Because I want to setup my crawlee project that continously running when it has no url to request, like waiting to a queue of urls (redis)
2 Replies
robust-apricot•2y ago
CriticalError | API | Crawlee
Errors of
CriticalError type will shut down the whole crawler.like-goldOP•2y ago
How can I do that? throw an CriticlError even its not encountering error?