Reclaiming failed request back to the list or queue. Navigation timed out after 60 seconds.
Hi guys, I just create a simple crawler from ts + crawlee template. The code is working on local but keep timeout on Apify.
The code has been modify to save to mongo-db.
https://github.com/blackmouse572/vnexpress-crawler
GitHub
GitHub - blackmouse572/vnexpress-crawler: crawler for vnexpress news
crawler for vnexpress news. Contribute to blackmouse572/vnexpress-crawler development by creating an account on GitHub.
3 Replies
Someone will reply to you shortly. In the meantime, this might help:
correct-apricotOP•6mo ago
A bit of update, I have create new template, copy the route contents, and remove
node-html-makrdown it run okay, but remove on my source code didn't work
correct-apricot•6mo ago
Try saving a screenshot (https://crawlee.dev/docs/next/examples/capture-screenshot) in the preNavigationHook (https://crawlee.dev/api/playwright-crawler/interface/PlaywrightCrawlerOptions#preNavigationHooks) to see what’s happening on the page and why it’s struggling to navigate. Also, check if your scraper is running with low memory on the platform. If that’s the case, consider increasing the navigationTimeout (https://crawlee.dev/api/next/playwright-crawler/interface/PlaywrightCrawlerOptions#navigationTimeoutSecs) in your configuration.