TimCoulter
CACrawlee & Apify
•Created by TimCoulter on 5/10/2025 in #apify-platform
Errors while attempting to use Apify Proxy
Thanks for your response (again 🙂).
I checked, using ipify, and you're right that the default Apify proxy configuration uses rotating IP addresses.
I also switched to using CheerioCrawler and it seems to be stable now (as well as offering lots of additional capabilities). TBH, I had hesitated about using Crawlee, because I assumed that was a heavyweight library, intended for more complex crawling operations (whereas I currently only need to scrape a pre-defined set of URLs with static HTML). But now that I have properly read the Crawlee docs, I can see that it can be used for simple or complex tasks, with no unwanted overhead. It's exactly what I need. Thanks for your guidance.
6 replies