How can I get my data to be scrapper faster?
Good day guys! Thanks again to stop by.
I have a ecommerce scrapper design to extra over 200k data. So far, I been testing some (Seller's front page) only with about 7k on data. Well, for those small seller the avg time to get the data is about 20 minutes using proxies or apify. I am wondering if there is a way to get data faster? Using celery? server local host? Any help will be appreciate it! Thanks
2 Replies
genetic-orangeOP•3y ago
Any suggestion will be appreciate it!
conscious-sapphire•3y ago
1. Apify allows you to boost memory or split the URLs accross multiple actors
2. The most important is how you scrape. Try ot use internal APIs or HTML, not browsers