Help for a Instagram data collection
Dear,
I’m trying to download Instagram posts with Apify thanks to “Instagram Post Scraper” (https://apify.com/apify/instagram-post-scraper). I wrote a Python code to automatize the pictures collection. Unfortunately, a lot of pictures URL are expired well before the 7 days of data retention. Could it be crashing because I tried to download more posts than allowed with the free trial or that my storage memory is overflowed ?
The second question is: if I pay for an upgrade subscription, can I be sure that my storage will increase according to my data size? I don’t want to pay for a subscription and that the downloaded URL are expired way too soon.
Have a great day,
Tiago
8 Replies
flat-fuchsia•3y ago
Hi @tiagojoseph, I checked the Instagram post scraper and the actor doesn't store scraped images in the Apify Storage. An upgraded subscription would unfortunately not help, as the expiration is happening on the Instagram side.
wise-whiteOP•3y ago
Hi! 🙂 Oh, oki, thanks a lot! What should i do, so? Dowload little by little?
flat-fuchsia•3y ago
The expiration of Instagram URLs is around 24 hours (might be more or less), not sure how big scrapes are you doing, but scraping in smaller chunks might help.
wise-whiteOP•3y ago
Ok, thanks a lot! 🙂 I deeply thank you for your answer!
xenial-black•3y ago
if you need help downloading images and store them let me know, i could help
@Moazzam Malek-SCRAPING EXPERT just advanced to level 2! Thanks for your contributions! 🎉
wise-whiteOP•3y ago
Woow, that could help me actually! 🙂 Right now everything is ok, just the data collection is really really slow with my python code. Do you have any adivce (one data collection at a time or on the opposite, several in the same time? ) 🙂
xenial-black•3y ago
how many you are collecting at a time?
if you can switch to nodeJs, you can do parallel execution