Instagram hashtag scraping doesn't return all the posts
I've been trying to use the Instagram hashtag scraper but after several scrapes I'm finding it only returns half the results and also returns multiple duplicates. I've upgraded my subscription and added IPs and Memory but still am not getting even half of the actual posts.
12 Replies
deep-jade•2y ago
Thanks for feedback! I quick check 500 results https://console.apify.com/view/runs/ePlFEfazw20agvJoZ and only duplicates confirmed: https://console.apify.com/view/runs/5TlzzqZ4RP5koiqYA
Can you please share run or full input?
unwilling-turquoiseOP•2y ago
duplicates tend to be missing some fields but if you sort on the URL field you can see which ones are
unwilling-turquoiseOP•2y ago
You can see the duplicates here don't have data for the 3 name fields but the post URL's are the same
@moz just advanced to level 1! Thanks for your contributions! 🎉
foreign-sapphire•2y ago
@moz could you please share the run id, so that we could check the input, build, and other details directly?
unwilling-turquoiseOP•2y ago
I did i sent it directly to Alexey
Hey guys any updates on this?
correct-apricot•2y ago
I am having the same problem. When trying to scrape all the posts of a given hashtag the actor says "finished" even if I can tell that by the number of results it hasn't made it all the way through. Inspecting the output, I can see a large number of duplicates (with varying degrees of completeness) and only data from the most recent months. When trying the same request multiple times sometimes it will get further than others. Any guidance here??
foreign-sapphire•2y ago
@Alexey Udovydchenko did you find out anything?
deep-jade•2y ago
Instagram migrating to new endpoints and applying harder data limits. Actor updated today and should work stable, however with less posts per profile or hashtag (expected hundreds at max). Will keep you informed about the progress.
correct-apricot•2y ago
I will wait for further updates and get back to working on my own scraping code; the functionality is pretty limited for my application if the actor only returns a couple hundred results
unwilling-turquoiseOP•2y ago
it would also be good to have a date range that can be set
I just ran it again, only got 261 results. Prior I was getting at least 2,000 once duplicates were removed.
evident-indigo•2y ago
Did anyone figure this out? I am also receiving a large number of duplicate records when calling the instagram-scraper. I have tried to extend the output function as follows:
but continue to return duplicated records
