Google Maps Scraper
Hello, how to run actor without duplicate data? i want scrape the data from Google Maps Scraper - Compass but i need the data is no duplicated data.
5 Replies
conscious-sapphire•2y ago
I'm not from the team, but if you're worried about duplicates (and not efficiency issues resulting from duplicates) then you should be able to filter the dataset in the coding language of your choice. There are also no-code solutions for removing duplicates (google remove duplicates from csv online) which should work as long as each duplicate has all the same elements and values as the original.
Here's one you could try, but any online should work: https://www.anyjson.in/csv-remove-duplicates
Simplest Online CSV Duplicate Remover | Remove Duplicate Rows
Use CSV Duplicate remover Tool to remove duplicate rows from csv data using this simplest online tool. Just upload or paste your CSV file and we will remove duplicates from csv
national-goldOP•2y ago
So the solution is make duplicate check on my apps ? but i worried about the "price" if i make another query from api if there any duplicate data 😄
conscious-sapphire•2y ago
Yeah sorry, if you're concerned about the price then it's up to the dev to work on. Compass seems to fix most issues listed on the issues page, so if you haven't already I'd recommend adding your issue there:
https://console.apify.com/actors/nwua9Gu5YrADL7ZDj/issues
national-goldOP•2y ago
Thank you Jonathan, i will check it.
rival-black•2y ago
Actually the scraper should not produce any duplicates within the same run by default.