CA
optimistic-gold
How to ignore already scraped tweets?
When scraping Twitter. Is there a way to configure Apify such that already scraped Tweets are ignored when a scheduled task runs again?
4 Replies
metropolitan-bronze•3y ago
I don't think the twitter public actors support it by default, will pass on the idea to the team (and will double check to confirm that it's really not possible atm). Some twitter scrapers have an option which could be a workaround for now - something like
Tweets newer than
Yeah, just got a confirmation that it's not an option at the moment...optimistic-goldOP•3y ago
Thanks for the response. Just to confirm, you're referring to the optional array listed on this page (https://apify.com/quacker/twitter-scraper/input-schema#mode)? Is there any way to feed twitter lists into this scraper? Or would I have to feed in the individual URLs of different members of the list?
Apify
Input · Twitter Scraper · Apify
Scrape tweets from any Twitter user profile. Top Twitter API alternative to scrape Twitter hashtags, threads, replies, followers, images, videos, statistics, and Twitter history. Download your data in any format, including JSON and Excel. Seamless integration with apps, reports, and databases.
metropolitan-bronze•3y ago
Apify
Input · Twitter Scraper · Apify
Scrape tweets from any Twitter user profile. Top Twitter API alternative to scrape Twitter hashtags, threads, replies, followers, images, videos, statistics, and Twitter history. Download your data in any format, including JSON and Excel. Seamless integration with apps, reports, and databases.
metropolitan-bronze•3y ago
As for the lists - well -
searchTerms is an array - so you could provide an array of search terms there.