Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻devs-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

national-gold

6/22/2023

Time Frame in Instagram Scraper, how only posts from "Today" ? It is not possible to set "today" ?

So how can I do so, thank you so much for your help.

optimistic-gold

6/21/2023

Very long scrapping for Twitter since 1.0.168

Hello, I have some Twitter scrapping tasks which have been very long since the release of the actor version 1.0.168 Is there an issue with it or is there a new Twitter feature which makes it more difficult to scrape ?...

foreign-sapphire

6/21/2023

Get New Reviews From Airbnb

Hello, I want to get new reviews every day ( or when a new review is created) from Airbnb(30 URLs) and send new reviews to the Slack channel. I used an Airbnb scraper actor. Of course, this returns all reviews. To send messages to Slack, I was going to use zapier. But the output of the apify actor on zapier is an object array and I can't format the output data....

genetic-orange

6/19/2023

Why did the requests stop?

Why did the script stop? ```{ "caseSensitive": false, "keywords": [ "Oberoi"...

genetic-orange

6/19/2023

using google maps scraper

Here's a quick demo of using the scraper to get cheese shops in Paris. https://youtu.be/Sf5mpLrUtYg

fascinating-indigo

6/18/2023

Scraping Instagram to Find all Profiles in a Niche Using Keywords

Is there a any way to search Instagram and find all the profiles that contain a keyword, such as "dentist". I see there are Instagram scrapers that start with a list of Instagram URLs/profiles, but I don't know the profiles I'm looking for, so I'm kinda stuck on square one.

typical-coral

6/17/2023

Puppeteer actor dockerfile via nixpacks?

Hello, I'm trying to build a typescript puppeteer crawler with nixpacks, I can't seem to get the puppeteer dependencies to work how they should. Build completes just fine.. Here's my nixpacks.toml file:...

adverse-sapphire

6/17/2023

Twitter Scaper Date Range

Hi everyone, I am new and I am using the free version of Apify for Twitter Scaper. I am trying to use rage date absolute to extrapolate some tweets, but it does not find any results. Removing the date range, on the other hand, returns me results from today until the previous days, stopping however at about 3500 results. What am I doing wrong? Are there any limits that I am not aware of? Thank you very much for your help in advance. ...

quickest-silver

6/16/2023

Volume Discounts

Hi all, Im trying to estimate my usage cost but i couldnt understand what this Volume Discount corresponds to. Why am I getting this discount? Will i get this discount if i upgrade to "Starter" as well? Would be great if you can give a little bit more information how this Volume Discount is used....

extended-yellow

6/16/2023

Can I use streams to upload video to Apify storage?

I am using screen recorder (https://github.com/prasanaworld/puppeteer-screen-recorder) and I want to be able to save it upon finishing the run. Can I use Apify storage for it?

genetic-orange

6/15/2023

keywords extractor has limited results against known positives

I'm testing KW extractor. I know 2 of the 4 sites have the target string but KW extractor only returns 1 positive. What do I need to change in my actor settings? Here's the video: https://www.youtube.com/watch?v=bGKr0t5P0P8 thank you...

genetic-orange

6/15/2023

best approach to scrape * sites with * strings?

Hello fellow Apifytes! I have 10 sites (for example). I want to scrape each site for any mention of 10 specific strings. Which 'ACTOR' would you recommend for this task?...

fascinating-indigo

6/14/2023

How to call apify actors from a separate server?

Context: - I'm currently using playwright on my nextjs api routes - these playwright scripts scrape a website & persist the data in my database (postgres) - since I need IP roration with session management though, I'd love to offload the scraping to crawlee - I'm considering apify as the deployment platform as this seems to be the recommended setup and apify apparently also supports webhooks -- e.g. can I get notified about the status of my actor run, such as job.started or job.finished events?...

stormy-gold

6/14/2023

please i need help because i work with Amazon Reviews Scraper in apify

please i work with Amazon Reviews Scraper in apify because i want to build app depend on Amazon Revi

stormy-gold

6/13/2023

Apify + Make no code

Hi! I´m not a dev, just a teacher and I´m trying to use Apify and Make (nocode) to use the data in Google Sheets. But, i cannot get the same database that the actors create. How can i start a task, use the database created and send to google sheets?

quickest-silver

6/10/2023

Twitter - Following Users

Hello friends I would like to extract the list of followers/following of a certain user on Twitter. Is this somehow possible or should I look for other solutions than Apify?

extended-salmon

6/10/2023

Reduce execution time

Hello, My site already has a good score on gt metrix, I use apify to scrape instagram data, the problem is that it takes too much time.. Click on buy follower, enter your instagram handle and wait until you see your profile picture. It takes between 9 and 20 seconds. I would like it to be immediate like my competitor....

like-gold

6/9/2023

Build Asynchronous pipeline

Hi There, I'm building a pipeline with Apify, but I want to run many jobs in parallel then later I will check their status and export completed jobs. is there anything I can use to export the data of any job using job id or something ?...

extended-yellow

6/8/2023

Static Proxy Provider (Alternatives to Apify)

Looking for an unmetered dedicated static proxy list provider with best possible subnet entropy (ie ideally to have each IP with different first 3 octets). I would appreciate if you could suggest or point out reliable choices on the market. We have tried the following options with little success:...

optimistic-gold

6/8/2023

Keboola Input Mapping and TikTok scraping

Hi, I want to scrape data from TikTok (@clockworks/free-tiktok-scraper) and use Keboola for that. That's why I'm using the Keboola Input Mapping actor @Deleted User/keboola-input-mapping. I have set up the Keboola Input Mapping actor as instructed in the information. Unfortunately the run ends with an ERROR error Actor run might be metamorphed only once. I have no idea where the error means. Could someone please advise? Run log: https://console.apify.com/view/runs/UzPqDQwCSDdrnpggp

Previous Next

Gaming

Programming

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Crawlee & Apify

This is the official developer community of Apify and Crawlee.