Crawlee & Apify

CA

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻devs-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

Hi,

Hi, I’m using APIFY to scrape reviews for a business that has around 500,000 reviews. I’m running the scraper daily, but I don’t want to re-fetch all 500K reviews every day. Most scrapers on APIFY don’t offer cursor, offset, or limit parameters....

+200 free users resulting in 0 paid users

Maybe a rental type? Pricing perhaps? $30. I have other actors with lower prices like $9.99, but they haven't resulted in anything. Use case? I created an open-source React Native app showing examples of how to use the data, even added AI. That's why I'm asking if anyone really makes money on Apify, or if it's just Apify making money with its services and super actors....

Some questions :)

Hello guys, Hope your going well. I'm just starting to use Apify for my company and getting some reviews of different websites. Glassdoor is the 1st one that I will implement and I would like to use this actor: https://apify.com/memo23/glassdoor-scraper-ppr...

Hey everyone! I’d like to know if I can

Hey everyone! I’d like to know if I can use Apify tools to create a SaaS, especially those that scrape Google (like Google Maps Reviews Scraper). Could this cause legal issues or problems with Google? Thanks for your feedback! 🙏

Sorry, I'm fish out of the water when it

Sorry, I'm fish out of the water when it comes to discord. I'm and IRC guy. I will figure out how to create a thread. Thanks.

I'm having hard time using apify proxy

I'm having hard time using apify proxy with playwright scraper. I keep getting net::ERR_TUNNEL_CONNECTION_FAILED although I passed the server, username, password correctly from ProxyInfo object 😢

Marches online

I want to set up an automation using make.com to scrape this website: https://www.marchesonline.com/appels-offres/en-cours. I would like to use the Apify module, specifically the "Web Scraper" actor, to perform the scraping. Specifically, I want to connect the actor to the site, which will click on each title (1,392 notices in total), scrape the data on the title's page, then go back to the main page and click on the second title, and then scrape the data on that page. It should do the same for...

hello 👋 I have a README.md file where I

hello 👋 I have a README.md file where I added a collapsible JSON code inside HTML, working locally (in my IDE) but not on Apify, any clue (I see the JSON inside the readme content in network tab, but there's no console errors I could find)?
No description

what do you mena by variable control?

what do you mena by variable control? the page load correctly both in normal chrome and headless puppeteer running locally. however, in Apify Actor without proxy, i get the bot dected message, and in Apify Actor with proxy, i get the Error Tunnel Failed. This is the page in question: https://a836-acris.nyc.gov/CP/LookUp/Index

I need some help with lead generation

I need some help with lead generation and i have questions, my english is bad, ty🙏

Hi All. Trying to test a Make automation

Hi All. Trying to test a Make automation for someone before I subscribe for them, but getting this error. Can you no longer access the API on a free plan? (status FAILED statusMessage You cannot use the API with the Free Plan. Please subscribe to a paid plan on Apify. )...

x-sap-sec

but shopee replace header with x-sap-sec key

hi everyone, a bit new here - just

hi everyone, a bit new here - just wanted to ask how everyone sorts data to match each row?

it's missing comps which i need. also

it's missing comps which i need. also coulnd't figure out how to search by address which is what i need, not search by map

Hi Apify Community,

Hi Apify Community, I successfully ran the LinkedIn scraper in the Apify environment and got the correct JSON response. However, I’m facing issues replicating the same in my local VSCode environment. I’ve followed the steps to pass the LinkedIn cookies (exported using the "EditThisCookie" extension) into my input configuration, but I'm still getting an error: "Input is not valid: Field input.cookie is required."...

I want to fetch some JSON with an

I want to fetch some JSON with an HttpCrawler and then from the JSON fetch a url with CheerioCrawler. How to combine these two?

Hi guys,

Hi guys, I tried using Apify linkedin job scraper but I can't scrape all the results. it only scrape 1000 results why? I used curious coder Linkedin scarper...

Hey Guys anyone able to help me with

Hey Guys anyone able to help me with scraping facebook business page followers of the pages scrape all the emails from the folllowers

Hey guys. why do i get the same url for

Hey guys. why do i get the same url for different sessions? what am in doing wrong? import json import random...
Next