Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻devs-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

raw-harlequin

7/5/2024

Crawler not very Efficient

I'm using the apify actor that crawls website contents, but it hasn't been very efficient some website weren't crawled at all. I need help

extended-salmon

7/4/2024

Google drive Integration not working

I am using an APIFY actor that scrapes twitter but since yesterday I'm trying to send the results to google Drive but I kepp getting the error you can see here. Is there any reason ?

sunny-green

7/4/2024

Are we able to split apify account on dev, and prod enviroments?

Hey we would like to have an option to split an account on dev, and prod, to be able to track which runs are on dev, and wich are on prod, except that, we would like not to pay twice for actors renting. As a pros, we would like to be able to see how much money we burn on each env. How we can do that?

national-gold

7/4/2024

Error in Cost calculation

Hi i see one of my runs costed all the available balance. i was expecting only 7USD but it charged me all the amount in my account 46USD. the total records output is 3K. but it wrongly saying 20K. this caused to calculate the bill wrongly. to whom i need to contact for this

unwilling-turquoise

7/3/2024

WARN ProxyConfiguration

I'm gettign the following error: WARN ProxyConfiguration: The Apify Proxy password you provided belongs to a different user than the Apify token you are using. Are you sure this is correct? Any ideas how to fix this error? Thank you...

deep-jade

7/1/2024

Is there an Actor who knows how to get me a list of followers from a certain profile?

Hey friends I have an excel with 10 profiles that I want to pull the followers of each such profile into one excel. Do I have such an option in Apify? Is there such an actor already ready? Thank you!...

other-emerald

6/30/2024

Need to have an identifier to link back scrapped data to main database

Hi, I am currently using Apify scrappers to scrape IG account information and FB page ad library information. The goal of using the scrapper is to get information about my leads and import back that information to my main leads database. However, depending on the scrapper, I sometime don't have an "identifier" to link back the data from the scraper, back to the correct lead in my main data base. The way I link the information I scrapped back to the right lead is using "lookup". for example: I look up the domain url in both my main database and the scrapped data to link the right information to the sheet....

fascinating-indigo

6/29/2024

Get dataset Make.com error

Hello, I want to get the results of my actor when I run it via make.com, but when I retrieve the dataset items, there’s no way of knowing when the actor will be done. My only solution is to make the actor wait the maximum amount of time it could possibly take to run, but as you can imagine, this is very ineffective to use in production....

sharp-indigo

6/26/2024

Actor insights page blank

When I try to go to the insights page for one of my actors, the page is blank. Empty charts flash on the screen initially but then disappear.

rival-black

6/25/2024

Unavailable to open any actors in Apify

I just opened my account and none of them is available to use. It pops up the message (see image).

!!!Joefree!!! 👑

6/22/2024

Actor Static DNS

I just realized some Actor maintained by Apify could have static DNS address example: https://super-scraper-api.apify.actor Is my Actors also have this DNS name ? If yes it could make things easier, for example the OAuth Authentication process Thank you. 🙏🏻...

!!!Joefree!!! 👑

6/22/2024

BUG Report 2024-06-22: Use custom SEO details

I set my Actor "Use custom SEO Details", and saved successfully. But when return to the setting page, its return to unchecked....

flat-fuchsia

6/21/2024

apify telegram group adder

ive been trying to run this scraper and group adder and it worked for like 5 invites, then it got banned right away.. anybody got an idea on how to setup this in a good way?

foreign-sapphire

6/20/2024

Apify console not working

Console just keep loading post auth

equal-aqua

6/19/2024

Pushing Multiple tables

Hi everyone, I have an actor that scrapes a table from various URLs. The table format varies across different URLs. Currently, I'm consolidating the data into a single table in the output. Is there a way to push the data into separate tables for each URL, similar to an Excel sheet? I've tried using the storage option, but the user prefers having the tables in the output instead....

foreign-sapphire

6/19/2024

Twitter Like Scraper

I came across a "Facebook Likes Scraper" I am looking for something like this for twitter. I would like a scraper from which I can get a all post related to a profile and some basic information like username of the accounts that like, share and repost the post . I have researched a lot and have not come across one if there is one I can use please guide me to it .

absent-sapphire

6/19/2024

Website Content Crawler Execution Speed

Hello, in the website crawler content actor (Apify developed), it's often taking longer than 2 minutes to scrape only one page. I'm wondering if we change the input param "requestTimeoutSecs": 60 does that force the actor to go faster? What other levers do we have to control execution speed. ...

absent-sapphire

6/18/2024

SSO is not working

SSO is not working when attempting to sign in to the platform. When selecting sign in with github, nothing happens.

metropolitan-bronze

6/17/2024

Help with Apify API: Getting Input Schema or OpenAPI Schema for each Agent.

hi Devs. anyone knows how can i get the input schema of an Actor? i've being trying this for several months with no success 🫠(Also works the OpenAPI Schema). I'm building an AI app that use Apify as backend for tools, obviously the users need to create an Apify Account for making this work. i know i need the ActorID, but from there i don't know what to do because when i want to Run Synchronous this agent i can't because its asking me for that input schema that i don't know where to find. The best result its a way to getting the openapi schema or input schema for each Agent using user's credentials....

rising-crimson

6/16/2024

customData in input isn’t going through to output

I am using google-search-scraper and am successfully defining some customData which is going into the input How do I get google-search-scraper to return the customData in the output?...

Previous Next

Gaming

Programming

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Crawlee & Apify

This is the official developer community of Apify and Crawlee.