crawlee-js
apify-platform
crawlee-python
💻hire-freelancers
🚀actor-promotion
💫feature-request
💻devs-and-apify
🗣general-chat
🎁giveaways
programming-memes
🌐apify-announcements
🕷crawlee-announcements
👥community
FACEBOOK POSTS - PHONE EXTRACTOR
Hi everybody in there,
I need to extract phone numbers form Facebook Posts in different Facebook Pages. Also need to set the date that the Scrap will be done. For example:
Scrap Post from this FCBK PAGE: xxxxx from date xx-xx-xx to xx-xx-xx
...
Actior stops running before finishing the proposed craping job
Hi everyone,
I could use some insights here. I'm a new user of Apify, and trying the platform as of now.
I've tried using the linkedin jobs scraper for specific roles, and it stops scraping all available jobs on the search, before completing the task. I tried resurrecting the actor, but still doesn't run. What could be the problem that I should be looking for?...
Function 'validateInputJSON' finished with error! Please check that your input is a valid JSON.
I'm trying to integrate Apify with Make.com (Integromat), but when I try to run Apify in Make.com, I get this error message in Make. What's wrong? (I have a Trial plan right now)
Limitations in startUrl field
Hello guys, I wonder that if there are some limitations in field startUrls like How many of url that I can put into these field.
...
run_input = {
"startUrls": [{ "url": "https://www.facebook.com/groups/647718825333067" }],
}
run_input = {
"startUrls": [{ "url": "https://www.facebook.com/groups/647718825333067" }],
}
Details are not revealed after clicking the button using puppeteer.
I have tried below code to get some hidden details after button click. After clicking the button, new divs popped out and show some additional details in page. Then there should be 12 elements but code results show only 8 elements which already visible before button click.
const addTitleSelector = 'h1.TextHeadline3TextHeadline3Wrapper-sc-10e1s2p-0';
const vehiclePriceSelector = '.TextHeadline2TextHeadline2Wrapper-sc-1itsg3n-0';
const vehicleParameterSelector = '.ExpandableContentContent-sc-11a0rym-0 .ParamsWithIconsWrapper-sc-hanfos-0 .TextCallout1__TextCallout1Wrapper-sc-swd73-0';...
Unknown format output - Instagram Profile Scraper
Fairly commonly, probably 30% of my profile queries on each actor run, are returned with an 'unknown format'. It happens seemingly random and doesn't always affect the same profile query- retrying the specific profile query can return the expected result.
Here are a couple of screenshots, from the log and storage:...
Google Maps Scraper
Hello, how to run actor without duplicate data? i want scrape the data from Google Maps Scraper - Compass but i need the data is no duplicated data.
Scraping my own TikTok profile?
Hi,
Is there a simple Actor that would scrape my own TikTok following profiles for name, link, profile img, profile info, etc. ?...
Instagram Hashtag Scraper
Hi,
I am looking to add an Instagram hashtag scraper to my website that allows the user to enter any hashtag and get usernames, emails, and phone numbers from accounts that have posted under that hashtag.
What is my best option? ...
Integrating webhook with Fivetran?
Hi! I am a one-man data team at a startup, so we are leveraging Fivetran to do a lot of our heavy lifting for social accounts. One thing they are missing is TikTok social data, since no API.
I like the Apify TikTok social scraper you have here, and would like to store it via webhook in my BigQuery instance via the webhook connector in Fivetran.
Does anybody have any experience to know if this use-case is compatible with what's currently on offer? I'm not seeing data flow in Fivetran, but I could be making a mistake somewhere. ...
Getting "Field input.proxy is required", trying to test API (Run actor) using the console. Getting
Hello, I am testing the API via the console before I actually right code using the node.js library. One basic call I am trying to run is:
https://api.apify.com/v2/acts/shu8hvrXbJbY3Eb9W/runs?token={my_token_here}&maxItems=1000
But the response I get is :...
Google Trends - How to use
Hello guys, let me start by saying that I am new to Apify. That said, I am trying to scrape Google Trends, but the result I'm getting back is not the expected data. I am getting dataset that looks like this:
```
[{
"Term / Date": "Socks",
"May 16 at 9:44 PM": "95",...
"Users", "Success Ratio" Meanings?
I was looking at some of the statistics and was wondering what "Users" and "Success Ratio" mean. I'm assuming that "Users" means the total number of trial or paid users ever rather than currently. Is this correct? (though I do see that I can monitor paying/trial users under publication statistics, I was wondering what "Success Ratio" means. Does it mean the percent of successful runs in total or from other users? Or does it mean the percent of trail users that end up paying? Thanks.
Facebook Page scraper
Hi,
I'm using "Facebook page posts checker", and I have three issues.
1. I would like to scrape the community of a page and not just the original posts, is there a way to accomplish that?
2. I know the Groups are beta in that scraper nonetheless, you can't limit the group scrape with a date only by quantity, how can I limit by date?
3. Ever since i started using it says the scraper is under maintenance, would that be resolved soon? ...
Facebook profile scraper
Hello, I am using facebook profile scrapper actor via api gateway and lambda. The time taken is from 30 secs to 2mins. Im using free account. Is it the normal time taken or something can be fine tuned.
API scraping
Hi, want to make a bot extract all the data from an API endpoint and store it on my database to create a history of the values.
I tried making the script without Apify, but the script took ages to finish and after some request my IP was blocked.
So the solution I found is to use Apify, because of the anti blocking solutions and the concurrency.
The problem is that I want to create a custom actor, and I did not find much information about scraping an API. I don't know if this is because this isn't the right tool for my problem.
In addition to that, I would love to know how are the actors expected to work, at the moment I have a script that create a bunch of URLs for all the pages (pagination) and another that go on each page and make the request....
About "Requests Queue"
what is the purpose of Request queues storages. Does it automatically fetch request, or it just purely just for storing URLs. thanks you in advance (sorry for noob question).
Amazon scraping stopped working suddenly last Friday
I have a scraper for Amazon pages. Everything was working fine for a month. I had lots of calls per actor run and all ultimately ended up being successful. An occasional 503 was returned but retries fixed the problem. As of last Friday I am getting a 503 for most calls. I was able to improve it a bit by:
- configuring proxies and retiring sessions
- residential proxies
- more headers, manually rotating user agent, etc.
- buying more IPs...
Is Quacker affiliated with Apify?
I'm a little confused as to whether Quacker is a third party or directly affiliated with Apify, as it seems to be worked on or mentioned by Apify's official team. (official Apify youtube video for example). Any clarification would help, thanks.
Bulk image downloader questions.
The Downloader seems to work great on getting the images from the URLs but how do you know what image came from what URL? I cant match the image to the source unless this is possible.....