crawlee-js
apify-platform
crawlee-python
💻hire-freelancers
🚀actor-promotion
💫feature-request
💻devs-and-apify
🗣general-chat
🎁giveaways
programming-memes
🌐apify-announcements
🕷crawlee-announcements
👥community
Scraping Threads Timeline. But Not able to get entire data.
I am making a scrapper to scrap threads.net using playwright and python and willing to host on Apify.
But It I am not able to scrap entire timeline. It is only giving some data and when scrolling down, I am not able to scrap the new data.
Any help or suggestion is highly appreciated....
Issues with Limited Results from YouTube Shorts Scraper
I want to scrape YouTube channel data, but I'm consistently receiving only 48 results, while on the target account are much more videos. MaxResult setting is empty .This issue persists even after switching from the 'youtube-shorts-scraper' to the 'Fast Youtube Channel Scraper'. I'm not sure if this is due to API limitations, scraper settings, or my usage approach.
I'm new here so appreciate a response how to get more results....
Faster Website Scraping
We are just testing Apify's Web Crawling API right now in a free account and will subscribe shortly.
Right now it is taking 57 seconds for 11 pages using Cheerio. My question is, will this be faster when we subscribe and get higher memory? Also are there any other suggestions to increase speed? What is the fastest method you have seen and how long should it take? This is a tad slow for us in production without some user experience changes....
TikTok Scraper Not Yielding More than 20 results
I've been using this scraper to scrape video urls from a TT profile, but it will only yield a max of 20 results. Is this a cap set by Apify or TikTok?
Scetchy scrolling distance (apify/web-scraper)
Hi,
I'm using the apify/web-scraper to scrape a website. The pages don't initially load all the content, you need to scroll down to see more.
Now i have set the scroll distance setting to the highest number possible but still it only works half of the time....
Using Output From One Actor As Input For Another
Curious if I'm missing something or not, but it feels like there should be a way to access the output from one actor, and use it as an input for another.
Specifically, let's say I want to connect the YouTube Channel Scraper (https://apify.com/streamers/youtube-channel-scraper) to the more generic YouTube Scraper (https://apify.com/streamers/youtube-scraper), which gives more details on videos than the Channel Scraper.
The Channel Scraper has as part of its output a list of video URLs, but I can't understand how to feed that in as an input to the generic Scraper, which asks for a text file / list of URLs. Is this even possible?...
Scrape app ratings from Google Play
Hi all, I'm very new to this community and I don’t have experience with coding or similar skills. I need to compile a list of apps from the Play Store with ratings less than 3.9 (excluding those without any reviews). Is there a step-by-step guide available that would enable a non-tech person to do this with ease? thanks!
tiktok scraper date not working
Hey I’m currently using the titkok scraper for finding viral videos relating to onlyfans, I’m trying to find videos from 1-2 days ago as most viral and using hashtags for this in seeing that the hashtag feature is working but not the date and I’m having to manually search the dates in inspect element any fixes?
Integrate google maps scraper actor with google maps email extractor
Hi, i'm not techy and I need to scrape whole data email included about bussines but the actors in apify are separeted and the only solution I think is better is to integrate both of them
I appreciate some help, thank you...
Pinterest Scraping Help
Hi!
I am using the pinteres scraper but it allows me only to scrape pinterest profiles. I am trying to scrape pins from a search.
Any idea on how to solve this?...
facebook comments scrape returning half of comments
Hi there,
I have been using the Facebook comments scrape to get comments on certain posts. The comment scraper only returns significantly fewer comments than the post has and than I specifically in the settings.
for example:...
SSL Error
Hi Guys! I'm getting SSL error when I put TripAdvisor URL, if I put other URL that works fine, please help me out.
GPT's Action - Error
Hello everyone, I'm trying to do this tutorial. https://blog.apify.com/add-custom-actions-to-your-gpts/#limitations-of-actors-and-gpts but I've a constant error on GPT that say "Error saving draft" 😦
Google Maps Scraper Question
Hello i'm scraping google maps using the compass tool. My question is lets say i want to scrape all restaurants in the state of california.
To do this would i just put “california” in the “location” field and then it will automatically scrape every single city place in the state of california also with the zooming feature on each individual place so it doesnt miss any.
Or...
Playwright - state.conditions.includes is not a function
I've done a new load to create a new actor using
npx crawlee create app, then selecting the Playwright Typescript template. Once all npm packages have been loaded I then cd into app and tried npm start. I get the below error, I then tryied to download the crawlee playwright packages npm install crawlee playwright however I still have this issue trying to run the actor. I have not modified the template that is provided when using npx crawlee create. Any help or guidance is appreciated.
node_modules\typescript\lib\typescript.js:41322
if (condition === "default" state.conditions.includes(condition) isApplicableVersionedTypesKey(state.conditions, condition)) {
^...Actions in GPTs
Hi,
I hope you doing well.
I try your new custom actions in GPTs,
I followed exactly this instruction but it doesn't works.
https://blog.apify.com/add-custom-actions-to-your-gpts/...
Not able to link Github organisation with Apify organisation
Hi,
I'm having issues linking our Github organisation/account with our Apify organisation.
I have our organisation selected in Apify, go through the process of setting up a new Actor and linking our Github organisation but I get redirected to https://console.apify.com/page-not-found after I have completed the Github step in the popup that opens with the message:...
is there's any coupon for free credit other than the satisfaction one?
I wanna test out the platform a bit more before I get a subscription, so it would be great if there's another coupon
thanks!...