crawlee-js
apify-platform
crawlee-python
💻hire-freelancers
🚀actor-promotion
💫feature-request
💻devs-and-apify
🗣general-chat
🎁giveaways
programming-memes
🌐apify-announcements
🕷crawlee-announcements
👥community
@HonzaS I understand there's no
@HonzaS I understand there's no guarantee from the users' perspective, but things like good reviews, quick response time, solid documentation, etc., are all positive signals. No one is being forced to set up their actors that way, but having the option to do so is valuable.
That said, I was just using that as a basic example to explain my point: from actors' developers perspective I believe we should offer free users enough convincing value while still encouraging them to subscribe.
From an actor developer’s perspective, it can be frustrating when you're using paid third-party APIs, and a free user comes in, scrapes 5K requests, and never returns(or when he does, he just creates a new Apify account again)...
Is there a way to extract all list of
Is there a way to extract all list of runs like this one for example. I wanted to extract like 1 year of runs data to do some cost analysis for my companies like average per runs.
There is data on CPU Usage, time, etc, but not on each runs.
Let me know if there is already a feature like this, tried to search but found nothing....
Hey there, it would be really useful for
Hey there, it would be really useful for the puppeter and playwright crawlers to be more considerate of their memory state while running. Ive had lots of occasions where crawlee ends up in a fatal OOM state where the autoscale pool tries to scale up, even when there isnt enough memory for the browser since the node process itself isnt consuming much RAM. eg, in a container with 4GB of memory, if node takes up 200mb and the browser takes up the remaining 3700mb, autoscaled pool would still try to...
Navigation to download handling
a better way to handle navigations that directly cause downloads such as the download button on this page: https://dca-global.org/file/view/12756/interact-case-study-cedaci
Playwright throws an error along the lines of net:ERR_BINDING_ABORTED
The best solution I have come up with is to set up a promise in a preNavigation hook that listens for download events and then handle them in the errorHandler but this obviously isnt ideal...
Feature requests
Feature requests
If we can sort the actors by the number of open issues it would help us to prioritise the fixes
Alert when paying users are reducing at an unusual rate for an actor
I don't see any use in total "Views" metrics on top of analytics page, but for individual actors, this will be meaningful to know...
Hello everyone, Is there a way to avoid
Hello everyone, Is there a way to avoid scraping same pages even if the crawler is restared ??? because I'm currently working on a news website crawler, However, with each run of the scraper, I'm encountering up to 80% duplicated news from previous runs. Any suggestions on how to address this issue effectively?
Hi everyone! I'd like to request a
Hi everyone! I'd like to request a feature for both organizational and personal accounts: the ability to reset proxy passwords directly through the interface. This feature would greatly enhance security, allowing users to quickly respond to potential breaches by updating their proxy credentials without delay. (my case after the password got leaked 😦 )
Thanks for considering this enhancement!...
also another feature request: some way
also another feature request: some way of private rental or way to allow run actor without sharing it's code etc.
context: I've got few orders for create private actors and I would love to share them just as runnable unit, so I can secure payment and reduce risk of being scammed...
🆇 Twitter Scraper · Apify
https://apify.com/shanes/tweet-flash anyone know what happened to this one?
Maybe I missing on the app, but why I
Maybe I missing on the app, but why I can't find actor calling snippet(py js ) on the actor page instead of digging in the docs...👌 It would be usefull if we have ready snippets with our keys....💡
@Lukas Krivka Hi. Sorry to post this in
@Lukas Krivka Hi. Sorry to post this in this channel, but my account was disabled for some reason, and I haven't heard back from support in a couple of days. Was wondering if I could get some info through another channel.
For the Airtable Exporter, is it
For the Airtable Exporter, is it possible to allow creating a new table every run? Instead of adding to a single, static table
Hi there, it would be great if i could
Hi there, it would be great if i could mark already checked actors and that i have skipped it. While searching in the store it occurs to me that i'm checking some actors twice because i can't remember this one. Have a nice day!
PS: Btw, why is there no Facebook actor that fetches likes and followers? I'd rent it right away.;-)...
as feature request I would like to see
as feature request I would like to see content of my agents publication statistics in one view
I want to be able to scrape an IG sound
I want to be able to scrape an IG sound for views, likes, comments etc.
Not sure if this is a feature request,
Not sure if this is a feature request, but it's an idea I implemented and there doesn't seem to be a better channel for general chatting.
When using Scrapy, there's this concept of Items. I was able to extend Items so that they contain declarative specification of Apify input types (see code):
```python...
hello. isnt it possible to retrieve last
hello. isnt it possible to retrieve last connection time within a whatsapp number? im using the scrapper, but unfortunately i only get profile pictures, and a boolean which i dont know what i means, but comes in false always