Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻devs-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

Louis Deconinck

2/7/2025

PPE charging

Is there any way how I can verify what the user will be charged when doing a run of a PPE (Pay Per Event) actor? How can I verify that the charging is set up correctly on my end?

Louis Deconinck

2/6/2025

RESIDENTIAL5 proxies

In the past I sometimes used RESIDENTIAL5 proxies, which I believed to be even better proxies than the regular RESIDENTIAL proxies. However, as of late they stopped working. Has anything changed in that regard? My scraper does no longer work and regular residential proxies keeps it blocked.

quickest-silver

2/5/2025

Free Trial User Abuse

Hello, what are some ways to prevent users from taking advantage of the free trial to get all the data they need and no longer using the actor. I had recently changed the trial duration from 3 days to 2 hours. I think I want to lower the trial period again but I can't until some time later due to having changed it recently.

genetic-orange

2/4/2025

Warning: Detected unsettled top-level await

Encountered below issue while deploying to Apify platform using apify push. This action is being perform in Github Actions. ----- Warning: Detected unsettled top-level await at file:///opt/hostedtoolcache/node/23.7.0/x64/lib/node_modules/apify-cli/bin/run.js:17 await execute({ development: false, dir: import.meta.url });...

secure-lavender

2/3/2025

Append data to an existing dataset

Is there a way to redirect the output of multiple runs of the same scraper to the same existing dataset, appending the new records? The order doesn't matter. Due to the limitations of the scraper I am using, I need to perform thousands of runs that produce a very small amount of output that I would like to add to an existing dataset (obviously having the same format or schema). I skim through the Apify API documentation and I did not find anything about it.

vicious-gold

2/2/2025

How to Share Data Across Public Actor Runs?

Is there a way to fetch data from a centralized source and update it when needed in public actor runs? What is the best way to manage this within Apify mechanisms without using external services (AWS DynamoDB, Firebase)? Since runs executed by each user in public actors occur in an isolated environment, the default KV store obtained using Actor.get_value() and Actor.set_value() is unique to each user. The share feature for a KV store created with a specific name is only applicable for specific cases where a username, etc., is provided. Is there a way to make this available to all public actor users?...

ratty-blush

1/30/2025

scraping continuing to next set of leads after a new run?

hey guys, not the most advanced apify user so i need help with scraping leads. Issue is i crape the max 5 k leads then try to restart the scraper and it rescrapes the same 5 k leads. how can i get it to scrape the next 5k leads

multiple-amethyst

1/30/2025

Is it possible to setup a slack notification for a low account balance?

We're using the proxy feature and our usage is somewhat difficult to predict, we'd like to either be notified via slack when our account balance goes below a given threshold OR we'd like to setup automatic account balance top offs. Are either of these possible?...

correct-apricot

1/29/2025

do I need to signup for a paid plan, if my requirement is only one time?

I can just use the free plan and refill some credits for pay as you go ? Becuase my usage is only one time thing.

!!!Joefree!!! 👑

1/29/2025

console WEBUI cosmetic BUG: icon with black border

When using a transparent icon for the actor (WEBP or PNG images), an unexpected black border appears (on Google Chrome 80% Zoom)

wise-white

1/26/2025

Apify cli can't find python

When I use apify run it say python can't be detected. It's installed and it's in PATH variable and everything and works from cmd and powershell like charm. Also, I updated node and npm to the latest version and reinstalled apify-cli

exotic-emerald

1/26/2025

Apify Proxy not working with https urls

I want to test the apify proxy and how it works to integrate it with my py code. Running a very simple check I found it's not working with https urls. here's a snippet: ```py import asyncio, httpx from apify import Actor...

stormy-gold

1/23/2025

nested tranformation

What is wrong with my transformation? everything under physicianInfo is not beeing displayed on joboverview...

message.txt

sensitive-blue

1/15/2025

Actor pay per event

Hi, I've seen mentions of a "pay per event" pricing model https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event & https://apify.com/mhamas/pay-per-event-example, but can't find how to use it for one of my actor, i only see rental or pay per result options. How can we use this pay per event pricing model?...

absent-sapphire

1/14/2025

Google Lens - extract product name

Hello, I would like to ask if any Apify tool can, for example, find a similar image - https://i.postimg.cc/KzRHFKQc/55.jpg and extract the product name from the links to CSV. We can use Google Lens? I want to use this to automatically name antique products. Thanks for the all informations and help! 👋...

multiple-amethyst

1/8/2025

403 response for new account when using proxy

I'm attempting to validate that the proxy works and am not having luck, should I expect the following to work? ``` ~ λ curl --proxy http://proxy.apify.com:8000 -U 'groups-RESIDENTIAL,country-US:apify_proxy_redacted' -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36" https://httpbin.org/ip curl: (56) CONNECT tunnel failed, response 403...

equal-aqua

1/8/2025

I Can't public my actor

my account: https://apify.com/wudizhangzhi actors: https://console.apify.com/actors/KAkfFaz8JVdvOQQ5F/source Error: Operation failed! (You currently don’t have the necessary permissions to publish an Actor. This is expected behavior. Please contact support for assistance in resolving the issue.) ...

exotic-emerald

1/7/2025

Where to get Personal ID or Business ID?

Guys im new to apify and i want to publish my newly built job scraper but when i setup monetization there are two business id and personal id option, where can i get this?

stormy-gold

1/7/2025

Google Maps crawler - Increasing place limit after initial run

Hi everyone, I recently ran a Google Maps scraper (https://apify.com/compass/crawler-google-places) to collect place data, and I've discovered that there are many more places available than what was initially collected in my first run. Current Situation: - Successfully completed an initial scrape - Have collected data for X places...

rival-black

1/5/2025

Apify - IG Data Extractor

How can I export the data acquired from the Instagram Profile Scraper (Apify) on autopilot back into google sheets everytime an actor runs whilst only importing specific data (such as Full Name, Biography). I just started my journy in coding and its incredible! thank you so much in advance

Previous Next

Gaming

Programming

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Crawlee & Apify

This is the official developer community of Apify and Crawlee.