Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻devs-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

dependent-tan

3/9/2025

Error running actor FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.

I'm trying to run my actor & it's giving this log: ``` 2025-03-09T00:13:41.538Z ACTOR: Pulling Docker image of build 20IgkKFk3QAzeFbk9 from repository. 2025-03-09T00:13:42.170Z ACTOR: Creating Docker container. 2025-03-09T00:13:42.237Z ACTOR: Starting Docker container....

adverse-sapphire

3/7/2025

Request options are not valid

Getting an error for a basic crawler when passing in my starting arguments. It says it the input must contain "url", which it does already. ```...

genetic-orange

3/6/2025

Tracking custom stats

Is there a way to add custom stats to a requestQueue? I know there is a key/value store for each run, but I don't see a solution for being able to add something like a counter to it as there is no locking or atomic access.

adverse-sapphire

3/5/2025

Python Template Issues

The python Crawlee + Playwright + Chrome template is broken. Selecting this template and immediately building/running it on the Apify platform Web IDE results in a Running as root without --no-sandbox is not supported error....

message.txt

foreign-sapphire

3/5/2025

📌 Passing a Custom Tag in Apify Actor & Webhook for Make.com

Hey everyone! 👋 I'm using a public Apify Actor in a Make.com scenario and want to pass a custom tag ({{custom_tag}}) in the input JSON when running the Actor. The goal is to have this tag included in the webhook payload that Apify sends back when the run completes. I can see that when I am starting the Apify Actor, that the {{custom_tag}} is included in the input to the Apify Actor but I do not know how I can get the Actor to output this {{custom_tag}} in its payload. I was thinking of using a custom webhook payload template to manually add {{custom_tag}} but I cannot find any solution to this....

conscious-sapphire

3/3/2025

Cannot create an account on the website with gmail address. Do not want to use Google SSO

Message => For better security, please use the "Sign up with Google" button to sign up with your Gmail account. I don't want to use Google SSO, I'm stuck at the sign-up form...

conscious-sapphire

3/2/2025

Force language?

Hello, In general on Apify, it is possible to force the browser language? (example: fr or es or en) I have tested with: ...

harsh-harlequin

2/28/2025

Website Content Crawler Actor - Get access to failed urls

Hi, I am using the actor Website Content Crawler (apify/website-content-crawler) to scrape a few thousand urls. These are a predecided list of a urls, so the depth is set to 0. I saw a few of these fail. Is there any way to get access to these failed urls from either the apify site UI or the integration? The Dataset under Storage only gives the successful urls....

harsh-harlequin

2/26/2025

Goodreads API?

New to the tool, very cool. When I dropped the link I wanted to scrape inside Actor it came out perfect. I'd like to build this into an automation though via Zapier. Ideally, I submit a google form with the link to the goodreads URL i want to scrape, zapier will do a webhook and pull the info, and then add it where I need it to go. I was using a tool for amazon with a format like this and wondering if there is a similar way to do it for this tool? https://api.scraperapi.com/?api_key={{api-key}}&...

rare-sapphire

2/25/2025

How to track a run's cost in USD?

I'm using the Python SDK to run apify actors and get data for my application> I want to track how much money I'm spending on each run so I can calculate my average costs. As I spoke with the Apify AI chat bot and by reading the documentation, I discovered that there is the 'usageTotalUsd' key in the run object that should represent how much I paid for the run. However, the data isn't right at all. In the apify dashboard I'll see one number and in my logs I will see a different value for how much I actually paid....

Louis Deconinck

2/24/2025

Set country for datacenter proxies

Is it possible to set a country for datacenter proxies? I currently have 30 Apify datacenter IPs which I think are from the US. I'm specifically looking to use EU proxies, country doesn't matter. Is there a way to specify multiple country codes? Example of how I do it with residential proxies....

flat-fuchsia

2/23/2025

❌ You hit hard limit for the total number of runs or records scraped by this actor for a free user.

Hello everyone, I'm currently building a web scraper via Apify. I want to scrape leads via Apollo (name: Apollo Scraper - Pay per leads) I have started and run the scraper several times but not one result came up. Now I have reached a hard limit. ...

memo23

2/20/2025

Use an HTTPS agent to enforce TLS 1.3 in got-scraping/CheerioCrawler

Is there a way to change tls version when scraping with CheerioCrawler?

correct-apricot

2/16/2025

[URGENT] Video Scraper for Reddit

I’m trying to use Apify to scrape videos from a specific subreddit. I only want to retrieve video posts, not images or text posts. I’ve tried using query prefixes like “type:video” and various filters, but I’m still getting mixed content. Has anyone successfully configured the scraper to return only video-based posts? What parameters or techniques did you use to achieve that? Any help would be appreciated. Thanks in advance!

correct-apricot

2/16/2025

[URGENT] Video Scraper for Reddit

Hi everyone, I’m trying to use Apify to scrape videos from a specific subreddit. I only want to retrieve video posts, not images or text posts. I’ve tried using query prefixes like “type:video” and various filters, but I’m still getting mixed content. Has anyone successfully configured the scraper to return only video-based posts? What parameters or techniques did you use to achieve that? Any help would be appreciated. Thanks in advance!

Louis Deconinck

2/13/2025

Purge all datasets using Apify CLI

with --purge you can delete the default dataset. This does not affect the other datasets. Is there a way to purge all datasets using the CLI?

stormy-gold

2/13/2025

Custom usage limit

This helps protect against unexpected platform overuse. You'll be notified if your monthly usage approaches the limit. While we strive to stay within the limit, small overages may occasionally occur. If exceeded, Apify platform services will be paused to prevent additional charges. Total monthly platform usage $49.00 of $49.00Edit limit...

Louis Deconinck

2/7/2025

Playwright chrome browser fails on Apify platform

I have a scraper using Playwright, which still works perfectly locally. It also used to work on Apify, but since today it no longer does. Has anything been changed about how Playwright is ran on Apify? The error talks about the old Chrome headles mode being removed? See attachment for the full logs....

message.txt

Louis Deconinck

2/7/2025

PPE charging

Is there any way how I can verify what the user will be charged when doing a run of a PPE (Pay Per Event) actor? How can I verify that the charging is set up correctly on my end?

Louis Deconinck

2/6/2025

RESIDENTIAL5 proxies

In the past I sometimes used RESIDENTIAL5 proxies, which I believed to be even better proxies than the regular RESIDENTIAL proxies. However, as of late they stopped working. Has anything changed in that regard? My scraper does no longer work and regular residential proxies keeps it blocked.

Previous Next

Gaming

Programming

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Crawlee & Apify

This is the official developer community of Apify and Crawlee.