Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻devs-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

other-emerald

4/11/2024

ERROR Actor failed with an exception

ERROR Actor failed with an exception
Traceback (most recent call last):
File "/usr/src/app/src/main.py", line 150, in main
actor_input = await Actor.get_input()
^^^^^^^^^^^^^^^^^^^^^^^ ...

optimistic-gold

3/29/2024

Hi, My wordpress websites are being

Hi, My wordpress websites are being massively scraped. I put reCaptcha V3 now is it enough to block scraping bots?

subsequent-cyan

3/27/2024

Hi Apify team, question on the Instagram

Hi Apify team, question on the Instagram Comments scraper. Is it possible to scrape only the top comments (with most likes and/or replies). As the results currently is max 50 and usually low quality comments (such as "great stuff" or "I love this")

wee-brown

3/26/2024

Anyone knows if there is a difference

Anyone knows if there is a difference between puppeteer’s evaluate() and chromium’s devtool console? I am currently scraping a site and doing a fetch api manually via the devtool console works perfectly, but im being blocked when using puppeteer’s evaluate(), despite using the same fetch function. Wonder if anyone could help me with this?

flat-fuchsia

3/19/2024

I'd like to understand the free tier.

I'd like to understand the free tier. The pricing page implies the $15 credit can be used for any of the cost items, but then on sign up it says "proxy trial", suggesting that the proxies are only available for a short time. But the whole point is to be a proxy so how does it work without one?

exotic-emerald

3/18/2024

Problem

Can you provide me with more details about the specific issues you're facing? What API are you trying to integrate, and what problems have you encountered so far?

vicious-gold

3/16/2024

Hey, has anyone an idea how to scrape

Hey, has anyone an idea how to scrape the imprint for names of CEOs if I have a list of domains? Is there already an actor for this? Thank you so much in advance!

conscious-sapphire

3/13/2024

Hello everyone, I've recently noticed

Hello everyone, I've recently noticed that to use the API, I now need to subscribe. Previously, I was using the free version. Has there been a change requiring payment for API access through a subscription plan?

provincial-silver

3/5/2024

is it possible to configure the use of

is it possible to configure the use of Actor RAM. Still, the impossibility of running two processes at the same time was related to this. Can this be configured for an already existing apify/web-scraper??? s it possible to set a limit? Is there a usage limit? for example, if we have 32 gigabytes of Ram and set the limit to 4, to be sure that I can definitely run 8 processes in parallel. I will be very grateful for your help

noble-gold

3/5/2024

Scaling Playwright Crawler on GCP Cloud Run

Yo everyone. Starting to try and hit my crawler with some real load. Playwright setup in GCP Cloud Run... do you think a tonne of info logs I haven't cleaned up could be impacting performance? If so that's my lowest hanging fruit 😂 Otherwise should I skip to looking at parallelization? What is performance like using lots of average/small/cheap instances versus overprovisioning a single cloud run instance and smacking it with load?...

fair-rose

3/3/2024

i have pay for extra data center proxy,

i have pay for extra data center proxy,using actorTweet Scraper V2 (Pay Per Result) - Twitter / X Scraper ,also add following code "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": [ "BUYPROXIES94952" ]...

funny-blue

2/29/2024

Hi folks need a little dev help. I am

Hi folks need a little dev help. I am basically performing some actions on a webpage. I want the actions to be performed only when the page is fully loaded. I have tried other ways such as window.document('') and other mentioned on stackoverflow but none worked for me. Need a little help. this is the below classname that is visible when the page is fuly loaded so my ideaa over here is to wait untill this className is present inside the DOM if it's present meaning the page is loaded and if not then it's not loaded

 datatable-body

 datatable-body

this is the classname...

deep-jade

2/27/2024

Hi everyone. I am trying to connect 2

Hi everyone. I am trying to connect 2 actors and I want to push data from the first into the input of the 2nd. How do I do this?

helpful-purple

2/19/2024

Good afternoon! My apify actor keeps

Good afternoon! My apify actor keeps running even after revoking the key, and removing all configurations for this. It keeps running without my permission, and is charging my account every 15 minutes. I would like to terminate the process. How can I do that?

foreign-sapphire

2/18/2024

Just try adding 4 users and the process

Just try adding 4 users and the process stops, any help please?

robust-apricot

2/18/2024

hi uh- im not sure if this is a good

hi uh- im not sure if this is a good place for this, but im a confused soul, know jack squat about these things, and was hoping for some pointers from literally anyone lol on this lil issue i gots. tryin to use a facebook scraper, gots the cookie thingie, export, copy paste...did it all and it says 'this field should be array' everytime i try to save and run and tbh i'm js lost and idk what to do lol. any sort of response or pointer to where to ask this is appreciated .^.

correct-apricot

2/15/2024

Hi, I'm building a scraper to scrape the

Hi, I'm building a scraper to scrape the text from a number of websites but I'm struggling to find how to clean the data correctly or if this an inbuilt function for crawlee? In Apify it's under "removeElementsCssSelector" but I can't see a corresponding options in crawlee

quickest-silver

2/14/2024

Hi, I need help with increasing the

Hi, I need help with increasing the default limit of the number of photos retrieved by the paid actor AliExpress Scraper after running the data extraction process. Currently, I'm only receiving a limited amount, typically six photos, and I want to know how to expand this limit.

solid-orange

2/14/2024

I need some help setting up the contact

I need some help setting up the contact information scrapper. Can someone help? I need it to automatically click the visit my website button for each profile and then srape the data from that page, but all I get is the url links from the main page.

stormy-gold

2/13/2024

Free Instagram Post Scraper · Apify

Hey guys, https://apify.com/apify/instagram-reel-scraper does this scraper returns the video file ? cause didn't see it, i saw some images.

Previous Next

Gaming

Programming

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Crawlee & Apify

This is the official developer community of Apify and Crawlee.