Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Join

crawlee-js

apify-platform

crawlee-python

💻hire-freelancers

🚀actor-promotion

💫feature-request

💻devs-and-apify

🗣general-chat

🎁giveaways

programming-memes

🌐apify-announcements

🕷crawlee-announcements

👥community

metropolitan-bronze

12/9/2023

Auto builds don't work at all, getting HTTP 500 errors from Apify both in UI and webhooks

When I click on the radio button Automatic builds, I get an error. Different browser doesn't help. The browser console says: ``` [31mERROR[39m Failed to handle request 'POST - /github-app/setup-webhook/...'[90m {"request":{"method":"post","url":"https://console-backend.apify.com/github-app/setup-webhook/...","headers":{"x-idempotency-key":"...","Authorization":"Bearer ..."},"params":{},"data":{"version":"0.0","enabled":true}},"response":{"status":"error","statusCode":500,"isClientSafe":true,"errorCode":"internal-server-error","errorMessage":"Cannot destructure property 'token' of '(intermediate value)' as it is null.","path":"/github-app/setup-webhook/..."}}[39m [object Object]...

metropolitan-bronze

12/7/2023

is the source code available for apify actors? most of the links are 404s

In the actors page, you can choose Information > Source Code > View on Github, but most of these links seem to 404. For example apify/google-search-scraper goes to https://github.com/apify-projects/store-google-search/tree/master/actors/google-search-main or apify/website-content-crawler goes to https://github.com/apify/store-website-content-crawler...

stormy-gold

12/7/2023

Instagram profile scraping via API

I have the following scenario: I want to enter Instagram username handle on my website and populate the whole profile with results from apify actor. How do I do it via API? I can not figure out how to send input data JSON from my input to the actor's input, run it and get results. I'd really appreciate some help....

rare-sapphire

12/6/2023

Is it safe to push .env file containing aws API keys to Apify?

rare-sapphire

12/6/2023

Does Apify supports webhooks

Hello everybody. Well as the title suggests, does Apify supports webhooks or similar methods that will invoke once an actor finished running? e.g. When Actor's done running, invoke a custom AWS lambda function that downloads the acquired data. ...

metropolitan-bronze

12/6/2023

Google query term and pull content from each of the result pages?

Hello friends, I'd like to do a Google search on a specified term, then for each of the URLs that are returned, I'd like to retrieve that page, and return the whole thing in a dataset. Is this possible? Thanks.

wise-white

12/6/2023

TripAdvisor info scraper

I am seeking assistance with collecting information regarding restaurants across Europe that offer alcoholic beverages, bars and hotel restaurants. The desired data includes email addresses, booking phone numbers, restaurant names, addresses, rankings, and website details, to be organized into an Excel spreadsheet format. While primarily focusing on scraping data from TripAdvisor, I am open to exploring alternative sources if available. I have already used a tripadvisor scraper but it have to individually enter every link which makes it a very slow process....

wee-brown

12/6/2023

Is there any twitter tweet scraper that work ?

I struggle to scrape a profile, i want all the tweets but also the threads, excluding retweets and replies to another person, i run every existing scraper on apify and no one work idk why

ambitious-aqua

12/6/2023

Actor is under maintenance

please we are facing a problem we don't know the issues coming from where , could u please help how i can solve this problem "This Actor is marked as under maintenance. Once you have addressed the issues, you can remove the under maintenance flag in the publication section." thank you...

vicious-gold

12/5/2023

Problem with adding a payment method

During the addition of a new payment method (credit card) I get an error that: The supplied card data failed tokenization. Please ensure you have sufficient funds on your card. This is not correct as I have funds on the card and I can see from my bank (revolte) that the transaction was accepted....

like-gold

12/4/2023

allegro crawler - missing data

I am using Allegro Scrapper - but it has mistake - sellerName is null, ad also IMAGES table is empty.... Attached - seller name and images JSON missings. SellerName was working at 3th november but after this date BOOOOM 🙂 We have two types of seller Names, we can see the difference with HTML nodes premium:...

fascinating-indigo

12/4/2023

Use env variables for actor at build time from apify-cli

Hi, I find out that it seems that it's possible to tell apify to apply env variable also during build time only from the GUI like this, there's no other way? I'm creating actor via apify-cli using 'apify-cli push' and planning to automating it via github actions and obviously in this way will not work

funny-blue

12/4/2023

Actor Origin WEB vs API

Hi, is there any difference in performance between running actor from WEB vs from API? I feel that sometimes the actor runs a bit slower when the origin is API..

rare-sapphire

12/3/2023

Creator Plan Question

Does the $500 on time credit get reapplied after 6 months if you pay again?

fascinating-indigo

12/2/2023

Problem building with private packages

Hi everyone, nice to be here 😁 I am writing to you about a problem with private packages on an Actor JS/Typescript- Engine Node 18. I followed these steps:...

stormy-gold

12/2/2023

How to increase Memory size of actor "Website Content Crawler"?

Hi, I'm a APIFY subscription user, though my account shows a total of 32GB memory. When I submit Run request via Python code, it's default to 4GB. Please help advise how to custom run time memory? "ApifyClient(APIFY_TOKEN).actor("aYG0l9s7dbB7j3gbS").start(input)" #Website-Content-Crawler

rare-sapphire

12/2/2023

Subscription

Is it possible to rent a actor for 1$/month without subscribing for 44$ dollar per month for apify? Or is there a cheaper option? Because thats too expensive for my case.

extended-salmon

12/1/2023

Force 3rd-party scraper to use local dataset?

Title says it all - is this possible, and how do you do it via the actor sdk/API? Thanks!

dependent-tan

12/1/2023

Scraping Threads Timeline. But Not able to get entire data.

I am making a scrapper to scrap threads.net using playwright and python and willing to host on Apify. But It I am not able to scrap entire timeline. It is only giving some data and when scrolling down, I am not able to scrap the new data. Any help or suggestion is highly appreciated....

sunny-green

11/30/2023

Issues with Limited Results from YouTube Shorts Scraper

I want to scrape YouTube channel data, but I'm consistently receiving only 48 results, while on the target account are much more videos. MaxResult setting is empty .This issue persists even after switching from the 'youtube-shorts-scraper' to the 'Fast Youtube Channel Scraper'. I'm not sure if this is due to API limitations, scraper settings, or my usage approach. I'm new here so appreciate a response how to get more results....

Previous Next

Gaming

Programming

Crawlee & Apify

This is the official developer community of Apify and Crawlee.

Crawlee & Apify

This is the official developer community of Apify and Crawlee.