Website Content Crawler With Site Login
The default Website Content Crawler has been great for my work, but I'm wondering if there's a version that can log into websites? Or is there a setting on the Input tab that I'm missing?
2 Replies
@arjuna just advanced to level 1! Thanks for your contributions! 🎉
Hi @arjuna ,
Depends on how the logign is implemented on the scraped website. I can see that Website Content Crawler allow to set the Cookie to the Actor's Run Input.
Check https://apify.com/apify/website-content-crawler/input-schema#initial-cookies
In case the login on website depends just on Cookies this could be a way, otherwise Iit may get quite complicated.
Apify
Input · Website Content Crawler · Apify
Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.