Trying to combine a content checker with a login on Apify (new to Apify and webscraping)
The content checker actor is what I need to get an alert when the content on a web page changes. The page is behind a login and I have learned how to export cookies.
But I can't seem to marry the two. What happens is:
(1) I keep getting a picture of the login page!
- So I think I need to know where to insert the cookies export. This actor type seems to be missing the "Initial Cookies" field that I see in other actors [like Puppeteer?] (but then those actors don't do the content check.. and I don't see how the content check actually works such that I could copy paste the code. I'm pretty unfamiliar with how these tools work!)
(2) I can paste the cookie export if I go to the JSON panel
- But I am just not sure the Content Checker was designed for this as I've noticed this gets erased/reset sometimes, eg., if I reload or return to the "regular editor" and save.
(3) How to specify the container properly.
- This is a syntax question but it can wait - if I got the whole page as a snapshot, that would work for me. I am not sure what CSS label to pick and I've tried a few options. I just don't think I've got this far yet because I am not sure if passing in the cookie export has worked (aside from the login page result, I've also in some tries gotten back the page banner as a result - so not inconclusive)
4 Replies
vicious-gold•3y ago
evident-indigo•3y ago
You can also refer to this newer tutorial:
https://developers.apify.com/academy/puppeteer-playwright/common-use-cases/logging-into-a-website
Apify
Logging into a website · Apify Developers
Understand the "login flow" - logging into a website, then maintaining a logged in status within different browser contexts for an efficient automation process.
quickest-silverOP•3y ago
Thanks for providing the guidance above - can you help me understand where I should be putting this code (I have used the Apify templates to create a content checker, for instance - but should I put this code in that actor? Or Is there a 'free text' actor I should be using)
evident-indigo•3y ago
The tutorial I linked contains the raw Puppeteer code
You can adapt it to a PuppeteerCrawler or PlaywrightCrawler in Crawlee