IrshaiD'
CACrawlee & Apify
•Created by IrshaiD' on 5/7/2025 in #apify-platform
How do we handle authenticated scrapers on apify cli locally
Yep got it . thanks guys
10 replies
CACrawlee & Apify
•Created by IrshaiD' on 5/7/2025 in #apify-platform
How do we handle authenticated scrapers on apify cli locally
My approach is to set up two separate actors with an integrated flow:
One actor login once and handles scraping the list pages.
The other actor scrapes the detail pages and sends the data to an S3 bucket.
This follows a divide-and-conquer approach. However, I want to avoid logging in every time in the details actor—I’d prefer to log in once and maintain the browser session across both actors.
10 replies
CACrawlee & Apify
•Created by IrshaiD' on 5/7/2025 in #apify-platform
How do we handle authenticated scrapers on apify cli locally
How would you approach this using Apify? Let me know if you have any suggestions.
10 replies
CACrawlee & Apify
•Created by IrshaiD' on 5/7/2025 in #apify-platform
How do we handle authenticated scrapers on apify cli locally
What I mean is: I need to scrape an goverment contracts website where the data is only accessible after logging in. It contains multiple contract listings and around 150,000 contract detail pages, so it's a large-scale operation. I want every request to carry the authenticated session.
10 replies