Crawlee & Apify•8mo ago

Puppeteer browser page stuck on redirections

when i use puppeteer & fingerprint injector with generator, some redirects make puppeteer on page firefox/chromium stuck after these redirections the page stops logging my interceptors (they just write the url), the page stops responding to the resize if I create a new page manually in this browser and follow the link with redirections, it's fine without injector and generator everything works fine too I'm not sure I can provide those links for some reason, but the problem is clearly related to the fingerprint injector the main link has pointers to other links

browser = await puppeteer
    .launch({
        browser: "firefox",
        headless: false,
        devtools: true,
        args: ['--disable-web-security', '--allow-running-insecure-content'],
    });

const fingerprintGenerator = new FingerprintGenerator();
const fingerprint = fingerprintGenerator.getFingerprint();
const injector = new FingerprintInjector();

let page = await browser.newPage();
await injector.attachFingerprintToPuppeteer(page, fingerprint);
await page.goto("...");

browser = await puppeteer
    .launch({
        browser: "firefox",
        headless: false,
        devtools: true,
        args: ['--disable-web-security', '--allow-running-insecure-content'],
    });

const fingerprintGenerator = new FingerprintGenerator();
const fingerprint = fingerprintGenerator.getFingerprint();
const injector = new FingerprintInjector();

let page = await browser.newPage();
await injector.attachFingerprintToPuppeteer(page, fingerprint);
await page.goto("...");

5 Replies

Hall•8mo ago

View post on community site

This post has been pushed to the community knowledgebase. Any replies in this thread will be synced to the community site.

Apify Community

sensitive-blueOP•8mo ago

deep-jade•8mo ago

Hi @Calypso Is there any reason why you are not using the crawlee which would make all of this for you?

sensitive-blueOP•8mo ago

I only knew about puppeteer xd

deep-jade•8mo ago

In that case I really recommend looking into crawlee https://crawlee.dev/docs/quick-start It will handle bunch of stuff for you. If you already know puppeteer you can use puppeteer crawler https://docs.apify.com/sdk/js/docs/examples/puppeteer-crawler

Quick Start | Crawlee · Build reliable crawlers. Fast.

Crawlee helps you build and maintain your crawlers. It's open source, but built by developers who scrape millions of pages every day for a living.

Puppeteer crawler | SDK for JavaScript | Apify Documentation

This example demonstrates how to use PuppeteerCrawler in combination

Gaming

Programming

Puppeteer browser page stuck on redirections

Did you find this page helpful?