eastern-cyan

TypeError Converting circular structure to JSON

Somehow I am passing a complex object but I don't know where I have to look at to solve this error. Any suggestions? Here is the product page which I am scraping : https://www.rexel.co.uk/uki/root-category/Tools%2C-Test-Equipment-%26-Fixings/2-Piece-HSS-metal-drill-bit-set-%C3%98-6-x-93-mm/p/2501419565

Bizline BIZ 700715 | 2 Piece HSS metal drill bit set Ø 6 x 93 mm | ...

Get 2 Piece HSS metal drill bit set Ø 6 x 93 mm at wholesale prices at Rexeluk - your electrical distributor. Join Now!

9 Replies

HonzaS•3y ago

my guess is puppeteerUtils takes some key as parameter actually it looks like, it takes page as the first parameter so the problem will be somewhere else you can put there some console.log to find out on which line is the problem

fascinating-indigo•3y ago

🧐 I don't have any errors with Playwright.

async requestHandler({ request, page, log }) {
        log.info(`GET ${request.url}  DONE`);

        await page.waitForSelector('button#acceptAllConsentSummary');
        await page.click('button#acceptAllConsentSummary');

        const results = await page.evaluate(() => {
            const productName = $('h1.font-weight-bold.mb-1').eq(0)?.text().trim();
            return { productName };
        });
        log.info(JSON.stringify(results));
}

async requestHandler({ request, page, log }) {
        log.info(`GET ${request.url}  DONE`);

        await page.waitForSelector('button#acceptAllConsentSummary');
        await page.click('button#acceptAllConsentSummary');

        const results = await page.evaluate(() => {
            const productName = $('h1.font-weight-bold.mb-1').eq(0)?.text().trim();
            return { productName };
        });
        log.info(JSON.stringify(results));
}

Result : INFO PlaywrightCrawler: {"productName":"2 Piece HSS metal drill bit set Ø 6 x 93 mm"}

eastern-cyanOP•3y ago

with more logs I still can't find he issue but I'll give Playwrite a try maybe Puppeteer is not the right method for this website..

optimistic-gold•3y ago

What happens: 1. The stack trace tells you the error is handleProduct function on line 60 so something about the page.evaluate 2. The "circular structure to JSON" usually means you are accidentally passing something different than you think. Basically instead of object or string, you are passing a class or something with methods that cannot be stringified. 3. This happens because everything coming in and out of page.evaluate is run through JSON.stringify and JSON.parse so you can only send stringifiable data. 4. Looking at the code, my guess is that the url param you are sending into page.evaluate is not a URL but something else you are passing in Puppeteer vs Playwright is totally irrelevant here, this is just typo bug

eastern-cyanOP•3y ago

Without the url in the return it is working, but how can I push the page url to the dataset?

eastern-cyanOP•3y ago

optimistic-gold•3y ago

The url itself is not a problem, you are just passing a wrong argument order to handleProduct from your main.js file

plain-purple•3y ago

🥲 I am also getting this circular thing error https://codeshare.io/6pbmAY

Pepa J•3y ago

The error in log should give you good clues on where the error occures, you may debug the code and se the cycling structure.

Gaming

Programming

TypeError Converting circular structure to JSON

Did you find this page helpful?