Crawlee & Apify•3y ago

How to click on pagination of js links?

The pagination links are not changing the url, they dont have href value. I think it must click on the first link, scrape the page, and then move to the next link, and it should happen in the same tab, there is no need for new tabs. You can see the pagination here: https://www.paz.co.il/he-IL/paz-stations Please help, thank you @thek1tten@Lukas Krivka @pczr

5 Replies

other-emerald•3y ago

you can do a click while using page.waitForNavigation() usually it's

const tabs = await page.$$('selector');
for (const tab of tabs) {
  await Promise.all([
    tab.click(),
    page.waitForNavigation(),
  ]);
}

const tabs = await page.$$('selector');
for (const tab of tabs) {
  await Promise.all([
    tab.click(),
    page.waitForNavigation(),
  ]);
}

adverse-sapphireOP•3y ago

Thanks but I can't get it right, can you help me modify the code?

message.txt

other-emerald•3y ago

your top const data = await page.$$eval('.result-row', ($posts) => { call needs to be inside the loop

other-emerald•3y ago

New_Text_Document.tx...

adverse-sapphireOP•3y ago

@pczr Can you please try to run it? It still scrapes only the results from the first page, and it scrape them many times again and again up Please help me solve this issue

Gaming

Programming

How to click on pagination of js links?

Did you find this page helpful?