Details are not revealed after clicking the button using puppeteer.
I have tried below code to get some hidden details after button click. After clicking the button, new divs popped out and show some additional details in page. Then there should be 12 elements but code results show only 8 elements which already visible before button click.
const addTitleSelector = 'h1.TextHeadline3TextHeadline3Wrapper-sc-10e1s2p-0';
const vehiclePriceSelector = '.TextHeadline2TextHeadline2Wrapper-sc-1itsg3n-0';
const vehicleParameterSelector = '.ExpandableContentContent-sc-11a0rym-0 .ParamsWithIconsWrapper-sc-hanfos-0 .TextCallout1TextCallout1Wrapper-sc-swd73-0';
const vehicleParametersExtendButtonSelector = 'button.ExpandableContentStyledShowMoreButton-sc-11a0rym-2';
router.addDefaultHandler(async ({ request, page, log }) => {
const title = await page.title();
log.info(
${title}, { url: request.loadedUrl });
const adTitle = await page.$eval(addTitleSelector, (item) => item.textContent);
const vehiclePrice = await page.$eval(vehiclePriceSelector, (item) => item.textContent);
await page.waitForSelector(vehicleParametersExtendButtonSelector);
await page.click(vehicleParametersExtendButtonSelector);
await page.waitForSelector(vehicleParameterSelector);
const vehicleParameters = await page.$$eval(vehicleParameterSelector, (items) => items.length)
await Dataset.pushData({
url: request.loadedUrl,
pageTitle: title,
adTitle,
vehiclePrice,
vehicleParameters
});
});1 Reply
stormy-gold•2y ago
Hey there! It's not really possible to check the code without knowing the website. Also - try checking the selectors on page. Looking at your code - site is using obfuscated selectors, so there's a chance the selectors that appear after click are slightly bit different.
Also keep in mind that if you click and then wait for selector (which is the same that already exisiting on page) waiting promise will resolve immediately
so you need to some some custom waiting, that would check number of items on page or something like that