harsh-harlequin

code not running on apify platform

using crawlee documentation for playwright i created a crawler on my local machine, it works fine, but putting same code on apify platform playwright scrapper it does not work, can anyone please help me. the code i am attempting to run is apify platform

async function pageFunction(context) {
    const { page, request, log } = context;
    //await waitFor('#vc-profile');

    const currentInvestingPosition = context.locator('.line-separated-row.row', { hasText: 'Current Investing Position' }).innerHTML();
    const investmentRange = context.locator('.line-separated-row.row', { hasText: 'Investment Range' }).innerHTML();
    const sweetSpot = context.locator('.line-separated-row.row', { hasText: 'Sweet Spot' }).innerHTML();
    const investmentsOnRecord = context.locator('.line-separated-row.row', { hasText: 'Investments On Record' }).innerHTML();
    const currentFundSize = context.locator('.line-separated-row.row', { hasText: 'Current Fund Size' }).innerHTML();

    const results = {
        url: context.request.url,
        current_investing_position: currentInvestingPosition,
        investment_range: investmentRange,
        sweet_spot: sweetSpot,
        investments_on_record: investmentsOnRecord,
        current_fund_size: currentFundSize
    }

    console.log(results)
    return {
        results
    };
    
}

async function pageFunction(context) {
    const { page, request, log } = context;
    //await waitFor('#vc-profile');

    const currentInvestingPosition = context.locator('.line-separated-row.row', { hasText: 'Current Investing Position' }).innerHTML();
    const investmentRange = context.locator('.line-separated-row.row', { hasText: 'Investment Range' }).innerHTML();
    const sweetSpot = context.locator('.line-separated-row.row', { hasText: 'Sweet Spot' }).innerHTML();
    const investmentsOnRecord = context.locator('.line-separated-row.row', { hasText: 'Investments On Record' }).innerHTML();
    const currentFundSize = context.locator('.line-separated-row.row', { hasText: 'Current Fund Size' }).innerHTML();

    const results = {
        url: context.request.url,
        current_investing_position: currentInvestingPosition,
        investment_range: investmentRange,
        sweet_spot: sweetSpot,
        investments_on_record: investmentsOnRecord,
        current_fund_size: currentFundSize
    }

    console.log(results)
    return {
        results
    };
    
}

8 Replies

modern-teal•3y ago

What error are you hitting?

harsh-harlequinOP•3y ago

nothing, it just keeps on looping into reclaiming request and then ends with empty results interestingly when i deployed my local crawlee code to apify platform, it runs well

modern-teal•3y ago

Hmmm if it’s reclaiming the request, then that means some error is occurring in the function. Do the log messages really say nothing? Do you have a run link you could share?

harsh-harlequinOP•3y ago

the one i published is private, the only share option is via email i guess. i can try sharing the playwright one though.

MEE6•3y ago

@dhruvgupta1729 just advanced to level 2! Thanks for your contributions! 🎉

harsh-harlequinOP•3y ago

this is the log file

Y67QbJ3BXtBSTdmsb.lo...

stormy-gold•3y ago

Maybe it's failing to find selector... Try to make a screenshot/get HTML of the failed page and check what is going on there. Maybe there are blocks/redirections or something like that.

harsh-harlequinOP•3y ago

it works fine on my local machine, same for if i publish actor from my local machine, it breaks when i add code to page function here https://console.apify.com/actors/MpRbnNmVAoj5RC1Ma i for sure need to look into proxy management, trying to sort it out today.

Apify

Apify Console

Manage the Apify platform and your account.

Gaming

Programming

code not running on apify platform

Did you find this page helpful?