Crawlee & Apify•3y ago

How to pass UserData when executing crawler

When I do await crawler.run(['https://crawlee.dev'], { userData: { depth: 0 } }); I got this error: Uncaught ArgumentError ArgumentError: Did not expect property userData to exist, got [object Object] in object options How can I set userData in option? `

4 Replies

equal-jade•3y ago

await crawler.run([{url: 'https://crawlee.dev', userData: { depth: 0 } }]);

quickest-silver•3y ago

userData is part of request, not part of crawler

fair-rose•3y ago

You passed the userData as the second parameter of the run() method instead of within the request. Hence the ArgumentError. Try this instead:

await crawler.run([{
    url: 'https://crawlee.dev',
    userData: {
        depth: 0,
    },
}]);

await crawler.run([{
    url: 'https://crawlee.dev',
    userData: {
        depth: 0,
    },
}]);

eager-peach•3y ago

Also, something that was not obvious to me was how to get the user data on the route. you can do it like so

const routeHandler = async ({ request, crawler, page, log }) => {
    // get user data
    let { depth } = request.userData  
}

const routeHandler = async ({ request, crawler, page, log }) => {
    // get user data
    let { depth } = request.userData  
}

Gaming

Programming

How to pass UserData when executing crawler

Did you find this page helpful?