How to add headers to `addRequests`
Hello, I'm using the addRequests function to crawl an array of urls. How do I add a header to this crawl request? Thanks!
2 Replies
@Plinpod just advanced to level 1! Thanks for your contributions! 🎉
absent-sapphireOP•3y ago
Solved this by mapping my array of urls to an array of Requests
urls = urls.map((url) => {
return new Request({
url,
headers: { 'header-key': 'value' },
})
})
crawler.addRequests(urls)