LowFatCheese - Hey guys, need some help with cr...
Hey guys, need some help with crawling multiple links in n8n;
I'm pulling the top (upto) 10 links for a SERP keyword, then wanting to get the markdown for further analysis etc.
I'm having trouble getting the firecrawl nodes going over all urls found, instead of just crawling the first url. Any thoughts?
3 Replies
Might be a noob question, so apologies, but yeah, do i use the http request node with a loop over items or am i just not seeing the right settings
I'm not super familiar with n8n but I think you need something to loop over all URLs returned and scrape them individually, and then aggregate. Alternatively, you can simply use Firecrawl search with scrape options specified, which will search and scrape all pages in one go.
@mogery thanks, got it sorted, used a loop over then aggregate