facing an issue here, while scraping one of the blog website on firecrawl
Hey, I am facing an issue here. while scraping one of the blog website, firecrawl still generating main website content. For ex - I am scraping for ||(www.main-brand.com/blog/any-blog-content), but its still giving me scrape content for (www.main-brand.com) only. it seems scraping is not working for any /blog subsection domain.
Is there any prerequisite for scraping blog website? Am I missing something here?
I have tried all feature correction like zero maxage, stealth mode, main content:false etc. but still no correct response.
Although scraping is working fine for other path like /about us etc
Is there any prerequisite for scraping blog website? Am I missing something here?
I have tried all feature correction like zero maxage, stealth mode, main content:false etc. but still no correct response.
Although scraping is working fine for other path like /about us etc
