A crawl on an ecommerce website with more than 500 products returns only 39 items. Do you know why ?
I am trying to get all perfumes from the website https://la-barfumerie.com/
I use the url https://la-barfumerie.com/products/* in the Extract tool.
I did it twice: first result contains 39 items and the second 37 items. The website contains more than 500 products !
I don't understand if the limitation comes from FireCrawl or if there is something like javascript or a protection technology on the website.
Thanks for your help and advise !
La Barfumerie Paris
La Barfumerie – Parfums de Luxe et Fragrances Exclusives en Ligne
Découvrez La Barfumerie Paris, votre boutique en ligne spécialisée dans les parfums de luxe. Explorez une sélection exclusive de fragrances raffinées pour tous les goûts.
3 Replies
if you figure it out, please let me know! I've had similar issues for ecomm sites
What kind of eComm sites are you guys scraping? @apatel @Bernard
Yess guys, ive also experienced a similar issue.
The total number of pages it should have scraped were 127, but it only scraped 30!
Instead of using the extract tool and making it crawl all of the webpages itself, i had to do the crawling myself. Luckily, they have a separate tool called map (it maps out all the urls available from the url you give it). This works quite well.
So i gave it the url, and got all 127 urls back. THEN i used the extract tool and gave it all 127 links to scrape.
A simple workaround, way more reliable. @Chowderr @apatel @Bernard i hope that helps?