Discrepancy in the count of URLs returned by the FireCrawl API v/s actual sitemap of the Website
I'm trying to get the list of URLs present in the sitemap of a website using FireCrawl API with following parameters in CRAWL mode -
params = {
'crawlerOptions': {
"returnOnlyUrls": True
}
}
firecrawl_urls = app.crawl_url(url, params=params)
But the count of URLs is not matching with the actual count of URLs present in the sitemap of the website.
e.g: for this website - eonhealth.com, the actual sitemap has around 106 pages but firecrawl is only returning 1 page in the results list.
Can anyone please clarify what could be the possible reason behind this?
@rafaelmiller, @Adobe.Flash Any thoughts on this?
params = {
'crawlerOptions': {
"returnOnlyUrls": True
}
}
firecrawl_urls = app.crawl_url(url, params=params)
But the count of URLs is not matching with the actual count of URLs present in the sitemap of the website.
e.g: for this website - eonhealth.com, the actual sitemap has around 106 pages but firecrawl is only returning 1 page in the results list.
Can anyone please clarify what could be the possible reason behind this?
@rafaelmiller, @Adobe.Flash Any thoughts on this?
