/map
I'm strongly considering migrating this process I've built to firecrawl. A general overview of what I'm trying to do: use firecrawl to obtain links when I perform a specific google query with /map. Pass the filtered links back to firecrawl's /scrape endpoint. However, what I'm noticing is that /map returns none of the links I see when I actually perform the google query. I'm getting back around 3K links from /map and none of them correspond to any links you'd see when you search that exact query up on google manually. All the links are the various links that come up whenever you search something, but isn't part of the actual search if that makes sense. For example: http://google.com/intl/zh-CN_cn/admob/landing/sign-up-002.html
http://google.com/intl/es_es/admob/announcements.html
http://google.com/intl/it_it/admob/businesskit/from-an-idea-to-an-app
Is my usecase incorrect? I
2 Replies
the map is for getting the urls of a site quickly (using sitemap and quick crawl). not for doing what you want
Have you tried using the Search endpoint? You're going to want to use that instead of Map
- Documentation: https://docs.firecrawl.dev/features/search
- Playground: https://www.firecrawl.dev/playground?mode=search