Removing 1000 limit cap on /map function
Hi, I need to remove the cap on the 1000 limit for results returned on the map/ function. I need to scrape all the available urls on a site and this limit is too low. How can I remove it? Thanks
2 Replies
Is this cap also on the api? Or on the website.
Hey! In the API, it defaults to 5000 and is limited to 30000. Here are instructions on how to configure it: https://docs.firecrawl.dev/api-reference/endpoint/map#body-limit
In the playground, its limit is also 5000, but is not configurable there. Here's an example which hits the 5000 limit: https://www.firecrawl.dev/app/playground?url=https%3A%2F%2Fdevelopers.google.com%2F&mode=map&limit=10&excludes=&includes=&formats=markdown&onlyMainContent=true&excludeTags=&includeTags=&proxy=undefined&includeSubdomains=true&mapSearch=&uniqueKey=1748041804231.
You'll notice that in the playground it says: "(Limited to 1000 'smart' results while in Alpha. Contact us for improved limits.)"
From the documentation (https://docs.firecrawl.dev/api-reference/endpoint/map#body-search):
Search query to use for mapping. During the Alpha phase, the 'smart' part of the search functionality is limited to 1000 search results. However, if map finds more results, there is no limit applied.