Does the self-hosted have a rate-limiter?
How could we rate limit crawling/scraping on the self-hosted Firecrawl?
3 Replies
You could use the delay parameter to control how many seconds between scrapes there are during a crawl.
Thank you!
Also, I wanna ask do we support scheduler? or is it something custom that we need to implement?
You will need to implement scheduling logic on your side to meet your custom needs.