python sdk crawl endpoint run non stop and run me out of credit
aI was testing firecrawl for the following use case: scrape the website of a startup for their product
I notice that with this website in particular, despite setting a limit of 2, the endpoint run non stop until I reach my credit limit
app = FirecrawlApp(api_key=[MY_API])
crawl_status = app.crawl_url(
'https://www.oceanfarmr.com/',
params={
'limit': 2,
'scrapeOptions': {'formats': ['markdown', 'html']}
},
)
I notice that with this website in particular, despite setting a limit of 2, the endpoint run non stop until I reach my credit limit
app = FirecrawlApp(api_key=[MY_API])
crawl_status = app.crawl_url(
'https://www.oceanfarmr.com/',
params={
'limit': 2,
'scrapeOptions': {'formats': ['markdown', 'html']}
},
)