Firecrawl api not giving output for crawl jobs
Hi folks, is there any outage on Firecrawl side?
Not getting output for the api requests on crawl api end point.
The response says status = WAITING
13 Replies
I'm having the same problem, can anyone provide information?
@Adobe.Flash can you help me?
@nick @rafaelmiller My jobs are also stuck in waiting status. Could you give us any information on what is happning?
@Janice @Sachin I'm taking a look
Ngl, already switched to selfhosted as it’s not possible to predict the reliability of your service.
Thank you
@Janice @Sachin everything's up and running smoothly now!
Thank you!!
Hey yall, quick update https://discord.com/channels/1226707384710332458/1261330279348572251/1261330647910191254
We experienced out of the ordinary high volume from 9 am - 10 am EST causing some jobs to get stuck in "waiting" or "stuck" or "failed" status. We have scaled and pushed a fix. Apologies for that. Everything should be back to normal.
Have you been able to get crawling working with self hosting? I just did a quick look at this using the docker setup and it looks like it only supports scraping.
Does scraping
didnt try crawling
So my jobs are running but failing within a 10 seconds (these jobs ran successfully yesterday).
Output from the check_crawl_status:
WARNING:root:crawl data returned is {'status': 'active', 'current': 1, 'current_url': '', 'current_step': 'SCRAPING', 'total': 100, 'data': None}
app-1 | WARNING:root:crawl data returned is {'status': 'failed', 'current': 1, 'current_url': '', 'current_step': 'SCRAPING', 'total': 100, 'data': []}
Is it possible to find out why these jobs are failing? @rafaelmiller @Adobe.Flash
@Janice can you dm me the url you are trying?
Thank you! Things just started working again.