Error in scrapeController: Error: Job wait
I recently switched over to v1, everything was working just fine before the switch. I added a console log to the scrape controller for reference.
I have tested this on different urls, all of which worked with v0.
Any insight would be greatly appreciate. Thank you!
7 Replies
I have also tried crawling a site for further testing. The api is working but scraping is not.
Are you using Docker? About 6 days ago the environment variable
FLY_PROCESS_GROUP was introduced. The environment variable for the worker has to be set to worker. The default is app so no workers are deployed if the variable is not set.
https://github.com/mendableai/firecrawl/blob/main/apps%2Fapi%2Fdocker-entrypoint.shGitHub
firecrawl/apps/api/docker-entrypoint.sh at main · mendableai/firecr...
🔥 Turn entire websites into LLM-ready markdown or structured data. Scrape, crawl and extract with a single API. - mendableai/firecrawl
Good catch @BrianJM ! Thank you!
Here is the pr that fixes it! Merging soon https://github.com/mendableai/firecrawl/pull/733
Thank you both.. I will wait for the merge to update. Glad to know I am not losing my mind
Just merged! Let us know if it works!
It’s working! Thank you so much 🙏