Firecrawl Crawl Issue - n8n Integration
Issue: Firecrawl crawl via n8n node (@mendable/n8n-nodes-firecrawl v1) has three problems:
Initial API error: First 2 attempts fail with scrapeOptions.formats validation error (expected array, received object), even though my config only has default empty headers. After 2-3 retries it starts.
Rate limiting: Hitting "too many requests" on free tier despite delay: 1000, maxConcurrency: 5, and batching.
Crawl never completes: Once started, crawl with limit: 5 stays in "running" indefinitely. Status never reaches "completed" and I have to manually stop it in the dashboard. No pages are returned.
Config: Using crawl operation with a prompt to generate paths, excludePaths: ["data/*"], limit: 5, and default scrapeOptions (empty headers only).
Main blocker: Crawl starts but never finishes, even for small limits.
Detailed description with full config and response examples attached below.
Initial API error: First 2 attempts fail with scrapeOptions.formats validation error (expected array, received object), even though my config only has default empty headers. After 2-3 retries it starts.
Rate limiting: Hitting "too many requests" on free tier despite delay: 1000, maxConcurrency: 5, and batching.
Crawl never completes: Once started, crawl with limit: 5 stays in "running" indefinitely. Status never reaches "completed" and I have to manually stop it in the dashboard. No pages are returned.
Config: Using crawl operation with a prompt to generate paths, excludePaths: ["data/*"], limit: 5, and default scrapeOptions (empty headers only).
Main blocker: Crawl starts but never finishes, even for small limits.
Detailed description with full config and response examples attached below.
message.txt6.22KB
