17 Replies
@Moderator pls help
Please share a URL where you're getting this
we're just getting 504 timeouts.. this isn't normal we're on the higher plans of yours
Are you getting status code 504 when connecting to Firecrawl? Or is the target site returning 504
I think target site
but we're also unable to extract any info with the scrape + llm extract connection
here's an example that's returning no data: barnhartbolt.com
Seems to work on the playground. Could you please share your scrape options/the code running the request?
does it also work with the llm extract?
let me check
What's your schema?
quick q: what's the rate limit for scrape + llm extract?
is it the same as the scrape limits?
Yes
okay that's odd we're getting rate limit errors when we're below the limits
okay let me share the schema
oh odd, i can see you getting back null as the extract result
I'm also getting this error.
{"key":"error","value":"(Internal server error) - All scraping engines failed!"}
oh i think it's b/c im trying to put multiple URLs into the scrape request - and not using the batch endpoint
oh hm. but it actually does work sometimes.
so maybe it is an error.What's the request you're sending?
{
"urls": ["https://en.wikipedia.org/wiki/Pearl_Jam", "https://en.wikipedia.org/wiki/Nirvana_(band)"],
"formats": [ "markdown" ],
"includeTags": [],
"excludeTags": [
"svg",
"img",
"picture",
"video",
"canvas",
"embed",
"object",
"iframe",
"audio",
"[data-src]",
"[style*='base64']"
],
"onlyMainContent": false,
"waitFor": 1000,
"removeBase64Images": true
}
just got another one.
i ran it 2x and it worked the 2nd time.
this is happening too many times for me to rely on firecrawl unfortunately - we're going to have to revert to using playwright.
just got another.
another:
another:
i get this 1 out of 6 times or so