extract api error
The error is:
Sep 29 10:33:00 Error extracting data with Firecrawl: Extract job failed. Error: [object Object]
Sep 29 10:33:00 Firecrawl error details: {
Sep 29 10:33:00 status: undefined,
Sep 29 10:33:00 statusCode: 500,
Sep 29 10:33:00 response: undefined,
Sep 29 10:33:00 config: undefined
Sep 29 10:33:00 }
Sep 29 10:33:00 [Firecrawl] extractWithSchema returned: NULL
9 Replies
Is it failing for a specific URL, and other APIs are working fine for you? If you could share the sample URL you're testing, it would help to debug and expedite the solution.
Sure. We've been using the same urls for a while now in development so I am not sure that is the issue. Examples include:
https://www.gratcliffe.co.uk/
My Site
G Ratcliffe Electrical Ltd | Electrical Engineers | England, UK
Do you need experienced electrical engineers? Contact G RATCLIFFE ELECTRICAL LTD for domestic and commercial electricians serving Porth and South Wales.
bdj-plumbing
BDJ Plumbing Services, Plumber Exeter, 07341 839206
BDJ Plumbing Services - Call Ben now for a plumber Exeter - Emergency plumber and small plumbing repairs. Plumbers in Exeter for taps, toilets, toilet repair, leaks, showers, bathroom plumbing, blockages, radiators and installs!
https://example.com
Have you made changes to the API in the last few days? Any updates or something like that that could have broken your backend?
The playground is working for me. This is an API request to the extract endpoint
seems to be an issue in your development environment, as I just now tested the
/extract endpoint on Postman with your sample url.
The team will get back to you via app-support on the environment-specific issue. you can also provide these details
Hi Guarav, I have been playing around since we spoke and believe I have foudn the issue.
In the zod JS schema for extract we were using enum. For example:
serviceMode: {
type: 'array',
items: {
type: 'string',
enum: ['at_your_location', 'at_our_location', 'online'],
},
maxItems: 1,
description:
'Single primary service mode - how the business primarily delivers services',
},
This was working on Friday and stopped working over the weekend. I have tested it by removing the enum property and the extract works correctly.
Can you please clarify what caused this issue to occur and how it went unnoticed?
As we are now live, this is an issue. Especially given the previous issues we've had with firecrawl
Okay, zod refinements were added - https://github.com/firecrawl/firecrawl/pull/2058 which seems to have caused this.
GitHub
fix: Add Zod validation to reject additionalProperties in schema pa...
fix: Add Zod validation to reject additionalProperties in schema parameters
Summary
Fixes an issue where schema parameters containing additionalProperties would cause silent failures in OpenAI API ...
@Alex are you on a self hosted version? exact issue here:
enum arrays being converted to objects
Although I've added a PR to handle this: https://github.com/firecrawl/firecrawl/pull/2224GitHub
fix: enum arrays being converted to objects by Chadha93 · Pull Req...
ref: https://discord.com/channels/1226707384710332458/1422199456929022013
Summary by cubic
Prevented enum arrays from being converted into objects during schema normalization, ensuring valid Open...