F
Firecrawl•5mo ago
miguelcgo

Able to scrap markdown but not json

Hey everyone I am trying to develop a script which scrapes a page and returns a json with information on that page. I can retrieve markdown for the entire page, however I can't retrieve anything in json. Here's the payload for my curlCommand 
const dataPayload = {
  url: TARGET_URL,
  formats: ["json"],
  jsonOptions: {
    schema: {
        type: "object",
        properties: {
            "artigo": {
                "type": "number"
            },
        },
        required: ["artigo"],
      }
    },
  location: {
    country: "PT",
    languages: ["pt-PT"]
  },
  waitFor: 10000,
};
const dataPayload = {
  url: TARGET_URL,
  formats: ["json"],
  jsonOptions: {
    schema: {
        type: "object",
        properties: {
            "artigo": {
                "type": "number"
            },
        },
        required: ["artigo"],
      }
    },
  location: {
    country: "PT",
    languages: ["pt-PT"]
  },
  waitFor: 10000,
};
When I use this payload I get nothing outside of metadata. After I add the format markdown then the entire markdown for the page gets returned. I also tried to use json without any shema but again I have no results. Anyone have a tip here?
2 Replies
SoulBloxXer
SoulBloxXer•5mo ago
Try enabling their fire 1 agent I cant explain why that would suddenly work but a similar problem i had was solved with that
Nalaso
Nalaso•2w ago
@miguelcgo were you able to fix this issue?

Did you find this page helpful?