F
Firecrawl•4w ago
VENGEΛNCE

Unable to scrape and extract data. Starts to Parallelise then nothing happens

URL is this: https://am-arrowmax.com/collections/* Prompt: 
Follow every product link to ensure all variants, including sold-out ones, are included. Capture the title, all variants (size, color, etc.), images per product and variant, price, and product/variant URLs. Ensure a clear mapping of product to variants to images to price to URLs.
Follow every product link to ensure all variants, including sold-out ones, are included. Capture the title, all variants (size, color, etc.), images per product and variant, price, and product/variant URLs. Ensure a clear mapping of product to variants to images to price to URLs.
The Schema was auto generated 
{
  "type": "object",
  "required": [],
  "properties": {
    "products": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [],
        "properties": {
          "title": {
            "type": "string"
          },
          "variants": {
            "type": "array",
            "items": {
              "type": "object",
              "required": [],
              "properties": {
                "size": {
                  "type": "string"
                },
                "color": {
                  "type": "string"
                },
                "images": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "required": [],
                    "properties": {}
                  }
                },
                "price": {
                  "type": "number"
                },
                "url": {
                  "type": "string"
                }
              }
            }
          },
          "images": {
            "type": "array",
            "items": {
              "type": "object",
              "required": [],
              "properties": {}
            }
          },
          "price": {
            "type": "number"
          },
          "url": {
            "type": "string"
          }
        }
      }
    }
  }
}
{
  "type": "object",
  "required": [],
  "properties": {
    "products": {
      "type": "array",
      "items": {
        "type": "object",
        "required": [],
        "properties": {
          "title": {
            "type": "string"
          },
          "variants": {
            "type": "array",
            "items": {
              "type": "object",
              "required": [],
              "properties": {
                "size": {
                  "type": "string"
                },
                "color": {
                  "type": "string"
                },
                "images": {
                  "type": "array",
                  "items": {
                    "type": "object",
                    "required": [],
                    "properties": {}
                  }
                },
                "price": {
                  "type": "number"
                },
                "url": {
                  "type": "string"
                }
              }
            }
          },
          "images": {
            "type": "array",
            "items": {
              "type": "object",
              "required": [],
              "properties": {}
            }
          },
          "price": {
            "type": "number"
          },
          "url": {
            "type": "string"
          }
        }
      }
    }
  }
}
No description
No description
8 Replies
VENGEΛNCE
VENGEΛNCEOP•4w ago
I messaged the AI on the website and it passed it to a human, but its Sunday in most of the world (Monday here in NZ) and I'd like to get this resolved today nothing shows in Activity Logs, I'm at 0/5 active concurrent browsers
micah.stairs
micah.stairs•4w ago
Can you try again? We fixed a bug this morning that was likely causing this.
VENGEΛNCE
VENGEΛNCEOP•4w ago
Just kicked off a new extract process so will see how it goes @micah.stairs I tried agian, nothing happened, cant even see the attempt in the extract overview, jus tthe previous failed attemps with empty files
micah.stairs
micah.stairs•4w ago
Hmm, so if you want more reliable (and cheaper) extract-like functionality, you should check out our JSON scraping mode, which is supported by endpoints like /scrape and /crawl. You can also pair that with custom actions, which is a powerful combo! Would this work for your use case?
harshithvh
harshithvh•4w ago
hey @VENGEΛNCE this seems to be working fine I spotted couple of issues in your setup 1. You have included https:// when adding URL, please remove it 2. Your prompt doesnt seem right, ex: you have included variants twice Make sure your prompt is simple and easy to parse for the parameters attaching a screenshot for your reference let me know if you have any further issues
No description
harshithvh
harshithvh•4w ago
@micah.stairs The team might consider adding/improving these 1. The URL for sharing is too long, can we shorten it? 2. Can we add the prompt/schema that was used for a run (under recent runs)
micah.stairs
micah.stairs•4w ago
Those are good idea! I just passed along those feature requests to the team. I will send a message here if we end up implementing either of those.
harshithvh
harshithvh•4w ago
sweet!

Did you find this page helpful?