Casper - Hello, do you know what is the scrape ...

Hello, do you know what is the scrape endpoint for open source local firecrawl?

6 Replies

Hi @Casper the endpoints will remain same, just that it will include the localhost - http://localhost:3002/v2/scrape note: the port is configurable. example:

# Single page scrape (v2)
curl -X POST http://localhost:3002/v2/scrape \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.firecrawl.dev/introduction",
    "formats": ["markdown", "html"]
  }'

# Start crawl job (v2)
curl -X POST http://localhost:3002/v2/crawl \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.firecrawl.dev",
    "limit": 50,
    "scrapeOptions": {
      "formats": ["markdown", "html"]
    }
  }'

# Check crawl status (v2)
curl -X GET http://localhost:3002/v2/crawl/{crawl_id}

# Map site (v2)
curl -X POST http://localhost:3002/v2/map \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.firecrawl.dev"
  }'

# Single page scrape (v2)
curl -X POST http://localhost:3002/v2/scrape \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.firecrawl.dev/introduction",
    "formats": ["markdown", "html"]
  }'

# Start crawl job (v2)
curl -X POST http://localhost:3002/v2/crawl \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.firecrawl.dev",
    "limit": 50,
    "scrapeOptions": {
      "formats": ["markdown", "html"]
    }
  }'

# Check crawl status (v2)
curl -X GET http://localhost:3002/v2/crawl/{crawl_id}

# Map site (v2)
curl -X POST http://localhost:3002/v2/map \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://docs.firecrawl.dev"
  }'

CasperOP•3mo ago

I finally figured out the endpoint was /v1/scrape - this should be better documented for sure Also an official docker image would be appreciated

Gaurav Chadha•3mo ago

v1 is legacy, we recommend using v2 endpoints.

CasperOP•3mo ago

i understand that but there is no offical docker image for v2 yet would you consider adding that or instruction how to do that?

Gaurav Chadha•3mo ago

We can, we will update you if we add an official firecrawl image. For now, we prefer to use docker-compose.yaml, which will support v2 endpoints as well - https://docs.firecrawl.dev/contributing/self-host

Firecrawl Docs

Self-hosting | Firecrawl

Learn how to self-host Firecrawl to run on your own and contribute to the project.

CasperOP•3mo ago

Thanks

Gaming

Programming

Casper - Hello, do you know what is the scrape ...

Did you find this page helpful?