F
Firecrawl•12mo ago
micah.stairs

Getting "Invalid cookie fields" error

I'm using the Python FireCrawl API and am trying to crawl a website behind authentication. It keeps failing when trying to scrape the first page. Am I doing something wrong? 
app.async_crawl_url(
    "https://portal2.vcinity.io/s/",
    params={
        "scrapeOptions": {
            "formats": ["markdown"],
            "headers": {
                "Cookie": "COOKIE_VALUE_HERE",
            },
        },
        "maxDepth": 10,
    },
)
app.async_crawl_url(
    "https://portal2.vcinity.io/s/",
    params={
        "scrapeOptions": {
            "formats": ["markdown"],
            "headers": {
                "Cookie": "COOKIE_VALUE_HERE",
            },
        },
        "maxDepth": 10,
    },
)
However that same cookie value seems to work correctly with curl: curl -X GET "https://portal2.vcinity.io/s/" -H 'Cookie: COOKIE_VALUE_HERE
1 Reply
micah.stairs
micah.stairsOP•11mo ago
I'll close this as obsolete.

Did you find this page helpful?