3 replies

🤔 There's a Cookie popup in my scrape

Hello.

Question: Is there a Firecrawl config setting I can use to prevent popups from getting into my scrapes?

All of a sudden to today I am getting "cookie popup content" when I scrape documentation files. I would like to prevent this if possible.

Attached:
- Screenshot of the cookie and site scraped
- Scraped file that now has the "cookie info" - lines [1-44] were added today, previous scrapes it wasn't there.

Here is my config (Firecrawl Python SDK):

    result = firecrawl.scrape(
        url,
        formats=["markdown"],
        only_main_content=True,  # Excl. nav menu, footer, sidebars, etc.
        remove_base64_images=True,  # Removes base64 strings (keeps alt text)
        wait_for=3000,  # Wait to capture dynamic content (3 seconds)
        max_age=86400000,  # Use cached content for speed (24 hours)
    )

    result = firecrawl.scrape(
        url,
        formats=["markdown"],
        only_main_content=True,  # Excl. nav menu, footer, sidebars, etc.
        remove_base64_images=True,  # Removes base64 strings (keeps alt text)
        wait_for=3000,  # Wait to capture dynamic content (3 seconds)
        max_age=86400000,  # Use cached content for speed (24 hours)
    )

Thanks in advance.

ai-sdk.md6.39KB

🤔 There's a Cookie popup in my scrape

Similar Threads

🤔 There's a Cookie popup in my scrape

Similar Threads

Similar Threads

Similar Threads