Firecrawl

Join builders, developers, users and turn any website into LLM-ready data, enabling developers to power their AI applications with clean, structured information crawled from the web.

Join

Firecrawl

Join builders, developers, users and turn any website into LLM-ready data, enabling developers to power their AI applications with clean, structured information crawled from the web.

Join

❓┃community-help

💬┃general

🛠️┃self-hosting

Enzo Surreal

10/6/2025

🔥 Challenge: Website requires 2FA to disconnect sessions + 3 session limit - is this solvable?

Hi! I successfully automated login on a website (JusBrasil) using Firecrawl actions and can scrape protected content perfectly. The problem: The website has a "max 3 concurrent browser sessions per account" policy. ...

Goateigh

10/5/2025

App keeps crashing, serverless function has timed out

I keep getting this error when trying to navigate through my dashboard. I also wasn't able to properly click on things, when I did nothing worked, so I had to hover over the links, get the URL, and then write it on my browser's URL bar. Not sure if it's just me. I've cleared my application memory and cache etc. but doesn't seem to work either.

Solution:

@JuanC @frishtik can you do a hard-refresh and try again, also try cleaning cache, it should work then. The is now deployed for all instances.

Lucas Mendonça

10/3/2025

Crawl does not work for a specific website, but scrape does

I’m trying to crawl this URL: https://www.investopedia.com/terms/r/request-for-proposal.asp/ Yes, including the trailing /. The crawl endpoint is returning nothing, while the scrape endpoint retrieves the correct information. Shouldn't both work?...

Hugo Faria

10/2/2025

I'm trying to use it in TRAE and I'm getting this error, what could it be?

'env' is not recognized as an internal or external command, operable program or batch file....

Lucas Mendonça

9/29/2025

Crawl endpoint retrieving wrong sourceURL

I’ve found a mismatch in the crawl endpoint. When I try scraping a specific website, the sourceURL is being returned as it was currentURL. However, when I run the same URL in scrape mode, the response looks correct and the sourceURL is fine. The URL I’m testing with is: https://forum.tufin.com/support/kc/ext/tm/ On the crawl I receive this sourceURL: https://forum.tufin.com/support/kc/ext/tm/Content/ext/tm/intro.htm On the scrape I receive this sourceURL: https://forum.tufin.com/support/kc/ext/tm/...

Rakesh

9/29/2025

Scraping site for logged-in users

can it scrape sites that require users to be logged in? Specifically, does Firecrawl support cookie-based sessions (or other session/auth replay methods)? If so, please describe how to provide cookies/session tokens, handle CSRF, and any limits or best practices (session rotation, rate limits, anti-bot handling). Thanks!

Wolverine

9/28/2025

Getting Empty Respose with Extract Node

Hi can someone help with the extract node in n8n its giving a success response with ID, when using this ID run get extract data, it giving empty response.tried hitting curl in postman also then also getting same issue. If i use the crul that work in playground , with that also getting issue. Kindly help....

Ayan_Khan

9/28/2025

JSON schema not working like earlier

I'm on a hobby plan and I was using json schema (v2) method to scrape market research websites. I used to extract information using the payload attached here. Now i'm almost getting no data regarding table of contents from the websites. Is something changed in the API? If not can someone help me with a json schema that firecrawl will respect. I'm facing pressure from the stakeholders as it affects production app.

message.txt

danny

9/26/2025

Getting intermittent error page from sites ONLY when using Firecrawl

hello there! i've successfully loaded the following site through other scrapers, including Playwright, which I know Firecrawl runs under the hood. https://www.capptegy.com/events ...

orykevin

9/25/2025

Why when I tried to scrape a page with pre tag / code block it's always empty in the markdown?

I want to get code block of the documentation site, but always get empty

...

BugCreator101

9/25/2025

Clarification on Firecrawl Data Servers, Retention, and Deletion

I had a quick question regarding Firecrawl’s data handling. Could you clarify where your servers are located (region-wise), how long data from crawls is stored by default, and whether users can fully delete their data (including logs/backups) if needed? Lastly, does Firecrawl comply with European data protection laws (like GDPR)

Rakesh

9/24/2025

User Segmentation and pincode level support

I’m using Dify workflow, I need to pull selling prices based on locality (like pincode) and also based on user types (premium, normal, new user). Does Firecrawl support extracting prices this way? Specifically, can it handle geo-based pricing and differentiate by user type or session? If not, any tips on how to achieve this would be awesome....

edsaur

9/23/2025

How can I make sure that my self-hosted firecrawl can handle large amount of request?

If the request is plenty how could I handle the large amounts of requests?

Samarth

9/23/2025

crawling isn't getting accurate data from site

when i am trying to crawl through my site using the playground, I am unable to retrieve all the numerical data from the site. and extract feature is also returning different results on re run even with similar configuration and prompt. link : https://softr.biglook.ai/...

yagomlcarvalho

9/23/2025

Extract all PDF URLs from webpage

Hi, guys! I've been testing the /scrape endpoint here with the json format to extract the PDF files from webpages. For pages with small number of files it's working very well, but for pages with lots of files sometimes it returns only a few and some other times ti returns the full set. Since it's AI generated it can be expected, but is there a way to pass some more instructions to the AI during the API call? I'm using the Python SDK

rno321

9/22/2025

How to set concurrency for Firecrawl self-hosted

I'm setting max_concurrency in my crawl options but doesn't seem to do anything. the docker service is still running scrapers serially.

edsaur

9/22/2025

Does the self-hosted have a rate-limiter?

How could we rate limit crawling/scraping on the self-hosted Firecrawl?

Enzo Surreal

9/22/2025

100% of URLs Domain?

Is there any way to make MAP return 100% of a domain's public URLs? I ask this because there are certain domains that return less than half of the URLs.

Ahmad Sabir

9/21/2025

JSON SCHEMA NOT WORKING

Hi team I am using scrape endpoint for json Schema output but after 50 or more URLs but not more than 80 the API request just hangs I am facing thsi issue from past 2 days. Can someone help me here. Is this a rate limit issue or what because if it is a rate limit issue why I am still facing same issue after 2 days. Thanks....

beardy

9/20/2025

n8n - looping through 300 records to search each result - 500 error

I am new to n8n + firecrawl. Building a workflow to iterate through 300 entries from json and firecrawl/search each. From n8n I am using generic HTTP POST with this payload: { "query": "{{ $json.name }}, {{$json.city}}, {{$json.state}}", "sources": ["news"],...

Previous Next

Gaming

Programming

Firecrawl

Join builders, developers, users and turn any website into LLM-ready data, enabling developers to power their AI applications with clean, structured information crawled from the web.

Firecrawl

Join builders, developers, users and turn any website into LLM-ready data, enabling developers to power their AI applications with clean, structured information crawled from the web.