Crawl a lead website to create a knowledge base
Hey guys, so I'm new to Firecrawl and I figured I'd find a response quicker if I post it here
For the context:
I'm creating a chatbots and voice ai assistants powered by LLMs for businesses.
Question:
given I have a fresh lead with the a web page URL and I want to create a knowledge base for my chatbot/voice agent LLM (Markdown or Embeddings)
-> what is the most straightforward way to approach gathering 'facts' or 'q&a' pairs from the webpage
Do I use extractor? Or /crawl? or gather a list of subpages from the website?
P.S. I now am at the stage of manual customer knowledge gathering process. But I do realize that I might want to create a ready-to-advertise system for potential leads automatically.
Please help me understand what my options are. Thanks!
For the context:
I'm creating a chatbots and voice ai assistants powered by LLMs for businesses.
Question:
given I have a fresh lead with the a web page URL and I want to create a knowledge base for my chatbot/voice agent LLM (Markdown or Embeddings)
-> what is the most straightforward way to approach gathering 'facts' or 'q&a' pairs from the webpage
Do I use extractor? Or /crawl? or gather a list of subpages from the website?
P.S. I now am at the stage of manual customer knowledge gathering process. But I do realize that I might want to create a ready-to-advertise system for potential leads automatically.
Please help me understand what my options are. Thanks!
