Search
Star
Feedback
Setup for Free
© 2026 Hedgehog Software, LLC
Twitter
GitHub
Discord
System
Light
Dark
More
Communities
Docs
About
Terms
Privacy
Not able to Crawl the website using Include Paths(Filter). - Firecrawl
F
Firecrawl
•
17mo ago
•
6 replies
Krishna
Not able to Crawl the website using Include Paths(Filter).
Hi
,
I am trying to crawl the webpages from this domain
https://www.peigenesis.com/
using path filters
.
But i am not able to get the associated web pages with that path
. I have been trying it from a long time
. Can someone pls help me
.
params2
=
{
'limit
'
: 4
,
'maxDepth
'
: 10
,
'includePaths
'
:
[
"
/part
-information
/
*
"
]
,
#
"excludePaths
"
:
[
]
,
#
"ignoreSitemap
"
: True
,
"allowBackwardLinks
"
: True
,
"allowExternalLinks
"
: False
,
#
"webhook
"
:
"
<string
>
"
,
"scrapeOptions
"
:
{
"formats
"
:
[
"markdown
"
,
"html
"
]
,
#
"headers
"
:
{
}
,
#
"includeTags
"
:
[
'
#parts a
'
,
'h1 span
'
,
'h2 span
'
,
'
#tools a
'
,
'
#tools
.fieldvalue
'
,
'
#mates a
'
,
'
#mates
.fieldvalue
'
]
,
"excludeTags
"
:
[
'img
'
]
,
"onlyMainContent
"
: True
,
"waitFor
"
: 2000
}
} this is the params that i am using while sending the api request
Thank you
.
Firecrawl
Join
Join builders, developers, users and turn any website into LLM-ready data, enabling developers to power their AI applications with clean, structured information crawled from the web.
15,303
Members
View on Discord
Resources
ModelContextProtocol
ModelContextProtocol
MCP Server
Recent Announcements
Similar Threads
Was this page helpful?
Yes
No
Similar Threads
Crawl "Include Only Paths" not working?
F
Firecrawl / ❓┃community-help
2y ago
Include Only Paths and Exclude Paths not working as intended
F
Firecrawl / ❓┃community-help
12mo ago
Crawl Is Scraping Website Too
F
Firecrawl / ❓┃community-help
7mo ago
Get all website url before crawl
F
Firecrawl / ❓┃community-help
2y ago