Scraping reddit
any work arounds for this?
[
{
"url": "https://www.reddit.com/r/Vaping/comments/15crn7u/what_are_your_opinions_on_the_smok_nord_4_kit/",
"content": "Error scraping https://www.reddit.com/r/Vaping/comments/15crn7u/what_are_your_opinions_on_the_smok_nord_4_kit/: 403 Client Error: Forbidden for url: https://api.firecrawl.dev/v0/scrape"
},
{
"url": "https://www.reddit.com/r/Vaping/comments/rw4ggw/is_the_smok_nord_4_a_good_vape/",
"content": "Error scraping https://www.reddit.com/r/Vaping/comments/rw4ggw/is_the_smok_nord_4_a_good_vape/: 403 Client Error: Forbidden for url: https://api.firecrawl.dev/v0/scrape"
},
{
"url": "https://www.reddit.com/r/Vaping/comments/12jkip3/thinking_about_buying_a_smok_nord_4_do_you/",
"content": "Error scraping https://www.reddit.com/r/Vaping/comments/12jkip3/thinking_about_buying_a_smok_nord_4_do_you/: 403 Client Error: Forbidden for url: https://api.firecrawl.dev/v0/scrape"
}
]
7 Replies
Hey there! We actually don't support reddit at this time. Are you using the cloud hosted version?
yep just using the APIs
any particular reason why it's not supported?
Any updates on supporting reddit?
Reddit's public content policy does not allow it
Does the self-hosted version allow Reddit?
Looks like Reddit offers an official API to fetch data: https://support.reddithelp.com/hc/en-us/articles/16160319875092-Reddit-Data-API-Wiki
In the meantime, are there any third-party tools that support crawling Reddit?