Docusaurus crawler
Hi all! New here, crawlee looks like an awesome tool. I'm currently building a docusaurus site crawler but wanted to ask around and see if anyone knows of an existing implementation before I go and reinvent something. If an existimg implementation doesn't exist I'd be happy to open source my own!
1 Reply
constant-blue•3y ago
You can try https://apify.com/lukaskrivka/article-extractor-smart#smart-article-extractor but it is not currently open source
Apify
Scrape and download articles and news · Apify
📰 Smart Article Extractor extracts articles from any scientific, academic, or news website with just one click. The extractor crawls the whole website and automatically distinguishes articles from other web pages. Download your data as HTML table, JSON, Excel, RSS feed, and more.