Easiest way to scrape a JSON URL?
I would like to scrape the URL below which returns a JSON dataset and then create a dataset from this by extracting selected json objects from this. I tried the Vanilla JS scraper and Cheerio scraper but did not manage to get a valid scrape. How would you approach this? Which (free) scraper would you choose? Does somebody have an example of a code that would acomplish this?
This is the URL: https://hypomat.glkb.ch/online-hypothek/api/hypomat/zinssatz
3 Replies
quickest-silver•3y ago
@Sesh you just make a request to the url and you'll get the data. don't need a scraper if its already an endpoint returning JSON
quickest-silver•3y ago
GeeksforGeeks
How to make ajax call from JavaScript ? - GeeksforGeeks
A Computer Science portal for geeks. It contains well written, well thought and well explained computer science and programming articles, quizzes and practice/competitive programming/company interview Questions.
rare-sapphire•3y ago
@Sesh with Apify you don't even really need your own actor, you can use https://apify.com/apify/cheerio-scraper with your custom function, see your json https://console.apify.com/view/runs/cqYiaMyVuqcSRFaaE
If you prefer own actor then CheerioCrawler should return
context.json to handler, if not it means you blocked, try under RESIDENTIAL proxies, as you can see from sample run data is availableApify
Cheerio Scraper - HTML scraping tool · Apify
Crawls websites using raw HTTP requests, parses the HTML with the Cheerio library, and extracts data from the pages using a Node.js code. Supports both recursive crawling and lists of URLs. This actor is a high-performance alternative to apify/web-scraper for websites that do not require JavaScript.