Failed to Fetch (JS)
A friend of mine sent me a website and I was curious as to how they were getting data. I found an API which I'm playing around with fetch data using JavaScript. I'm attempting to read an API, but it returns "TypeError: Failed to fetch"
Here's my code:
Typing the API in browser returns data, but when I run this function it returns an error. What am I doing wrong?
41 Replies
you're using
let
don't think that the variable exists inside the function
nope, i was wrongI read on the documentation that I need to use GET
Do I need to do something like this, with method, mode, cache, credentials, headers, etc?
you're being bitten by cors
if you paste your code in a random tab, you get this:
if you paste it into roblox, then it works
Would that mean I have to do something like this?
or do I have to use a proxy server? Something like https://corsproxy.github.io/
don't think you're supposed to access this api
Hmm
It's on their website https://catalog.roblox.com/docs/index.html
and it's not under their robots.txt
i have no idea then
okay
the error you get suggests the solution:
This got rid of the error
it does fetch
I don't know much about scraping, just playing around to see if I can get data
CORS is used to prevent scraping, adding mode no-cors bypasses this?
CORS can be used for that, but it's up to the server to enforce it properly
How could I get this to console log?
and when I try to log as json:
mode: "no-cors" doesn't magically hack the server or anything, it just sends a request without CORS headers
it didn't for me
you use it like this:
[...].then((response) => response.json()).[...]
yeah thats what I wrote that spit out that log
it automatically parses the json code
Do people scrape data that is blocked by CORS using something like Selenium?
depends on what you're familiar with
it's being weird, I see the response JSON in the console, but parsing it with response.json() produces a warning that it isn't valid JSON
probably the russian characters should be escaped
but it should work anyways
Just reading some random stackoverflow posts, and people say to get around this kind of issue, you need to request through a server? Similar to a proxy?
I found a github regarding scraping from the site --> https://github.com/noblox/noblox.js
Seems like what they're doing is using node.js and getting a user account's cookie which they then use to request from the account
so maybe to successfully scrape this site, you'd need to make the api calls while logged into an account (on their website) to avoid CORS
https://noblox.js.org/ <-- there's stuff like this, so, you probably don't need
This NPM package enables operations froms the Roblox website to be executed via NodeJS;
this package requests setCookie() which is an account loginnpm
roblox
This module is intended to be used to access the Roblox platform and handle (the best it can) mumbojumbo like
X-CSRF-Token
headers and throttling/retries.. Latest version: 1.0.40, last published: a year ago. Start using roblox in your project by running npm i roblox
. There is 1 other project in the npm registry using roblox.and this?
This stuff is too far out of my scope
I figured I would be able to just call the api and get data
read the code and try to figure out what they did, and try to reproduce it yourself
I am but it requires node
I haven't touched node at all I figured I could do this with vanilla js
node is just javascript without the browser bits