Failed to Fetch (JS)

A friend of mine sent me a website and I was curious as to how they were getting data. I found an API which I'm playing around with fetch data using JavaScript. I'm attempting to read an API, but it returns "TypeError: Failed to fetch" Here's my code:
let api = "https://users.roblox.com/v1/users/4060137991";

function fetchData() {
fetch(api)
.then(response => console.log(response))
.catch(err => console.log("error:" + err))
}

fetchData()
let api = "https://users.roblox.com/v1/users/4060137991";

function fetchData() {
fetch(api)
.then(response => console.log(response))
.catch(err => console.log("error:" + err))
}

fetchData()
Typing the API in browser returns data, but when I run this function it returns an error. What am I doing wrong?
41 Replies
ἔρως
ἔρως14mo ago
you're using let don't think that the variable exists inside the function nope, i was wrong
Matt
Matt14mo ago
I read on the documentation that I need to use GET
Matt
Matt14mo ago
No description
Matt
Matt14mo ago
Do I need to do something like this, with method, mode, cache, credentials, headers, etc?
ἔρως
ἔρως14mo ago
you're being bitten by cors
ἔρως
ἔρως14mo ago
if you paste your code in a random tab, you get this:
No description
ἔρως
ἔρως14mo ago
if you paste it into roblox, then it works
Matt
Matt14mo ago
Would that mean I have to do something like this? or do I have to use a proxy server? Something like https://corsproxy.github.io/
ἔρως
ἔρως14mo ago
don't think you're supposed to access this api
Matt
Matt14mo ago
Hmm It's on their website https://catalog.roblox.com/docs/index.html and it's not under their robots.txt
ἔρως
ἔρως14mo ago
i have no idea then
Matt
Matt14mo ago
okay
Jochem
Jochem14mo ago
the error you get suggests the solution:
fetch(api, {mode: 'no-cors'})
fetch(api, {mode: 'no-cors'})
Matt
Matt14mo ago
This got rid of the error
ἔρως
ἔρως14mo ago
but doesn't fetch https://users.roblox.com/docs/index.html <-- i did find the api
Jochem
Jochem14mo ago
it does fetch
Matt
Matt14mo ago
I don't know much about scraping, just playing around to see if I can get data CORS is used to prevent scraping, adding mode no-cors bypasses this?
Jochem
Jochem14mo ago
No description
Jochem
Jochem14mo ago
CORS can be used for that, but it's up to the server to enforce it properly
Matt
Matt14mo ago
How could I get this to console log?
Matt
Matt14mo ago
No description
Matt
Matt14mo ago
and when I try to log as json:
Jochem
Jochem14mo ago
mode: "no-cors" doesn't magically hack the server or anything, it just sends a request without CORS headers
Matt
Matt14mo ago
No description
ἔρως
ἔρως14mo ago
it didn't for me
No description
ἔρως
ἔρως14mo ago
you use it like this: [...].then((response) => response.json()).[...]
Matt
Matt14mo ago
yeah thats what I wrote that spit out that log
ἔρως
ἔρως14mo ago
it automatically parses the json code
Matt
Matt14mo ago
Do people scrape data that is blocked by CORS using something like Selenium?
ἔρως
ἔρως14mo ago
depends on what you're familiar with
Jochem
Jochem14mo ago
it's being weird, I see the response JSON in the console, but parsing it with response.json() produces a warning that it isn't valid JSON
ἔρως
ἔρως14mo ago
probably the russian characters should be escaped but it should work anyways
Matt
Matt14mo ago
Just reading some random stackoverflow posts, and people say to get around this kind of issue, you need to request through a server? Similar to a proxy? I found a github regarding scraping from the site --> https://github.com/noblox/noblox.js Seems like what they're doing is using node.js and getting a user account's cookie which they then use to request from the account so maybe to successfully scrape this site, you'd need to make the api calls while logged into an account (on their website) to avoid CORS
ἔρως
ἔρως14mo ago
https://noblox.js.org/ <-- there's stuff like this, so, you probably don't need
Matt
Matt14mo ago
This NPM package enables operations froms the Roblox website to be executed via NodeJS; this package requests setCookie() which is an account login
ἔρως
ἔρως14mo ago
npm
roblox
This module is intended to be used to access the Roblox platform and handle (the best it can) mumbojumbo like X-CSRF-Token headers and throttling/retries.. Latest version: 1.0.40, last published: a year ago. Start using roblox in your project by running npm i roblox. There is 1 other project in the npm registry using roblox.
ἔρως
ἔρως14mo ago
and this?
Matt
Matt14mo ago
This stuff is too far out of my scope I figured I would be able to just call the api and get data
ἔρως
ἔρως14mo ago
read the code and try to figure out what they did, and try to reproduce it yourself
Matt
Matt14mo ago
I am but it requires node I haven't touched node at all I figured I could do this with vanilla js
ἔρως
ἔρως14mo ago
node is just javascript without the browser bits
Want results from more Discord servers?
Add your server