WEB SCRAPPING
"Lately, I’ve been doing some web scraping projects and spending time learning more about it, but I keep wondering if I’m wasting my time. Is web scraping actually useful for companies? Can I get a job with web scraping skills? And what’s the best language for it — Python or Node.js? These questions are always on my mind. I wonder if anyone else has gone through the same thing.
28 Replies
web scraping is of questionable legality, depending on what you do with the content
the best language is legalise, allowing you to scrape the website
you probably can get a job with web scraping: there's lots of shady companies that do that
but, if those companies are shady and slimy enough to use web scraping for things they aren't allowed to, you think they also won't be shady and slimy with your paycheck?
your right, but scrapping allows me to understand the website and also programming i just do it for fun
no, it doesn't
all you understand is how to find elements
you practice a little of the language and that's it
im also learning C because university but i dont is the way i want to go
its very hard
what do you think about C
pointers
depends
all that stuff is
hard
in what context?
and i cant find a good project
yeah, I agree it's probably not good practice to learn web development. There's nothing terribly wrong with web scraping for personal projects, but anything that starts making money is going to get shut down in a hurry.
other languages have something similar, but they call it "references", which aren't exactly the same
usually with a very strongly worded letter
written by a lawyer
something with "CEASE AND DESIST" on the top of the letter
or a gaggle of them
(unless they just IP-ban you)
or do both
in university i need to choose between programming side or IT side (cloud, network, infrestracture, etc)
what do you think ?
C is a language that has its uses still. It's still likely a good teaching tool, but it's very limited and there's a lot of better tools for almost all the tasks it used to be used for
it's still very prevalent in embed systems
entirely up to you and what you would like to do 🤷
and hardware too
but rust is slowly creeping over
and pretty much nowhere else
yes but i never try nothing about IT but think its good
try it then, on your own
try to setup some docker and play about with it
just have a field day or 2, trying a few things
but like... learning a language like C will introduce you to concepts that will translate to different languages. Programming languages aren't like regular ones. I don't get a massive bonus to learning Italian just because I speak Dutch. But with programming, 95+% of what you're learning is how to solve problems and break them down into manageable chunks. The concepts of how you accomplish those chunks translate almost entirely between different languages
docker is used in both sides
that's very true
like, I'm convinced I could write something in Rust, if I could write it in javascript, simply because I can just google syntax and built-in methods
we're getting very off topic for the original topic of the thread though...
yup, we are
in short: scrapping bad, usually not legal, avoid it