Polished & Refined evidence & arguments for X-Risk

For the next time that there is a 'pro AI' letter/PR thing, we should have a very well researched response for the reasons to be concerned about AI. IMPORTANT: This should not just be something that people in PauseAI or agreeing with PauseAI's message agree with.
This should be something that e/accs, normies, as many people as possible are persuaded by, something which is very robust against counter arguments. Something tested by someone who is very open to talking to other groups in a respectful way.

full-green•3/7/24, 10:16 PM

This would be really good to hash out with a group of technically-minded and well-educated people, including alignment researchers.

It seems like everyone has their favorite way of characterizing the problem, some of which are very easy to disagree with along some narrow technical or philosophical line.

For a very general and basic argument, I personally like the following:

Well-resourced companies are trying to build artificially intelligent systems with general capabilities
The possession of general capabilities necessarily implies the ability to do things we want ("good things") as well as things we don't want ("bad things")
The more capable the system is, the better the good things it can do, and the worse the bad things it can do
There is no expert consensus on whether or how we can steer a powerful general system robustly enough to ensure that it does not do anything so bad that we go to zero
We cannot recover from zero
The people of the world have not given their informed consent for this experiment to be conducted on their lives
Is-to-ought bonus:
Therefore, no amount of potential or actual benefit can ever outweigh a unilateral risk

technical-ivory•3/7/24, 10:17 PM

It is so great!

Ffull-green This would be really good to hash out with a group of technically-minded and wel...

spotty-amberOP•3/8/24, 6:58 PM

the counter-arguments to that tend to be that by not building it, we're risking those who are already suffering and dying

Sspotty-amber the counter-arguments to that tend to be that by not building it, we're risking ...

full-green•3/8/24, 7:03 PM

That just isn't true, though. Our current situation isn't an additional risk. It's just our current situation. Adding catastrophic risks on top of that isn't going to help those people.

Besides which, the point still stands that the suffering and dying have not been asked. I'm pretty sure they don't want to risk ending the world, either. Most people aren't that selfish.

spotty-amberOP•3/8/24, 7:04 PM

My suggestion: present the evidence of harms which have already happened, the lack of liability/justice for the victims and show how they're growing
The counters to this: these problems might be presented/twisted to be a lack of capabilities, not alignment

Ffull-green That just isn't true, though. Our current situation isn't an additional risk. It...

spotty-amberOP•3/8/24, 7:04 PM

I personally 100% agree with this

spotty-amberOP•3/8/24, 7:04 PM

just thinking about how someone would try to counter it- and how I've seen it be countered