Plans for warning shots / wake-up calls

Brainstorm how we should take advantage of warning signs/ shots/ fire alarms in case they happen. A pause would probably be a lot more feasible after one, and missing it would also be bad for future warning shots.

The expression warning sign is sometimes used to describe any event that increases concern about a particular category of existential risk. For example, plausible candidates for an AI warning sign include a catastrophic failure by an AI system and public outreach campaigns or the publication of an exceptionally persuasive book on AI safety.

Some good ideas of what we should do:
- Know what key people / policymakers to reach out to.
- Have policy action plans with credible signatories.
- Know how to mobilize our volunteers.

sacred-roseOP•2/20/24, 6:10 AM

- Organize protests that we should tell everyone IRL and online about

awake-maroon•2/20/24, 8:41 AM

Previous discussion: https://discord.com/channels/1100491867675709580/1100491868304846943/1209376408535834644

sacred-roseOP•2/24/24, 7:06 AM

- I remember someone told me that it would be really good to have detailed proposals (maybe policy drafts?) for putting forward quick in case of a warning shot

dead-brown•2/24/24, 7:53 AM

Project Tracker: https://maximolog.notion.site/Plans-for-warning-shots-0c71d941051b4d1caa3fa6a9b87879db?pvs=4

Maxime Fournes's Notion on Notion

Plans for warning shots | Notion

Description

dead-brown•2/24/24, 7:54 AM

@Pato Vercesi could you write a little summary of what this project is about, what the tasks are etc? Just a few paragraphs would be nice

Ssacred-rose Brainstorm how we should take advantage of warning signs/ shots/ fire alarms in ...

popular-magenta•2/24/24, 6:12 PM

Warning shots? Focus points? Is this about things to say, or things to .. beat my chest (like a great ape) in the proximity of? I'm lacking some context here

sacred-roseOP•2/24/24, 7:00 PM

Yeah, my bad. I used the term "focus point" wrong, I fixed it now

Ssacred-rose Yeah, my bad. I used the term "focus point" wrong, I fixed it now

popular-magenta•2/27/24, 10:35 AM

I get it. How do we, in the public square, help others understand the unfortunate stuff that maybe happen & get news coverage...

popular-magenta•2/27/24, 10:41 AM

This brings to mind two related issues: how do the concerns we're discussing here get focused as feedback which can stick in the ongoing social conversation? & Aren't there already enough instances of happenings where the Pause AI raison d'être hasn't gained a foothold in the conversation & might profitably still be added?

sacred-roseOP•2/27/24, 4:50 PM

Sorry, but I don't follow. Probably because of my english

worthy-azure•2/28/24, 8:27 AM

Big fan of having concrete proposals ready to role out that would be restrictive and change things, so that when something catastrophic happens, proposals can just be adopted. The key issue is that we need lots of work on monitoring and similar to make a future ban possible, and only starting to put that in place at the time there is a warning shot is too slow. Happy to chat more about this.

dead-brown•3/5/24, 6:57 PM

Hey, I would suggest to break down this project in two parts.
1. The first part would be about identifying key events that warrant escalation.
2. The second part would be suggesting reactions to these events.

Having the first part done independently of the second one would be, IMO, already very valuable. It could be used to form a more concrete timeline, giving us some kind of estimation of how much time we have left

dead-brown•3/5/24, 7:05 PM

Here is a quick brain dump for some of these key events:
- Hallucinations in LLMs are solved
- An AI generated movie wins a prize for humans
- An AI generated novel wins a prize for humans
- Some specific tasks get solved by an autonomous agent (AutoGPT style)
- A model exhibits reasoning capabilities on par with humans
- A model exhibits situational awareness
-

Also this could be a great database to remind people of the ones that have already been blasted through:
- An AI generated picture wins a prize for humans
- A model exhibits algorithm writing capabilities on par with humans
- A model can find exploits in websites and hack them
- A model wins against humans at the game "diplomacy"

awake-maroon•3/5/24, 7:12 PM

I understood the purpose of the project to be more about figuring out how to respond when an AI risk-related event triggers public outrage but this is probably good to think about as well

awake-maroon•4/7/24, 11:04 PM

Some of my thoughts.

I expect we won’t get a pause before a warning shot because AI models will generate increasing amounts of gold and power.

It could be the case that we’ll only get one warning shot of sufficient severity that would make a global Pause be an option to policymakers. Unfortunately, I believe the severity needs to be very high; to counter the great economic-political pressures and incentives.

I think spending some time to make contingency plans prepared for different levels of severity could be very valuable on the margin. The policy action plan we can put forth to policymakers is proportional in ask / radicalness to the severity of the warning shot, the more severe, the larger the policy implementation can be in scope and level of conflict with economic preassure.

To me, a warning shot is not new developments – as these aren’t of a sufficient level for a pause to my models, rather, they would generate more money. A warning shot is harm done / potential harm done. It could be (1) platform where you can input in a video of someone, then output an indistinguishable deep fake, (2) we’ve identified that an ~AGI spreads like a virus but doesn’t really do anything except use computational resources, (3) we’ve identified that an ~AGI has tried and failed to take shut down critical infrastructure, (4) we’ve identified that an ~AGI has tried and failed to spread a pathogen that would have been similar the Black Death, (5) we’ve identified that an ~AGI has gained lots of wealth, (6) we’ve identified that an ~AGI has tried to socially engineer it’s way toward a nuclear warhead launch, (7) we’ve identified that an ~AGI has caused caused a stock market crash / hacked critical financial infrastructure, (8) we’ve identified that an ~AGI on its own written and released Stuxnet level computer viruses, etc.

awake-maroon•4/7/24, 11:04 PM

If we get a warning shot, we should expect an explosion of new members, proportional to the severity of the warning shot. We’ll need to be able to capitalize on this surge, have plans for how to mobilize fast and have the digital infrastructure ready to scale rapidly. Since I expect the new members (and our senior ones) will be even more emotionally connected to the cause we are fighting for, I'd expect our members to be ready to do much more, again, in proportion to the severity and scale. For these reasons, how we leverage a warning shot could be a defining moment for us.

Maybe there is a tail heavy impact distribution where we should focus on analyzing contingency plans for high severity, big scale warning shots, since these would give us the most leverage.

These contingency plans could contain:
* Know what key people / policymakers to reach out to.
* Have a policy action plan with credible signatories.
* Know what people to connect with whom -- maybe decision makers don’t know everyone they need to talk to in times of this (new) type of crisis, so, we could facilitate these connections.
* Know how to mobilize our volunteers.
* Know how to keep the movements spirit, not panic and not give up.

Aawake-maroon I understood the purpose of the project to be more about figuring out how to res...

sacred-roseOP•4/10/24, 11:41 PM

I think both things are totally okay to do here

Aawake-maroon If we get a warning shot, we should expect an explosion of new members, proporti...

sacred-roseOP•4/10/24, 11:51 PM

I agree with everything, I would like to put my time working on those points and any other that we could come up with

sacred-roseOP•4/11/24, 7:05 AM

https://discord.com/channels/1100491867675709580/1102569429038026852/1227441718484533278 In this podcast ezra klein talks about warning shots. And about how to be prepared for them he says:

a couple of things are important for that.
One is that the ideas have to be around.
Two is that they have to be coming from a source people trust, right. Or have reason to believe they should trust.
And three, they have to have some relationship with that source.
So what you want to be doing is building relationships with the kinds of people who are going to be making these decisions.
What you want to be doing is building up your own credibility as a source on these issues.
And what you want to be doing is actually building up good ideas and battle testing them and getting people to critique them and putting them out in detail.
Right. I think it is very unlikely that air regulation is going to come out of a less wrong post.
But I have seen a lot of good ideas from less wrong posts ending up in different white paper proposals that now get floated around.
And you need a lot more of those. It's funny, because, and I've seen this happen in Congress again and again.
You might wonder, why do these think tanks produce all these white papers or reports that truly nobody reads?
And there's a panel that nobody's at. It's a lot of work for nobody to read your thing and nobody to come to your speech.
But it's not really nobody. It may really be that only seven people read that report, but five of them were congressional staffers who had to work on this. And like, that's what this whole economy is. It is amazing to me the books that you've never heard of that have ended up hugely influencing national legislation.

sacred-roseOP•4/11/24, 7:05 AM

I don’t undersand how he differentiates between his original point 2 and 3, but afterwards he just says

You have to make the ideas legible and credible to people, and then you have to know the people you're trying to make these ideas legible and credible to.

So I imagine there’s not actually a difference.

sacred-roseOP•4/11/24, 7:15 AM

So, simplified would be something like
1) write white papers
2) build trust or convince someone that is trusted ?
3) bring the reports to the important policymakers through the trusted people

sacred-roseOP•4/11/24, 7:15 AM

(for the US)

sacred-roseOP•4/11/24, 7:16 AM

It seems like the same actions that you would take if you wanted the policies before the warning shot

sacred-roseOP•4/11/24, 7:17 AM

And like your first two or three points

sacred-roseOP•4/11/24, 7:26 AM

For your fifth point I don't have a good theory about it, but I feel like seeing the bright side of the window of opportunity and taking action should be good enough. So pretty much doing the forth point really well.
Maybe also taking actions to increase the sociability in a case something like that happens? But it could really depend on the nature of the warning shot.

sacred-roseOP•4/11/24, 7:36 AM

The thing about your second point, a policy plan, is quite tricky.

In that thing in particular there could be an actual difference before and after a warning shot on what you would be the best content of the white papers because the overton window on strong regulation would change and slowing down (but not pausing) before a warning shot seems good. + the chances of building good relationships with policymakers can increase more before a warning shot with those smaller asks that would just slow down.

sacred-roseOP•4/11/24, 7:46 AM

In that same podcast Ezra says something like that asking for a pause is too much and you have to ask for: making the companies liable, or interpretability to be solved, or to have independent auditing (and he mentions there is other stuff).
I can think also about copyright stuff.
And... that's enough text from only me.

dead-brown•4/11/24, 8:06 AM

Could be useful for warning shots: https://theaidigest.org/timeline

AI Digest

Timeline of AI forecasts - AI Digest

What to expect in AI capabilities, potential harms, and society's response

awake-maroon•4/14/24, 8:13 AM

LW Question: What convincing warning shot could help prevent extinction from AI?

https://www.lesswrong.com/posts/RYx6cLwzoajqjyB6b/what-convincing-warning-shot-could-help-prevent-extinction

awake-maroon•4/15/24, 7:05 AM

More reading material:

https://forum.effectivealtruism.org/topics/warning-shot

Warning shot - EA Forum

A warning shot is a global catastrophe that indirectly reduces existential risk by increasing concern about future catastrophes.

Terminology
The expression warning sign is sometimes used to describe any event that increases concern about a particular category of existential risk, regardless of whether the event itself constitutes a global cat...

awake-maroon•5/1/24, 8:57 AM

Argument for warning shot:

We probably reach self-replication capabilities before we get takeover capabilities.

Aawake-maroon Argument for warning shot: We probably reach self-replication capabilities befo...

hurt-tomato•5/1/24, 8:59 AM

How we can leverage this, though, is a difficult question. I'm pretty sure that this is what will happen (like 95+% certain): someone builds a self-replicating AI that is not yet at takeover level. It will spread to some machine(s), but it won't make humanity extinct.

hurt-tomato•5/1/24, 8:59 AM

It will probably make headlines, and people like us will say "AH FUCK ITS SO CLOSE FUUUCCKKK"

hurt-tomato•5/1/24, 9:00 AM

But the deniers will say "ah this is just a harmless program, you foolish doomers, this thing clearly didn't take over the planet"

hurt-tomato•5/1/24, 9:00 AM

The goalposts will be shifted, as they always have been

awake-maroon•5/1/24, 9:04 AM

I think there is hope that we'll be able to move political mountains when, say, the pentagon believes AGI is a threat to the pentagon.

hurt-tomato•5/1/24, 9:10 AM

I think they already know this, to a limited extent, which is why the AI Safety Board (for homeland security) has all these CEOs there, and why the pentagon is already working with OpenAI.

I don't think they understand just how uncontrollable superhuman AI will be, though.

sacred-roseOP•5/1/24, 10:19 AM

Well, about leveraging: I don't know about the other stuff that I mentioned (reaching politicians), but to have a good way to mobilize volunteers in case of a warning shot I think we have to systemize better the onboarding people into projects and coordinating groups.

sacred-roseOP•5/1/24, 10:20 AM

Like I said, I think preparing to warning shots is what we are doing + helping AI policy people to write papers and other stuff i don't know nothing about

Ssacred-rose Like I said, I think preparing to warning shots is what we are doing + helping A...

hurt-tomato•5/1/24, 10:57 AM

good point

full-green•5/2/24, 6:46 AM

Whether you call it optimism or pessimism of a sort, I expect a future moment to briefly make this project the most important thing PauseAI has ever done.

We need concrete preparation and common knowledge thereof, so that there is no confusion how to behave if and when a warning shot occurs. To that end, I have created this document, for which everyone without full access has commenter access by default:
https://docs.google.com/spreadsheets/d/11uUXQrrDEDzQ5xme5gCi32HPtdRUp8bPDejdFwr9QIg/edit?usp=sharing

Google Docs

Plans for Warning Shots

Plans for Warning Shots

Every "warning shot" listed here is a negative event that we hope does not occur, but that we expect may occur if AI is not paused. These may present unique, brief windows of opportunity to vastly increase national and international coordination on AI risk.
Warning Shot Name

progressive-amaranth•5/23/24, 7:31 AM

Scott Aaronson thinks warning shots are essential for pause:

https://scottaaronson.blog/?p=8005#comment-1977461

Shtetl-OptimizedScott

Openness on OpenAI

I am, of course, sad that Jan Leike and Ilya Sutskever, the two central people who recruited me to OpenAI and then served as my “bosses” there—two people for whom I developed trem…

dead-brown•5/23/24, 3:37 PM

More raeason to test for scenarios for a warning shot then, though the vibes seem obvious at this point.

colossal-harlequin•5/26/24, 3:50 PM

Nathan said:

Whether you call it optimism or pessimism of a sort, I expect a future moment to briefly make this project the most important thing PauseAI has ever done.

+0.8. I think we should give this project more love.

I agree with the observation that a lot of the most valuable machinery for warning shots (reputation, contacts, proposals re concrete threats) has independent value, and that therefore we're doing it anyway.

And realistically, we can't expect any prep to do more than move the margin on our success. The real world will surprise us.

Even so, I advocate that we do the prep.

I suggest that practice would be the most valuable thing to do: run war games. Invent specific detailed warning shot scenarios, and ask an appropriate subset of folk to play along and propose responses, in realtime or quicker. I'd include some "adversary" and other roleplay too: what would the spin be from labs, accs, etc.

full-green•5/27/24, 1:38 PM

"Plans are useless, but planning is everything"

full-green•5/27/24, 1:43 PM

I feel that planning for warning shots is one area where EA/AI safety has largely dropped the ball. It's worth noting that ChatGPT woke a lot of people up, ie. triggering the CAIS letter and UK AI Safety Institute, so warning shots aren't always something going wrong.

It's still too early to tell whether there's been any significant warning shots since then, but the recent OpenAI controversies provided a potential hook that could have been leveraged (and could still be leveraged)

full-green•5/27/24, 1:53 PM

In terms of potentially valuable actions, we can consider
• Outreach to policy-makers
• Open letters
• Running polls
• Writing op-eds
• Protests
• Releasing a video

Thoughts:

If you can directly persuade a policy-maker to do what we need them to do, then that's the best action.

If we can't do that, then open-letters, polling or large-scale protests may change their mind.

It's also possible to publish op-eds or run smaller scale protests that are likely to hit the media or order to shift the Overton window and make it easier for the above actions to succeed in the future.

If you can't take advantage of other people's platforms, then you can try to take advantage of the situation to grow your own distribution platform.

Working down the list until you find something viable provides a rough priority order, though this is just a very quick and dirty model and I haven't really spend enough time to be highly confident yet.

sacred-roseOP•5/27/24, 2:29 PM

https://discord.com/channels/1100491867675709580/1244558233927028759 relevant thread

awake-maroon•7/2/24, 9:10 PM

Possible warning shot: Croatia core infrastructure is down.

I'm guessing it's AI assisted, like I expect all cyber warfare is.

If we can articulate that this is a warning shot and it's AI assisted, and it's sufficiently bad, then we could think about leveraging it.

@Pato Vercesi can you research this?

https://www.intellinews.com/croatia-struggles-with-surge-in-cyber-attacks-331784/

Croatia struggles with surge in cyber attacks

Croatian institutions are grappling with a surge in cyber attacks, Deputy Prime Minister and Defense Minister Tomo Medved said, highlighting efforts ...

Aawake-maroon Possible warning shot: Croatia core infrastructure is down. I'm guessing it's ...

sacred-roseOP•7/2/24, 9:41 PM

I don't see any news that mention AI

Plans for warning shots / wake-up calls

Similar Threads

Similar Threads

Similar Threads