NTIA Solicits Comments on Open-Weight AI Models (due within 30 days)

U.S. Department of Commerce’s National Telecommunications and Information Administration (NTIA) launched a Request for Comment on the risks, benefits and potential policy related to advanced artificial intelligence (AI) models with widely available model weights.

Comments are due within 30 days of publication of the Request for Comment in the Federal Register.

https://www.ntia.gov/federal-register-notice/2024/dual-use-foundation-artificial-intelligence-models-widely-available

Press release: https://www.commerce.gov/news/press-releases/2024/02/ntia-solicits-comments-open-weight-ai-models

Dual Use Foundation Artificial Intelligence Models with Widely Avai...

SUMMARY On October 30, 2023, President Biden issued an Executive Order on “Safe, Secure, and Trustworthy Development and Use of Artificial In...

U.S. Department of Commerce

NTIA Solicits Comments on Open-Weight AI Models

Today, the Department of Commerce’s National Telecommunications and Information Administration (NTIA) launched a Request for Comment on the risks, benefits and potential policy related to advanced artificial intelligence (AI) models with widely available model weights – the core component of AI systems.

dead-brown•3/4/24, 11:57 AM

Project Tracker: https://maximolog.notion.site/NTIA-Solicits-Comments-on-Open-Weight-AI-Models-due-within-30-days-2751cb96888f477a83570cafe58fc297?pvs=4

Maxime Fournes's Notion on Notion

NTIA Solicits Comments on Open-Weight AI Models (due within 30 days...

Description

urgent-maroon•3/19/24, 4:28 AM

anyone working on this, or interested in doing so? I'm thinking about it...anything I submit will not be remotely authoritative or comprehensive, but it could be worthwhile

urgent-maroon•3/19/24, 4:53 AM

very rough list of materials that might be relevant in preparing a comment (some that i'd want to quote/cite explicitly, others that i'd just want to keep in mind):
RAND report that I haven't read yet, no doubt someone at NTIA is reading it but maybe there are parts worth drawing their attention to: https://www.rand.org/pubs/working_papers/WRA2849-1.html
https://arxiv.org/abs/2310.20624 on unRLHFing Llama 2-Chat and associated LW posts (https://www.lesswrong.com/posts/qmQFHCgCyEEjuy5a7/lora-fine-tuning-efficiently-undoes-safety-training-from#comments, https://www.lesswrong.com/posts/3eqHYxfWb5x4Qfz8C/unrlhf-efficiently-undoing-llm-safeguards#comments)
Gladstone report mentions some concerns with this briefly in section 1.5.1: https://discord.com/channels/1100491867675709580/1216810494992847019/1216842819109519561
Anthropic RSP is evidence they take it seriously: https://www.anthropic.com/news/anthropics-responsible-scaling-policy
for the proposition that "some regulation (especially restricting the publication of AI research and sharing of model weights) would differentially slow foreign AI progress!" - https://www.lesswrong.com/posts/YguseW2zMYe8tMCbW/cruxes-on-us-lead-for-some-domestic-ai-regulation
Paul Christiano has a couple of thoughts re: OpenAI preparedness framework and model weights: https://www.lesswrong.com/posts/G2AghCGjGCkhWiFDd/openai-s-preparedness-framework-praise-and-recommendations
Zvi has a little discussion here: https://www.lesswrong.com/posts/emo2hAvq6p7Pn4Pps/ai-42-the-wrong-answer#Open_Foundation_Model_Weights_Are_Unsafe_And_Nothing_Can_Fix_This
some discussion on a couple PauseAI pages: https://pauseai.info/cybersecurity-risks#mitigating-ai-cybersecurity-risks, https://pauseai.info/scenarios#cyberterrorism

urgent-maroon•3/19/24, 5:04 AM

tagging @casejp since you seemed to feel strongly about this a while back (https://discord.com/channels/1100491867675709580/1119539399034683462/1169496434639376385) - have you done any writing or research toward the RFC that you'd like to share?

urgent-maroon•3/21/24, 5:21 AM

here's what I've got so far. Hoping to add some more on other parts of questions 2, 5, 7, and 8, but I'm not sure how much I'll get done before the deadline or how long a comment it's useful to submit. I'm very interested in feedback on what I have so far, suggestions, or best of all, inspiring others here to submit their own comments

modelweight-draft.docx9.65KB

urgent-maroon•3/21/24, 5:23 AM

so far this is all very much in harm-mitigation mode, though I hope to put in a few words for a more general pause in a response to a later question

moderate-tomatoOP•3/21/24, 9:54 PM

Written comments must be received on or before March 27, 2024.

urgent-maroon•3/22/24, 5:40 AM

this paper from Qi et al is imo more impressive (i.e. scary w.r.t. the risks of LLM fine-tuning) than the Llama UnRLHF paper I linked above, but it doesn't focus solely on open foundation models - they get similar results from fine-tuning GPT 3.5 Turbo via the provided APIs
https://arxiv.org/abs/2310.03693
I am planning to mention it but not sure how much to focus on it/whether it would be worth someone else focusing a comment on it specifically. I feel like it's a useful part of a case for pausing, but I'm not sure if it's useful for discussion of public model weights specifically (and maybe it's even counterproductive, if it persuades them that closed models aren't any better)

arXiv.org

Fine-tuning Aligned Language Models Compromises Safety, Even When U...

Optimizing large language models (LLMs) for downstream use cases often involves the customization of pre-trained LLMs through further fine-tuning. Meta's open release of Llama models and OpenAI's APIs for fine-tuning GPT-3.5 Turbo on custom datasets also encourage this practice. But, what are the safety costs associated with such custom fine-tun...

urgent-maroon•3/22/24, 5:43 AM

I think the thing to emphasize is maybe that with closed models the foundation model developer still has some chance to do something about the problem

they can shut down fine-tuning APIs if they're being misused, or try to better control how the APIs are used [to be clear I am not optimistic about any of this as a general alignment strategy, but it seems better than not being able to do that] - whereas with public model weights there's no such opportunity

Uurgent-maroon tagging @casejp since you seemed to feel strongly about this a while back (https...

full-green•3/22/24, 2:08 PM

Yeah, I haven't really worked on this tbh. Sorry, I'm spread pretty thin atm

urgent-maroon•3/22/24, 2:09 PM

No worries, just figured it was worth asking!

full-green•3/23/24, 5:39 PM

AI-Plans is hosting an event for this starting on March 26, 2024:
https://lu.ma/RFC-Law-a-Thon
(Sorry that it's a little last-minute!)

AI Law-a-Thon · Luma

Overview
AI Plans is hosting a Law-a-Thon, pairing lawyers and people versed in AI Alignment, aiming to give high-quality feedback for an NTIA Request For Comment (RFC).
Join the AI Plans...

urgent-maroon•3/24/24, 1:32 PM

won't be able to make it, but I'm glad you're putting that event together!

moderate-tomatoOP•3/24/24, 10:12 PM

Great event! Sorry, it’s outside my time zone so I won’t be able to attend.

Best for Americans to attend this one since NTIA is a U.S. agency

urgent-maroon•3/25/24, 8:53 AM

here's what I have at this point, definitely still interested in feedback but not likely to add a ton of content now
I am concerned that the paragraph where I advocate for a pause (7a) actually makes the whole thing worse in terms of advocacy for caution with model weights

modelweight-draft.docx14.14KB

urgent-maroon•3/26/24, 9:52 PM

For anyone who went to the AI-Plans event on this RFC, do you have any info/takeaways from it that you'd like to share here?

moderate-tomatoOP•3/27/24, 2:48 AM

One day left for comments

https://x.com/daniel_271828/status/1772803359285903538?s=46&t=_xX_kazwTFLs2wukisAuLg

Daniel Eth (yes, Eth is my actual last name) (@daniel_271828) on X

The NTIA is gathering public comments on "potential benefits, risks, and implications of [open sourcing of] dual-use foundation models." There is 1 day left to comment, & after a month of comments being open there are only 157 comments. If you have thoughts, comment!

Twitter

•

3/27/24, 1:50 AM

Uurgent-maroon For anyone who went to the AI-Plans event on this RFC, do you have any info/take...

moderate-tomatoOP•3/27/24, 2:55 AM

@Nathan Metzger, @Kabir Kumar

urgent-maroon•3/27/24, 3:51 AM

Just submitted my comment. Note that the comment field on regulations.gov is limited to 5000 characters - I put an abridged version there and my full comment as an attachment

spotty-amber•3/28/24, 1:56 AM

we'll do another law-a-thon for this

Uurgent-maroon For anyone who went to the AI-Plans event on this RFC, do you have any info/take...

spotty-amber•3/28/24, 1:56 AM

in my personal opinion, the previous RFC (haven't looked at this one yet) showed that whoever wrote it basically has no idea what they're talking about

spotty-amber•3/28/24, 1:57 AM

publishing the results of the law-a-thon in a few mins

Sspotty-amber in my personal opinion, the previous RFC (haven't looked at this one yet) showed...

full-green•3/28/24, 3:20 AM

Not sure what you mean by previous RFC and this RFC. Isn't there only one, due today?

spotty-amber•3/28/24, 3:20 AM

There's more by NTIA and other orgs

spotty-amber•3/28/24, 3:20 AM

Is this one not the one in April/May?

full-green•3/28/24, 3:22 AM

This is the one for March 27th. I haven't heard of any others. Those might need their own project post, or perhaps this post could be converted to being about RFCs in general.

spotty-amber•3/28/24, 3:22 AM

Yeah, the march 27th one seems like it's needing a lot of info

spotty-amber•3/28/24, 3:22 AM

The questions were off

spotty-amber•3/28/24, 3:23 AM

Lots of desire to want open source to be good

moderate-tomatoOP•3/28/24, 7:55 AM

Congrats to everyone who submitted a comment

NTIA Solicits Comments on Open-Weight AI Models (due within 30 days)

Similar Threads

Similar Threads

Similar Threads