I have a feeling many people asking for AI regulation, are really asking for regulation to protect them or others from job losses they see as inevitable.
This seems totally contrary to regulation around de-risking AI development so that it can safely accelerate.
You touched on this toward the end, but I wonder if you have any thoughts on how to deal with this contradiction. If any regulation in the US happens now it feels much more likely (to me) to be EU style than what you propose.
In theory, the basic concept undergirding this bears merit. But boy is there real concern with centralizing all authority on AI development to a select few “industry leaders” (step #1). I think this is a very bad idea supported with honestly good intentions.
If AI is as powerful as indicated, and thus deserves a Manhattan-style development safety structure, ensuring that power is controlled by a very select few gives those people nearly unlimited control of humanity and its future. And it does so behind closed doors. Even Kim-Jong-Un would blush at such an opportunity.
The concepts mentioned in this article are an understandable starting ground for ideation, but require dramatically expanded considerations.
It is not obviously feasible to come up with any sort of principled definition of "Green" / "Orange" / "Red" types of AI research, given that (a) the plausible "experts" seem very far apart on the question of what exactly is high-risk and (b) they don't and in some sense can't give justifications for their positions grounded in real-world evidence rather than thought experiments. You can have principled risk levels in types of *application* of research, analogous to the DO-178C levels of software deployment risk in avionics, but that is a very different thing.
"While this runs the risk of leapfrogging to a Yudkowsky-style superintelligence, I’d rather we test the “hard takeoff” hypothesis under controlled conditions where we might catch an unfriendly AI in the act. If alignment is as hard as the Yuddites think, the project would at worst provide a non-speculative basis for shutting it all down."
You're basically assuming away hard takeoff scenarios where what humans call "airgap" can be bypassed with sufficient subtelty and/or that this hypothetical intelligence can't fake alignment enough to get access to non-airgapped systems.
If you're wrong, you just accelerate destruction. Why are you so confident that this is a good idea?
Great piece. Lots to unpack. Could use an application of the two-but rule for each plank of the plan. Like this: https://www.2buts.com/p/ai-regulation
"The future may be hidden behind a dense fog, but we can at least turn on our high-beams."
...you're not supposed to turn on your high beams in fog. The water molecules reflect the water back at you and make it harder to see.
I have a feeling many people asking for AI regulation, are really asking for regulation to protect them or others from job losses they see as inevitable.
This seems totally contrary to regulation around de-risking AI development so that it can safely accelerate.
You touched on this toward the end, but I wonder if you have any thoughts on how to deal with this contradiction. If any regulation in the US happens now it feels much more likely (to me) to be EU style than what you propose.
In theory, the basic concept undergirding this bears merit. But boy is there real concern with centralizing all authority on AI development to a select few “industry leaders” (step #1). I think this is a very bad idea supported with honestly good intentions.
If AI is as powerful as indicated, and thus deserves a Manhattan-style development safety structure, ensuring that power is controlled by a very select few gives those people nearly unlimited control of humanity and its future. And it does so behind closed doors. Even Kim-Jong-Un would blush at such an opportunity.
The concepts mentioned in this article are an understandable starting ground for ideation, but require dramatically expanded considerations.
It is not obviously feasible to come up with any sort of principled definition of "Green" / "Orange" / "Red" types of AI research, given that (a) the plausible "experts" seem very far apart on the question of what exactly is high-risk and (b) they don't and in some sense can't give justifications for their positions grounded in real-world evidence rather than thought experiments. You can have principled risk levels in types of *application* of research, analogous to the DO-178C levels of software deployment risk in avionics, but that is a very different thing.
I don't understand this point:
"While this runs the risk of leapfrogging to a Yudkowsky-style superintelligence, I’d rather we test the “hard takeoff” hypothesis under controlled conditions where we might catch an unfriendly AI in the act. If alignment is as hard as the Yuddites think, the project would at worst provide a non-speculative basis for shutting it all down."
You're basically assuming away hard takeoff scenarios where what humans call "airgap" can be bypassed with sufficient subtelty and/or that this hypothetical intelligence can't fake alignment enough to get access to non-airgapped systems.
If you're wrong, you just accelerate destruction. Why are you so confident that this is a good idea?