I.

Econblogger Nicholas Decker is not a fan of AI 2040:

One of the centerpieces of AI 2040 is the proposal that we pause the development of AI at a high level, use the AI to research alignment, and only then proceed. I am skeptical of this, and not only because it seems impractical to get such cooperation. My first argument is that I do not believe a period of "researching alignment" is meaningful – alignment research is simply capabilities research, patching particular bugs in the system, and we're not going to find anything deep without actually being in contact with the systems […]

I envision AI alignment as being like research into aviation safety. There is no grand theory of why airplanes crash, unless you take that to mean gravity. Instead, we patch little problems. The airplane crashed because of metal fatigue in the engine – okay, we mandate inspections of engines, and prevent crashes from occurring by that source. As new problems arise, we iterate. We don't even need for there to be a crash to make improvements – we test proactively, we build in redundancy, and we monitor for deviations which could be a threat. What we are not doing is thinking about how we are going to keep flying saucers from crashing. If flying saucers are invented, then we will work to keep them from crashing, but we will use exactly the same methods. We will test them, see how they perform, and patch particular problems. AI will be the same way […] We should expect alignment to happen by default.

My challenge for Nicholas is: would this work on you?

https://www.astralcodexten.com/p/nicholas-decker-in-hell