AI as a science, and three obstacles to alignment strategies

Executive summary: The post argues that AI alignment is difficult due to the intertwining of alignment and capabilities research, the difficulty of distinguishing real vs. fake solutions, and the high risk of theories failing on their first critical application.

Key points:

1. Efforts to understand and aim AI systems often also uncover ways to make them more capable, allowing unchecked development to continue.
2. Distinguishing scientifically-grounded alignment solutions from insufficient ones will be very difficult for regulators.
3. Even rigorous alignment theories may fail catastrophically the first time they are tested in a real superintelligence.
4. Recommends civilization pursue non-AI routes to transhumanism like uploading instead.
5. Developing scientific theories of artificial cognition is important but risks accelerating progress.
6. Minimal pivotal tasks are worth considering to contain failures of new alignment theories.

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.