"If we go extinct due to misaligned AI, at least nature will continue, right? ... right?"

I think literal extinction is unlikely even conditional on misaligned AI takeover due to:

• The potential for the AI to be at least a tiny bit "kind" (same as humans probably wouldn't kill all aliens).^[1]
• Decision theory/trade reasons

This is discussed in more detail here and here.

Insofar as humans and/or aliens care about nature, similar arguments apply there too, though this is mostly beside the point: if humans survive and have (even a tiny bit of) resources they can preserve some natural easily.

I find it annoying how confident this article is without really bother to engage with the relevant arguments here.

(Same goes for many other posts asserting that AIs will disassemble humans for their atoms.)

(This comment echos Owen's to some extent.)

1. ^{^}

   This includes the potential for the AI to have preferences that are morally valueable from a typical human perspective.

I just wanted to say that the new aisafety.info website looks great! I have not looked at everything in detail, just clicking around a bit, but the article seem of good quality to me.

I will probably mainly recommend aisafety as an introductory resource.

I think this is a plausible consequence, but not a clear one.

Many people put significant value on conservation. It is plausible that some version of this would survive in an AI which was somewhat misaligned (especially since conservation might be a reasonably simple goal to point towards), such that it would spend some fraction of its resources towards preserving nature -- and one planet is a tiny fraction of the resources it could expect to end up with.

The most straightforward argument against this is that such an AI maybe wouldn't wipe out all humans. I tend to agree, and a good amount of my probability mass on "existential catastrophe from misaligned AI" does not involve human extinction. But I think there's some possible middle ground where an AI was not capable of reliably seizing power without driving humans extinct, but was capable if it allowed itself to do so, could wipe them out without eliminating nature (which would presumably pose much less threat to its ascendancy).