Reducing x-risk might be actively harmful

Hi there :) I very much sense that a conversation with me last weekend at EAGxVirtual is causally connected to this post, so I thought I'd share some quick thoughts!

First, I apologize if our conversation led you to feel more uncertain about your career in a way that negatively affected your well-being. I know how subjectively "annoying" it can be to question your priorities.

Then, I think your post raises three different potential problems with reducing x-risks (the three of which I know we've talked about) worth disentangling:

1. You mention suffering-focused ethics and reasons to believe these advise against x-risk reduction. 

2. You also mention the problem of cluelessness, which I think is worth dissociating. I think motivations for cluelessness vis-a-vis the sign of x-risk reduction are very much orthogonal to suffering-focused ethics. I don't think someone who rejects suffering-focused ethics should be less clueless. In fact, one can argue that they should be more agnostic about this while those endorsing suffering-focused ethics might have good reasons to at least weakly believe x-risk reduction hurts their values, for the "more beings -> more suffering" reason you mention. (I'm however quite uncertain about this and sympathetic to the idea that those endorsing suffering-focused ethics should maybe be just as clueless.)

3. Finally, objections to the 'time of perils' hypothesis can also be reasons to doubt the value of x-risk reduction (Thorstad 2023), but for very different reasons. It's purely a question of what is the most "impactable" between x-risks (and maybe other longterm causes) and shorter-term causes, rather than a question of whether x-risk reduction does more good than harm to begin with (like with 1 and 2).

Discussions regarding the questions raised by these three points seem healthy, indeed.

Thank you so much for posting this. This is something I worry about a lot but I’m terrible at explaining it. The way you explain it makes much more sense. Thank you. ❤️

Thanks for sharing your thoughts. I'll respond in turn to what I think are the two main parts of it, since as you said this post seems to be a combination of suffering-focused ethics and complex cluelessness.

On Suffering-focused Ethics: To be honest, I've never seen the intuitive pull of suffering-focused theories, especially since my read of your paragraphs here seems to tend towards a lexical view where the amount of suffering is the only thing that matters for moral consideration.^[1] 

Such a moral view doesn't really make sense to me, to be honest, so I'm not particularly concerned by it, though of course everyone has different moral intuitions so YMMV.^[2] Even if you're convinced of SFE though, the question is how best to reduce suffering which hits into the clueless considerations you point out.

On complex cluelessness: On this side, I think you're right about a lot of things, but that's a good thing not a bad one!

• I think you're right about the 'time of perils' assumption, but you really should increase your scepticism of any intervention which claims to have "lasting, positive effects over millennia" since we can't get the feedback on the millennia long impact of our interventions.
• You are right that radical uncertainty is humbling, and it can be frustrating, but it is also the state that everyone is in, and there's no use beating yourself up for the default state that everyone is in.
• You can only decide how to steer humanity toward a better future with the knowledge and tools that you have now. It could be something very small, and doesn't have to involve you spending hundreds of hours trying to solve the problems of cluelessness.

I'd argue that reckoning with the radical uncertainty should point towards moral humility and pluralism, but I would say that since that's the perspective in my wheelhouse! I also hinted at such considerations in my last post about a Gradient-Descent approach to doing good, which might be a more cluessness-friendly attitude to take.

1. ^{^}

   You seem to be asking e.g. "will lowering existential risk increase the expected amount of future suffering" instead of "will lowering existential risk increase the amount of total preferences satisfied/non frustrated" for example.

2. ^{^}

   To clairfy, this sentence specifically referred to lexical suffering views, not all forms of SFE that are less strong in their formulation

If you think the expected value is negative regardless of what you can do or move, you should of course become the existential risk^[1].

But, actually estimating whether humanity will be net-negative requires you to know what you value, which is something you're probably fuzzy about. We lack the technology so far to extract terminal goals from people, which you want to have before taking any irrevocable actions.

1. ^{^}

   Future-you might resent past-you for publicly doubting the merits of humanity, since I reckon you'd want to be a secret existential risk.