Merging Everybody Into Hell
By Tim Chan @ 2026-10-13T14:32 (+6)
Here’s a thought experiment. As far as I know, no one has written about it yet.
There's a mind fated for hell. Its suffering will last a very long time, and nothing can change that.
You get to decide what happens at the end of everybody else's life. There are two options:
Option A: Everybody's life just ends normally and nothing happens. Assume there's no afterlife.
Option B: At the end of everybody's lives, everybody is merged into the mind fated for hell. Assume this is possible. Merging doesn’t make the mind’s hell any more intense or any longer, and it doesn’t create extra copies of the mind. But from every person's point of view, they close their eyes at the end of their lives and wake up as that one mind in hell. In return, $1 goes to your favorite charity.
Which do you choose?
I'd choose A. Everybody going to hell is obviously bad, and definitely not worth a dollar to any charity.
And yet standard utilitarianism chooses B. By its count, merging neither adds nor subtracts welfare. There's still one mind in hell, even though it came from a merge of many minds. So, it first says the two options are equal in value. Then, $1 breaks the tie. Now it says you should send everybody to hell.
This is the verdict of any standard utilitarianism that adds up the welfare of experiences - total, average, and even negative - as long as $1 does some good and the merging is considered morally neutral.
The mirror case works too. Offer $1 to charity for letting everybody's lives simply end, instead of merging them into a mind fated for heaven, and standard utilitarianism accepts the dollar, depriving everybody of heaven.
That can't be right. What's going on?
The problem here is that standard utilitarianism doesn't explicitly track what each person goes on to experience. It only adds up experiences as seen from the outside, from a third-person 'view from nowhere'. What you subjectively continue onto is none of its business. That's mostly fine for linear lives, where each experience simply leads to the next, but not for lives that can be copied and merged, and you can construct more counterexamples like this one with those operations.
One could bite the bullet and say only the hell itself matters, but not the fact that multiple people would come to experience it. But then one would have to be indifferent between dying peacefully and waking up in that hell tomorrow, everything else unchanged. I don’t think many people are indifferent.
I think the fix is to care about each perspective’s subjective futures, in addition to its immediate welfare. I’ve proposed that utilitarianism should be based on a Markov reward process over types of experiences. It handles this case, and I think it also fixes problems with identical copies that others have noted.
It feels like we should have started discussing this counterexample a long time ago. Ethics has only handled the extremely special case of linear lives. We may soon be headed for takeoff, yet as a civilization we haven’t even worked out the general case and what it implies.
turchin @ 2026-10-13T17:53 (+6)
Similar argument was presented as utilitarian argument against quantum immortality by some writers on LW.
They argue: The number of branches of multiverse where I survive is declining after quantum suicide, so the total utility is declining, so I should not care about QI.
Quantum immortality instead is based on the idea of preserving continuity.
Traditional utilitarianism doesn't care about continuity (and also about uniqueness and about complexity of experience). I think that we should update utilitarianism, as it was created as an apoximation of what is really good and should not be taken as religious dogma.
Brad West🔸 @ 2026-10-13T16:02 (+2)
This seems deeply confused. You are discussing more consciousnesses going into an extreme state of disvalue (hell). You say that there is no multiplicity of minds, but this is creating more negatively valenced experience, which is what utilitarianism seeks to prevent.
Whatever faults utilitarianism has, it does not endorse creating enormous disvalue, at least without creating more enormous value or reduction of disvalue elsewhere. Your dollar to charity comes no where close to offsetting the immense suffering of option B per utilitarianism.
Tim Chan @ 2026-10-13T16:28 (+1)
I agree with you that B is much worse. But I disagree that standard utilitarianism, as it is formulated now, reaches our conclusion that B is worse than A.
In the thought experiment, merging into one mind doesn’t add any negative experiences. The hell mind’s experiences are exactly the same in both options. It's still just one mind suffering in hell, and that seems to be all standard utilitarianism counts.
Rather than just counting immediate experience, I think it makes more sense to also account for subjective continuity, i.e. that multiple minds lead into the one mind in hell.
dan.pandori 🔸 @ 2026-10-13T15:25 (+1)
If two toll roads merge into one, should the merged road cost twice as much per mile? No, because its now one road. Having two entries does not make it twice as valuable. Why are minds different?
Also, the notion of merging minds only makes sense (to me) under some form of dualism. Ie, you need some non-physical mind stuff to do a complete merge. I don't think imagining me & you merging minds can result in one entity that carries both 100% of my weight and 100% of your weight. At best we blur together and get a fraction of each. More likely, both of us are functionally dead post-merge and there is a new 3rd conscious being.
Tim Chan @ 2026-10-13T15:39 (+1)
I think that analogy doesn’t quite fit. The road doesn’t have a point of view, but people do. Two people headed to hell seems worse than one.
My question is really “if merging were possible, would we merge everybody into hell?” Copying minds didn’t seem possible a decade ago, but assuming digital sentience is a thing, it now seems quite routine. If we wouldn’t merge everybody into hell, then there are some unresolved issues in utilitarianism.
dan.pandori 🔸 @ 2026-10-13T17:30 (+2)
Merging & copying gets weird, no questions.
I think your notion of counting distinct paths has a less satisfactory answer than 'standard' util though.
Here's a scenario I think shows the weirdness. Imagine a single mind. We copy this mind into two instantiations, which have slightly different experiences for a tenth of a second. We then merge these minds. We repeat this process a thousand times. There are then 2^1000 distinct experience paths here. But counting the future experiences of the 2^1000 merged mind at that full multiple seems quite silly.
Tim Chan @ 2026-10-13T19:53 (+1)
Agreed that counting paths at a full multiple would seem silly.
My proposed fix is a bit different. It cares about immediate welfare plus the welfare of future types of observer-moments, where futures are weighted by how likely we are to continue into each one, so each time step adds up to a weight of one. In your example, splitting shares the future between the copies and merging adds the shares back up, so the merged mind’s future isn’t multiplied by 2^1000.
Each brief copy does count as its own observer-moment with its own future, so the splitting adds a little. But that grows with the number of experiences that actually happened, not exponentially.
I don’t go into the fix in this post. Post was mainly to raise the question.