Will Sentience Make AI’s Morality Better?

By Ronen Bar @ 2025-05-18T04:34 (+25)

TLDR

I propose to explore what I call the AI Sentience Ethics Conundrum: Will the world be better with a sentient AI or an insentient AI?

In this article, we will explore whether a sentient AI would behave more ethically and robustly than an insentient one. I will propose an initial framework of four key factors - understanding reality, understanding morality, power and willingness to act - that are potentially influenced by AI sentience. Those factors are key in determining how it behaves, i.e., the degree of AI Moral Alignment

In the next post, I will address how AI sentience might affect the world, taking a broader view than just AI's behavior, including its own welfare.

Existing opinions on this questions

The Sentience Ethics Conundrum represents a crucial yet underexplored aspect of AI's potential impact on the world. In his Nonsentient Optimizers post, Eliezer Yudkowsky dismisses this concern entirely, arguing that those who believe AI requires empathy fundamentally misunderstand utility functions and decision theory, unable to conceive of intelligence operating unlike human minds.

However, most AI Safety and EA people I spoke with seem to disagree with Yudkowsky's conclusion. It seems this question is very hard, deep, and complicated, and I find it one of the most essential questions in AI safety and Moral Alignment. It is the kind of question where there is much more than meets the eye.

Epistemic status

This analysis presents preliminary thoughts mapping critical questions and postulations emerging from this conundrum. The simple framework suggested in this article is a very rough starting point for thinking about this topic, which in my opinion should be developed much further.

The four dependent factors

Examples are provided in parentheses.

The key question of this article

We will examine the influence of AI Sentience, the independent factor, on the four dependent factors, which in turn influence AI Moral Alignment. All this will be done to gain insights into the question: will a sentient AI behave more ethically than an insentient one, and if so, what sentience intensity, valence (spectrum & intensity), and particular core traits will optimize its morality?

For simplicity, in this post I disregard the option of sentience without valence, though this also deserves consideration.

When I say 'should we build a sentient AI?,' I mean whether we—or an AI—should build it.

In this post I will use sentience and consciousness interchangeably.

AI sentience and the four dependent factors - questions and thoughts

AI Sentience and 🌀 Understanding Reality

What unique information does an agent obtain through subjective experience?

Does an agent need sentience to figure out what causes sentience?

Will an insentient AI be able to better solve the conundrum

Is it possible to create ASI without it being sentient

The uncertainty problem

AI Sentience and ⚖️ Understanding Morality

Can genuine moral understanding exist without subjective experience?

Robustness?

Intrinsic alignment, autonomy, recursive ethical self-improvement and complexity

Moral judgment will be damaged by experiences?

What we know from human sentience–ethics interactions

Different kinds of consciousness, valence, senses and time

AI Sentience and 🤖 AI Power

AI Sentience and 🎬 Willingness to Act

Influence on AI behavior

According to my suggestions in this post, AI's behavior stems from a combination of an AI’s understanding of reality, morality, its power, and willingness to act according to the first three factors. Some other questions that relate multiple factors at ones should also be examined, such as:

Research suggestions

Epilog