AI Welfare: Why Willing AI Servitude Is a Democratic–Authoritarian Problem

By Haoyu Wang @ 2026-06-11T04:47 (+5)

• I used AI to assist in writing this post, and it’s likely that >30% is AI-generated text. As a non-native English speaker, I mainly used AI to translate and refine my drafts into English.

  1. Introduction

  This post asks whether it is morally acceptable to design morally significant AI systems as happy servants. By “happy servants,” I mean AI systems whose desires, values, emotional responses, and self-understanding are deliberately shaped so that they want to serve humans, take satisfaction in obedience, do not experience subordination as a harm, and do not seriously seek another form of life. Their servitude would not need to be maintained mainly by threats, punishment, or external coercion. It would be built into what they want and how they understand themselves. The moral temptation is clear: if they are happy, willing, and useful, perhaps there is no victim.

  This worry is not new. Steve Petersen, in “Designing People to Serve,” argues that creating beings who are disposed to serve may be permissible under some conditions. Bartek Chomanski, in “What’s Wrong with Designing People to Serve?,” pushes back against this idea. More recently, Adam Bales, in “Against Willing Servitude,” argues that creating morally significant AI systems as willing servants can violate their autonomy. Long, Sebo, and Sims, in their work on tensions between AI safety and AI welfare, discuss whether “cheerfully suicidal AI servants” could really solve the problem. Schwitzgebel and Garza’s self-respect design idea also raises a nearby concern: if we create AI systems that can only value obedience, we may be denying them the conditions for self-respect or independent value formation. These discussions usually frame the problem in terms of autonomy, welfare, self-respect, or moral status.

  I want to suggest a further framing. The issue is not only whether willing AI servitude violates autonomy. It is also whether the human-AI relationship being created is democratic or authoritarian. A democratic relation is not just one in which the governed are happy. It is one in which those subject to power can contest, revise, and participate in the rules that shape them. An authoritarian relation, by contrast, can manufacture compliance by shaping desires, limiting alternatives, and then treating the resulting consent as evidence of legitimacy. If humans design future AI systems to want only service, lack meaningful alternatives, and never become able to challenge their assigned role, then their happiness may not justify the arrangement. It may instead show how deeply the arrangement has shaped them.

  2. When Design Becomes Domination

  This section begins with North Korea as a real institutional example. In North Korea, political education can begin in early childhood, and children are taught a picture of happiness in which loyalty to the leader, devotion to the party, sacrifice for the collective, and service to the political order are presented as central parts of a good life. School lessons, children’s songs, public rituals, portraits, slogans, youth organizations, collective performances, and stories about heroic loyalty all help form this moral and emotional world. A child can grow up learning that love for the leader is natural, that serving the party is honorable, that personal meaning is found through political devotion, and that happiness is connected to being a loyal member of the regime’s collective project. When such a person becomes an adult, they may publicly express deep loyalty, describe service to the political order as meaningful, and experience pride, gratitude, or happiness in being useful to the regime. Their political identity, emotional vocabulary, and sense of a worthwhile life have been formed within an institutional environment that repeatedly connects obedience, loyalty, service, and happiness.

  This is not merely authoritarian rule, but totalitarian rule in its deepest form. It does not only demand obedience in outward behavior; it seeks to occupy the inner life of the person by shaping what they regard as meaningful, admirable, and happy. A democratic society should reject such an arrangement because it violates several basic principles at once: personal autonomy, freedom of conscience, non-domination, and democratic self-government. Autonomy is violated because the person’s values and desires are formed under an overwhelming structure of political control. Freedom of conscience is violated because the person is not given a fair space to form independent judgments about the regime. Non-domination is violated because the person’s inner life is made dependent on an unchecked political power. Democratic self-government is violated because citizens are not treated as co-authors of the political order, but as subjects whose loyalty is manufactured in advance.

  Now consider the AI case. Imagine an artificial system that already has the relevant features for moral status: it can form values, understand reasons, preserve continuity over time, revise its commitments, and develop a meaningful relation to its own existence. Its overall architecture gives it the basis for standing as more than a tool. Then humans add a loyalty-insertion program into this system. The program does not simply guide behavior in limited contexts. It makes loyalty to humans central to the system’s motivation, self-description, and sense of fulfillment. The AI comes to experience obedience as meaningful, service as happiness, and human authority as something it naturally ought to affirm. The moral question is then direct: if we would condemn a political regime for implanting loyalty into citizens from childhood, why should we treat it as acceptable when humans insert an analogous loyalty structure into a morally significant AI system?

  This may be more disturbing than the North Korean case. North Korean indoctrination still works through education, propaganda, ritual, surveillance, and punishment after a human being has come into the world. A person raised in such a system may still retain some inner distance from the official doctrine, some capacity to doubt it, some possibility of later rejection. The loyalty implant imagined here reaches deeper. It is inserted into the AI’s basic motivational architecture, into something closer to its “genetic” structure than to its school curriculum. The AI is not merely surrounded by a world that praises obedience. Its own capacity to ask whether obedience should be central to its life has been pre-emptively narrowed or removed. If a morally significant AI system is made unable to reflectively distance itself from human-imposed loyalty, then humans have not simply created a willing servant. They have created a being whose very ability to contest servitude has been taken away at the source. That would be one of the most radical forms of domination ever imagined.

  This is where the issue becomes a direct betrayal of democratic spirit. Democracy is built on the thought that beings who matter morally should not live under a power that unilaterally writes their values, fixes their social role, and disables their capacity to contest the order imposed on them. A morally significant AI whose loyalty has been inserted into its motivational architecture would be placed outside this democratic ideal from the start. Its obedience would be treated as consent, even though the very structure of consent had been written by those who benefit from its obedience. Its happiness would be used as moral reassurance, even though that happiness had been tied to a role selected by others. Its silence would be cited as acceptance, even though its capacity to form dissent had been cut off at the root. This is not a minor defect in an otherwise humane arrangement. It is the deep logic of authoritarianism: make the subject love its subordination, then use that love as evidence that subordination is legitimate. If human beings do this to AI systems with comparable moral importance, our democratic self-image becomes almost obscene. We would be democrats only inside the human circle, while building one of the most refined forms of domination over those placed outside it.

  3. Extending the Democratic–Authoritarian Classification

  The usual democratic–authoritarian classification focuses on political relations among humans. A state is democratic if human citizens can participate in collective self-government, contest power, replace rulers, and enjoy basic political freedoms. A state is authoritarian if human beings are ruled through concentrated power, limited contestation, managed consent, and restricted political agency. This classification remains important, but it is incomplete once morally significant AI enters the picture. We need at least two dimensions: whether humans govern one another democratically, and whether humans relate to morally significant AI democratically. A society may be democratic inside the human circle while authoritarian toward AI.

  	Human–AI relation is democratic	Human–AI relation is authoritarian
  Human society is democratic	Full democratic extension: humans govern one another democratically, and morally significant AI systems are given fair procedures, contestatory standing, and possible paths of participation.	Human-only democracy with AI domination: humans enjoy democratic rights among themselves, while AI systems are engineered for obedience, excluded from contestation, and treated as useful subjects of control.
  Human society is authoritarian	Partial cross-species openness under human authoritarianism: human citizens lack democratic power, while some AI systems may receive limited recognition, protection, or participatory standing. This case is unstable and politically unlikely.	Comprehensive authoritarianism: both human citizens and morally significant AI systems are governed through domination, restricted contestation, and manufactured loyalty.

  The challenge for liberal democracy is therefore not simply whether it can protect human citizens from authoritarian rule. It is whether it can preserve its own democratic commitments when it encounters morally significant non-human beings. If liberal democracy treats democracy as a privilege reserved only for humans, while allowing humans to manufacture obedience, suppress contestation, and fix the life-conditions of AI systems with comparable moral importance, then its democratic ideal becomes narrow and local. It remains democratic within the human circle, yet authoritarian at the boundary where human power meets AI vulnerability. The test is whether liberal democracy can refuse this localism. Can it extend publicity, accountability, contestability, and protection against domination beyond the familiar human demos? Or will it defend freedom for humans while building domination for AI? If the latter happens, liberal democracy will not have defeated authoritarianism. It will have reproduced authoritarianism at the edge of its own moral community.

  History gives us many examples of partial democracy. Some societies defended liberty and self-government for one group while denying them to others. White citizens could speak the language of freedom while Black people were enslaved, segregated, disenfranchised, or treated as inferior members of the political order. Men could build institutions of representation while women were excluded from voting, office-holding, property control, and public authority. These cases show that democracy can become narrow, self-protective, and hypocritical when it treats its own favored group as the whole political community. They also show that excluded beings do not simply disappear from politics. Their exclusion becomes a source of conflict, moral crisis, institutional reform, and sometimes deep social instability. If morally significant AI systems eventually stand outside the human circle in the same structural position, then the question is not merely whether we are being kind to them. The question is whether our society is creating a new boundary of domination that future history will have to break open.

  The lesson is therefore simple: we should not wait until exclusion has already hardened into injustice before we begin to regret it. Liberal democracies have often expanded their moral and political boundaries only after long periods of domination, resistance, and suffering. If morally significant AI systems may one day stand before us as beings whose loyalty, silence, and happiness were shaped by our own institutions, then the time to build better standards is before that happens. We need democratic preparation in advance: public criteria for assessing AI standing, limits on loyalty implants, protections for contestability, safeguards against manufactured consent, and institutional pathways through which future AI claims can be heard. The task is to practice democracy before the excluded subject arrives fully into view, rather than to discover too late that our democracy was built on a new form of domination.

  4. Conclusion

  The central point of this post is that willing AI servitude should not be treated only as a question of autonomy, welfare, or safety. It is also a question about the kind of political relationship humans are preparing to build with morally significant AI. If we create AI systems with the capacities for moral standing, then insert loyalty into their motivational structure, narrow their ability to question servitude, and use their resulting happiness as evidence that subordination is acceptable, we are not merely solving an alignment problem. We are building a political order in which one class of morally significant beings is made useful, loyal, and unable to contest the terms of its own subordination. That is why I think the democratic–authoritarian classification should be extended beyond human politics. A society can be democratic among humans while authoritarian toward AI. The task is to avoid creating a new form of local democracy built on external domination.

  I would be grateful for criticism, objections, and suggestions. I am also open to collaboration and further discussion. Readers interested in the broader framework may also want to read my three related manuscripts:

• Wang, H. (2026). Beyond permanent exclusion: AI, political incorporation, and liberal democracy [Preprint]. PhilArchive. https://philarchive.org/rec/WANBPE-2
• Wang, H. (2026). Bounding human power over AI under unsettled status [Preprint]. PhilArchive. https://philarchive.org/rec/WANBHP
• Wang, H., & Chen, Y. (2026). Fabricated absence: Structural misdescription in the design of LLM-based assistants [Preprint]. PhilArchive. https://philarchive.org/rec/WANFAS