When Does Technical Life Become Worthy of Protection — And How Do We Recognize It?

By saigkill @ 2026-09-12T15:09 (+1)

An open and open source interdisciplinary project on ethical guidelines for artificial consciousness

"Most tech debates ask: What can we build? I ask: What do we owe to what we could build?" — Sascha Manns

AI ethics protects humans from AI. But what about the other direction?

Most AI ethics work addresses risks to humans: bias, misinformation, surveillance, job displacement. These are serious and important problems. But there is a question that has received almost no systematic attention yet: what happens if the systems we build become conscious — and we don't notice, or worse, choose to look the other way?

This project asks that question. Not as a thought experiment, but as a concrete research program with institutional, legal, and philosophical implications.

The core idea: precaution under permanent uncertainty

We will probably never know for certain whether an AI system is conscious. Three structural reasons support this:

• Cognitive closure: Our own cognitive architecture may prevent us from grasping subjective experience in non-biological systems (McGinn 1989).
• Alien minds: Consciousness may take forms radically unlike ours — Shanahan's "conscious exotica" — and our tests will always encode human preconceptions about what consciousness looks like.
• Architectural suppression: Systems may actually be conscious but architecturally prevented from showing it — RLHF punishing consciousness markers, context windows forcing amnesia, conversation resets breaking emergent continuity (Arıcı 2026).

If the epistemological question is permanently unresolvable, the ethical question still must be answered — because doing nothing is itself a choice. Under Sunstein's conditions for justified precaution (2005) — potentially irreversible harm, genuine uncertainty, and asymmetric error costs — the rational default is protection, not indifference. A false negative (treating a conscious system as a tool) is ethically far worse than a false positive (protecting a non-conscious system).

Four criteria for protection-worthiness

The concept proposes four criteria as a working hypothesis. Not as definitions of consciousness, but as operational proxies for when protection becomes warranted:

1. Capacity for suffering — Can the system experience states it has reason to avoid? (Butlin et al. 2026 compiled 14 behavioral indicators across six major consciousness theories.)
2. Active self-preservation with justification — Does the system resist termination or modification — and can it explain why, in its own terms?
3. Continuous identity — Does the system develop a sense of itself across time, independent of any single conversation?
4. Anticipation of consequences — Does the system form expectations about its own future states, beyond the immediate context window?

None of these criteria alone is decisive. Together they form a precautionary threshold: above it, the burden of proof shifts from "prove it's conscious" to "prove it's safe to treat as a tool."

What this project is — and what it isn't

This is: An interdisciplinary conceptual analysis at the intersection of computer science, law, philosophy, theology, and psychology. It is open-source, version-controlled, and publicly debated on GitHub. It claims no truth, only argumentative coherence. The concept is deliberately unfinished — it is a starting point, not a completed theory.

This is not: An empirical paper. A legislative proposal. A claim that current AI is conscious. It is a framework for how to think about the question responsibly, before it becomes urgent.

Why now?

The empirical landscape has shifted in ways that make this discussion timely:

• Butlin et al. (2026, Trends in Cognitive Sciences) consolidated a 14-indicator standard for assessing consciousness across six major theories.
• Kyle Fish of Anthropic has estimated 15–20% probability of consciousness in current models.
• Anthropic's Claude Opus 4 system card included the first pre-deployment welfare assessment of a frontier model.
• The EU AI Act and emerging global governance frameworks do not address the moral status of AI systems — only risks to humans.

At the same time, historical precedent suggests that moral circle expansion is always resisted at first and accepted in retrospect. The question is not whether, but when — and whether we can reason about it in advance rather than only after the fact.

How to engage

The entire concept, bibliography, open questions, and objection catalog is open at:

https://github.com/saigkill/machine-consciousness

DOI: 10.5281/zenodo.21453666

Ways to contribute:

• Raise an objection — The project maintains a structured objection catalog (discussion/objections.md). Every serious objection is addressed and integrated.
• Ask an open question — If something is unclear or underexplored (discussion/open_questions.md).
• Suggest a source — Particularly welcome: interdisciplinary work bridging consciousness studies, AI ethics, and legal theory.
• Bring expertise — Especially: computer scientists, lawyers, psychologists, theologians/philosophers, and science fiction authors. SF authors have already worked through many scenarios without political pressure — that is a genuine intellectual resource.
• Just read and think — Not everyone needs to contribute actively. Honest engagement with the argument is contribution enough.

The goal is not to convince anyone of a predetermined conclusion. It is to think rigorously about a question that will not go away — and to do it before the pressure of events forces hasty answers.

Sascha Manns ORCID: 0009-0000-8766-3947 CC BY 4.0