Deep Democracy as a promising target for positive AGI futures

Upvoted; I have some some concerns about this proposal but I do think "Deep Democracy"-aligned AGI would be significantly better than the default outcome (assuming alignment is solved, which is a big assumption) that AGI is aligned to whoever happens to be in control of it. And I think this is an important discussion to have.

And "make AGI democratic" seems much more tractable than "convince everyone that utilitarianism is true, and then make AGI utilitarian".

Suitably deep kinds of democracy avoid the tyranny of the majority, where if 51% of people say they want something, it happens. Instead decisions are sensitive to everyone's values. This means that if you personally value something really weird, that doesn't get stamped out by majority values, it still gets a place in the future.

How does this work mechanically? Say 1% of people care about wild animal suffering, 49% care about spreading nature, and 50% don't care about either. How do you satisfy both the 1% and the 49%? How do the 1%—who have the actually correct values—not get trampled?

You object to aligning AGI(s) to your own values for principled reasons. It would be highly uncooperative, undemocratic, coercive, and basically cartoon supervillain evil.

I value cooperation and not-being-evil, though! If I align AGI to my own values, then the AGI will be nice to everyone—probably nicer than if it's aligned to some non-extrapolated aggregate of the values of all currently-living humans. The Michael-aligned AGI will surely be nicer to animals and digital minds than the democratically-aligned AGI would be.

(This is hypothetical of course; there will never be a situation where AGI is aligned specifically to my values.)

It also seems a bit circular because if you want to build a Deep Democracy AGI, then that means you value Deep Democracy, so you're still aligning AGI to your values, it's just that you value including everyone else's values. Why is that any better than (say) building a Utilitarian AGI, which incorporates everyone's preferences?

P.S. I appreciated the length of this article, when I saw the title I thought it was going to be a 10,000 word slog but it ended up being quite easy to read. (Observing this preference in myself makes me think I need to try harder to make my own writing shorter.)

Could you say a bit more about the power law point?

A related thing I've been thinking about is that some kinds of deep democracy and some kinds of better futures-style reasoning (for sufficiently risk-neutral, utilitarian, confident in their moral views, etc etc etc kinds of agents, assume all the necessary qualifiers here) will end up being in tension — after all, why compromise between lots of moral views when this means you miss out on a bunch of feasible moral value? (More precisely, why choose the compromise it's-just-ok future when you could optimise really hard according to the moral view you favour and have some small chance of getting almost all feasible value?)

I think that some versions of the power law point might make moral compromise look more appealing, which is why I'm interested. (I'm personally on team compromise!)

Thanks for the post! Sure seems like an important topic. Unnecessarily grumpy thoughts to follow —

First, I feel unsure what you're imagining this ‘deep democracy’ thing is an answer to. You write, “If you want the long-term future to go well by the lights of a certain value function[…]” Who is “you” here?

An individual? If so, it seems clear to me that the best way to bring about things I care about, holding fixed others' strategies, is indeed to get my AI(s) to pursue things I care about, and clearly not best or desirable from any almost individual's point of view (certainly not an equilibrium) to get their AI(s) to pursue some kind of democratic conglomeration of everybody's interests.

An AI company? If so, I imagine (at least by default) that the companies are just trying to serve demand with more and better models, complying with regulation, and maybe proactively adding in side constraints and various revenue-sacrificing behaviours for prosocial or reputational reasons. It's not obvious to me that there comes a point where the executives sit down to discuss: "what set of values should we align AI to?", where the candidate answers are various (pseudo-)moral views.

In the case where we're considering a company considering the target of alignment, I wonder if my confusion comes from starting from different assumptions about how 'unitary' their product is. One outcome, which I currently view as a default, is just an extrapolation of the status quo today: companies train base models, then those base models are post-trained and broken up into various specialised variants, which can in turn be fine-tuned for specific use cases, and all eventually served for a million different uses. On this picture, it's not clear how much influence an AI company can have on the moral vision of the models they ultimately serve. The main reason is just that the vast majority of what these AIs are doing, in my mind, are just helpful or economically useful tasks based on (direct or delegated) specific human instructions, not following a grand impartial moral plan. And if the models are too eager to break ranks and pursue an abstract moral vision, people won't use them. This is what runs through my head when people talk about "the AGI" — what is that?

Of course, there are some reasons for thinking the AI landscape will be more unitary, and this picture could be wrong. Maybe a corporate monopoly, maybe a centralised (state-led) project, maybe a coup. Let's consider the extreme case where "you" are a lab exec, you hold total power over the world through some single alignable AI system, and you face the decision of what to tell it to do. Here I'd zoom in on the part where you say it would be "uncooperative, undemocratic, coercive" to implement your values. One rejoinder here is to make the point that, AI aside, you should (i) be morally uncertain, (ii) interested in figuring out what's good through deliberation, and (iii) care about other people. So if the hegemon-leader had a reasonable moral view, directly implementing it through an AI hegemon doesn't strike me as obviously worse in expectation than ceding influence. If the hegemon-leader has a moral view which is obviously bad, then I don't think it's very interesting that a democratic thing seems better.

In any case, I agree that the main reasons against the hegemon-leader directly implementing their vision of what's good. But (at a gut level) this is, as you say, because it would be illegitimate, uncooperative, etc., not to mention practically likely to fail (we haven't tested global totalitarianism, but most new political experiments fail). And I think democractic-but-hegemonic AI probably fares pretty badly by those lights, too, compared to actually just ceding power or not becoming a hegemon in the first place?

I do feel like I'm being unfair or missing something here. Maybe another reading is: look, anarchy is bad, especially when everyone has an army of AIs to carry out their bidding, and everyone is rich enough to start caring about scary, scope-sensitive, ideologically-motivated outcomes. The result is a bunch of winner-take-all conflict, and general destructive competition. So we need some governance system (national and/or global?) which curbs this destruction, but also does the best job possible aggregating people's values. And this is what "deep democracy" should be doing.

The part of this I agree with is that, as far as we have voting systems, they could be much improved post-AGI, in a million ways. Thumbs up to people imagining what those tools and approaches could look like, to make democratic political procedures more flexible, effective, rich in information, great at finding win-win compromises, and so on.

But there's a part that feels underspecified, and a part that I'm more sceptical of. The part that feels underspecified is what "deep democracy" actually is. The way you've phrased it, and I'm being a bit unfair here, is close to being good by definition ("deeply capturing and being responsive to every single person's values" — I mean, sure!) I expect this is one of those cases where, once forced to actually specify the system, you make salient the fact that any particular system has to make tradeoffs (cf Arrow, though that's a bit overblown).

The part I'm more sceptical of is that the anarchic alternative is chaotic and destructive, and the best way to aggregate preferencesis via setting up some centralised monopoly on force, and figuring out what centralised process it follows. Consider the international ~anarchy. War is a really unattractive option, even for neighboring expansionary states, so in theory (and often in practice) compromise is virtually always preferred to war. And that's the hard (fully anarchic) case — smaller-scale conflict is avoided because of criminal and civil laws which make it very not worth it.

Finally, I'd suggest that, in a sense, we have a way to allocate resources and efforts towards what people want in a granular, deep, preference-aggregating way: trade. My sense is to think about this as (even today) the main means by which society is arranged to make everyone better-off; and then consider cases where centralised processes (like voting) are necessary or valuable. One example, which seems potentially very important, is if something like "moral worth" becomes even more divorced from wealth. Of course in designing a democratic process you don't have perfectly neutral ground to stand on; you have to make a call on who gets to partake. But you can give more of a voice to people who are otherwise practically totally disenfranchised because they lack resources; while the ultra-wealthy otherwise would dominate outcomes. That's already an issue and could become a bigger issue, but does suggest an alternative answer to the question you're asking, which is (potentially a lot of) wealth redistribution.

If value is a power law or similarly distributed, then you have a high chance of at least capturing some of the stuff that is astronomically more valuable than everything else, rather than losing out on this stuff entirely.

Of course there are a bunch of methods in social choice which do this, like quadratic voting; though it's notable that most electoral democracies are not good examples, and anything like "go with the majority while protecting the rights of the minority" seems apt to highly underrate cases where some voters think that a particular issue is astronimically higher in stakes than others think. But this is also a case where I don't think there's an especially neutral, non-theory-laden approach for how to recognise how and when to give people's views more weight because (intuitively) they think some issue is higher stakes. Then again, I think this is a general problem, not a specific issue with designing democratic methods.

Ok sorry for the chaotically written thoughts, I think they probably understate how much I'm a fan of this line of thinking. And your comments in reply to this were clarifying and made me realise I was in fact a bit confused on a couple points. Thanks again for writing.

I generally agree with this post that it is a good target. This is effectively equivalent to Coherent Extrapolated Volition^[1], which has been proposed as a target for alignment. The difference is whether you consider extrapolated values by default or not.

1. ^{^}

   https://www.lesswrong.com/w/coherent-extrapolated-volition-alignment-target

Nice post. I've been vaguely thinking in this direction so it's helpful to have a name for this idea and some early food for thought.

this was my premise much more so with election by jury.

https://www.electionbyjury.org/manifesto

full disclosure: i co-founded the center for election science which advocates approval voting.

quadratic voting has been pretty deeply debunked.