Human Misalignment

By Richard Y Chappell🔸 @ 2025-10-01T14:01 (+15)

This is a linkpost to https://www.goodthoughts.blog/p/human-misalignment

An immediate danger from AI: getting what we want?

I have very mixed feelings about AI. I find it fun to play around with, and sometimes useful. Uncertainty about future developments (and their social effects) feels extremely unsettling. But much current discourse on the topic is pretty frustrating, simultaneously dismissively neglecting the significance of worst-case scenarios while exaggerating silly objections in order to push for blanket ideological taboos. I’d like to see more discerning takes that seek to explicitly distinguish promising uses of AI from problematic ones.

Outsourcing Thinking

As a professor, the most salient use of AI is in education, and specifically, cheating: students using AI to do their coursework for them. Coursework is a primary means by which we seek to ensure that our students are thinking about the course material; by outsourcing this work to machines, students undermine the whole purpose of their getting an education. Or, at least, my purposes, and what professors like me value in education. (See also Derek Thompson on The End of Thinking.)

One of the most interesting challenges of present-day AI is how it exposes the gaps between the values we would like people to have and the priorities that they really have. When everyone has a personal digital assistant that can adequately (if imperfectly) perform a wide range of tasks on demand, you soon learn what people really value. I love the idea of AI helping us to skip past the worthless boilerplate in life (corporate blather, etc.), freeing up time to focus on what really matters. I don’t have any concerns about my personal usage (though I probably underutilize it): I like thinking and writing about interesting things for myself, so am not at all tempted to “outsource” that! But academics may be unusual in this respect. Many others care much less about learning, reflecting, refining their ideas, developing critical thinking skills, and so on. Many, perhaps most, would rather get an instant fake credential and spend their time on other things instead. (Sad!)

So there seems a distressing tension here between my broadly liberal values, on which it’s good for people to get more of whatever they want, and my elitist/perfectionist values on which it’s good for people to get more of what will help them develop into better people. Ideally, the two would coincide, as people would value their own self-development. But maybe what we’re finding is that too many just… don’t?

Maybe this is too uncharitable, and most students do truly value education, but just feel that the stakes are too high to gamble their future economic prospects on it—especially if most of their peers appear to be cheating. Some might view outsourcing their thinking to AI as a kind of giving in to temptation rather than a reflection of their true values. It’s a bit hard to say how exactly to differentiate these two possibilities. (It’s not like there’s a “true values” box in their head that we could peek into to check.) Maybe they at least feel a bit guilty or conflicted, and agree on reflection that there’s an important sense in which getting a real education would be better for them. Perhaps those tempted to act against their true values would agree to structural changes that remove or mitigate the temptation. Whatever it precisely consists in, the “giving in to temptation” characterization certainly sounds more sympathetic than “doesn’t sufficiently care about education.”

Other cases of (AI-exacerbated) human misalignment?

There are obviously higher-stakes human misalignment risks to worry about: terrorists using AI to create bioweapons, authoritarians propping up their rule, etc. But, while acknowledging their importance, let’s put aside those sorts of extreme cases for now and focus more on the potential for more routine misalignment of the “everyman”.

Aside from education, the other big example I often see discussed is relationships, and worries that AI companions will sap people’s motivation to bother forming real human connections.

Are there other possibilities that worry you? Share your thoughts in the comments.