What are the bad EA memes? How could we reframe them?

At first with the question of "bad EA memes" I also wondered if it might include "things lots of EAs believe that make them less effective at doing good"

Summary: It was bad optics for EA to associate itself with memes that misrepresent what the movement is really about. It was mistaken branding efforts in the first few years of EA that has gotten EA stuck with these inaccurate interpretations about the movement.

It's both. Common misconceptions about EA are not only inaccurate presentations of what EA is about. They're the consequence of EA misrepresenting itself. That's why it was bad optics. 

The impression I've gotten from other comments on this post is that people aren't very aware that the these misconceptions about EA were caused by EA branding itself with memes like the one Hauke Hillebrandt references in this comment. 

I don't know if it's because most people have joined the EA movement before it got stuck with these misconceptions. 

Yet I've participated in EA for a decade and I remember for the first few years we associated ourselves with earning to give and overly simplistic utilitarian (pseudo-utilitarian?) approaches. 

I made that mistake a lot. It's hard to overstate how much we, the first 'cohort' of EA, made that mistake. (Linch, I'm aware you've been in EA for a long time too but I don't mean to imply you're part of that first cohort or whatever.) It took only a few years for us to fully recognize we were making these mistakes and attempt to rectify so many misconceptions about EA. Yet a decade later we're still stuck with them.

If you strongly think I'll do this, I will but it will be a bit of a faff.

The question wasn't about misconceptions, as memes that misrepresent what EA is really about, but how EA has spread memes that misrepresent what it's really about. In other words, the question is about what better memes EA should choose to send messages and represent itself better.

EA was never explicitly about mitigating global poverty, earning to give, ignoring change or utilitarianism. Yet for its first few years, EA disproportionately focused its public messaging on those memes. That's caused these misconceptions about EA to persist for years longer and the point is to recognize how the cause of that is the mistake EA itself made.

That has always been a strawman of what earning to give is about. My opinion is that at this point it's better for EA to assert itself in the face of misrepresentations instead of trying to defuse them through conciliatory dialogue that has never worked. 

You know, my take on this is that instead of resisting comparisons to Terminator and The Matrix, they should just be embraced (mostly). "Yeah, like that! We're trying to prevent those things from happening. More or less."

The thing is, when you're talking about something that sounds kind of far out, you can take one of two tactics: you can try to engineer the concept and your language around it so that it sounds more normal/ordinary, or you can just embrace the fact that it is kind of crazy, and use language that makes it clear you understand that perception.

So like, "AI Apocalypse Prevention"?

When I introduce AI risk to someone, I generally start by talking about how we don't actually know what's going on inside of our ml systems, that we're bad at making their goals what we actually want, and we have no way of trusting that the systems actually have the goals we're telling them to optimize for.

Next I say this is a problem because as the state of the art of AI progresses, we're going to be giving more and more power to these systems to make decisions for us, and if they are optimizing for goals different from ours this could have terrible effects.

I then note that we've already seen this happen in YouTube's algorithm a few years ago: they told it to maximize the time spent on the platform, thinking it would just show people videos they liked. But in reality it learned that there were a few videos it could show people which would radicalize them into a political extreme, and by doing this it was far easier to judge which videos would keep them on the platform the longest: those which showed those they agreed with doing good things & being right, and which showed those they disagreed with doing bad things & being wrong. This has since been fixed, but the point is that we thought we were telling it to do one thing, but then it did something we really didn't want it to do. If this system had more power (for instance, running drone swarms or being the CEO-equivalent of a business), it would be far harder to both understand what it was doing wrong, and be able to physically change it's code.

I then say the situation becomes even worse if the AI is smarter than the typical human. There are many people who have malicious goals, and are as smart as the average person, but who are able to stay in positions of power through politically outmaneuvering their rivals. If the AI is better than these people at manipulating humans (which seems very likely, given the thing AIs are known for best nowadays is manipulating humans to do what the company they serve wants), then it is hopeless to attempt to remove them from power.

I think something about properly testing powerful new technologies and making sure they're not used to hurt people sounds pretty intuitive. I think people intuitively get that anything with military applications can cause serious accidents or be misused by bad actors.

Why do they object to it? 

My experience has been that those who don't participate in EA at all have a better reception of "AI risk" in general than near-termists in EA. 

I expect long-termists care as much if not more about what others outside of EA think of AI risk as a concept than near-termists in EA. 

I also recently asked a related question on LessWrong about the distinction between AI risk and AI alignment as concepts.

Avoid catostrophic industrial/research accidents?