Rationality, EA and being a movement

I work for CEA, but these views are my own -- though they are, naturally, informed by my work experience.

----

First, and most important: Thank you for taking the time to write this up. It's not easy to summarize conversations like this, especially when they touch on controversial topics, but it's good to have this kind of thing out in public (even anonymized).

----

I found the concrete point about Open Phil research hires to be interesting, though the claimed numbers for CFAR seem higher than I'd expect, and I strongly expect that some of the most recent research hires came to Open Phil through the EA movement:

• Open Phil recruited for these roles by directly contacting many people (I'd estimate well over a hundred, perhaps 300-400) using a variety of EA networks. For example, I received an email with the following statement: "I don't know you personally, but from your technical experience and your experience as an EA student group founder and leader, I wonder if you might be a fit for an RA position at Open Philanthropy."
• Luke Muehlhauser’s writeup of the hiring round noted that there were a lot of very strong applicants, including multiple candidates who weren’t hired but might excel in a research role in the future. I can’t guarantee that many of the strong applicants applied because of their EA involvement, but it seems likely.
• While I wasn't hired as an RA, I was a finalist for the role. Bastian Stern, one of the new researchers mentioned in this post, founded a chapter of Giving What We Can in college, and new researcher Jacob Trefethen was also a member of that chapter. If there hadn't been an EA movement for them to join, would they have heard about the role? Several other Open Phil researchers (whose work includes the long-term future) also have backgrounds in EA community-building.

I'll be curious to see whether, if Open Phil makes another grant to CFAR, they will note CFAR's usefulness as a recruiting pipeline (they didn't in January 2018, but this was before their major 2018 hiring round happened).

Also, regarding claims about 80,000 Hours specifically:

• Getting good ops hires is still very important, and I don’t think it makes sense to downplay that.
• Even assuming that none of the research hires were coached by 80K (I assume it’s true, but I don’t have independent knowledge of that):
• We don’t know how many of the very close candidates came through 80,000 Hours…
• ...or how many actual hires were helped by 80K’s other resources…
• ...or how many researchers at other organizations received career coaching.
• Open Phil’s enormous follow-on grant to 80K in early 2019 seems to indicate their continued belief that 80K’s work is valuable in at least some of the ways Open Phil cares about.

----

As for the statements about "compromising on a commitment to truth"... there aren't enough examples or detailed arguments to say much.

I've attended a CFAR workshop, a mini-workshop, and a reunion, and I've also run operations for two separate CFAR workshops (over a span of four years, alongside people from multiple "eras" of CFAR/rationality). I've also spent nearly a year working at CEA, before which I founded two EA groups and worked loosely with various direct and meta organizations in the movement.

Some beliefs I've come to have, as a result of this experience (corresponding to each point):

1. "Protecting reputation" and "gaining social status" are not limited to EA or rationality. Both movements care about this to varying degrees -- sometimes too much (in my view), and sometimes not enough. Sometimes, it is good to have a good reputation and high status, because these things both make your work easier and signify actual virtues of your movement/organization.

2. I've met some of the most rigorous thinkers I've ever known in the rationality movement -- and in the EA movement, including EA-aligned people who aren't involved with the rationality side very much or at all. On the other hand, I've seen bad arguments and intellectual confusion pop up in both movements from time to time (usually quashed after a while). On the whole, I've been impressed by the rigor of the people who run various major EA orgs, and I don't think that the less-rigorous people who speak at conferences have much of an influence over what the major orgs do. (I'd be really interested to hear counterarguments to this, of course!)

3. There are certainly people from whom various EA orgs have wanted to dissociate (sometimes successfully, sometimes not). My impression is that high-profile dissociation generally happens for good reasons (the highest-profile case I can think of is Gleb Tsipursky, who had some interesting ideas but on the whole exemplified what the rationalists quoted in your post were afraid of -- and was publicly criticized in exacting detail).

I'd love to hear specific examples of "low-status" people whose ideas have been ignored to the detriment of EA, but no one comes to mind; Forum posts attacking mainstream EA orgs are some of the most popular on the entire site, and typically produce lots of discussion/heat (though perhaps less light).

I've heard from many people who are reluctant to voice their views in public around EA topics -- but as often as not, these are high-profile members of the community, or at least people whose ideas aren't very controversial.

They aren't reluctant to speak because they don’t have status — it’s often the opposite, because having status gives you something to lose, and being popular and widely-read often means getting more criticism over even minor points than an unknown person would. I’ve heard similar complaints about LessWrong from both well-known and “unknown” writers; many responses in EA/rationalist spaces take a lot of time to address and aren’t especially helpful. (This isn’t unique to us, of course — it’s a symptom of the internet — but it’s not something that necessarily indicates the suppression of unpopular ideas.)

That said, I am an employee of CEA, so people with controversial views may not want to speak to me at all -- but I can't comment on what I haven't heard.

4. Again, I'd be happy to hear specific cases, but otherwise it's hard to figure out which people are "interested in EA's resources, instead of the mission", or which "truth-finding processes" have been corrupted. I don't agree with every grant EA orgs have ever made, but on the whole, I don't see evidence of systemic epistemic damage.

----

The same difficulties apply to much of the rest of the conversation -- there's not enough content to allow for a thorough counterargument. Part of the difficulty is that the question "who is doing the best AI safety research?" is controversial, not especially objective, and tinged by one's perspective on the best "direction" for safety research (some directions are more associated with the rationality community than others). I can point to people in the EA community whose longtermist work has been impressive to me, but I'm not an AI expert, so my opinion means very little here.

As a final thought: I wonder what the most prominent thinkers/public faces of the rationality movement would think about the claims here? My impression from working in both movements is that there’s a lot of mutual respect between the people most involved in each one, but it’s possible that respect for EA’s leaders wouldn’t extend to respect for its growth strategy/overall epistemics.

Thanks for the summary! I don't know if that came up during your discussion, but I would have found concrete examples useful for judging the arguments.

"ii) This causes EA to prioritise building relationships with high-status people, such as offering them major speaking slots at EA conferences, even when they aren't particularly rigorous thinker"

I'd hope that bad arguments from high-status people will be pointed out and the discussion moves forward (e.g. Steven Pinker strawmanning worries about x-risks).

"iii) It also causes EA to want to dissociate from low-status people who produce ideas worth paying attention to."

For example I find it unlikely that an anonymous writer with good ideas and comments won't be read and discussed on the forum. Maybe it's different on conferences and behind the scences at EA orgs, though?

"iv) By acquiring resources and status EA had drawn the attention of people who were interested in these resources, instead of the mission of EA. These people would damage the epistemic norms by attempting to shift the outcomes of truth-finding processes towards outcomes that would benefit them."

EAs seem to mostly interact with research groups (part of the institution with the best track record in truth-finding) and non-profits. I'm not worried that research groups pose a significant threat to EAs epistemic standards, rather I expect researchers to 1) enrich them and 2) be a good match for altruistic/ethical motivations and being rigorous about this. Examples that come to mind are OpenPhil causing/convincing bioriks researchers to shift their research in the direction of existential threats.

Does someone know of examples or mechanism how non-profits might manipulate or have manipulated discussions? Maybe they find very consequential & self-serving arguments that are very difficult to evaluate? I believe some people think about AI Safety in this way, but my impression is that this issue has enjoyed a lot of scrutiny.