Open Phil EA/LT Survey 2020: EA Groups

By Eli Rose @ 2021-09-01T01:01 (+35)

We'll be posting results from this survey to the frontpage one at a time, since a few people asked us not to share all of them at once and risk flooding the Forum. You can see the full set of results at our sequence.

This post is part of a series on Open Phil’s 2020 survey of a subset of people doing (or interested in) longtermist priority work. See the first post for an introduction and a discussion of the high-level takeaways from this survey. Previous posts discussed the survey’s methodology, background statistics about our respondent pool, and our findings on what helped and hindered our respondents re: their positive impact on the world. This post discusses our results about the impact from EA groups.

You can read this whole sequence as a single Google Doc here, if you prefer having everything in one place. Feel free to comment either on the doc or on the Forum. Comments on the Forum are likely better for substantive discussion, while comments on the doc may be better for quick, localized questions.

We found evidence of a lot of impact from EA groups. One question asked our respondents to list the top few things that impacted them (unanchored), and on this question, our respondents listed EA groups more frequently, by far, than anything else on the list of 34 EA/EA-adjacent things we specifically looked for in responses. On other metrics which aggregated large and small impacts, (“impact points” and “mentions”) they also performed well. How is this impact distributed across different groups and types of groups?

Summary of Key Results

Results

66 distinct groups were mentioned by respondents. These encompassed both university-based groups, like Yale EA, and geographically-based groups, like EA London. A small number of topic-specific groups (e.g. an AI Safety reading group) were also included. Respondents didn’t always tell us explicitly which group they had attended, but when they didn’t we were generally able to figure it out from reading free-text responses[1].

We also calculated impact points for individual EA groups[2]. Note that because of differences in how they are calculated, the sum of impact points for individual EA groups is slightly different from the impact points total assigned to the thing called “Local EA groups, including student groups” in our Influences section.

Note also that a fair number of our respondents were actually founders/organizers of the groups they said had an impact on them, and some mentioned that this was important to how the group had impacted them (e.g. they got more into EA ideas because they were in this role).

This analysis doesn’t include rationality/LessWrong/Slate Star Codex meetup groups, though they were one of the 34 influences we asked about by name.

The below charts show the breakdown of unweighted and weighted impact points for groups with the top ten largest totals[3], lumping all other groups under “Other.” I’ve redacted the names of some groups in order to protect respondent anonymity.

Groups by unweighted impact points

Groups by weighted impact points

The data for top factor listings by individual group was very noisy because of the small number of data points for each group, so I don’t include it here.

Here is how weighted and unweighted impact points break down across various subsets of the groups. One subset is “top-15 universities”; see our definition of that term here[4].
Share of unweighted impact points from groups Share of weighted impact points from groups
top 10 groups of any kind 63% 73%
university groups 64% 76%
groups at a top-15 university 56% 61%[5]
Just Oxford, Cambridge, Stanford, and Harvard groups[6] 44% 53%

The top 10 out of these 66 groups by impact points are responsible for most of the points, and their share goes up when you consider respondent weight. The same is true when looking at university groups. Most of the impact from university groups is coming from top-15 university groups.

Impact also looks very unevenly distributed even among groups at top-15 universities. Between 40-50% of all impact from groups (by these metrics), and the vast majority of all impact (by these metrics) from groups at top-15 universities, looks to be coming from just the groups at Oxford, Cambridge, Stanford, and Harvard. (This is counting all of Harvard’s groups, not just the undergrad one.)

Note that, to the extent we want to use them to forecast the future, comparisons between groups’ historical impacts here are confounded by their different ages; some groups at top universities didn’t get started until 2016 or later, while Stanford EA, EA Oxford, EA Cambridge, and Harvard EA have been around for a much longer time. Also keep in mind that these results are based on only our respondents, who are likely a small fraction of the total group members for each group.

  1. 155 of our respondents had had some interaction with a local group. In 10 of the cases, there wasn’t enough information in the free-text answers for us to confidently discern which group it was. ↩︎

  2. Technically, this is different from ‘impact points’ as used above because it is more holistic: we looked at more than just a respondent’s multiple-choice selection in the influences section to determine whether they had interacted with a group. We also applied some of our judgment to what the ‘helpfulness’ rating for a group should be — in 8/155 cases, we modified the respondent’s answer to the multiple-choice selection on how much the group helped them, adjusting it up or down depending on other information in the survey (e.g. in one case we changed a report of “helped” to “slightly helped” because the group didn’t seem to be a key part of the story the respondent gave in the factors section.) But the two don’t diverge too much. ↩︎

  3. Harvard has four distinct groups that were mentioned in our survey, with distinct group leaders, events, etc. The one called “Harvard EA” here is the Harvard College group, for undergrads. ↩︎

  4. Our definition of top-15 university does not include UC Berkeley, and including it would meaningfully change the weighted numbers, in the ways I’ve footnoted below. As far as I’m aware, this is the only debatable inclusion in the top-universities list which would meaningfully change the results. ↩︎

  5. Including UC Berkeley would bring this number up to 65%. ↩︎

  6. Counts all of Harvard’s groups, not just the undergrad-focused one. ↩︎