EA Survey 2019 Series: Engagement Levels

By Neil_Dullaghan🔹 @ 2020-06-23T04:54 (+42)

Summary

Introduction

In recent years there has been significant interest in the levels of engagement of EAs and in particular, the most engaged members of the community. The focus and resources devoted to those considered "contributors" or "core"(1, 2) are likely very different to those considered part of the EA "network". CEA has noted that the level of involvement with EA of typical applicants to EA Globals has risen so much that the typical person who didn’t get accepted to attend has changed from someone who was barely involved with EA to someone who is probably pretty knowledgeable about EA and has been involved for at least a few years.

It is important to state at the outset that being involved in the community by posting on the Forum, attending lots of events, or working at EA related organizations is not necessarily the same as dedication to EA or having the most impact.

Here we examine how engaged in different activities are the 2,513 EAs who responded to the 2019 EA Survey. Naturally, what measure we should use to determine the most engaged EAs in the survey is controversial. Any proxy we select will also necessarily be imperfect as there will almost certainly be exceptions to the rule. In this post we examine potential proxies, which may each capture distinct modes of EA involvement; group membership, activities, and self-reported engagement.

Measures of engagement

Group Membership

We find reasonably large numbers of EAs are members of at least one group (the EA Facebook page, Local EA groups, the EA Forum, Giving What We Can, LessWrong), but much smaller numbers are involved across multiple groups. Just as in the EA Survey data from 2018, we see a high number of EAs reported being members of a local EA group. This was 907 individuals (43% of EAs in our sample who answered a group membership question),[1] the second largest grouping behind the EA Facebook group. In the 2019 Local Group Organizer’s survey, 2,144 people were reported to be regular attendees of local EA group events, and 1,513 members were reported to be highly engaged in EA.

72% of respondents were a member of at least one of these groups, and almost 30% were a member of three or more groups.

Most respondents who were members of the EA Forum were also members of the EA Facebook group (81%) while below half (48%) of EA Facebook members were EA Forum members. Below we can see that 119 respondents were members of the EA Forum but not the EA Facebook page,532 were members of the EA Facebook page but not the EA Forum, 494 were members of both the EA Forum and EA Facebook page, 1,386 were not members of either.

Just over 65% of Forum members are members of local groups. Among local group members, 44% are also members of the EA Forum.

EA Facebook and EA local groups were the two groups with the largest membership within our sample and while there was substantial overlap, many EAs who were members of EA Facebook were not members of a local group and vice versa.

Unlike in last year’s EA Survey data, where our sample contained similar numbers of EA Forum and LessWrong members but low rates of overlap between the two groups, 60% of LessWrong members were also members of the EA Forum, though only about 38% of EA Forum members were also members of LessWrong. This makes some sense since our aim was to primarily sample EAs, not LessWrong members. The majority of respondents (1,736, 69%) were members of neither the EA Forum nor LessWrong.

Activities

We asked respondents to select which activities they have engaged in from a pre-set select-all-that-apply list. There appears to be a clear narrowing of modes of engagement, with many EAs having read an EA book, made an EA donation or career change, but far fewer posting on the EA Forum, receiving 80,000 Hours career coaching, or working at an EA organization.

81% (1,623) reported that they had made a donation influenced by EA principles. 90% of these offered information about their donations. The second most popular activity was reading an EA book. There were a set of activities that a smaller number of survey respondents report having done. 283 (13%) previously worked at an EA organization and 207 (10%) currently work at an organization, with 15% (296) of EAs in the survey currently work and/or previously worked at an EA organization. 12% (241) had received personal career coaching from 80,000 Hours, and 13% (258) had posted on the EA Forum. These latter four narrower activities do not have a high degree of overlap, with only 2.5% (61) of EAs having work(ed) at an EA organization, posted on the EA Forum, and done 80,000 Hours career coaching.

Self-reported Engagement Level

This year the survey asked respondents to rank their level of engagement in the EA community on a scale from no engagement to high engagement, with certain prompts given to provide proxies for each level:

This scale is an imperfect proxy for engagement in EA and may not capture dimensions such as commitment or dedication to EA. The items listed as examples in each category also likely created some bias. For example, people who live near a location where in-person EA Globals happen more often (London and San Francisco) are much more likely to be able to identify as considerable or high engagement than someone equally as engaged but geographically further away or unable to attend for other reasons. Indeed, in our post on the geographic distribution of EAs, we noted that EAs living outside of the USA and Europe reported the largest shares of non engaged or only mildly engaged EAs, while EAs living in the UK had the largest share reporting to be highly engaged (31%) and USA EAs made up a plurality (38%) of highly engaged EAs.

Our analysis excludes respondents who answered “no” to either of our two EA screener questions[2] asking about whether they consider themselves an EA so it is perhaps not surprising that very few respondents in our analysis make up the “No engagement” category. There were only ~24 respondents who answered no to an EA screener question and selected no engagement. The bulk (~105) chose moderately engaged in EA so this would not drastically change the overall picture below.

Differences in levels of engagement across groups

How do these various measures of activity differ across demographic and other groupings of EAs? The data suggests male EAs in the survey are more likely to report making an EA donation, posting on the EA forum, and being members of the EA Forum and LessWrong than female EAs. Those identifying as white are more likely to report donating also. Younger EAs and those who have been engaged in the movement at least 3 years and still remain active today are likely to be more engaged than older and newer EAs. For the sake of simplicity and clearer illustration we present here only the most interesting and clear results. For those interested, more details and tables are available here.

Time in EA

Those who have been involved in EA 8 or more years appear especially more likely to be highly engaged today than newer EAs. The majority of EAs who joined 3 or more years ago and are still active in the community report being considerably or highly engaged now. To the extent that this data can shed light on how long it takes for people to become highly engaged in EA, it suggests after 3 years most EAs that remain in the movement consider themselves very engaged. We can also see that there is a trend towards higher engagement that peaks at around 5 years in. There are ~15% of EAs who report having first heard of EA 9 or more years ago but are now only mildly or not engaged in EA.

If we categorised those who selected 1-3 or 4-5 on the engagement scale as “low” or “high” in engagement respectively, as in our analysis in this post , we can see that high engagement EAs make up smaller shares of newer cohorts than older cohorts. It may be that many of these low engagement EAs in the more recent cohorts will become more highly engaged after 3 or so years. This may also not be so surprising if one assumes that EAs who joined 5 or more years ago and did not become highly engaged simply dropped out of the movement and so do not appear in the survey. For more research on attrition rates in EA see our earlier posts here and here.

Gender

How diverse are the most engaged EAs? Since the EA Survey population is majority male it is not surprising that the majority of most engagement categories are male. However, there does not appear to be a marked difference in the rates of self-reported engagement levels between genders.[3] For example, similar shares of male and female EAs are highly engaged.

There appear to be significant[4] associations between gender and membership of the EA Forum and LessWrong. Men appear more likely to report being members of the EA Forum and LessWrong than women. It is somewhat notable that there does not appear to be a significant difference in membership of local EA groups between men and women.

We also see that men are more likely to report commenting on the EA Forum or making a donation than women.[5] As we noted in our careers and skills post, there does not appear to be a difference between the shares of men and women working at an EA organization.

Age

There is a statistically significant difference in the underlying distributions of ages between the different self-reported engagement levels.[6] More engaged EAs appear to be younger on average than less engaged EAs. One speculative hypothesis to explain this is that younger people tend to have more free time for whatever form of engagement they pursue. Students do appear in higher proportions among the higher engagement levels than the lower levels (29%-36% in levels 3 to 5 and 16-19% in levels 1 to 2).[7]

There are significant differences in the average ages of group members and nonmembers,[8] with most members being ~2 years younger than nonmembers and 6-7 years younger than EAs who report being a member of none of these groups.

Race

EAs identifying as white in the survey were more likely to report donating according to EA principles but there were no significant differences in self-reported levels of engagement or group membership between EAs who identified as white and those who did not.

Income

There does seem to be somewhat of a negative association between income and higher self-reported engagement levels.[9] Limiting the analysis to compare only full-time employed nonstudents there is a significant difference in the underlying distributions of income between the engagement levels, however it does not appear to be a simple linear association.[10]

Non members of groups appear to earn more on average. Forum, Local group, and Facebook members appear to earn a significant amount less than nonmembers of these groups respectively.[11] One might assume Giving What We Can members are more likely to be following an earning-to-give path in a high-paying career than nonmembers. There is no difference in the share of GWWC members and nonmembers who reported following an earning-to-give path career path (38% respectively) and no statistically significant difference in the average income of GWWC members and nonmembers.

Relationships between measures of engagement

Activities and engagement

Many of the activities listed above had corresponding items in the descriptions of the engagement levels. An examination of the activities that correspond to selections on the EA engagement scale illustrates that those who identified with the highest engagement category are likely to have engaged in virtually all activities, while those in the lower categories are likely to only have made a donation, read an EA related book, and thought about changing their career path. The engagement selections do appear to correspond well to the wording of the survey. For example, individuals in the high engagement category were the most likely to have applied for an EA organization position, to be employed by an EA organization or to be a leader of a local EA group and the mild engagement category maps well onto reading a book. Interestingly, although it was not included in the prompts, individuals with a considerable or high engagement ranking were more likely to have posted or commented on EA Forum. This is in line with our expectations from last year where we selected EA Forum membership as the best available proxy for engagement. In the bar chart below the number of respondents answering yes for each engagement activity are arranged by engagement level and stacked on top of each other so the totals in each bar exceed the true number of respondents (1,838 respondents). This figure serves to illustrate that more people are engaging in more activities overall for each subsequent self-reported engagement level.

Some specific activities appear to be better predictors of engagement ranking than others. For example, a high percentage of respondents across all categories report making donations based on EA, so this is a relatively uninformative activity. However, the percentage of those in the high engagement category that report having attended an EA Global is relatively large (74%),[12] but close to zero for the mild and moderate categories, so this activity tends to divide the engagement scale into two groups. This should not be so surprising since attending an EA Global was specifically mentioned in the description of the second highest level of engagement so it does not tell us whether attending an EA Global makes someone more engaged or whether more engaged EAs are more likely to attend EA Globals. Other activities, such as applying for an EA organization job rise steadily with an increase in self-ranked engagement.

Group membership and engagement

Group membership was less tied to engagement ranking than activity. Individuals in the no engagement to moderate engagement categories were more likely to select the “none of the above” category when queried about membership. Individuals in the high engagement category were most likely to be members of EA Forum, the EA Facebook group and a local group. Note again that in the bar chart below the bars show the number of respondents in each group stacked on top of each other so the totals in each bar exceed the true number of respondents in each engagement level.

Group membership and activities

How do EA Forum, EA Facebook, local group, GWWC, LessWrong members score across activities? Unsurprisingly, EA Forum members were very likely to have commented on the EA Forum (close to 60%). The share of EA Forum members undertaking an activity was larger than the shares of other groups for almost every activity in fact. Of course, we should remember that group membership is not exclusive and as we noted above many groups have overlapping membership. GWWC members were very likely to have made an EA donation (more than 80%). EA Forum members and LessWrong members were as likely as each other to write a post about EA somewhere other than the EA Forum (recalling that 60% of LessWrong members were Forum members and 38% of Forum members were LessWrong members).

Given these associations can we simplify all the measures into 1 or more dimensions? Are there distinct modes of being engaged or are all the forms of engagement pretty much just equally associated with each other and can be condensed into simply ‘more engagement’ in general? We looked for patterns and associations through a range of methods: multiple correspondence analysis, (ordered) logistic regression and cluster analysis. Our multiple correspondence analysis divided our sample into two clusters and suggested 14% of the variance (a typical amount for this type of multivariate analysis) in these engagement scale groupings could be explained by two dimensions that appear to map onto high engagement in numerous activities for high engagement EAs and nonmembership of EA groups for low engagement EAs. The other methods also suggested that the self-reported engagement scale seems to be well correlated with activity question responses and lower levels of engagement were related to nonmembership of groups. The most important predictor of a high level of engagement was membership in the EA forum and increasing time in the EA community had a strong impact on increasing engagement. For those interested, more details on the models are available here.

Conclusion

For those interested in focusing on the highly engaged EAs in the community it seems useful to have a measure of this engagement. We can see that while most EAs engage by donating, reading, and making career decisions with EA principles in mind there appears to be a demarcation between those in the top two self-reported levels of engagement and the rest. Those who consider themselves as being highly engaged in EA report taking on positions in local EA groups and EA organizations and engaging in the EA Forum and EA events at higher rates than their peers in the movement who consider themselves less involved. EA Forum membership appears to be the best single concrete metric we have associated with engagement (as measured in this report according to activities and self-reported level of engagement). On the whole, there is little evidence of demographic differences in engagement that we looked at.

Credits

The annual EA Survey is a project of Rethink Charity with analysis and commentary from researchers at Rethink Priorities.

This essay was written by Neil Dullaghan, Kim Cuddington, and David Moss. Thanks to Jason Schukraft and Peter Hurford for comments.

If you like our work, please consider subscribing to our newsletter. You can see all our work to date here.

Other articles in the EA Survey 2019 Series can be found here.

  1. GWWC membership was asked as a separate question, so the total number of respondents is the combination of those who answered the group membership question and those who answered the GWWC pledge question. ↩︎

  2. We included only those who selected “Yes” to both EA screener questions, “Do you broadly subscribe to the basic ideas behind effective altruism?”, “Could you, however loosely, be described as an "Effective Altruist"?” and “Yes” to the honesty check question, “Are you giving genuine, sincere answers?”. There were approximately 360 respondents who answered “No” to at least one of the EA screener questions, but 40-60% of these did not complete the survey. For ease of comparison with previous surveys we continue to exclude these respondents from analyses. ↩︎

  3. Kruskall-Wallis: chi-squared = 2.327 with 1 d.f. probability = 0.1271 chi-squared with ties = 2.509 with 1 d.f. probability = 0.1132 ↩︎

  4. chi-squared p<0.05 using bonferroni corrected p-values ↩︎

  5. chi-squared p<0.05 using bonferroni corrected p-values ↩︎

  6. Median test results Pearson chi2(4) = 53.9919 Pr = 0.000 ↩︎

  7. Omnibus significance testing suggested a statistically significant association between student status and engagement level, but this test doesn’t point to what is driving the association. ↩︎

  8. All welch t-tests p<0.0001 ↩︎

  9. Median exact test p<0.0001 ↩︎

  10. Median exact test p<0.0001 ↩︎

  11. Welch t-tests on the log of income p<0.05 ↩︎

  12. 57% of those who reported having attended an EAG self-reported being highly engaged in EA. ↩︎

Nathan Young @ 2021-03-19T14:19 (+1)

Those who consider themselves as being highly engaged in EA report taking on positions in local EA groups and EA organizations and engaging in the EA Forum and EA events at higher rates than their peers in the movement who consider themselves less involved.

To me this looks like an artefact of the original categories.

High engagement: I am heavily involved in the effective altruism community, perhaps helping to lead an EA group or working at an EA-aligned organization. I make heavy use of the principles of effective altruism when I make decisions about my career or charitable donations.

It looks to me that to self define as a high engagement EA you are being asked if you are lead a group or or work at an EA org. It doesn't seem surprising that the two correlate. 

I think I recall thinking this while I took the survey, tbh. I feel engaged, but I'm not sure I fit into that category. 

The idea that people who give significant amounts of money by EA lights*  aren't really  engaged because they don't work in EA orgs or lead groups seems unhelpful. This seems to be the case since moderate-engagement EAs both earn more and a higher percentage say they give. I say this not because it is the most important factor but because it's the one I can see in the data.

 

I might suggest that in future either this is rephrased entirely to be about "feeling very much a part of the movement" or that the first  sentence reads as follows
 

"I am heavily involved in the effective altruism community. This might mean many things,  perhaps helping to lead an EA group, attending a number of groups, giving  an amount of money  I consider significant by EA principles or working in an EA-aligned career."

Thanks again for your work.

Z @ 2020-07-15T07:04 (+1)

Thanks for doing the survey and analyzing the data.

What does GWWC membership mean here - anything above 10% or 1% pledge?

Neil_Dullaghan @ 2020-07-16T05:50 (+1)

GWWC membership here refers to anyone who provided a year in response to the question in the EA Survey "If you have taken the Giving What We Can pledge, please note the year:"