How do attendees get value from EA community-building events?

By OllieBase @ 2023-07-05T10:12 (+31)

This post is part of the Community Events Retrospective sequence.

Earlier this year, I surveyed ~500 people who had attended an EAGx event or an event funded by the Community Events Programme. The key sources of value that attendees report getting are: 

• New connections in their network;
• Learning about new career options; and
• Learning about EA cause areas.

This finding is quite rough and we plan on getting more clarity on sources of value from our events in future surveys.

Results

In the survey, I asked attendees about routes to valuable outcomes from our events,^[1] and then coded attendee responses into the following baskets:^[2]

• Connections
• Learning about new career options 
• Learning more about EA cause areas or taking them more seriously
• Increased motivation or positivity for EA broadly
• Increased motivation or positivity for EA-aligned work
• Finding specific sessions interesting or valuable
• More likely to attend other EA-aligned events/opportunities
• Inspired or motivated to take a specific action
• Strengthening existing connections
• Making friends or social connections
• Event enjoyment (just having a really positive experience)

These are the results:

A key limitation to this analysis was that we explicitly asked attendees to report how connections were valuable to them, meaning the “Connections” response is likely artificially high. Learning about career options and EA cause areas (incl. via sessions) came through as a frequent source of value for our attendees. I had expected more attendees to report general positivity about the event, but this appeared far less frequently than learning or connections.

Some of these options have significant overlap. To get a slightly clearer picture of how “learning” in general might compare to “connections” in general, I grouped options together into the following baskets:

• Connections (“Connections”, “Strengthening existing connections” and “Making friends or social connections”)
• Learning (“Learned about new career options”, “Finding specific sessions interesting or valuable” and “Learning more about EA cause areas or taking them more seriously”)
• Motivation / Positivity (“Increased motivation or positivity for EA broadly” and “Increased motiviation or positivity for EA-aligned work”)
• Action (“More likely to attend other EA-aligned events/opportunities” and “Inspired or motivated to take a specific action”)
• Other (“Event enjoyment” and “Other”).
   

These are the results using the grouped baskets:

With this grouping, which is more favourable to the “Learning” basket (previously split into frequently cited categories), it’s striking that this basket outperforms Connections, despite the survey explicitly prompting attendees to report on connections. 

This updates me towards thinking that learning (about career options or ideas) is a large source of value from EA community-building events. Like connections, it’s a source of value where tangible outcomes that might occur downstream of this value will be difficult to track - attendees might well just form a deeper understanding about a key EA cause area, take action later on, but not report an event as being a pivotal moment in their journey. 

That said, providing opportunities for learning might not be the comparative advantage for events. Information about EA-aligned career opportunities and ideas can be found online (e.g. via the 80,000 Hours website, the 80,000 Hours podcast^[3], the EA Forum or via the many EA newsletters) whereas in-person events are one of the few ways EA community members can meet other community members in person and build their network. This weakly suggests that connections might still be the source of value event organisers should focus on.
 

My thanks to Callum Calvert, Jona Glade, Michel Justen, Sophie Thomson, Oscar Howie, Ben West, Eli Nathan, Ivan Burduk and Amy Labenz for comments and feedback.

1. ^{^}

    Specifically, we asked how many new connections attendee made, “How were these new connections valuable to you, specifically?” and “What other sources of value did you get from attending the event, which aren’t captured by connections?”.

2. ^{^}

   My thanks to our virtual assistant, George Go, who actually coded all of the answers with my input and feedback.

3. ^{^}

   Conflict of interest: my fiancé, Luisa Rodriguez, is a co-host of this podcast.

David_Moss @ 2023-07-05T12:36 (+4)

it’s striking that this basket [Learning] outperforms Connections, despite the survey explicitly prompting attendees to report on connections. This updates me towards thinking that learning (about career options or ideas) is a large source of value from EA community-building events.

 

In the EA Survey, we also found that there was a surprisingly low gap between learning something important and making a new connection, in cases where one might have expected an influence to strongly favour one or the other. (For example, it was mentioned to me that the EA Forum producing so many connections as it did was surprising).

I think these results are probably partly due to measurement error (specifically, I think a general 'positive feeling about the influence' factor makes people more likely to attribute either learning or connection to the factor, but not entirely.

It's perhaps also worth noting that in these results EAG and EAGx come out with more Connections than Learnings (not weighted for importance), though they also perform appreciably well compared to other sources for Learnings. 

I think these earlier results also confirm the importance of scale noted in your post, i.e. the dominant factors just seem to be those operating at much larger scale, rather than smaller, more targeted interventions (this seems to be a recurring theme across our results) and the difference in scale between different EA projects (in terms of how many people they reach) varies enormously:

OllieBase @ 2023-07-17T09:43 (+4)

Thanks for sharing, David!

James Herbert @ 2023-07-06T13:53 (+1)

Hey David, your observation that 'the dominant factors just seem to be those operating at much larger scale, rather than smaller, more targeted interventions' is a recurring theme is very interesting! 

To check I understand, are you saying that things like websites, podcasts, and other scalable things seem to be having much more impact than things such as 1-1s? I'm asking because we at EA Netherlands sometimes wonder how much we ought to be investing in our website vs smaller and more targeted interventions. 

David_Moss @ 2023-07-07T13:04 (+11)

are you saying that things like websites, podcasts, and other scalable things seem to be having much more impact than things such as 1-1s?

 

Basically (though I'm thinking more about the scale of programs (i.e. whether the program actually reaches a large scale) than the scalability of kinds of program). 

We observe that the factors which are cited by the largest numbers of people as being among the most important influences on them are those with very large reach, i.e. those which many EAs have been exposed to (e.g. 80K website), followed by those with smaller reach (e.g. EA Forum), and then by smaller programs. 

Indeed, when we look at the association between the number of people in the EA Survey reporting having interacted with a given factor in the last 12 months and indicating that it was one of the most important influences on their ability to have a personal impact, we observe a very strong correlation (r=0.8, [0.5-0.9], p=0.001]. That might seem truistic, but I think (per the OP's other post), many people expect some smaller, more targeted programs to be dramatically more influential than more 'mass' outreach.

Some important qualifiers:

• Obviously this is looking at reported exposures among people who are taking the EA Survey. It should not be taken to capture all exposures (e.g. many people could view the 80K or GiveWell websites and never have any further interaction with EA), so this shouldn't be taken to indicate a general ratio of impacts per exposure.
• This data comes from two separate questions with different numbers of respondents (so it's possible for an influence to have more impacts than exposures) which is another reason why it shouldn't be taken as giving a literal exposures:impacts ratio.
• This is looking at numbers of people citing the factor as among the most important influences on their ability to have an impact, but these are not weighted. So this is compatible with certain influences being much more important than others, though as noted, the post linked above may give some reasons counting against that.
• This strong correlation is compatible with some large differences (visible on the plot) between influences which have similar numbers of exposures but which lead to very different numbers of impacts.

That said, I think this still serves as a somewhat useful illustration of the extent to which the factors with the largest number of impacts are those with the largest number of exposures among EAs. Given the very large differences in scale between programs, smaller programs needs to be getting a dramatically higher number (or higher value) of hits to compete with the larger influences.

I'm asking because we at EA Netherlands sometimes wonder how much we ought to be investing in our website vs smaller and more targeted interventions. 

When it comes to thinking about EAN's particular options, I would add some additional caveats: 

•  The differences in scale between the largest programs (e.g. the most prominent EA websites / podcasts) and the smaller programs, discussed above, may differ when comparing the possible EAN website / podcast / smaller programs (e.g. the likely differences in scale may be a lot smaller. And in your particular situation, you might be dramatically better placed than others to run a website/podcast/other program.
• In other data, we observe that the impact of different factors like websites and podcasts is very skewed e.g. the most effective podcast reaches many more EAs than any of the others. So even if podcasts as a category seems like it large scale, the typical new podcast/website might be expected to have much smaller reach, more comparable to a less scalable kind of program. 

James Herbert @ 2023-07-07T16:12 (+1)

Thanks so much for the detailed reply! This is very helpful :)