EA Forum engagement doubled in the last year

By Ben_West🔸 @ 2021-11-04T10:44 (+74)

For CEA's Q3 update, we're sharing multiple posts on different aspects of our work.

The Effective Altruism Forum aims to be the central place for collaborative discussion about how to do the most good.

Metric status

Hours of engagement are up by 100.2% year-over-year as of October 1. This meets our target of 100%, and means we’ve doubled engagement hours this year.

We weren't tracking this metric until 2020, but based on other data we have, we estimate that the previous year's growth rate was 90%, for two-year growth of ~380%.

This chart shows daily engagement hours for 2020 (yellow) and 2021 (blue). The red line shows our daily "target" (the numbers we'd need to hit to reach 100% YoY growth if our engagement grew linearly).

In response to a comment, we'll also share our graph for monthly active users (people who viewed at least one page while logged in):

We went from 727 MAUs in 10/2019 to 1140 in 10/2020 and 1954 in 10/2021.

Progress this quarter

Key progress:

Other progress:

Reflections

I estimate that engineering output tripled in September (e.g. all three of the user-visible features I mentioned were released in September.) This is in line with the tripling of team size. Given that Sarah just started this month, I’m pretty happy with that increase in productivity, though I also think that I could have made the onboarding process even smoother.

lukeprog @ 2021-11-04T15:21 (+44)

FWIW the EA forum seems subjectively much better to me than it did ~2 years ago, both in platform and in content, and much of that intuitively seems plausibly traceable to specific labor of the EA forum team. Thanks for all your work!

Ben_West @ 2021-11-11T01:13 (+2)

Thanks Luke! We appreciate the kind words.

NunoSempere @ 2021-11-04T11:15 (+19)

Nice to see!

Ben_West @ 2021-11-11T01:14 (+4)

Thanks Nuno!

 What is the cumulative number of hours for 2021 so far? 

CEA uses a goal year which starts October 1, i.e. we have just started our 2022 year. The above post was about our prior goal year October 1, 2020-October 1, 2021, which is roughly YTD Calendar year 2021.

We have had 12,000 hours of engagement so far in goal year 2022:

 

 Do you have the figures spent in the EA Wiki specifically? 

Between October 1, 2020 and October 1, 2021, these pages received 65,000 views and 1,800 hrs.

I imagine that there is some way to know when someone is just leaving a tab open for a while, but can you elaborate on how you deal with that?

We use document.hidden and document.visibilityState to determine if the tab is open but not visible, and use mouse move, keypress, and scroll events to determine if the tab is open but idle. Relevant code is here.

Relatedly, what is engagement driven by? By a few users who use it a lot, or by very many users who use it a little?

Relative to most online content platforms, I think we have a small but dedicated audience. (E.g. because many platforms rely on their posts randomly going viral on social media, whereas this does not tend to be the source of very much engagement for us.) So I think the direct answer to your question is “a few users who use it a lot”, at least relative to something like BuzzFeed.

Here’s the PDF of how much time users spent engaging over the past month:

This would look like a negative sloping line for most sites, but you can see it’s relatively flat for us.


 

Sophia @ 2021-11-09T10:49 (+14)

This sort of thing is hard to judge but I'd guess that having a moderator who seems enthusiastic, approachable and friendly makes a huge difference.

I think Aaron Gertler might have made the forum a much less intimidating place to contribute to the discussion than it would have been without his comments and posts (and also that this sort of thing compounds over time). Many of his additions seemed to nudge the forum culture in a more open, more curious, more patient and more welcoming direction (I have read the forum regularly for years and I noticed the change in tone - which seemed in no small part to the huge number of generally encouraging, patient and kind posts/comments by Aaron - changing how much I impulsively wanted to engage).

Ben_West @ 2021-11-11T01:15 (+3)

Thanks Sophia! I really appreciate you sharing this – it’s helpful for us to get a sense of which aspects of our work are motivating to people – and I agree that Aaron is great!

Ramiro @ 2021-11-05T19:33 (+12)

Thanks and Congrats.
I wonder if part of these effects (+ engagement, +headcount, +funding) could be temporary due to covid etc. - instead of a stable trend. Can we  rule this hypothesis out?

Ben_West @ 2021-11-11T01:15 (+3)

Hey Ramiro, it does seem likely to me that part of the growth is exogenous. EA is growing rapidly as a community, and COVID also probably resulted in people spending more time online.

Some evidence that growth is at least partially endogenous to the Forum though includes that LessWrong and 80000hours.org didn’t grow very much over the past year, whereas we doubled.

kyle_fish @ 2021-11-04T11:19 (+6)

This is great to see! Do you have a sense of what fraction of the EA community is engaging with the forum? I'm curious how much of this growth is driven by the increased size of the EA community, versus an increased percentage of community members using the forum.

Benjamin_Todd @ 2021-11-04T12:15 (+9)

Would be useful to see the number of unique users over time, rather than just engagement hours.

Aaron Gertler @ 2021-11-04T16:08 (+4)

I've added a chart to the post.

David_Moss @ 2021-11-04T15:54 (+6)

Among respondents to the EA Survey, in 2020, 38% of respondents were EA Forum members. In 2019 it was 30%. In 2018 it was 20%. 

Those numbers are doubtless inflated though, because EA Forum members (a very disproportionately highly engaged group: >80% are levels 4-5 out of 5 in self-reported engagement) are more likely to take the survey. The question is how many less engaged (who are less likely to be on the Forum) there are, which is  less easy to estimate, although there is a model in this post.

David_Moss @ 2021-11-07T18:17 (+4)

A useful comparison point might be how many EAs are members of a local group. A priori, one might think that being a member of an in-person group is a higher bar/more demanding than being a member of an EA Forum, but historically that has not been the case.

One other thing that may be of interest is that we don't see much of  difference in the increase in EA group EA forum membership between EAS 2019 and EAS 2020. But, apparently, there's been a lot more support for groups too, so perhaps that's not surprising. (One other possible thing of note (not shown) is that the EA Forum also seems to be increasing its share among newer EAs, where previously it had been almost exclusively the preserve of established EAs).

Ben_West @ 2021-11-11T01:15 (+2)

A couple of people have already posted some data sources, but I will add one more: my attempt to identify the percentage of people who engaged with one CEA program that went on to use the Forum in the next year. This relied on naïve matching of email addresses, so it’s probably a substantial underestimate, but found 25-40% of event attendees read a Forum post within 2021.

I also track a data set of people who are in leadership positions at EA organizations, and this group averages reading 0.7 posts/day (i.e. they are super engaged on the Forum).

Penetration is substantially lower among people who are newer to EA. Over the next year, we are considering focusing more of our attention on them.