Effective Altruism Forum web traffic from Google Analytics

By vipulnaik @ 2016-12-31T21:23 (+10)

Note: If you are using uBlock you may not be able to see the images, because they have "/google-analytics" in the name and uBlock is a bit ... paranoid ... about blocking file names with that substring. Please disable uBlock for effective-altruism.com or temporarily disable it to see images.

As 2016 comes to a close, I thought it'd be interesting to look at traffic trends on the Effective Altruism Forum (which I'll call EAF for this article) since its launch around September 7, 2014. Some key takeaways follow, after which I discuss more of the data.

Key takeaways

I close with some speculative thoughts on implications.

Some additional graphs and data (that weren't important enough to make it to the main post) are in the Miscellanea section.

Annual cycle and long-term growth trends

Here's a graph of monthly pageviews in Google Analytics. Note that since the site was launched after a few days of September 2014 were already over, the September 2014 number represents a period shorter than a month.

You can see that December and January have higher traffic than the surrounding months for the end of 2014 end-of-year Giving Season. For the 2015 end-of-year Giving Season, February (so February 2016) actually has slightly higher traffic than January, but you can still see the same qualitative picture: a sharp rise in traffic in December, a decline in January, and a return to normalcy soon. For the 2016 end-of-year Giving Season, we only have December data, and we see a pretty huge rise from November to December.

An year-over-year comparison between EAF's first and second year (September 7, 2014 to September 6, 2015, compared with September 7, 2015 to September 6, 2016) shows a 9.86% increase in pageviews (from 314,789 to 345,832), a 21.53% increase in sessions, and a 26.97% increase in users. Pages/session and average session duration fell.

Looking at some specific months shows how noise overwhelms this slight long-term increase. For instance, in September 2014 (the month of launch, where a few days were missing since those were pre-launch), there were 26,256 pageviews. The corresponding numbers in September 2015 and September 2016 were 30,467 (+16.0% year-over-year) and 26,347 (-14.5% year-over-year). For October, the numbers in 2014, 2015, and 2016 were respectively 25,237, 26,259 (+4.0% year-over-year), and 32,880 (+25.2% year-over-year).

With that said, there is clearer evidence of year-over-year growth for November and December. As a result, the gap between Giving Season and other months has been increasing from 2014 to 2015 to 2016.

Data is very limited but one tentative theory that fits the evidence so far is that the growth of the EA movement has been strongest among readers whose interest in EAF content is high only during Giving Season. Another is that content creators are focusing their content creation and publishing efforts more during Giving Season and less during other months, and this is increasing the traffic difference between Giving Season and other months.

One interesting trend between late 2014 and late 2015: although the increase in number of pageviews was small, sessions increased quite a bit. Pages/session dropped to compensate. This could be because people began using EAF more on their phones (thus increasing the number of sessions and users recorded by GA). It could be because the number of people accessing EAF increased, but the interest and dedication of the early adopters declined.

I also compared this growth rate against that of other websites that cater to similar audiences. I was limited to the ones that have released some public data, namely, 80,000 Hours and GiveWell. Here's the comparison:

Impact of EA events

The main story regarding EA events is the non-story: there is no increase in traffic to EAF in or around EA Global or major events. I got a list of events from Effects of major events on EA activity by Eric Yu. His post didn't discuss effects on EAF traffic, although the original post by Peter Hurford that his post answered had asked for that data. So I thought this would be a good set of events to look at:

Post traffic lifecycle: norms and exceptions

The vast majority of EAF posts get the vast majority of their traffic within a few days of publication, and all the associated engagement activity (post upvotes and comments) also occurs within that time. This time can vary from a day to a little over a week. Typical examples:

The fact that most posts have short lifespans is part of the explanation for why, despite the cumulative content on EAF being much much more now than it was during launch in September 2014, total pageviews aren't up that much. If the posts had longer lifespans, and continued to get steady trickles of views, these steady trickles would add up to a lot of total pageviews, and that number would increase as the total number of published posts went up. However, since posts typically die quickly after being published, we don't see the steady accumulation of pageviews. So pageviews in a given month are determined more by new content created at the margin rather than the total corpus of content so far.

However, the top viewed items of all time display a different pattern. Like the other posts, they have a spike around the time of initial publication (or initial import into EAF), and much of the engagement activity (upvotes, comments) occurs within that time. However, unlike other posts, the top viewed items continue to get a steady trickle of views later in life, or additional later spikes in traffic, long after active engagement with them (in the form of comments and upvotes) has died down.

Another noteworthy feature of the top posts is that many of them were imported into EAF, or cross-posted to EAF, from other places (such as LessWrong, or the old Effective Altruism blog that was hosted on the same domain).

The situation with Facebook likes + comments + shares seems similar to long-term traffic (and therefore different from comments and upvotes on EAF): posts that continue to get a steady trickle of views also continue to get a steady trickle of Facebook likes, comments, and shares. However, I don't have good time series data on Facebook likes, comments, and shares, so this is just a rough estimate based on what i do have.

Here are the top five posts:

Beyond the top five, the picture becomes more mixed. The sixth and seventh article, namely Philosophical Critiques of Effective Altruism by Prof Jeff McMahan and Cheerfully, had only one spike at time of publication, but do still get a very slow but still steady trickle of traffic. In eighth place is the wide-interest article Six Ways To Get Along With People Who Are Totally Wrong*. This has had three big spikes and a much much smaller steady stream of views.

The most-commented article I could find was Concerns with Intentional Insights, with 39 net upvotes and 182 comments, and 60 Facebook likes + comments + shares. This placed ninth in terms of lifetime pageviews. The traffic to this article had a single, really huge spike at the time of launch, and has since been pretty low.

Another way of trying to gauge the relative importance of steady views to top-performing content versus views of new content is to compare total pageviews against pageviews of the top five articles of all time over time. I have included a graph below with this data. All time, the top five articles account for 52,872 pageviews out of 467,380. In December 2016 (after any of them was published, so it misses the initial peak of all five) the top five accounted for 2,703 out of 28,284 views. Overall, then, the top five of all time account for something like 10% of pageviews in a given month.

Speculative thoughts on implications

The slow increase in total traffic to EAF suggests some tempering of the widespread narrative of rapid EA movement growth. It would be interesting to understand why EAF traffic lags other widely cited indicators of movement growth, such as GWWC pledge sign-ups and GiveWell money moved. A number of plausible theories can be proposed. For instance, it could be that EAF appeals only to a segment of EAs (and EA-penumbra people) and this segment is not represented well in new recruits. On the other hand, it could be that EAF has huge churn, so that even though the EA movement is growing, EAF membership at any given time isn't growing that fast because people leave quickly. It's also possible that the problems are on the supply side (not enough quality content on EAF, or more competition for readers' attention).

More interesting (to me) than the overall traffic trend is the question of what the pageviews and other metrics for specific posts can tell us about the impact of those posts. Content creators get feedback on posts based on initial traction in the form of comments and upvotes, which correlate well with initial pageviews. But for the best-performing posts in the long term, a lot of the pageviews happen over time, and the long-term pageview trend is not as correlated with the initial spike in pageviews as we might think.

One possible implication is that content creators on EAF may be overproducing content that gets a lot of initial traction, at the expense of creating evergreen content with long-term value. I mean this more in the sense that people are reluctant to produce evergreen content because they don't see it getting that much traction, rather than that they are cynically generating content of short-term value to increase their karma. Of course, this implication is far from proven, as it's not clear that long-term pageviews are a good indicator of long-term value. But even if it's not, analogous reasoning might apply for long-term value.

Take a look at the top five posts. Four of them are generic, evergreen content whose relevance declines very slowly over time. The fifth one (on donating to the Wikimedia Foundation) is more specific, but the topic it touches on, namely donations to Wikipedia, is something that a sizable fraction of the Internet-using population comes across during Giving Season. Contrast this with a post like Concerns with Intentional Insights that gets a lot of interest when it is published but does not address a recurring, long-term human need, and therefore dies down quickly. Note that it's still possible (and many have argued) that the Intentional Insights post did have a huge indirect long-term impact not reflected in recurring pageviews, by changing norms around fundraising and accountability that would serve as cautionary lessons for future donors and organizations. My point of using it as an example was mostly to simply highlight it as a case where direct interest (as measured by pageviews) dropped sharply after an initial period. There is also a possibility that the post will see another traffic surge when similar issues crop up in evaluating other organizations, just like the Wikimedia Foundation post has had traffic surges triggered by Wikipedia banner fundraising.

Another implication is around the way we link to older material. A lot of content is high-quality, but fails to get visited after a huge amount of traffic in the first week. Regulars forget about the content, newcomers never hear of it. Better, stronger linking practices to older content, and more high-quality followup posts that help revive older content, could help address this. Discussions and debates advance when people build on top of past material, rather than regenerate or rehash it, so a strong memory of past content, reinforced through extensive linking to it, can help.

Miscellanea

Prior to the Effective Altruism Forum, the same domain name hosted an invite-only blog that had a handful of posts. These ports were ported over to EAF when it started (along with some more seed content). Below is the Google Analytics for the pre-EAF site. You can see it was significantly less than traffic to EAF. Data is available only starting October 2013. The drop in September 2014 coincides with the launch of EAF during the month.

Below, I paste the pageview data for the new Effective Altruism Forum at daily and weekly granularity respectively for the period September 1, 2014 to December 31, 2016, and then the hourly pageview data for December 2016. By looking at this data you can verify that there is a clear daily cycle in traffic, but no weekly cycle.

Data exports

Although I have Read&Analyze access to the EAF's Google Analytics, I don't have the authority to grant others access. You can contact Ryan Carey to give you Read&Analyze access if you are interested.

If there are specific EAF posts that you think are important enough to have their traffic patterns covered in this post, please list them in the comments and I'll add their traffic patterns to the post when I get time.

Related reading

Related reading on movement size and growth metrics for EAF and related websites and communities:

Public data on Effective Altruism Forum web traffic:

More normative discussions of EA movement growth:

Related reading on website traffic variation and understanding website audiences:

UPDATE June 17, 2017: Starting around late April 2017, I no longer have access to the Effective Altruism Forum Google Analytics data. Thus, I will not be able to update the existing exports or publish follow-up posts on the Forum's analytics.

undefined @ 2017-01-01T20:29 (+5)

Clarification: You're using EAF to mean EA Forum, while I usually see it used to mean EA Foundation.

undefined @ 2017-01-01T21:18 (+2)

Ah, I was really confused during the whole article and thought that this had something to do with traffic to the EA Foundation websites. Thanks for clarifying!

undefined @ 2017-01-01T21:45 (+1)

Thanks, I added the explication of the acronym at the beginning.

undefined @ 2017-01-02T19:06 (+3)

Thanks for the high-quality analysis!

The lack of growth of the EA Forum coupled with the large growth in EA Survey respondents confuses me. The 2015 EA Survey found 350 self-reported users of the EA Forum, but unfortunately the 2014 survey did not ask about that, so there's no longitudinal data (yet) to compare.

undefined @ 2017-01-02T19:17 (+1)

Perhaps a combination of i) different degrees of intensity of engagement and ii) changing methodology.

1 - The degree of high-intensity engagement (reading and writing intellectual arguments) is meant to be much harder to grow than total people who affiliate with EA.

2 - How did the Survey methodology change between years?

undefined @ 2016-12-31T21:27 (+2)

What traffic would you estimate the facebook group or other community venues to have?

undefined @ 2016-12-31T21:32 (+4)

You can get data on the Facebook group(s) using tools like http://sociograph.io -- however, they can take a while to load all the data. A full analysis of that data would be worth another post.

undefined @ 2017-01-02T19:01 (+2)

That would be really cool to see!

undefined @ 2017-01-03T10:36 (+1)

I think a big factor is simply that we don't promote the forum. There's little attempt to get new EAs to use it (e.g. no prominent links on EA.org or 80k), and there's little attempt to market the forum more broadly (e.g. content marketing strategies).

We could easily grow the forum's traffic if we made it a focus.

I think the point about a lack of evergreen content that's not highly internal to the EA movement is a good one, and a shame. If we had more content like that then I would expect to see more growth. On the other hand, it's difficult to make. Major 80k pieces can easily take a week.

undefined @ 2017-01-03T04:09 (+1)

.

undefined @ 2017-04-16T20:44 (+1)

Public link with up-to-date data https://www.reddit.com/r/EffectiveAltruism/about/traffic

undefined @ 2017-01-02T20:12 (+1)

I think Facebook and email dominate a large portion of nearly everyone's internet usage. I've seen other groups try to establish separate sites or discussion boards outside of Facebook and it never seems to work (EA Forum is probably successful relatively) because people just have so much social investment on Facebook.

CarlShulman @ 2017-01-02T22:05 (+1)

The EA Facebook group also has had slower growth in traffic than GWWC pledges or GW donations.