Sarah Cheng's Quick takes

Some quick responses to Nuño’s article about EA Forum stewardship

I work on the CEA Online Team, which runs the Forum, but I am only speaking for myself. Others on my team may disagree with me. I wrote this relatively quickly so I wouldn’t be surprised if I changed my mind on things upon reflection.

Overall, I really appreciated Nuño’s article, and did not find it to be harsh, overly confrontational, or unpleasant to read. I appreciated the nice messages that he included to me, a person working on the Forum, at the start and end of the piece.

On the design change and addition of features:

People have different aesthetic preferences, and I personally think the current Forum design looks nicer than the 2018 version, plus I think it has better usability in various ways. I like minimalism in some contexts, but I care more about doing good than about making the Forum visually pleasing to me. To that end, I think it is correct for the Forum to have more than just a simple frontpage list of posts plus a “recent discussion” feed (which seems to be the entirety of the 2018 version).

For example, I think adding the “quick takes” and “popular comments” sections to the home page have been really successful. By making quick takes more salient, we’ve encouraged additional discussions on the site (since quick takes are intended for less polished posts). “Popular comments” helps to highlight ongoing discussions in posts that may not be visible elsewhere. I take the fact that LessWrong borrowed these sections for their site as further evidence of their value, and in fact LessWrong has had some impactful discussions happen in their quick takes section. As another example, features like the “Groups directory” and the “People directory” are not available anywhere else online, and I view them as more like “essential infrastructure for the EA community”. I think it’s reasonable for those to live on the EA Forum, where people already gather to talk about EA things and look for some EA resources. 

On the costs of the Forum:

Here, I want to clarify what the actual costs are. I will focus on budgeted costs over true costs for privacy reasons.

• I believe that the $2M/year was the highest ever (overall, budgeted) cost of the CEA Online Team, when we were a team of 8 people, and it was only the case for approximately 6 months (between Jan 2023 and July 2023). We were probably 7 people for another ~6 months total, and less than 7 people for the rest of the time.
• We are currently at 6 people, and our team’s budget for the year is close to $1.6M/year. I joined in late 2021, when the team was 5 people (including me). 6 is a more representative number for how many people have been on the Online Team in the past few years.
• My understanding is that about $960K/year of the budget is staff costs, broadly construed.^[1] The rest are things such as SaaS costs (which we have made significant strides in reducing in the past year).
• Our true staff costs are lower, because multiple people on the Online Team choose to reduce their take-home pay. We budget for their full salaries because CEA needs to be able to pay them that salary if they choose to take it. I think it’s reasonable to allow people to keep this information private, so we do not report the true staff costs.
• The Online Team does not spend all of its time working on the Forum (although it is a majority of our time). The actual time split depends on when other projects arise, either via external partnerships or internal initiatives (such as the big EA.org redesign in 2022). I would broadly estimate that we spent 10% of Online Team staff time on other projects over the past 3 years.
• Therefore, the actual average yearly cost of Forum staff time has been roughly $860K minus voluntary salary reductions.
• In the future, I expect more Online Team staff time to be used for non-Forum projects, so I expect that cost to naturally decrease over time. (We spent roughly 20% of staff time on non-Forum projects in the past 3 months, for example.)

EDIT: I will also note that there are additional overheads to staff that are not captured even in the "broadly construed" figures above. For example, we have a People Ops team whose job is focused on CEA staff, so one could argue that part of the People Ops budget goes towards the Forum team. Part of the rest of the $1.6M/year is also ops overhead related to having staff. You could even argue that some of the SaaS costs are overhead on staff. I didn't want to go line by line and delay posting this quick take, but I'm adding this here to highlight that "staff costs" can vary a lot depending on what you consider part of that cost. I just picked a "staff cost" value that I thought would generally match the definition that readers would have in mind (i.e. the most direct costs to the company, which is mostly salary plus benefits). I would guess that from the CEA perspective (like if we were considering hiring another person on our team), we would take into account other overhead costs to some extent.

[I will skip responding to the moderation section because I have not been involved with that.]

On Forum culture evolution:

I guess I generally think things like, making the Forum more accessible and decreasing barriers to entry, are good. There are many people in the world who can do good via the broad project of effective altruism, and I want the Forum to work for them as much as is reasonable. Of course there are tradeoffs, and we do need to keep the interests of more experienced users in mind and be mindful not to lose parts of the Forum culture that are valuable, but I don’t see why that should mean we stop making the Forum more accessible. In my opinion, new Forum users are some of the most valuable contributors, and it would be to our detriment to ignore them.

On cost-effectiveness:

I think we agree that what matters is marginal cost-effectiveness. I believe that the marginal expense of the Online Team is a software engineer. Our team has done an internal marginal impact analysis to evaluate how much value the marginal engineer produces via the Forum (broadly, by looking at the software engineering work we have done in a year, BOTEC-ing how much counterfactual value it has produced, then attributing an appropriate fraction to the marginal engineer), and the resulting 90% confidence interval was above break even.^[2]

1. ^{^}

   Inclusive of benefits, travel, allowances for equipment, etc. (Not all of which is spent.)

2. ^{^}

   Multiple people within CEA red-teammed it (and there is not full agreement on the current version). The last time I looked at the Squiggle model, the analysis had over 100 factors. That doesn’t prove its accuracy but hopefully it demonstrates that we take this seriously. I hope to publish more about this publicly, but it will take a good chunk of time to extract the private information and write a version that is clear for a public audience. And, as I mentioned in another comment, I actually don’t know how to incorporate the time I spend on writing it up into the marginal impact analysis, so by default I feel like it’s a poor use of my work time (like, by spending time writing a public version, I would actually be decreasing the value of the marginal engineer, though this is slightly complicated by the fact that I’m currently technically in a non-engineering interim role).

Mini EA Forum Update

You can now subscribe to be notified every time a user comments (thanks to the LessWrong team for building the functionality!), and we’ve updated the design of the notification option menus. You can see more details in GitHub here.

Mini EA Forum Update

You can now subscribe to be notified when posts are added to a sequence. You can see more details in GitHub here.

We’ve also made it a bit easier to create and edit sequences, including allowing users to delete sequences they’ve made.

I've been thinking a bit about how to improve sequences, so I'd be curious to hear:

1. How you use them
2. What you'd like to be able to do with them
3. Any other thoughts/feedback

Mini EA Forum Update

We’ve updated our new user onboarding flow! You can see more details in GitHub here.

In addition to making it way prettier, we’re trying out adding some optional steps, including:

1. You can select topics you’re interested in, to make your frontpage more relevant to you.
   1. You can also click the “Customize feed” button on the frontpage - see details here.
2. You can choose some authors to subscribe to. You will be notified when an author you are subscribed to publishes a post.
   1. You can also subscribe from any user’s profile page.
3. You’re prompted to fill in some profile information to give other users context on who you are.
   1. You can also edit your profile here.

I hope that these additional optional steps help new users get more out of the Forum. We will continue to iterate on this flow based on usage and feedback - feel free to reply to this quick take with your thoughts!

New: floating audio player for posts

You can already listen to Forum posts via the audio player on the post page and via podcast feeds, thanks to Type III Audio. Recently, we enabled their new floating audio player. Now, the player becomes fixed to the bottom of the screen when you scroll down, making it easier to read along with the audio. In addition, you can click on the play buttons next to section headings to skip directly to them.

As always, we’d love to hear your feedback! Do you prefer to read or listen to posts? How can we improve the listening experience? Feel free to respond here or contact us in other ways.

New: “card view” for frontpage posts

We’re testing out a new “card view” for the main post list on the home page. You can toggle the layout by clicking the dropdown circled in red below. You can see more details in GitHub here.

Let us know what you think! :)

The CEA Online Team (which runs this Forum) has finalized our OKRs for this new half-quarter, and I've updated the public doc, so I'm reposting the link to the doc here.