Moving community discussion to a separate tab (a test we might run)

By Lizka, Clifford @ 2023-02-06T21:36 (+197)

TL;DR: we (the Forum team) might make a pretty significant change on Frontpage for a month as a test. We’re sharing the outline of the proposal to get feedback from the community. The change we’ll test is hiding “Community” posts from the Frontpage for everybody by default, and moving those posts to a prominent tab. 

Where we are right now: 

We’re probably going to run the test, although we might not depending on the feedback we get as a result of this post and we might significantly change our approach. After the test, it’s pretty unclear to us whether we’ll keep the change — and we might want to extend the test period — but we think that we will probably want to keep something like the change after the month goes by. 

We will review the feedback we get, and if we decide to go for it, we’ll likely start the test within a week or two. 

Summary of the change we’d test: moving community discussion to a separate tab

For the test, we’d move all “Community” posts off the top section of the Frontpage for everyone by default (by setting the default topic filter for “Community” to “hidden,” meaning that people will be able to opt out of the change). We will probably also remove “Community” posts from Recent Discussion on the Frontpage (you will be able to change this back to the default in your user settings). 

Then we’ll probably either add a tab on the Frontpage or a section lower down on the Frontpage, which would feature a limited number of “Community” posts. You could then load more articles or visit the tab. 

There’s a chance that we’ll tweak our classification of what we mean by “Community” posts, but currently, it’s something like what’s described in this footnote.^[1] 

Here are two mockups of the layouts we’re considering testing (if we go for one of these, we’ll probably continue testing and improving it):

A “Community” tab

A section for “Community” posts on the Frontpage

Why consider doing this at all?

We’re pretty worried about the Forum voting structure driving more engagement to some topics systematically (without the community endorsing this prioritization), and related issues.

Some topics and discussions tend to get a lot more Forum karma and attention than others, as Ben and Lizka outlined in this post. These tend to be: 

• About the community (because it interests almost everyone at least a little bit)
• Accessible to everyone, or on topics where everyone has an opinion 

This phenomenon means that posts that interest a smaller fraction of the community — even if they interest their target readers a lot more and are more useful to their audience (and even if you consider that some other posts’ audiences are larger), will get a lot less attention than might be helpful. 

We want to do something to address this imbalance, and we think that separating out “Community” posts might get us a lot of the benefit while still letting important conversations happen. (We checked a lot of posts we thought were getting extra engagement via this effect; almost all were “Community” posts.)

Note that this mechanism “overvalues” “Community” posts whether or not you think that “Community” posts are currently over-valued by the average Forum user (the Forum team has mixed opinions on this^[2]); given whatever the community actually cares about, things that are easier to discuss or think about will get more engagement.^[3] (See more in this comment.)

We also want the Frontpage to feature different kinds of content that’s particularly interesting to different groups, instead of being full of posts that interest the majority of Forum users to at least some extent (in particular, posts about the EA community). This is how we get cross-pollination between different cause areas or types of work, give people who focus mostly on one cause area the opportunity to engage with excellent content from a different cause area, etc. The status quo makes this significantly harder. (This is not the primary driver of the change.)

We’ve been hearing about the Forum fixating on certain discussions for over a year: this has been making a lot of people sad for a long time.

The Forum team has been hearing about how the Forum often gets inundated with community-oriented posts (and how excellent cause-specific posts get missed) for a long time. (Aaron Gertler, who ran the content side of the Forum until December 2021, tells us that he was hearing this a lot during his time.) This is also often paired with worries that posts that are emotionally charged or on topics where opinions are really strong are particularly likely to encourage people to engage even when they don’t endorse engaging. 

These concerns often look like: 

• “Hey, I notice that I often get sucked into discussions that are vaguely relevant to me, but which I don’t think are important. I’ve started using the Forum less as a result.” 
• “When I go on the Frontpage, I just see posts about the community that stick around for a long time. I miss useful posts because they’re a bit more niche and don’t quite neatly fall into the niches I know to look for.”
• “I wasted hours yesterday reading all the comments on a post to see which comments I agreed with. I wish I hadn’t done that; I wasted time and was upset.”
• “I don’t want to share the Forum with people who learned about EA and are really excited about working on impactful projects because if they go on the Forum, it’ll look like all we care about is random community stuff. I don’t think it’s all we care about, but I don’t know if new people will understand that.” 
• Etc. (Note that these quotes are made up.)

We sought out more feedback and got more of the same concerns from people from different parts of the EA community. We also personally agreed with a lot of the concerns that people were sharing. There are also examples of people bringing up similar issues or even very similar solutions (see e.g. this comment) on the Forum and getting a lot of agreement, and Ben and Lizka’s post got a lot of karma (and agreement in the comments — along with some good pushback, but we were not flooded with concerns about the potential changes), which was also a signal that a lot of people were interested in this change.

We’ve tried a number of things to address these concerns: 

1. Giving people the option to filter topics and customize their Frontpage based on their interests, and promoting this option repeatedly.
   1. Unfortunately, most people tend to leave settings on the default. (Different people on the team have different opinions on the usefulness of this feature.)
2. In June, we down-weighted “community” posts for new users and logged-out users by default (more). 
   1. We didn’t modify the settings for existing users and now think that this was a mistake. 
3. In September, we decided to introduce curation.
   1. We think this has been helping a bit, but not really significantly counteracting the effects described. 
4. We were working on a project to develop subforums in the hope that it would help nurture important conversations relevant to people in different fields. We’ll be posting an update on that project soon, but it doesn’t seem likely to accomplish this goal. 

None of these really solved the issue. 

We’ve had a lot of discussions about this and want to test a change to get some answers.

We’ve had a bunch of conversations about this internally, and a lot of user interviews and conversations with different people from different parts of the EA community. A thing that often happens when you talk to people is that they’ll tell you they’re really excited about a potential feature or change you might push, but then when you make the change, they don’t like it, or don’t use it. We might find out that something like this is going on here. We might also discover that this is a type of change that sounds good in principle, but it turns out that everyone has slightly different interpretations of what it will involve or there are other practical details that are impossible to get right, and almost everyone hates it. 

What do we hope to find out through the test? 

We’d be tracking some things like: 

• Engagement with different types of posts, and how it might change
• Whether tricky discussions go better or worse after the change
• Feedback from the community

If engagement with the Frontpage drops a lot, if tricky conversations are even harder to have well, if we get important negative feedback, or if we see other things that worry us, we will consider the test to be indicating that we should revert the change and go back to the drawing board.

This would all be useful information that’s hard to get without running a test. 

What are our worries about the change we want to test, and what are we doing to mitigate these worries?

Note: for each of these, we’d be very interested in hearing more suggestions about how we can mitigate them. 

1. If we significantly decrease the visibility of community-oriented discussions, we might miss out on important conversations the community should have (especially things like suggested improvements to our norms, keeping key institutions in EA accountable, etc.). 
   1. To counter this, we will make sure that the “Community” tab is prominent or that “Community” posts get a dedicated spot on the Frontpage, and we’re considering facilitating some conversations like this more directly. We also plan to continue sharing top community posts in the Digest, etc. 
2. If the “Community” side of the Forum is what is most engaging to people (whether or not they endorse this engagement or this tendency), people may just move towards engaging more with the “Community” tab and engage less with the Frontpage (or engage less with the Forum in general). I.e. it’s possible that a significant part of the particularly useful engagement on the Forum is via a “drive-by” effect where you go on the Forum for a discussion about discussion norms, but accidentally see a technical report on a topic of interest to you. 
   1. We’ll be tracking data on this. 
3. Separating community-oriented discussion and more object-level discussion may lead to a drop in cross-pollination between the people who focus more on the community and those who focus more on the object-level. E.g. we’ll have posts about event strategy that don’t get useful feedback from domain experts in biosecurity who’ve been to interesting conferences, etc. 
   1. We’ll try to get feedback on this to find out if something like this happened as a result of the change. 
4. This change might make the Forum more confusing and structurally messy.
   1. We’ll gather feedback on how people understand the tabs/separate spaces, to see if it makes sense to users. Agnes, the UX designer on the team, will be designing the layout; we’re hopeful that it will make sense. But it’ll still probably add complexity, and we’ll need to decide if that’s a price we are willing to pay. 
5. Newcomers who visit the Forum might be surprised to not see discussions on recent events that are highly relevant to the EA community, or more generally just won’t quickly find discussions they’d expect from a community that’s alive, conscious of its problems, hosting events, etc. 
   1. The Frontpage is pretty active even if we remove “Community” posts, so that’s not a major concern. When there are important events that are getting a lot of coverage and attention even outside of EA, we might often have threads on the Frontpage for discussing them, so that might address some of the concern. We also might come up with better solutions to this problem if we see it realized. 
6. We may simply land on a significantly subpar solution to the problem (or accidentally solve the symptom of a worse problem that remains unsolved).
   1. If we decide to keep the change, we’ll still be on the lookout for other improvements. 

What sort of feedback is most useful, and how to give it

Please feel free to comment under this post, and we’ve set up two comment polls. You can also reach out privately by emailing forum@centreforeffectivealtruism.org.

1. ^{^}

   This piece of writing is primarily impactful via the EA community as a phenomenon. It's not significantly relevant to a non-meta organization, field of research, type of real-work-in-the-world, etc. This includes posts about the community, announcements from community-building or infrastructure organizations that are aimed at a familiar-with-EA audience, etc. It doesn’t include things like, “here’s a mindset that helps me really internalize the scope of a problem,” or “A guide on information security as an impactful career path from 80,000 Hours.”

2. ^{^}

   People on the Forum team have somewhat varying opinions on whether the EA community currently explicitly over-values “meta” posts — posts about the community — too much right now. Several people believe something like: “it would be useful for the average Forum user to spend a bit more time reading posts that are specific to a cause area in EA, over spending a bit more time reading posts about the community-as-a-phenomenon.” We’re worried that this is making us biased here, but we don’t view this as a key motivator of the change.

3. ^{^}

   We think that a lot of the discussions the community really needs to have are quite difficult, and we want those discussions to get the attention they need. This is either because they’re technical or depend on complicated arguments, or because they’re tricky community issues that we want to discuss rigorously. The karma phenomenon described in the post hurts those (more difficult) discussions in particular, and we want to counteract it.

Lizka @ 2023-02-06T21:37 (+51)

Agree-vote (✅) with this comment if you are overall optimistic about the test that we’re outlining in the post (e.g. because you think we’ll find out something useful, or because you think that the change will likely be helpful). 

Disagree-vote (❎) if you’re overall pessimistic about the test. 

Please use normal-strength votes (we’ll check for strong-agree-votes and cancel them). (We’re posting two comments as a quick poll to get a sense of what people who might be less comfortable commenting think. Please note that we won’t defer entirely to the results of these polls. See the other comment.)

Nathan Young @ 2023-02-07T01:30 (+7)

I think @ChanaMessinger makes a reasonable case that strong votes are much less reasonable in regard to agree/disagreevotes. If you think so here, maybe we shouldn't get them in general. I'm pretty ambivolent.

RobBensinger @ 2023-02-07T09:25 (+8)

I've told the LW admins before that I wish there weren't a way to strong-agree or strong-disagree. Or, alternatively, I wish there were hover text that lets you see how many people have strong-agreed vs. weak-agreed vs. weak-disagreed vs. strong-disagreed.

I think ideally I'd also like to see a breakdown into karma buckets, so you can explicitly see how many veteran users vs. new users agreed or disagreed, as opposed to veteran users just quietly getting more voting power.

Maybe something like three buckets: one for people who (based on their karma) probably are pretty new to the forum (or post pretty low-quality stuff), one for veteran users with tons of karma, and one for users who are somewhere in the middle. E.g., an image that shows up when you hover, like (imagining a hypothetical comment with net 12 agreevotes):

 

 

(Obviously all of this would have to be only for future votes, not past ones, since people didn't make their past agreevotes with an expectation that this would be public info.)

Lizka @ 2023-02-07T13:56 (+14)

I don't have a strong view one way or another on the topic of whether you should be able to use strong-agree-votes anywhere. I don't know what others on the team think. 

I'm quite excited about potentially developing or otherwise adding an easy-to-use poll feature on the Forum, though. It's not an obvious part of any of our major projects for the near future, though, and it's not clear to me that it's worth prioritizing over other improvements (in part because I don't know how much use it would get), so I don't know when exactly we might get to it, if ever — but this is something that would make me personally happy. 

Ivy_Mazzola @ 2023-02-07T17:02 (+5)

That's interesting, I'd have thought they are more necessary with agree/disagree. Because people with more karma may be more likely to better informed about the topic? But then again people with more karma are also more informed on forum and epistemic norms (upvote/downvote dichotomy?) So maybe both are important

I'm in favor of a weekly cap on how frequently users can use their strongvotes though!

To me it also seems that agree/disagree should be on the left and upvote/downvote on the right. If people are only going to vote on thing, they should prob vote on whether they think the statement is correct, right?