Our plan to share "PR FAQs" for new Forum features

By Ben_West🔸 @ 2021-07-29T04:10 (+48)

Want to shape the future of this Forum?

CEA’s new design process lets you provide early feedback on changes to our online projects, so that the online community is more the place you want it to be.

CEA has started using a design process popularized by Amazon known as "press release frequently asked questions" (PR FAQ). A short description of this process can be found here, and a longer description in this book. The idea is that product developers start the design process with a press release, rather than ending with one. This encourages developers to start with an important customer need and work backwards to solve the engineering constraints required to meet that need, rather than vice versa. 

(This document is an example of a PR FAQ.)

CEA's online team is going to pilot publishing a handful of PR FAQs on the Forum in August. Our hope is that this will:

1. Let us design better features, by getting feedback from the community early and often
2. Prioritize features, by seeing how important users think different changes are
3. Increase engagement, through users feeling more bought into and excited about features they participated in designing

This will be a one-time trial, with no guarantee of continuation. If you would like us to keep publishing PR FAQs, please comment on them!

Fake Quotes

"The subscription feature proposed by CEA was not something I would ever use, but they listened to my feedback and it's now replaced my RSS feed!" – Forum User

“I felt like we were spinning our wheels in internal discussion, but after seeing how strongly people reacted to the mockups I published, it’s clear that we should make the Events feature more prominent in the sidebar.” – Forum Developer

FAQs

Which PR FAQs will be published?

For August, each member of the online team (Aaron, Ben, Jonathan, JP) will publish one PR FAQ. They are intended to be for:

1. Medium-sized features which can be deployed within about one month
2. Publicly facing projects about which the median Forum user could realistically provide feedback

Depending on how successful this pilot is, we may change the scope of which ones get published.

How can we tell if this pilot is successful?

We will measure the amount of engagement we get and the number of meaningful changes to features we made as a result of feedback from users. We would also like to consider whether features get more engagement after they are released as a result of this process, though the counterfactual is harder to estimate there.

Lastly, we will consider feedback on this PR FAQ to gauge whether the community is excited about this process.

Will publishing these PR FAQs lead to the community interpreting them as commitments rather than projects we are considering?

This is a real risk, and something we will have to track. There are a couple reasons to think the benefits outweigh the costs, though:

1. Much of our work would not change others' behavior, even if it was a commitment. E.g. if we add avatars to user profiles, that's not really going to influence anyone else's work (except maybe LessWrong + Greater Wrong). Very few people are in a position where they would e.g. have counterfactually used a different Forum if they had known that our feature set would be different.
2. To the extent our work does overlap with others’ (e.g. some Forum features might interact with work the EA Hub does), PR FAQs might prompt those people to coordinate with us, making this process a net win for coordination.
3. Engagement with our users is extremely important. This forum has ~0 value without people engaging on it. One advantage of having a relatively small community is that we can realistically consider and respond to all the feedback we get, and we should take advantage of that.

Can feedback on a PR FAQ result in CEA deciding to drop the project entirely?

Yes, if people are negative about some proposed feature, then the process is working as expected and we should respond to their feedback appropriately by dropping the project or dramatically changing our approach.

Why do we think that the benefits of communicating more publicly will outweigh the costs?

Public communication is often more costly than one would naïvely expect. The Open Philanthropy Project has written about shifts in their public communications strategy, as one notable example in the EA community.

Conversely though, talking regularly with your users often has more benefits than one would naïvely expect. “Spend more time talking to your users” is perhaps the most common piece of advice given to startups, and many companies have sites where users can provide feedback.

I suspect we (the online team) are closer to a tech startup than we are to OpenPhil: for example, our design decisions are more likely to be driven by aspects of the forum easily understood by any power user, as opposed to private knowledge about an esoteric subject.

Nonetheless, this is a pilot precisely because we aren’t sure whether the benefits will outweigh the costs, and it is something we will have to evaluate.

Will PR FAQs go through CEA’s document review process?

No. They are intended to be semi-informal communication with the community, and we are accepting the cost of having some typos in them in order to streamline the communication process.

They will be reviewed by the online team as part of our monthly planning process.

Will PR FAQs be displayed in any special way on the Forum?

We will give them the Effective Altruism Forum tag, which users can choose to subscribe to (or hide from the front page) if they wish. If we continue this process, we can consider whether more changes are warranted.