CEA Mid-year update (2020)

By MaxDalton @ 2020-08-11T10:06 (+86)

This is a linkpost to https://www.centreforeffectivealtruism.org/blog/cea-mid-year-update-2020/

We'd like to share an update on our recent progress. 

In a previous post, we set out five goals for 2020:

1. Developing our strategy: We're on track, with more work needed on metrics and data.
2. Narrowing our scope by considering spinning off EA Funds and Giving What We Can: We're slightly ahead of expectations; we hired leaders and set an initial strategic direction for each project.
3. Expanding group and community health support: We're somewhat behind expectations.
4. Improving online discussion: We're somewhat ahead of expectations. We learned a lot about how to run virtual events, and EA Forum engagement has grown significantly.
5. Streamlining internal collaboration and processes: We're on track or ahead of expectations on finances, customer relationship management (CRM), and culture. We're behind on hiring and the Oxford office move.

Overall, I think we're making good progress toward our goals for the year.

Specific progress:

• We hired Jonas Vollmer to lead EA Funds, and Luke Freeman to lead Giving What We Can. Jonas and Luke are experienced EAs with executive experience and excellent skillsets for the roles. We plan to investigate spinning off both organisations from CEA in the next year or so, although we will continue to support them both operationally.
• We hired Catherine Low as a contractor to improve support for groups, and she's had 60+ calls with organisers who gave the calls an NPS of 70+, which is excellent. We've also updated 16 resources for local groups, including an improved introductory fellowship curriculum template, discussion group guides, and advice on how to host virtual events in light of COVID-19 restrictions. Further progress here is a major focus.
• We renewed several Community Building Grants and made three new grants to EA Brown, EA NYC, and EA MIT.
• Events collaborated with EAGx organisers from around the world to create EAGx Virtual. With over 1,400 attendees, it was the largest EA event ever held. Virtual events are unlikely to be as valuable as in-person events, but now that we've developed the capacity to hold virtual events, we will likely continue to do so (in addition to holding live events), since they are less expensive and more accessible.
• Our key Forum metric (views of good posts) has doubled so far this year, meaning that more good content is being posted to the Forum and more people are reading that content. We also introduced highly-demanded features like tags, and some core introductory content.
• We've focused our community health work on mentorship and networking for underrepresented EA groups, and developing long-term plans to further improve diversity, equity, and belonging in the community.

I also feel that we've fixed a lot of internal issues over the last 18 months and that we continue to gradually improve. We have 13 months of runway. Staff performance, morale, and retention appear to be solid and improving. We've improved our accounting systems, grantmaking records, and HR compliance/onboarding, and invested our reserves to protect against inflation.

Program updates

Groups

We’re in the early stages of improving our support for groups. This is a major focus at the moment.

Groups support

• We are working to improve the introductory EA fellowship template with Community Building Grant recipients Huw Thomas, James Aung, and Alex Holness-Tofts. Introductory EA fellowships are eight-week reading and discussion groups run on university campuses. We estimate that approximately 400 students participated in an intro fellowship last year.
• We brought on Catherine Low, who had been an extraordinarily active volunteer supporter of EA groups, as a contractor to improve our support of groups. Catherine has had 60+ calls with group organisers, which so far score an average Net Promoter Score of 70+, which is excellent.
• We’ve partnered with organisers to improve 16 resources on the EA Hub related to starting and running groups, including discussion group guides and advice on running virtual events for groups transitioning to online meetups due to COVID-19.
• We’ve helped to run three monthly virtual meetups (with attendance ranging from 40-70 organisers) where organisers made connections across different geographies, explored resources, and discussed ways to improve their groups.

Community Building Grants

We regranted a total of $325,000, including grants to the following:

• Renewals
  • EA London: 1 FTE, 1 year
  • EA France: 1.2 FTE, 1 year
  • EA Toronto: 0.5 FTE, 1 year
  • EA Norway: 2 FTE, 1 year
  • EA Berlin: 0.5 FTE, 1 year
• New Grants
  • EA Brown: 0.5 FTE, 6 months
  • EA MIT: 0.5 FTE, 3 months
  • EA NYC: 2 FTE, 1 year

We collaborated with the current NYC organisers to run an application round for an EA NYC Community Director. We were excited by the quality of the applications we received, and decided to make offers to Arushi Gupta and Aaron Mayer to be EA NYC Co-Directors (rather than just making one offer). This is the first time that we’ve collaborated with a group to run an application round for a specific location. We’re pleased with how the process went, and it helped us further develop our processes for evaluating grant applications more generally. We may want to run similar application rounds in the future — though it took a relatively long time, so we don’t expect to be able to do this for many locations.

Events

EA Global: Virtual

In response to COVID-19, we replaced EA Global: San Francisco 2020 with a virtual event.

As the event was scheduled for March, just as the extent of the coronavirus epidemic was becoming clear, there was not much time to plan this transition. I think the team did a good job anticipating the change and adapting.

We met about half of our goals for the event.

Some key stats:

• Date: March 21-22, 2020
• Grip (networking app): 456 active users
• YouTube: 325 peak concurrent views
• Slack (group discussion): 270 members
• Likelihood to recommend: 7.82 (average)

Goals we achieved:

• Find three case studies of strong impact from the conference
• Improve welcomingness (such that >80% of respondents agree that EAG is a place where they feel welcome); 89% of survey respondents reported feeling this way

Goals we didn’t achieve:

• Average attendee feels comfortable reaching out to 5+ new people
  • New connections per attendee (average): Virtual = 4.4 | London = 9.6
  • Attendees with 5 or more new connections: Virtual = 37% | London = 80%
  • See chart for more data
• Maintain or improve recommendation scores compared to EA Global: London 2019 (see chart below for scores from previous events)

See more details in the wrap report.

Virtual EAGx

In June, we collaborated with EAGx organisers from around the world to run the largest ever EA conference, with over 1,400 attendees from 57 countries. For many community members without a local group nearby, this was their first chance to attend an EA event and meet other EAs from across the world. The event resulted in 10,456 connections and 2,495 meetings within the Grip networking app, and over 3,000 hours of content views.

Some key stats:

• Date: June 13-14, 2020
• Grip (networking app): 1,389 active users
• YouTube: 434 peak concurrent views
• Slack (group discussion): 1,025 members
• Likelihood to recommend: 8.44 (average)

This event’s “likelihood to recommend” score was a big improvement over our first virtual event, and was closer to the usual score for an in-person event (see chart above).

Although the proportion of attendees making important connections was lower than at previous events, 50% of survey respondents reported making a minor plan change, and over a third changed their mind about something.

Goals:
We achieved almost all of our goals for the event, including:

• 1,000 EAs attending over the course of the event (actual number: 1,424)
• Facilitating at least 700 one-on-one meetings (actual number: 2,495)
• Achieving an NPS over 30 (actual number: 43)

Systems improvements

We shifted our admissions system to new software, began batching responses to applications, and hired extra capacity to help with the admissions process. This seems to be saving time on admissions.

It seems that virtual events are worth running — they are normally not quite as good as in-person events, but cheaper to do and easier for people to access. We’re tentatively thinking of continuing to run virtual events in addition to in-person events in the future.

Forum

Metric progress

The Forum's key metric (number of views of good posts) has more than doubled since the start of the year.

While we've spent a lot of time fine-tuning and validating this metric, we want to flag that the definition of a "good post" is inherently subjective. Experienced EAs outside of CEA have generally agreed with our assessments when we've asked them to evaluate posts, but we want to avoid drawing strong evidence of the Forum's success from this metric alone.

(Some examples of "good posts" from this year include writings on randomista development, personal cruxes on AI safety, and considering a wider range of longtermist jobs.)
We've also seen many other metrics of Forum activity increase since the start of this year. For example, views of all posts by logged-in users have roughly doubled, and the number of posts with 2+ karma has increased by ~80%.

Technical progress

We continue to port features across from LessWrong. We recently imported tags, pingbacks and link previews. (Tags are our most requested feature ever.) We also contribute to LW’s codebase.

We also launched a sequence of posts introducing people to EA (this is a first draft — we welcome feedback via this form). We’ve begun to do more outreach to potential top posters, including promotion at EAGxVirtual, which resulted in our biggest spike in new users ever. We’ve also been trying to increase the Forum’s visibility on search engines (though we are struggling to make improvements, despite engaging SEO consultants).

We continue to offer editing for Forum posts, and to run the Forum prize, EA Newsletter, EA/CEA social media, etc.

Key issues

Low satisfaction scores: We asked users: "On a scale of 0 to 10, how likely would you be to recommend the Forum to someone who recently became interested in effective altruism?" The average response was 6.9. We think that this low score was partly because people felt that the Forum is relatively insider-y, and so not suitable for new people. However, we'd like to get those scores up.

Demographics:  According to Google Analytics, 41% of Forum viewers are female (higher than the proportion of community members who are female), but we believe* that males make up a higher proportion of authors than their proportion in the community. We are considering how we could encourage more women to post.

*Many authors are anonymous, so we aren't certain of this.

Community health

The community health team tries to enable EA's long-term success by responding to media enquiries, mediating community disputes and complaints, and providing advice and support to local organisers who are facing particularly difficult problems.

One of our goals is to be available to the community to help with our areas of specialty. Here's the breakdown of cases we've assisted with during 2020 so far:

• 87 new media requests or situations
• 12 situations or questions on diversity, equity, and belonging
• 7 situations involving harmful or unethical behavior
• 6 personal or mental health struggles
• Around 60 more minor cases or requests

Some of our proactive work:

• Sky Mayhew joined the board of directors of WANBAM. The data-sharing system she set up allows us to recruit WANBAM members as speakers for EAG and other events, meetup hosts, job applicants, etc.
• We've made some improvements in integrating diversity, equity, and belonging intp our hiring process (e.g. adding related skills in job descriptions for public-facing roles, improving sourcing) but still have outstanding work related to improving our assessment skills in these areas and adjusting our scorecard weighting.
• We've been working with Encompass and with CEA leadership to clarify our org-wide long-term goals with respect to diversity, equity, and belonging.
• We updated our page on diversity, equity, and belonging, published an accompanying blog post, and collected resources on racial justice for the Forum and the groups newsletter. Julia Wise gave a talk on related topics.
• We funded Ewelina Tur for pro-bono psychotherapy for 140 free or discounted sessions she provided to EAs over the past year.
• We increased capacity for the future by:
  • Hiring extra capacity to help with the admissions process (~.3FTE / yr)
  • Closing EA Grants (with some grant-making transferred to EA Funds), to free up Nicole Ross's time/focus
  • Contracting and aligning with a communications advisor who can coach EAs before media interviews

Funds

Hiring and strategy

Jonas Vollmer has joined CEA as Head of EA Funds. Jonas was previously Co-Executive Director of the Center on Long-Term Risk (previously the EA Foundation). We were impressed by Jonas' analysis of the strategic options for Funds, and by his plans for active grantmaking (that is, seeking out potential grantees more proactively rather than only granting to people who applied for funding).

If we think that Funds is making good progress, we'll consider spinning off the project in the next year or so. Jonas would report to our board, but continue to be supported operationally by our team (like 80,000 Hours is).

Progress

Although we have been mostly focused on strategy and hiring so far this year, cumulative donations YTD to the four main funds are up about 36% (~$4.5m vs. ~$3.3m). Including pass-through donations (e.g. people donating to MIRI), we're at $5.9m compared to $4.1m last year and $5.6m the year before. Note that donations are very lumpy, so this could partly be random deviation.

Giving What We Can (GWWC)

Hiring

Luke Freeman just joined us to lead GWWC, with a view to spinning off the project from CEA.

Luke is a long-standing GWWC member with a background in startups, digital marketing, and local community building in Australia. We've been impressed by his supportive and welcoming approach to community building, and plans for spreading awareness of the pledge.

If we think that GWWC is making good progress, we'll consider spinning off the project in the next six months or so. Luke would report to our board, but continue to be supported operationally by our team (like 80,000 Hours is).

Progress

New pledge rates have held relatively steady (around 10 per week).

As detailed in the announcement post, Luke is looking for feedback, volunteers, member stories, and introductions, and has been holding some online events for members.

Org-wide updates

Finances

At the mid-year point, we are 36% below budget, due to cancelling in-person events such as EA Global, slower hiring, lower-than-anticipated salary increases, and community-building grantmaking falling more in the second half of the year. Reforecasting our spend to the year end suggests we will be 15% under budget for 2020 overall.

Taking the above into account, we currently have 13 months of runway, providing financial security for CEA until around halfway through 2021.

We’ve been tidying up our financial systems and improving our internal reporting to allow more accurate and faster forecasting. We’ve moved some of our funds to interest-bearing accounts, which yields around $3k/week in interest payments. We also implemented a grant management system, which should allow us to automate more of our grantmaking, increase compliance, and improve monitoring and evaluation.

Strategy

We continue to test and improve our strategy for the organisation, including developing metrics that can track our progress. This is still a work in progress, so we’re not ready to share more publicly at the moment. I’m overall slightly disappointed with the rate of progress here, and I’m aiming to focus more on this in the final six months of the year.

Staff

Morale

I feel that staff morale is now solid and improving. The data support this view, but I think it’s extremely hard to measure this reliably. (The linear regression over the last year is for self-reported morale to increase by an average of 0.71 points per year on a 10-point scale. This could partly be explained by changes in the question we ask staff, or how people were approaching the question, but I think this is at least partly a real effect.)

Retention

Retention is the percentage of staff at the beginning of the period who still work for the org at the end of the period.

For historical reasons, we report this from February to February. Note that retention and turnover figures for 2020 might change over the course of the year.

Conclusion: progress so far

As I discussed above, I think we’re making good progress toward our goals for the year. We plan to review our progress again at the end of the year.

David_Moss @ 2020-08-11T10:54 (+28)

Demographics:  According to Google Analytics, 41% of Forum viewers are female (higher than the proportion of community members who are female), but we believe* that males make up a higher proportion of authors than their proportion in the community...

*Many authors are anonymous, so we aren't certain of this.

 

For what it's worth EA Survey 2019 data suggests this too. Obviously classifying Forum posts directly gives a more comprehensive sample, but as you note has the issue of authors being of indeterminate gender. The chi square tests are, however, not significant for posting on the EA Forum (p=0.052) and writing about EA (not on the EA Forum) (p=0.057), while the others were p<0.001.

Maxdalton @ 2020-08-13T19:01 (+6)

Thanks for sharing this! I found it somewhat surprising that the scale of effect looks like it's bigger for comments vs. posts. (I imagine that the difference in significance is also partly that the sample size for posters is much smaller, so it's harder to reach a significance threshhold.)