CEA update: Q1 2021

By MaxDalton @ 2021-04-22T05:36 (+93)

This is a linkpost to https://www.centreforeffectivealtruism.org/blog/cea-update-q1-2021

This update covers CEA's work in the first quarter of 2021.

Background

Our mission is to build a community of students and professionals acting on EA principles, by creating and sustaining high-quality discussion spaces.

In 2019, we focused on stabilizing the organization and improving execution. In 2020, we clarified and narrowed our scope (by setting strategy and spinning off Funds and GWWC).

In 2021, we are focused on working towards our annual goals, as well as growing our team.

Program progress

These are brief summaries; you can find more details for each program further down in this post.

Groups

Forum

Events (EA Global: Reconnect)

Community health

Expanding capacity

My (Max’s) main focus in 2021 is on hiring.

Reasons for this:

In Q1 we opened two rounds, for a finance lead and a full-stack engineer. We also began to redraft our careers page and made some improvements to our hiring process. We’re on track to open more rounds in Q2.

Other progress

Program updates

Groups

We help local group organizers by advising them, providing resources, and creating online spaces where they can share resources and support each other.

Support and advice

Fellowships

Enhanced support

We’re testing out providing additional support for specific populations within EA to see if we can foster better growth and retention there.

Other

Much of our work with groups happens in collaboration with group leaders, and some of this impact should be attributed to them.

You can find more details about this work in the appendix.

Forum/online

The Effective Altruism Forum (EA Forum) aims to be the central place for discussion about how to do the most good.

Progress

Key progress

Other progress

Technical progress has been slower than expected. We are in the process of hiring another developer. (The application period is over; we are currently conducting interviews.)

Metrics

Q1 update: Forum metrics

This shows the number of views on the EA Forum by logged-in users. Each day's data point represents the average of the previous 30 days. We believe the spike in mid-2020 was largely the result of an issue where bots created false accounts before we made a technical fix.

Q1 update: Content projects

This shows engagement for a variety of content projects over the last year.

Events

Events enable attendees to make new connections, learn about core concepts, share and discuss new research, and coordinate on projects.

EA Global: Reconnect

This quarter was spent planning and executing EA Global: Reconnect. This event was focused on increasing the numbers of connections between highly-engaged community members. We have not yet deeply analyzed the data, so we are sharing our initial impressions.

New ideas we tested:

Outcomes

We did not hit the majority of the goals we set for the event, and we’re a bit disappointed in the outcomes, although we still think it was worth doing. We’ll reflect on this over the next month. While we’ve learned a lot about virtual events this year, we hope this will be one of the last events in which “Zoom fatigue” will be a major factor.

We also began to process applications for teams organizing EAGx conferences, and we aim to support more EAGx conferences this year than last.

Community health

The Community Health team aims to preserve paths to value and mitigate key risks to the future of the EA community. Our work includes fostering a healthy culture, improving diversity, mitigating harm done by risky actors, reducing the harm of negative PR, and identifying risks to early field-building.

Reactive work

We handled:

Examples of different cases from this quarter:

Proactive work

Brand

Diversity, equity, and inclusion

Culture

Epistemics

Reflections

Operations

The Operations team provides support across finance, legal, HR, grantmaking, office management, cybersecurity, and fundraising. This support enables CEA, 80,000 Hours, Forethought, EA Funds, and Giving What We Can to run efficiently.

Progress this quarter

Finance lead hire

Office refurbishment

Customer Relationship Management (CRM) software

HR and cybersecurity

Other

Morale and retention

Q1 update: Staff morale

Average morale in Q1 was 6.45/10, which represents a slight drop from the previous quarter (6.61). Some of this may be due to a change in how we’re capturing this morale data, as we’ve shifted from using 15Five to using a weekly Slack poll.

Louis Dixon (finance lead) decided to leave to pursue study in computer science. We’re grateful that he gave us ample notice, and optimistic about finding a solid replacement.

Appendix: Additional groups information

Support

Advice

Resources

We’ve seen a gap between students completing an Introductory Fellowship and actually taking significant actions based on EA principles. As a result, we’ve supported / run the following activities:

Seeding new groups

Funding

We continued to provide funding for group events and initiatives, at a similar level to last year.

Fellowships

Q1 update: Fellowship metrics

Virtual programs

This quarter, Emma Abele, a contractor and EA Brown leader, launched EA Virtual Programs. Over 600 participants and 100 facilitators took part.

Group-run fellowships

Community Building Grants

Community Building Grants (CBGs) allow group organizers to professionally engage in local community building activities.

Our main priority for this quarter was evaluating renewal applications for funding. Harri has decided to transition out of his role as Community Building Grants specialist. Over the next quarter, he will be exploring other possible roles at CEA and externally, and will continue to maintain the program until we hire additional team members.

We previously expected to reopen new applications for funding in January. We decided against this due to low capacity. We made a mistake in not communicating our timeline for deciding whether to re-open applications before the decision had been made. We have now made a public commitment to provide an update about our plans by June 1st, in advance of the June 13th EA Infrastructure Fund round.

In Q1, we granted ~$420,000, which was broadly on track with our expectations.

Renewal city / national grants made:

New grants made:

Group health cases

26 independent group-related community health issues were raised, and we spent significant time on four.[8]

Categories of cases (some of which fit into more than one category):


  1. The Introductory Fellowship (which will soon be referred to as the Introductory Program) is an eight-week program meant to introduce new group members to some of the core ideas in effective altruism. Dozens of groups have run this program for hundreds of people, but the content hadn’t been available in a publicly accessible format until now. ↩︎

  2. People from Black and Hispanic/Latinx backgrounds are especially underrepresented in the community. We are prioritizing engagement of people from these backgrounds, based on the hypothesis that they may feel particularly isolated. We typically mean people who are living in places where they are in the minority, such as Black and Hispanic/Latinx people in the US or UK. ↩︎

  3. This was proposed and driven by student leaders at Brown and Harvard. It might have existed without CEA funding, but we think funding helped organizers have enough time to actually run the project. ↩︎

  4. Topics include: group history, how boards/committees are structured, current activities they run, open strategic questions, etc. ↩︎

  5. As some students leave fellowships before finishing, and fellowships are run independently through groups, our estimates of the number of fellowships in Q4 and participants across Q1 / Q4 are somewhat uncertain. ↩︎

  6. 7 support calls, 2 meetings through EA Global, and 6 longer email correspondences. ↩︎

  7. Brown is not one of our focus universities, but we believe that many of our focus university groups can learn from the Brown group’s innovative practices. ↩︎

  8. By “significant time”, we mean interfacing closely with the Community Health team and holding at least one in-depth call with the organizer(s) involved. ↩︎


Peter_Hurford @ 2021-04-22T07:01 (+29)

This is really cool to see! Thanks for sharing this level of detail and transparency from one of the most important EA organizations!

BrianTan @ 2021-04-22T07:04 (+21)

This update and CEA's plans for 2021 mention the term "highly-ranked university groups" and "focus universities" a lot. Could you clarify what you mean by one or both of those terms? (i.e. are you looking at the top 40 universities globally based on a specific website)? Thanks!

MaxDalton @ 2021-04-23T07:58 (+8)

These terms are generally referring to 19 university groups which we give some additional support (e.g. we offer extra 1:1 calls with them, and we pilot some programs with them). This is on top of the support we offer all groups (e.g. online resources, funding for events, 1:1 calls, advice over Slack/email).

The groups are chosen primarily based on the university’s track record of having highly influential graduates (e.g. Nobel prize winners, politicians, major philanthropists). We also place some weight on university rankings, universities in regions with rapidly-growing global influence, the group’s track record, and leader quality. 

Current focus university groups in no particular order: Harvard, Swarthmore, Oxford, London School of Economics (LSE), Cambridge, Georgetown, Stanford, Hong Kong University, Yale, Princeton, MIT, Caltech, Berkeley, University of Chicago, Columbia, Penn.

BrianTan @ 2021-04-23T10:16 (+2)

Got it! You listed only 16 universities though, but you mentioned you're referring to 19 groups. Do some universities have more than one group (i.e. Harvard and Harvard Law?) 

MaxDalton @ 2021-04-23T12:46 (+4)

Yes, that's right.

Khorton @ 2021-04-22T12:56 (+13)

"We have ... improved our cybersecurity, and streamlined a number of HR systems."

Hurray! Well done

MaxDalton @ 2021-04-23T16:26 (+2)

Thanks - I'll pass this on to the people involved! 

Habryka @ 2021-04-22T07:09 (+13)

Thank you for posting this!

Harry_Taussig @ 2021-04-23T21:24 (+12)

Are you able to reveal who this YouTube creator is? I'm surprised by how little EA YouTube content there is aside from recorded talks. I feel like an EA-related Veritasium or Kurzgesagt could be super helpful and popular as per this post.

sky @ 2021-04-26T17:23 (+2)

Thanks for asking. I’m not able to say more at this point about that specific creator. 
I think you’re asking a good, implied question I share though: which comms channels would be most promising, for creating or sharing additional EA content?
I’m interested in analysis of those sorts of questions, and see them as part of the strategic comms role we’re hoping to hire for this year.
(I work at CEA).

james_aung @ 2021-04-22T17:20 (+11)

Would you be able to provide a Net Promoter Score analysis of your Likelihood to Recommend metrics? I find NPS yields different, interesting information from an averaged LTR and should be very straightforward to compute.

MaxDalton @ 2021-04-23T16:22 (+15)

Sure! I've asked the relevant people to respond with the NPS figures if it's quick/easy for them to do so, but they might prioritize other things.

Btw, I disagree about how useful NPS is. I think it's quite a weird metric (with very strong threshold effects between 6/7 and 8/9, and no discrimination between a 6 and a 1). That's why we switched to the mean. I do think that looking at a histogram is often useful though- in most cases the mean doesn't give you a strong sense of the distribution.

james_aung @ 2021-04-24T18:02 (+9)

Thanks! I guess I think NPS is useful precisely because of those threshold effects, but agree not sure that it handles the discrimination between 6 and 1 well. Histograms seem great!

MaxDalton @ 2021-04-25T10:00 (+17)

Hmm, I still think the threshold effects are kinda weird, and so NPS shouldn't be the main measure. (I know you're just asking for it as supplementary info, and I think we'd maybe both prefer mean + histogram.)

There's a prima facie case, that's like: the threshold effects say that you care totally about the 6/7 and 8/9 boundaries, and not-at-all about the 5/6, 7/8, 9/10 boundaries. That's weird!

I could imagine a view that's like "it's really important to have enthusiastic promoters because they help spread the word about your product" or something, but then why would that view want you to care not-at-all about the 9/10 boundary? I imagine 10s are more enthusiastic promoters, and it seems plausible to me that the 9/10 differential is the same or greater than the 8/9 differential. 

And why would it want you to care not-at-all about the 7/8 boundary? I imagine 8s could be enthusiastic promoters, more so than 7s.

 And similar comments for a view that's like "it's really important to avoid having detractors, because they put people off".

I could also imagine a kinda startup-y view that's like "it's really important to get excellent product market fit, which means focusing on getting some people to really love your product, rather than a large group of people to like it". But on that view,  why ignore the 9/10 boundary? And why care about detractors?

I also think that maybe all of the above views make more sense when your aim is to predict whether your product will grow virally (not our focus), vs. whether it's generally high quality/providing something that people want (more our focus). So they might just not carry over well to our case.

james_aung @ 2021-04-25T20:43 (+5)

Thanks for explaining your view! I don’t really have super strong views here, so don’t want to labour the point, but just thought I’d share my intuition for where I’m coming from. For me it makes sense to have a thresholds at the places because it does actually carve up the buckets of reactions better than the linear scale suggests.

For example, some people feel weird rating something really low and so they “express dislike” by rating it 6/10. So to me the lowest scorers and the 6/10ers are actually probably have more similar experiences than their linear score suggests. I claim this is driven by weird habits/something psychological of how people are used to rating things.

I think there’s a similar thing at the 7/8/9 distinction. I think when people think something is “okay” they just rate it 7/10. But when someone is actually impressed by something they rate it 9/10, which is only 2 points more but actually captures a quite different sentiment. From experience also I’ve noticed some people use 9/10 in place of 10/10 because they just never give anything 10/10 (e.g they understand what it means for something to be 10/10 differently to others)

The short of it is that I claim people don’t seem to use the linear scale as an actual linear scale , and so it makes sense to normalise things with the thresholds, and I claim that the thresholds are at the right place mostly just from my (very limited) experience

MaxDalton @ 2021-04-26T05:26 (+2)

Thanks for explaining! The guess about how people use the scale seems pretty plausible to me.

MaxDalton @ 2021-04-23T16:24 (+5)

EA Global: Reconnect NPS was 20%

MaxDalton @ 2021-04-26T05:53 (+2)

For groups support calls, one staff member's NPS was 83% and another's was 55%. (They were talking to different user groups, which probably explains some of the discrepancy.)

David_Kristoffersson @ 2021-04-22T12:15 (+7)

Thanks for posting this. I find it quite useful to get an overview of how the EA community is being managed and developed.

BrianTan @ 2021-04-22T07:03 (+7)

Thanks for writing and publishing this! Lots of exciting progress. I have some questions, which I'll separate into different comments:

Group model write-ups: James Aung coordinated organizers at CBG-funded university groups (Oxford, Cambridge, Harvard, Yale, Stanford, and Brown) to write up documents explaining how their groups work.

Is it possible to get links to these documents? I and other student chapter leaders in EA Philippines would be interested to read them, and I think other student group organizers would be interested too. In particular, we think a lot about what the best organizational structure is for uni groups, and what strategies to use for a) goal-setting, b) deciding what projects to run, and c) dividing roles and responsibilities.

james_aung @ 2021-04-22T09:32 (+5)

Hey Brian. I'd have to ask the individuals who wrote up their docs, but the plan is definitely to eventually share more of these type of group writeups widely. They weren't written with a broad audience in mind, but I feel like several leaders would be keen to share their writeups more publicly after cleaning them up a bit. I'll nudge people on this and ask if they're keen

BrianTan @ 2021-04-22T10:28 (+2)

Got it, thanks! If they could be compiled and put on the EA Hub Resources, such as on this page, that would probably be the best place to compile them?

BrianTan @ 2021-04-22T07:28 (+6)

On Fellowship Data:

  1. If a fellowship starts in Feb but ends in April, does that count toward Q1 data or Q2 data? 
  2. Regarding the number of participants for fellowships, I'm not sure how that data is collected, but maybe Marie Buhl reached out directly privately to fellowship organizers to collect this data. Anyway, there's a good chance that data for EA Philippines's chapters aren't accurately included in Q1 data.

    For example, 2 of our student chapters in EA Philippines,  EA UP Diliman and EA Blue, are both running intro fellowships since Feb and March respectively. They have a combined 67 participants (36 and 31 respectively), though a few have already dropped out or are not set to graduate. I assume both universities are non-focus universities. So if our participant data is part of your Q1 data set, then that means 29% of fellows (67/230) from non-focus universities are from our universities. I think that's too high, so my assumption would be most or all of their fellows are not yet in the data set.
  3. Related to #2, I think it would be useful to have a public Google Sheet with rows for which groups are running fellowships, and columns for what country and university they're from, when they've started, when they will end, how many participants, and how many graduates (if data exists on that already). I assume most fellowship organizers would be okay with this data being public. I think having this public Google Sheet can let us easily know which groups are and aren't on the list yet. I think it's also good for others to know which universities/cities have fellowships - maybe they can recommend friends from those universities/cities to join those fellowships.
  4. I think it would be good to include data for both participants and graduates. Would be interesting to know what the avg. drop-off rate is for these fellowships.
  5. Also, a separate Google Sheet that lists which groups are going to run fellowships or reading groups in the next 1-3 months (and whether they accept people virtually or not) could also help people interested to know what fellowships or reading groups are coming up. But I think this is less important to do than the other things I suggested or mentioned above.
JoanGass @ 2021-04-23T17:11 (+4)

Hi Brian, 

Great to hear about your enthusiasm for fellowships! 

  1. Q1 data - we count fellowships based on when they start
  2. We collect data on this at the end of each fellowship, so the non-EAVP participant numbers in the report are guesses based on Marie’s conversations with group leaders. For groups Huw was not in touch with on a regular basis, Marie assumed an average of around 12 participants/group, so it’s possible that the number is higher based on EA Philippines numbers (although a number of fellowships are smaller than 12 participants).
  3. Marie is planning to make a spreadsheet like this from next quarter and will post it on the EA Groups Slack.
  4. We’re collecting data on both starting and finishing numbers of participants at the end of each fellowship.
  5. Marie is planning to include future fellowships in the spreadsheet. Adding a tab for other reading groups seems like a good idea. 

Feel free to reach out to Marie directly on the EA Groups slack if you’d like to discuss more

BrianTan @ 2021-04-24T12:51 (+4)

On #1 and 2: Got it! I guess CEA should be more cautious (i.e. by putting significant caveats or not reporting the data yet) then about reporting participant data for non-EA Virtual Programs participants, since you collect data at the end of fellowships, and the data are just guesses before then.

On #3-5: Great!

JoanGass @ 2021-04-26T11:34 (+3)

On #1 and #2: In our report in footnote 5 where we reported this data we said:  "As some students leave fellowships before finishing, and fellowships are run independently through groups, our estimates of the number of fellowships in Q4 and participants across Q1 / Q4 are somewhat uncertain."

I do think the benefits of reporting estimates are more valuable than only reporting precise information, but we do try to add additional detail about where the uncertainty comes from. I'll keep this comment in mind when we do our Q2 report as well. 

BrianTan @ 2021-04-26T15:03 (+2)

Yeah estimates are probably better than nothing. Maybe making the caveat/uncertainties about the data more easily seen, i.e. as asterisks beside some numbers on the fellowship data table, rather than as a footnote might help. But yeah it's a minor thing!

Adam Binks @ 2021-04-25T23:22 (+3)

Thanks for this, really interesting! I am surprised that the total attendance of fellowships isn't even higher - do you have a feel for whether they're typically constrained by mentors or signups? In my experience helping run fellowships, many people are surprisingly interested but haven't heard about EA, have you looked at ways to reach more of these people?

JoanGass @ 2021-04-26T15:36 (+2)

For the large EA Virtual Program round, at first we were worried about having enough facilitators. But then we actually had quite a number of volunteer facilitators (over 100!) so then we focused on getting more participants. In the end we ended up having participant demand that matched available facilitators. As we mentioned, we're working to build more operations capacity for the virtual programs version of our fellowship. Once we do this, we hope to be able to offer them on a more consistent basis so more people can sign up.