Retrospective: CEA’s Fall ‘22 University Group Accelerator Program (UGAP)

By Joris 🔸, jessica_mccurdy🔸, Jake McKinnon, Uni Groups Team @ 2023-02-24T18:38 (+40)

Summary

• From August - December 2022, CEA’s University Groups Team ran the fourth round of UGAP, the University Group Accelerator Program.
• 63 ~new university groups completed the Fall ‘22 (Northern Hemisphere) round of UGAP. Through the program, 112 organizers received mentorship and got access to curated resources. 104 of these organizers also received a stipend.
• UGAP’s Likelihood to Recommend was 9.04 (n=58 organizers), the average rating of mentors was 8.7 (n=56).
• Participants rated mentorship, the UGAP resource guide, and the stipends as the most valuable aspects of UGAP.
• While we are exploring other metrics, we’re quite happy with the number of ~counterfactual HEAs these organizers helped ‘generate’ with their groups, compared to how much time it cost us to run this program. We’re also excited about the quality of a lot of the participating organizers. 
• In the past, UGAP was the main program our team ran. We are likely to continue running it, but as a significantly smaller percentage of our team’s portfolio.
  • Sign up here to be informed when applications open for a next round of UGAP!

About this UGAP cohort

Number of groups in the Fall ‘22 (Northern Hemisphere) UGAP cohort: 63, spread over 21 countries. We had about 112 organizers participate, of which 104 received a stipend.^[1] [2]

Running this cohort (including the retreat) cost CEA’s Uni Groups team about 0.9FTE.^[3]

This round of UGAP was preceded by the UGAP Starter Program, a 2-week program providing organizers with resources and a mentorship call. It had a lower barrier to entry than UGAP. We are not likely to run the starter program again - see here for a full retrospective on the Starter Program.

Feedback from UGAP participants

“How likely would you be to recommend that a new uni group apply for UGAP?”: 9.04 (n=58 organizers, down from a 9.2 rating from the previous cohort (with n=38))

Mentorship

UGAP participants received mentorship on a weekly basis for about 1-3 months, and on a biweekly basis for the remainder of the semester.

• UGAP mentors are experienced university group organizers. Matches were made based on level of experience, areas of strength and our best guess at whether people would get along. 
• Participants rated mentor meetings as the most valuable thing UGAP offers. These are the things that participants highlighted as most valuable about mentorship:
  • Provided feedback on their group strategy
  • Being a sounding board & giving advice on practical problems
  • Providing accountability and emotional support
• Average rating of mentor: 8.7 (n=56)

Other aspects of UGAP

Some things to highlight:

• We are trying to figure out how important stipends actually were. For example: what kinds of organizers reported that stipends were valuable to them? Are these the people we are excited about? 
  • There are a few organizers we are excited about who did explicitly mention stipends but we would like to do more user interviews around this. It seems plausible that stipends should be more opt-in in the future or based on need.
• Some of the templates and tools we had available weren’t used as much as we expected. We deprioritized promoting the ones that people found least useful when launching the current UGAP cohort.
• UGAP provides organizers with a handbook that helps them do outreach on their campus, which a number of organizers highlighted as useful to them.

HEAs

We asked organizers how many of their group members they thought met the definition of HEA. We are quite skeptical of HEA as a good metric.^[4] We also think group organizers are likely to overestimate the number of HEAs in their group, and maybe also whether these people got engaged with EA counterfactually. Yet, we think it’s good to share some results!

• 27 groups of the past UGAP cohort answered the question above. Their organizers self-reported a total of 104 - 119 HEAs in all their groups combined (so an average of 3.9 - 4.4 HEAs per group). 
  • Of those HEAs, group organizers said a total of 55 - 66 became highly engaged EAs this semester (so an average of 2.0 - 2.4 new HEAs per group).
• Since we think group organizers often overestimate the number of HEAs, our very, very rough guess on the number of counterfactual HEAs^[5] is 15 - 25.

It’s unclear how many of these people are counterfactual (see below), but we’re overall quite excited about these numbers!

Fellowships

We asked group organizers to include a quiz in their post-fellowship survey, to check the EA understanding of their fellows. The results were better than baselines, but we still think these numbers could be higher, as the questions asked are fairly easy.

We asked organizers what their semester would’ve looked like if they hadn’t participated in UGAP. Summarizing their answers is hard, as it was an open-ended question. But these are some answers that were given often (20 organizers answered this question):

• Semester would look the same / very similar: 10
• Would still run a fellowship: 11
• Would not have run a fellowship: 6
• Would not be an organizer: 3
• Would have lacked a sense of support / would have felt less confident: almost all respondents

94.4% of the organizers said they’ll host another fellowship at their university. The remaining organizers answered “maybe”; none answered “no” (n=54).

Impact stories

Some impact stories we’re excited about:

• “At least 3 seniors in my group have shared that the fellowship and EA has had a major influence on their career choices.”
• “Two of our fellows and my co-organiser are starting an AI safety hub in [university city]”
• “One of my fellows applied to Charity Entrepreneurship's Incubation Program”
• “We had a group member who went on an AI bootcamp as a result of hearing about it from EA [university].”
• “At least one member is interning with an EA organization in Berkeley, which they only found because their university group mentioned it to them”

Additionally, across all of the UGAP groups of this cohort:^[6]

• Over 800 events were organized in the last 6 months
• Almost 4,000 people attended events in the last year
• Almost 700 people completed the intro fellowship in 2022
• 250 people attended EAG(x)s in 2022

UGAP Retreat

We ran a retreat for 38 of the UGAP participants who we thought could benefit most from attending it. We think the retreat was quite successful. You can find a review here.

Continuing UGAP

Mistakes we made & things to improve

The UGAP Starter Program had a number of problems that we’re addressing in various ways (including not running the Starter Program anymore). See the Starter Program Retrospective.

Other feedback we received, and improvements we’re working on:

• Various more experienced organizers flagged that not all parts of UGAP were helpful for them. 
  • We are piloting OSP, which provides mentorship for more experienced university group organizers.
  • Going forward, we don’t expect to accept more experienced organizers in UGAP.
• Various organizers flagged they want to be in touch with other organizers more, and are looking at what might address this.
• We are continuing to better calibrate our application process.
• We have implemented changes to help preserve valuable mentor time.

Why we want to keep running UGAP

The data collected from this round of UGAP makes us think that we’re providing a valuable service to new group organizers. The main costs of UGAP are a portion of our team’s time (0.9 for this past cohort, 0.6 FTE expected in the future) and mentor time (and the opportunity costs of those), but we think those costs are outweighed by the main benefits of UGAP: we think we found a repeatable program with product-market fit that supports new group organizers, helps identify talent, and contributes to building a global network of university group organizers.

Sign up here to be informed when applications open for a next round of UGAP!

1. ^{^}

   A significant number of participants in this cohort (maybe 20%) were more experienced organizers than the level we normally expect UGAP participants to be at (namely: never organized a group before). Our program was less tailored to these already-existing groups, but we are still happy we accepted them in our program. We are now piloting an Organizer Support Program (OSP), which supports existing groups and their more experienced university group organizers.

2. ^{^}

   The Spring ‘23 cohort of UGAP has significantly fewer groups (27) and organizers (42, 33 with stipend). We expect more people in the Fall ‘23 cohort, but not as many as during the Fall ‘22 semester.

3. ^{^}

   ~2 months of 3 people working on this 0.8 of their time during the admissions & kickoff period + ~4 months of the equivalent of one team member doing mentor calls full time + ~2 months of full-time work of one team member on the UGAP retreat.

4. ^{^}

   We don’t think the HEA definition properly measures what we care about - it doesn’t properly define what significant actions we care about, doesn’t tell us anything about the ‘vibe’ of the group, and different organizers interpret the definition very differently. However, it was the best metric we had at the time (we are looking into alternatives) and it allows us to do some comparisons.

5. ^{^}

   Counterfactual on the group existing, not necessarily of them participating in UGAP.

6. ^{^}

   These numbers are not the main things that we evaluate UGAP or individual groups on, but they are interesting data points. In general, we are wary of (over)optimizing for quantity.