Redesigning the Intro Fellowship: Results, Resources, Reflections, & more

By Sam_E_24 @ 2023-08-22T01:59 (+34)

Special thanks to Sarah Pomeranz, Alex Roman, Jarom Gordon, Sam Battis, Nils Voelker, Henna Dettani, Steve Thompson, Jessica McCurdy, Jesse Rothman, and many more for their feedback and support. Last but not least, thanks to all of the EA Florida students who dedicated their time and effort over the last year.

For community builders with questions/concerns, reach out here. I'm happy to discuss ideas and be helpful wherever I can.

Executive Summary
 

The University of Florida EA group (EAF) piloted a redesigned introductory fellowship, the Problem Solving for Social Impact Certificate (SIC), during the Fall 2022 and Spring 2023 semesters. The eight-week program includes core concepts of effective altruism, career planning support, and professional upskilling. Each session consists of a group discussion as well as a presentation on supplementary EA materials and cross-applicable tools to position students for success in any professional field. EAF aimed to provide more value to participants by selecting a set of materials with an emphasis on practicality and accessibility. 
 

Feedback from the two semesters indicated the program was able to communicate core concepts of EA while achieving some success at upskilling participants. Depending mostly on organizer capacity and the target audience on campus, I believe some university groups should consider running this program or something similar instead of standard intro fellowships.

Abbreviated Meta Views on EA and Community Building

To me, EA is really just about asking questions. Specifically, the most consequential questions in the world. This is why I was drawn to it, why I love it, and why I’m probably not going anywhere any time soon. 

Although you should definitely still have a strategy, community builders at universities shouldn’t be too caught up with how many “EAs” they’re producing. Hitting your KPIs is great, but you should really just focus on  getting students to be more rigorous about their impulse to help others.  

Introduction

Brief Context 

EAF is a student organization I started at the University of Florida as a third-year economics student in January, 2022. I gave five presentations during the first semester: three standalone — to very small audiences — and two co-hosted by other clubs. By the end of the semester, the club basically was still just me and two other people (mandatory to have three people for registration) who didn’t have prior knowledge of EA. This piece is about the club’s main activities for the following two semesters.

Motivation 

Redesigning the program to be more skill-based was largely due to the following beliefs: 

1. EA should and can be more inclusive beyond the most represented demographics (e.g. com sci and philosophy backgrounds) 
2. EA-style thinking and frameworks can be used to increase impact in wide variety of contexts beyond conventional EA cause areas
3. It would be difficult to get a lot of UF students to sign up for a program that required 2 hours of independent prep per week 
4. Engagement would increase if the curriculum was more action-oriented and less theoretical 
5. Longtermism, existential risk, and AI have generally less mass appeal and are not especially useful/relevant topics for the majority of majors 
   1. This was specifically true among the group of students I reached during the previous semester and expected to participate in the fellowship
6. Besides general competence, one’s ability to be successful in lots of non-technical jobs comes down to being good at using a relatively small set of tools, frameworks, and thinking-styles
   1. Colleges are often not teaching these
   2. At least some of these are not prohibitively complex to the extent that 
      1. I could probably teach them at a reasonably adequate level without significant presentation time
      2. Students would actionably benefit from these presentations
   3. Having a rudimentary understanding of these can significantly set you apart from your peers during internships and  entry-level jobs
   4. Early career capital — specifically when students are still undergrads — compounds at a disproportionately high rate

I wanted something to attract more interest in the club and appeal to a greater audience; adjusting the program content and repositioning the program’s marketing around professional development felt like a natural way to do this. 

Broadly speaking, my goals became the following: 

• Facilitate engagement with core EA principles with an emphasis on practicality and how actionable changes can be made in your lives
• Reduce preparation time, somewhat drastically 
• Teach the things I’ve learned, things I wish I knew, and things I would like to learn more about that represent ~20% of the skills you need to do ~80% of generic desk jobs

Theory of Change 
 

Program Development

The revisions and non-EA additions to the original curriculum, created and provided by the Centre for Effective Altruism, were predominantly based on my professional experiences and input from members of the EA Consulting Network (EACN). I also received feedback from members of the university groups team at the Centre for Effective Altruism and university group organizers from different schools.  
 

The presentations at the end of each week were meant to be tangible yet by no means comprehensive; no one can actually solve problems like management consultants from 15 minutes of slides, let alone 15 minutes of slides presented by an undergrad. Knowing that some people would resonate with some ideas more than others, I mainly wanted to introduce topics and provide great materials for people to look at afterwards by creating a new syllabus section: Learning Reinforcement. 

My biggest personal influences on the non-EA content came from participating in Capacity Ventures, a one-month upskilling program run by Training for Good, consulting in Cape Town, and working at a startup in Tel Aviv. The Training for Good program helped participants make weighted factor models for potential career options, which I felt was particularly valuable. 

I looked to consulting for content inspiration because of my prior interest in the field and the strong reputation the industry had for being generally competent. To me, consultants are basically professional problem solvers. I think their work is really quite similar to one of the core projects of EA: they’re good at what they do because they can navigate situations of uncertainty. Bruce Henderson, the founder of Boston Consulting Group, viewed the concept of strategy itself as a phenomenon that sprung from “the ability to foresee the future consequences of present initiatives.” 

So when adding content, I was basically asking the question: what are the most useful tools and frameworks that tangibly elevate the abilities of consultants over those of the people who pay for consultants?

Addressing Failure Modes 

For brevity I will only include what I saw — and what was communicated to me — as the two main failure modes: 

1. Students will get the wrong impression of what EA is and isn’t

I took this quite seriously and ended up implementing a color coding system to designate content affiliations. I felt like I went to great lengths to specify what was and wasn’t from Centre for Effective Altruism or content that was representative of EA as a movement (eventually to the annoyance of my students). Not that I wouldn’t do this again, but it often felt  impractical; it seemed fairly obvious that my efforts were causing an unnecessary level of confusion. 

1. This program will push away more students who only have pure interest in EA ideas, ones who would typically be the most engaged with a theory-based, discussion-only format 

I think this could be true at many schools, but I don’t think University of Florida is one of them. There is really no student involvement that is remotely close to the program I ran; this would still be any student’s best opportunity to explore these ideas with on-campus peers. I could certainly see this program turning away kids who already know a ton about EA, but this is an issue that would be inherent in any fellowship branded as “intro.”

Program Overview 

Structural Differences from Standard University Group Fellowships*

*referring mainly to the 2022 Centre for Effective Altruism intro program as I understood it a year ago. The Arete Fellowship is also quite popular and largely based on CEA’s syllabus, but my understanding is that the Arete program might vary from school to school

** this varied and wasn't always sequential. Presentations often had slides on both EA and non-EA topics

Week-by-week Content Comparison

I tried to arrange for the non-EA topics to have some connection to the weekly EA theme and/or required readings, but there wasn’t a natural match for each week. These pairings ended up also heavily influenced by how discussion-friendly the EA theme was as well as the importance and expected time requirement of the non-EA topic.

Examples of Alternative Content Delivery 

• Although “The Precipice” is one of my favorite books ever, I don’t think it is the most engaging or efficient material for explaining concepts to undergrads. Given my time constraint of promising to keep weekly prep at around 20 minutes, I opted for Nick Bostrom’s Tedx Talk on The Vulnerable World Hypothesis. College students prefer YouTube over dense books and I think it’s a very digestible introduction to the broader topic of existential risk. 
• To illustrate triage during the week 1 session, I showed about 5 minutes of the ESPN short titled “The Man in the Red Bandana,” the story of a young man who helped save as many as 12 people in the South Tower of the World Trade Center.. I then connected his heroism to the historically unique 21st Century circumstance people in wealthy countries find themselves in: most of us can save more lives over the course of our careers by effectively donating a small fraction of our income.
• Students used fermi estimates —specifically the estimation by bounding method — to approximate predictions for how much money they will earn over the course of their careers. I then had them put these figures into this calculator from The Life You Can Save that tells you how much impact you can have with your donations.

Results 

Participation 

• There were two cohorts in the Fall that merged into one and three in the Spring that merged into two (both due to students dropping out, which was expected)
• The cohorts in the Fall were primarily separated based on predicted engagement levels 
• The cohorts in the Spring were primarily separated based on field of study and students’ schedules
  • My availability was also a limiting factor; there were some weeks where I could only do two instead of three presentations

Key Performance Indicators & more

(n=21)

“Test score” refers to a final concepts check of 5 questions with each counting for 1 point. Given it was meant to be quick — the language on the survey was“We're looking for you to briefly show your understanding of a topic rather than write everything you know”— my grading was fairly lenient. I basically just took points off for anything that was incorrect or blatantly missing.

Definition/Notes from 80,000 Hours on “Criteria-based plan changes”

Feedback on the Program Structure

Other Notable Results 

• 1v1s were rated as pretty much the most consistently valuable part of the program. Between prep, call time, and writing personalized follow-up emails, I was typically spending > 90 minutes per student
• Reduced prep time actually did not seem to matter for most people when they were deciding whether or not to apply
• People really liked having the discussion questions up on the screen 

Here's a copy of the full completion survey results 

• Here’s a copy of the weekly feedback I collected for more data on specific readings and receptivity to specific lessons. 

Mistakes 

• Weighted factor model assignment proved to be too complex
  • In the Fall, the most common mistake was putting salary figures where the “estimated value of impact measured by $ of donations to effective charities” was supposed to go. Other than that, I felt like people had a pretty good grasp of the assignment. I still tried to simplify it and wrote what I viewed as clearer instructions for the Spring kids, but they struggled notably more.
    • This might’ve been in part due to the first group being all upperclassman and the second group being mostly underclassman. 
• Going overtime during sessions and having inconsistent session times
  • Especially in the Fall, the session lengths varied pretty widely
  • Sometimes we wouldn’t get to certain discussion questions, and sometimes we barely got to the upskilling slides / skimmed them pointlessly fast
• Dogmatism
  • I sometimes conveyed ideas as if they were more true/certain than they actually are. In some cases this was a delivery issue, and in others it was probably more of not having a deep enough understanding of the topic 
• Neglecting the “community” aspect of the club. My main focus was on program development and content delivery, but I could have had a much better effort to nurture the social dynamics of participating in the fellowship. 
• Not publicly tracking mistakes in the Fall and discontinuing tracking in the Spring. I didn't prioritize it because I got feedback indicating most people probably weren’t going to look at it, but it still would’ve been optimal to have maintained this SIC Mistakes and Corrections 

That list was certainly not exhaustive.

Recommended Curriculum Adjustments 

Structural Changes 

Schedule sessions for 75 minutes. Some sessions might run closer to an hour, but communicate that this is the default setting. Explain it is permissible to leave early at the 1 hour mark if students have prior commitments.

Weekly Topics 

I would replace week 3 with the decision-making workshop I helped create for the EACN’s case competition at Brown University from March of this year. It introduces how a Botec could be done for an EA-relevant problem, incorporating fermi estimates and expected value. These are concepts introduced in week 2, so placing it in week 3 would offer good reinforcement.

I would also replace the forecasting week with an introduction to Theory of Change (ToT) and  monitoring and evaluation (M&E). Week 4’s upskilling topic was forecasting; I mainly had this interactive calibration game from clearer thinking on the agenda, but I ended up never allocating time for it during the sessions. This was fairly intentional/expected; I knew longtermism was going to be a more discussion-heavy week and I was more keen on skipping the calibration game than I was for the other upskilling topics. ToT and how it can be used to map your organization’s M&E is a much more useful and cross-applicable topic. I only had a short required reading on ToT in week 8, so there’s certainly space in the program to spend more time on this. 

I would also replace the week 5 creativity section with a summary of this Spencer Greenberg presentation on how to come up with new ideas. 

The last major change I would make is combining communication with project management into the week 6 upskilling slides, rebranding the theme to  “Critiquing EA + Effective Teamwork.” The content for both of these topics was pretty easy to breeze through, which was probably a sign that some of it could be cut out. I’d be sure to keep top-down communication and separating values and probabilities from Holden Karnofsky’s Bayesian Mindset piece (unless you move the latter to the rationality week, which also seems fine).

WFM Assignment 

I would change the uniform instruction to every student doing this 80,000 Hours guided exercise at the end of the career workshop session (week 7), time capped at around 10 minutes. I did this as a group exercise with one of the Spring cohorts, and I think it was a good way to ease into the task (though it’s worth noting the more advanced students will probably work ahead of the group pace). During the following week, I would take the same approach that I did in the Spring and keep the majority of class time as a workshop for students to try making their models more complex. If you are holding 1v1s earlier in the program, you can introduce the assignment —- especially to the more engaged students — so they can start thinking ahead and ideally ask more advanced questions during the in-person sessions.
 

Reflections and Advice for University Group Organizers

Specific Notes
 

• Prioritize 1v1s
• Tailor your pitch to your audience. 
  • The honors history student should hear, “EA basically started off as a handful of Oxford philosophy nerds who donated 10% of their income to the best charities.”
  • The frat guy should hear, “EA is kinda like Moneyball but for charity.” 
  • Your friends’ parent who works in finance should hear about how two Bridgewater guys quit in order to evaluate charities with the same level of rigor they used to evaluate stocks.
• Soften the “here’s why you should do EA things” spiel by acknowledging that some people may have a comparative advantage in other areas. You can bring it full circle by emphasizing the exploratory, impartial nature of EA and/or scout mindset. 
• It’s very difficult to find the balance between practicing solid epistemic modesty and being compelling. You don’t want to make the mistake of being overconfident and misrepresenting ideas, but you also can’t always be second guessing yourself
• I find that neglectedness is pretty sensible and relevant to bring up when introducing EA (ideally in a compelling but not salesperson-esque manner) to prospective students. However, I also sometimes find that it can be awkward to end on that note because it might imply you think they are wasting their time doing something else to some extent (you might actually think this and you might even be right for doing so). 
• After doing a bit of market segmentation, you might discover your target audience includes people who are members and/or leaders of other philanthropic organizations. Given their passion for the work they currently do, you need to be especially careful to not turn them off or come across as pretentious and superior. Nobody wants to hear that they spend their free time on things that aren’t important. 
• If you’re going to be experimental, own your mistakes. Representing the EA community to the best of your ability matters. You and only you are responsible for getting it right.
  • E.g. since people weren’t fully understanding the WFM assignment, I went into every student’s file after the program formally ended and spent  20+ minutes per student making comments
• Faculty driven outreach is the best way to get applicants 
• Try to host events with guest speakers. The EA community is full of people doing amazing things and most of them are probably happy to talk to your group.
• Consider market segmentation when doing outreach but don’t forget EAs aren’t all cookie cutter; you’re doing it wrong if you’re only looking for philosophy and com sci kids 
• Pursue co-marketing with other organizations
• The highest attendance meetings/presentations I’ve given were all times where I was a guest speaker. Contact the leaders of other pro-social clubs and ask for 10-20 minutes of their time to give an intro talk. Take whatever time they give you.
• It’s hard to predict engagement levels and which ideas will resonate with which people (Echoed by this Yale EA group forum post)
• There will be things you find intuitive that your audience will struggle with 
• Spend time briefly reviewing the most important concepts in the sessions after you first introduce them. 
• Do things that don’t scale.
• Similar to advice given by Kuhan during his appearance on 80k after hours podcast
  • It went something like this: care more about the outcome rather than getting caught up on who’s responsible for what and/or placing blame. 
• This might happen more often than you’d like when you’re starting out but don’t get discouraged. 
• If you visibly go the extra mile to provide as much individualized value as you can, people will notice and appreciate it. 
• Find a balance between letting people flesh out their thoughts during discussion and keeping the session on track. 

I think a lot of us, particularly wide-eyed university organizers just starting out, might not give enough credence to the fact that not everyone has a sufficient propensity for sacrifice and risk taking to actually act on EA principles. For some, I do not think it ultimately matters if it is me in front of the classroom or Will MacAskill himself; if there are eight weeks of sessions or eight months; if a presentation demonstrates practical applications or if it only contains discussion. After all, pivoting from a lifelong career plan to a path you just learned about presents a very real and daunting set of uncertainties.

Should University Group Organizers Run this Program?

It depends. Do you have an organizer with capacity, relevant interests, and reasonably sufficient abilities/experiences (no one is going to be perfectly qualified to do this)? How many programs are you already running? What school do you organize at? What type of students do you want to reach? How many facilitators do you have?
 

Again, feel free to reach out if you want to discuss key cruxes, content decisions, and/or general strategy. 
 

Final Thoughts 

So many people go through college —- and life — without questioning things. They become physical therapists because they loved lacrosse in high school but got sidelined to injury during their senior year. They become orthodontists because braces completely transformed their smile. There’s absolutely nothing wrong with physical therapists or dentists; however, I do think it’s wrong to not ask yourself if the path ahead is the right one. You shouldn’t be upset if a student makes a weighted factor model and orthodontist still comes out on top; you should feel amazing and grateful that you played a role in helping someone think critically about a really important decision.
 

EA isn’t for everyone, and we should be ok with that. Nevertheless, people with a perceived low potential for completely changing their planned allocation of time and resources should not be neglected by EA. A large proportion of the value from community building, and the effective altruism movement itself, comes from normalizing higher levels of sacrifice and shifting the Overton window for what it means to be altruistic. Despite all of the mistakes and mishaps over the last year, I’m proud and incredibly grateful to say I’ve at least done that. 

Resources 

All the materials you would need for running this program plus various templates, marketing content, and more. 

https://drive.google.com/drive/folders/1VRaXZ9wEOP1pH7_m66MXtRPtw6YktxxJ?usp=sharing