German EA Intro Program Report - Summer 2023 (and before)

By Evander H. 🔸, Ruben Heller @ 2023-11-03T17:50 (+21)

For whom might this post be useful?

• Other national or local groups can get inspiration on how to run an EA Intro Program.
• People who want to know more about what is happening regarding Community Building in Germany.

Summary

• Description: What is the history and the size of the Intro Program?
  • The EA Intro Program (formerly Intro Fellowship) has been around since Winter 2020, taking place once per semester. It wants to introduce people to the key ideas of Effective Altruism and connect people with the EA Community (in Germany).
  • The Intro Program started as an independent project and is now facilitated in cooperation with local groups and EA Germany. Active participant numbers over the years: 20, 25, 10, 70, 75, 85. In the latest iterations, there were about 100 applicants each.
• Inspiration: What is unique about the German Intro Program compared to others?
  • Coordinated advertisement in Germany
  • Structure: 4 weeks (advertised), 6 weeks (default), 8 weeks (opt-in)
  • Combining online with in-person
  • Structured 1on1s with moderators
  • LinkedIn certificates, book scholarship
  • Journal as content
    • Less reading than EA Handbook
    • Questions to answer
  • EAD Retreat afterward
• Call to action: How can I copy and adapt the materials of the Intro Program?
  • Official website of the German EA Intro Program: https://www.effektiveraltruismus.de/intro-program 
  • More info: https://bit.ly/ea-intro-program-de 
  • All templates
    • Outreach material
    • Journal (content of the intro program)
    • Discussion document
    • Forms
    • Info sheet (internal coordination)
    • Names and Faces Template
    • How to: Organize Intro Program

Number of participants and moderators

• Participants
  • Total: 107
  • Dropouts in first weeks: 47
  • Finished 8 weeks: 8 
  • Finished 6 weeks: 25 
    • (+ 8 = 33)
  • Finished 4 weeks: 27 
    • (+ 25 + 8 = 60)
  • Attended at least one meetup: 25 
    • (+25 +8 +27 = 85)
• Moderators
  • 17 moderators facilitating
  • 8 online groups
  • 9 in-person groups

Feedback

• We took feedback after each discussion to evaluate and improve the moderator's facilitation in discussion sessions. 
• In addition to that, we took feedback at the end, to get an overall picture of the effect of the Intro Program on the participants and moderators.

Summary 

• Impact
  • We received pretty good overall ratings from both participants and moderators (>8/10 ratings on average), with people rating it on average twice as valuable compared to what they would have done otherwise.
  • Career
    • 92.5% of respondents reported plans of doing career related activities
    • The most mentioned activities are making a 1on1 in the Intro Program, making a copy of the 80 k career planning template and applying to a mentoring by EA Germany
    • Career dedication to doing good: Avg: 4.12/5, but in the application it was even higher, 4.18
  • Other
    • What has the Intro Program impacted the most?
      • Career Avg: 7.27, EA Community/Engagement Avg: 7.22, Donations Avg: 6.90
• Organization
  • We got the feedback that it’s decently organized. Many participants would recommend it (Avg: 8.78/10). We still want to improve the grouping and communication of the intro program besides other things.
• Interpretation
  • There are maybe more impactful community building projects, e.g. targeted outreach, more advanced programs or cause area-specific projects. 
  • Still, the data combined with general reasoning, that it’s good to have such a low-key general offer as the Intro Program to learn more about EA.

1. Participants

40 respondents took the final feedback form. 

General

• How would you rate the Intro Program overall? (1-10)
  • Avg: 8.32
  • Median: 8

• How likely is it that you would recommend the Intro Program to a friend or colleague with similar interests to your own? (1-10)
  • Avg: 8.78
  • Median: 9

• Preparation time: How many minutes did you spend on average preparing for the weekly discussion group? 
  • Avg: 105
  • Median: 90
• Weeks Feedback (range -2 to 2, average)
  • Week 1: 1,38
  • Week 2: 1,24
  • Week 3: 1,13
  • Week 4: 1,18
  • Week 5: 0,78
  • Week 6: 1,47
  • (Week 7: 0,5) <- n=2
  • (Week 8: 1,33) <- n=
     
• Grouping Process: How did you like the grouping process in the beginning? (1-10)
  • Avg: 8.76
  • Median: 10
  • Learning: Was high variance. For most people it worked well, for some not. We will ask in the future iterations already in the application from which days would probably work for the participants.
• Communication: How satisfied were you with the communication in the Intro Program (emails/forms, signal groups)? (1-10)
  • Avg: 8.73
  • Median: 9

• How would you rate your moderator in general? (1-10)
  • Avg: 8.68
  • Median: 9
• General: How valuable was the Intro Program for you compared to what you would have done otherwise? (0.5 = half as valuable, 2 = twice as valuable, …) 
  • Avg: 1.995
  • Median: 2

Impact on participants

Career

• Career: How likely are you to dedicate your career to doing good? (1-5)
  • Avg: 4.12
  • Median: 4
  • Note: We had the same question in the application form: Avg: 4.18; Median: 4

• Career choices: What impact did the Intro Program have on your career choices in the future (e.g., Internships, Bachelor Thesis, Change to high-impact cause etc.)? (1-10)
  • Avg: 7.27
  • Median: 7

• Career: 92.5% of respondents reported plans of doing career related activities such as 
  • Make a copy of the 80000 hours career template
  • Registered for a 1on1 in the Intro Program to discuss your career plans
  • Apply for career advice at EA Germany
  • Apply for career advice at 80000 hours
  • Switch career path in the long term [~ 5 years] to a high impact cause.

• Career: some of the considered career paths are
  • AI safety
  • Research communication
  • Public Policy
  • Mental Health / Facilitation for EA
  • Politics (for different cause areas)
  • Research

Donations

• Donations: What impact did the Intro Program have on your donations in the future (e.g. starting to donate, Trial Pledge)? (1-10)
  • Avg: 6.90
  • Median: 7

• Donation: 80% of the respondents reported planning to do one or more of the following:
  • Regularly donate effectively
  • Take the Giving Pledge
  • Talking with people about Effective Giving

EA Community/Engagement

• EA Community/Engagement: Based on your experience in the Intro Program, how much do you feel like you belong in the effective altruism community? (1-10)
  • Avg: 6.66
  • Median: 7

• EA Community/Engagement: What impact did the Intro Program have on your engagement in the future (e.g., in the EA Community, Party Politics, other)? (1-10)
  • Avg: 7.22
  • Median: 7

• EA Community/Engagement: How motivated do you feel to be more active in the community or do an EA project than before the Intro Program? (0.5 = half as valuable, 2 = twice as valuable, …) 
  • Avg: 1.79
  • Median: 1,5
• EA Community/Engagement: How many additional people in the EA community did you feel comfortable reaching out to after the Intro Program? (1-10)
  • Avg: 4.88
  • Median: 2
  • Quite a few people had 0 as an answer, while some people put down 20+
• Engagement: 90% of respondents reported plans to do at least one of the following in the future or did them during the Intro Program
  • Go to events of my local group
  • Go to an EA retreat
  • Apply to an EAGx or EAG
  • Make an EA forum account
  • Join the EA Germany Slack
  • Start helping to organize EA events

Understanding

• Learnings: 90% of respondents reported having at least one of these experiences
  • I got a deeper understanding of a key idea
  • I made a minor change to my plans
  • I made an important social connection
  • I changed my mind about a key idea

Knowledge Test

We also added some multiple choice knowledge questions into the final feedback form, to check for understanding of key topics of EA. These questions are, to our knowledge, also asked after other Intro Program internationally. All in all, the great majority of respondents showed thorough knowledge about EA key concepts and topics. The questions were as follows:

• What are the three factors that EA often uses when doing cause prioritization? (Answer: Scale, neglectedness, solvability)
  • 38 out of 40 respondents answered correctly (95%)

• Some effective altruists argue that we should strive to be impartial when it comes to doing good, so we avoid privileging the interest of anyone based on arbitrary factors, such as race or gender. They also suggest that we should NOT discriminate against people based on these two factors: (Answer: Where they live and when they are born)
  • 32 out of 40 respondents answered correctly (80%)
  • Common wrong answer was “How intelligent they are and where they live”

• Why does Peter Singer argue that we should give non-human animals moral concern? (Answer: They can experience suffering)
  • 38 out of 40 respondents answered correctly (95%)

• In the Precipice, why does Toby Ord think that 100% of humans dying is so much worse than the case of 99% of humans dying, despite a mere 1% difference? (Answer: Humanity’s future potential would be permanently destroyed)
  • 35 out of 40 respondents answered correctly (87.5%)

• Effective altruism as a social movement is all about… (Answer: Using evidence and careful reasoning to take actions that help others as much as possible)
  • 38 out of 40 respondents answered correctly (95%)

• "In a study, respondents were asked how much they were willing to pay to save migrating birds from drowning in uncovered oil ponds. They responded with the following answers:

- They’re willing to pay $80 to save a total of 2,000 birds

- They’re willing to pay $78 to save a total of 20,000 birds

- They’re willing to pay $88 to save a total of 200,000 birds

Note: they’re NOT saying they’re willing to pay $80, $78, or $88 for one bird each.

Which cognitive bias did they exhibit?" (Answer: Scope insensitivity)

• 36 out of 40 respondents answered correctly (90%)

• Which of the following interventions have a higher expected value? 
  • 36 out of 40 respondents answered correctly (90%)

• What organization within EA specializes in helping people plan high impact career pathways? (Answer: 80,000 Hours)
  • 39 out of 40 respondents answered correctly (97.5%)

2. Moderators

• 7 moderators took the final feedback form, from a total of 15 (not counting 2 organizers who were mods themselves)

General

• Impact estimation: How impactful do you think is the Intro Program?

• How would you rate the Intro Program overall? (1-10)
  • Avg: 8.29

• How likely is it that you would recommend moderating in the Intro Program to a friend or colleague in a similar situation to your own?
  • Avg: 8.14

• Time investment: How many minutes did you spend on average per week on this Intro Program?
  • Avg: 193.58
  • Median: 170 → around 3 hours

• Grouping Process: How did you like the grouping process in the beginning?
  • Avg: 9

• Journal: How much did you like the journal of the Intro program?
  • Avg: 8

• Communication: How did you find the communication between mods/organizer?

Impact on moderators

• Time usage: How valuable was moderating in the Intro Program for you compared to what you would have done otherwise?
  • Avg: 1.37
  • Median: 1.2

• Time usage: What do you think: how well was your time spent?
  •  
  • 1/7 mentioned that he/she could have spent the time probably better → seems alright
• Impact on you: Did you have any of these experiences at the Intro Program?

Other

• We had several participants taking up the initiative to create a career planning group together, where they would hold each other accountable and support each other through feedback on their thoughts & plans. 
• One participant wrote up his career plan and posted it online and in the discussion group to get further feedback on it.
• 1on1s
  • In week 2, participants can apply to speak to a moderator in a 1on1. The participants have to formulate the topic and the person they want to talk to.
  • The theory of change is that through that participants reflect EA stronger in their personal lives and have one contact person for the future.
  • In theory, this sounds good. In practice, it takes the facilitators a total of about 120 minutes (scheduling, preparation, follow-up) and often the contact with the participant diminishes over time.
  • We are not sure if we should continue the 1on1s. Some data regarding evaluating the 1on1s can be found here (n=3).
• EAD Retreat
  • Around 20 people participated afterward in the EAD Retreat. More info about the retreat here.
• Internal organization
  • We used Asana, Google Workspace and Zoom for organizing the Intro Program. Maybe Airtable and Notion could improve some workflows. Currently, we aren’t planning to switch.

Learnings/Improvement

We got a lot of feedback regarding different aspects of the Intro Program. Going forward, we want to implement these points to improve the Intro Program (Again templates are here).

• Promotion
  • Extend promotion to further channels. Have more materials like flyers etc.
  • Decide if there should be an EA Introductory Program newsletter to let interested people know when the next application period is open.
• Application form
  • Have a strict deadline.
  • Ask on which day in the afternoon the applicant has probably time. → Improves likelihood that grouping works.
  • Have an automated mail (add-on in Google forms) after filling in the form. Content: Application received. On [date] you will hear from us latest whether you got accepted. Reminder to attend kickoff. 
• Grouping Process
  • Online groups: building the groups based on when people have time. Ask for weekday availability in the registration form, then assign groups based on that. 
    • Currently, we ask for weekday availability after we assign the groups. This is probably suboptimal from the users' perspective and maybe even from the organizing perspective, as we have to coordinate participants switching groups if they can’t find a time that fits everybody. On the other hand, it is probably computationally easier than hashing out everything before. 
  • Merge shrinking groups sooner to have more lively discussion. This could mean having closer monitoring over attendance in groups. As a start, a group size of around 7 people seems good.
• Moderators
  • Make an optional 90 - 120-minute training session before. Mandatory for new moderators.
  • Decide if there should be 1on1s in the next iteration. Improve the workflow so that it's less time-consuming for the moderators.
• Names and Faces
  • In general, a good thing to have. It wasn’t that often used last time. → Give more nudges to make comments to each other profiles. 
• Journal
  • Design should be less crowded, cleaner
  • Time estimates weren’t realistic (took more time to read) - but the data how much time people spent preparing was fine
  • Terms stated at the beginning should be extractable from the texts provided
  • Implement the first suggestions and ask publicly for more feedback several before the next iteration.
• Discussion Sessions
  • Make the document shorter (e.g. leave out summaries)
  • Make the voting algorithm less complicated
  • Quite a few people asked for longer sessions (~120 min), want to have more time for discussion
  • Build a sense of group cohesion in discussion groups, e.g. with informal check-ins in the beginning and nudges to reach out to other participants even outside the Intro Program
• Communication
  • Have 2 mails in the beginning. Communicate other opportunities e.g. 1on1s registration, EAD Retreat, EAGx application in the discussion groups.
  • Too many channels of information. Improvement: Have one 
  • Too many forms and forms are too long. Streamline and focus on the important ones.
  • Mods: Slack Channel wasn’t used much → Signal group would be better
• Events
  • Don’t offer socials or a giving game. In the last iterations, not that many people [around 1 to 5] showed up and so it wasn’t worth it.
• Documentation
  • Complete the documentation how to organize the Intro Program.
• Diversity & Feedback
  • Add an anonymous feedback form.
  • Improve the internal gathering of feedback and acting on it.
• Final Form
  • Add: Name (optional) and description: In our analysis, we'll compare your answers on the application with your answers on the final form. We will publish the impact and learnings of the Intro Program in an anonymized way.
  • Food basket lottery for filling in the form → too much work, do something else e.g. additional free book, donation to favorite charity or so
• 3 months follow-up
  • Start doing a 3-month follow-up and gather data.

History of the Intro Program

Statistics & History

Remarks

• The first three iterations were a volunteer project only loosely in contact with EA Germany. Probably the impact of these is pretty counterfactual no one else was interested in running Intro Programs in Germany. These iterations were also more selective. Out of memory, roughly 20% of applicants got rejected.
• In summer 2022, it was run less like an independent project and was more integrated in the semester plan of each local group. 
• In winter 2022, the intro program was included in the foundational programs of EA Germany providing financial and organizational support.

This report was written by Evander Hammer and Ruben Heller. All mistakes are our own. We appreciate feedback or questions about the Intro Program in the comments.