Review of EA Global 2016

By Amy Labenz @ 2016-09-20T15:51 (+22)

My name is Amy Labenz, and I was Executive Producer and Curator of EA Global 2016. On behalf of the team, I’m posting to provide the EA community with information about our:

1. Goals for EA Global 2016
2. Decisions and lessons
3. Evaluation of EA Global 2016
4. Plans for EA Global 2017

Summary

What was it?  EA Global 2016 was the largest gathering of the effective altruism community to date. Just over 1,000 attendees joined us for a two and a half day conference at the University of California, Berkeley where we had 3 stages, 3 workshop rooms, and 5 discussion rooms. You can see the complete schedule here.

What were the goals?  We set three goals for EA Global 2016: (1) to create connections and facilitate coordination among people in the effective altruism community, (2) to create an opportunity for people to immerse themselves in EA concepts and effectively communicate complex ideas, and (3) to encourage the development of the EA movement as an intellectual community.

How did it go?  Overall, we think EA Global 2016 was a success. In particular, we were pleased with our ability to encourage new connections between our 1,000 attendees as well as the positive feedback that we received about the program, theme, and production. We did, however, make mistakes, and received some negative feedback as a result, particularly concerning persistent marketing emails, late announcement of the program and dates, late admissions decisions, and last-minute program changes.  

What will we change? Going forward, I will transition to managing a dedicated Events Team full-time and will start preparing for EA Global 2017 much earlier. We intend to engage more with the community throughout the planning process to get input on programming decisions and to help potential attendees evaluate the value of attending. In addition, as part of the CEA merger, we have reallocated staff and have begun making improvements for next year - for example, with the new eaglobal.org.

Goals for EA Global 2016

We selected three main goals, based on an analysis of how EA Global, EAGx, and related events have created value in the past:

1. To create connections and facilitate coordination among people in the effective altruism community,
2. To create an opportunity for people to immerse themselves in EA concepts and effectively communicate complex ideas, and
3. To encourage the development of the EA movement as an intellectual community.

The first two goals were easy to agree upon, but we were more cautious in deciding the third. Goal #3 reflects our view that the EA movement should cultivate an atmosphere of intellectual modesty and self-skepticism. Our reasons for this view are: (1) our expectation that new high-priority cause areas will continue to emerge and that a community-wide focus on cause prioritization remains important, and (2) feedback from community members that previous events had been too focused on specific causes.

Decisions and Lessons

Our goals influenced our decision-making process, particularly on strategic questions.

Programming

To help build connections between attendees, the schedule had many long breaks to allow people to mingle, as well as receptions and an afterparty to facilitate networking. To promote immersive information exchange, we had a large number of workshops and discussion groups on particular topic areas. We recorded all of the talks on the two main stages so that attendees were able to participate in workshops and discussions and did not feel rushed to leave valuable conversations. In addition, our talks and panels investigated research challenges and potential breakthroughs rather than particular organizations. We selected topics to attract experts who could discuss issues such as how EAs can influence policy and how to navigate intellectual disagreements. Finally, we introduced the Effective Altruism Global Research Meeting, which included lightning talks and academic poster presentations to promote intellectual exchange and a collaborative atmosphere.

Our main programming missteps were: (1) we did not post the program on the website early enough and failed to adequately communicate schedule and programming changes at the event, (2) the large number of concurrent activities limited the number of shared experiences at the conference, (3) the workshop rooms were not large enough, so some people who were excited to participate in the most popular workshops were disappointed, and (4) a number of the recorded talks and panels had poor attendance, which was uncomfortable for the speakers. We may want to consider having less complexity in the future, with fewer stages or fewer talks and more workshops with larger workshop spaces. We should also better communicate which talks will be recorded, and may consider recording additional or more varied content. 

Production and Logistics

Based on feedback from previous EA Global events, we tried to improve the production and logistics for EA Global 2016 so that people could better focus on the content of the event. This year, we spent more time working on room layout design, signage to direct people, catering estimates to ensure that there would be sufficient food, and volunteer recruitment, role descriptions, and training so that we could anticipate and resolve issues. In general, we made large improvements over previous years, but we still had issues with poor on-site communication of schedule changes that resulted in confusion and frustration for attendees.

Date and Venue

We decided to hold EA Global in August because: (1) it is a convenient time for students to travel, (2) university facilities and housing are more readily available, and (3) vendors lower prices to compete for business during their off-season. Given our goal of fostering an intellectual community, we thought that these benefits outweighed the fact that some high-value attendees, such as Bay Area VCs, might be less likely to attend at this time. People were generally happy with the venue and reported that its physical layout was one of the things they would like to see again. This layout, which was a large improvement over the sprawling Google venue from 2015, was one of the other main considerations that led us to select UC Berkeley. In addition, working with the EAs of Berkeley allowed us to get large discounts. One challenge of working with the student organization was that it introduced major bureaucratic hurdles, particularly with finalizing contracts with the university and reserving rooms. This caused us to delay our announcement of the event for months, which probably prevented some people from attending, and resulted in disruptive room changes, such as the last-minute changes to the workshop rooms on Friday.

Diversity

We’d like EA Global to help further diversity within the movement by attracting both attendees and speakers from underrepresented groups, particularly in terms of nationality, race, and gender. To recruit speakers, we consulted with the CEA team and leaders in the community to create a master list, which we evaluated on a number of dimensions. We optimized for fit with the program, selected experts in fields that we believe to be high value, and prioritized speakers based on their familiarity with effective altruism, their name recognition, and our personal connection. As part of that process, we made specific efforts to invite people from traditionally underrepresented groups. In addition, when building the program we made sure there were no all-male panels (we did have one panel entirely composed of women - this was not by design, but we noticed it with time to make adjustments and decided not to). We made significant improvements to our process over EA Global 2015; however, we’re not satisfied with our progress here and would like to do better in the future. We hope that starting the speaker selection process earlier will give us more opportunity to find speakers who better represent the diversity that we’d like to see in the movement.

Speakers

Overall, we did not optimize for attracting “big name” speakers, though we did invite some in that category. We found in the past that headline-making speakers can result in inaccurate perceptions outside of the community. For example, at EA Global 2015, Elon Musk participated in an AI side event, but because of his status and the team’s decision to feature him prominently in some event marketing, he was seen as the face of EA Global 2015. This contributed heavily to the impression that EA Global 2105 was focussed on technology and AI risk, even though this was not the intention of the organizing team. With more lead time and careful presentation of featured speakers and topics, we think it will be possible to select for “big name” speakers that also fit well with conference themes. This year, a number of speakers in this category were interested but had scheduling conflicts. Next year we will start the invitation process earlier. If you would like to nominate a speaker, please do so here.

This year we had ~80 speakers. Alison Woodman, our Speaker Liaison Team Leader, managed speaker communications before the event and helped train volunteers on the Speaker Liaison Team.  Each speaker had a Speaker Liaison to escort them to their stages and to make sure they were comfortable and understood the event layout. We also hosted a speaker reception where we connected speakers with one another and with EA organizations, and we expect to see friendships and collaborations as a result. Generally, our speaker interactions went well, and we have received positive feedback from many speakers. Unfortunately, we had communication delays, particularly about panel descriptions and lightning talk acceptances, which were likely stressful for the speakers. This was largely caused by a process mistake where we had only one employee drafting the descriptions and making decisions about lightning talks. Once we invited additional teammates to approve talk applications and write panel descriptions, things went more smoothly, but because of delays in making that call we probably appeared less professional than we would have liked.

Admissions and Marketing

We have an admissions process because we want to select for people that will get the most value out of the conference and who will contribute to the positive experiences of other attendees. We look for altruistic people who are curious and have an analytic mindset and try to screen out people who have histories of disruptive behavior. This year we received 2,152 applications for a target conference size of 1,000 people, which we successfully achieved. The application process, however, did have several problems: (1) conversion on our applications was lower than expected, and as a result, we did a push for additional applications that was seen as overly persistent or misleading (See Kerry Vaughan’s Marketing post), (2) spelling mistakes made some of the pre-event communications appear unprofessional, (3) some questions in the application may have been off-putting or discouraging, and (4) promotions, such as allowing some attendees to bring a friend, created inconsistent admissions standards. We plan to address these issues by opening applications earlier and making more use of waitlisting next year. This will allow for reconsideration of those who expressed early interest and will give us more time to build promotions that are consistent with our admissions standards. We also plan to have a focus group led by our Community Director, Julia Wise, in order to check the application for tone. Julia will also review the pre-event communications and marketing plan to make sure they do not clash with the intellectual and professional goals of the event.

Location

Our biggest considerations when determining where to hold EA Global 2016 were the convenience for and cost to the community. We considered (1) where most EAs that would like to attend are located (based on the map of Effective Altruists on the EA Hub, and the results of the most recent EA survey), (2) the strength of the existing communities in the various locations, (3) the cost of flights for the EA community, and (4) the location of EA organizations. Based on our analysis, we decided on the Bay Area. Our delayed release of the location likely prevented a number of Europe-based EAs from making the trip, and we plan to make a decision much earlier this year. Please let us know your location preferences here.

Food

Food was the largest single budget item for the conference; we spent approximately $100 / person for the weekend. We decided to have food on-site for the opening and closing receptions and for lunches because we wanted to create opportunities for interaction between conference goers. After discussions internally and on Facebook, we chose a vegetarian and vegan menu. We did this acknowledging that the community contains many who do not believe that personal diet changes are an effective way to reduce suffering, and also many who believe that a meatless menu is a basic gesture of respect to animal advocates. We included a restaurant guide in the program with the hope that nearby restaurants could provide further options if people found that our catering didn’t meet their needs. Food is always a touchy subject at conferences, and our survey results showed mixed responses on the meat question but trended positive in terms of overall food satisfaction.

Finances

CEA leadership set three high-level financial goals for the event: (1) use resources wisely, in keeping with our community values, while ensuring that attendees could focus on the content in a pleasant, stress-free environment, (2) keep tickets as affordable as possible while making scholarships available to applicants who could not have attended without assistance, and (3) roughly break even on the event. In the end, we were quite successful in achieving these goals. The total budget for the event was ~$250K (excluding the cost of staff time and travel). We estimate that CEA will spend $20K on costs not covered by ticket purchases and sponsorships (~$230K). We are still waiting on a full accounting from our bookkeeper and will provide an update when we have the final numbers. Here is a rough summary:

Note: financial aid and scholarships were our largest source of foregone revenue, however, we did not account for them as a “cost” in our spending overview. Students were given automatic discounts of $100 off, and everyone who applied for financial aid received some financial assistance. We heard substantial negative feedback about the posted ticket price, which suggests that we could have better communicated our financial aid policy. We instituted a pricing strategy where the ticket price was considerably higher than the price we expected the average attendee to pay, based on our estimates of the number of people who would need scholarships. It seems that the current pricing strategy might discourage value-aligned people who need financial assistance but are hesitant to ask for it. We plan to consider alternative pricing strategies and are open to suggestions. We will also consider adding more financial aid to our budget for next year.  

Evaluation of EA Global 2016

EA Global takes thousands of hours of EA time, so it is important that we earnestly assess how we did. We collected data from two surveys: (1) Closing Survey: 255 people responded to a 5-minute survey after the closing talk, (2) Follow-up Survey: 92 people responded to a survey that we sent a week after the event. The surveys were a mix of free-form responses and numerical ratings. (See the links for summaries and Oliver Habryka’s upcoming post for more detail).

On our Closing Survey, attendee satisfaction with the conference overall got the highest rating, with a score of 6.1 out of 7 (displayed as 2.1 below). Similarly, our Follow-up Survey results showed a 4.1 out of 5 overall satisfaction rating.

In our Follow-up Survey, we asked attendees to rate how much different parts of the event were issues for them. The mean was 1.5 on a 4 point scale, suggesting that none of these were terribly bothersome on average; however, of the issues listed, the clarity of the program, logistics communication, and the ticket price appear to have been the biggest concerns for attendees. This matches the feedback that we have received in-person and online.  

In our Closing Survey, we saw a significant impact as measured by the number of attendees who reported having changed their minds on particular topics. A large majority of people surveyed listed at least one area where they changed their minds (though this result may have been a consequence of the survey design).

In addition, in response to our Follow-up Survey, 14% of the ~90 respondents said that they changed their life plans significantly as a result of EA Global, 27% said that EA Global changed how much they intend to give to effective charities, and 5% of people reported that they took the Giving What We Can Pledge as a result of EA Global. We cannot simply extrapolate this proportion to all attendees of the event since there are likely selection effects at work (those responding to the Follow-up Survey may have been more moved by the event). It appears that 23 attendees signed the GWWC Pledge at EA Global or in the weeks after the event and 8 more signed up for Try Giving (only 3 cited the conference as how they first heard about GWWC, though this required a write-in answer, which may have resulted in underreporting).

We expect to see additional sources of value in the months to come, but based on attendee and EA organization feedback, as well as our own conversations with potential collaborators, potential hires, and new donors, we think that EA Global 2016 was a valuable use of CEA team time. On the margin, we might have saved time by outsourcing more of the work, or by limiting the complexity and/or size of the event. Given that we received feedback suggesting that people would like to see more facilitated interactions & networking and relatively fewer panels & interviews, CEA might be able to plan a version of EA Global with less curated content and more workshops, which would demand less staff time while better satisfying attendee preferences. 

The Communications and Outreach Team is currently working on improved methods for evaluating CEA projects against one another, but this is beyond the scope of this particular post.

Plans for EA Global 2017

For EA Global 2017, we will start the planning process earlier and invite more community participation throughout. We aim to start planning EA Global 2017 this month, with a goal of selecting a location and identifying keynote speakers in the next month, and working from there to select a date. If you are interested in contributing content or in having your organization represented at the event, please use our preliminary workshop submission form here.

To build on the success of EA Global 2016 and shore up our weak spots, I am transitioning to running the Events Team full-time, which will include EA Global, EAGx, and additional in-person outreach. The team will include Roxanne Heston, Julia Wise, and another teammate (description  here, apply here), and will receive support from the rest of the Communications and Outreach Team, led by Tara Mac Aulay. We will bring on a new teammate to manage CEA US Operations (description here, apply here). As a result of the new team structure, we have already started making improvements to our process for next year. For example, we have a new website, thanks to Sam Deere. With the new system, many staff members can edit the website and push changes quickly. This will allow us to share information about EA Global, including information about the program, speakers, and schedule, without having a single bottleneck.

Please visit the new website to see the videos and pictures from this year, and to get information about EAGx events and EA Global 2017. I appreciate all of the feedback that we have gotten so far and I look forward to EA Global 2017! If you have feedback but haven’t had a chance to share it yet, please do so in the comments or email us at hello@eaglobal.org.

Thank you!

Thank you to the attendees, speakers, staff, and volunteers who helped make EA Global 2016 possible. Special thanks to: Oliver Habryka (Director of Programming) and Julia Wise (Community Director) for help with this post. Thanks also to the rest of the EA Global team: Alison Woodman (Speaker Liaison Team Leader), Peter Buckley (Assistant Producer), Kerry Vaughan (Director of Marketing), Larissa Rowe (Marketing and Social Media), Tara Mac Aulay (COO of CEA), Will MacAskill (CEO of CEA), Anjali Gopal (Volunteer Team Leader), and our 50  volunteers. Also, thank you to our MCs: Nathan Labenz, Roxanne Heston, and Jonathan Courtney, and to Tessa Alexanian for helping me and others with our slides, Andrew Lapinski-Barker for creating the signs for the event, Katherine Xiang for designing the programs, Michelle Hutchinson for organizing the Research Meeting, and the many Pareto Fellows who joined in to help.