EAGxBoston 2022: Retrospective

By Kaleem, OllieBase, Rockwell, Miriam Huerta, Juan Gil @ 2022-07-14T13:09 (+59)

TLDR: We think EAGxBoston was a success. As the largest EAGx ever, the organizers, and CEA staff supporting them, learned lots of valuable lessons about how to plan large(er) EAGxs in the future. When asked “How likely is it that you would recommend EAGx to a friend or colleague with similar interests to your own?”, attendees gave a mean average response of 8.99, with 70.6% of attendees giving >=9/10.

What is the purpose of this post? 

This post is a mixture of advice for future EAGx planners, feedback for the CEA events team from our EAGx planning team, a general recap/update about the success of the event for attendees and non-attendees, and hopefully a catalyst for comments about how we can improve EA conferences in the future^[1]. 

High-level facts and figures

• Date: 1st-3rd of April 2022
• Location: Boston Convention and Exhibition Center, Boston MA, USA.
• 1283 Applications  --> 1005 accepted applications --> 950 registered attendees
• 44 Speakers
• 75 sessions of programming (talks, panels, speed friending, meetups, and office hours)
• 22 Orgs represented at the Career fair

Things we could have done better: 

The main challenge with the planning and execution of EAGxBoston was the short timeline from inception to the weekend of the event, which was around 3 months. This led to organizers immediately needing to put in close to 1FTE planning the conference, vendors having to deal with accelerated timelines, and attendees and speakers having to deal with short timelines and often-delayed communication, both leading up to and during the event. Below is a truncated version of our much longer list which can be found in the full public document. 

Deciding a date for the conference

We took a long time to decide a date for the conference, which caused a cascade of delays, because we couldn't lock-in any vendors, speakers, or attendees as a result. 

Securing a venue sooner

Once we had decided on a conference date, we still took 2-3 weeks longer than  needed deciding which of the venues which we'd secured to go with. 

Swapcard: 

• Launching the Swapcard earlier (maybe a week earlier).
• Offer attendees a training on how to use Swapcard
• Have better training materials for organizers on how to use Swapcard (CEA now has this)

Confirming the schedule earlier

Organising hotel accommodation earlier and better

Registration

• Add more info to badges
• Use barcode scanning at the registration desk to verify registration.
• We didn’t have enough blank attendee/volunteer name badges.

T-shirts 

• Attendee T-shirt
  • Size distribution wasn't good and meant that we didn't have the right sizes for everyone
• Volunteer T-shirt
  • Volunteers didn't like how unwearable they'd be post-conference
  • Size distribution wasn't good and meant that we didn't have the right sizes for everyone

Have (a) volunteer(s) dealing with routine emails earlier 

• We might have lost 25-50% of organizer time answering  emails which required simple answers.

Make sure that all food allergy labels were totally correct, crystal clear

Strategy & goal setting

When planning EAGxBoston, we knew that the majority of potential attendees would be US-based college students. We were relatively confident about this, given that we estimated ~1500 US-based students would have completed an EA intro or in-depth fellowship program in person or online since the last EA conference to happen in the US (2019). Given that we were hoping to have a 1000 person conference, where we’d hopefully have between 60%-75% of attendees being “new” or “inexperienced” EAs (vs 40%-25% “experienced” or mentors or content experts), we had the following goals for the conference: 

1. Major: To maximize the number of meaningful connections made, and interesting ideas heard, by attendees.
2. Major: To make attendees as excited and hyped-up about EA ideas and the EA community as possible
3. Minor: to make attendees consider reducing or eliminating animal products from their diets.

Core team & volunteers^[2]

The organizing team consisted of Kaleem Ahmid and Juan Gil (both based in Boston), Rachel Woodard and Rockwell Schwartz (both based in New York City), and Miriam Huerta (based in New Haven). They were supported by Ollie Base, Ivan Burduk, and other members of the CEA events team (in the UK). In the weeks leading up to the conference, key volunteers included Megan Nelson, Annie Barnett and Mat Allen.

Budgeting, finances, and reimbursements

Budget and costs

The overall cost of the conference was $484,881.86 ($535.78 per attendee). This was more than CEA or the organizers had originally budgeted for, although the original budget was not proposed with ~1000 attendees in mind. Compared to the recent EA Global conferences in London, it seems that the costs of the venue and catering were significantly higher than expected, although they were within industry standards in the city of Boston (they were similar to the costs of all the other quotes we received from venues in the RFP stage of planning). 

Even though this seems like a lot of money for an EA conference, it is important to put the cost into perspective. The mean estimate of the cost of a ticket to a similar event, according to the seasoned conference venue professionals who we worked with, was between $400-800. Most attendees at EAGxBoston didn’t have to pay anything to attend – if we required everyone to buy a ticket, we could have probably reduced the total cost by only ~25%. In addition, if you think about what the counterfactual value of a getting a young person to make a 100% commitment to a high impact career is (lets say ~$1M, conservatively)^[3], the conference would pay for itself if it had that small of change overall – which would be equivalent to ~0.1% of a shift towards a high impact career per attendee, which seems like a likely effect based on the outcomes attendees reported. 

For those planning on hosting an event in the future and are interested in a detailed breakdown of costs, please request viewing permission from Ollie Base here. 

Reimbursements

The process for applying for, approving, sending, and receiving travel reimbursements seems to be largely functional and reliable, but has a lot of built in points of failure. 

Applying: 

• It is clear that many people applying to the conference do not know if they should be applying for financial assistance, or what costs can/cannot be reimbursed. We know this because a large number of people emailed after the conference asking if they could be reimbursed after finding out about reimbursement from someone else, and from people including or excluding items in their cost breakdown (also, a lot of people don’t include a cost breakdown, which probably increases the variance of funding given to people who should receive the same amount of funding – a cost breakdown should either be mandatory in order to receive funding, or we should use a standardized formula to determine funding amounts).

Applications 

Applications for EAGxBoston opened on the 15th of January. 

• The application was structured so that applicants could tick boxes for each conference they were applying for, and use the same submission for each i.e someone would fill in the whole application and then select which of EAGxOxford, EAGxBoston, EAGxPrague, EAG London etc they would like to be considered for.
• Applications were run through Zoho.
• Applications were screened by a team of organizers and volunteers form EAGxBoston and CEA, with each member having to go through a period of calibration training to ensure consistency in decision making.
• Given that EAGxOxford was happening the weekend before EAGxBoston, attendees from The Americas were prioritized for EAGxBoston, whilst applicants from the UK and Europe were encouraged to apply to EAGxOxford.
• Exceptions were made for attendees from the UK and Europe for a number of reasons including community building expertise, subject expertise, or an imminent relocation to the US.
• We estimate ~30 people attended both conferences.
• Applicants from countries who’d need a visa to attend both EAGxBoston and EAGxOxford were encouraged to apply for EAGxOxford, as obtaining a US visa was significantly more expensive and difficult than obtaining a UK visa in the months leading up to EAGxBoston.
• In some cases, applicants were also encouraged to apply for EAGxPrague or EAGxSingapore or EAGxAustralia instead of EAGxBoston or EAGxOxford if it seemed that the distance was especially far.  It would have also been helpful to launch the application sooner.

However, the main issue with the timeline for applications was not the launch date, but rather how potential attendees treated the application deadline. Even with the application being open for ~3 months, the deadline being clearly stated in multiple places, and numerous emails sent to people who’d partially completed their application or had previously attended an EA conference, we still received ~100 applications after it had closed. It is worth noting that the majority of these applications were not from first-time attendees or intro-level EAs. We think this is bad for a number of reasons: 

1. It meant that we did not know the final number of attendees for the event until a few days before the event. This means that vendors who needed quantities (such as catering, merchandise, security) were given rough estimates which had to be overestimations, further raising costs.
2. The week before the event is the most demanding week of the planning process for organizers, and having to process an additional 100 applications during that week probably cost us 0.5-1FTE of organizer time, which could have been avoided if the applications were processed alongside the rest of the applications earlier in the year.
3. This follows a trend in the community where a small subset of regular EA conference attendees has been allowed to get away with applying after the deadline and still been accepted. As a result, there is now an expectation that if you apply after the deadline, or email the organizers (especially if you have a personal connection with them) asking to register when registration is closed, that your request will be granted. We should be striving to be better than this as a community. We would encourage future conference organizers to have a strict policy of not allowing late applications or registrations to EA conferences, in order to establish a norm of applying on time, in order to maintain fairness and equitability in the community and prevent organizers from having to choose between integrity and social cost, and in order to reduce extra and unnecessary strain on organizers. 
    

Creative

We took the aesthetics of the conference quite seriously. The reasons for this were 

1) we wanted to break our community’s habit of placing low value on aesthetics

 2) we wanted our attendees to really like our conference and merchandise so that they’d be happy to show it off afterwards, creating some positive spillover after the conference in the form of brand exposure.

 We designed our own logo, name badges, 2 t-shirts, 2 tote bags, and stickers to give to attendees (all pictured somewhere in this post).We’ve received a lot of positive feedback about the merch offered at EAGxBoston – in particular, the tan and orange t-shirt has been very positively received – people have been seen wearing them at EAG London and EAGxPrague. We have also been asked by at least 100 people who weren’t at the conference if we could send them one, and many group organizers and other EAGx organizers have asked us what programs we used and how we went about designing our merch. We hope we’ve started a trend that future EAG(x) t-shirts will be interesting and fun. 

Subsequently, Marka Ellertson (now of EA t-shirt design fame) has agreed to design the t-shirt for the upcoming EA Global San Francisco!

Survey results

In classic style, attendees were asked to sit in silence and complete the feedback form en masse at the end of the conference. We’ve reported some relevant metrics from those responses below:^[4]

Notes on these results: 

• We had 935 repondants for the first question (which was the first question on the survey), but the response rate dropped off significantly after that, to the point where some questions only had ~50 reponses.
• an LTR of 8.99 is a very high rating - the industry heuristic is that “your best friend’s wedding will have an LTR of 8.0” - so, apparently, attendees of EAGxBoston will be disappointed by their best friend’s wedding not living up to their experience at the conference?!
• An average of 9.61 is close to the number of connections made by attendees at EAG London 2022, and slightly lower than EAG London 2021.
  • We think this is likely high because so many of our attendees had never been to an event before and many had learned about EA during the pandemic, so their ability to make connections over the past 3 years had been stifled.
  • It’s also worth noting that this is a slightly dubious metric, because it is plausible that the more connections someone reports, the less likely they are to actually reach out to any of them in the future (e.g. “Charlotte reported only making 2 connections at EAGxBoston, because she met two people who she made very deep connections with and whom she’s now planning on founding an organization with” vs “Daisy reported making 10 connections because she added 10 people on whatsapp after singing Queen for 2 hours with them at the after party”).
• The percentage of respondents on the survey reporting to be female was very low (26.64%). This was worrying, as initial reactions were that we had done a bad job of encouraging gender equitable attendance. However on inspection of attendees self-reported gender identity in the application reponses, the percentage reporting to be female was ~45%.
  • It is hard to know what to make of this inconsistency, but one possibility is that the sample of attendees responding to the post-conference survey was coincidentally male-skewed (maybe male attendees are more likely to fill in the survey?), or that something about the survey made female attendees stop responding sooner than male attendees. It could also somehow be true that female attendees were less likely to be at the conference at the closing ceremony, or maybe many of the female registrants didn't show up - these seem like unlikely reasons.
• Welcomingness (Q: "How welcome did you feel at EAGxBoston?"), Openness (Q: "How open do you think other attendees of EAGxBoston were to hearing ideas different from theirs?"), Humility (Q: "How humbly do you think other attendees of EAGxBoston expressed their ideas?"), and motivation (Q:"how motivated are you to pursue a high impact project or career because of your attendance at EAGxBoston?") were all measured on 5 point Likert scales, and were all satisfactorily highly scored.
  • We are especially proud of the welcommingness score of 4.56/5 (the joint highest ever score for an EA conference - with EAGxOxford), as this relates to one of our main goals, which was to make first-time attendees feel excited about and included in the EA community.
     

Other topics in the Retrospective doc: 

• Choosing a venue
• Setting up the event space
• Communications
• Content
• Catering
• Lighting
• Signage
• Acoustics
• Decor
• On Naming
  
  All pictures from the conference can be found here! 

Thanks to Justis from The Forum Feedback Service and Ollie Base for their extensive feedback on this post and the longer document. 

1. ^{^}

   Views expressed by the organizing team are made solely in our independent, non-professional capacity – the publication of these views does not imply the endorsement of our employers or CEA’s events team.

2. ^{^}

   Thanks to our volunteers!: Aleksandr Lyzhov, Andie Hansen, Anjay Friedman, Annie Barnett, Anshul Khandelwal, Arjun Durbhakula, Blake Hannagan, Charlotte Darnell, Claire Nyquist, Daniel Scott Mcintosh, Dmitrii (Mit) Troitskii, Elika Somani, Erik Halperin, Evelyn Fitzgerald, Gintare Slauzgalvyte, Grace Hoglund, Harrison Gietz, Isaac Blumenthal, Jade Zaslavsky, Jean Cadet, Jeremy Campbell, Jerod Richards-Walsh, Jess Austin, João Bosco De Lucena, Johanna Schandera, Joshua Preuss, Juan Pablo Ramos, Kathleen Yan, Khalid Ansari, Lauren Mangla, Louise Van, Den Heuvel, Madina Zauzanova, Maya Deutchman, Megan Nelson, Michael Hewlett, Miranda Zhang, Nicholas Seier, Oraya Srimokla, Patryk Jarmakowicz, Philippe Rivet, Robi Rahman, Roshni Dadlani, Sarah Cheng, Tereza Flidrova, Tzu Kit Chan, and Xuan

3. ^{^}

   It's hard to say what the value of a counterfactual high-impact career is (We tried to explain this in this post). However it might be easiest and most conservative to simply just say that the median college graduate might earn ~$1 over their career, and therefore that's the value of their career counterfactually, even if we think that a high impact career might on expectation have a value orders of magnitude greater than that. Its also worth noting that "committing to having a high impact career" has a wide variety of ways to play-out in real life, some of which might not be valuable at all. In addition, there are a whole bunch of other things which might make a conference valuable that aren't "getting students to commit to a high impact career", such as counterfactually taking the GWWC pledge, or starting their own local groups, or supporting future organizing efforts, or spreading the word somewhat about EA, or feeling more positively about EA and spread this somewhat through the broader culture, or  become energised and protected somewhat against burnout/isolation/value drift.

4. ^{^}

   LTR= Likelihood to recommend