EAGxBerkeley 2016 Retrospective

By Rohin Shah @ 2016-09-11T06:27 (+18)

(Note: All EA Berkeley retrospectives can be found here. This post includes the summary and some selected sections of our EAGxBerkeley 2016 retrospective because the full document is 13 pages long. We chose to include sections that we believe would be relevant to most EAs instead of just other EAGx organizers.)

This is a retrospective on the EAGxBerkeley conference held all day on April 30, 2016. It was written mainly by Rohin Shah, with contributions from Ajeya Cotra. The content and structure is inspired by EAGxBoston’s post on the EA Forum. Email eaofberkeley@gmail.com with any questions, or comment on this post!

(Note: This table of contents links to an external Google Doc)

Summary and Lessons Learned

Impact Evaluation

Money and Time Costs

Impact

Comparison to other EA Berkeley work

Conference Format

Target Audience

Logistics

Contacting Speakers

Venue

Food

A/V

Marketing

Facebook

Tickets

The Day

Budget

Interaction with EA Outreach

Appendix

Timeline

EA Outreach Impact Survey

Summary and Lessons Learned

The EA Berkeley club started organizing EAGxBerkeley in earnest on February 13 and put on EAGxBerkeley on April 30, with around 80 attendees. There were a few hiccups with the organization but overall the conference was decent. However, we do not feel that the conference was very impactful compared to other EA Berkeley club activities. We may still put on an EAGx conference again -- it seems possible to us that we could improve on EAGx to the point where it would have similar impact as other club activities.

We want to echo almost everything that EAGxBoston said:

• We would not recommend such a short timeline -- if possible, it would be better to have 4-6 months of time. Each organizer put in 2-5 hours per week initially, which ramped up as the conference got closer.

• Choosing a date early was critical -- it allowed us to specify an exact day and time when contacting speakers, which helped speed up the process.

• We had trouble with funding -- in the end, we simply didn’t ask for any funding from EA Outreach because they had not given us the grant application, and so we funded the conference from our club funds.

We also learned some other major lessons from EAGxBerkeley:

• Even though we tried to target Berkeley students who were new to EA, a significant fraction of our attendees were quite familiar with EA already. If there are a lot of EAs nearby an EAGx conference, it seems likely that they will attend. (However, it is worth noting that we did not explicitly say that the event was geared toward newcomers -- we simply marketed directly to Berkeley students.)

• It is very likely worthwhile to have “early bird” discount tickets, so that people who are interested buy tickets immediately instead of waiting until they are sure they will come.

• It is very important to contact speakers early (more than two months before the event date). Many of our potential speakers said they would have liked to speak at the conference, but had some obligation on the day of the conference.

• Even in Berkeley, where vegan food is easy to find, it is hard to get a good vegan breakfast. It makes sense to have breakfast be vegetarian with vegan options, though lunch and dinner can still be fully vegan.

• We had some trouble interacting with EA Outreach, which made the organizing experience a little harder. Hopefully this will become easier as more EAGx conferences happen, but future organizers should be aware of this.

More minor lessons about organizing a conference can be found in this doc.

Impact Evaluation

Money and Time Costs

We spent $2,721.07 to organize EAGx, and earned $816.20 from ticket sales, which means that we used $1,904.87 of club funds. Details can be found in the budget section.

The two main organizers tracked their time using Toggl (see here for details), and found that they put in 67 and 65 hours respectively for organizing the conference. Using an estimate of [30-50] hours for each of the other three organizers, we come out with around [220-280] person-hours spent organizing the conference.

We had six volunteers during the conference day itself. We estimate that each person spent around [4-6] hours actively volunteering, including time spent before the event, for a total of [24-36] person-hours.

One of the organizers also gave the intro talk, and spent 30 hours preparing for it. We would guess that most of the other speakers did not put nearly as much time into their talks, so we conservatively estimate around [10-20] hours of work for each speaker on average. (This includes time spent meeting with us to vet talks, time spent on logistical work, time delivering the actual talk, time spent preparing for the talk, etc.) With 15 speakers, this comes out to [150-300] hours.

In total, this comes out to [240-320] person-hours contributed by members of EA Berkeley for organizing the conference, and [390-620] person-hours if you include the speaker time as well.

Impact

It is very hard for us to estimate the impact of EAGxBerkeley, since we don’t have much concrete data to work with. We sent a survey to attendees at the end of the conference, but this was primarily to get feedback about the conference organization. We did ask whether people had changed their mind about anything, and many of them had, but we are doubtful that this is that impactful.

Here are some quantifiable effects:

• Two of the attendees will very likely be active club members next semester, though one of them would have been a member anyway.

• Two of the attendees have taken the Giving What We Can pledge between the day of EAGxBerkeley and the time of this writing (August 25, 2016). We are not sure of the causal impact attributable to EAGxBerkeley for these pledges, but would guess that it is low.

EA Outreach sent out an impact survey after the event, which we have analyzed here. Based on that survey, it seems that we were fairly good at informing people about typical EA thought, and moved people somewhat towards those positions. In addition, 17% of respondents said they didn’t previously look at effectiveness when donating but now would, and 7% said that one of the talks or panels led to a career plan change. We aren’t that impressed by this because in our experience, very few of these will actually happen.

Here are some less quantifiable effects:

• Attendees got a chance to network, which may result in big life changes and more good done. (For example, EA Berkeley was started by two students who met each other at EA Global.)

• Attendees changed their minds about some issue of importance that leads to them doing more good. (For example, perhaps an attendee is convinced that systemic change is tractable and goes on to significantly reduce corruption.)

• Organizers built valuable skills by organizing this conference.

• EA Berkeley becomes more reputable since we have put on a conference.

• EA Berkeley built stronger ties with the EA community by putting on this conference.

However, we do not feel that these effects are very impactful. These all seem like they have a fairly low probability of having a high impact, and may not have much counterfactual impact (these effects would likely have happened through some other means even if EAGxBerkeley had not happened). We would prefer to get 1-2 more GWWC pledges instead of all of these effects combined.

Overall, we would prefer to get 4-6 more GWWC pledges instead of all of EAGxBerkeley’s effects. So if we fix the impact at 4 pledges, using a value of $60,000 per pledge we get $240,000 worth of value for $1,904.87, around 280 organizer hours, and around 220 speaker hours.

Comparison to other EA Berkeley work

We have written up a report on EA Berkeley work over Spring 2016 here. Excluding EAGxBerkeley, the co-presidents collectively put in 387 person-hours over the semester. We would estimate the rest of the members put in around 150-200 person-hours (our most active members worked primarily on EAGx), for a total of around 550 person-hours. We spent $3,443.89 on these activities.

For this, we have gotten 9-10 concrete GWWC pledges, that would almost certainly not have happened otherwise. This alone makes EA Berkeley activities slightly more cost-effective than EAGx. However, there are plenty of other beneficial effects -- for example:

• Two of the pledgees will also be active members next semester. (In general, we expect active members to be more committed and more knowledgeable, and so we value them more than a single GWWC pledge.)

• At least two students became vegetarian because of us.

• Our members became more knowledgeable about EA, which we expect will lead to higher impact in the future.

• Over $1,100 of the money we “spent” was regranted to effective charities through Giving Games.

• We expect that we have affected our members’ career plans, though we are not sure to what extent.

There are several other beneficial effects as well. Overall, we would estimate that EA Berkeley had ~5x the impact of EAGxBerkeley, making it 2-3x more cost-effective. If you include speaker hours as a cost for EAGxBerkeley, then EA Berkeley activities are ~5x more cost-effective. In addition, the evidence of impact for EA Berkeley is much stronger and more robust than for EAGxBerkeley.

Conference Format

Target Audience

We specifically targeted Berkeley students who were very new to EA, since our primary goal was to get new people interested in EA. Unfortunately we were not very successful at this -- of the 89 people who answered “How familiar are you with the Effective Altruism movement?”:

• 12 chose “This is the first time I’ve heard the term”

• 16 chose “I’ve heard of it before but don’t really know what it is”

• 46 chose “I know about GiveWell, 80,000 Hours, AMF, GiveDirectly and Giving What We Can”

• 15 chose “I know as much as or more than the organizers”

However, for many of the people we personally know who chose “I know about GiveWell, …”, we think that the conference would be valuable for them and they were in our target audience. So, in hindsight, our options may need more tweaking -- perhaps we should have simply used a 1-7 scale. Either way though, we are fairly confident that over a third of our attendees, and probably half, were not in our target audience. We aren’t sure whether we should change our target audience or change our marketing next time.

Interaction with EA Outreach

Note: Roxanne and other EA Outreach employees reviewed this section before publication.

One hard part of organizing EAGxBerkeley was interacting with EA Outreach (EAO). Most of this was with Roxanne, the main point of contact at EAO. She has said that this was because she bit off more than she could chew over the last semester. There were multiple instances where Roxanne did not respond to our messages for days, and almost every time that Roxanne said EAO would do something for us, that thing would not be done by the time they said it would be done. Some examples:

• When we were accepted for EAGx in February, we were told that we could apply for grants from EAO to fund the conference. We or another EAGx team asked for the grant application three times, and all three times Roxanne failed to meet the deadline she said it would be done by. Eventually, we gave up on EAO grants and used club funds, especially since EAGxBoston was having a lot of trouble getting EAO funding.

• Roxanne told us that EAO wanted to create an intro talk for the conference, or at least be part of the process if we were to make our own intro talk. However, we never heard about that again.

That said, we do not think that all of the blame lies with Roxanne. Based on our limited knowledge, it seems that in some instances there were other issues:

• At one point, Roxanne had the grant application ready to go, but then got conflicting funding information at the last moment from funders, and so it was never released.

• Roxanne asked other EAO staff to help with the grant application, but they were not able to finish it either.

• After our trial assignment for EAGx, it sounded to us that Roxanne was on board but needed to make a final determination with the rest of the team. That took a week to come, which was hard for us since we already had a very compressed timeline.

• Our impression is that a lot of the constraints on EAGx events were created by the team as a whole, not Roxanne alone. (These are things like “you must have an application”, “we will give the intro talk, or at least have input into it”, and so on.) It was apparent to us well before the conference date that Roxanne/EAO was overburdened, and yet these constraints were created that made the burden even larger.

• Regardless of everything else, there should have been someone at EAO who was checking in on Roxanne, especially since she is only working part-time.

These are the only instances which we heard about, but we think it is probable that there were others that we don’t know about (for example, perhaps someone was in charge of creating an intro talk but couldn’t finish it in time).

All of that said, we think it is likely that future EAGx organizers will have a much easier time -- a lot of the questions that came up with our conference have been decided and should not cause significant delays any more. In particular, the grant application and intro talk should be ready to go immediately, whereas for us they were being created in parallel to us organizing the event.