More info on EA Global admissions

My current sense (as someone who organized EAG in the past and has thought about the effects of EAG a lot) is that it would be better to increase the size of the event, and if that's not financially viable, reduce the size of the subsidies for attendees to make that possible.

I don't think the effect size of "some people felt like the event was too big" is comparable to the effect size of allowing up to 50% more people to participate in the event, and so I think 1000+ person EAG events are probably worth it.

My experience from finding venues is that it is quite doable to find 1000+ person sized venues for reasonable prices, and I didn't share the impression that venue-price seems to increase drastically for more than 500-600 people. I do think the price-per-head might increase a bit, but I would be surprised if it increased by more than 15%.

Ben_West @ 2020-01-03T23:45 (+6)

Thanks for the feedback, Oliver. Do you have opinions on our hypothesis that we should focus on EAGx over more/bigger EA Globals?

Habryka @ 2020-01-04T20:26 (+41)

So, I have a few considerations that tend to argue against that. Here are some of them:

1. Common knowledge is built better by having everyone actually in the same space

I think having common knowledge of norms, ideas and future plans is often very important, and is better achieved by having everyone in the same place. If you split up the event into multiple events, even if all the same people attend, the participants of those events can now no longer verify who else is at the event, and as such can no longer build common knowledge with those other people about the things that have been discussed.

2. EAGx events have historically been of lower quality

I have been to 3 EAGx events, all three of which seemed to me to be just generally much worse run than EAG, both in terms of content and operations. And to be clear, I don't think this reflects particularly badly on the organizers, running a conference is just hard and requires a lot of time, which most EAGx organizers don't tend to have. In general I am in favor of specialization here. Obviously you helping the organizers might be able to address this consideration, so this might be moot.

3. EAGx events should go deep, whereas EAG events should go wide

When I designed the original goal-document for EAGx together with the EAO team, the goal of EAGx was in large parts to allow the creation of more specialist conferences, in which participants could go significantly more in-depth into a topic, and overall feel more like researcher conferences. I think for a variety of reasons that never ended up happening, but one of the reasons is I think that we did try to compensate for the lack of space in the EAG events by encouraging people to go to EAGx events instead.

My current sense is that we do actually also want distributed intro events, and we might want a separate brand from EAGx for that. But for now, I think encouraging usual EAG attendees to go to EAGx events as a replacement will prevent more specialist EAGx-type events from happening, which seems sad to me.

4. The value of a conference does scale to a meaningful degree with n^2

Metcalfe's Law states that

the [value] of a telecommunications network is proportional to the square of the number of connected users of the system (n^2)

I don't think this fully applies to conferences, but I do think it applies to a large degree. The value of an event to me is somewhat proportional to the number of people at that event, so I think there are strong increasing returns to conference size, at least from that perspective.

5. Group membership is in significant parts determined by who attends EAG, and not by who attends EAGx, and I feel somewhat uncomfortable with the degree of control CEA has over that

I think there is a meaningful sense in which people's intuitive sense of "who is an active member of the EA community" is closely related to who attended past EAG events, and so I think preventing people from attending EAG is actually something that has a pretty significant effect on people's social standing. I think having smaller events introduces a lot of noise into that system, and I also don't currently trust CEA to make a lot of the relevant decisions here, and would prefer CEA to on the margin have less control over EA group membership.

I have some more concerns, but these are the ones that I felt like I could write up easily.

Denise_Melchin @ 2020-01-04T23:19 (+25)

I don't feel like I get more value out of large conferences and I'd be curious about seeing more data on this question. For me, having more people at a conference makes it harder to physically find the people I actually want to talk to. They make up a smaller fraction of attendees and are more spread out. I have also had the impression that conversations at large conferences are shorter. In combination, I get much less value out of very large events compared to small or medium sized ones.

The event size was one of the main reasons I decided not to attend EAG London this year for the first time. It is too big for me to get sufficient value out of it.

Milan_Griffes @ 2020-01-04T21:13 (+14)

Thank you for this fantastic comment.

5. Group membership is in significant parts determined by who attends EAG, and not by who attends EAGx, and I feel somewhat uncomfortable with the degree of control CEA has over that

Strongly agree. EAG attendance is a Schelling point for who is "an EA" and who isn't, even if EAG organizers don't endorse this, and even if "being an EA" isn't an endorsed and/or fully coherent concept.

ellenm @ 2020-01-06T05:04 (+22)

Agree wholeheartedly! Especially for those who fall under "You’ve been several times before and we want to give a spot to a first-timer". I imagine if you go to EAG every year and were suddenly rejected you'd feel like you were kicked out of the club. A huge part of community building in professional associations is going to an annual conference and getting to catch up with your peers, EAG is that way for those of us who don't live in largely populated EA cities.

Milan_Griffes @ 2020-01-06T16:48 (+7)

A huge part of community building in professional associations is going to an annual conference and getting to catch up with your peers, EAG is that way for those of us who don't live in largely populated EA cities.

+1 to the analogy of EA Global as a professional association's annual conference.

willbradshaw @ 2020-01-15T23:19 (+3)

I also think this is a valuable analogy.

Ben_West @ 2020-01-15T17:54 (+11)

Thanks for the detailed thoughts Oli.

I think having common knowledge of norms, ideas and future plans is often very important, and is better achieved by having everyone in the same place. If you split up the event into multiple events, even if all the same people attend, the participants of those events can now no longer verify who else is at the event, and as such can no longer build common knowledge with those other people about the things that have been discussed.

Interesting, this doesn’t fit with my experience for two reasons: a) attendance is so far past Dunbar’s number that I have a hard time knowing who attended any individual EA Global and b) even if I know that someone attended a given EA Global, I’m not sure whether they attended any individual talk/workshop/etc. (since many people don’t attend the same talks, or even any talks at all).

I’m curious if you have examples of “norms, ideas, or future plans” which were successfully shared in 2016 (when we had just the one large EA Global) that you think would not have successfully been shared if we had multiple events?

I have been to 3 EAGx events, all three of which seemed to me to be just generally much worse run than EAG, both in terms of content and operations

We have heard concerns similar to yours about logistics and content in the past, and we are providing more support for EAGx organizers this year, including creating a “playbook” to document best practices, having monthly check-in calls between the organizers and CEA’s events team, and hosting a training for the organizers (which is happening this week).

At least in recent years, the comparison of the Net Promoter Score of EAG and EAGx events indicate that the attendees themselves are positive about EAGx, though there are obviously lots of confounding factors:

(More information about EAGx can be found here.)

The value of a conference does scale to a meaningful degree with n^2… I think there are strong increasing returns to conference size

Echoing Denise, I would be curious for evidence here. My intuition is that marginal returns are diminishing, not increasing, and I think this is a common view (e.g. ticket prices for conferences don’t seem to scale with the square of the number of attendees).

Group membership is in significant parts determined by who attends EAG, and not by who attends EAGx, and I feel somewhat uncomfortable with the degree of control CEA has over that

Do you have examples of groups (events, programs, etc.) which use EA Global attendance as a “significant” membership criterion?

My impression is that many people who are highly involved in EA do not attend EA Global (some EA organization staff do not attend, for example), so I would be pretty skeptical of using it.

Meta Note

To clarify my above responses: I (and the Events team, who are currently running a retreat with the EAGx organizers) believe that more people being able to attend EA Global is good, all other things being equal. Even though I’m less positive about the specific things you are pointing to here than you are, I generally agree that you are pointing to legitimate sources of value.

Habryka @ 2020-01-16T21:15 (+22)

I’m curious if you have examples of “norms, ideas, or future plans” which were successfully shared in 2016 (when we had just the one large EA Global) that you think would not have successfully been shared if we had multiple events?

I think EAG 2016 was the last time that I felt like there was a strong shared EA culture. These days I feel quite isolated from the european EA culture, and feel like there is a significant amount of tension between the different cultural clusters (though this is probably worsened by me no longer visiting the UK very much, which I tended to do more during my time at CEA). I think that tension has always been there, but I feel like I am now much more disconnected from how EA is going in other places around the world (and more broadly, don't see a path forward for cultural recombination and reconciliation) because the two clusters just have their own events. I also feel somewhat similar about east-coast and west-coast cultural differences.

More concrete examples would be propagating ongoing shifts in cause-priorities. Many surveys suggest there has been an ongoing shift to more long-term causes, and my sense is that there is a buildup of social tension associated with that, that I think is hard to resolve without building common knowledge.

I think EAG 2016 very concretely actually did a lot by creating common-knowledge of that shift in cause-priorities, as well as a broader shift towards more macro-scale modeling, instead of more narrow RCT-based thinking that I think many assumed to be "what EA is about". I.e. I think EAG 2016 did a lot to establish that EA wasn't just primarily GiveWell and GiveWell style approaches.

A lot of the information I expect to be exchanged here is not going to be straightforward facts, but much more related to attitudes and social expectations, so it's hard to be very concrete about these things, which I regret.

Importantly, I think a lot of this information spreads even when not everyone is attending the same talk. At all EAGs I went to, basically everyone knew by the end what the main points of the opening talks were, because people talked to each other about the content of the opening talks (if they were well-delivered), even if they didn't attend, so there is a lot of diffusion of information that makes literally everyone being in the same talk not fully necessary (and where probabilistic common-knowledge can still be built). The information flow of people who attended separate EA Globals is still present, just many orders of magnitude weaker.

At least in recent years, the comparison of the Net Promoter Score of EAG and EAGx events indicate that the attendees themselves are positive about EAGx, though there are obviously lots of confounding factors:

These graphs are great and surprising to me. I don't yet have great models of how I expect the Net Promoter Score to vary for different types of events like this, so I am not sure yet how to update.

Echoing Denise, I would be curious for evidence here. My intuition is that marginal returns are diminishing, not increasing, and I think this is a common view

At this year's EAG there were many core people in EA that I had hoped I could talk to, but that weren't attending, and when I inquired about their presence, they said they were just planning to attend EAG London, since that was more convenient for them. I also heard other people say that they weren't attending because they didn't really expect a lot of the "best people" to be around, which is a negative feedback loop that I think is at least partially caused by having many events, without one clear Schelling event that everyone is expected to show up to.

(e.g. ticket prices for conferences don’t seem to scale with the square of the number of attendees).

This assumes a model of perfect monopoly for conferences. In a perfectly competitive conference landscape, you expect ticket prices to be equal to marginal costs, which would be decreasing with size. I expect the actual conference landscape to be somewhere in-between, with a curve that does increase in prize proportional to size for a bit, but definitely not completely. Because of that, I don't think price is much evidence either way on this issue.

Do you have examples of groups (events, programs, etc.) which use EA Global attendance as a “significant” membership criterion?

I think I do to some significant extend. I definitely have a significantly different relationship to how I treat people who I met at EA Global. I also think that if someone tells me that they tried to get into EA Global but didn't get in, then I do make a pretty significant update on the degree to which they are core to EA, though the post above has definitely changed that some for me (since it made it more clear that CEA was handling acceptances quite differently than I thought they were). But I don't expect everyone to have read the post in as much detail as I have, and I expect people will continue to think that EAG attendance is in significant parts screening for involvement and knowledge about EA.

I have a variety of other thoughts, but probably won't have time to engage much more. So this will most likely be my last comment on the thread (unless someone asks a question or makes a comment that ends up feeling particularly easy or fun to reply to).

Milan_Griffes @ 2020-01-15T18:13 (+7)

My impression is that many people who are highly involved in EA do not attend EA Global (some EA organization staff do not attend, for example), so I would be pretty skeptical of using it.

On the "group membership" dimension, attending EAG is less important for EA org staff as they have other signifiers of membership in the group.

Khorton @ 2020-01-05T14:11 (+7)

I agree with Denise that I prefer smaller conferences, but what I really want to attend is a small specialist conference (for example, an EAGx for the top 150 policy experts in EA). Your suggestion here that a large EAG could lead to smaller EAGx conferences has gotten my attention.

ellenm @ 2020-01-01T00:22 (+3)

Is travel price ever taken into consideration? Non-US and/or east cost attendees can easily spend $1k+ just for flights/lodging which probably causes them to take a large discount on the ticket price.

Julia_Wise @ 2020-01-08T17:58 (+4)

Yes, we do take attendees' travel costs into consideration. This is part of why we went back to having multiple EA Global conferences after trying out having only one in 2016, and why we have continued to have EAGx events, so that more people have something near them. We recognize that reduced travel costs would be one benefit to having more events.

This might be a slightly silly suggestion and I'm not sure how best to implement it, but I think it might be useful to remind potential attendees that attending EAG is not obligatory just because you are part of the EA Community and/or care a lot about doing good well. I heard from a few people who weren't particularly excited about attending EAG, but still did it because that's 'what you do as an EA'. It seems sad that these people take up spots from people who are actually keen on EAG itself.

It only occurred to me fairly late last year that attending EAG is actually entirely optional. On a side note, rising ticket prices did help me come to the realisation that I did not actually want to go (and therefore didn't take up a spot from someone who was more keen on going).

This was a super helpful write-up! My only point of feedback would be to write a short version of this into the waitlist/rejection emails for future EAGs (or even the next wave for this EAG). It was really confusing to get waitlisted after being accepted for the last 4 years with a form rejection email without any explanation. Maybe add it in for people who answer that they've attended EAG before? I imagine that would help from turning people off to EA/EAG. I definitely had an immediate reaction of not feeling valued by the community after being involved for ~5 years (and was sad I won't see my EA friends/acquaintances this year), hopefully other people didn't feel that way.

It's good to hear more resources are being put into EAGx events, but if they are "oriented to students and other earlier-stage EAs who are less likely to be able to get into EA Global", will there be some in-between events for people who are older and more experienced but didn't get into EAG this year?

Apologies if any of this came off as mean/bitter, EAG is one of my favorite times of the year (I'm one of the people that loves going to talks)! :)

This seems to be a good problem to have, as it seems we're growing to the point where we can't accommodate everyone in one place. I'd be interested to hear about future plans to accommodate the growth of having more "advanced" EAs across the US/world since right now it seems like if we don't live in a major EA city, EAG is our only chance to see other EAs and learn new things.

Thanks to everyone who helps make these events possible. I assume UC Berkeley in the summer that accommodated ~1000 people in 2016 was not more expensive, so you would describe it as less suited to the event? Why is that? It had the large advantage of very inexpensive housing in the dorms. That is understandable if CEA only wants to subsidize a certain number of tickets, but I would think there are significant number of additional people who would pay the full cost. I'm interested in the estimate of the percent reduction in value to the first ~600 participants associated with a larger conference and how that was weighed against the value that additional participants could get. With fewer EAGx events, I expect the value of the latter would be larger this year than other years.

What about making more EAG propers (not EAGx's)?

Thanks for writing this. I don't have a solution but I'm just registering that I would expect plenty of rejected applicants to feel alienated from the EA community despite this post.

I wanted to share an update: for the past month, our events team (Amy, Barry, and Kate) have been brainstorming ways to allow more people to attend EA Global SF 2020. Our previous bottleneck was the number of seats available for lunch: even with us buying out the restaurant next to the dome (M.Y. China), we only had space for 550 people. (Tap 415, another nearby restaurant which we had used in prior years, has gone out of business.)

We have now updated our agreements with the venue and contractors and brainstormed some additional changes that will allow more attendees in sessions and at lunch. This has increased our capacity by 70 (from 550 to 620).

(As a reference point: EA Global SF had 499 attendees in 2019.)

Why wouldn't randomisation be a good fit for EAG? I suspect that the ability of the organisers to finely distinguish between similarly-promising applicants is minimal anyway*, so a strategy of, say, roughly scoring applicants into buckets and then randomising among those who fall between "obvious shoo-in" and "clearly unsuitable" could work quite well, as well as being much quicker and easier for the organisers.

(This is roughly how the proposal to randomise scientific grantmaking would work: apply a basic check for suitability/competence and then randomise among those who make that cut. I think this would be a big improvement over the current system and would apply the same reasoning in many other domains with similar features, such as university admissions.)

* Not because I have a low opinion of the organisers, just because I think this is generally true.

Also how did you do that co-author thing?

I appreciate how thoughtful the admissions team seems to be about creating a useful experience for attendees!

With this explanation (maybe moreso than in previous years when the average applicant was less involved in EA), it sounds like there are some large categories of people where your message is "We'd like you to apply, but you probably won't get in, and we know that's discouraging you from applying." That seems like a tough spot to put applicants in, particularly given that individual EAs can be very self-critical and might be inclined to forgo applying if they imagine you might mistakenly decide they're a better applicant than someone else in the same loose category.

One question I have related to this is about what steps you've considered or taken (other than this post!) towards encouraging applications from people who might be discouraged about their likelihood of getting in. For example, have you adjusted the way you promote the conference in general? Do you suggest that colleges, organizations, or local groups try to encourage every member who would like to attend EAG to apply, or conversely that they try to coordinate so that not too many people from their group apply? Have you made changes to the application itself to make it easier to fill out or more clear about the range of people you want to see apply? Other things?

I'm also curious whether you have considered more strongly theming some EAG or EAGx events in order to provide a clearer signal about who would get the most benefit out of a particular conference and perhaps provide more focused environments for attendees. (Any type of theme, whether focusing on some range of cause areas, some stage of attendee careers, some common concern like research implementation...) Or if you have tried themes in the past, have they had any of that type of effect, or were people still looking for an experience that felt like it covered a broad spectrum of EA topics and concerns?

This seems like it involved some difficult decisions. Thanks for selecting what sounds like it was most reasonable and explaining here.

I guess my main take-away is not anything about the details of this event, but rather the point that it seems like it's become more urgent/important for more work to be done in this area.

It seems to me like the previous few EAGs were done with relatively few people, and now there's an opportunity for a bunch more work.

I'm a bit curious; how big of a challenge does this feel for CEA, and are there ways you think community members could help out? (For instance, applying to CEA to help run similar events, or holding more local meetups, etc.)