Deference Culture in EA

EA has a high deference culture? Compared to what other cultures? Idk but I feel like the difference between EA and other groups of people I've been in (grad students, City Year people, law students...) may not be that EAs defer more on average but rather that they are much more likely to explicitly flag when they are doing so. In EA the default expectation is that you do your own thinking and back up your decisions and claims with evidence*, and deference is a legitimate source of evidence so people cite it. But in other communities people would just say "I think X" or "I'm doing X" and not bother to explain why (and perhaps not even know why, because they didn't really think that much about it).

*Other communities have this norm too, I think, but not to the same extent.

Here are choice parts of my model of deference:

1. Whether you should defer or not depends not only on your estimation of relative expertise but also on what kind of role you want to fill in the community, in order to increase the altruistic impact of the community.  I call it role-based social epistemology, and I really should write it longly at some point.
2. You can think of the roles as occupying different points on the production possibilities frontier for the explore-exploit trade-off. If you think of rationality as an individual project, you might reason that you should aim for a healthy balance between exploring and exploiting due to potential diminishing returns to either one. But if you instead take the perspective of "how can I coordinate with my community in order to maximize the impact we produce?" you start to see why specializing could be optimal.
3. If you are a Decision-Maker, you're optimizing for allocating resources efficiently (e.g. money, work, power, etc.), and the impact of your allocation depends on how accurate your related beliefs are. And because accurate beliefs are so important to your decisions, you should opportunistically defer to people whenever you think they might have better information than you (Aumann-agreement style), as long as you think you're decently calibrated and you're deferring to advice with sufficient bandwidth. You should be Exploiting existing knowledge and expertise by deferring to it. But because you frequently defer to others, you may not be safe to defer to in turn due to potential negative externalities associated with information cascades that can be hard to correct.
4. If you are an Explorer, your job is to optimize for the chance of discovering important insights that can help the community make progress on important open problems. This is fundamentally a different project compared to just trying to acquire accurate beliefs. Now, you want to actively avoid ending up with the same belief states as other people to some extent. Notice that the problems are still open, which means that existing tools and angles-of-attack may be insufficient for the task. Evaluate paradigms/approaches for how neglected they are. Remember, it doesn't matter whether you're right about what other people are right about as long as you are extremely right about what other people are wrong about. So if you want to maximize the chance that the community ends up solving the problem, you want to coordinate with other explorers in order to search separate parts of the idea-tree. What matters is that the right fruits are picked, not that you end up picking them. We're in a parallel tree search paradigm, and this has implications for how we individually should balance the explore-exploit trade-off.
5. If you are an Expert/Forecaster, your job is to acquire accurate beliefs that are safe to defer to. If there's a difficult and important question (crucial consideration) for which better forecasts could marginally improve the careers/donations of a lot of people, this could be an important way to produce impact. Your impact here depends on the accuracy of your beliefs, so unlike the Explorer, you don't have strong reasons to avoid common belief states. Your impact also depends on how safe you are to defer to, because you can potentially do a lot of harm by reinforcing false information cascades. And these considerations are newcomblike, so you should act by that rule which, when followed by the proportion of other experts you predict will follow it due to the same reasoning as you, maximizes community impact. Sometimes that means you want to report your independent impressions, and sometimes that means you want to share and elicit likelihood ratios instead of posterior beliefs. A common failure mode here is to over-optimize for making your beliefs legible, which in extreme cases turns into a race to the bottom, and in median cases turns into myopic empiricism where you predictably end up astray because you refuse to update on a large class of illegible (but Bayesian) evidence.
6. The limiting case of a Decision-Maker always reporting their independent impressions is (roughly) an Expert. But only insofar as it's psychologically feasible to maintain a long-term separation between independent and all-things-considered impressions, and I have my doubts.
7. What kind of knowledge-work you want to do depends not only on your comparative advantages but also on your model of how the community produces altruistic impact. If on your model community impact is marginally bottlenecked by insights, you should probably consider aiming for ambitious insight-production. If on the other hand, you think you can have more impact by contributing to marginally better forecasts about what problems are most important to work on, maybe consider aiming for producing deference-safe predictions. And if you just happen to have a bunch of money lying around, you just don't have the luxury of recklessly diverging from expert consensus, and you should use everything in your toolbox to make sure you're allocating them efficiently.
8. No one is pure any of these. The roles are separated by what optimization criteria they use, and you optimize for different things in different areas of your life, and over your lifetime. But I think it's usefwl to carve out the roles, so you can notice when you need to put which hat on, and the different things that implies for how you should play.

Thanks for detailing this aspect of EA. I think much of the deference culture is driven by early EA orgs like GiveWell, as you mention. There is a tendency to map the strong deference that GiveWell merits in global health onto other cause areas where it may not apply. For instance, GWWC recommends giving to several funds in different cause areas. The presentation suggests that funds are roughly equal in quality for their respective cause areas. GiveWell has about ~5x more staff than Animal Charity Evaluators, ~10x+ more staff compared to the EA Infrastructure Fund, and ~10x+ more staff compared to Founders Pledge's climate team. To the extent that a larger team size means more research hours and more research hours means better funding decisions, there is a significant difference in funding quality among the different fund recommendations. This difference isn't communicated in public-facing EA media like the GWWC webpage and videos.

As someone who is an expert in a cause area where the EA fund has comparatively little analytical capacity (climate change mitigation), I find the deference and marketing of the climate fund as the most effective giving option a continual source of frustration. I've written about that here and here. I'm also worried about people mapping weak deference onto causes where they should have greater deference: many people early in their EA engagement care about climate change as a cause area. If they have some level of expertise, they may find the climate fund recommendations underwhelming and then incorrectly assume funds in other causes areas have similarly low levels of research behind them. There may be some attrition in getting more people more involved in EA because of this, though it is a tiny niche. I don't think the answer to the comparative deference problem is to do something like delist fund options from the GWWC page. But we do need some way to communicate the differential level of rigor.

I like that you contrast deference with investigation, rather than unilaterism. So many discussions and posts about deference devolve into discussion about unilateralism. Example: https://forum.effectivealtruism.org/posts/Jx6ncakmergiC74kG/deference-culture-in-ea?commentId=epR5HxT6nkdSCtMCf

But arguments against unilateralism can't be applied as arguments against investigation. Investigation grows the intellectual commons. Empirically it's clear there is much to investigate. EAs generally agree that AI risk is the most important problem yet there is no plan to move forward (aside from help out OpenAI and hope that this somehow turns out to be a good idea instead of an apocalyptic one). 

I like this breakdown a lot. Another related reason for deferring less and building your own inside view is for figuring out your career within a field.

Choosing research questions, deciding which roles and orgs to apply to, finding role models and plotting a career trajectory, and proposing new projects can be parts of your job in just about any field, and it’ll be hard to do them well if you’re constantly deferring to experts. On niche topics, it’s even difficult to learn who the experts are and what they believe.

Personally I’ve deferred to 80,000 Hours on which high-level cause areas offer the highest potential for impact. But after spending a few months to years learning about a single cause area, I feel much less clueless about the field and have a real inside view.

"the EA (Effective Altruism) movement has a pretty strong deference culture."

 Is this some kind of demographic thing? I haven't noticed it except in terms of college students/recent grads being a bit too attached to the idea of working for EA orgs. I defer when I don't feel like I have the appropriate knowledge and can't acquire it in reasonable time, and don't otherwise. 

As someone who was a solo-EA, without knowing there was a whole EA movement, for well over a decade, it's really nice to be able to rely on other people's judgment sometimes instead of having to analyze every little thing for myself. But that deference comes from some intuitive sense of cost-benefit tradeoffs involved in investing my time to dive deeper into something, not from a general idea that I should be deferential, and it goes away the moment I sense that the cost-benefit analysis has flipped.  And I don't feel like some kind of outlier for doing this. Another EA once called me an SBF bootlicker just for supporting Carrick Flynn, for example. 

When someone defers, they defer to a person or group. And presumably, as someone learns more about a person it becomes easier to decide when to defer to their judgment. 

I agree that learning more about a subject would help someone assess any person/group who claims to have knowledge about that subject.