What wheels do EAs and rationalists reinvent?
By Arepo @ 2024-10-17T04:42 (+26)
A common criticism of EA/rationalist discussion is that we reinvent the wheel - specifically, that concepts which become part of the community have close analogies that have been better studied in academic literature. Or in some cases, that we fixate on some particular academically sourced notion to the exclusion of many similar or competing theories.
I think we can simultaneously test and address this purported problem by crowd sourcing an open database mapping EA concepts to related academic concepts, and in particular citing papers that investigate the latter. In this thread I propose the following format:
- 'Answers' name an EA or rat concept either that you suspect might or know has mappings to a broader set of academic literature.
- Replies to answers cite at least one academic work (or a good Wikipedia article) describing a related phenomenon or concept. In some cases, an EA/rat concept might be an amalgam of multiple other concepts, so please give as many replies to answers as seem appropriate.
- Feel free but not obliged to add context to replies (as long as they link a good source)
- Feel free to reply to your own answer
I'll add any responses this thread gets to a commentable Google sheet (which I can keep updating), and share that sheet afterwards. Hopefully this will be a valuable resource both for fans of effective altruism to learn more about their areas of interest, and for critics to asserting the reinventing-of-wheelness of EA/rat to prove instances of their case (where an answer gets convincing replies) or refute them (where an answer gets no or only loosely related replies).
I'll seed the discussion with a handful of answers of my own, most of which I have at best tentative mappings.
[ETA I would ask people not to downvote answers to this thread. If the system I proposed is functioning accurately, then every answer is a service to the community, whether it ends up being mapped (and therefore validated as an instance of people re) or not mapped (and therefore refuted). If you think this is a bad system, then please downvote the top level post, rather than disincentivising the people who are trying to make it work.]
Ben Millwood🔸 @ 2024-10-17T11:12 (+11)
concepts which become part of the community have close analogies that have been better studied in academic literature
If they got into the community from the academic literature, this isn't reinventing the wheel, right? At worst it's rebranding the wheel, which feels like a different thing.
For example, is conservation of expected evidence an instance of reinventing the wheel, because this particular name for it is (as far as I know) a LessWrong innovation? I'm sure they (we?) didn't rediscover the theorem from basic principles.
I suppose you might still regard this as a point of criticism insofar as it creates jargon barriers, or insofar as you draw indirect (and IMO tenuous) inferences about a lack of collaboration with the mainstream (i.e. we can only get away with using different words because people who use the "normal" words don't talk to us). But I wouldn't want people drawing from this that LW is unfamiliar with mainstream probability theory.
Arepo @ 2024-10-17T04:45 (+6)
Arepo @ 2024-10-17T04:46 (+5)
Mo Putera @ 2024-10-17T06:47 (+1)
Not really, "coordination failure due to positional arms race" is better.
Arepo @ 2024-10-17T08:24 (+4)
I'm not sure I take a throwaway comment by someone closely socially tied to the author of the comment as evidence that it isn't equivalent.
Also it doesn't need to be literally equivalent to them. The criticism, if there is one, would be that Scott's concept doesn't add anything to the work done by academics - although that criticism would be false if it unified hitherto un-unified fields in a useful way.
Mo Putera @ 2024-10-17T09:13 (+1)
That's fair, no need to take it.
The criticism, if there is one, would be that Scott's concept doesn't add anything to the work done by academics
Stuart Armstrong (author of the OP in the link above) seems to think it was academically inspiring, cf. the passage starting with
Academic Moloch
Ok, now to the point. The "Moloch" idea is very interesting, and, at the FHI, we may try to do some research in this area (naming it something more respectable/boring, of course, something like "how to avoid stable value-losing civilization attractors").
The project hasn't started yet, but a few caveats to the Moloch idea have already occurred to me. ...
Not sure if that counts for you.
(I'm not socially tied to Luke in any way. I had the same misconception as you a long time ago, remember reading that comment as clarifying, and thought you would appreciate the share.)
Linch @ 2024-10-17T05:09 (+2)
Moloch is just a fanciful term for coordination traps right?
Arepo @ 2024-10-17T06:07 (+2)
Do you have a citation for coordination traps specifically? Coordination games seem pretty closely related, but Googling for the former I find only casual/informal references to it being a game (possibly a coordination game specifically) with multiple equilibria, some worse than others, such that players might get trapped in a suboptimal equilibrium.
Karthik Tadepalli @ 2024-10-21T16:35 (+2)
I agree with Linch that the idea that "a game can have multiple equilibria that are Pareto-rankable" is trivial. Then the existence of multiple equilibria automatically means players can get trapped in a suboptimal equilibrium – after all, that's what an equilibrium is.
What specific element of "coordination traps" goes beyond that core idea?
Linch @ 2024-10-17T20:55 (+2)
Not really; rationalist jargon is often more memetically fit than academic jargon so it's often hard for me to remember the original language even when I first learned something from non-rationalist sources. But there's a sense in which the core idea (Nash equilibria may not be Pareto efficient) is ~trivial, even if meditating on it gets you something deep/surprising eventually.
I don't really think of presenting this as Moloch as "reinventing the wheel," more like seeing the same problem from a different angle, and hopefully a pedagogically better one.
Henry Howard🔸 @ 2024-10-17T07:22 (+5)
Development Economics
One of the forum's highest rated posts is about how we should simply improve economic growth in poor countries
Erich_Grunewald 🔸 @ 2024-10-18T02:06 (+5)
I don't see how this is reinventing the wheel? The post makes many references to development economics (11 mentions to be precise). It was not an instance of independently developing something that ended up being close to development economics.
Henry Howard🔸 @ 2024-10-18T04:17 (+5)
The post suggests that 4 person-years of “careful analysis” will find “promising funding opportunities in this space”.
Development economics does that careful analysis already, why would we make breakthroughs reinventing it?
Erich_Grunewald 🔸 @ 2024-10-18T16:35 (+5)
That still does not seem like reinventing the wheel to me. My read of that post is that it's not saying "EAs should do these analyses that have already been done, from scratch" but something closer to "EAs should pay more attention to strategies from development economics and identify specific, cost-effective funding opportunities there". Unless you think development economics is solved, there is presumably still work to be done, e.g., to evaluate and compare different opportunities. For example, GiveWell definitely engages with experts in global health, but still also needs to rigorously evaluate and compare different interventions and programs.
And again, the article mentions development economics repeatedly and cites development economics texts -- why would someone mention a field, cite texts from a field, and then suggest reinventing it without giving any reason?
Linch @ 2024-10-18T20:04 (+2)
This job posting seems related.
Matrice Jacobine @ 2024-10-19T14:20 (+1)
My experience is that many global-poverty-focused EA likes to refer to their field as "global health and development" but the existing literature in institutional development economics has been mostly ignored in favor of constantly retreading the same old streetlight-illuminated ground of bednets and deworming. This may in part because it might be problematic for EA Political Orthodoxy. @Ben Kuhn has made this point cogently here and here.
Ian Turner @ 2024-10-19T21:41 (+1)
I’m not sure I understood your point. What charities or programs do you think GiveWell should be funding, but aren’t, that are supported by “existing literature in institutional development economics”?
Matrice Jacobine @ 2024-10-19T22:31 (+1)
Institutional reforms to help LDCs escape the poverty trap.
Ian Turner @ 2024-10-20T14:36 (+1)
That doesn’t sound like a charity or charitable program to me?
Matrice Jacobine @ 2024-10-20T15:01 (+1)
That's what I mean by "constantly retreading the streetlight-illuminated ground". And lack of established charities hasn't stopped the longtermist wing (and to a certain extent the animalist wing) of EA before?
Ian Turner @ 2024-10-20T15:13 (+1)
Establishing a new charity is one thing, but I haven’t seen you propose a charitable program or intervention yet?
Matrice Jacobine @ 2024-10-20T15:26 (+1)
Why are the animalist and longtermist wings of EA the only wings that consider policy change an intervention?
Ian Turner @ 2024-10-21T03:57 (+1)
Is it possible we’re talking to past each other? “Institutional reforms” isn’t something a donor can spend money or donate to. But EA global health efforts are open to working on policy change; an example is the Lead Exposure Elimination Project.
I still feel that you haven’t really answered the question, what do you think GiveWell should recommend, which they currently aren’t?
Matrice Jacobine @ 2024-10-21T12:23 (+1)
I don't know how to make it clearer. Longtermist nonprofits get to research world problems and their possible solutions without having to immediately show a randomized controlled trial following the ITN framework on policies that don't exist yet. Why is the same thing seemingly impossible for dealing with global poverty?
Jason @ 2024-10-22T01:09 (+4)
The academic fields most relevant to GH&D work are fairly mature. Because of that, it's reasonable for GH&D to focus less on producing stuff that is more like basic research / theory generation (academia is often strong in this and had a big head start) and devote its resources more toward setting up a tractable implementation of something (which is often not academia's comparative advantage for various reasons).
GH&D also has a clearly successful baseline with near-infinite room for more funding, and so more speculative projects need to clear that baseline before they become viable. You haven't identified any specific proposed area to study, but my suspicion is that most of them would require sustained political commitment over many years in the LDC and/or large cash infusions beyond the bankroll of EA GH&D to potentially work.
Matrice Jacobine @ 2024-10-22T12:31 (+1)
GH&D also has a clearly successful baseline with near-infinite room for more funding, and so more speculative projects need to clear that baseline before they become viable.
Again, that is exactly what I am calling "constantly retreading the streetlight-illuminated ground". I do not think most institutional development economists would endorse the idea that LDCs can escape the poverty trap through short-term health interventions alone.
Jason @ 2024-10-22T15:39 (+2)
I don't think most development economists would endorse the idea that a viable pathway exists for LDCs to escape the poverty trap based on ~$600-800MM/year in EA funding (even assuming you could concentrate all GH&D funding on a single project) and near-zero relevant political influence, either. And those are the resources that GH&D EA has on the table right now in my estimation.
To fund something at even the early stages, one needs either the ability to execute any resulting project or the ability to persuade those who do. The type of projects you're implying are very likely to require boatloads of cash, widespread and painful-to-some changes in the LDCs, or both. Even conditioned on a consensus within development economics, I am skeptical that EA has that much ability to get Western foreign aid departments and LDC politicians to do what the development economists say they should be doing.
Matrice Jacobine @ 2024-10-22T16:32 (+1)
Okay, so why is the faction of EA with ostensibly the most funds the one with "near-zero relevant political influence" while one of the animalist faction's top projects is creating an animalist movement in East Asia from scratch, and the longtermist faction has the president of RAND? That seems like a choice to divide influence that way in the first place.
Karthik Tadepalli @ 2024-10-21T16:31 (+4)
Conservation of expected evidence
Karthik Tadepalli @ 2024-10-21T16:37 (+9)
This is a restatement of the law of iterated expectations. LIE says . Replace with an indicator variable for whether some hypothesis is true, and interpret as an indicator for binary evidence about . Then this immediately gives you a conservation of expected evidence: if , then , since is an average of the two of them so it must be in between them.
You could argue this is just an intuitive connection of the LIE to problems of decisionmaking, rather than a reinvention. But there's no acknowledgement of the LIE anywhere in the original post or comments. In fact, it's treated as a consequence of Bayesianism, when it follows from probability axioms. (Though one comment does point this out.)
To see it formulated in a context explicitly about beliefs, see Box 1 in these macroeconomics lecture notes.
Arepo @ 2024-10-22T05:24 (+2)
Thanks - agree or disagree with it, this is a really nice example of what I was hoping for.
Ulrik Horn @ 2024-10-22T11:28 (+3)
Moral circle. There are so many frameworks from psychology on morality, empathy etc. But maybe I am missing some nuance that makes moral circle distinct from all of these but to date I have not seen it.
Matrice Jacobine @ 2024-10-22T12:28 (+5)
The concept was coined by Singer, who is an EA, but he coined it in 1981 and it has been a term of mainstream moral philosophy for a while.
Ulrik Horn @ 2024-10-22T16:20 (+2)
Ah that might explain it - it is coming from philosophy not psychology!
Robi Rahman @ 2024-10-17T22:28 (+3)
I saw a lot of criticism of the EA approach to x-risks on the grounds that we're just reinventing the wheel, and that these already exist in government disaster preparedness and the insurance industry. I looked into the fields that we're supposedly reinventing, and they weren't the same at all, in that the scale of catastrophes previously investigated was far smaller, only up to regional things like natural disasters. No one in any position of authority had prepared a serious plan for what to do in any situation where human extinction was a possibility, even the ones the general public has heard of (nuclear winter, asteroids, climate change).
Arepo @ 2024-10-19T03:32 (+2)
The extent to which you think they're the same is going to depend heavily on
- your long term moral discounting rate (if it's high, then you're going to be equally concerned between highly destructive events that very likely won't kill everyone and comparably destructive events that might),
- your priors on specific events leading to human extinction (which, given the lack of data, will have a strong impact on your conclusion), and
- your change in credence of civilisation flourishing post-catastrophe.
Given the high uncertainty behind each of those considerations (arguably excluding the first), I think it's too strong to say they're 'not the same at all'. I don't know what you mean by fields only looking into regional disasters - how are you differentiating those investigations from the fields that you mention that the general public has heard of in large part because a ton of academic and governmental effort has gone into it?
Robi Rahman @ 2024-10-22T20:15 (+2)
I agree with your numbered points, especially that if your discount rate is very high, then a catastrophe that kills almost everyone is similar in badness to a catastrophe that kills everyone.
But one of the key differences between EA/LT and these fields is that we're almost the only ones who think future people are (almost) as important as present people, and that the discount rate shouldn't be very high. Under that assumption, the work done is indeed very different in what it accomplishes.
I don't know what you mean by fields only looking into regional disasters - how are you differentiating those investigations from the fields that you mention that the general public has heard of in large part because a ton of academic and governmental effort has gone into it?
I'm skeptical that the insurance industry isn't bothering to protect against asteroids and nuclear winter just because they think the government is already handling those scenarios. For one, any event that kills all humans is uninsurable, so a profit-motivated mitigation plan will be underincentivized and ineffective. Furthermore, I don't agree that the government has any good plan to deal with x-risks. (Perhaps they have a secret, very effective, classified plan that I'm not aware of, but I doubt it.)
Jason @ 2024-10-17T23:47 (+2)
I'm confused on how to interpret a disagreevote on a top-level "answer" which is supposed to merely list a concept. In contrast, one could disagreevote a reply to an answer because it doesn't map well to the top-level answer.
MichaelDickens @ 2024-10-18T02:04 (+6)
I interpreted disagreement to mean "EA is not reinventing the wheel with this concept".
Chris Leong @ 2024-10-17T08:42 (+2)
I'd suggest going with a Wiki page rather than a Google Sheet. Google Sheets are more suited to the task, but almost inevitably become outdated at some point.
Ben Millwood🔸 @ 2024-10-17T11:06 (+2)
what wiki did you have in mind? wouldn't it also be at risk of becoming outdated?
Chris Leong @ 2024-10-17T11:30 (+2)
EA or LW. Just less dependent on a single editor adding/approving changes.
Saul Munn @ 2024-10-17T06:39 (+2)
thanks for making this post/question!
Arepo @ 2024-10-17T05:01 (+2)
Erich_Grunewald 🔸 @ 2024-10-18T02:08 (+6)
It would be helpful if you mentioned who the original inventor was.
Arepo @ 2024-10-17T04:55 (+2)
Arepo @ 2024-10-17T04:55 (+2)
Arepo @ 2024-10-17T04:48 (+2)
Global catastrophic risk studies
Arepo @ 2024-10-17T04:53 (+2)
I had the impression there was a field of '(global) resilience studies' I'd seen before, but on a first look at the moment can't find anything convincingly on point.
GideonF @ 2024-10-17T10:25 (+6)
Pretty sure EA basically invented that (yes people were working on stuff before then and outside of it, but still that seems different to 'reinventing the wheel')
Arepo @ 2024-10-17T04:43 (+1)
Arepo @ 2024-10-17T04:43 (+1)
The Sorites Paradox
Chris Leong @ 2024-10-17T08:44 (+3)
This is very different. I'd reference Wittgenstein's Family Resemblances instead.