We don’t need AGI for an amazing future

Karl - thanks for a wonderful, clear, and compelling essay. 

I'm disappointed that you got so many weirdly hostile reactions on LessWrong (although that's not uncommon on LessWrong), and no engagement so far here on EA Forum. Your thoughts deserve more positive attention.

I agree that almost everything specific that we can imagine AGI delivering, can also be delivered by good, safe, aligned, narrow AI -- whether that's innovations in regenerative medicine and longevity, or better methods for geoengineering to ameliorate climate change, or software to fight against mass-customized political propaganda, or whatever. The people saying that only AGI can deliver what we want often seem pretty unfamiliar with how narrow AI has already been helping to advance research in many specific areas. (Indeed, almost every piece of software that we routinely use in the behavioral sciences -- including Google search, Web of Science, Qualtrics, and SPSS stats software -- would have been considered 'advanced but narrow AI' by 1970s standards.) 

An analogy to cell differentiation in organic life might help to clarify the relative pros and cons of narrow AI versus AGI. Pluripotent stem cells are kind of analogous to AGI: they are powerfully 'general purpose' (in some sense), so they can differentiate into any other kind of cell, to grow into any kind of tissue in any kind of organ. But stem cells are especially likely to turn into cancers that kill the whole body. To avoid this, organic life evolves so that most stem cells early in fetal development commit to differentiate into specific kinds of cells, with much more limited options for further differentiation, built-in limits on further replication, and gene-regulatory limits on what portion of the proteome they are permitted to produce. But these cells that are 'committed to narrowness' are actually much more useful in specific adaptations (as neurons in brains, erythrocytes in blood, or osteoblasts in bone) -- and much less likely to turn lethally cancerous. 

So, out of the 30+ trillion cells in the adult human body, the vast majority are 'narrow-purpose', differentiated cells committed to some particular 'cell fate', working away doing their little jobs in some particular organ -- whereas only about 200,000 cells are kept in reserve as stem cells, sequestered away mostly in bone marrow. That ratio -- about 150 million 'narrow cells' for every 'general-purpose' stem cell -- has proven evolutionarily prudent. 

But of course, that doesn't mean we can afford one AGI for every 150 million narrow AIs. Evolution has had hundreds of millions of generations of multicellular evolution to solve the 'biological alignment problem' of managing the stem cell/cancer risk problem. We might want to keep the ratio closer to zero AGIs for every million ANIs, until we have a much better idea what we're doing. (This point is loosely related to my recent essay on 'biomimetic alignment', here.)