Risks from Atomically Precise Manufacturing

By MichaelA🔸 @ 2020-08-25T09:53 (+29)

This is a linkpost to https://www.openphilanthropy.org/research/cause-reports/atomically-precise-manufacturing

That link is to a 2015 “shallow investigation” by the Open Philanthropy Project. Their summary reads:

Background: Atomically precise manufacturing is a proposed technology for assembling macroscopic objects defined by data files by using very small parts to build the objects with atomic precision using earth-abundant materials. There is little consensus about its feasibility, how to develop it, or when, if ever, it might be developed. This page focuses primarily on potential risks from atomically precise manufacturing. We may separately examine its potential benefits and development pathways in more detail in the future.

What is the problem? If created, atomically precise manufacturing would likely radically lower costs and expand capabilities in computing, materials, medicine, and other areas. However, it would likely also make it substantially easier to develop new weapons and quickly and inexpensively produce them at scale with an extremely small manufacturing base. In addition, some argue that it would help make it possible to create tiny self-replicating machines that could consume the Earth’s resources in a scenario known as “grey goo,” but such machines would have to be designed deliberately and we are highly uncertain of whether it would be possible to make them.

What are possible interventions? A philanthropist could seek to influence research and development directions or support policy research. Potential goals could include achieving consensus regarding the feasibility of atomically precise manufacturing, identifying promising development strategies, and/or mitigating risks from possible military applications. We are highly uncertain about how to weigh the possible risks and benefits from accelerating progress toward APM and about the effectiveness of policy research in the absence of greater consensus regarding the feasibility of the technology.

Who else is working on it? A few small non-profit organizations have explicitly focused on research, development, and policy analysis related to atomically precise manufacturing. Atomically precise manufacturing receives little explicit attention in academia, but potential enabling technologies such as DNA nanotechnology and scanning probe microscopy are active fields of research.

A key passage I’d highlight is:

Unless APM is developed in a secret “Manhattan Project”—and there is disagreement about how plausible that is —the people we spoke with believe it would be extremely unlikely for an observer closely watching the field to be surprised by a sudden increase in potentially dangerous APM capabilities.

That said, their list of “Questions for further investigation” includes:

How confident can we be that there will be substantial lead time between early signs that APM is feasible and the deployment of APM?

Also on this topic, 80,000 Hours write:

Both the risks and benefits of advances in this technology seem like they might be significant, and there is currently little effort to shape its trajectory. However, there is also relatively little investment going into making atomic-scale manufacturing work right now, which reduces the urgency of the issue.

Why I’m posting this here

The handful of public, quantitative existential risk estimates that exist suggest APM - or perhaps nanotechnology more broadly - may be one of the largest sources of existential risk. (Of course, it's hard to say what conclusions to draw from these estimates, for many reasons.)

It also seems like APM/nanotechnology was among the most prominently discussed existential risks until sometime around 2010, but that it’s been discussed less since then. And I think there has been relatively little public discussion of why that shift in focus occurred. (See also.)

So I think it’d be good if it was a bit easier for people to: 

To that end, I’ve made this link post, as well as the tag Atomically Precise Manufacturing.

Feel free to use this comment section for general discussion of whether EAs should pay more attention to APM, why or why not, and what in particular we should consider doing about APM (if anything).

Sanjay @ 2020-08-25T14:34 (+10)

I raised a similar question on the Effective Altruism fb group last year.

Notable responses included the comment from Howie Lempel which reiterated the points in the Open Phil article about how it seemed unlikely that someone watching the field would fail to notice if there was a sudden increase in capabilities.

Also Rob Wiblin commented to ask to make it clear that 80,000 hours doesn't necessarily endorse the view that nanotech/APM is as high a risk as that survey suggests.

MichaelA @ 2020-09-15T09:23 (+6)

It looks like FHI now want to start looking into nanotechnology/APM more, and build more capacity in that area: They're hiring for researchers in a bunch of areas, one of which is: 

Nanotechnology: analysing roadmaps to atomically precise manufacturing and related technologies, including possible intersections with advances in artificial intelligence, and potential impacts and strategic implications of progress in these areas.

Jamie_Harris @ 2021-11-29T09:29 (+5)

That's interesting. As far as I can tell, Eric Drexler was basically the person who kicked off interest + concern about this tech in the 1980s onwards.* His publications on the topic have accrued tens of thousands of citations. But Drexler's work at FHI now focuses on AI.

(I came to this year-old post because some of the early transhumanist / proto-EA content (e.g. Bostrom and Kurzweil) seems to mention nanotech very prominently, sometimes preceding discussion of superintelligent AI, and I wanted to see if any aspiring EAs were still talking about it.)

 

*General impression from some of the transhumanist stuff I've been reading. The Wikipedia page on nanotechnology says:

The term "nano-technology" was first used by Norio Taniguchi in 1974, though it was not widely known. Inspired by Feynman's concepts, K. Eric Drexler used the term "nanotechnology" in his 1986 book Engines of Creation: The Coming Era of Nanotechnology, which proposed the idea of a nanoscale "assembler" which would be able to build a copy of itself and of other items of arbitrary complexity with atomic control. Also in 1986, Drexler co-founded The Foresight Institute (with which he is no longer affiliated) to help increase public awareness and understanding of nanotechnology concepts and implications. The emergence of nanotechnology as a field in the 1980s occurred through convergence of Drexler's theoretical and public work, which developed and popularized a conceptual framework for nanotechnology, and high-visibility experimental advances that drew additional wide-scale attention to the prospects of atomic control of matter.

MichaelA @ 2021-11-29T12:32 (+5)

One somewhat tangential thing you might find interesting is how prominent nanotech seems to be in many of the "Late 2021 MIRI Conversations". Though none of the mentions there seem to be suggesting anyone should try to study or influence nanotech itself, more so that nanotech could be a key tool used by agentic AI systems.