A disentanglement project for the nuclear security cause area

By Sarah Weiler @ 2022-06-03T05:29 (+16)

Hi all :)

Over the last few months, I participated in a part-time research fellowship, in the course of which I made a very exploratory attempt to “disentangle the nuclear security cause area”, i.e. to get a better and more clear model of nuclear risks and ways to mitigate them (as an individual or group/community/movement). Below is a summary write-up of what I did and what came out of it, as well as a list of thougths and questions I have on methodology, and some ideas for further investigation (a full compilation of my work and findings can be found in this GoogleDoc). Feedback (no matter how "unpolished) on any of these sections, and especially on the questions and ideas for further research, is highly welcome!

Huge thanks go to the two mentors I had during the fellowship, as well as to a close friend (not on the Forum) who provided feedback and helped make the ToC diagrams look a lot less muddled and ugly.

A disentanglement project for the nuclear security cause area - A short(-ish) Summary

The idea for this investigation was developed in response to the confusion and open questions I encountered while trying to figure out how I conceive of nuclear risks and potential ways to reduce them: During the first half of a research fellowship I did this spring, I wrote up my thoughts on the pathways by which a nuclear weapons-induced existential/extreme catastrophe may be averted, and on the strategies a community like EA could adopt to help push the world onto these pathways (see this section in the GoogleDoc). I was able to list a number of intermediate goals (e.g., arms reduction treaties) and broad intervention options (e.g., advocacy for certain policies) that seem relevant for reducing nuclear risk; however, I didn’t succeed in building a neat theory of change, nor in coming up with clear-cut, actionable recommendations for how to work on nuclear risk reduction. To quote myself (from the summary of the write-up linked to above):

“the list [of intermediate goals and broad intervention options] alone doesn’t yield many workable insights; in order to figure out how to contribute to the highest-level goal (reducing global catastrophic risks related to nuclear issues), I would need to have some model of how specific actions/interventions contribute causally to chains of events that eventually impact on the highest-level goal. I am currently very uncertain about what such a model would be and about how to come up with such a model. This seems a crucial sticking point preventing me from figuring out how I can and want to contribute to this cause area.”

Based on comments from and a subsequent conversation with my mentor (Luisa Rodriguez), the following research idea was developed as a means to get closer to a “model of how specific actions/interventions contribute causally to chains of events that eventually impact on the highest-level goal [of averting nuclear catastrophe]”: Study how people who already work on nuclear issues/risks think about the topic; try to identify the theory of change that underlies their efforts, the goals they pursue, and the reasons for doing so; try to reconstruct the model they use to decide what to work on, and see if that can inform your own model. The shortened label I gave that research interest/aim is “identify existing nuclear risk world views”.

I then started with a shallow and broad investigation to see whether any classifications of “nuclear risk world views” exist already in the academic literature. That investigation could certainly be expanded upon and there were some leads to suggest that a more comprehensive search of the academic literature could help generate/discover a good (theoretically and empirically well informed) classification.

However, because I wasn’t sure how long the more comprehensive survey would take (and whether it would eventually lead anywhere) and because the time period for the fellowship was limited, I decided to pick out three specific organizations active in the US policy sphere and only study their particular views on nuclear risks (for now): the Nuclear Threat Initiative (NTI), the International Campaign to Abolish Nuclear Weapons (ICAN), and the RAND Corporation.

The three case studies I then compiled are still fairly shallow and could benefit from a more in-depth investigation and analysis, but they do give a sense for where research of this kind could lead and how it can be approached. I put together an illustration of the Theories of Change that the work of these three organizations seems to be guided by (see example below), set up an extremely preliminary draft of a database to store and catalog different goals and actions (as well as organizations that pursue them), took note of the challenges I faced during the investigation, and came up with a list of follow-up research projects to push this disentanglement enterprise forwards.

Meta: Thoughts on the challenges and limitations of this project

• Information overload and challenge of bringing it all together:
  • I’m still unsure about the appropriate way to organize and present information. Should I focus on the figure (plus some added explanation in-text)? Or is it better to summarize the goals pursued in a table? Or should I try to construct an even simpler theory of change first and then focus on identifying assumptions and crucial considerations therein? Should I go beyond what’s on the webpage and focus on the goals/actions and debates/questions which they neglect?
  • It’s hard to identify what the different goals are and how they hang together (Which goal is part of which other goal? How are the goals “ordered”, i.e. how do they feed into each other?)
  • It’s hard to give a concise outline of the reasoning behind the overall theory of change, and behind each goal - how long should that outline be? 
  • It’s hard to know whether/how far to dissect the reasoning employed to justify the organization’s work
  • A much more fine-grained differentiation and description of goals and how they hang together could be derived from the organizations’ webpages (see incomplete tables here). This seems like it could easily grow out of hand, i.e. become too complicated and/or meandering to be of much use (on the other hand, I also feel drawn towards that more thorough approach, since the topic seems too important to treat it on a shallow level). I didn’t pursue this further for now, but remain unsure about what would be the optimal/a good way to organize information from the organiazations’ websites.
  • Since RAND isn’t focused solely or mainly on nuclear weapons, the information gathering and retrieval task is quite different from the other two orgs. Interestingly, the task kind of feels easier that way (because there’s less information, thus less of a challenge to bring info together and extract core takeaways?)
• What to do about stuff that is mostly missing from the figures and which isn’t really on the websites (but which seems relevant for any model of nuclear risks)?
  • Partly on the websites: Assumptions about how different stages in the theory of change hang together (i.e., why certain intermediate goals lead to higher-level goals)
  • Not on websites: Reasons for prioritization of goals and interventions
  • Not on websites: List of alternative goals and actions, and reasons why they are not prioritized

Ideas/Plans for further work

• More sophisticated methodology to study existing “nuclear risk world views”
  • Do a more comprehensive survey of the academic literature
  • Conduct interviews to complement the webpage investigation
  • Analyze a more diverse set of organizations and/or actors
  • Create/Develop a database on possible goals&actions, and their (causal) interactions
    • This is one starting example/trial for how that could look
      • One table with a row for each org/actor, and cols for formal info (location, type of org, etc) and multi-select cols for high-level goals, intermediate goals, and actions taken; maybe also a col for "anti-goals"
      • One table with a row for each action/goal, and cols for its type (goal in itself, instrumental goal, action), the goals it precedes causally, maybe the goals/action that precede it, and the orgs/actors that pursue and those which counteract it
      • Link those tables 
      • Input lots of orgs (incl. from different countries, private and public, etc.)
      • Then get stats for which goals are pursued how often (and at which level) → network analysis
      • But: what does the frequency with which different nuclear risk world views are held tell us? Is a goal more legitimate because multiple orgs/actors pursue it?
    • Could also take a very different form, tough (e.g., an epistemic map as suggested in this Forum post?). Uncertainty on how to do this effectively/usefully remains high.
  • Conduct a qualitative analysis of the theories of change of existing organizations/actors
    • Single out a few key orgs or actors
    • Give a detailed description of their theories of change
    • Scrutinize the reasoning that underlies their theory of change
    • Compare between these orgs and identify possible crucial considerations/points of contention
    • Give recommendations/thoughts for how the approach of these orgs/actors ought to influence mine/that of EA/that of people in general
• Find some other way to further disentangle nuclear risks and ways for reducing them 
  • E.g., the Nuclear risk research ideas compiled by Michael Aird and Will Aldred, and especially their proposed project of researching Intermediate goals for nuclear risk reduction
  • But could also be something completely different, not necessarily affiliated to or inspired by the effective altruism movement (a diverse range of perspectives on the topic seems to have high potential value [though could also backfire and create more confusion than illumination; but that’s impossible to tell in advance, and I think we should just try and find out])

See also the open questions that I compiled when first writing up / trying to disentangle my thoughts/model of nuclear security as a cause area (here).

What next?

I might work on one or several of the listed follow-up projects during the upcoming summer (as part of the CERI Research Fellowship) or at a later point, but I also strongly encourage and animate readers who are interested in the nuclear security cause area to consider contributing to the disentanglement goals I outlined themselves, either by taking up an idea from my list of suggested projects or by coming up with your own. My main takeaway from this investigation is that reducing nuclear risks is a daunting task, for which many decision-relevant questions remain unresolved, but also that we can make progress on finding ways to tackle the challenge effectively. I thus believe that there would be quite a lot of value in having people with different backgrounds and perspectives take a close look at the problem and offer their ideas and attempts to enhance our understanding of it!

If you don’t have the time or interest to do a full research project on the topic, there’s another way that you could contribute: let me (and others on the Forum) know about your reaction (thoughts, comments, criticism, questions, ideas, etc.) to what I did during the Fellowship in the comments below (or as comments in the GoogleDoc). Any feedback is greatly appreciated :)!