Statement on Pluralism in Existential Risk Studies

By Gideon Futerman @ 2023-08-16T14:29 (+29)

This is a copy of the English version a statement released yesterday by a group of academics that can be seen at https://www.existentialriskstudies.org/statement/. The Spanish translation, by Mónica A. Ulloa Ruiz, will be put on the forum soon

This statement was drawn up by a group of researchers from a variety of institutions who attended the FHI and CSER Workshop on Pluralisms in Existential Risk Studies from 11th-14th May 2023. It conveys our support for the necessity for the community concerned with existential risk to be pluralistic, containing a diversity of methods, approaches and perspectives, that can foster difference and disagreement in a constructive manner. We recognise that the field has not yet achieved this necessary pluralism, and commit to bring about such pluralism. A list of researchers who support the statement is included at the end. In the spirit of pluralism, whilst we signatories do not all necessarily endorse every point contained within the statement, we stand sufficiently behind the thrust of the statement that we think it is well worth making. 

We believe it is important that the community concerned with existential risk is well adapted to its purposes, and that pluralism is necessary for this community to achieve various  epistemic, ethical and pragmatic goals that each of us hold to be important.

Existential risk, as a concept, is defined in a multitude of ways in our community. Some in this group think it to be ‘risks of human extinction, societal collapse and other events associated with these’^[1], others conceptions involve ‘risk of permanent loss of humanity’s potential’^[2] or the risk of the loss of large amounts of expected value of the future^[3]. Some see it as an inseparable part of the broader class of risks to the existence of individuals, communities or specific 'worlds'^[4]. While differing definitions of existential risk can and do lead to relevant divergence,it also seems we (the signatories) regularly have enough of a shared interest in working together, learning from each other (even through our disagreements) that being part of the same, pluralistic community appears to be mutually beneficial. This needn’t mean we will come to the same conclusions, or always agree; rather it suggests that we would all benefit from the existence of a healthy, pluralistic intellectual ecosystem concerned with issues around the plurality of conceptions of existential risk. 

We disagree, or take different approaches, on a lot. We differ on the methods we think best for studying existential risk^[5]. We differ on the epistemic aims of such research^[6]. We differ on what should be accepted as evidence^[7] and how this evidence ought to be interpreted^[8], evaluated^[9] and aggregated^[10]. We differ on what core assumptions should underpin our visions of existential risk^[11] and how to research them^[12]. We differ on our visions of the future. We differ as to what is ethically important or acceptable^[13]. We differ on our politics and how this relates to our thinking and practice with regards to existential risk. We have different methodologies, epistemologies and ethics. 

These differences do not emerge from nowhere; they are both a feature of intellectual differences between schools of thought and research cultures, as well as a feature of deeper global structural factors such as inequality and deep rooted cultural difference^[14]. Indeed, one of the reasons these deep disagreements are often obscured is because the same global structures of inequality also lead to many important voices that give expression to such different perspectives not being present in the field. Thus, any movement towards pluralism ought to acknowledge these structural forces and the role they have played in the development of the field and the production of existential risk thus far. Preventing  the reproduction of these structures will be essential to broadening the conversation around existential risk; why such broadening is important in the aim of reducing existential risk will be discussed below. 

Differences are a strength. We think our community can be better by each of our own lights through a plurality of approaches, even those about which we disagree amongst ourselves. Further, we suggest that the power to confer support for different approaches should be distributed among the community rather than allocated by a few actors and funders, as no single individual can adequately manifest the epistemic and ethical diversity we deem necessary. This position is justified by the following reasons. 

Firstly, we recognise the high level of uncertainty^[15] involved in studying a large class of existential risks. Much of our reasoning is based on novel ideas and unexplored models of the world, often with sparse or heavily contested evidence to support the assumptions that fundamentally underpin them^[16]. Whilst it is possible to make evidence-based claims, evidence seems so sparse that no decisive case can be made for a single approach. It may be equally as reasonable, given the sparse evidence available, to base enquiry off a variety of different assumptions. As such, it is prudent to adopt a pluralistic approach, to allow us to ‘hedge our bets’ and develop a variety of lines of inquiry. Moreover, under such uncertainty, scientific creativity is important^[17], which thrives better in pluralistic communities than in homogeneous ones. We are still uncertain with regard to which questions to ask, let alone the correct methods and assumptions to underpin the process of answering them, that taking many approaches appears to us the correct course of action.

Secondly, disagreement can be constructive, so a pluralistic community can allow us to make discoveries or develop ideas none of us individually would have reached. This is not to suggest that these new ideas come from convergence towards a singular truth, but rather that the process of disagreement, and the exposure to new concepts, can play a key role in the epistemic process.  Creating an intellectual ecosystem that encourages cross-fertilization through encounter and disagreement may help to clarify useful courses of action for mitigating existential risk, whilst avoiding premature convergence; indeed, convergence on a single course of action may be imprudent given our epistemic situation^[18].

Thirdly, having a pluralistic community can be instrumentally useful, to allow us to understand how to reduce existential risk from several perspectives, and allow us to build coalitions to combat risk. Much of existential risk reduction is political^[19], so diverse and broad based coalitions can give us a useful political basis for action. Many different types of existential risk may have similar political causes^[20], and a pluralistic community may open up new avenues for collaboration with those whom we each have common cause^[21]. This is not to say this will always happen; indeed, an important aspect of pluralism is its openness to a proliferation of possibilities; but having a pluralistic community will allow for fruitful possibilities for collaboration when unified political aims are present. Thus, in the conditions where it is possible, the pluralistic community would be the site for incompletely theorised agreements of necessary political action. 

Finally, there are also reasons to want a community where a plurality of normative ideas are represented. While many of us are deeply concerned with the idea of building futures consistent with each of our best ethical theories, we all have at least some degree of ethical uncertainty about what views are best. It may be that different ethical theorists are all “climbing the same mountain”, or it may be that they are deeply and irreconcilably divided. However, none of us are able to answer this question with certainty (indeed, many of us think this is unanswerable). Nevertheless existential risk, and how we should respond to it, is deeply bound up with our normative beliefs about what would be good and bad^[22]. Highlighting, collaborating with or respecting people of different ethical viewpoints is therefore vital to working together within a pluralistic community. Indeed, having a variety of ethical perspectives heard and empowered may be vital to the achievement of justice within existential risk, an ethical priority that for many of us is crucial.

We want to acknowledge, draw benefit from, and support the existing pluralism of the community, and to actively cultivate a greater plurality of perspectives, methodologies, worldviews, and values. In the academic community, this will mean exploring and encouraging disciplines that are often neglected in studying existential risk, such as science and technology studies (STS), political science, peace and conflict studies, archaeology, anthropology and sociology. It may mean utilising existing methods from a wider range of disciplines and attempting to innovate our own methods. It also means genuine respect for and inclusion of those people who are outside of groups typically considered ‘experts’. This will also likely require more care put into publication practices for those engaged in research, including greater consideration and conversation of the appropriate role of different fora in the epistemic process, including peer reviewed publications, than has currently been the case to date. A greater focus on creating a research culture that actively promotes creativity and diversity of evidence-based thinking will be key. 

Moreover, all of this entails greater geographic, socio-economic, cultural, gender, racial and ability diversity, both in terms of those who may have interest in being a part of the community, and those whom the community may learn from. In particular, providing opportunities for those outside the traditional centres of the community concerned with existential risk (the USA and Northern Europe) to engage as part of or in dialogue with the community we are creating is vitally important. This diversification and increased inclusion would only be beneficial if engaged in a manner of mutual respect and equality rather than attempting to leverage existing power imbalances to allow for a particular perspective to dominate; it cannot just be on the terms of those of us with already entrenched privilege. Dialogue and openness towards those with different ethical perspectives, particularly non-Western ethical perspectives, different definitions of existential risk and different bases with which they start reasoning about existential risk from should also be a key priority; expanding the community such that we can learn and collaborate with those from a diverse range of perspectives is ethically and epistemically important. 

We recognise that this vision will have tradeoffs. A community like this will take time, money and energy to help build and maintain that could be used for other purposes. A more diverse and pluralistic community would involve the ceding of some power within the community by those with the most influence and authority at present, and a ceding of the dominance of the present dominant viewpoints; such a process will be difficult. Engaging with a diversity of viewpoints will compromise some of the unity that exists in the field, which will inevitably lead to more conflict than at present, and reduce the ability of the field to rapidly converge on a set of similar viewpoints, including around prioritisation. Nonetheless, pushing the field in the direction of greater pluralism is worth such a cost given the benefits for the epistemic and ethical health of the community in achieving its plurality of purposes. 

We thus declare our support for moving the existential risk community towards greater pluralism, and acknowledge our responsibility in trying to bring this about. We have only given a rough sketch of what pluralism must entail, and hope that the process of building such a community will help concretise what it means in practice. We don’t wish to merely respect each other at a distance, but desire more active engagement, sharing ideas and disagreement with people from across the community, and empowerment of each other to take different approaches. We don’t merely wish for the perspectives of those of us with the most power to dominate, but rather to create a culture where a genuine proliferation of evidence-based insights can occur. This requires support in many different forms from different actors in the community, which we are optimistic can happen; if we are to bring about a less endangered world, it must. 

Signed by:

Gideon Futerman (University of Oxford, Coordinator and principal author of this statement)

SJ Beard (Senior Research Associate, Centre for the Study of Existential Risk, University of Cambridge)

Anders Sandberg (Senior Research Fellow, Future of Humanity Institute, University of Oxford)

Paul N Edwards (Director, Stanford Existential Risk Initiative, Stanford University)

Erica Thompson (Associate Professor, Department of Science, Technology, Engineering and Public Policy, University College London)

Thomas Meier (Director, Centre for Apocalyptic and Post Apocalyptic Studies, University of Heidelberg)

Paul Ingram (Senior Research Associate, Centre for the Study of Existential Risk, University of Cambridge)

Matthijs Maas (Senior Research Fellow, Law and AI and Head of AI Research at the Legal Priorities Project)

Florian Ulrich Jehn (Senior Researcher, Alliance to Feed the Earth in Disasters

David Thorstad (Postdoctoral Research Fellow, Global Priorities Institute, University of Oxford)

Karim Jebari (Institute for Futures Studies)

Mónica A. Ulloa Ruiz (Policy Transfer Officer, Riesgos Catastroficós Globales)

Bill Anderson-Samways (AI Governance Researcher)

Ximena Barker Huesca (Kings College London)

Lin Bowker-Lonneker (University of Oxford)

Nadia Mir-Montazeri (University of Bonn)

Phillip Spillman (University of Cambridge)

Noah Taylor (Peace Studies Researcher)

1. ^{^}

    Beard, S., Rowe, T. and Fox, J. (2020) ‘An analysis and evaluation of methods currently used to quantify the likelihood of existential hazards’, Futures, 115, p. 102469.

2. ^{^}

   Bostrom, N. (2002) ‘Existential risks: analyzing human extinction scenarios and related hazards’, Journal of evolution and technology / WTA, 9. 

3. ^{^}

   Cotton-Barratt, O. and Ord, T. (2015) Existential risk and existential hope: Definitions. Available at: http://files.tobyord.com/existential-risk-and-existential-hope.pdf (Accessed: 22 June 2023).

4. ^{^}

   Mitchell, A. and Chaudhury, A. (2020) ‘Worlding beyond “the” “end” of “the world”: white apocalyptic visions and BIPOC futurisms’, International relations, p. 004711782094893.

5. ^{^}

   Such as the different methods laid out in: Beard, S., Rowe, T. and Fox, J. (2020) ‘An analysis and evaluation of methods currently used to quantify the likelihood of existential hazards’, Futures, 115, p. 102469.

6. ^{^}

   Including how important generating probabilistic forecasts are, the importance of specific causal-chain threat models, the utility of speculation and more.

7. ^{^}

   For example, how far individual case studies can be taken as evidence.

8. ^{^}

   For example, whether worries of tipping points and unknown feedbacks can be taken as evidence for existential risk or not.

9. ^{^}

   For example, how far we should rely on the updating of  individual credences by a small number of decision-makers.

10. ^{^}

    For example whether we can or should adequately differentiate between GCR and XRisk, or how likely we are to identify correct threat models and hazards in advance.

11. ^{^}

    For example whether looking at sub-existential catastrophes as evidence of cascades dominating will appropriately scale to existential catastrophes.

12. ^{^}

    For example whether looking at sub-existential catastrophes as evidence of cascades dominating will appropriately scale to existential catastrophes.

13. ^{^}

      For example, with regards to how much we agree with strong longtermism

14. ^{^}

     Ibid 4

15. ^{^}

    Currie, A. (2019) ‘Existential risk, creativity & well-adapted science’, Studies in history and philosophy of science, 76, pp. 39–48.

16. ^{^}

    Sundaram, Lalitha, Matthijs M. Maas, and S. J. Beard. ‘Seven Questions for Existential Risk Studies’, 25 May 2022. https://doi.org/10.2139/ssrn.4118618.

17. ^{^}

     Ibid 15

18. ^{^}

    Ibid 15

19. ^{^}

     In the sense of being involved in addressing and utilising power in public life.

20. ^{^}

    Cotton-Barratt, O., Daniel, M. and Sandberg, A. (2020) ‘Defence in Depth Against Human Extinction: Prevention, Response, Resilience, and Why They All Matter’, Global policy, 11(3), pp. 271–282.

21. ^{^}

    Such has been discussed within the space of people discussing the risks and harms of AI: Stix, C. and Maas, M.M. (2021) ‘Bridging the gap: the case for an “Incompletely Theorized Agreement” on AI policy’, AI and ethics, 1(3), pp. 261–271.

22. ^{^}

    Cremer, Carla Zoe, and Luke Kemp. (2021). ‘Democratising Risk: In Search of a Methodology to Study Existential Risk’. https://papers.ssrn.com/abstract=3995225.

Larks @ 2023-08-16T18:57 (+46)

Thanks very much for sharing this! I definitely agree with the importance of including a broad range of political perspectives. As you know, the EA and existential risk movements are currently overwhelmingly left wing, and appear to becoming more homogeneous over time; do you have any thoughts on what could be done to attract and include more conservatives / Republicans etc.?

 

Gideon Futerman @ 2023-08-16T21:50 (+7)

I think its important to not simplify political pluralism down to simply party politics, particularly in one country, so I'm simply not sure this simple left-right axis can at all accurately conceptualise the politics of the space and I think if we are to fully capitalise on the politics of this space and how to diversify them, we really need to look more expansively.

Addressing your question directly, I firstly don't claim to know or have all the answers, although I can think of places it would definitely be useful. One potential example would be to develop a political philosophy of xrisk reduction based on Burkeian ideas, which I think may be a useful addition to an 'agents of doom' agenda. I'm sure other intellectual currents in conseravtive thought could also be useful, particularly given how inherently conservative the idea of XRisk reduction is (eg see Mitchell and Chaudrey's paper, and I actually agree this inherent conseravtivism is problematic) and so conservatives ought to have a lot to say about xrisk reduction. 

John G. Halstead @ 2023-08-17T08:18 (+17)

In the post, you say "Much of existential risk reduction is political[19], so diverse and broad based coalitions can give us a useful political basis for action. Many different types of existential risk may have similar political causes[20], and a pluralistic community may open up new avenues for collaboration with those whom we each have common cause[21]." It does seem strange not to say in the main post that apparently almost all EAs would vote probably vote Labour or Democrat, so clearly something is amiss here by your own lights

Gideon Futerman @ 2023-08-17T08:46 (+8)

I do basically agree that we should have people who we would say are on the right (hence my suggestion) and I can see why my previous comment may come across dismissive (apologies), I just don't agree that breaking it down to party politics(eg Republican) or left-right given how broad it is is necessarily that useful. (Again, please remember I'm not just talking about EA but ERS, but I think your point still applies)

Ben_West @ 2023-08-17T21:53 (+3)

I just wanted to say thanks for taking the question seriously, even though you don't fully endorse the frame!

quinn @ 2023-08-17T09:18 (+2)

what's the inherent conservatism of xrisk reduction, or the version that's salient to you, or to Mitchell/Chaudrey? I know of a few approaches. I guess Mitchell/Chaudrey might plausibly think the "everyone dying bad" frame solves the whiteness problem (i.e. someone like Ord throws in a bunch of extra stuff on top which you could accuse of a kind of WEIRD supremacy / insufficiently anticapitalist / etc). I remain slightly confused why "positive longtermism and negative longtermism" have not been adopted since I'm frequently aggressively reminded that it's a source of cruxes for different people. 

FJehn @ 2023-09-23T06:29 (+3)

Just a thought here. I am not sure if you can literally read this as EA being overwhelmingly left, as it depends a lot on your view point and what you define as "left". EA exists both in the US and Europe. Policy positions that are seen as left and especially center left in the US would often be more on the center or center right spectrum in Europe.