Update on the Simon Institute: Year One

By konrad, maxime @ 2022-04-07T09:45 (+88)

- We thank Jacob Arbeid for his immensely helpful edits & comments.

Summary

A year ago, we launched the Simon Institute for Longterm Governance (SI). In this post, we 

 

Based in Geneva, Switzerland, SI is building the science-policy interface for longtermist engagement with the UN system.[1] The UN, its networks and activities provide international fora for exchange framed under multilateralist ideals. As a source of symbolism with some agenda-setting and policy design power, the UN eases the promotion of common sense longtermism among influential institutions and individuals. However, making valuable contributions requires context-specific knowledge, as well as the right contacts, collaborators and reputation.

Throughout its first year, SI proved its capacity and value in connecting longtermist actors to international civil servants and national delegates, especially through workshops and high-level dinners with e.g. CSER and FHI. SI’s research is attracting valuable attention and has in a first part passed academic peer review. This combination of activities has granted SI mandates from UN institutions, as well as the Swiss government, to directly work on policy processes relevant to existential risk reduction. 

In the next step SI will focus on the development of the UN’s “Our Common Agenda” (OCA) as a particularly impactful opportunity to focus international and national attention on future generations and existential risk. The value of this engagement can come from e.g. identifying allies, contributing to the redefinition of the international policy narrative, and easing other longtermists’ national policy efforts by helping create global reference frameworks.

Due to its positioning and first achievements, SI finds itself with this rare window of opportunity to inculcate longtermism and existential risk reduction at the heart of the institutions that can claim to speak for all humanity, and which can affect the priorities of national governments. Our success is now dependent on growing our funding and talent base, which we hope will mainly come from the value-aligned EA community. We believe we have room for more funding of $ ~4 million until 2025.

1. What is SI?

The vision of the Simon Institute for Longterm Governance (SI) is for governments and multilateral organizations to adequately account for the interests of future generations.

Most pressingly, though, we as humanity need to reduce risks to our flourishing effectively. For all key risks, humanity’s path to existential security cannot be brought about by the actions of any single country, making more effective international cooperation essential. Improving international cooperation requires better management of uncertainty and understanding of technological development by policymakers, as well as better representation of future generations - all while preserving political legitimacy for present generations.

Such improvements require policy change at international levels (e.g. new or modified treaties) and national levels (e.g. treaty ratification and implementation). Longtermist research can and should inform such policy change. But to accomplish that, we need intermediary actors who can build connections to relevant policy ecosystems and translate between stakeholders. SI is one such intermediary actor that the community needs. 

To improve international existential risk governance, SI works at the science-policy interface. We

  1. Engage policymakers by providing space for reflection and advice from external experts.
  2. Support researchers in their policy engagement by providing knowledge of policy processes and opportunities for engagement.
  3. Foster interaction between researchers and policymakers to increase trust and alignment by organizing events and facilitating exchanges.
  4. Conduct and curate research to substantiate and reflect on our efforts.

 

Until at least 2025, we will focus on existential risk mitigation. While long-term governance can focus on many other policy problems (e.g. sustaining progress), existential risk mitigation seems most urgent to advance. We currently focus on general risk governance instead of specific threats (e.g. transformative AI or engineered pathogens). This approach better suits the international discourse and avoids us being branded as too partisan. That said, we do operate within cause-specific networks when connecting longtermist researchers to specific policy audiences, e.g. around the Biological Weapons Convention.

SI focuses primarily on fostering information processing. Unlike many think tanks, SI does not focus its theory of change around producing in-house policy analyses or recommendations. Instead, SI works with policymakers and academics to efficiently translate research into practice. This is why SI could fill a gap in an information-rich but time-scarce environment.

SI’s theory of change

 

SI aims to be policy-relevant. SI is embedded in one of the few international policy hubs - Geneva - and adapts its strategy in response to arising opportunities. 2021 has marked a breakthrough in international policymakers’ responsiveness to longtermist concerns. In the United Nations Office of the Secretary General’s ‘Our Common Agenda’, SG Antonio Guterres states:

We are at an inflection point in history. The COVID-19 pandemic has served as a wake-up call and with the climate crisis now looming, the world is experiencing its biggest shared test since the Second World War. Humanity faces a stark and urgent choice: breakdown or breakthrough. The choices we make — or fail to make — today could result in further breakdown and a future of perpetual crises, or a breakthrough to a better, more sustainable, peaceful future for our people and planet.” 

Toby Ord, as well as staff from CSER and FLI, helped introduce the concepts of “existential risk” and “intergenerational global public goods” to this publication thanks to their personal contacts in the SG Office. Leveraging this window of opportunity, SI works with them to reduce existential risks and further long-term governance via the impact pathways of the international system.

2. Review of year one

In brief, we 

Impact

To illustrate how SI is making a difference and building up further potential, this section highlights three ways in which SI has already proven useful. We asked ourselves “what would have happened without our contribution” to provide counterfactual impact estimates. These are, at best, educated guesses and not to be taken with high degrees of confidence.[2] For a simple list of all outputs, see the appendix.

 

We played a significant role in at least two important policy engagement projects of key longtermist research organizations.

Workshop with CSER to build a Science-Policy Interface for GCR Reduction

Researchers and policy brokers learning to speak each other’s language

 

Dr. Clarissa Rios Rojas from the Centre for the Study of Existential Risk (CSER) reached out to us to co-develop a workshop series bringing together a selected audience of more than 30 global catastrophic risk (GCR) researchers and policy brokers in the disarmament and disaster risk reduction sphere. The goal was to contribute to CSER’s ongoing efforts to build a science-policy interface for GCRs (abbreviated ‘GCR-SPI’). We were well-positioned to co-design this workshop due to our understanding of CSER’s work and mission, as well as our practical experience with current affairs and common frames in international policy.

Clarissa reported that our joint brainstorming sessions, together with our framings and workshop facilitation were instrumental in the process of designing a productive GCR-SPI group. The participants also reported that they were surprised by the opportunities and insights created through the workshop. SI’s experience discussing GCRs with international policymakers and academics allowed for a more effective bridging and detection of gaps in understanding between researchers and policy practitioners, as well as between the longtermist field and more established paradigms.

The workshop resulted in improved relationships between almost three dozen science-policy actors and academic researchers. Some have expressed interest in shifting their personal career’s focus on neglected GCRs and are discussing applications to appropriate funding programs with us. Some international organizations, like the UN Office for Disaster Risk Reduction (UNDRR) and the International Science Council, also explicitly stated an interest in developing a better understanding of GCRs as a result.
 

High-level dinner with senior leadership at the BWC’s Meeting of State Parties

Getting a feel for the public diplomatic discourse around the BWC

 

The Future of Humanity Institute’s Biosecurity Research Group (FHI Bio) reached out to us for their visit to the Biological Weapons Convention’s Meeting of States Parties in Geneva in November 2021. Beyond supplying them with practical information, we organized dinners and lunches to create and strengthen connections between us, other researchers, young diplomats, and senior leadership in biosecurity. A longtermist biosecurity grantmaker reported that they would have paid approximately $ 50,000 for the high-level dinner with senior leadership from key national delegations and international organizations. Our expenses for this dinner, including staff time, were between $ 6,000-7,000.

Without SI, FHI Bio would have been less likely to get an in-depth look behind the scenes of international diplomacy and unlikely to connect to key stakeholders in a setting where personal discussion allowed for the divulging of insider information and personal opinion. These insights and contacts have already proven useful to other long-term-oriented endeavors, too.

 

We have made at least some positive difference through various other activities.

Our working paper “Policymaking for the Long-term Future: Improving Institutional Fit” has been received with great interest and positive feedback from practitioners and academics alike. We are on track to publishing an academic paper on the framework to enable further research on how to advance the spatial, temporal, functional, and representational fit of governing institutions to safeguard the long-term future. We are discussing the practical implications of these dimensions of long-term institutional fit with the Swiss government. This validates at least some of SI’s theoretical basis and capacity to produce practically useful knowledge. 

Another paper, titled “A Computational Turn in Policy Process Studies: Coevolving Dynamics of Policy Change”, has been accepted to the journal Complexity. Building on a critical review of the application of complexity theory to policy process studies, we present and implement a baseline model of policy processes using the logic of coevolving networks. The editor and reviewers have congratulated us for a unique piece that initiates important reflections. This suggests we are capable of building on and contributing to the study of policymaking in a technical, hypothesis-driven way. This research is not meant to be directly useful for policymakers, but rather serves as a way to document and contribute our knowledge to enable further research. 

We have added further value to CSER’s work through our ongoing engagement. We have brought in CSER’s Lalitha Sundaram for a talk on global catastrophic biological risks (GCBR) at the Biological Weapons Convention’s Meeting of Experts in Geneva in Summer 2021, which was welcomed by the BWC’s Implementation Support Unit as a way of increasing the issue’s visibility. We have discussed international policy engagement with several of CSER’s researchers and how to increase the usefulness of their work during a visit to Cambridge for a presentation of SI. These exchanges have contributed to their discussion of policy impact, showing how existing projects could be made useful to key policy audiences, and how they could plug into our network.

Similarly, our exchange with leadership and researchers at Oxford's Global Priorities Institute seems to have contributed to minor adjustments to their plans. A well-attended presentation of our work to >15 GPI researchers indicated lively interest from leading academics of direct relevance to SI’s goals. In discussion, we were able to highlight how rigorous research and its authors can support longtermist policy engagement through public backing. Hilary Greaves reported that she will help clarify concepts of existential risk for effective policy engagement as a result.

Career changes have plausibly been another result of our work. 80,000 Hours, Effective Thesis, and others in the EA community have been forwarding promising individuals to us for advice (~33 calls). Our advice is at least perceived as helpful. Examples range from a young leader deciding to stay in a major French think tank instead of moving on; a promising young scholar deciding to pursue a public policy master's instead of going into politics; and a research consultant who decided to make their work more policy-relevant. We expect most of the value to show only after a few years.

We have extensively engaged with and advised other policy-related projects on their strategy, communications, or the content of their workshops. We are actively coordinating with key policy actors and EA community leaders to monitor and reduce potential harm from risky activities and actors. We also, on a roughly weekly basis, receive inquiries about job openings, internships, or our Longterm Governance Network from junior talent looking for mentorship and exchange, indicating a currently unserved need for more such network support.

 

We have strengthened our network and knowledge base

The development of our tabletop exercise on pandemic preparedness opened doors with potential collaborators, substantially deepened our knowledge of GCBRs and their governance, and synthesized a lot of knowledge into an engaging 3h format to learn about the governance of engineered pathogen outbreaks.

We also gathered useful knowledge on the challenges to better risk management, as perceived by policy actors, by conducting 28 expert interviews for training on existential risk and decision-making at the UN High-Level Political Forum in July 2021.

Roadblocks and lessons learned

1. Fundraising was more difficult than expected

Our main fundraising targeted the EA Funds, resulting in six proposals for a total of ca. $ 600,000. Of these proposals, only $ 46,000 was granted. Another application to the Survival and Flourishing Fund for network-building activities was rejected without comment. Founders Pledge investigated our plans and concluded SI to be promising but with an insufficient track record for a recommendation.

The main reason for worry seems to have been our limited track record in downside-risk-conscious policy engagement, given SI’s ambitious, public-facing plans. Thus, the EA Infrastructure Fund decided to fund a minimal version of SI to first gather more data for a more telling evaluation after March 2022. 

As institutional credibility is necessary to execute our plans, applying as an organization was necessary but made SI appear too risky of an investment. An organization is harder to evaluate than two individuals and funders worried it would be harder to shut down in case of failure as they did not have the capacity to properly vet all involved parties.

To ensure strategic alignment and coordination between SI and value-aligned funders, we invested in months-long exchanges and accepted the condition of not pursuing other grants above CHF 100,000 until March 2022. From September 2021 onwards, we thus deprioritized fundraising and focused on delivering high-quality outputs with the available manpower.

We have signed a grant agreement of CHF 50,000 from the Swiss Government’s International Public Law Division for a project on existential risk governance led by SI’s board member Igor Linkov. Our collaboration with the Geneva Science-Policy Interface has yielded another grant agreement of CHF 30,000 for work on the tabletop exercise on pandemic preparedness. We have also received generous donations from several private donors, amounting to CHF ~68,000. 

Since February 2022, our impact and learning have made us sufficiently confident to start applying for larger grants again. We recently have handed in proposals to EA Funds, the Survival and Flourishing Fund, FTX Future Fund, and are in touch with Effective Giving.

 

2. Deployment of in-person programs was almost impossible

Because of COVID-19, key events with the UN in March 2021 and at the High-Level Political Forum in July 2021, as well as others were converted to online programs. This made participation and hosting sessions less valuable, especially since our intensive in-person formats did not fit the shortened time slots. 

The only public in-person event we could run was Lalitha’s talk next to the Biological Weapons Convention’s Meeting of Experts. Due to COVID-19 restrictions, turn-out and interactions remained limited but feedback was positive. The only other major in-person event was the successful high-level dinner on GCBRs next to the BWC’s Meeting of States Parties.

Additionally, running our table-top exercise on pandemic preparedness proved more difficult than anticipated. Beyond taking a significant amount of time out of their busy days, many diplomats would also have needed security clearance to participate. Therefore, we will need to adapt our deployment strategy to enter the vetted spaces of our target audience instead of getting them into ours. While those spaces are more often national than international, we believe local missions and training programs to be a promising starting point for deployment in Geneva.


 

3. Communicating our work has so far traded off against generating results

Our theory of change, centered around the UN system, is difficult to understand without knowledge of the context within which we operate as an organization based in ‘international Geneva’. Further, many components need to come together for SI to have an impact. While we have gathered evidence that our theory of change is valid, it remains challenging for outsiders to judge. We are planning to change this by publishing further details of our models this year. 

Moreover, throughout the year, we have produced a lot of outputs - from interviews to events and publications - but did not advertise them beyond their immediate use case. They were thus nearly unobservable to the EA community. 

We usually could not go the extra mile to promote what we developed beyond its immediate audience. For instance, we have not posted on the Forum since our founding. We have received feedback from external partners that it is surprising how much we do with very little funding. We’re looking to build the capacity to process and share our work more widely, through more blog and Forum posts, as well as additional content on other platforms.

Furthermore, as SI works at the interface between existential risk research and related policymaking, we have to frame our work differently to various sides and communicate these choices transparently to all sides. Some framings appear insufficiently technical to existential risk researchers while others appear too theoretical to our policy audiences. We are planning to conduct message testing studies to improve cross-context coherence, beyond local salience.
 

4. With the right framings or opportunities, we can focus directly on long-term issues

Existential risk research lacks many accessible examples and policy recommendations to articulate a constructive, positive narrative around existential risk mitigation. Talk about existential risk often puts off overly optimistic policy actors because they want to focus purely on progress. Conversely, proposed solutions to existential risk are often perceived as too reliant on techno-utopian worldviews, which are incompatible with many policy actors’ understanding of a sustainable lifestyle within planetary bounds.

Different UN agencies and conventions rely on different narratives (e.g. security, global development, sustainability, human rights, etc.). SI needs to adapt to different narratives, depending on the context (e.g. which UN Convention, member-states, history). A potential negative consequence of this reality is that SI will end up working with different framings for different audiences, which may cause confusion. However, this negative consequence seems less likely if we accompany these adaptations in messaging with efforts to foster cross-contextual understanding. 

Successful existential risk mitigation is thus highly dependent on the translation of the research into practical recommendations for policy change that are motivating and coherent across cultures and institutions. COVID-19, and more recently the Ukraine crisis and accompanying nuclear risks, have in this sense been a blessing in disguise. They provide reference points of global salience to exemplify to policy audiences how things could easily be even worse. 

Building on these reference points, the UN’s Our Common Agenda presents a rare opportunity to directly work on future generations and existential risks in national and international political agendas. This is because they are explicitly introduced in the report and 17 out of its 69 proposals are directly relevant to longtermist goals. We need surge capacity to take advantage of the upcoming windows of opportunity it provides. Blog posts, talks, interviews, etc. are likely cheaper and plausibly more effective than academic articles for promoting ideas. 

Continuous, iterative engagement with our target audience will be needed to ensure that longtermist considerations are well-understood. Many policy actors understand risk distributions even less than we initially thought, which makes the case for prioritizing existential risks difficult (“why focus on unlikely things when 99% of what’s already happening seems dangerous, too?”). Building understanding requires sustained exchange with policy actors, or common authoritative reference frames, such as shared experiences (e.g. COVID-19) or intergovernmental agendas (e.g. Our Common Agenda). Once key actors understand existential risks better and focus on them, it is much easier to design effective action.

 

5. We can engage and provide value to both our research and policy audiences

We proved that we can facilitate interactions between academic and policy organizations, as well as support researchers in their policy engagement.

Among non-EA actors, we were accepted to contribute to the UN High-Level Political Forum, BWC Meeting of Experts, and BWC Meeting of States Parties. The UN Library and Geneva Centre for Security Policy invited us to run tabletop exercises. The Swiss Government initiated a research collaboration, as well as inviting Max to be a youth delegate on disaster risk reduction. We were able to run a high-level dinner as the sole convener and attracted senior diplomats and delegates from key state actors, as well as senior researchers and knowledge brokers.

Matchmaking has shown itself to be useful, as researchers are keen to be supported in translating their work into impact. If you can make yourself useful to policy actors, it seems that many doors are open. Traveling to the places where we want to have connections is an effective means to make our perspective accessible and useful to others; inviting guests to Geneva is very useful in helping them understand the local reality and plug into our networks.

The international policy perspective that we developed as part of founding and running SI seems unique and complements others well, including at organizations like FHI and CSER. Organizations similar to SI but focused on engaging with policy in individual nations, e.g. CLTR and Pour Demain, are coordinating their internationally relevant activities with us. 

Our branding, network and early work have positioned us such that key policy actors we would like to collaborate with reach out to us proactively. The Swiss government proposed a collaboration on international law and existential risks via our board members. We had initially planned to devote significant resources to engaging with organizations involved in the implementation process of the UN’s Our Common Agenda: we were pleasantly surprised when, based on our workshops and publications, they reached out to us first.

Overall, we have spent most of our time developing content, framings, and formats to productively engage our audience. We can now dedicate more time to facilitating input and sessions, and less time to developing them.

 

6. It is possible to engage in international policy with low downside risks

As mentioned, some observers have expressed concern that engaging with the international policy sphere carries risks. Given our evaluation and the received feedback on our work, it seems unlikely that SI is causing harm. Longtermists with experience in international policy processes have reported that our work is not particularly risky. Conversely, those with less familiarity with the UN ecosystem are understandably more worried about downside risks. 

Negative impact would most likely come from bad frames, activities or events that confuse key longtermist or policy concepts, put off allies, or focus their efforts on low-impact opportunities. Other risks we see are insufficient capacity for public relations management in case of unexpected crises, as well as claiming too much space that could discourage other, much-needed longtermist policy engagement.

We are weighing different opinions on possible downside risks of our work on a case-by-case basis. One general choice we made is that we do not make policy recommendations ourselves. Instead, we always work jointly with subject matter experts to provide policy ideas. We focus on facilitation and general improvements in risk literacy, agenda-setting and decision-making processes. This builds capacity for the time when well-vetted recommendations can be widely promoted and reflects a consensus among longtermist actors that robust policy ideas are lacking, especially at the international level.  

Why facilitation seems impactful

 

By being locally embedded in the Geneva policy ecosystem, we can make engaging with international processes less risky for external longtermist actors. We have been able to provide guidance to researchers, increasing their flexibility and reducing the probability of complications. 

Our lack of funding plausibly increases the risks associated with our work because we have limited capacity to continuously adapt. We lack surge capacity to manage bigger crises should they arise, despite our cautious approach.

 

7. Our current research capacity is limited but our thinking valued

Research has not and won’t be SI’s main focus. Yet it seems important to document our theory of change, evidence base, lessons learned, and ideas for research we would like to see. Thus, throughout its first 12 months, SI has produced a diverse set of publications. Much of these reflected work that contributed to the foundation of SI. To increase confidence in our approach and build our field’s legitimacy, we decided to submit it to academic peer review.

In the process of writing and revising, we realized that we would save a lot of time with an editor and experienced research staff. Nevertheless, due to SI’s competitive advantage of getting policymakers and academics to exchange productively, we do not plan on building up in-house research, unless we unexpectedly receive donations worth more than two times our estimated room for more funding and recruit a highly qualified research lead.

Due to SI’s practice-oriented theory of change, we decided to write practical reference pieces instead of academic publications. Nonetheless, we received highly encouraging peer reviews of “Computational Policy Process Studies” and our concept of “Long-term Institutional Fit”. In discussions with researchers at FHI, GPI and CSER, we have also been able to provide useful input. Further, we have received more than 50 inquiries for thesis supervision, research internships, strategy and career advice.

Given this, it could be valuable for somebody to further develop our research interests. For now, the opportunity costs in policy engagement seem too high for SI to build up more capacity.

3. Our plans for the next two years

Our biggest upcoming opportunity is that we have been solicited by the UN SG Office to contribute to the development of Our Common Agenda. We evaluate it to be a unique opportunity for impact as it has introduced the concepts of existential risk and intergenerational global public goods to the international policy discourse, as well as put a strong focus on future generations in a way that is separate from youth engagement. 

We plan on exploring what can be done at this agenda-setting level of international affairs, until at least after the UN’s Summit of the Future, scheduled for September 2023. We expect the development of this agenda and its resulting processes to 

 

To make the most of this opportunity, SI will conduct the following activities building up to its participation at the Summit of the Future:

 

Many in the longtermist policy field have suggested the need for an actor that aggregates information, tracks opportunities and precipitates action around windows of opportunity, such as OCA. SI aims to position itself in this role by building the infrastructure and bandwidth for scalable, internationally coherent longtermist policy engagement.

We will continuously assess progress made and need surge capacity to seize windows of opportunity as announcements are made on short timelines. We expect various national conferences and workshops throughout the next 18 months to be useful tools to achieve an increased focus on existential risk at the summit itself. To improve coordination, we are planning to host a retreat with key longtermist policy actors in January 2023. We hope to contribute to workshops with delegations and representatives throughout 2023 in countries around the world. After the Summit For the Future, we will publish an evaluation of our engagement and reconsider our mid-term planning.

To solidify SI’s standing in the eyes of its local policy ecosystem, we will also: 

 

To scale the organization sustainably, we plan to hire in waves, such that our team culture is consolidated and our core activities are not disrupted. We currently believe that we would be able to hire 3-5 people before needing to consolidate and restructure a first time.

4. Call for funding

Looking back on SI’s achievements and at the opportunities ahead, by far, our biggest bottleneck is funding. To illustrate:

  1. Konrad is the only person who works full-time for SI and his salary is guaranteed until June 2022 only.
  2. Max works at most 60% for SI, as his salary comes from the University of Geneva, forcing him to stay employed there and work at ~140% FTE.
  3. There is no funding for additional staff and we have only one contractor who started this March.

 

Our current funding needs are the following (we are happy to share budgets on request):

  1. Minimal scenario - for Konrad and Max to work full-time for two years: CHF ~600,000 ($650,000)
  2. Mainline scenario - for building a team focused on UN OCA engagement until 2025: CHF ~4,000,000 ($4,300,000)
  3. Ambitious scenario - for hiring experts to develop cause-specific engagement teams in biosecurity and AI governance: CHF ~8,000,000  ($8,600,000)

 

We have handed in grant proposals to the FTX Future Fund, the Survival Flourishing Fund and the EA Long-term Future Fund and are awaiting feedback. We are in touch with Effective Giving and Founders Pledge as well as a few private donors. We do not expect to receive major grants before the end of June. The sooner we get funding confirmed, the sooner Max can quit his job and we can start scaling up to leverage the UN’s Our Common Agenda and similarly promising opportunities for the mitigation of existential risks and the flourishing of future generations. 

You can donate crypto, $, £ and € tax-deductibly from the US, UK and NL via the EA Funds. You can donate CHF tax-deductibly via EA Geneva or bank transfer. For donations from Germany, please, get in touch. 

5. How to get in touch

If you want to find out more, tell us where you think we’re making mistakes, suggest a collaboration or arrange a general chat, you can contact us at:

6. Ask us anything

Please use the comments to ask us anything - feel free to ask for clarifications, question our plans, point out likely mistakes or propose ideas you think we’ve missed! 

7. Appendix

Overview of outputs

A list of all our outputs in our first year:

  1. Developed and ran a 2-day online workshop on building the global catastrophic risk science-policy interface in collaboration with the Centre for the Study of Existential Risk. The workshop included around 30 participants - half of them existential risk researchers and half risk management policy actors within international organizations. The workshop will lead to a joint report with CSER in 2022.
  2. Developed a tabletop exercise on pandemic preparedness.
  3. Developed a 2-day workshop for Trajan House researchers on understanding and engaging with the United Nations system. Canceled to reduce COVID-19 risks. Ran a 2-hour online session instead.
  4. Developed and ran a short course on the Philosophy, Science, and Practice of Longterm Governance at the University of Geneva’s summer school on the future of global governance.
  5. Organized an event on navigating complexity and information overload in decision-making together with the Geneva Science-Policy Interface, the United Nations Research Institute for Social Development, and with speakers from the European Commission, World Health Organization, and the Institute for Government. 80 people attended and scored the event either good (4/5) or excellent (5/5).
  6. Organized a session and gave a talk on the behavioral science of evidence-based decision-making at the UN High-Level Political Forum. Was supposed to be a 2h in-person workshop on behavioral science & existential risk but canceled due to COVID-19.
  7. Ran in-person event on “long-term decision-making in the face of global catastrophic biological risks” at the BWC’s Meeting of Experts. Low attendance due to COVID restrictions and unavailability of the Palais des Nations.
  8. Held a dinner during BWC MSP with high-profile attendees from diplomatic delegations, international organizations, and existential risk research organizations. Successfully framed existential risks and connected policymakers to academics.
  9. Gave 10 talks on
    1. Tabletop exercise for resilience at the Resilience Studies Symposium of the Geneva Transformative Governance Lab
    2. Tactics for evidence-based policymaking at University College London
    3. Computational policy processes at the Geneva Transformative Governance Lab
    4. Computational policy processes at the Graduate Institute of International and Development Studies
    5. Global catastrophic risk science-policy interface at Swiss Existential Risk Initiative
    6. Long-term governance to Effective Altruism Norway
    7. Long-term governance at EA Germany fellowship weekend
    8. Long-term governance at Centre for the Study of Existential Risk, Cambridge
    9. Long-term governance at Global Priorities Institute, Oxford
    10. Decision-making under uncertainty at Project King
  10. Formalized a collaboration with the Swiss Department of Foreign Affairs’ Directorate of International Law to work on international law and existential risks
  11. Published or co-authored 3 publications
    1. Co-authored paper on Resilience to COVID-19 Pandemic
    2. Co-authored policy brief on Resilience to COVID-19 Pandemic
    3. Published book chapter on Modeling Armed Conflict
  12. Submitted 3 papers to peer-review
    1. Finalized 1st working paper on ‘Policymaking for the Long-term Future’, published on the website and rejected by Futures. Got valuable peer-review feedback, revamp in the works.
    2. Submitted paper on ’Computational Policy Processes’ to Complexity, got accepted after major revisions.
      1. Forthcoming & completely revamped preprint ”A Computational Turn in Policy Process Studies: Co-evolving Network Dynamics of Policy Change” here.
    3. Finalized paper on Strengthening Decisions in Policymaking: A Rapid Review of the Evidence on Four Strategies and submitted to Government Information Quarterly, got rejected.
  13. Published 6 blog posts
    1. Published written piece on ‘Setting Expectations for Extreme Risk Mitigation through Policy Change
    2. Published written piece on ‘Building the Field of Long-term Governance - SI’s Research Approach
    3. Published written piece on ‘United Nations for the future - a collection of key international texts for long-term governance
    4. Published GSPI blog post on Decision-making in the face of global catastrophic biological risks
    5. Published an overview of fields to improve decision-making in policy systems
    6. Published a blog post announcing the launch of SI
  14. Published a Very Short Guide to Decision-making on Wicked Problems
  15. Developed 3 research projects
    1. Finalized protocol for a systematic review on existential risk decision-making and received feedback
    2. Developed a funding proposal on international risk governance for the Swiss National Science Foundation, but decided to postpone our submission.
    3. Partnered in a submission on Predicting Collective Intelligence from Non-Verbal Social Interactions in International Decision-Making for Data Science Impulse
  1. ^

    In this post, we don’t exhaustively justify the value of working with the UN system. We plan to discuss this in a separate post as it seems important for the community to discuss detailed models of how nation states might deliver intergenerational global public goods (e.g. x-risk mitigation).

  2. ^

    We will publish a more systematic assessment of our impact in Q3 2022. 


RyanCarey @ 2022-04-07T16:23 (+37)

 We have handed in grant proposals to the FTX Future Fund, the Survival Flourishing Fund and the EA Long-term Future Fund and are awaiting feedback... The sooner we get funding confirmed, the sooner Max can quit his job and we can start scaling up to leverage the UN’s Our Common Agenda and similarly promising opportunities for the mitigation of existential risks and the flourishing of future generations. 

Seems like it might be useful for a more agile granter, such as perhaps an individual regranter to offer a backstop, e.g.  to say "if SI doesn't receieve funding from any of these sources, then I will offer the minimal amount so that Max would be comfortable quitting his job", so that you can just get on with things. Assuming this would not be too much money, and there are no big hidden risks.

konrad @ 2022-04-08T08:44 (+11)

Thanks very much for highlighting this so clearly, yes indeed. We are currently in touch with one potential such grantmaker. If you know of others we could talk to, that would be great.

The amount isn't trivial at ~600k. Max' salary also guarantees my financial stability beyond the ~6 months of runway I have. It's what has allowed us to make mid-term plans and me to quit my CBG.

NegativeNuno @ 2022-04-08T19:19 (+21)

Do you want an overly negative and perhaps inaccurate comment from this account, under Crocker's rules?

konrad @ 2022-04-08T22:36 (+9)

Yes, happily!

NegativeNuno @ 2022-04-21T16:58 (+27)

Epistemic status: not too sure. See account description.

Overall thoughts

  • The first few sections of this post came across to me as a bit "fake-ish", and really put me off as a reader. Some sardonic notes on that below.
  • Depending on the details, the work on the UN's "Our Common Agenda" (OCA) and your work with the Swiss government seems fairly to very exciting! I'd be curious to get a few more details on it

What parts I'm most excited about, and how I would have structured this post

  1. Sections 2.5. We can engage and provide value to both our research and policy audiences, and 2.7. Our current research capacity is limited but our thinking valued are fairly strong. On 2.7., you could also commission or suggest the kinds of research you are most excited about, even if you don't have the ability to do them yourself.

  2. I also thought that these paragraphs were very strong:

Building on these reference points, the UN’s Our Common Agenda presents a rare opportunity to directly work on future generations and existential risks in national and international political agendas. This is because they are explicitly introduced in the report and 17 out of its 69 proposals are directly relevant to longtermist goals

Our biggest upcoming opportunity is that we have been solicited by the UN SG Office to contribute to the development of Our Common Agenda. We evaluate it to be a unique opportunity for impact as it has introduced the concepts of existential risk and intergenerational global public goods to the international policy discourse, as well as put a strong focus on future generations in a way that is separate from youth engagement.

  1. The fact that your funding appeal is directly connected to your work on the UN's "Our common agenda" specifically makes it stronger.

I would have put them at the beginning, rather than somewhere in the post. I would have also been more explicit at the beginning, rather than talking about a "mandate" in the abstract. What exactly does the mandate involve?

I would also have liked to see more emphasis on your work as experimentation. This writeup strikes me as the kind of writeup that a more mature organization might produce. But your organization is much younger and has just one to three FTEs, so I would have liked to see an emphasis on your capacity for experimentation, ability to pivot, and general competence.

Key points

Some key points I would look at if I had unlimited time:

  • Does the UN actually matter? I come away with the impression that you believe that this is the case. But my prior on the UN being decidedly mediocre and not that relevant is actually quite strong, and it is actually not clear to me how much I should defer to your expert understanding.
    • For instance, weapon reduction treaties such as New START were not conceived as part of a UN framework. Similarly, the fact that the Biological Weapons Convention has so low a budget makes me update very negatively on the UN's competence.
    • One factor contributing to this is the US' recluctance to take part in UN's structures, rather than doing its own thing.
  • Because you have mostly been doing collaborations, I am really unsure about the Simon Institute's (SI) Shapley and counterfactual value.

Overall, these seem hard to resolve. But this grant seems like it still makes a lot of sense from a hits-based perspective.

Thoughts on fundraising

Our main fundraising targeted the EA Funds, resulting in six proposals for a total of ca. $ 600,000. Of these proposals, only $ 46,000 was granted. Another application to the Survival and Flourishing Fund for network-building activities was rejected without comment. Founders Pledge investigated our plans and concluded SI to be promising but with an insufficient track record for a recommendation.

The main reason for worry seems to have been our limited track record in downside-risk-conscious policy engagement, given SI’s ambitious, public-facing plans. Thus, the EA Infrastructure Fund decided to fund a minimal version of SI to first gather more data for a more telling evaluation after March 2022.

So my understanding here is that EA Funds didn't really trust your judgment, and wanted to see a track record before funding you further. I'm not sure to what extent this post achieves that goal with regards to the good judgment part.

Furthermore, as SI works at the interface between existential risk research and related policymaking, we have to frame our work differently to various sides and communicate these choices transparently to all sides. Some framings appear insufficiently technical to existential risk researchers while others appear too theoretical to our policy audiences. We are planning to conduct message testing studies to improve cross-context coherence, beyond local salience.

Yeah, this is tricky. For instance, I'm not sure to what extent you are thinking clearly but expressing your thoughts a bit too verbosely or in essentially a different dialect, or directly thinking unclearly.

Parting thoughts

I've put some specific nitpicks in a separate comment below. Surprisingly, many of these nitpicks mostly relate to the way you word things, rather than to your actual content. But I think that the way you word things formally also has you leave out a fair amount of detail.

Particularly around the UN's "Our Common Agenda". I can imagine situations that would make it worth it for you to be fully funded, but also situations where it's not all that exciting. A year ago I spent some time trying to figure out how the Sustainable Development Goals were decided, and I really couldn't exactly tell. I also couldn't tell by how much the SDGs ended up affecting, for instance, US aid funding. I think you probably have a bunch of implicit models around how this could be valuable, but this post would have been a good time to write them up.

In an ideal world, you could also have an estimate of how much better $4M to you would make the OCA, and how much funding/resources the OCA would influence, to get a reasonable estimate on impact. For instance, here is a guesstimate model that assumes that the only pathway to funding is Simon Institute -> OCA -> SDG -> Foreign aid. I'm guessing that this is wrong, but I also don't have a good sense of by what the impact of OCA is mediated through. So I'd be curious to get a less fake estimate using your own models.

Overall, I'm left hoping that EA would have its own Kissinger to be able to evaluate whether this kind of thing is really promising.

NegativeNuno @ 2022-04-21T16:59 (+9)

Specific nitpicks

These were written as I was reading the post, so some of them are addressed by points brought up later. They are also a bit too sardonic.

  • "For all key risks, humanity’s path to existential security cannot be brought about by the actions of any single country, making more effective international cooperation essential"
    • Is this actually true? Not sure. For instance, if the US, China and maybe the UK decide to not do anything too crazy like getting into an AI arms race, that seems like it might leave us in a decent position, AI policy-wise.
  • "This combination of activities has granted SI mandates from UN institutions, as well as the Swiss government, to directly work on policy processes relevant to existential risk reduction"
    • Mandates but no money? "Mandates" sounds good, but not sure what it means.
    • [note: my initial impression was wrong, for instance, later you say] "We have signed a grant agreement of CHF 50,000 from the Swiss Government’s International Public Law Division for a project on existential risk governance led by SI’s board member Igor Linkov. Our collaboration with the Geneva Science-Policy Interface has yielded another grant agreement of CHF 30,000 for work on the tabletop exercise on pandemic preparedness.". Nice. But is this the same "mandate".
  • "Science-policy interface" is a really neat construction, but I wouldn't call Global Priorities research a "science"
  • "This is why SI could fill a gap in an information-rich but time-scarce environment". More plainly expressed sentences could also fill a gap in verbiage-rich but transparency-scare environments. Ok, this is mean. But, for instance, I think I could get a better idea of what you are doing if you word this as: "We try to build relationships with and make recommendations to really busy bureacrats who are nonetheless a bit altruistically inclined. Eventually, we could position ourselves so as to build international institutions for existential risk reduction, like a new treaty or the International Atomic Energy Agency". But is that what you are doing? I sort of get the vague feeling that I don't know how you are spending most of your hours.
  • "SI is embedded in one of the few international policy hubs - Geneva - and adapts its strategy in response to arising opportunities". The word "embedded" rubs me the wrong way.
  • "2021 has marked a breakthrough in international policymakers’ responsiveness to longtermist concerns." => Policymakers are now more worried about weird and unexpected things because COVID provides a salient example.
  • "Leveraging this window of opportunity, SI works with them to reduce existential risks and further long-term governance via the impact pathways of the international system.". I am really not sure what proportion of impact of this leveraging SI is claiming, or what proportion should be allocated to it.
  • "Began to work on the UN’s Our Common Agenda processes". Directly??? In what capacity?
  • "co-develop a workshop series": Would the series have happened in your absence? What % of the value was due to your participation?
  • "Some international organizations, like the UN Office for Disaster Risk Reduction (UNDRR) and the International Science Council, also explicitly stated an interest in developing a better understanding of GCRs as a result". My first thought on reading this is "nice!". But "stated an interest in developing a better understanding" is actually extremely non-committal. What do you think it will/could cash out to?
  • "Without SI, FHI Bio would have been less likely to get an in-depth look behind the scenes of international diplomacy and unlikely to connect to key stakeholders in a setting where personal discussion allowed for the divulging of insider information and personal opinion". How much less likely?
  • "Beyond taking a significant amount of time out of their busy days, many diplomats would also have needed security clearance to participate.". Why the security clearance?
NegativeNuno @ 2022-04-28T14:33 (+6)

Circling back to this, this report hits almost none of the notes in lukeprog's Features that make a report especially helpful to me, which might be one reason why I got the impression that the authors were speaking a different dialect.

konrad @ 2022-09-21T19:58 (+2)

Dear Nuño, thank you very much for the very reasonable critiques! I had intended to respond in depth but it's continuously not the best use of time. I hope you understand. Your effort has been thoroughly appreciated and continues to be integrated into our communications with the EA community. 

We have now secured around 2 years of funding and are ramping up our capacity . Until we can bridge the inferential gap more broadly, our blog offers insight into what we're up to. However, it is written for a UN audience and non-exhaustive, thus you may understandably remain on the fence.

nathanhb @ 2022-04-18T18:13 (+3)

Sounds like good work. Trying to get the right information to the right people who can pass that on to decision makers seems useful. 

 

Two things I would like you to consider having some cached thoughts on to offer to the right people would be:

1. food security under adverse circumstances (e.g. highlights from the work of AllFed like seaweed farming).

2. Global cooling work (e.g. Silver Lining, and doubling up with reef-protection by seawater spraying over reefs), and the combined work of carbon sequestration and soil quality improvement of Biochar as an agricultural practice. Biochar is neat because it's advantageous to the individual farmer, as well as being good for the world, so the local incentive structure is aligned.