Announcing the Open Philanthropy AI Worldviews Contest

By Jason Schukraft, Peter Favaloro @ 2023-03-10T02:33 (+137)

This is a linkpost to https://www.openphilanthropy.org/open-philanthropy-ai-worldviews-contest/

Update: We've now chosen the winning entries; you can read them here.

We are pleased to announce the 2023 Open Philanthropy AI Worldviews Contest.

The goal of the contest is to surface novel considerations that could influence our views on AI timelines and AI risk. We plan to distribute $225,000 in prize money across six winning entries. This is the same contest we preannounced late last year, which is itself the spiritual successor to the now-defunct Future Fund competition. Part of our hope is that our (much smaller) prizes might encourage people who already started work for the Future Fund competition to share it publicly.

The contest deadline is May 31, 2023. All work posted for the first time on or after September 23, 2022 is eligible. Use this form to submit your entry.

Prize Conditions and Amounts

Essays should address one of these two questions:

Question 1: What is the probability that AGI is developed by January 1, 2043?^[1]

Question 2: Conditional on AGI being developed by 2070, what is the probability that humanity will suffer an existential catastrophe due to loss of control over an AGI system?

Essays should be clearly targeted at one of the questions, not both.

Winning essays will be determined by the extent to which they substantively inform the thinking of a panel of Open Phil employees. There are several ways an essay could substantively inform the thinking of a panelist:

• An essay could cause a panelist to change their central estimate of the probability of AGI by 2043 or the probability of existential catastrophe conditional on AGI by 2070.
• An essay could cause a panelist to change the shape of their probability distribution for AGI by 2043 or existential catastrophe conditional on AGI by 2070, which could have strategic implications even if it doesn’t alter the panelist’s central estimate.
• An essay could clarify a concept or identify a crux in a way that made it clearer what further research would be valuable to conduct (even if the essay doesn’t change anybody’s probability distribution or central estimate).

We will keep the composition of the panel anonymous to avoid participants targeting their work too closely to the beliefs of any one person. The panel includes representatives from both our Global Health & Wellbeing team and our Longtermism team. Open Phil’s published body of work on AI^[2] broadly represents the views of the panel.

Panelist credences on the probability of AGI by 2043 range from ~10% to ~45%. Conditional on AGI being developed by 2070, panelist credences on the probability of existential catastrophe range from ~5% to ~50%.

We will award a total of six prizes across three tiers:

• First prize (two awards): $50,000
• Second prize (two awards): $37,500
• Third prize (two awards): $25,000

Eligibility

• Submissions must be original work, published for the first time on or after September 23, 2022 and before 11:59 pm EDT May 31, 2023.
• All authors must be 18 years or older.
• Submissions must be written in English.
• No official word limit — but we expect to find it harder to engage with pieces longer than 5,000 words (not counting footnotes and references).
• Open Phil employees and their immediate family members are ineligible.
• The following groups are also ineligible:
  • People who are residing in, or nationals of, Puerto Rico, Quebec, or countries or jurisdictions that prohibit such contests by law
  • People who are specifically sanctioned by the United States or based in a US-sanctioned country (North Korea, Iran, Russia, Myanmar, Afghanistan, Syria, Venezuela, and Cuba at time of writing)
• You can submit as many entries as you want, but you can only win one prize.
• Co-authorship is fine.
• See here for additional details and fine print. 

Submission

Use this form to submit your entries. We strongly encourage (but do not require) that you post your entry on the EA Forum and/or LessWrong. However, if your essay contains infohazardous material, please do not post the essay publicly. 

Note that submissions will be hosted on a Google server and viewable by Open Phil staff. We don’t think that (m)any submissions will warrant more security than this. However, if you believe that your submission merits a more secure procedure, reach out to AIWorldviewsContest@openphilanthropy.org, and we will make appropriate arrangements.

Judging Process and Criteria

There will be three rounds of judging.

Round 1: An initial screening panel will evaluate all submitted essays by blind grading to determine whether each essay is a good-faith entry. All good-faith entries will advance to Round 2.

Round 2: Out of the good-faith entries advancing from Round 1, a panel of judges will select at least twenty-four finalists.

Round 3: Out of the finalists advancing from Round 2, the judges will select two first-place entries, two second-place entries, and two third-place entries.

In Rounds 2 and 3, the judges will make their decision using the criteria described below:

• The extent to which an essay uncovers considerations that change a judge’s beliefs about the probability of AGI arriving by 2043 or the threat that AGI systems might pose. (67%)
• The extent to which an essay clarifies the underlying concepts that ought to inform one’s views about the probability of AGI arriving by 2043 or the threat that AGI systems might pose. (33%)

Questions?

Please email AIWorldviewsContest@openphilanthropy.org with any questions, comments, or concerns.

1. ^{^}

   By “AGI” we mean something like “AI that can quickly and affordably be trained to perform nearly all economically and strategically valuable tasks at roughly human cost or less.” AGI is a notoriously thorny concept to define precisely. What we’re actually interested in is the potential existential threat posed by advanced AI systems. To that end, we welcome submissions that are oriented around related concepts, such as transformative AI, human-level AI, or PASTA.

2. ^{^}

   This includes the research published on our website, as well as material from Ajeya Cotra, Holden Karnofsky, Joe Carlsmith, and Tom Davidson.