Announcing the Forecasting Research Institute (we’re hiring)

By Tegan @ 2022-12-13T12:11 (+168)

The Forecasting Research Institute (FRI) is a new organization focused on advancing the science of forecasting for the public good. 

All decision-making implicitly relies on prediction, so improving prediction accuracy should lead to better decisions. And forecasting has shown early promise in the first-generation research conducted by FRI Chief Scientist Phil Tetlock and coauthors. But despite burgeoning popular interest in the practice of forecasting (especially among EAs), it has yet to realize its potential as a tool to inform decision-making.

Early forecasting work focused on establishing a rigorous standard for accuracy, in experimental conditions chosen to provide the cleanest, most precise evidence possible about forecasting itself—a proof of concept, rather than a roadmap for using forecasting in real-world conditions. A great deal of work, both foundational and translational, is still needed to shape forecasting into a tool with practical value.

That’s why our team is pursuing a two-pronged strategy. One is foundational, aimed at filling in the gaps in the science of forecasting that represent critical barriers to some of the most important uses of forecasting—like how to handle low probability events, long-run and unobservable outcomes, or complex topics that cannot be captured in a single forecast. The other prong is translational, focused on adapting forecasting methods to practical purposes: increasing the decision-relevance of questions, using forecasting to map important disagreements, and identifying the contexts in which forecasting will be most useful.

Over the next two years we plan to launch multiple research projects aimed at the key outstanding questions for forecasting. We will also analyze and report on our group’s recently completed project, the Existential Risk Persuasion Tournament (XPT). This tournament brought together over 200 domain experts and highly skilled forecasters to explore, debate, and forecast potential threats to humanity in the next century, creating a wealth of rich data that our team is mining for forecasting and policy insights.

In our upcoming projects, we’ll be conducting large, high-powered studies on a new research platform, customized for the demands of forecasting research. We’ll also work closely with selected organizations and policymakers to create forecasting tools informed by practical use-cases. Our planned projects include:

• Developing a forecasting proficiency test for quickly and cheaply identifying accurate forecasters
• Identifying leading indicators of increased risk to humanity from AI by building “AI-risk conditional trees” with the help of domain experts (overview of conditional trees here, pg. 13)
• Exploring ways of judging (and incentivizing) answers to unresolvable and far-future questions, such as reciprocal scoring
• Conducting “Epistemic Audits” to help organizations reduce uncertainty, identify action-relevant disagreement, and guide their decision processes.

(For more on our research priorities, see here and here.)

We’re excited to begin FRI’s work at such an auspicious time for the field of forecasting, with the many great projects, people and ideas that currently inhabit it—spanning the gamut from heavyweight organizations like Metaculus and GJI, to the numerous innovative projects run by small teams and individuals. This environment presents a wealth of opportunities for collaboration and cooperation, and we’re looking forward to being a part of such a dynamic community.

Our core team consists of Phil Tetlock, Michael Page, Josh Rosenberg, Ezra Karger, Tegan McCaslin, and Zachary Jacobs. We also work with various contractors and external collaborators in the forecasting space. We’re looking to expand our team to include additional research analysts, data analysts, content editors and research assistants; more information on the roles and an application form can be found by following those links.