Ten Commandments for Aspiring Superforecasters
By Vasco Grilo🔸 @ 2024-02-20T13:01 (+13)
This is a linkpost to https://goodjudgment.com/philip-tetlocks-10-commandments-of-superforecasting/
This is a linkpost for Ten Commandments for Aspiring Superforecasters by Good Judgment[1]. I think the lessons also apply outside of explicit forecasting.
In Superforecasting: The Art and Science of Prediction, Good Judgment co-founder Philip Tetlock and his co-author Dan Gardner summarize the Good Judgment Project research findings in the form of “Ten Commandments for Aspiring Superforecasters.” These commandments describe behaviors that have been “experimentally demonstrated to boost [forecasting] accuracy.” You can learn more about these commandments—and practice applying them under the guidance of professional Superforecasters—at one of Good Judgment’s training workshops.
Here are the 10 commandments:
- Triage. "Focus on questions where your hard work is likely to pay off".
- Break seemingly intractable problems into tractable sub-problems.
- Strike the right balance between inside and outside views.
- Strike the right balance between under- and overreacting to evidence.
- Look for the clashing causal forces at work in each problem.
- Strive to distinguish as many degrees of doubt as the problem permits but no more.
- Strike the right balance between under- and overconfidence, between prudence and decisiveness.
- Look for the errors behind your mistakes but beware of rearview-mirror hindsight biases.
- Bring out the best in others and let others bring out the best in you.
- Master the error-balancing bicycle. "Implementing each commandment requires balancing opposing errors".
Last but not least, there is the “11th commandment”:
“It is impossible to lay down binding rules,” Helmuth von Moltke warned, “because two cases will never be exactly the same.”
- ^
I wanted this to be a crosspost, but it was not possible due to copyright restrictions.
MathiasKB @ 2024-02-20T16:13 (+3)
I think this leaves out what is perhaps the most important step in making a quality forecast, which is to consider the baserates!
Vasco Grilo @ 2024-02-20T18:12 (+2)
Nice point, Mathias! I agree reference class forecasting is super important. I think it is supposed to be included in the 3rd commandment about the inside and outside view:
Superforecasters know that there is nothing new under the sun. Nothing is 100% “unique.” Language purists be damned: uniqueness is a matter of degree. So Superforecasters conduct creative searches for comparison classes even for seemingly unique events, such as the outcome of a hunt for a high-profile terrorist (Joseph Kony) or the standoff between a new socialist government in Athens and Greece’s creditors. Superforecasters are in the habit of posing the outside-view question: How often do things of this sort happen in situations of this sort?