Some estimation work in the horizon

By NunoSempere @ 2023-03-29T22:18 (+25)

This is a linkpost to https://nunosempere.com/blog/2023/03/20/estimation-in-the-horizon/

This post outlines some work in altruistic estimation that seems currently doable. Some of it might be pursued by, for example, my team at the Quantified Uncertainty Research Institute. But together this work adds up to more than what our small team can achieve.

Two downsides of this post are that a) it looks at things that are more salient to me, and doesn’t comprehensively review all estimation work being done, and b) it could use more examples.

Saruman in Isengard looking at an army of orcs
Saruman in Isengard looking at an army of orcs

Directions

Produce more specific high-value estimates

The idea here is to produce more estimates in a way that results in better decisions.

Some examples might be:

The value of these estimates generally has two moving parts:

  1. Their immediate payoff in terms of affecting near-term decisions
  2. Making it easier to create future estimates of a similar kind

For example, if I help someone estimate the value of various career options they are considering, that has the benefit of improving their decision about which career moves they make. But if those estimates are then posted on the EA Forum, or in some repository of models, they might make it easier for people to reuse and tweak these estimates, which would be cheaper than coming up with their own estimates anew.

At QURI in particular, we might work on estimating the value of some of 80,000 hours' top career paths, building upon the results from this contest, because this seems like a somewhat ambitious type of estimation that would produce useful templates and allow us to explore new strategies. And I personally expect to produce a few such estimates this year.

Explore different types of estimates

At QURI, we have been experimenting mainly with two broad classes of estimation:

As we do more work there, we learn and begin to implement some stuff, like:

But in general, are there any ingenious types of estimates which would make estimation easier, more scalable, or doable for previously inaccessible topics?

Build estimation pipelines

In August 2020, @Erich_Grunewald estimated that there was a 5-10% chance that EA would lose a billionaire in the next few years. In hindsight, this estimate seems decent. But it wasn’t vetted, distributed, and incorporated into people’s decisions. And although it was part of some QURI contests, it seems more like a one-off than a continuous series of estimations of importance.

Instead, we could have pipelines that continuously produce estimates in a way which produces value. Some pipelines which it might be valuable to build might be:

Here, by pipeline, I mean something distinct from a one-off, and that starts to produce value and can continue to do so. My sense is that those types of work are more valuable.

Work on the theory behind estimation

Some topics which could benefit from a theoretical treatment might be:

The hope here would be that taking a step back and theorizing could add some clarity which could improve your day to day estimation.

Produce better estimation tooling

Better tooling seems like it could lead to the production of more and better estimates, faster. For an extreme example, here is a simple BOTEC in C (a low level estimation language), and here is that same estimate in Squiggle, a language for rapid estimation.

Some progress in the tooling domain might involve:

One consideration for this kind of enterprise is that it can be fairly time intensive, and fairly long until you learn whether it has been worth it or not.

Who is working on this type of thing?

Within or adjacent to EA:

Outside EA, probably too many to mention at once, but some which stand out to me are: