My notes on: Sequence thinking vs. cluster thinking

By Vasco Grilo @ 2022-05-25T15:03 (+24)

I have recently read GiveWell's article, written by Holden Karnofsky, about Sequence thinking vs. cluster thinking. Below are my notes, which are essentially structured transcriptions. Any errors/misinterpretations are my own.

Definitions

Sequence thinking:

• Decision making is based on a single model of the world:
  • Breaking down the decision into a set of key questions, taking one’s best guess on each question, and accepting the conclusion that is implied by the set of best guesses.
    • An excellent example of this sort of thinking is Robin Hanson’s discussion of cryonics.
  • It has the form: “A, and B, and C … and N; therefore X”.
• Advantage of making one’s assumptions and beliefs highly transparent.
  • As such, it is often associated with finding ways to make counterintuitive comparisons.

Cluster thinking:

• Generally the more common kind of thinking.
• Decision-making is approached from multiple perspectives (which might also be called “mental models”):
  • Observing which decision would be implied by each perspective, and weighing the perspectives in order to arrive at a final decision.
  • Cluster thinking has the form: “Perspective 1 implies X; perspective 2 implies not-X; perspective 3 implies X; … therefore, weighing these different perspectives and taking into account how much uncertainty I have about each, X”.
  • Each perspective might represent a relatively crude or limited pattern-match (e.g., “This plan seems similar to other plans that have had bad results”), or a highly complex model.
  • The different perspectives are combined by weighing their conclusions against each other, rather than by constructing a single unified model that tries to account for all available information.
• Perspectives associated with high uncertainty are stopped from carrying strong weight in the final decision, even when such perspectives involve extreme claims.
  • E.g., a low-certainty argument that “animal welfare is 100,000x as promising a cause as global poverty” receives no more weight than if it were an argument that “animal welfare is 10x as promising a cause as global poverty”.
• “Regression to normality”: the stranger and more unusual the action-relevant implications of a perspective, the higher the bar for taking it seriously (“extraordinary claims require extraordinary evidence”).

Why Cluster Thinking?

Sequence thinking is prone to reaching badly wrong conclusions based on a single missing, or poorly estimated, parameter:

• In general, missing parameters and overestimated probabilities will lead to overestimating the likelihood that actions play out as hoped.
• Thus overestimating the desirability of deviating from “tried and true” behavior and behavior backed by outside views.

Cluster thinking is more similar to empirically effective prediction methods:

• Sequence thinking may incorporate many considerations, but all are translated into a single language, a single mental model, and in some sense a single “formula”.
• Successful prediction systems, whether in finance, software, or domains such as political forecasting, generally combine the predictions of multiple models in ways that purposefully avoid letting any one model (especially a low-certainty one) carry too much weight when it contradicts the others.

A cluster-thinking-style “regression to normality” seems to prevent some obviously problematic behavior relating to knowably impaired judgment:

• One thought experiment is imagining that one is clearly and knowably impaired at the moment (e.g. being drunk or a young child), and contemplating a chain of reasoning that suggests high expected value for some unusual and extreme action (such as jumping from a height).
• It seems that the person in question should recognize their own elevated fallibility and take special precautions to avoid deviating from “normal” behavior.

Sequence thinking seems to tend toward excessive comfort with “ends justify the means” type thinking:

• The basic structure of cluster thinking does set up more hurdles for arguments about “the ends” (large-magnitude but speculative down-the-line outcomes) to justify “the means” (actions whose consequences are nearer and clearer).
• The worse the “means,” the more robust (and not just large in claimed magnitude) one’s case for “the ends” ought to be.

When uncertainty is high, “unknown unknowns” can dominate the impacts of our actions, and cluster thinking may be better suited to optimizing “unknown unknown” impacts:

• Sequence thinking seems, by its nature, to rely on listing the possible outcomes of an action and evaluating the action according to its probability of achieving these outcomes.
• The sort of outside views that tend to get more weight in cluster thinking are often good predictors of “unknown unknowns”.

Broad market efficiency:

• The most efficient markets can be consistently beaten only by the most talented/dedicated players, while the least efficient ones can be beaten with fairly little in the way of talent and dedication.
• When one is considering a topic or action that one knows little about, one should consider the broad market to be highly efficient.

Sequence thinking seems to over-encourage “exploiting” as opposed to “exploring” one’s best guesses:

• Cluster thinking highlights many consequential areas of uncertainty and promises returns to clearing up any of them, leading to more traction on learning and more reduction in “unknown unknowns” over time.
• Sequence thinking has a tendency to make different options seem to differ more in value.
• Cluster thinking tends to have heavier penalties for uncertainty.

Advantages of sequence thinking

Sequence thinking can generate robust conclusions that then inform cluster thinking:

• There are times when a long chain of reasoning can be constructed that has relatively little uncertainty involved.
• The extreme case of this is in some science and engineering applications.
• A less extreme case is when someone simply puts a great deal of work into doing as much reflection and investigation as they can of the parameters in their model.

Sequence thinking is more favorable to generating creative, unconventional, and nonconformist ideas:

• Sequence thinking provides a way of seeing where a chain of reasoning goes when historical observations, conventional wisdom, expert opinion and other “outside views” are suspended.

Sequence thinking is better-suited to transparency, discussion and reflection:

• I generally find it very hard to formalize and explain what “outside views” I am bringing to a decision, how I am weighing them against each other, and why I have the level of certainty I do in each view.
• Sequence thinking tends to consist of breaking a decision down along lines that are well-suited to communication, often in terms of a chain of causality.

Sequence thinking can lead to deeper understanding:

• It is better-suited to explicit discussion and reflection.
• It tends to focus on chains of causality without deep integration of poorly-understood but empirically observed “outside view” patterns.

Other considerations:

• Sequence thinking can be a good antidote to scope insensitivity.

Cluster thinking and argumentation

It is important to ask not just whether there are explicit problems with one’s argument, but:

• How much uncertainty there is in one’s argument (even if such uncertainty doesn’t clearly skew the calculation in one direction or another).
• Whether other arguments, using substantially different mental models, give the same conclusion.

The balance I try to strike

As implied above, I believe:

• Sequence thinking is valuable for idea generation, reflection and discussion.
• Cluster thinking is best for making the final choice between options.

For me, a basic rule of thumb is that it’s worth making some degree of bet on novel ideas, even when the ideas are likely flawed, when it’s the kind of bet that:

• Facilitates the stress-testing, refinement, and growing influence of these ideas.
• Does not interfere with other, more promising bets on other novel ideas.

The above line of argument justifies behavior that can seem otherwise strange and self-contradictory:

• It can justify advocating and acting to some degree on a novel idea, while not living one’s life fully consistently with this idea
  • E.g., working to promote Peter Singer’s ideas about the case for giving more generously, while not actually giving as much as his ideas would literally imply one should.
• Asking not only “Does this idea seem valid to me?”, but “Am I acting on this idea in a way that promotes it and facilitates its evolution, and does not interfere with my promotion of other more promising ideas?”.