Types of information hazards

By Vasco Grilo🔸 @ 2022-05-29T14:30 (+15)

The definitions of the types and subtypes of information hazards described in Bostrom 2011, by information transfer mode and effect, are presented below.

Information hazard: A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm.

By information transfer mode

Data hazard: Specific data, such as the genetic sequence of a lethal pathogen or a blueprint for making a thermonuclear weapon, if disseminated, create risk.

Idea hazard: A general idea, if disseminated, creates a risk, even without a data-rich detailed specification.

Attention hazard: The mere drawing of attention to some particularly potent or relevant ideas or data increases risk, even when these ideas or data are already “known”.

Template hazard: The presentation of a template enables distinctive modes of information transfer and thereby creates risk.

Signaling hazard: Verbal and non-verbal actions can indirectly transmit information about some hidden quality of the sender, and such social signaling creates risk.

Evocation hazard: There can be a risk that the particular mode of presentation used to convey some content can activate undesirable mental states and processes.

By effect

Adversarial risks

Competiveness hazard: There is a risk that, by obtaining information, some competitor of ours will become stronger, thereby weakening our competitive position. Subtypes:

• Enemy hazard: By obtaining information our enemy or potential enemy becomes stronger and this increases the threat he poses to us.
• Intellectual property hazard: A faces the risk that some other firm B will obtain A’s intellectual property, thereby weakening A’s competitive position.
• Commitment hazard: There is a risk that the obtainment of some information will weaken one’s ability credibly to commit to some course of action.
• Knowing-too-much hazard: Our possessing some information makes us a potential target or object of dislike.

Risks to social organization and markets

Norm hazard: Some social norms depend on a coordination of beliefs or expectations among many subjects; and a risk is posed by information that could disrupt these expectations for the worse. Subtypes:

• Information asymmetry hazard: When one party to a transaction has the potential to gain information that the others lack, a market failure can result.
• Unveiling hazard: The functioning of some markets, and the support for some social policies, depends on the existence of a shared “veil of ignorance”; and the lifting of which veil can undermine those markets and policies.
• Recognition hazard: Some social fiction depends on some shared knowledge not becoming common knowledge or not being publicly acknowledged; but public release of information could ruin the pretense.

Risks of irrationality and error

Ideological hazard: An idea might, by entering into an ecology populated by other ideas, interact in ways which, in the context of extant institutional and social structures, produce a harmful outcome, even in the absence of any intention to harm.

Distraction and temptation hazards: Information can harm us by distracting us or presenting us with temptation.

Role model hazard: We can be corrupted and deformed by exposure to bad role models.

Biasing hazard: When we are biased, we can be led further away from the truth by exposure to information that triggers or amplifies our biases.

De-biasing hazard: When our biases have individual or social benefits, harm could result from information that erodes these biases.

Neuropsychological hazard: Information might have negative effects on our psyches because of the particular ways in which our brains are structured, effects that would not arise in more “idealized” cognitive architectures.

Information-burying hazard: Irrelevant information can make relevant information harder to find, thereby increasing search costs for agents with limited computational resources.

Risks to valuable states and activities

Psychological reaction hazard: Information can reduce well-being by causing sadness, disappointment, or some other psychological effect in the receiver. Subtypes:

• Disappointment hazard: Our emotional well-being can be adversely affected by the receipt of bad news.
• Spoiler hazard: Fun that depends on ignorance and suspense is at risk of being destroyed by premature disclosure of truth.
• Mindset hazard: Our basic attitude or mindset might change in undesirable ways as a consequence of exposure to information of certain kinds.

Belief-constituted value hazard: If some component of well-being depends constitutively on epistemic or attentional states, then information that alters those states might thereby directly impact well-being. Subtype:

• Embarrassment hazard: We may suffer psychological distress or reputational damage as a result of embarrassing facts about ourselves being disclosed.

Risks from information technology systems

Information system hazard: The behavior of some (non-human) information system can be adversely affected by some informational inputs or system interactions. Subtypes:

• Information infrastructure failure hazard: There is a risk that some information system will malfunction, either accidentally or as result of cyber attack; and as a consequence, the owners or users of the system may be inconvenienced, or third parties whose welfare depends on the system may be harmed, or the malfunction might propagate through some dependent network, causing a wider disturbance.
• Information infrastructure misuse hazard: There is a risk that some information system, while functioning according to specifications, will service some harmful purpose and will facilitate the achievement of said purpose by providing useful information infrastructure.
• Artificial intelligence hazard: There could be computer-related risks in which the threat would derive primarily from the cognitive sophistication of the program rather than the specific properties of any actuators to which the system initially has access.

Risks from development

Development hazard: Progress in some field of knowledge can lead to enhanced technological, organizational, or economic capabilities, which can produce negative consequences (independently of any particular extant competitive context).