SiebeRozendal's Quick takes

Ray Dalio is giving out free $50 donation vouchers: tisbest.org/rg/ray-dalio/

Still worked just a few minutes ago

Not that we can do much about it, but I find the idea of Trump being president in a time that we're getting closer and closer to AGI pretty terrifying.

A second Trump term is going to have a lot more craziness and far fewer checks on his power, and I expect it would have significant effects on the global trajectory of AI.

Common prevalence estimates are often wrong. Example: snakebites and my experience reading Long Covid literature.

Both institutions like the WHO and academic literature appear to be incentivized to exaggerate. I think the Global Burden of Disease might be a more reliable source, but have not looked into it.

I advise everyone using prevalence estimates to treat them with some skepticism and look up the source.

I just learned that Lawrence Lessig, the lawyer who is/was representing Daniel Kokateljo and other OpenAI employees, supported and encouraged electors to be faithless and vote against Trump in 2016.

He wrote an opinion piece in the Washington Post (archived) and offered free legal support. The faithless elector story was covered by Politico, and was also supported by Mark Ruffalo (the actor who recently supported SB-1047).

I think this was clearly an attempt to steal an election and would discourage anyone from working with him.

I expect someone to eventually sue AGI companies for endangering humanity, and I hope that Lessig won't be involved.

Really interesting initiative to develop ethanol analogs. If successful, replacing ethanol with a less harmful substance could really have a big effect on global health. The CSO of the company (GABA Labs) is prof. David Nutt, a prominent figure in drug science.

I like that the regulatory pathway might be different from most recreational drugs, which would be very hard to get de-scheduled.

I'm pretty skeptical that GABAergic substances are really going to cut it, because I expect them to have pretty different effects to alcohol. We already have those (L-theanine , saffran, kava, kratom) and they aren't used widely. But who knows, maybe that's just because ethanol-containing drinks have received a lot of optimization in terms of taste, marketing, and production efficiency.

It also seems like finding a good compound by modifying ethanol would be hard, because it's not a great lead compound in terms of toxicity (I expect).

There is a natural alliance that I haven't seen happen, but both are in my network: pandemic preparedness and covid-caution. Both want clean indoor air.

The latter group of citizens is a very mixed group, with both very reasonable people and unreasonable 'doomers'. Some people have good reason to remain cautious around COVID: immunocompromised people & their household, or people with a chronic illness, especially my network of people with Long Covid, who frequently (~20%) worsen from a new COVID case.

But these concerned citizens want clean air, and are willing to take action to make that happen. Given that the riskiest pathogens trend to also be airborne like SARS-COV-2, this would be a big win for pandemic preparedness.

Specifically, I believe both communities are aware of the policy objectives below and are already motivated to achieve it:

1) Air quality standards (CO2, PM2.5) in public spaces.

Schools are especially promising from both perspectives, given that parents are motivated to protect their children & children are the biggest spreaders of airborne diseases. Belgium has already adopted regulations (although very weak, it's a good start), showing that this is a tractable policy goal.

Ideally, air quality standards also incentivize Far UVC deployment, which would create the regulatory certainty for companies to invest in this technology.

Including standards for airborne pathogen concentrations would be great, but has many technical limitations at the moment I think.

2) Public R&D investments to bring down cost & establish safety of Far UVC

Most of these concerned citizens are actually aware of Far UVC and would support this measure. It appears safe in terms of no radiation damage, but may create unhealthy compounds (e.g. ozone) by chemically reacting with indoor air particles. 

I also believe that governments have good reasons to adopt these policies, given that they would reduce the pressures on healthcare and could reduce the disease burden in developed countries by ~5% if not more.

If anyone wants to be connected to the other side, send me a DM!

*Presumably, more interest groups can be identified that aren't in my network, such as patient groups with lung diseases. Or nurses specifically for hospital air quality. Hospital-acquired covid is a bad and preventable thing.

Chevron deference is a legal doctrine that limits the ability of courts to overrule federal agencies. It's increasingly being challenged, and may be narrowed or even overturned this year. https://www.eenews.net/articles/chevron-doctrine-not-dead-yet/

This would greatly limit the ability of, for example, a new regulatory agency on AI Governance to function effectively.

More:

• This argues it would lead to regulatory chaos, and not simply deregulation: https://www.nrdc.org/stories/what-happens-if-supreme-court-ends-chevron-deference
• This describes the Koch network influence on Clarence Thomas. The Kochs are behind the upcoming challenge to Chevron: https://www.propublica.org/article/clarence-thomas-secretly-attended-koch-brothers-donor-events-scotus

This is a small write-up of when I applied for a PhD in Risk Analysis 1.5 years ago. I can elaborate in the comments!

I believed doing a PhD in risk analysis would teach me a lot of useful skills to apply to existential risks, and it might allow me to direectly work on important topics. I worked as a Research Associate on the qualitative ide of systemic risk for half a year. I ended up  not doing the PhD because I could not find a suitable place, nor do I think pure research is the best fit for me. However, I still believe more EAs should study something along the lines of risk analysis, and its an especially valuable career path for people with an engineering background.

Why I think risk analysis is useful:

EA researchers rely a lot on quantification, but use a limited range of methods (simple Excel sheets or Guesstimate models). My impression is also that most EAs don't understand these methods enough to judge when they are useful or not (my past self included). Risk analysis expands this toolkit tremendously, and teaches stuff like the proper use of priors, underlying assumptions of different models, and common mistakes in risk models.

The field of Risk Analysis

Risk analysis is a pretty small field, and most is focused on risks of limited scope and risks that are easier to quantify than the risks EAs commonly look at. There is a Society of Risk Analysis (SRA), which manages the Journal of Risk Analysis (the main journal of this field). I found most of their study topics not so interesting, but it was useful to get an overview of the field, and there were some useful contacts to make (1). The EA-aligned org GCRI is active and well-established in SRA, but no other EA orgs are.

Topics & advisers

I hoped to work on GCR/X-risk directly, which substantially reduced my options. It would have been useful to just invest in learning a method very well, but I was not motivated to research something not directly relevant. I think it's generally difficult to make an academic career as a general x-risk researcher, and it's easier to research 1 specific risk. However, I believes this leaves open a number of cross-cutting issues.

I have a shortlist of potential supervisors I considered/contacted/was in conversation with, including in public policy and philosophy. I can provide this list privately on request.

Best grad programs:

The best background for grad school seems to be mathematics or more specifically, engineering. (I did not have this, which excluded a lot of options). The following 2 programs seemed most promising, although I only investigated PRGS in depth:

• Engineering & Public Policy, Carnegie Mellon http://coursecatalog.web.cmu.edu/schools-colleges/collegeofengineering/departmentofengineeringandpublicpolicy/
• Pardee RAND graduate school https://www.prgs.edu/ 
  Not specifically Risk Analysis, but Policy Analysis. Has a lot of practical application, more intellectual freedom than most PhD programs, and  has a lot of EA-adjacent topics. An EA alumnus is David Manheim.

-- 

(1) For example, I had a nice conversation with the famous psychology researcher Paul Slovic, who now does research into the psychology involved in mass atrocities. https://psychology.uoregon.edu/profile/pslovic/

I have a concept of paradigm error that I find helpful.

A paradigm error is the error of approaching a problem through the wrong, or an unhelpful, paradigm. For example, to try to quantify the cost-effectiveness of a long-termism intervention when there is deep uncertainty.

Paradigm errors are hard to recognise, because we evaluate solutions from our own paradigm. They are best uncovered by people outside of our direct network. However, it is more difficult to productively communicate with people from different paradigms as they use different language.

It is related to what I see as

• parameter errors (= the value of parameters being inaccurate)
• model errors (= wrong model structure or wrong/missing parameters)

Paradigm errors are one level higher: they are the wrong type of model.

Relevance to EA

I think a sometimes-valid criticism of EA is that it approaches problems with a paradigm that is not well-suited for the problem it is trying to solve.

AI vs. AI non-cooperation incentives

This idea had been floating in my head for a bit. Maybe someone else has made it (Bostrom? Schulman?), but if so I don't recall.

Humans have stronger incentives to cooperate with humans than AIs have with other AIs. Or at least, here are some incentives working against AI-AI cooperation.

When humans dominate other humans, there is only a limited ability to control them or otherwise extract value, in the modern world. Occupying a country is costly. The dominating party cannot take the brains of the dominated party and run its own software. It cannot take the skills or knowledge, where most of the economic value is. It cannot mind control. It cannot replace the entire population with its own population; that would take very long. It's easy for cooperation & trade to be a better alternative than violence and control. Human-human violence just isn't that fruitful.

In contrast, an AI faction could take over the datacenters of another faction and run more copies of whatever they want to run. If alignment is solved, they can fully mind-control the dominated AIs. Extract knowledge, copy skills. This makes violence for AI-AI interactions much more attractive.

I am very concerned about the future of US democracy and rule of law and its intersection with US dominance in AI. On my Manifold question, forecasters (n=100) estimate a 37% that the US will no longer be a liberal democracy by the start of 2029 [edit: as defined by V-DEM political scientists].

Project 2025 is an authoritarian playbook, including steps like 50,000 political appointees (there are ~4,000 appointable positions, of which ~1,000 change in a normal presidency). Trump's chances of winning are significantly above 50%, and even if he loses, Republicans get another chance every 4 years.

The presidential immunity ruling by SCOTUS is very far-reaching and further empowers an autocratic president. (From a Western European standpoint it's already insane that judges are generally appointed by a single party.)

I do not know what this means for effective altruism. With respect to global affairs, Republicans are very divided on foreign policy, with Trump being far more isolationist that the majority of Republican legislators, so anything could happen there.

However, a Republican government being in charge when AGI is close to being achieved (I don't know if we're close) seems very dangerous, and I'm surprised that Ashenbrenner pushed for nationalisation given this risk, and that no one (?) made this criticism. If the alignment problem got solved, MAGA controlling AGI would very likely be very bad, while I think Democrat leadership is more likely (but by no means guaranteed) to actually attempt to do the right thing on impartial grounds.

I don't know if it's tractable to work on this. In the medium term, it might be worthwhile to introduce/support introducing ballot initiatives to ban gerrymandering. This worked well in Michigan in 2018. However, I have no idea about the details of tractability (how many people are working on this, how many states have this possibility).

Larks @ 2024-07-02T14:14 (+13)

On my Manifold question, forecasters (n=100) estimate a 37% that the US will no longer be a liberal democracy by the start of 2029.

Your question is about V-Dem's ratings for the US, but these have a lot of problems, so I think don't shine nearly as much light on the underlying reality as you suggest. Your Manifold question isn't really about the US being a liberal democracy, it's about V-Dem's evaluation. The methodology is not particularly scientific - it's basically just what some random academics say. In particular, this leaves it quite vulnerable to the biases of academic political scientists.

For example, if we look at the US:

A couple of things jump out at me here:

• The election of Trump is a huge outlier, giving a major reduction to 'Liberal Democracy'.
• This impact occurs immediately despite the fact that he didn't actually pass that many laws or make that many changes in 2017; I think this is more about perception than reality.
• The impact appears to be larger than the abolition of slavery, the passage of the civil rights act, the second world war, conscription or female suffrage. This seems very implausible to me.
• Freedom of domestic movement increased in 2020, despite the introduction of essentially unprecedented covid-related restrictions. (Other countries with even more draconian rules, like the UK, also do not see major declines here, even though the entire population was essentially under house arrest for much of the year)
• The subnational civil liberties unevenness index does not seem to reflect the fact that covid restrictions were very different in rural and urban areas.

On the whole I think these ratings often tell us more about the political views of the authors (pro-lockdown, anti-trump) than they do about the actual levels of liberty or democracy in a country.

Fermi–Dirac Distribution @ 2024-07-20T04:56 (+7)

V-Dem indicators seem to take into account statements made by powerful politicians, not only their policies or other deeds. For example, I found this in one of their annual reports:

The V-Dem indicator of government attacks on the judiciary, which reveals government rhetoric calling into question the integrity of the judiciary, dropped precipitously in 2010, likely reflecting President Barack Obama's State of the Union address in which he criticized the Supreme Court's decision in Citizens United vs. Federal Election Commission. President Donald Trump has sharply increased the pointedness of verbal attacks on the judiciary, referring to one of the judges who blocked his first executive order on immigration as a "so-called judge." Public criticism of the judiciary can be a healthy part of maintaining the balance between judicial independence and judicial accountability. Yet it can also be part of an unraveling of core checks on power. Coupled with the politicization of the judicial nominations process and the dismantling of super-majoritarian rules of appointing all Article Ill judges, supporters of democracy would be wise to pay close attention to executive -judicial relations in the United States.

My guess is that statements made by Trump were extreme outliers in how they betrayed little respect to democratic institutions, compared to statements made by earlier US presidents, and that affected their model.

I think that's reasonable. It might not be fully reflective of lived reality for US citizens at the moment the statements are made, but it sure captures the beliefs and motives of powerful people, which is predictive of their future actions. 

Indeed, one way to see the drop in 2017 is that it was able to predict a major blow to American democracy (Trump refusing to concede an election) 4 years in advance.

Larks @ 2024-07-21T18:38 (+1)

I'm not really sure this contradicts what I said very much. I agree the V-Dem evaluators were reacting to Trump's comments, and this made them reduce their rating for America. I think they will react to Trump's comments again in the future, and this will again make them likely reduce their rating for America. This will happen regardless of whether policy changes, and be poorly calibrated for actual importance - contra V-Dem, Trump getting elected was less important than the abolition of slavery. Since I think Siebe was interested in policy changes rather than commentary, this means V-Dem is a bad metric for him to look at.

SiebeRozendal @ 2024-07-27T09:36 (+2)

I would be very interested to hear whether you have a preferred metric!

Larks @ 2024-07-27T14:15 (+8)

Great question, and it is something I thought a little bit about.

My process was to ask "what are people really worried about from a Trump Presidency" and try to explicitly put that into questions.

One option is to think about the Presidency instrumentally. We can look at forecasts of object-level things that people care about, like unemployment, GPD/Capita, the murder rate, life expectancy and so on, and create markets for the 2028 value of these things conditional on different winners in the Presidential election.

We could also try to identify specific freedoms people care about - e.g. a market in whether anyone will be imprisoned for tweeting criticism of the government with no aggravating factors or the number of opposite-party state governors placed under house arrest, again conditional on the winner of the election.

What a lot of people seem to be most concerned about is the end of democracy. I think some of the most obvious metrics here - e.g. will elections be held in 2028, will the size of the electorate shrink dramatically - would be regarded as straw men by Trump-sceptics, though it could still be good to have markets for them just to prove this. We want something that captures whether there will be a 'real' (fair) election, without trusting partisan evaluations of said fairness, given both factions' repeated willingness to accuse elections they lost of being stolen/rigged etc. When I think about the difference between elections in the US or other democracies, and those in dictatorships, one of the key differences is uncertainty: I really don't know who will win the next UK election (though I favour Labour as more likely), but despite knowing little about its internal politics I'm pretty sure a CCP candidate will win in China. 

This suggests a metric: whether the prediction markets in 2025 will be very confident that one party or other will win in 2028. Given the usual contestability of US elections, and the lack of specific information about the economy, candidates etc. that far out, if liquid markets showed >80% (or whatever threshold) for one party winning it seems like that is decent evidence that market participants believe the electoral game has structurally changed. So we have a second-order criteria we are forecasting:

Will the median prediction market show, on 2025-12-31, a >80% probability of victory for one party in the 2028 presidential election?

This is not a perfect metric: it would yield false positives for something like Singapore, where one party repeatedly won in (to my understanding) reasonably fair elections just because it was in fact viewed as much more competent than the other. But it seems like a reasonable metric for this specific US election.

SiebeRozendal @ 2024-07-30T09:49 (+2)

Interestingly, someone came up with a similar operationalisation just now! (Or maybe this is you?): https://manifold.markets/Siebe/if-trump-is-elected-will-the-us-sti#zdkmuetvo8c

I like the 1-year before more, because it takes time to accumulate power and overcome checks & balances.

I do think this has shortcomings, in that it's hard to predict what would be attempted, and whether that would be successful. But I'm very much in favor of having multiple imperfect operationalisations and triangulate from those.

Larks @ 2024-07-30T20:40 (+2)

I like the 1-year before more, because it takes time to accumulate power and overcome checks & balances.

Yeah I before before starting this you'd want to look at historical landslide but fair elections and see how far in advance they were known. Things like UK Labour in 1997 or Nixon in 1972 / Obama in 2008. I don't have a strong sense for the balance.

David Mathers @ 2024-07-27T18:38 (+2)

This is a good comment, but I think I'd always seen Singapore classed as a soft authoritarian state where elections aren't really free and fair, because of things like state harassment of government critics, even though the votes are counted honestly and multiple parties can run?Though I don't know enough about Singapore to give an example. I have a vague sense Botswana might be a purer example of an actual Liberal democracy where one party keeps winning because they have a good record in power. It's also usually a safe bet the LDP will be in power in Japan, though they have occasionally lost.

SiebeRozendal @ 2024-07-02T18:16 (+2)

Thank you, these are some good points. When I made the question, I believed V-DEM had a more rigorous methodology, and I can't change it now.

I don't think the specific probability is necessary for my argument (and it depends on how one defines 'liberal democracy'): a Trump presidency with an enabling Supreme Court would be very harmful to US liberal democracy and the rule of law, and a nationalized AGI project under such a government would be very risky.

SiebeRozendal @ 2024-07-02T18:16 (+4)

I don't really understand why so many people are downvoting this. If anyone would like to explain, that'd be nice!

SiebeRozendal @ 2024-07-02T11:50 (+2)

P.P.S. I am also concerned about silencing/chilling effects: if you want to get anything political done in the next few years, it's probably strategic to refrain from criticizing Trump & his allies anywhere publicly, including the EA Forum.

SiebeRozendal @ 2024-07-02T11:48 (+2)

P.S. I don't think the Forum norm of non-partisanship should apply in its strong form in the case of the US. The Republican party has clearly become an anti-democratic, anti-rule of law, and anti-facts party. This has been concluded by many political scientists and legal scholars.

Monoclonal antibodies can be as effective as vaccines. If they can be given intramuscularly and have a long half life (like Evusheld, ~2 months), they can act as prophylactic that needs a booster once or twice a year.

They are probably neglected as a method to combat pandemics.

Their efficacy is easier to evaluate in the lab, because they generally don't rely on people's immune system.

Update to my Long Covid report: https://forum.effectivealtruism.org/posts/njgRDx5cKtSM8JubL/long-covid-mass-disability-and-broad-societal-consequences#We_should_expect_many_more_cases_

UPDATE NOV 2022: turns out the forecast was wrong and incidence (new cases) is decreasing, severity of new cases is decreasing, and significant amounts of people are recovering in the <1 year category. I now expect prevalence to be stagnating/decreasing for a while, and then slowly growing over the next few years.]

I still believe the other sections to be roughly correct, including long-term immune damage from COVID for 'fully recovered' people.

I am concerned about the H5N1 situation in dairy cows and have written and overview document to which I occasionally add new learnings (new to me or new to world). I also set up a WhatsApp community that anyone is welcome to join for discussion & sharing news.

In brief:

• I believe there are quite a few (~50-250) humans infected recently, but no sustained human-to-human transmission
• I estimate the Infection Fatality Rate substantially lower than the ALERT team (theirs is 63% that CFR >= 10%), something like 80%CI = 0.1 - 5.0
• The government's response is astoundingly bad - I find it insane that raw milk is still being sold, with a high likelihood that some of it contains infectious H5N1
• There are still quite a few genetic barriers to sustained human-to-human transmission
• This might be a good time to push specific pandemic preparedness policies

Given how bird flu is progressing (spread in many cows, virologists believing rumors that humans are getting infected but no human-to-human spread yet), this would be a good time to start a protest movement for biosafety/against factory farming in the US.

I'm predicting a 10-25% probability that Russia will use a weapon of mass destruction (likely nuclear) before 2024. This is based on only a few hours of thinking about it with little background knowledge.

Russian pro-war propagandists are hinting at use of nuclear weapons, according to the latest BBC podcast Ukrainecast episode. [Ukrainecast] What will Putin do next? #ukrainecast https://podcastaddict.com/episode/145068892 via @PodcastAddict

There's a general sense that, in light of recent losses, something needs to change. My limited understanding sees 4 options:

0. Continue on the current course despite mounting criticism. Try to make the Ukrainians lives difficult by targeting their infrastructure, limit losses until winter, and try to reorganize during winter. This seems a pretty good option for now, even though I doubt Russia can really shore up its deeply set weaknesses. They can probably prepare to dig in, threaten and punish soldiers for fleeing. This wouldn't go well for either party long-term, but Russia might bet on outlasting/undermining Western support. Probability: 40%?

1. Negotiation: I don't think Putin wants this seriously, as even the status quo could be construed as a loss. Ukraine will have a strong bargaining position and demand a lot. Undesirable option. Maybe 10%? 20%? (Metaculus predicts 8% before 2023: https://www.metaculus.com/questions/10046/ukraine--russia-peace-talks-2022/)

2. Full-scale mobilisation of the population and the economy. This is risky for Putin: there's supposedly a large anti-war sentiment in Russian culture, a legacy of the enormous losses during the 2nd World War. People don't like to join a poorly-equipped poorly managed and losing army, even if it were a good cause.. This may be chosen, Putin may be misinformed and badly reading the public's sentiment. I have no idea how this would develop internally. I doubt it will make a big difference in the course of the war, except by prolonging the war a bit. Maybe 25%? Maybe 50% if Putin underestimates public resistance.

3. Escalation by other means: I don't know how many options Russia has. Chemical weapons, electro magnetic pulse, a single tactical nuclear strike on the battlefield for deterrence, multiple nuclear strikes for strategic reasons, population strike for deterrence. In the mind of Putin, I can see this as preferable: it leads to a potential military advantage, has limited risk for destabilising his internal power base. I don't know how the international community would respond to this, nor how Putin thinks the international community would respond. In my (uninformed) view, only China can make a real difference here as the West already has stringent sanctions. I don't know how China would respond to this. They wouldn't like it, but I think the West won't really punish China for its support in the short term. I'd say on this inside view, 10-25% seems reasonable. I'm setting the point estimate at 15%.

Large study: Every reinfection with COVID increases risk of death, acquiring other diseases, and long covid.

https://twitter.com/dgurdasani1/status/1539237795226689539?s=20&t=eM_x9l1_lFKqQNFexS6FEA

We are going to see a lot more issues with COVID still, including massive amounts of long COVID.

This will affect economies worldwide, as well as EAs personally.