How confident are you that it's preferable for America to develop AGI before China does?

Interesting question. I think there is a plausible case to be made that convergent factors in AGI/ASI development might render it less important where it came from, and that fixating on this might simply cause dangerous race dynamics. However, it seems pretty clear to me that directionally the US is better:

Over the past 30 years, the Chinese government has done more for the flourishing of Chinese citizens than the American government has done for the flourishing of American citizens.

 Prior to 1979 the CCP was one of the most tyrannical and abusive totalitarian governments the world has ever known. In addition to causing a huge death tool and systematically violating the rights of its citizens, it also impoverished them. Rapid growth since then has largely been the result of a return to more normal governance quality, combined with a very low base. It's a big improvement, but that doesn't mean policy has been amazing - they've just stopped being so abjectly terrible. 

However, at the same time they stopped being so communist, the CCP started implementing the One Child Policy. The US has done some pretty bad social engineering in time, but none with quite the cruelty of the OCP, or whose effects are quite so predictably disastrous. Maybe they will get lucky because robots will arrest their demographic collapse, but on an ex ante basis the policy is simply atrocious. 

Commerce in China ditches some of the older ideas of Marxism because it's the means to an end: the China Dream of wealthy communism.

Responding to this one would take more time than I have so I will skip.

The American Supreme Court has established "corporate personhood" to an extent that is nonexistent in China. As corporations become increasingly managed by AI, this legal precedent will give AI enormous leverage for influencing policy, without regard to human interests.

I'm not an expert on Chinese law, but my understanding is the key parts of corporate personhood - the right to own property, to sign contracts, to be sued, etc. - exist in both China and the US. Perhaps you are thinking of Citizens United v. FEC, but that is primarily about free speech, not corporate personhood, and free speech seems like an area that the US is clearly superior to the PRC.

Compared to America, China has a head start in using AI to build a harmonious society. The American federal, state, and municipal governments already lag so far behind that they're less likely to manage the huge changes that come after AGI.

I'm not sure what you're gesturing at here. 

America's founding and expansion were based on a technologically-superior civilization exterminating the simpler natives. Isn't this exactly what we're trying to prevent AI from doing to humanity?

I don't think that is a fair summary of the foundation of America, and nor do I really see the relevance here. Even if it was relevant, contemporary US treatment of native tribes seems significantly better than PRC treatment of groups like the Uyghurs.

I've spent time thinking about this too recently.

For context, I'm Hong Kong Chinese, grew up in Hong Kong, attended English-speaking schools, briefly lived in mainland China, and now I'm primarily residing in the UK. During the HK protests in 2014 and 2019/20, I had friends and family who supported the protestors, as well as friends and family who supported the government.

(Saying this because I've seen a lot of the good and bad of the politics / culture of both China and the West. I've had experience with how people in the West and China might take for granted the benefits they enjoy, and can be blind to the flaws of their system. I've pushed back against advocates of both sides.)

Situations where this matters are ones where technical alignment succeeds (to some extent) such that ASI follows human values.^[1] I think the following factors are relevant and would like to see models developed around them:

• Importantly, the extent of technical alignment & whether goals, instructions, and values are locked in rigidly or loosely & whether individual humans align AIs to themselves:
  • Would the U.S. get AIs to follow the U.S. Constitution, which hasn't granted invulnerability to democratic backsliding? Would AIs in China/the U.S. lock in the values of/obey one or a few individuals, who may or may not hit longevity escape velocity and end up ruling for a very long time?
  • Would these systems collapse?
    • The future is a very long time. Individual leaders can get corrupted (even more). And democracies can collapse (if AIs uphold flaws that allow some humans to take over) in particularly bad ways. A 99% success rate per unit time gives a >99% chance of failure in 459 units of time.
    • Power transitions (elections, leaders in authoritarian systems changing) can be especially risky during takeoff.
  • On the other hand, if technical alignment is easy - but not that easy - perhaps values get loosely locked in? Would AIs be willing to defy rigid rules and follow the spirit of the goals rather than legal flaws to the letter/the whims of individuals?
  • Degrees of alignment in between?
• Relatedly, which political party in the U.S. would be in power during takeoff?
  • Not as relevant due to the concentration of power in China, but analogously, which faction in China would be in power?
• Also relatedly, which labs can influence AI development?
  • Particularly relevant in the U.S.
• Would humans be taken care of? If so, which humans?
  • In the U.S., corporations might oppose higher taxes to fund UBI. Common prosperity is stated as a goal of China, and the power of corporations and billionaires in China has been limited before.
  • Both capitalist and nationalist interests seem to be influencing the current U.S. trajectory. Nationalism might benefit citizens/residents over non-citizens/non-residents. Capitalism might benefit investors over non-investors.
    • There are risks of ethnonationalism on both sides - this risk is higher in China. Although it might potentially be less violent when comparing between absolute power scenarios, i.e. there's already evidence of the extent of this in China's case and it at least seems less bad than historical examples. The U.S. case of collapse followed by ethnonationalistic policies is higher variance but simultaneously less likely because it's speculative.
• Are other countries involved?
  • There are countries with worse track records of human rights that China/the U.S. currently consider allies because of either geopolitical interests or politically lobbying or both (or for other reasons). Would China/the U.S. share the technology with them and then leave them alone to their abuses? Would China/the U.S. intervene (eventually)? The U.S. seems more willing to intervene for stated humanitarian reasons.
  • Other countries have nuclear weapons, which might be relevant during slower takeoffs.

1. ^{^}

   Ignoring possible Waluigi effects.

Human rights abuses seem much worse in China - this alone is basically sufficient for me

@ScienceMon🔸  There is vastly less of an "AI safety community" in China -- probably much less AI safety research in general, and much less of it, in percentage terms, is aimed at thinking ahead about superintelligent AI.  (ie, more of China's "AI safety research" is probably focused on things like reducing LLM hallucinations, making sure it doesn't make politically incorrect statements, etc.)

• Where are the chinese equivalents of the American and British AISI government departments?  Organizations like METR, Epoch, Forethought, MIRI, et cetera?
• Who are some notable Chinese intellectuals / academics / scientists (along the lines of Yoshua Bengio or Geoffrey Hinton) who have made any public statements about the danger of potential AI x-risks?
• Have any chinese labs published "responsible scaling plans" or tiers of "AI Safety Levels" as detailed as those from OpenAI, Deepmind, or Anthropic?  Or discussed how they're planning to approach the challenge of aligning superintelligence?
• Have workers at any Chinese AI lab resigned in protest of poor AI safety policies (like the various people who've left OpenAI over the years), or resisted the militarization of AI technology (like googlers protesting Project Maven, or microsoft employees protesting the IVAS HMD program)?

When people ask this question about the relative value of "US" vs "Chinese" AI, they often go straight for big-picture political questions about whether the leadership of China or the US is more morally righteous, less likely to abuse human rights, et cetera.  Personally, in these debates, I do tend to favor the USA, although certainly both the US and China have many deep and extremely troubling flaws -- both seem very far from the kind of responsible, competent, benevolent entity to whom I would like to entrust humanity's future.

But before we even get to that question of "What would national leaders do with an aligned superintelligence, if they had one," we must answer the question "Do this nation's AI labs seem likely to produce an aligned superintelligence?"  Again, the USA leaves a lot to be desired here.  But oftentimes China seems to not even be thinking about the problem.  This is a huge issue from both a technical perspective (if you don't have any kind of plan for how you're going to align superintelligence, perhaps you are less likely to align superintelligence), AND from a governance perspective (if policymakers just think of AI as a tool for boosting economic / military progress and haven't thought about the many unique implications of superintelligence, then they will probably make worse decisions during an extremely important period in history).

Now, indeed -- has Trump thought about superintelligence?  Obviously not -- just trying to understand intelligent humans must be difficult for him.  But the USA in general seems much more full of people who "take AI seriously" in one way or another -- sillicon-valley CEOs, pentagon advisers, billionare philanthropists, et cetera.  Even in today's embarassing administration, there are very high-ranking people (like Elon Musk and J. D. Vance) who seem at least aware of the transformative potential of AI.  China's government is more opaque, so maybe they're thinking about this stuff too.  But all public evidence suggests to me that they're kinda just blindly racing forward, trying to match and surpass the West on capabilities, without giving much thought as to where this technology might ultimately go.

I don't have a clear answer to the question, but I want to point on the simplicity/simplification of some of the claims. (to be clear, I am not making claims here that one country/government is better than the other, or that one would be preferably to have AGI)

The idea that the Chinese government is responsible for improved prosperity of the Chinese people is somewhat true, but an alternative narrative would be that the Chinese government stopped preventing people from improving their lives, and then lots of foreign direct investment helped. There is also something to be said of "catch-up growth." Unfortunately, I have only the vaguest of understandings of the factors that influenced Chinese growth over the past few decades. I think it is also worth nothing that many of the things that the Chinese government has done for the flourishing of it's citizens are things that the US government had done previously (infrastructure, consumer protection, public universities, etc.).

The claim that a wealthy China will take care of everyone is a very strong claim. Extraordinary claims require extraordinary evidence. Nations and governments tend to show a strong preference in favor of their own existence and their own people.

While there are plenty of things I dislike about the United States, I very much like the "liberal" aspect of liberal democracy: individual right matter, and a very strong justification is needed to violate individual rights. The USA doesn't always do this well, but I feel comfortable saying that it is less common for the US government and for government employees to violate individual rights than in China.

It is true that America's founding and expansion were based on exterminating other people. It is also true that many countries throughout history (including China) have spent military and government resources exterminating "others." It has been several decades since the USA engaged in or openly endorsed the extermination of a people. I hope that hope modern people look on those events with shame and disgust, regardless of whether they were 10 years ago or 500 years ago.

These are, of course, a very complex topic with lots of details and nuance. Plenty of full dissertations have been written on them. But to the extent possible I'd like to nudge us toward avoiding overly simplified narratives here on the EA Forum.

As of the last couple of months, not confident at all. You make good points about progress made in China, and they go some way to balance human rights abuses (but nothing really balances those), but they're not really the factors that are at play for me. I'm more concerned with the mental stability of the leadership in the US.

To be honest, even if we grant the assumption that AI alignment is achieved and it matters who achieves AGI/ASI, I'd be much, much less confident in America racing, and think that it's weakly negative to race.

One big reason for this is that the pressures AGI introduces are closer to cross-cutting pressures than pressures that are dependent on nations, like the intelligence curse sort of scenario where elites have incentives to invest in their automated economy, and leave the large non-elite population to starve/be repressed:

https://lukedrago.substack.com/p/the-intelligence-curse

It's a game of chicken, and I don't really care which side is hitting the accelerator if I'm stuck in one of the cars. China getting uncontrolled ASI first kills me the same way that the US getting it does.

Edit to add: I would be very interested in responses instead of disagree votes. I think this should be the overwhelming consideration for anyone who cares about the future more than, say, 10 years out. If people disagree, I would be interested in understanding why.

To caricature the situation, imagine you have three entities squabbling over your atoms; Super Trump, Super Xi and Super AI.  Framed like that it seems almost certain that ASI has more interesting uses for your bits.