PSA: Apropos of nothing, did you known you can hide the community section?

(You can get rid of it entirely in your settings as well.)

Vaidehi Agarwalla @ 2023-09-08T14:38 (+16)

Is there a way to snooze the community tab or snooze / hide certain posts? I would use this feature.

Lorenzo Buonanno @ 2023-09-08T15:38 (+8)

Is there a way to do this for community quick takes? Most of the quick takes at this moment seem to be about the community.

MathiasKB @ 2023-09-08T14:29 (+6)

Thanks, you just bought me days of productivity

Appreciation post for Saulius

I realized recently that the same author that made the corporate commitments post and the misleading cost effectiveness post also made all three of these excellent posts on neglected animal welfare concerns that I remembered reading.

Fish used as live bait by recreational fishermen

Rodents farmed for pet snake food

35-150 billion fish are raised in captivity to be released into the wild every year

For the first he got this notable comment from OpenPhil's Lewis Bollard. Honorable mention includes this post which I also remembered, doing good epistemic work fact-checking a commonly cited comparison.

Offer of help with hands-free input

If you're experiencing wrist pain, you might want to take a break from typing. But the prospect of not been able to interact with the world might be holding you back. I want to help you try voice input. It's been helpful for me to go from being scared about my career and impact to being confident that I can still be productive without my hands. (In fact this post is brought to you by nothing but my voice.) Right now I think you're the best fit if you:

• Have ever written code, even a small amount, or otherwise feel comfortable editing a config file
• Are willing to give it a few days
• Have a quiet room where you can talk to your computer

Make sure you have a professional microphone — order this mic if not, which should arrive the next day.

Then you can book a call with me. Make sure your mic will arrive by the time the call is scheduled.

The Forum went down this morning, for what is I believe the longest outage we've ever had. A bot was breaking our robots.txt and performing a miniature (unintentional) DOS attack on us. This was perfectly timed with an upgrade to our infrastructure that made it hard to diagnose what was going on. We eventually figured out what was going on and blocked the bot, and learned some lessons along the way. My apologies on behalf of the Forum team :(

Here’s a puzzle I’ve thought about a few times recently:

The impact of an activity ($I$) is due to two factors, $X$ and $Y$. Those factors combine multiplicatively to produce impact. Examples include:

• The funding of an organization and the people working at the org
• A manager of a team who acts as a lever on the work of their reports
• The EA Forum acts as a lever on top of the efforts of the authors
• A product manager joins a team of engineers

Let’s assume in all of these scenarios that you are only one of the players in the situation, and you can only control your own actions.

From a counterfactual analysis, if you can increase your contribution by 10%, then you increase the impact by 10%, end of story.

From a Shapley Value perspective, it’s a bit more complicated, but we can start with a prior that you split your impact evenly with the other players.

Both these perspectives have a lot going for them! The counterfactual analysis has important correspondences to reality. If you do 10% better at your job the world gets $0.1I$ better. Shapley Values prevent the scenario where the multiplicative impact causes the involved agents to collectively contribute too much.

I notice myself feeling relatively more philosophically comfortable running with the Shapely Value analysis in the scenario where I feel aligned with the other players in the game. And potentially the Shapley Value approach downsides go down if I actually run the math (Fake edit: I ran a really hacky guess as to how I’d calculate this using this calculator and it wasn’t that helpful).

But I don’t feel 100% bought-in to the Shapley Value approach, and think there’s a value in paying attention to the counterfactuals. My unprincipled compromise approach would be to take some weighted geometric mean and call it a day.

Interested in comments.

With the US presidential election coming up this year, some of y’all will probably want to discuss it.^[1] I think it’s a good time to restate our politics policy. tl;dr Partisan politics content is allowed, but will be restricted to the Personal Blog category. On-topic policy discussions are still eligible as frontpage material.

1. ^{^}

   Or the expected UK elections.

If you're copy-pasting from a Google Doc, you can now copy-paste footnotes into the WYSIWYG editor in the following way (which allowed us to get around the impossibility of selecting text + footnotes in a Google Doc): 

1. Publish your Google Doc to web
   1. You can do this by clicking on File > Share > Publish to web
   2. Then approve the pop-up asking you to confirm (hit "Publish")
   3. Then open the link that you'll be given; this is now the published-to-web version of your document.
2. Select the whole text, including footnotes, and copy that. (If you'd like, you can now unpublish the document.)
3. Open the Forum text editor (WYSIWYG), and paste the selection.

We'll announce this in a top level post as part of a broader feature announcement soon, but I wanted to post this on my shortform to get this in to the minds of some of the authors of the Forum.

Thanks to Jonathan Mustin for writing the code that made this work.

The new Forum turns 1 year old today.

🎵Happy Birthday to us 🎶

We now have google-docs-style collaborative editing. Thanks a bunch to Jim Babcock of LessWrong for developing this feature.

How hard should one work?

Some thoughts on optimal allocation for people who are selfless but nevertheless human.

Baseline: 40 hours a week.

Tiny brain: Work more get more done.

Normal brain: Working more doesn’t really make you more productive, focus on working less to avoid burnout.

Bigger brain: Burnout’s not really caused by overwork, furthermore when you work more you spend more time thinking about your work. You crowd out other distractions that take away your limited attention.

Galaxy brain: Most EA work is creative work that benefits from:

• Real obsession, which means you can’t force yourself to do it.
• Fresh perspective, which can turn thinking about something all the time into a liability.
• Excellent prioritization and execution on the most important parts. If you try to do either of those while tired, you can really fuck it up and lose most of the value.

Here are some other considerations that I think are important:

• If you work hard you contribute to a culture of working hard, which could be helpful for attracting the most impactful people, who are more likely than average in my experience to be hardworking.
• Many people will have individual-specific reasons not to work hard. Some people have mental health issues that empirically seem to get worse if they work too hard, or they would get migraines or similar. Others will just find that they know themselves well enough to know when they should call it quits, for reasons captured elsewhere in this doc or not. This makes me usually very reluctant to call someone else out for not working hard enough.

A word on selflessness — I’m analyzing this from the perspective of someone trying to be purely selfless. I think it’s a useful frame. But I also think most people should make the decision about how much they work from the perspective of someone with the actual goals they have. It is a whole nother much more complicated blog post to flesh that out.

Finally, I want to say that although this post makes it seem like I’m coming down on the side of working less hard, I do overall think the question is complicated, and I definitely don’t know what the right answer is. This is mostly me writing in response to my own thinking, and to a conversation I recently had with my friend. My feeling from reading the discussions the Forum’s had about it, the conversation rarely gets past the normal brain take, plausibly because it seems like a bad look to argue the case for working harder. If I were writing to try to shift the state of public discussion, I would probably argue the bigger brain take more. But this is shortform, so it’s written for me.

I'd like to try my hand at summarizing / paraphrasing Matthew Barnett's interesting twitter thread on the FLI letter.^[1]

The tl;dr is that trying to ban AI progress will increase the hardware overhang, and risk the ban getting lifted all of a sudden in a way that causes a dangerous jump in capabilities.

Background reading: this summary will rely on an understanding of hardware overhangs (second link), which is a somewhat slippery concept, and I myself wish I understood at a deeper level.

***

Barnett Against Model Scaling Bans

Effectiveness of regulation and the counterfactual

It is hard to prevent AI progress. There's a large monetary incentive to make progress in AI, and companies can make algorithmic progress on smaller models. "Larger experiments don't appear vastly more informative than medium sized experiments."^[2] The current proposals on the table on ban the largest runs.

Your only other option is draconian regulation, which will be hard to do well and will unpredictable and bad effects.

Conversely, by default, Matthew is optimistic about companies putting lots of effort into alignment. It's economically incentivized. And we can see this happening: OpenAI has put more effort into aligning its models over time, and GPT-4 seems more aligned than GPT-2.

But maybe some delay on the margin will have good effects anyway? Not necessarily:

Overhang

Matthew's arguments above about algorithmic progress still occurring imply that AI progress will occur during a ban.^[3] Given that, the amount of AI power that can be wrung out humanity's hardware stock will be higher at the end of the ban than at the start. What are these consequences of that? Nothing good, says Matthew:

First, we need to account for the sudden jump in capabilities when the ban is relaxed. Companies will suddenly train up to the economically incentivized levels, leading to a discontinuous jump in capabilities. Discontinuous jumps in capabilities are more dangerous than incremental improvements. This is the core of the argument, according to my read.

Maybe we can continue the ban indefinitely? Sounds extremely risky to Matthew. Matthew is worried that we will try, and then the overhang will get worse and worse until for some reason (global power conflict?) regulators decide to relax the ban, and a more massive discontinuous jump occurs.

As an additional consideration, there will now be more competitive pressure, as the regulation has evened the playing field by holding back the largest labs, leading to more incentive to cut corners in safety.

Fixed budget of delays

[Caveat lector: It seems likely to me that I'm not grokking Matthew here, this section is rougher and more likely to mislead you]: Matthew claims that the there's only so much budget of delays that humanity has. He argues that humanity should spend that budget later, when AI is closer to being an existential threat, rather than now.

Matthew considers the argument that we will wait too long to spend our budget and reach AGI before we do, but rejects that because he believes it will be obvious when AI is getting dangerous.

***

I consider the arguments presented here to be overall quite strong. I may come back and write up a response, but this is basically just a passthrough of Matthew, if I've done it right. Which brings me to:

Disclaimer: I say "Matthew is worried", and other such statements. I have not run this by Matthew. I am writing this based on my understand from his twitter thread, see the original for the ground truth of what he actually said.

1. ^{^}

   Thanks to Lizka for pointing me to it.

2. ^{^}

   Seems like a controversial claim, I'd be curious for someone more knowledgable than I to weigh in.

   It's not obvious that this is crux-y for me though. See next bullet point.

3. ^{^}

   Matthew does not mention the ongoing march of compute progress. I don't understand why. It seems to me to make his argument stronger. Compute progress will lead to overhang just as surely as algorithmic progress, and seems about as hard to stop.

Note: due to the presence of spammers submitting an unusual volume of spam to the EA Topics Wiki, we've disabled creation and editing of topics for all users with <10 karma.

I've just released the first step towards having EA Forum tags become places where you can learn more about a topic after read a post. Posting about it because it's a pretty noticeable change:

The tag at the top of the post makes it easier to notice that there's a category to this post. (It's the most relevant core tag to the post.) We had heard feedback from new users that they didn't know tags were a thing.

Once you get to the tag page, there's now a header image. More changes to the tag pages to give it more structure are coming.

Consider time discounting in your cost-benefit calculations of avoiding tail risks

[Epistemic status: bit unendorsed, written quickly]

For many questions of tail risks (covid, nuclear war risk) I've seen EAs^[1] doing a bunch of modeling of how likely the tail risk is, and what the impact will be, maybe spending 10s of hours on it. And then when it comes time to compare that tail risk vs short term tradeoffs, they do some incredibly naive multiplication, like life expectancy * risk of death.

Maybe if you view yourself as a pure hedonist with zero discounting, but that's a hell of a mix. I at least care significantly about impact in my life, and the impact part of my life has heavy instrumental discounting. If I'm working on growing the EA community, which is exponentially growing at ~20% per year, then I should discount my future productivity at 20% per year.

[This section especially might be wrong] So altruistically, I should be weighing the future at a factor of about (not gonna show my work)  ${\int }_0^{40}{e}^{-.2t}dt$ times my productivity for a year, which turns out to be a factor of about 5.

Maybe you disagree with this! Good! But now this important part of the cost-benefit calculation is getting the attention it deserves, rather than being a complete afterthought.

1. ^{^}

   Myself included.

Posting this on shortform rather than as a comment because I feel like it's more personal musings than a contribution to the audience of the original post —

Things I'm confused about after reading Will's post, Are we living at the most influential time in history?:

What should my prior be about the likelihood of being at the hinge of history? I feel really interested in this question, but haven't even fully read the comments on the subject. TODO.

How much evidence do I have for the Yudkowsky-Bostrom framework? I'd like to get better at comparing the strength of an argument to the power of a study.

Suppose I think that this argument holds. Then it seems like I can make claims about AI occurring because I've thought about the prior that I have a lot of influence. I keep going back and forth about whether this is a valid move. I think it just is, but I assign some credence that I'd reject it if I thought more about it.

What should my estimate of the likelihood we're at the HoH if I'm 90% confident in the arguments presented in the post?

This first shortform comment on the EA Forum will be both a seed for the page and a description.

Shortform is an experimental feature brought in from LessWrong to allow posters a place to put quickly written thoughts down, with less pressure to make it to the length / quality of a full post.

Thus starts the most embarrassing post-mortem I've ever written.
The EA Forum went down for 5 minutes today. My sincere apologies to anyone who's Forum activity was interrupted.
I was first alerted by Pingdom, which I am very glad we set up. I immediately knew what was wrong. I had just hit "Stop" on the (long unused and just archived) CEA Staff Forum, which we built as a test of the technology. Except I actually hit stop on the EA Forum itself. I turned it back on and it took a long minute or two, but was soon back up.
...
Lessons learned:
* I've seen sites that, after pressing the big red button that says "Delete", makes you enter the name of the service / repository / etc. you want to delete. I like those, but did not think of porting it to sites without that feature. I think I should install a TAP that whenever I hit a big red button, I confirm the name of the service I am stopping.
* The speed of the fix leaned heavily on the fact that Pingdom was set up. But it doesn't catch everything. In case it doesn't catch something, I just changed it so that anyone can email me with "urgent" in the subject line and I will get notified on my phone, even if it is on silent. My email is jp at organizationwebsite.

On the incentives of climate science

Alright, the title sounds super conspiratorial, but I hope the content is just boring. Epistemic status: speculating, somewhat confident in the dynamic existing.

Climate science as published by the IPCC tends to

1) Be pretty rigorous

2) Not spend much effort on the tail risks

I have a model that they do this because of their incentives for what they're trying to accomplish.

They're in a politicized field, where the methodology is combed over and mistakes are harshly criticized. Also, they want to show enough damage from climate change to make it clear that it's a good idea to institute policies reducing greenhouse gas emissions.

Thus they only need to show some significant damage, not a global catastrophic one. And they want to maintain as much rigor as possible to prevent the discovery of mistakes, and it's easier to be rigorous about things that are likely than about tail risks.

Yet I think longtermist EAs should be more interested in the tail risks. If I'm right, then the questions we're most interested in are underrepresented in the literature.

Here's a thought you might have: "AI timelines are so short, it's significantly reducing the net present value^[1] of {high school, college} students." I think that view is tempting based on eg (1, 2). However, I claim that there's a "play to your outs" effect here. Not in a "AGI is hard" way, but "we slow down the progress of AI capability development".

In those worlds, we get more time. And, with that more time, it sure would seem great if we could have a substantial fraction of the young workforce care about X-Risk / be thinking with EA principles. Given the success we've had historically in convincing young people of the value of these ideas, it still seems pretty promising to have some portion of our community continue to put work into doing so.

1. ^{^}

   By this I mean, the lifetime impact of the student, discounted over time.

Note from the Forum team: The Forum was intermittently down for roughly an hour and change last night/this morning, due to an issue with our domain provider.

Temporary site update: I've taken down the allPosts page. It appears we have a bot hitting the page, and it's causing the site to be slow. While I investigate, I've simply taken the page down. My apologies for the inconvenience.

Here's an interesting tweet from a thread by Ajeya Cotra:

But I'm not aware of anyone who successfully took complex deliberate *non-obvious* action many years or decades ahead of time on the basis of speculation about how some technology would change the future.

I'm curious for this particular line to get more discussion, and would be interested in takes here.

A tax, not a ban

In which JP tries to learn about AI governance by writing up a take. Take tl;dr: Overhang concerns and desire to avoid catchup effects seem super real. But that need not imply speeding ahead towards our doom. Why not try to slow everything down uniformly? — Please tell me why I’m wrong.

After the FLI letter, the debate in EA has coalesced into “6 month pause” vs “shut it all down” vs some complicated shrug. I’m broadly sympathetic to concerns (1, 2) that a moratorium might make the hardware overhang worse, or make competitive dynamics worse.

To put it provocatively (at least around these parts), it seems like there’s something to the OpenAI “Planning for AGI and beyond” justification for their behavior. I think that sudden discontinuous jumps are bad.

Ok, but it remains true that OpenAI has burned a whole bunch of timelines, and that’s bad. It seems to me that speeding up algorithmic improvements is incredibly dubious. And the large economic incentives they’ve created for AI chips seems really bad.^[1]

So, how do we balance these things? Proposal: ““we”” reduce the economic incentive to speed ahead with AI. If we’re successful, we could slow down hardware progress, and algorithmic progress, for OpenAI, and all its competitors.

How would this work? This could be the weak point of my analysis, but you could put a tax on “AI products”. This would be terrible and distortionary, but it would probably be effective at slashing the most centrally AGI companies. You can also put a tax on GPUs.

Note: a way to think about this is a Pigouvian tax.

Counterarguments

China. Yep. This is a counterargument against a moratorium as well. I think I’m willing to bite this bullet.

Something like: we really need the cooperation of chip companies. If we tax their chips they’ll sell to China immediately. This is the major reason why I’m more optimistic about taxing “AI products” than GPUs, which would be easier to tax.

IDK, JP, a moratorium would be easier to coordinate around, is more likely to actually happen, and isn’t that bad. We should put our wood behind that arrow. Seems plausible, but I really don’t think this is a long term solution, and I tentatively think the tax thing is.

***

Again, will be very appreciative of different takes, etc here.

See also: Notes on Potential Future AI Tax Policy. This post was written before reading that one. Sadly that post spends too much time in the weeds arguing against a specific implementation which Zvi apparently really doesn’t like, and not enough time discussing overall dynamics, IMO.

1. ^{^}

   See also here, which I haven’t entirely reread for a good related discussion.

We're planning Q4 goals for the Forum.

Do you use the Forum? (Probably, considering.) Do you have feelings about the Forum?

If you send me a PM, one of the CEA staffers running the Forum (myself or Aaron) will set up a call call where you can tell me all the things you think we should do.

I want to write a post saying why Aaron and I* think the Forum is valuable, which technical features currently enable it to produce that value, and what other features I’m planning on building to achieve that value. However, I've wanted to write that post for a long time and the muse of public transparency and openness (you remember that one, right?) hasn't visited.

Here's a more mundane but still informative post, about how we relate to the codebase we forked off of. I promise the space metaphor is necessary. I don't know whether to apologize for it or hype it.

———

You can think of the LessWrong codebase as a planet-sized spaceship. They're traveling through the galaxy of forum-space, and we're a smaller spacecraft following along. We spend some energy following them, but benefit from their gravitational pull.

(The real-world correlate of their gravity pulling us along is that they make features which we benefit from.)

We have less developer-power than they do (1 dev vs 2.5-3.5, depending on how you count.) So they can move faster than we can, and generally go in directions we want to go. We can go further away from the LW planet-ship (by writing our own features), but this causes their gravitational pull to be weaker and we have to spend more fuel to keep up with them (more time adapting their changes for our codebase).

I view the best strategy as making features that LW also wants (moving both ships in directions I want), and then, when necessary, making changes that only I want.

———

One implication of this is that feature requests are more likely to be implemented, and implemented quickly, if they are compelling to both the EA Forum and LessWrong. These features keep the spaceships close together, helping them burn less fuel in the process.**

*(and Max and Ben)

** I was going to write something about how this could be a promising climate-change reduction strategy, until I remembered that carbon emissions don’t matter in outer space.

Scoreboard

Or: Let impact outcomes settle debates

When American sports players try to trash talk their opponents, the opponents will sometimes retort with the word "scoreboard" along with pointing at the screen showing that they plainly have more points. This is to say, basically, "whatever bro, I'm winning". It's a cocky response, but it's hard to argue with. I'll try to keep this post not-cocky.

I think of this sometimes in the context of the EA ecosystem. Sometimes I am astounded by how disorganized someone might be, and because of that I might be tempted to think of them as fundamentally incompetent. But if I happen to notice that they're sitting on a huge pile of impact, then I should be more humble. Something's obviously working for them. And maybe their disorganization is linked to them being so impactful. Despite this, there's an attitude I sometimes see in myself or others, which is to bring it up in a rather unhelpful or condescending way, which is especially apparent if someone is clearly doing something right.

The example of a disorganized person seems extremely clear to me. But can I still use this concept in a more difficult case? What if I have a strategic disagreement? Sometimes I might be tempted to tell someone that their strategy means that their apparent impact is illusory. But I think a really valuable skill is noticing when I kinda, you know, want that to be true, but it isn't, and in fact the scoreboard simply shows just the opposite.

For a long time the EA Forum has struggled with what to call its tag / wiki thing. Our interface had several references to "this wiki-tag". As part of our work to improve the tagging feature, we've now renamed it to Topics.

I should write up an update about the Decade Review. Do you have questions about it? I can answer them here, but it will also help me inform the update.

Tip: if you want a way to view Will's AMA answers despite the long thread, you can see all his comments on his user profile.

Working questions

A mental technique I’ve been starting to use recently: “working questions.” When tackling a fuzzy concept, I’ve heard of people using “working definitions” and “working hypotheses.” Those terms help you move forward on understanding a problem without locking yourself into a frame, allowing you to focus on other parts of your investigation.

Often, it seems to me, I know I want to investigate a problem without being quite clear on what exactly I want to investigate. And the exact question I want to answer is quite important! And instead of needing to be precise about the question from the beginning, I’ve found it helpful to think about a “working question” that I’ll then refine into a more precise question as I move forward.

An example: “something about the EA Forum’s brand/reputation” -> “What do potential writers think about the costs and benefits of posting on the Forum?” -> “Do writers think they will reach a substantial fraction of the people they want to reach, if they post on the EA Forum?”

We've been experiencing intermittent outages recently. Multiple possible causes and fixes have not turned out to fix it, so we're still working on it. If you see an error saying:
"503 Service Unavailable: No healthy endpoints to handle the request. [...]"
Try refreshing, or waiting 30 seconds and then refreshing; they're very transient errors.
Our apologies for the disruption.