Jaime Sevilla's Quick takes

My overall impression is that the CEA community health team (CHT from now on) are well intentioned but sometimes understaffed and other times downright incompetent. It's hard to me to be impartial here, and I understand that their failures are more salient to me than their successes. Yet I endorse the need for change, at the very least including 1) removing people from the CHT that serve as a advisors to any EA funds or have other conflict of interest positions, 2) hiring HR and mental health specialists with credentials, 3) publicly clarifying their role and mandate. 

My impression is that the most valuable function that the CHT provides is as support of community building teams across the world, from advising community builders to preventing problematic community builders from receiving support. If this is the case, I think it would be best to rebrand the CHT as a CEA HR department, and for CEA to properly hire the community builders who are now supported as grantees, which one could argue is an employee misclassification.

I would not be comfortable discussing these issues openly out of concern for the people affected, but here are some horror stories:

1. A CHT staff pressured a community builder to put through with and include a community member with whom they weren't comfortable interacting.
2. A CHT staff pressured a community builder to not press charges against a community member who they felt harassed by.
3. After a restraining order was set by the police in place in this last case, the CHT refused to liaison with the EA Global team to deny access to the person restrained, even knowing that the affected community builder would be attending the event.
4. My overall sense is that CHT is not very mindful of the needs of community builders in other contexts. Two very promising professionals I've mentored have dissociated from EA, and rejected a grant, in large part because of how they were treated by the CHT.
5. My impression is that the CHT staff undermines the legitimacy of local communities to make their own decisions. CEA is often perceived as a source of authority, and the CHT has a lot of sway in funding decisions. This makes it so that it is really hard for local groups to go against the wishes of the CHT, who is the main intermediary with groups. I wish this relation was more transparent, so they could be hold accountable for it.

To be clear, I think that these stories have a lot of nuance to them and are in each cases the result of the CHT making what they thought were the best decisions they could make with the tools they had, but in each of them I noticed that I ended up disagreeing with the decisions made and feeling very uncomfortable with how the whole community structure was set up.

A pattern I've recently encountered twice in different contexts is that I would recommend a more junior (but with relevant expertise) staff for a talk and I've been asked if I could not recommend a more "senior" profile instead.

I thought it was with the best of intentions, but it still rubbed me the wrong way. First, this diminishes the expertise of my staff, who have demonstrated mastery over their subject matter. Second, this systematically denies young people of valuable experience and exposure. You cannot become a "senior" profile without access to these speaking opportunities!

Please give speaking opportunities to less experienced professionals too!

Within Epoch there is a months-long debate about how we should report growth rates for certain key quantities such as the amount of compute used for training runs.

I have been an advocate of an unusual choice: orders-of-magnitude per year (abbreviated OOMs/year). Why is that? Let's look at other popular choices.

Doubling times. This has become the standard in AI forecasting, and its a terrible metric. On the positive, it is an intuitive metric that both policy makers and researchers are familiar with. But it is absolutely horrid to make calculations. For example, if I know that the cost of AI training runs is doubling every 0.6 years, and the FLOP/$ is doubling every 2.5 years, then the FLOP per training run is doubling every ${(\frac{1}{0.6}+\frac{1}{2.5})}^{-1}$ years, which is very difficult to solve in your head! ^[1] [2]

Percent growth. This is a choice often favoured in economics, where eg the growth rate of GDP is often reported as 3%, etc. Unlike doubling times, percent growth composes nicely - you just have to add them up! ^[3] However, I also find percent changes somewhat prone to confusion. For instance, when I tell people that model size has increased 200% since X years ago, I sometimes have had people misunderstand this as saying that it has increased by a factor of 2. 

Ultimately, a very common operation that I find myself doing in my head is "if the effective FLOP used in AI training runs grows at a certain rate, how quickly will we traverse from the scale of current training runs (1e25 FLOP) to a certain threshold (eg 1e30)". OOMs/year makes this computation easy, even if I need to account for multiple factors such as hardware increases, investment and algorithmic improvements. Eg if these grow respectively as 0.4 OOM/year, 0.1 OOM/year and 0.5 OOM/year, then I know the total effective growth is 1.0 OOM/year, and it will take 5 years to cross that 5 OOM scale gap. And if investment suddenly stopped growing, then I would be able to quickly understand that the pace would be halved, and the gap would then take 10 years to cross.

Sadly OOMs/year is uncommon, and both researchers and policy makers struggle to understand it. I think this is a missed opportunity, and that AI forecasting would be easier to reason about if we moved to it, or at the very least abandoned the very badly behaved doubling time framing.

What do you think? Do you agree we should move past doubling times to a better choice? Which choice would you favour?

1. ^{^}

    I won't enter into the technical details, but it also has some very unintuitive results when combining the results with uncertainty. Once we had a discussion because some doubling times looked like they had to be wrong. They spanned from days to years! But it turns out that doubling times are very sensitive to noise, which led to our intuitions being wrong.

2. ^{^}

   I'd also argue that Christiano's operationalization of slow takeoff is a terrible definition, and that a big part of that terribleness stems from doubling times being very unintuitive.

3. ^{^}

   This is because have the useful property that $\ln (1+g)\approx g$ for percent changes $g$ close to zero, which links percent growth to a straightforward model of growth such as $x_t=x_0\exp \left\{gt\right\}$. But this approximation breaks down for percent changes over 1 (which are often seen in AI forecasting).

Quick guide on the agree/disagree voting system:
 

• When you upvote a post/comment, you are recommending that more people ought to read and engage with it.
• When you agree vote a post/comment, you are communicating that you endorse its conclusions/recommendations.
• Symmetrically, if you downvote a post/comment you are recommending against engaging with it.
• And similarly, if you disagree vote a post/comment you are communicating that you don't endorse it's conclusions/recommendations.

Upvotes determine the order of posts of comments and determine which comments are automatically hidden, so have a measurable effect on how many people read them.

Agree votes AFAIK do not affect content recommendations, but are helpful to understand whether there is community support for a conclusion, and if so in which direction.

On getting research collaborators

(adapted from a private conversation)

The 80/20 advice I would give is: be proactive in reaching out to other people and suggesting to them to work for an evening on a small project, like writing a post. Afterwards you both can decide if you are excited enough to work together on something bigger, like a paper.

For more in depth advice, here are some ways I've started collaborations in the past:

• Deconfusion sessions. 
  I often invite other researchers for short sessions of 1-2 hours to focus on a topic, with the goal of coding together a barebones prototype or a sketch of a paper.
  
  For example, I engaged in conversation with Pablo Moreno about Quantum Computing and AI Aligment. We found we disagreed, so I invited him to spend one hour discussing the topic more in depth. During the conversation we wrote down the key points of disagrement, and we resolved to expand them into an article.
   
• Advertise through intermediate outputs. 
  I found it useful for many reasons to split big research projects into post-size bits. One of those reasons is to let other people know what I am working on, and that I am interested in collaborating.
  
  For example, for the project on studying macroscopic trends in Machine Learning, we resolved to first write a short article about parameter counts. I then advertised the post asking for potential collaborators to reach out.
   
• Interview people on their interests. 
  Asking people what motivates them and what they want to work on can segue into an opportunity to say "actually, I am also interested in X, do you want to work together on it?". I think this requires some finesse, but it is a skill that can be practiced.
  
  For example, I had an in depth conversation with Laura González about her interests and what kinds of things she wanted to work on. It came up that she was interested in game design, so I prodded her on whether she would be interested in helping me refine a board game prototype I had previously shown her. This started our collaboration.
   
• Join communities of practice.
  I found it quite useful to participate in small communities of people working towards similar goals. 
  
  For example, my supervisor helped me join a Slack group for people working on AI Explainability. I reached out to the people for one-on-one conversations, and suggested working together to a few. Miruna Clinciu accepted - and now we are buiding a small research project.

Our team at Epoch recently updated the org's website.
I'd be curious to receive feedback if anyone has any!
What do you like about the design? What do you dislike?
How can we make it more useful for you?

Just a quick comment for the devs - I saw the "More posts like this" lateral bar and it felt quite jarring. I liked it way better when it was at the end. Having it randomly in the middle of a post felt distracting and puzzling. 

ETA: the Give feedback button does not seem to work either. Also its purpose its unclear (give feedback on the post selection? on the feature?)