Feedback I've been giving to junior x-risk researchers

By Will Aldred @ 2022-08-15T20:46 (+145)

Context

The CERI and SERI summer research fellowships are each coming to a close. My roles – co-leading CERI, and leading SERI's nuclear risk cause area – involve giving feedback to fellows on their research.^[1] I found myself giving similar-ish feedback to different fellows, and so I'm writing up some of the feedback I gave (as best as I can remember, most of it was verbal) since I suspect this might be useful to aspiring or junior EA/x-risk researchers.

Feedback examples

‘Looks like your target audience for your paper is grantmaker x. You should reach out to them (early!), explain what your research project is about, and ask for their feedback on what bottom lines would be most useful/decision-relevant to them.’

…

‘I can see a couple immediate counterarguments to your claim [in section y in your draft], which you don't appear to acknowledge. I suggest you set a 5 minute timer to brainstorm counterarguments. (Note: Don't also brainstorm responses to the counters in those 5 mins.)’

…

‘It's not clear how the project you've proposed, even if executed really well, will result in anyone in the world taking any action. You say your aim is to reduce hostility and increase peace between [redacted]... what is the step immediately prior to this increased peace? Who are the actors? What is the backward chain from these actors to your research project?

[response from fellow]

Ah, okay, so you see advocacy as a route to influencing these actors... Do you have a sense for how much this kind of advocacy would cost? [Followed by further questions to tease out the plausability of their implicit path to impact.]’

…

(different project to the one above)

‘Who are the 'we' you're referring to? Seems to me that pinning down this 'we' would be really helpful for crisping up your theory of change.’

…

‘You should form and state a bottom line view. Even if you're not confident, you now know more about this topic than ~anyone who'll read your paper, so in expectation your bottom line will update your readers in the direction of truth.’

…

‘Write a summary at the top of your post. Your summary should include your bottom lines and any actionable insights. Ideally, the summary should give the reader ~80% of the value of the whole post: this means the summary should be more informative than an academic paper's abstract. I like to include a table in my summaries, see, for example, here. And for more examples of good summaries, see here, here, and here.

The same goes for each section of your post. Put your bottom lines up front in each section. And, if your sections are long, I'd recommend you write a summary at the start of each.’

…

‘Feel free to include an epistemic status at the top of your post, and/or even at various points throughout your post, to let readers know how confident you are in what you've written.’

…

‘Where you've written 'likely', 'possibly', 'plausibly', you should try to attach a quantitative estimate (e.g., '30% likely').’

…

‘Try to make clear the scope of your paper. Resist the temptation to increase the scope of your paper as you go. You can't tackle everything, and that's fine: better a well-executed narrow paper than a rough or unfinished broader paper.’

…

‘Make clear how what you're working on fits into the overall project of solving x-risk. Think of your project as one rung in the EA/x-risk community's ladder.

More concretely, flag what you see as cruxes or important considerations, but probably don't attempt to figure out a bottom line for all of them. State the ones you don't tackle as directions for future work.’

…

‘Where you're constraining the scope of your research, be sure you make this clear. Otherwise, readers might be left confused by why you've left out some considerations that they can think of.’

…

‘Your writing appears very 'academic' in style - i.e., long sentences and paragraphs, formal tone. As a result, I found it tough to read. Tough in the sense of:

1. 'It was hard to pick out key bits of info, and/or to disentangle different claims'.
2. 'It just wasn't a very fun read.'

My suggestions:

• On 1., use bullet points or numbered lists where you can, instead of prose. Also, use headings and bolding. These things make your post easier to read, and also easier to skim.
• On 2., try to avoid back to back to back long sentences. Also, less importantly, feel free to just loosen up a bit – having some humour in your writing rarely hurts.’

…

‘Try to keep orderings consistent, e.g., pick one of 'the US and China' or 'China and the US' and stick with that, instead of flip-flopping between the two.’

…

‘Hyperlinks are your friend. You should always link to jargon the first time you use it. You should also try to always link to, e.g., organization websites, where you cite orgs.’

…

‘There's no need to include a bibliography, in my opinion. Linking to papers throughout the body of your text is fully sufficient.’

…

‘I think you spent too long sifting through the academic literature to try and define [redacted], given that it's not central to your paper. In future when you can't quickly find a definition, I'd recommend you just 'make up' a definition that makes sense to you (and that'll make sense to readers) and run with that.

Broader point: try to always keep in mind the intended end product. You could spend anywhere from 2 minutes to 5 years on pretty much any piece of your research - at the end of the day it really comes down to time tradeoffs and being smart about which parts of your paper you should spend more or less time on.’

…

‘One extreme is to spend too much time reading, and to only start writing pretty late on. In this extreme, it turns out that you read a bunch of stuff that was only tangential to answering your research question. Another extreme is to start writing too early, before you have a decent model of the problem you're working on.

My take is that the reading-too-much extreme is more common (i.e., most junior researchers I've seen err on the side of starting to write too late) and also the worse of the two evils. Why do I think the reading-too-much extreme is worse than the writing-too-early extreme? I think that if you start writing early, you'll notice the places where you'll need to do more background reading, and you'll thus be able to focus your reading on the things you actually need to read to produce impactful research. Contrast this with doing lots of background reading, most of it unrelated to what you eventually write.’

…

‘Reading 'too much' is possibly the optimal strategy if you're mainly trying to skill up (e.g., through increased domain knowledge), rather than have direct impact now. But also bear in mind that becoming more efficient at direct impact is itself a form of skilling up, and this pushes back toward 'writing early' as the better extreme.’

…

‘It's all right, and in fact probably optimal given your career stage, for your theory of change to be mainly about skilling you up for future impact, as opposed to having direct impact now.’

High-level recommendations

According to me, there are two resources every junior EA researcher should read/watch, really understand, and put into practice:

1. Michael Aird's theory of change workshop (recording, slides, worksheet)
2. ‘Reasoning Transparency’ (Muehlhauser, 2017)

Moreover, junior researchers, I think, are often not quite sure what to aim for, because they haven't seen many (or any) examples of impactful research. I claim that most posts and reports put out by Rethink Priorities and Open Philanthropy are unusually impactful, and that a junior researcher should check out their research databases (here and here, respectively) to begin bootstrapping their model of what impactful research looks like.

See also

• Ashwin Acharya's (2022) ‘Using the “executive summary” style: writing that respects your reader’s time’
• Nuño Sempere's (2021) ‘Frank Feedback Given To Very Junior Researchers’
• Michael Aird's (2022) ‘Interested in EA/longtermist research careers? Here are my top recommended resources’
• Max Daniel's (2020) ‘Giving and receiving feedback’

I'm grateful to Nandini Shiralkar for helpful comments on an earlier draft.

1. ^{^}

   The intended output for most CERI research projects is a Forum post or a document shared with an EA decision maker.