How do EA researchers decide on which topics to write on, and how much time to spend on it?

By BrianTan @ 2020-12-31T14:33 (+24)

Hey all, I have two questions for EA researchers, although researchers in non-EA orgs can also answer:

1. How do you decide or prioritize which specific research topic to pursue among many possible topics?

I'm talking more about topics within a specific cause area, but how to pick between topics across cause areas is also useful. I have seen Charity Entrepreneurship's research process, and I think it's quite a good process of deciding what to do research on. But I wonder if there are easier versions or alternative versions of this done by other researchers or EA organizations.

2. How do you decide how much time to allocate to researching and writing a specific write-up? 

Also, what do you do to make sure you are on track to finishing on time, and what do you do if you do not meet the deadline you set?

While I was working as a product designer in the software development industry, I learned the concept of "fixed time, variable scope or "fixed scope, variable time" releases. "Fixed time, variable scope" means the team would ship only the features they have finished by the deadline. Alternatively, they could choose not to ship it at all, and put the project on pause (or to not continue it at all). Meanwhile, "fixed scope, variable time" means that the team would only launch after all the features specified for that release have been finished. I wonder if researchers use either of these frameworks, such as setting a deadline and limiting the scope of research work so that it could meet the deadline, or if they just allow for deadlines to be moved (or don't set deadlines at all) if the research takes longer than expected.

---

I'd love to get answers to these questions. I ask because I'm currently leading EA Philippines' local cause prioritization and career advice research, where we aim to figure out what are the top local causes Filipinos should contribute to, and which career paths could be the most effective for them to take to do the most good. We're working with a few volunteers (mostly students) who are helping us do this research. We aim to write problem profiles like the ones 80,000 Hours has written. Here's a sample of our work on the scale and neglectedness of mental health in the Philippines.

However, I don't really have a background as a researcher, other than a couple of research papers I wrote in university. So I'm not sure what are the best practices when it comes to deciding what specific topics to pick, how to set deadlines well, or how to properly manage the scope of research projects. Any advice would be appreciated! If you know of any resources I could read or consume too, feel free to link them. And if you're also happy to have a 30-60 minute call with me on the above topics or questions, I'm happy to book a call with you as well. Thanks!

NunoSempere @ 2020-12-31T15:16 (+11)

I predict the value of each project in terms of "microbostroms" (or, Quality Adjusted Research Papers), more on which here, or expected microbostroms per unit of resources, and then carry out the most promising ones. See here for a rubric for that unit.  

This has recently led to some fairly well received posts, such as: 

• https://www.lesswrong.com/posts/6bSjRezJDxR2omHKE/real-life-examples-of-prediction-systems-interfering-with
• https://forum.effectivealtruism.org/posts/SCqRu6shoa8ySvRAa/big-list-of-cause-candidates
• https://forum.effectivealtruism.org/posts/udGBF8YWshCKwRKTp/an-experiment-to-evaluate-the-value-of-one-researcher-s-work
• https://forum.effectivealtruism.org/posts/qb56nicbnj9asSemx/predicting-the-value-of-small-altruistic-projects-a-proof-of

so I'm probably going to continue doing this. I initially started with a simple google sheet, which you can see here. But I recently moved to a foretold community, which looks something like this:

I'm actually looking to see if this has a chance of being useful for other people, so if you or other researchers want to send me a list of projects you're considering, I'm happy to get you set up on foretold and predict an initial estimate of their value. 

BrianTan @ 2021-01-01T11:17 (+3)

Thanks for this answer Nuno! I've read the first 1/3-1/2 of the post you linked so far, and I think it's a cool framework. I think estimating value as "microbostroms"/"milibostroms" or QARPs, and framing things as an order of magnitude away from each other, are a good idea. 

I'm curious how much time you expect to take to predict an initial estimate of the value of 1 research project? I don't have a list of topics yet, but if I do have one, I'm wondering how much time it would take you to predict their value.

Also, after you're done predicting (or crowdsourcing predictions) of the value of your projects, how do you then estimate how long each research question/project would take? Do you also do forecasts for that? And do you then just divide the milibostroms by the estimated number of hours to find what's most cost-effective to pursue?

NunoSempere @ 2021-01-01T12:17 (+4)

The posts linked under "fairly well received posts" are just as proof of capabilities (i.e., my forecasting system isn't suggesting terrible projects). They are also fairly long, so I wouldn't suggest reading all of them.

Right now, the time I take to predict the value of a project ranges from almost instantaneous to something like 5 mins.

how do you then estimate how long each research question/project would take? Do you also do forecasts for that? 

Yes, I also have an estimate for hours, but that's a bit more tricky, and I'm not that great at it. 

And do you then just divide the milibostroms by the estimated number of hours to find what's most cost-effective to pursue?

Yes, see column H of the spreadsheet. 

BrianTan @ 2021-01-01T12:52 (+3)

Ah I meant I've only read 1/3-1/2 of the "An experiment to evaluate the value of one researcher's work". I'll try to finish reading it within the next few days. 

That's cool to hear though that it takes you 0-5 mins to predict the value of a project. I may want to book a call with you too to dig deeper about your forecasting system for projects. Could you DM me on the Forum your Calendly link (if you have one)? :)