Review of FHI's Summer Research Fellowship 2020

By rosehadshar @ 2020-11-19T11:32 (+124)

This post reviews the Future of Humanity Institute’s Summer Research Fellowship 2020 in detail. If you’re in a hurry, we recommend reading the summary of activities, lessons learned, and comparing costs and benefits sections for a quicker take.

Thanks to Owen Cotton-Barratt, Max Daniel, Eliana Lorch, Tanya Singh and the summer fellows for reviewing and improving this post.

Summary of activities

• 27 fellows spent 6 weeks on a remote fellowship programme.
  • We got ~300 applications and interviewed ~50 people.
  • Backgrounds ranged from undergraduate students to postdocs.
• By default, each fellow had a mentor and worked on a research project (though there were some exceptions, and many fellows also spent significant time exploring).
  • The mentors were researchers and research managers at FHI, CSER, OpenPhil, OpenAI, GPI, and MIRI.
  • Approximately, the projects related to AI governance (13 fellows), technical AI safety (3 fellows), biosecurity (4 fellows), macrostrategy broadly construed (5 fellows), and policy (2 fellows).
• This fellowship grew out of the CEA summer fellowship (which CEA wanted to stop running), though was significantly different in that a) it was remote, and b) we took more fellows (in 2019, the CEA fellowship took 11 fellows).

Lessons learned

Meta: we’re stating these confidently to make clear which direction the summer fellowship has updated us in. Obviously this was just one summer programme, and in many cases there might be important differences which mean that these takeaways wouldn’t generalise. The takeaways are in rough order of importance according to us.

• Remote fellowships can provide substantial value to fellows and organizers, and are worth considering where in person options are not available, or are especially costly.
  • Initially we put some probability on the whole fellowship being awkward and unproductive, but in fact fellows had good experiences and some good things seem to have come from the programme. Remoteness also allowed us to take more fellows, and in the particular case of Covid-19, also came with lower opportunity costs for many fellows and organisers.
    • In spite of remoteness, it was possible to create a positive, friendly and open culture. Many fellows remarked on this and appreciated it, and we guess that it helped to increase engagement, support those experiencing difficulties, and deepen fellows’ experiences. Modelling open and authentic communication on the part of the organisers seems to have been a major contributing factor to this culture.
  • That said, we still think that there are many ways in which an in person fellowship would have been better, particularly in terms of culture, networking and some kinds of logistics.
• There is more interest than we expected in mentoring people amongst researchers in a range of different EA organisations. We successfully found 18 mentors for 24 fellows (3 fellows didn’t have formal mentors for various reasons), and we contacted at least a further 10 people who we think would have mentored if there had been a fellow working on the right topic. We also didn’t do a comprehensive search for mentors, and instead reached out ad hoc to people in our network, so we expect we would have found more interest if we had searched a bit harder.
• There were more promising applicants for this programme than we expected. In 2019 the CEA fellowship received ~90 applications, and this fellowship received ~300. We think that the number of strong applicants was correspondingly larger.
• Minor difficulties with mental health were common, and more serious ones not extremely rare (although this is confounded by remoteness and Covid-19). 1-1s and other conversations seemed to help support people with this. We think it would be good for future organisers to invest more time in thinking about how to mitigate the risks from a) imposter syndrome, and b) the open ended nature of the fellowship and the lack of clear success metrics.
• The personal fit between mentor and fellow might potentially be very impactful. In the two cases where the fellowship led directly on to further remunerated collaboration between the fellow and the mentor, we selected the mentorship pairing in part because we expected the two people to get on well together personally. It’s unclear how important this was, but we think it’s worth paying attention to in future pairings.
• A significant part of the value of (this) fellowship came from providing mentorship and network.
• There’s significant interest in the fellowship being longer than 6 weeks, from fellows and mentors, though it’s not clear to us if this is net worth it.
• Many fellows thought that a mentor with more subject expertise would have been able to support them more. However, most of these fellows also had a strong positive experience with their actual mentor, suggesting that this is a nice to have rather than an essential.
• Low-input attempts to create good online office space can be net harmful. We set up a default remote office space, which hampered some more energetic attempts from fellows to create good online environments.
• Putting on talks especially for fellows was high time cost and only somewhat valuable. Especially in the context of a remote programme or in the case of recorded talks, we expect that it would have been better to either not do this at all, or to invest even more time in making the series really good.
• Communicate clearly to fellows before the start of the programme that there are multiple different outcomes that look like success, if this is the case. We communicated this at the start of the programme, which we think was better than not communicating it at all or communicating it later, but we think that communicating earlier than this would likely have better helped fellows to plan and orient.
• The learning value of running such a programme for organisers can be very high. Rose had already co-ran a fellowship similar to this one, but still learned a lot (in part because this fellowship was remote, in part because she had a more empowered role on this fellowship). Eliana hadn’t previously run a fellowship like this and also learned a lot.
  • Both Rose and Eliana had a lot of vision for this fellowship, but not for a remote version of it, and were initially unexcited about a remote version. We were surprised that it went well given that we didn’t have a clear vision of how to run this remotely.

Goals of the fellowship

The main goal of the fellowship is to help young people to do better things from a longtermist perspective than they would otherwise have done.

We expect most of this change to happen through:

• Changing fellows’ career plans, by putting fellows in the way of concrete opportunities, shifting fellows’ motivations, and helping fellows to test their fit.
• Improving fellows’ networks, both among other fellows and with more senior people.
• Improving fellows’ knowledge and skills in a range of ways, including research taste, research experience, and psychological empowerment.

We try to meet this goal by:

• Selecting high potential young people where we think the fellowship is better than what they could counterfactually be spending time on.
• Bringing them into a rich longtermist environment.
• Giving them the affordance to work on a relevant project.
• Providing them with a mentor who can give them feedback.
• Introducing them to each other and relevant researchers in the space.

Detail on activities

Timeline of activities

• January/February 2020: Rose and Eliana decided to take on the project together
• 19th February: application page goes live (deadline 22nd March)
  • We asked for a CV and a research proposal
    • The research proposal was for the purposes of evaluation; people were free to work on other things during the programme
• March-April: desk rejects and interview decisions
• Late April: interviews
• Early May: offers
• 13th July: fellowship started
• 24th August: fellowship ended

Summer fellows

There were 27 fellows in all. Below is a list of those fellows who explicitly gave us permission to name them in this post. See here for more extended bios.

Evaluation of aspects of the fellowship

The fellowship as a whole

We asked fellows the following questions:

• How much better or worse do you feel about the summer fellowship than about what you would have done otherwise?
  • Average 8.58 out of 10, min 5
• If the same program (same organizers, same structure etc.) happened next year, how strongly would you recommend a friend (with similar background to you before the fellowship) to apply?
  • Average 8.89 out of 10, min 2 (then 7)

We think these are fairly good responses overall. The minimum responses to both questions came from the same fellow.

Length of the programme

Fellows’ opinions

• We asked fellows ‘What % of the value you've gotten from the fellowship so far would you attribute to each week of the fellowship?’
  • The average responses were 15%, 17%, 16%, 15%, 14% and 16% for weeks 1,2,3,4,5 and 6
  • This suggests that returns weren’t diminishing over the 6 weeks
• Of the 27 fellows, 5 said in the final survey that they wished the fellowship had been longer. We guess that an additional 3 might also have benefitted from a longer fellowship

Mentors’ opinions

• 2 mentors suggested making the fellowship longer in future

Our thoughts

• It’s worth noting that the CEA summer fellowship was 6 weeks in 2018, and fellows said it was too short. The 2019 CEA fellowship was therefore lengthened to 8 weeks, and fellows again said it was too short. However, the organisers thought that the 2019 fellowship was a success, and that an additional 2 weeks would not have increased the impact of the programme much.
• There’s some tension between the content of the fellowship and where we think the impact comes from. Fellows undertake a research project during the fellowship; we think that 6 weeks is a little too short for this, especially given the extra time needed to orient to a new environment. However, we don’t think that the main value of the fellowship comes from completed research outputs. Instead, we expect most of the value to come for career updates, networking and gaining knowledge and skills. 6 weeks seems enough to capture significant value here.
• We think that organisers next year should seriously consider making the fellowship longer, but we don’t think it’s clear that they should.

Program components

• We asked fellows ‘How valuable did you find the following [programme components]?
• The options were Harmful, ~Neutral, Valuable, Very Valuable, and NA
• Assigning -1 to Harmful, 0 to ~Neutral, 1 to Valuable, and 2 to Very valuable, the programme components were rated in the following way:

Mentoring

For fellows

We asked fellows the following questions:

• Would you like to see your mentor mentoring other people in the future?
  • Average 4.77 out of 5
• In what ways do you think another mentor might have been able to support you more?
  • 8 fellows: more subject expertise
    • NB 7 of these fellows mostly scored the question about their mentor mentoring other people 5/5 (the final person rated it as 3/5), and 6 rated mentoring very valuable (the other 2 rated it as valuable)
  • 4 fellows: more accountability/management/structure
  • 2 fellows: more active mentorship in general
  • 1 fellow each:
    • Feedback on written work
    • Different thinking style to me
    • More directive/forceful on research directions
    • More introductions

We think that subject expertise is a nice to have property in a mentor, but not strictly necessary for valuable mentoring pairs to be formed.

For mentors

We asked mentors the following questions:

• How much better or worse do you feel about your mentoring than about what you would have done otherwise? (Counting both benefits to your mentee(s) and to you.)
  • Average 7.2 out of 10 (min 6)
• If the same program (same organizers, same structure etc.) happened next year, how strongly would you recommend a friend (with similar background to you before the fellowship) to be a mentor?
  • Average 7.5 out of 10 (min 5)
• I've learned a lot about how to mentor researchers.
  • Average 3.5 out of 5 (min 2)
• I'm glad that I volunteered to be a mentor.
  • Average 4.3 out of 5 (min 3)
• If we ran the fellowship again, what would you recommend we change?
  • 2 mentors mentioned providing more advice and prompting on mentoring
  • 1 mentor each mentioned
    • Sharing templates for mentors to request feedback from mentees
    • More involvement of mentors in the selection process
    • Nudging mentors to make introductions
    • Updating mentors on the career planning session we ran

In one sense, the ratings mentors gave seem quite low. However, they clear the bar of ‘worth my time overall’.

Culture

We asked fellows ‘Do you have any comments on the culture of the Summer Research Fellowship?’

• 12 fellows commented on the culture being quite good to fantastic in a generic way
• 9 fellows mentioned the culture being warm/kind/friendly
• 3 fellows appreciated the openness of the culture (in terms of sharing struggles and difficulties)
• 1 fellow mentioned each of the following as a problem with the culture:
  • Lack of Slack engagement
  • Sharing curiosities not being sufficiently common
  • Insufficient steps taken to mitigate imposter syndrome
  • Too much of an 'in' culture around EA/rationality
• Fellows also commented on what had helped contribute to the culture
  • 7 fellows mentioned modelling from Rose and Eliana
  • 2 fellows mentioned the opening session

We think the culture was surprisingly good for a remote fellowship, and we guess that it helped to increase engagement, support those experiencing difficulties, and deepen fellows’ experiences.

Mental health

• We asked fellows, ‘Did you experience any difficulties with mental health during the fellowship?’
  • 15 people reported some difficulties, of whom 3 reported significant difficulties (according to us)
• Of those 15 people, 4 reported that SRF contributed to the difficulties they experienced, specifically through:
  • Open ended commitments and unclear paths to success (2 fellows)
  • Feelings of imposter syndrome (1 fellow)
  • Logistical stuff to do with time zones (1 fellow, and we know of at least one additional fellow for whom this posed non-trivial challenges)
• 13 fellows reported that the fellowship supported them in their difficulties, mostly via 1-1s and other conversations
  • The intersection between this group of 13 and the group of 15 who reported difficulties is 12
  • The intersection between this group of 13 and the group of 4 where SRF contributed to their difficulties is 3
• 2 fellows sometimes felt unwelcome
  • In one case the fellow felt unwelcome in some heavily EA conversations where they had less background than the other participants
  • In another case the fellow felt unwelcome in some fellow-organised discussion groups

We think that given that the fellowship was remote, this rate of mental health difficulties isn’t surprising, and also that the counterfactual is unclear given the global pandemic. 

We also think that anyone having difficulties with mental health that are exacerbated by the fellowship is bad, and we should endeavour to do better. In particular, we think future organisers should think more about ways of mitigating the mental health risks of the fellowship being open ended and (in its current form) having many and therefore unclear paths to success. It’s worth noting that we think that the opposite extreme (of a very clear and particular goal) is likely to be challenging for other fellows, and that the right solution will take variation among fellows into account.

Slack

We created a separate Slack workspace for the fellows and their mentors. Mentor engagement with the Slack workspace was optional, and in the end, minimal. We did not add fellows to the regular FHI Slack workspace, as it contains potentially sensitive information, and we didn’t want to flood the Slack with new members.

The Slack worked well as a coordination tool, but didn’t become a rich intellectual space. There weren’t many object-level research discussions, or much social engagement.

Not adding fellows to the FHI Slack worked well. We think we still managed to introduce fellows to FHIers in a way that was valuable.

Costs of running the fellowship

To FHI

• The financial costs were all spent on stipends for fellows.
• Researcher time talking to fellows. As organisers, we set up roughly 80 1-1 pairings between FHI researchers and fellows. Probably some of these never actually took place, but in addition many mentors made introductions too. Assuming that 1-1s lasted 60 minutes on average, this cost maybe 100 hours of FHI researcher time.
• Admin time administering selection, setting up stipends and scheduling events (perhaps 110-130 hours).
• A small number of people (2-6) gave us feedback on our plans, which cost a small amount of time (maybe 4-20 hours overall).
• Around 20 FHIers spent some time reviewing some applications. We guess this cost 40-60 hours total.

To mentors

• There were 18 mentors, who were researchers and research managers at FHI, CSER, OpenPhil, OpenAI, GPI and MIRI.
• In addition, there were 2 informal mentors who agreed to speak more occasionally with a fellow. These informal mentors were at CSET and GPI.
• All mentors were volunteers.
• Most mentors met with fellows once a week for 30-60 minutes. These 18 mentors had 24 fellows between them.
  • Guessing that two thirds of the time mentors spent was literally in meeting fellows, this makes for a net time cost to mentors of roughly ~170 hours.
• NB all mentors thought mentoring was net better than their counterfactual (average 7.2/10 compared with the counterfactual; minimum 6/10).

To the SRF organisers

• Eliana Lorch and Rose Hadshar ran the fellowship. Neither tracked their time, but we guess that combined we spent the following amount of time on the fellowship:
  • Selection: maybe 3 weeks of work time
  • Planning the programme: maybe 1.5 weeks of work time
  • Running the programme: maybe 2.5 weeks of work time
• We feel that this time was better spent on the fellowship than on our counterfactuals (in Rose’s case, more investment into RSP culture and programme structure; in Eliana’s case, more work on modeling).
• This makes for a total of ~280 organising hours. The CEA fellowship in 2019 cost more like ~200 hours to its organisers, so this version was more time intensive. We expect that this comes in large part from the fact that we had triple the number of applications, took nearly three times the number of fellows, and needed to rethink the programme substantially to make it remote.
• Organising was stressful for both Rose and Eliana at times.

To fellows

• 2 fellows reported feeling isolated at points during the fellowship
• 3 fellows reported that the fellowship exacerbated significant mental health difficulties they were facing
  • In one case, the fellowship contributed via pressure and feelings of imposter syndrome
  • In the other two cases, the fellowship contributed via unclear expectations and success criteria
• Opportunity costs
  • 1 fellow rated the fellowship as 5/10 compared with their counterfactual and we think had a frustrating experience of the fellowship
  • All other fellows rated the fellowship as net better than their counterfactual (average 8.6/10 compared with the counterfactual)

Benefits produced by the fellowship

Meta: 

• Unless stated otherwise, the following is based on a mix of (i) responses to feedback forms all fellows and mentors filled out at the end of the programme, (ii) Rose’s and Eliana’s impressions based on interactions with the fellows, their mentors etc.
• A lot of this is subjective.
• It’s also somewhat sensitive as it relates to particular people. We’ve anonymised as far as possible.
• We think that most of the benefits of the fellowship won’t be visible until more time has passed. We intend to survey fellows again 6 months after the fellowship, and possibly 2 years afterwards too.

To fellows

Career updates

• Concrete opportunities
  • 1 fellow found a paid research position working for the person who mentored them on the fellowship
  • 1 fellow received two grants to continue their fellowship project and to work on related research
  • 2 fellows have ongoing collaboration/mentorship relationships
  • 1 fellow became an affiliate at a longtermist research organisation
  • 3 fellows have potential future collaborators
• Motivations
  • 4 fellows became more likely to work on things with longtermist implications
• Studying
  • 4 fellows updated towards further study (with longtermist relevance)
• Research updates
  • 4 fellows got a clearer idea of what to work on during their PhD or masters degree
  • 2 fellows updated against research careers
  • 2 fellows updated towards global priorities research
  • 1 fellow shifted their idea of what to apply to do a PhD on
• Meta
  • 11 fellows have mentors who intend to speak with them in future about their career plans
    • In one case the mentor intends to substantially engage in discussions around choosing a PhD topic

Network

• The median fellow reported knowing 5 people before the fellowship who met the following criteria:
  • a) have spent more time than you thinking about or working on questions of relevance to the long-term future, and
  • b) you’d feel comfortable reaching out to for advice or feedback
• The median fellow reported knowing an additional 6 people as a result of the fellowship who met the same criteria
• This represents an average increase of fellows’ long termist networks of around 100%
• One fellow who had very little exposure to EA prior to the fellowship reported that it was a positive introduction to EA for them

Knowledge and skills

• 9 fellows gained significant knowledge on various aspects of longtermism
• 8 fellows gained significant research skills
  • In most cases, these were fellows with little prior research experience
  • In a few cases, skills were gained by more experienced researchers around research taste
• 4 fellows reported improving on other skills or traits
  • Agenti-ness, productivity, and confidence were named

Outputs

This list contains outputs where there was a publicly accessible link, and/or where fellows were happy to include their name and email address against their work. There were many other outputs where neither of these things applied, which we have excluded from this list.

Note that in some cases only parts of the output were produced during the fellowship. However, we think that for most if not all of these, these texts would not exist, or would have been published later or with worse quality, if not for the fellowship.

Completed outputs:

• Clarifying “What failure looks like” (part 1)

Notable works in progress include:

• Draft papers on:
  • Labor, Capital, and the Optimal Growth of Social Movements
  • Promoting versatile vaccine development for emerging pandemics (Joshua Monrad joshua.monrad@gmail.com, with Jonas Sandbrink and Neil Cherian. Currently under review with an academic journal)
  • Metagenomic sequencing and public health surveillance (Sid Sharma, sidsharma1125@gmail.com)
  • A discussion on the statistical nature of threats to historical civilizations (Nithin Sivadas, nithin@bu.edu)
• A draft LessWrong post on future trajectories for democracy and voting assistants (Wiona Glänzer, wiona.glaenzer@icloud.com)
• A draft EA forum post with the working title ‘A Review of Moral Psychology Research Relevant to Moral Circle Expansion’ (Megan Kinniment, megan.k.w.12@gmail.com)
• An article titled ‘National Interests or the Greater Good? Dual Moral Obligations and International Cooperation’ (Jenny Xiao, j.xiao@columbia.edu)

Other effects

• Two fellows and a third collaborator won the 2020 Next Generation for Biosecurity Competition with a joint paper, Widening the Framework for Regulation of Dual-Use Research in the Wake of the COVID-19 Pandemic. This collaboration would not have happened without the collaborators meeting through SRF
• One fellow was able to facilitate in-depth exchange between RSP management and another research organisation on running research teams
• One fellow changed the focus of their masters dissertation to AI safety
• One fellow was able to better prepare for starting their job at a think tank

To mentors

• 1 mentor’s fellow was working on a project from the mentor which otherwise the mentor would have wanted to do themselves. The fellow’s time was a bit under 1:1 equivalent of the mentor’s time on the project, and the fellow extended the paper following on from the fellowship
• 3 mentors updated towards mentoring in future, significantly in 2 cases
• 2 mentors clarified some of their own research thinking
• 4 mentors learned some useful object level things
• 1 mentor reported improving at mentoring
• Mentors updated on how promising 9 fellows were in the course of interacting with them

Comparing costs and benefits

We think that the fellowship was pretty clearly net positive, but that it also wasn’t clearly an outstanding success. 

In particular, at the end of the 2019 CEA fellowship, we think there were proportionally more clear success cases to point to than there are at the same point for the 2020 FHI fellowship. This might in part be because there were fewer fellows in 2019, which might mean a) fellows had more input per head and this created more value, and b) organisers had a better sense of how individual fellows had developed, making it easier to spot successes. It might also be because the fellowship was remote and this made it less valuable. Small sample size and frequency of outliers might also mean that this is explained just by luck.

We expect more of the impact of this fellowship to become clear later as fellows progress in their careers, and intend to survey fellows again 6 months after the fellowship and possibly 2 years after.

Mistakes

We’re sure we made other mistakes too, but the ones that stand out in chronological order are:

• Rose invested insufficient time in managing the person providing administrative support during the application process. This created unnecessary inefficiency, confusion and delays for organisers and applicants.
• We'd make some hiring decisions differently now, and we think for some of them we could have realized this ex ante.
• We didn’t communicate promptly enough with fellows about the expectations we had of them, or the structure and schedule of the programme.
• In the first week the timings of a number of events were changed last minute, and this created confusion that seemed particularly costly at the beginning of the fellowship.
• We instituted a default online office space, but didn’t invest any energy or thought into integrating this well with other parts of the fellowship or fellows’ initiatives. This led to a fairly inactive online office space, and probably hindered some more energetic initiatives from fellows.
• We didn’t communicate clearly with fellows about how to prepare and give good lightning talks, and this made these sessions less valuable than otherwise.
• We didn’t put conscious attention on how to elicit negative feedback (anonymous feedback options, options to give feedback to non-organisers). We expect this leads to a positive skew in our evaluation here.

We may consider sharing some materials and resources with organisers of similar programmes in future. If you think these materials would be useful to you, please email srf@philosophy.ox.ac.uk explaining why this would be useful, and we'll consider your request.