My experience on a summer research programme

[I co-directed the summer programme described in the OP, together with Rose Hadshar. Views are my own.]

Hi Denise and Siebe, thank you very much for your comments. I think they are valuable contributions to the conversation about the opportunities and risks of work at EA organizations.

I'm actually concerned (and have been so for a while) that many people in the community overestimate the value of work at EA organizations compared to alternatives. For example, if a friend told me that the only option they've considered for their summer was this summer programme, I'd strongly advise them to also look at opportunities outside the EA community. I think that widely promoted messages, e.g. about the value of 'direct work', have contributed to this (what I think is a) misperception (NB I don't believe that the sources of these messages actually intended to promote this perception, more that the nuance of their intentions got inadvertently lost as messages were propagated through the community). I therefore particularly appreciate that you point out some specific downsides a summer programme such as the one discussed here, and give some actionable reasons to consider alternatives. (And that you do this in a public place, accessible to those with limited opportunity to have high-bandwidth conversations with people working at EA organizations or other relevant experience.)

That being said, my current view is that this summer programme can be a good choice for some people in some circumstances. Based on my experience this summer and the feedback from our summer fellows, I'd also guess that for several of them the summer fellowship was actually more valuable than what they'd have done otherwise. I'd like to gesture at my reasoning, in the hope that this is helpful for people considering applying to a similar programme. (Caveat: as the OP also points out, this programme might look quite different next year, if it'll be run at all. I recommend checking with whoever will be directing it.)

• Just to prevent misunderstandings, I'd like to mention that this year's programme differed in major ways from previous versions (and was run by different people). In fact, I had been participating in the programme in 2018, and had a mixed experience; when planning this year's programme, I've deliberately tried to learn from this experience (I had also reached out to my fellow fellows to ask them about their experience). E.g., I had appreciated the intellectual autonomy and the opportunity to informally talk to staff at CEA, FHI, and GPI (which shared an office at the time), but I had also often felt lonely and isolated and would have liked more supervision for one of my two projects. As a result, for this year we designed several activities aimed at fostering a collaborative and open culture among the summer fellows, giving them opportunities to introduce themselves and their projects to other staff at the office, and paired each summer fellow with a mentor. 
• Overall, I think my experience last year was very unlike the experience of this year's fellows: For example, after a prompt to consider what they'd have done otherwise, this year 10 out of 11 people answered 9 or 10 on a 1-10 scale for "How much better or worse do you feel about the summer fellowship than about what you would have done otherwise?" (and the other one with an 8); on the same scale for "If the same program (same organizers, same structure etc.) happened next year, how strongly would you recommend a friend (with similar background to you before the fellowship) to apply?", 9 answered with a 9 or 10 (and the other ones with 7 and 8). While I have concerns about the validity of such self-reported impressions for assessing the value of the programme (which I think will depend on things like future career outcomes), I'm confident that I'd have given significantly less positive responses about my experience last year. 
• Denise, you're right that this year's programme did "not by default result in job offers being made to the best performers," and wasn't intended to do so. I don't believe this is necessarily a downside, however. In my view, one of the main benefit of this programme is to help people decide whether they'll want to aim for similar work in the future. I think this will often be valuable for people who wouldn't be available for full employment after the programme, e.g. because they'll continue their university studies. In addition, I believe that several of the benefits of this year's programme were enabled by us fostering a relatively non-evaluative atmosphere, which I worry would not have been possible in a programme designed to identify future hires. So overall I think that (a) programmes that are essentially extended "work trials" and (b) programmes that won't usually lead to future employment have different upsides and downsides, and that there is room for both. (I do believe it's important to be transparent about which of the two one is running. I'm not aware of any of this year's summer fellows erroneously believing the programme would result in job offers, but I'd consider it a severe communications mistake on our part if this has happened.) As an aside, I think that a programme like this can to some extent further people's careers. For example, for the summer fellows I mentored I'm now able to provide a credible reference for applications to jobs in EA, and in fact expect to do so over the next few weeks in 1-2 cases. I do, however, agree that these benefits are importantly different from the prospect of a relevant job offer, and I doubt that these benefits alone would make the summer programme worthwhile.
• My impression is that our mentoring setup worked quite well, at least from the fellows' perspective: 7 said they found mentoring "very valuable", 3 "valuable", and 1 "~neutral;" similarly, 10 out of 11 people said they'd like their mentor to mentor other people in the future, 9 of them maximally strongly. This seems consistent with responses on specific ways in which mentoring provided value and ways in which it could have been better. That being said, I do think there is room for improvement - for example, we weren't in all cases able to find a mentor with relevant domain expertise. My loose impression from this year was that cases in which the mentor had domain expertise were a much "safer bet" in terms of mentoring being valuable. In other cases, I think we were able to overcome this challenge to some extent: for example, I worked with two of my mentees to help them reach out to various domain experts, as a result of which they had several conversations; I still don't expect this provided as much value as direct and regular supervision from someone with domain expertise would have. (On the other hand, I think domain expertise is helpful all else equal, but far from sufficient for good mentoring. We did try to provide guidance to mentors, though I think with mixed success.) Anecdotally, based on my own experience in academia (and the distribution among my friends), my guess is that we've provided significantly more mentoring than the median academic internship, though except in 1-4 cases not as much quality-weighted mentorship as in the right tail (say, >95th percentile) of the academic mentorship distribution. (My wild guess is the situation is better in industry, though with other downsides, e.g. less autonomy.)
• As mentioned earlier, I think the programme did a decent job at helping people to decide if they want to work in similar roles in the future - and does so in a way that is less costly to people than, say, applying for a full-time job at these organizations. So in this sense I do think it had career-related benefits, albeit by improving future decisions rather than by getting a job immediately. For instance, I think I would have benefitted significantly from the opportunity to work at CEA or FHI for a few weeks before deciding whether to accept the offer I got for FHI's Research Scholars Programme, a 2-year role. (As I said, I participated in last year's summer programme - however, I needed to decide to apply to RSP before the summer programme started, and I got the RSP offer quite early during the summer.) In fact, I believe that in at least one case the summer fellowship helped a fellow to decide that they do not want to accept an offer to work at an EA organization, and I think this was among the most tangible, valuable outcomes of this summer. Siebe, I agree with you that this has to weighed against the potential to get more information about other paths by doing internships elsewhere. However, I think that conversely these other internships would provide less information on how much one would like to work at, say, FHI. I think that it depends on the specifics of someone's circumstances which of these things is more valuable to explore.
• Summer fellows were paid. (With the exception of a very small number of people who couldn't be paid for legal reasons.) That being said, I do believe there might be room for improvement here: for instance, on a scale from 1 to 5 on whether they felt the amount of financial support was adequate 8 out of 11 fellows responded with 4 or 5 - I think that ideally this would be 11 out of 11. I'm also concerned that 10 out of 11 fellows indicated that their savings stayed constant or (in 3 cases) decreased; while I don't think this is catastrophic given the short duration of the programme, I think it suggests the programme is by far not as accessible to people with a less comfortable financial situation than I would like.
• Siebe, I agree that "EA-branded organizations are young and often run these programs without much prior experience." For example, my highest degree is a master's in an unrelated field, and I've had maybe 2 years of work experience relevant to me running such a programme and mentoring researchers (though potentially somewhat less or much more depending on how you count things I did during my university years outside of a paid role). I agree this is a drawback, all else being equal. However, I think it might be relevant to note that (i) Rose Hadshar, the co-director of this year's programme, has been the Project Manager for the Research Scholars Programme since last fall, which I think is very relevant work experience (more so than my previous one), (ii) we consulted with several people with significantly more relevant experience, including staff involved in running previous versions of the summer programme. I also think the "all else being equal" clause is important, and possibly not true in expectation: for example, I think it's fair to say that Rose and I really cared about this programme and were highly motivated to make it go as well as we could. By contrast, I've heard many stories from my friends in which they did an internship at an established institution, working with people that had decades of work experience, but came away very frustrated because they felt the internship programme had been poorly thought out, consisted of boring work, or suffered from an uninspiring organizational culture. Clearly, many internship programmes outside of the EA community won't suffer from these problems (and some at EA organizations will), but overall I believe it's worth finding out as much as possible about the specifics of an internship programme one is considering rather than relying too much on proxies such as prior experience. 

I think most of these points provide complementary information to the considerations you raised, rather than undermining them. I think they're relevant if someone wanted to form an overall view on whether to apply to such a programme, but I'm not trying to dispute the value of your comments. I'm curious to what extent the points I raised change your overall view on the upsides and downsides of this summer programme, or if you have any further questions.