Best Practices for early-career Research Management

By Tilman, Gideon Futerman @ 2023-09-15T13:31 (+25)

This post is part of the “Insights of an ERA: Existential Risk Research Talent Development” sequence, outlining lessons learned from running the Existential Risk Alliance (ERA) Fellowship programme in 2023. The post is also part of the career conversations week.

Executive Summary

• ERA used a research management model involving Mentors, Research Managers (RMs), and fellows. Mentors provided subject matter expertise, while Research Managers offered project oversight and professional development support.
• Key management responsibilities included scoping projects, setting goals/timelines, providing feedback, solving issues, and facilitating networking.
• Techniques like regular check-ins, collaborative goal-setting, and reviewing progress helped keep fellows on track.
• While RMs sometimes lacked domain expertise, scoping projects involved leveraging fellows' proposals, open-ended questioning, and connecting fellows to relevant resources.
• Dedication to proactivity, good feedback mechanisms, and random "water cooler conversations" were noted as highly valuable aspects of the research management.

Introduction

Research management plays a vital yet often underappreciated role in maximising research outcomes, especially for early-career researchers. The practices used to support and guide fellows have a direct impact on their progress, learning, and overall success during formative periods.

Effective management is crucial for aligning research efforts with strategic goals, addressing bottlenecks, facilitating important connections, and ensuring work remains on track to achieve its intended impact. For those just embarking on an academic or cause-focused career, the guidance, oversight, and relationship with a Research Manager (RM) can be instrumental in shaping their skills, networks, and long-term trajectory.

This post examines the research management model used in the ERA Cambridge Fellowship. First, we cover general recommendations for effective research management, drawing from Managing to Change the World (and these notes by Peter Wildeford) and other sources. After outlining established practices, we make some notes on how full-time research management is different from research management for fellowships. With this in mind, we focus on our experience managing researchers, insights gained, and areas for improvement. This post is long, so feel free to only read the relevant sections. 

We aim to not only document our learnings but also foster thoughtful discussion around how to support early-career researchers best. We aim to iteratively strengthen our model through open collaboration with the research community.

Research Management Setup at ERA

The ERA Fellowship structure is fellow, Mentor, and Research Manager, as outlined in the previous post.

Mentor:

• (before the fellowship) adjusting the scope of the research project,
• (during the fellowship) discussing domain-specific research with the fellow, and
• Giving feedback on research output.

Research Manager:

• (before the fellowship) hires fellows, advises on the research proposal, helps to figure out mentorship,
• (during the fellowship) keeps track of research progress, helps to solve bottlenecks of research fellows, helps to improve Mentor-fellow communication, and gives additional project or cause-specific advice.

What is Research Management?

The goal of research management is to improve the research output of researchers.

Mindset of a Manager

• Do what is most important to do and prioritise the most important task over other, perhaps easier, tasks.
  • Even if some tasks are simple to address upfront, their importance - often measured by urgency or impact - should be the primary consideration.
• Delegate work to other people as much as possible.
  • If tasks can be delegated, they should be. Tasks should be handed down the managing pipeline (imagine, e.g., CEO → Head of Development → Project Manager → Engineer).
  • Handing down tasks should happen until you reach the person who can do the task well enough.
• Managers spend time managing and don’t necessarily do direct work. That encompasses reviewing documents, revisiting meetings- and overarching goals notes, 1-1 meetings, etc.
  • A common failure mode for new managers is trying to squeeze in managing between their other work commitments. 
  • Managing shouldn’t be an inconvenience.
• The people you manage shouldn’t think your meetings are a waste of time.

Key Responsibilities

Scoping a research project 

• (usually takes some context on the research itself, though managers usually lack domain-specific knowledge.)
• Managers help clarify the theory of change for a project.
• Managers assist in creating a project plan when researchers expect to achieve goals/milestones.
• Helpful mnemonic “Five W’s and an H”:
  • Who should be involved?
  • What does success look like?
  • When is the project due?
  • Where might the fellow go for help?
  • Why does this project matter?
    • How should the fellow approach it?

Weekly check-ins

• Staying engaged with the research is important to ensure all deadlines are met.
• Meetings should cover check-in, current bottlenecks, progress on last week’s task, and next week's goals.
  • Personal preference over these meetings plays a role.
• Managers should value these meetings highly since this is the main uninterrupted interaction they get with their fellows.
• Best practices:
  • Recurring meetings at the same time (e.g., every Friday at 3:30 PM)
  • Meetings should start on a light note.
  • Preparing meetings with agendas of what to cover during the meetings (there might be a default structure). Try to avoid adding things spontaneously during the meetings since you want your fellow to be as prepared as possible for these meetings.
    • An example of a prepared meeting agenda can be found in the appendix.
  • The manager leads the meeting and takes notes. Ask your fellow whether they want to add anything to the notes.
• Managers and fellows should together agree on the next goals.

How to help researchers improve skills

• Managing encompasses improving the staff’s performance in some specific skill.
• How to coach researchers effectively
  • Step 1: Collaborate to set a goal
    • Describe a behaviour or result you want to achieve by a date when you want it achieved.
    • Urgency: Create short-term tasks and give short deadlines
  • Step 2: Collaborate to brainstorm resources
  • Step 3: Collaborate to create a plan
  • Step 4: The researcher acts and reports in the weekly meetings on the plan

Building Relationships and Hiring

• Create and encourage Work-with-me-docs
• Feedback Mechanisms
  • Getting feedback is super important for the health of an organisation
  • Create enough room for staff and researchers to give feedback.
  • See ToC for ERA
• Hiring
  • See hiring retrospective
  • In short: Try to be as fair as possible and reasonable.

Fellowship vs. full-time research management

• Since we accept fellows without pre-existing mentorship arrangements, a big part of the role of the RM before the fellowship consists of
  • Refining the research proposal of the fellow
  • Clarifying the theory of change of the research
  • Getting suitable mentorship for the fellows
• Less focus on long-term satisfaction
  • In normal research management, there is a big focus on retaining talent for your organisation.
  • That means you want to ensure that the person you manage is happy with their work and, more crucially, is satisfied with their work over an extended period.
    • That usually means that you want your researcher to grow over time and be regularly excited about their challenges. ^[1]
  • Since fellowships only run between 8-12 weeks, usually, there is less time to discuss and support the overall growth of the researcher.
• Less focus on fitting research into an overall agenda 
  • Because the Fellowship is agnostic to specific agendas and focuses on improving researchers, usually, there is little thought spent on how research fits into a holistic picture (besides being generally useful for mitigating x-risk).

Research Managers’ Perspectives

This section outlines insights directly from the RMs at ERA. It covers their key responsibilities and techniques found valuable for supporting fellows during the programme. Our goal is to share hands-on experiences that may inform effective research management, especially for short-term, intensive fellowships like ERA.

• Preamble: The needs of individual fellows varied widely
  • One valuable lesson for most RMs was to ‘adjust the seat’ to each fellow
  • It is helpful to ask before starting the management relationship what type of adjustments are most suitable for the fellow
  • However, there is also the dynamic of adjusting practices during the run of the fellowship. Partially, this is because both the fellow and RM learned about one another, but also, the requirements of research management changed depending on when it was in the fellowship, the availability and role of the mentor, and other surrounding factors (e.g., the fellow’s changing interests over the fellowship)
• Weekly meetings
  • Having weekly meetings with prepared action items made it easier to stay on track with important topics, like recurring issues, accountability for various tasks, or overarching goals for the fellowship.
  • Most RMs used a pre-prepared template to cover the significant issues for the week before discussing other concerns or ideas that fellows had (see Appendix for an example template).
  • Discussing next-week’s goals in these meetings was for some participants crucial for not falling prey to planning fallacies (“Writing this section will only take a day”) and unwelcome surprises (“I’ll use the existing implementation in the library”^[2])
    • Realistic self-assessment and goal-setting regularly help with pre-emptively mitigating blockers by aligning expectations with actual outcomes.
  • Per fellow:1 meeting-hour per week, 30min/week preparation 
• Clarifying the theory of change of the research before(/early) on during the fellowship
  • This was helpful in the cases where research pivoted during the fellowship
  • Having a theory of change laid out made it easier to figure out where to “start again” and rethink where assumptions were wrong
  • Per fellow: 1-3 meeting-hours before the fellowship, 1-2 hours reading documents
• Giving feedback on work
  • This covered reading sections of text, important mail, or erroneous code.
  • We also created a set of criteria by which to give feedback, specifically around reasoning transparency, etc.
  • Per fellow: 0-30 minutes per week, 1-2 hours post-fellowship for the deliverables
• Exploring ideas with fellows
  • This covered discussing personal development with the fellows, including their career options and biggest uncertainties.
  • It also included discussing ideas around x-risk and their views on it in general and within the cause area.
  • Per fellow: 0-60 minutes per week 
• Random encounters
  • Chatting on random encounters about work, as well as chatting about issues on Slack, was noticeably more than we expected and also varied a lot between fellows.
  • Some fellows explicitly wanted to be asked about their projects during coffee breaks.
    • We assume it would be stressful for others, so the default was only to do it if it came naturally (or through the fellow).
  • This also sometimes involved talking about their general ideas on important issues, as discussed above.
  • Per fellow: 0-1h randomly chatting per week, 0-30min chatting on Slack 
    • Note that this varied most between RMs that were a lot in-office and remote 
• Preparing the Programme for fellows
  • This will be covered thoroughly in a future post (stay tuned!).
  • 3 hours before the fellowship, 3 hours throughout the fellowship 
    • Not including time spent during the events
    • Note that one team member who owns programming spent significantly more time on these tasks.
• Connecting fellows with the research community at large
  • Providing introductions between fellows and other researchers was deemed valuable.
  • The need for introductions varied a lot across cause areas.

Impact of Research Management at ERA

This section outlines insights on the impact of research management during the ERA Fellowship based on survey responses from ERA fellows after completing the fellowship. We sent out anonymous surveys and received responses from the majority. 

While the survey results were largely positive regarding the role of RMs, it is important to note that some fellows did not fill out the survey. Those with more critical perspectives may have been less inclined to provide feedback. We aim to mitigate this potential bias by also gathering candid verbal input and being self-reflective when reviewing areas needing improvement.

With that context, here are the key themes on research management that emerged from the survey responses:

Managers provide valuable holistic support for fellows

• Multiple fellows praised the proactiveness and availability of their managers, and this was often considered much of the value RMs provided.
• RMs provided technical and emotional support. This included attempting to increase the confidence of the fellows when necessary. Helping combat imposter syndrome was one important aspect of the RM’s role; approaches here included highlighting the work the fellows had done so far, our excitement about their projects, and highlighting how given how personalised the interactions in the programme and rigorous the application process we knew the fellows clearly deserved to be at ERA.
• Fellows noted that RMs made them feel safe to speak up with ideas, questions, and concerns on a variety of topics.
• Having multiple managers around the office was deemed valuable, with fellows appreciating having multiple people to ask for advice.

Fast Feedback through the Manager

• RMs were seen as supportive, providing timely and detailed feedback.
• Feedback mechanisms like regular updates, goal setting, and checking the viability of plans were appreciated.
• Commenting on documents was seen as a mechanism for useful feedback alongside meetings and conversations.
• Helping to formulate and manage the project structure, including reminding fellows of their original goals, was a key management role.
• Questioning of the fellows' ideas, research, and plans by a supportive manager also provided large amounts of value in refining their projects.
• RMs were not perceived to be pressuring, and some fellows appreciated a more ‘hands-off’ approach from their RM.
• Having a ‘second pair of eyes’ or someone to bounce ideas off was one of the most useful aspects of the RM.

Connecting fellows to the broader community

• Fellows appreciated opportunities for 1-1 sessions with researchers, enabled through RMs
• Job opportunities shared by the RMs were also useful to them.
• The RMs were useful in connecting the fellows to others in their networks, which may lead to further future collaboration, research, or jobs.
• There were positive remarks on conversations (often informal, ‘water-cooler’ conversations) with RMs about their cause area and x-risk. Some fellows found this the most valuable part of their interaction with RMs.

Time Management & Goal Setting

• Fellows expressed a need for guidance in setting realistic goals, defining the scope of their projects, and managing time.
• RMs were useful in helping fellows prioritise between ideas.
• Generally, more clarity over what would be expected at different times was seen as needed, including laying out what the schedule would look like to fellows before the fellowship.
• Some fellows suggested that there should have been more engagement with fellows before the fellowship, although whether this would have been feasible with many fellows is unclear.
• Some fellows wished for more reminders about their original, ambitious goals for the fellowship.
• One fellow noted a mismatch between the level of ambition that they had for their project and the level of ambition their RM had for their project. While this didn't cause any major problems, ensuring fellow and RM are aligned on the goals for the project may be beneficial. Consistent clarity of communication around these goals and making more space for feedback around them may have been useful to help solve this problem. 

Cause Area Expertise

• Fellows differed in how important they felt it was for their RM to have object-level knowledge about their project. For some fellows, this was very helpful and profoundly shaped the direction of the projects, while for others, such input was less necessary.
• This seemed to differ depending on both the mentor and the project type. While most fellows felt that object-level interactions with their mentors were considerably more valuable than with their RMs, this was not universally the case, and most fellows found object-level interactions with RMs somewhat useful.
• Many fellows commented on the importance of having the RM to discuss specific issues around their cause area, even if this wasn’t specifically responsible for the project. In general, having knowledgeable people around seemed to be of high value.

Juniority

• The juniority of the RMs was noted as a negative by some fellows; however, it is unclear whether more senior RMs would have wanted to participate in ERA. ^[3]
• It was noted that having RMs with expertise in topics they were willing to talk to fellows about was deeply useful; it is possible for junior RMs to have such expertise, although this is obviously less common than in more senior individuals.
• One way this may have been particularly impactful is a general feeling that some aspects of the fellowship were less ‘professional’ than may have been optimal; for example, last-minute urgent messages or the non-standardised/streamlined process for reaching out to possible mentors.

Mentorship and Support by RM

• Some fellows felt that their weekly meetings with RMs could have included more open, unstructured time, focusing more on quick feedback and reflections.
• Proposals were made for more frequent meetings involving the fellow, RM, and Mentor.
• Fellows desired more non-work related interactions with the RMs; other fellows praised how well RMs from all cause areas interacted with them.
• While most fellows had a solid connection to their RM, one reported feeling that their RM was not very invested in the project, seemingly due to a mismatch in perceived worldviews/culture. This is something that ERA would have to rectify in future iterations, including working on ways to pick this up earlier in the programme. Notably, however, this didn’t seem to reduce the fellow's satisfaction with the programme as a whole substantially.

So, how valuable were RMs?

• In general, the fellows rated the research management of the fellowship highly.
• However, how valuable fellows felt the RMs were significantly differed, likely due to several factors, including:
  • The involvement and availability of their Mentor
  • The RM’s experience in their area
  • How much support they felt they needed from their RM

Conclusion

The ERA Fellowship is an ongoing endeavour to support and grow the existential risk research community. While we have outlined our learnings and insights to the best of our ability, we recognise there is always room for improvement.

We believe open discussion and collaboration are crucial for refining our approach. Thus, we warmly invite feedback from researchers, managers, and others. By incorporating different perspectives, we can enhance the fellowship programme and contribute more effectively to building capacity for existential risk reduction.

If you have thoughts to share, please comment below or email us directly at tilman@erafellowship.org. We also have an anonymous feedback form available. Your input is invaluable as we work to provide the best possible fellowship experience.

Appendix

Example Meeting Agenda during ERA (Week 2)

Note: This agenda is from an internal document written by Nandini. RMs used adapted versions of these agendas for their weekly meetings.

1. Check-in
   • A quick check-in on any pressing questions or concerns.
2. Midpoint presentations
   • Discuss the upcoming midpoint presentation.
   • Emphasise that presentation content is more important than polished slides.
   • Review key points to be addressed and offer guidance on how to effectively structure and present their research.
3. Feedback / Reflections so far
   • How’s everything going? How have their goals shifted since they started ERA?
4. Open Space
   • Discuss whatever your fellow finds useful, e.g., their research progress, general trends in the field, etc.
5. Questions & Next Steps
   • Address any other questions or concerns from the research fellow, especially any challenges encountered in the first week.
   • Recap next steps and actions, and set concrete deliverables for the next week.

Credits

This post was written by Tilman Räuker and Gideon Futerman, part of the Existential Risk Alliance (ERA) team, a fiscally sponsored project of Rethink Priorities. We are incredibly grateful to Nandini, Oscar, Moritz, Joël, Irina, and others who kindly shared their insights and provided constructive feedback on this post. However, it's important to note that their feedback doesn't imply complete endorsement of every viewpoint expressed in this post. Any inaccuracies, errors, or omissions are solely our responsibility. We encourage readers to engage critically with the content and look forward to incorporating further feedback as we refine our understanding and approach.

1. ^{^}

   Similar to the flow quadrant.

2. ^{^}

   There was not an existing implementation.

3. ^{^}

   It should be noted that the role was initially called “Cause Area Lead” and thus may have attracted a different subset of people than if it was called “Research Manager” from the onset.