Underinvestment at the top: what I discovered coaching a dozen EA leaders

By Tee @ 2022-04-01T12:36 (+102)

Table of Contents & Key Links

I’d like to give special thanks to the trial participants – those who were comfortable giving public testimonials include Angus Mercer, Michael Plant (Happier Lives Institute), Luke Freeman (Giving What We Can), Bridget Williams (Center for Population-Level Bioethics), Kat Woods (Nonlinear Fund), Nikita Patel (Fortify Health), Varsha Venugopal (Suvita), Irena Kotikova (EA Czechia) and Dominika Kropucin (Rethink Priorities). More thanks in this section

Navigating this post

I’ve tried to make this rather lengthy report more ‘click-to-expand’ in order to help readers navigate it nonlinearly (more like Seinfeld, not Game of Thrones). While the quantified results carry interesting potential implications, I think pairing the Executive Summary & Overview and Takeaways & Lessons Learned sections would provide for the reader a summary together with richer insights into what happened during the coaching trials, and what I think it could mean for the Effective Altruism community. The anticipated questions section is also pretty revealing of my experience and motivations in conducting the trial and producing the report. 

Reach out at tee@teebarnett.com or my website if you’re interested in receiving my coaching, collaborating and/or helping fund my efforts to continue building a professionalized support infrastructure in the EA community. 

Executive Summary & Overview

I’m Tee – you can find more contextualizing sections about me, about my style of coaching and references for the general case for coaching towards the end of the report. 

Many of the insights contained within this report are derived from November 2021 – February 2022, where I conducted a dozen free trials of my coaching with EA leaders that I feel went pretty well.[1] Across several methods of approximating value and effects, the feedback seems to suggest that the coaching is both desired and effective for those undertaking the complexities that come with doing scalable good. I think there could be something profound here for the community to consider. 

Given how highly the Effective Altruism community appears to value certain forms of direct work, it feels obvious to say that, for important individuals doing this direct work, investing in the development of their capabilities could be quite impactful. 

From my prior experience coaching other high-performers, I had good reason to think that coaching could be a potentially potent way to develop EA leaders (EDs/CEOs, senior-level managers & thought leaders). Coaching is only beginning to catch on within the community it seems. However, what is currently on offer is almost exclusively reliant on opaque patterns of in-network endorsement, varies widely in quality and intended purpose, approximations of its value are almost completely anecdotal, and there are several nontrivial and very understandable barriers to access. 

I thought a more rigorous trial of my coaching would be fairly revealing in regards to key questions about my own practice. But more importantly, I thought a structured trial would give me a better sense of how my coaching (and perhaps by extension, coaching in general) could meet the developmental needs of individuals who are quite important to EA. 

What if, for example, key agents that we’re counting on to solve important problems can noticeably improve their capabilities? Leading an organization and/or coordinating others to achieve scalable good is incredibly challenging. Basically nobody is natively skilled in all domains required to be a great leader. 

To be clear, this report is not a commentary on the leadership fitness of the trial participants, but instead the reality that different people require many types of support. The trials made it clearer to me that investing in developing the capacities of our leaders is neglected and undervalued, and that having a vetted support infrastructure of high quality coaches for EA leaders could be hugely impactful.

Notable Results

What effects does the coaching have? 

According to quantified survey data of a dozen EA leaders, the top categories of self-reported value are the following: 

1 – "Strategy towards personal and professional - I have greater ability to navigate how my personal life interacts with my professional life."

Concrete examples derived from qualitative answers

“Tee Barnett Coaching has been really helpful for me across a range of areas, from the practical to the theoretical. We've explored areas from the meaning of intelligence, to the role of a CEO, to building operational systems, to impostor syndrome. I leave each session with a whole lot of new insights, and an option to innovate and improve in my professional life." 

Nikita Patel, Co-founder & CEO @ Fortify Health

2 – "Perception shift - the underlying paradigm that I use to interpret large areas of my perception have meaningfully shifted in a way that is beneficial to me."

Concrete examples derived from qualitative answers

“[Tee] was careful in asking questions that would help both of us to understand how I perceive situations and how misperceptions are limiting my ability to do good work and have meaningful relationships. He was generous enough to go into some difficult issues and did so with care and compassion. The coaching has helped me to identify mental and behavioural patterns that are holding me back, and provided me with some strategies to correct these." 

– Bridget Williams, Postdoctoral Associate @ The Center for Population-Level Bioethics, Rutgers University

3 – "Skill & mental models - mental models that I received were directly useful to what I wanted to do and/or sharpening a specific skill"

Concrete examples derived from qualitative answers

"Tee nudged me to consider new ways to approach several areas of my work- interaction with colleagues, thinking strategically on hiring and letting go, and fundraising. I can already see my changed approach paying dividends. Tee is brilliant at listening deeply and providing appropriate mental models to understand yourself and your approaches." 

Varsha Venugopal, Co-founder @ Suvita

Notable results by the numbers

Basic trial structure by the numbers

Quantitative Results

Note: I do not take the numerical valuations of the coaching literally, nor do I think it would be wise for anyone interpreting this report to anchor too hard on this. They’re simply meant to be a snapshot of potential value. I am happily soliciting feedback on how these questions were constructed. 

Spreadsheet of quantitative answers and methodology notes section

1) “If you could have received a cash grant instead of the trial package of coaching sessions, how much money would it take for you to be indifferent between the grant and coaching?” 

(12 of 12 respondents answered) 

2) “If you were earning substantially more than you currently do (e.g 2 - 5x, how much would it have been worth to pay for the trial package of sessions if you were paying out of pocket?”

(12 of 12 respondents answered)  

3) “By what percent would you estimate coaching changed the amount you accomplished over the past month?” 

(12 of 12 respondents answered) 

4) [optional question later in the survey][2] “How much would it have been worth to pay for the trial package of sessions, if you were paying out of pocket? (per hour)”[2]

Quantitative Visuals

These visualizations have been pulled directly from the data inputted by respondents via Typeform. 

Satisfaction Matrix – Please rate the areas where you received value from the coaching

(See category labels below)

Category Labels

Top 5 areas of value across trial respondents

  1. Strategy towards personal and professional - I have greater ability to navigate how my personal life interacts with my professional life.
  2. Perception shift - the underlying paradigm that I use to interpret large areas of my perception have meaningfully shifted in a way that is beneficial to me
  3. Skill & mental models - mental models that I received were directly useful to what I wanted to do and/or sharpening a specific skill
  4. Discovery of self - I was able to uncover information about myself that I can now use to interpret meaningful things in the present
  5. Mental tools - I picked up valuable tools from the coaching that will meaningfully affect the way that I think about and approach situations in the future

How satisfied are you with Tee Barnett Coaching overall? 

Did Tee Barnett Coaching meet your expectations? 

Are there ways that Tee’s Coaching had a meaningful effect on you and/or what you’re capable of doing? 


Were there any situations where a shift generated by Tee’s coaching resulted in you taking different actions than you otherwise would have?

Do you feel Tee’s Coaching could be beneficial to other leaders in EA? 

Would you like to recommend that someone try Tee’s coaching? 

Would you like to continue being coached by Tee Barnett? 

Within which compensation structure would you be open to continue being coached by Tee Barnett? 

Takeaways & Lessons Learned

EA Leadership & Community Insights

Note: these are a collection of impressions derived from my experience coaching EA leaders, my experience as a co-founder of Rethink Charity and numerous informal conversations. It is not meant to come across as objective, conclusive or exhaustive by any means, though I would encourage community funders to consider directing resources toward further exploration and research in this area. Once again, these insights are not a commentary on the leadership fitness of the trial participants, but instead the reality that different people require many types of support.

Coaching Insights

Tl;dr – somewhat surprisingly, what I outline below roughly corresponds with the quantified survey results on what seems to be most useful and potent about my style of coaching. In short, I gather that it is quite valuable to spend a dedicated period of time with a skilled guide who is well-versed in a major component of their worldview (EA and resulting effects on thinking) who also falls outside of their professional and social circles. The ‘personal strategy’ style of coaching traverses and analyzes the emotional connection between the personal and professional, likely making sessions feel more emotionally resonant (and therefore impactful) than professional coaching, and more real-world and action-oriented than some forms of talk therapy. And finally, we often address the ‘social layer’ in a tangible way that leads to benefits from addressing social dynamics in a way that isn’t as passively theoretical/abstract as some rationalist community-derived practices (though I’m indebted to thought leaders in that area for their openly shared models and methods). 

Below are some of those insights in greater detail: 

Trial & Operational Insights

Things to do differently (moving forward) 

Selected Qualitative Answers

View appendix of qualitative answers for more

“Are there ways that Tee’s Coaching had a meaningful effect on you and/or what you’re capable of doing?” 

“I've had a number of profound breakthroughs during our sessions. These were crucial shifts that I was able to make regarding my workflow and I'm already seeing results from it.”

“Tee worked with me to decide upon and take actionable steps in regards to the development and management of my staff, my personal ways of thinking and behaviours, and improving my stakeholder engagement.”

“Tee helped me reach some conclusions about myself and what I bring to my current role that caused perceptual shifts which I believe have made me more valuable to my team. He also helped me discard at least two bad plans in favor of better plans.”

“Were there any situations where a shift generated by Tee’s coaching resulted in you taking different actions than you otherwise would have?”

View appendix of qualitative answers for more

“I adjusted the strategy for my organisation, my own management style, how I thought about fundraising, and how to restructure my own role.”

“I had some questions around the role of a CEO, as well as questions around feeling stuck on some overwhelmingly large tasks. Tee helped me realistically analyse what I was bringing to my role, and to critically analyse my own views around the meaning of intelligence. He also suggested practical action points on one task I was especially stuck on, around an operational systems overhaul, which helped me break down the process into bite sized pieces; from there, I've been able to make significant progress on this task, that previously felt bottlenecked. Tee also helped me build confidence in saying 'no' where it was needed, to prevent future instances of burnout; I've been a little more successful at setting boundaries and protecting my time since my coaching with Tee began.”

“In one case, I changed my approach to an important conversation with an employee in a direction that was much more positive and productive. In another case, I discarded a plan that Tee made me realize was likely to fail and crafted a better, more robust plan.”

EA Leadership Recommendation – ”Do you feel Tee’s coaching could be beneficial to other leaders in EA?” 

(See publicly displayed testimonials on this question and view appendix of qualitative answers for more) 

“I think Tee has a unique ability put people at ease, actively listen, and develop intelligent, nuanced feedback, even with little information. He knows how to ask the right questions, and to prompt his clients to think about things in ways they may not have before. This, combined with his experience of EA, puts him in a brilliant position to support others in the EA community.” 


“He does an amazing job of skillfully using a wide array of psychological techniques without you even noticing them. And then he does the best part for an EA - he explains what he was doing so you can understand it - and yourself - better. Tee is a fantastic coach. I highly recommend him if you want to be happier and more productive.” 

“Generally speaking, I think that working with a knowledgeable coach - in this case Tee, can be a valuable experience to all leaders and individuals working in roles that may affect others (for example People Managers).” 

What are the main reasons why you hadn't tried coaching before?

(Optional section – selected ‘no’ for having had coaching before) 

“I wasn't aware of a coach would work for my needs and my style.” 

“I simply hadn't considered myself a candidate for it.” 

“I guess the main constraint is the cost/fee. Paying out of pocket can get a bit expensive, especially if it is an ongoing commitment.” 

About me

I’m Tee – a self-styled ‘personal strategist’ currently based in Prague. You can learn more about my coaching practice from this section of the report and my website. At this point in time, I’m still coaching clients that I believe to have high impact potential, issuing more trials of my coaching, launching a podcast, and exploring potential ways to collaborate on the creation of a professionalized support infrastructure to serve those trying to achieve scalable good. 

I'm a co-founder, former executive director, and current board director of Rethink Charity (RC), a project collective that launched and/or incubated several EA community building projects, including Rethink Priorities, RC Forward, the EA Hub, the EA Survey, EA Giving Tuesday, and fiscal sponsorship for numerous startup EA-aligned projects (Dao Foods, AI Safety Support, Effective Thesis, LEAP conference). 

Big picture stuff – my ultimate aim is to utilize my life and work to substantially improve the flourishing of all sentient beings. I’m channeling these efforts through coaching, which seeks to cause deeply valuable perceptual shifts for influential individuals that have unusually high impact potential in the world (e.g. Executive Directors, CEOs, influencers, thought leaders)

The general case for coaching

Evaluating a more general case for coaching that invokes and analyzes several types of empirical data is outside the scope of this report. Additional recommended reading that originates from the EA community includes a section from Sebastian Schmidt’s post on further empirical support for the idea that skilled coaching can be materially valuable for coachees. I also found Lynette Bye’s cost-benefit analysis on her own coaching worth checking out as well. 

What’s Tee's coaching like? 

Working with me is about having a ‘personal strategist’ that helps guide the discovery, navigation and refinement of deep perceptual constructs that meaningfully affect your personal and professional life. Clients report that ‘shifts in perception’ are often the most valuable result of the coaching. This is meant to be done in a way that is truth- and reality-tracking better than the ingrained and patternized ways that people typically interpret their situations. 

In short, the idea is that working through surface-level issues will often lead to the exploration of deeper patterns and structures of sensemaking and interpretation. Addressing these deep structures causes important and lasting effects.

The subject matter of sessions and my employed methods are varied and specifically tailored for the individual. (Anonymized client story example). 

This type of coaching is nonetheless distinct from therapy. While we may work through emotional issues associated with the relevant challenges, several areas (e.g. trauma processing) are outside of the scope of this coaching. What’s taken from any given session shouldn’t be misconstrued as therapy or the advice of a credentialed clinician. 

What do I hope comes as a result of this report?

Anticipated Questions, objections and inquiries

Methodology Notes

Additional Thanks

My opening thanks to those who put the time and energy into reviewing this report, including Kerry Vaughan, Sebastian Schmidt, Luke Freeman, David Moss, Jamie Elsey, Mark Lee, Dominika Kropucin, Monica Chen, Emily Crotteau, Kristina Nemcova, Oliver Crook and Jess Smith. 

These coaching trials and the resulting report felt like a big step for me putting myself out there. Innumerable people are to thank for this and hopefully I’ve not been shy in private about how important their contribution has been. 

Most relevantly, the trial participants who were kind enough to part with their very valuable time to test out the coaching. These folks are dedicating the best working years of their lives (at least) to effective causes, which I greatly admire. The independent funders who believed in me and the potential for this trial to yield valuable information when institutional funders did not, or were not a great fit. From being afforded the opportunity to make a living conducting this trial, and my hardening experience fundraising for Rethink Charity, I cannot stress enough how important small- to medium-sized donors could be in helping new projects get off the ground. 

And finally, to those who contributed in my efforts to gain still in coaching – I couldn’t be more thankful. Many individuals spent countless hours patiently training me in the art of helping others. It’s something that changed my life and I would never trade away. 
 

  1. ^

    This period also included a handful of subsidized trials, though only the free trial feedback is reflected in the data. 

  2. ^

    I mention in the Methodology Notes section how it was certainly an oversight to not have asked for trial package valuations side-by-side with hourly valuations. There's good reason to believe the valuations would be different


PeterBrietbart @ 2022-04-04T15:19 (+22)

Thanks for the detailed post! Public posts on projects as personal as this like this can be a bit scary to  write, and I really appreciate the openness and detail. 

As someone who has known him for a fair while, one thing I think this post doesn't quite do justice to is how genuinely lovely and helpful Tee is. Coaching - like therapy - gets a lot of its efficacy from the relationship between coach and coach-ee, and so I'm not surprised by the positive feedback showcased above. 

Tee is unlikely to comment on how great he is, so I'm going to do it here instead :)

Tee @ 2022-04-05T11:33 (+10)

This was arrestingly sweet of you, Peter. Thank you. It's one of the best things that's come out of writing this post. I hope these types of comments get normalized in the community more broadly! 

 

jlemien @ 2022-04-02T03:46 (+13)

Tee, I really enjoyed this. I have a gut feeling that lines up with a lot of the points you described, which I will roughly summarize as "giving people in these situations coaching can vastly improve the 'how much stress I endure to how much I accomplish' ratio." Especially given the specific cultural habits/perspectives of EAs, I think that having many of us push a bit more for these kind of "support services" would be valuable.

I've been thinking about getting more EA-aligned coaches, and I was wondering if you could you describe how you got trained/certified as a coach, and if you would recommend that particular training program.

Tee @ 2022-04-02T12:59 (+10)

Heartened to see that you enjoyed it! And great prompts/questions. Lovely to hear that this post could go some way in nudging you toward coaching. I have lots of thoughts on how to find a coach that might turn into another post, but some about getting the vibe right and trialing with more than one coach I mention here in this post. Hope it helps

There’s a lot to say about how coaching can improve the metabolization of stressors. In many cases, I’m pairing remedial efforts (working through emotional fallout and imprints) with methods that often have the effect of building more flexibility into the client’s ways of making sense and interpreting things. We’re also proactively aiming for a more elegant way of being and acting that causes less emotional shear (ie psychological toll) in local contexts. This can be approached and achieved in many ways as you might imagine. IMO it’s always a different set of moves, methods and timing for each person.

On recommending a coaching program – I’d almost never recommend a specific program offhand. My probably unsatisfying (though very on-brand) answer is that the way you pursue coaching skill and credentialing is mediated through what kind of coach you want to be, how you think the world works, and what you believe the path looks like to get there. (e.g. “I want to make a career change. How do I make a career change? Learn a new skill well enough to earn a living. How do I do that? Get a degree in a different field of study. That way, I’ll know what to do and people will take me seriously if I have gone through a course/get a degree”. This isn’t necessarily incorrect, but it’s a line of reasoning that will result in a particular sequence of specific actions)

The subject of how to develop skill and how to think about credentialing in this ‘industry’ also super interesting to get into. My rough approximation of the credentialing landscape is the following:

  • Some pockets of high-quality programs that are narrowly specialized. These often require considerable time and monetary investment.
  • Lots of low-grade, mass-scale credentialing bodies that basically take aspiring coaches from 0 to 0.1. (Usually the first thing that most people reach for in order to find the permission to make a career change and qualify to get listed on coaching registries. I’m not knocking it because I’m sure some really great coaches got started that way. But the typical use-case is good to know)
  • Many ideologically intense woo-woo or pseudo-scientific-claims-about-maximizing-human-potentiality programs (where some content/model gems do exist)
  • A subset of (usually individual personality-driven) coaching programs with little substance that try to intensely upsell people. These irritate me to no end and I find many of them pretty predatory or manipulative in gross way.

I wouldn’t say I’ve exhaustively canvassed what’s out there, so if anyone reading this has any suggestions for high-quality credentialing programs, particularly ones that encourage the integration of multiple methods and paradigms, I’d be curious to hear about them!

I’d characterize my own situation as a sustained (over years) combination of structured coaching training, informal coaching training, being mentored by senior coaches and therapists, and building skill through my own practice. I’ve no credentials issued by an official body. My introduction to coaching was through Paradigm Academy, where I received the more structured coaching training. After that, I preferred to pursue coaches and subject-matter experts that I felt could upgrade my ongoing practice in some way. I’ve done most of that my own dime personally, but also in professional contexts. In my time at Counterfactual Ventures, we designed our founder selection and development program alongside a cognitive developmental theory-oriented consultancy co-founded by Bill Torbort. Developmental paradigms derived from Piaget’s work popularized by Kegan, Kohlberg, Fischer and Torbort etc. have influenced me a lot.

Because of my own models of what coach I want to be, how the world works and how to get there, unless there’s something really amazing that’s not on my radar, I’ll probably opt for undertaking a handful of highly specialized courses/training (and possibly getting credentialed) that will hopefully result in a varied and potent repertoire of coaching methods.

Apologies if that was a lot to take in! Happy to chat with you about it more if you'd like. Feel free to reach out at tee@teebarnett.com if you'd like to continue the discussion elsewhere. 

jlemien @ 2022-04-02T13:41 (+6)

Thanks for such a detailed response. I'll be thinking and processing all of this for a while. Great food for thought.

SebastianSchmidt @ 2022-04-06T17:49 (+2)

Overall I like this approach a lot and agree with a lot of it (I'm also a coach)

Could you share an example of each of the four categories of coaching courses that you mention? 

While I broadly agree with your assessment, I recognize that I could also easily be wrong as I haven't done a particularly systematic search and I've only done one coaching course myself and spent 15-180 minutes researching 7 other courses.

Tee @ 2022-04-07T09:51 (+4)

Getting back to this comment might take a bit longer than usual for me to dig up exemplars of each category and even decide whether I think it's a good idea to promote coaching types of a certain category (i.e. I'd rather be quite selective of what I choose to promote, rather than highlighting less good things in an attempt to be comprehensive.) 

Also this from above!  "I wouldn’t say I’ve exhaustively canvassed what’s out there, so if anyone reading this has any suggestions for high-quality credentialing programs, particularly ones that encourage the integration of multiple methods and paradigms, I’d be curious to hear about them!" 

SebastianSchmidt @ 2022-04-11T15:32 (+1)

I appreciate this although having a list of not-recommended programs for people starting might also be highly valuable. Especially, as it takes quite a bit of nuance to steer clear of the lower quality ones. With that said, I'm curious to hear what high quality you'd like to promote. I'm guessing Paradigm?

Tee @ 2022-04-11T17:10 (+6)

Yeah. On the face of it, I could see how this feels like an easy ask, but I intentionally constructed this post in such a way as to have my work stand up and be evaluated on its own, without being associated with (or positioned against) other programs, coaches, or theoretical paradigms for now. I'll have to spend a bit more time thinking through the differences between displaying in terms of 'highlighting', 'promoting', 'recommending' and 'publicly outing'. What to look out for in both positive and negative senses sounds like something that actually could be a great post on its own. Maybe we could co-author that. 

With that said, I'm curious to hear what high quality you'd like to promote. I'm guessing Paradigm?

This answer might make the above make more sense. My understanding is that Paradigm isn't currently active, but were it still an option, I would restrict the scope of my recommendation to attending their workshops and working closely with specific coaches. For someone looking for a well-structured coaching program and hoping for a widely-recognized credential to earn, it wouldn't be a very good choice. For me personally, my style of learning is boosted tremendously by fruitful individual relationships (great mentors, coaches, etc.) I like to think it worked out quite well in that sense

SebastianSchmidt @ 2022-04-06T18:01 (+6)

If I were to add a gross simplification of an alternative strategy (that I've used over the past 2 years)

  1. Experiment with different coaches and find a coach whose theory of change, style, and approach you like.
  2. Become an "apprentice" of that coach by getting regular coaching from him/her while reverse engineering/deliberately practice that coaching method.
  3. Get your first coachees and use the simple versions you've acquired this far to incrementally build your own coaching "style". #LearnByDoing
    1. Crucially, get good at requesting feedback from your coachees to grow.
  4. Find peers and mentors with whom you can have many conversations to reflect on and improve your coaching. Hero version: Move together with other coaches.
  5. Follow situational inspiration (i.e., use the challenges and questions you encounter from your individual coaching practice to seek out models, skills, and additional mentors)
  6. Build your map of the coaching landscape and experiment with an at least moderately promising coaching course.
Tee @ 2022-04-07T11:55 (+9)

Cool to see your path to this Sebastian. Some great tips here. What's both tricky and exhilarating about navigating this space is how free-form it is. I have lots of respect for people who are this damn resourceful. 

I'd call your "alternative strategy" instead a "potential pathway" to gaining skill as a coach. What I outlined was more like a scaffolding set of considerations for thinking about how to gain skill and become a coach, within which innumerable pathways could be pursued. But I did like that you provided a personal example. It's probably a lot more accessible for others to model off of than the prompts I gave. 

Writing out your journey in this way does make me want to write out something of my own that's similar. Like the Coaching Insights section of this post but for how one could work towards becoming a professional coach. Could be interesting for people whose pathway is currently under consideration (or for those looking to fold in different approaches) 

SebastianSchmidt @ 2022-04-11T15:50 (+3)

Thanks for this. I agree, it's a potential pathway to becoming a coach which involves more than building skills. E.g., forming an identity as a coach - for me it took a long time to be comfortable in the skin of being a coach (likely amplified by doing this during covid and moving to another country where I knew one person only). Ideally, I would have added more nuance and an illustration as it's not a simple linearly progressing approach but when we're new to something we need simplicity.

Would love to read about your path!

SebastianSchmidt @ 2022-04-05T09:44 (+9)

The specifics of coaching leaders and the trial:

  1. I'm struck by the effects reported after just around 4 sessions ~ 7 hours. I can't help but question whether these effects will last for more than a month after the coaching. When did they fill in the survey relative to the coaching? For how long do you predict that the effects will last?
  2. What do you think the ideal coaching frequency is for people in this reference class? I.e., every week, every other week, once per month? (Assume that we'll have unlimited supply of high quality coaches).
  3. One of the main rooms for improvement (from my perspective) might be if the evaluation had been conducted by a third person and I'll probably see if I can find someone like this if/when I do a trial myself. Do you have any thoughts or reactions to that?
Tee @ 2022-04-07T18:04 (+4)

I'm struck by the effects reported after just around 4 sessions ~ 7 hours. I can't help but question whether these effects will last for more than a month after the coaching. When did they fill in the survey relative to the coaching? For how long do you predict that the effects will last?


Good questions – I think the set of claims that I'm more comfortable standing behind are that the coaching seems to be quite valuable and important during the period that the coachee is engaged, rather than trying to predict what the consequent effects will be after a pre-packaged period of time for the trial. A follow-up on the stickiness and potency of consequent effects would be interesting though.  I'm taking this suggestion pretty seriously. 

The set of claims I'm more comfortable standing behind is particularly true if that pre-packaged period of time for the trial is constructed for reasons that aren't all aimed at maximizing effects (e.g. if I had unlimited resources to run this trial in order to cause effects, the duration and frequencies might have been different)

Nearly all filled in the survey after the 4th session. The turnaround time on getting a completed survey ranged between 2 days and 2-3 weeks, depending on the person's responsiveness. A more rigorous trial would probably be more hardcore about when final feedback surveys are issued and completed. I didn't feel that I was in a position to draw hard lines on when these leaders submitted the surveys. 

What do you think the ideal coaching frequency is for people in this reference class? I.e., every week, every other week, once per month? (Assume that we'll have unlimited supply of high quality coaches).

Short answer is that fortnightly (once every two weeks) seems to be the sweet spot for fairly busy leaders undertaking complex roles. But the frequency we end up going with is unique to the individual and varies according to a constellation of things – a non-exhaustive list includes what their goals are/what the subject matter is, how inclined they are to test out new actions and outlooks and the time horizons on those feedback loops, how inclined they are to take time to pause and reflect (ie have they taken the time to think through what they felt was important to think through), their mental space and general availability, personal financial situation, etc. I'm sure their pre-existing models of what they need to work on and how long it will take to bring those things to a good place also plays an important factor. 

One of the main rooms for improvement (from my perspective) might be if the evaluation had been conducted by a third person and I'll probably see if I can find someone like this if/when I do a trial myself. Do you have any thoughts or reactions to that?

Good point – I flirted with this idea and I'm still quite interested in doing this. My primary hesitation is that I'd be concerned about off the bat about whether there's enough epistemic alignment on the 'metrics' that are chosen, and furthermore what the implications of certain metrics are. (For example, if someone over-engineered the quantitative metrics and anchored too hard in the importance of them, the results could be pretty damaging to how people look at your practice in a way that doesn't seem justified to me.)

Anticipating epistemic idiosyncrasies in the wide variety of readers out there, I personally chose a variety of metrics that would likely resonate in different ways with different people. I was shooting for producing a collage of valuations that cut across different paradigms. 

Following from that, I think it would actually be cool to have sections of a single unified evaluation designed by different people that measure along different paradigms. 

SebastianSchmidt @ 2022-04-05T09:44 (+9)

Thanks so much for this. Super inspiring that you decided to do something as rigorous and transparent as this and thus contributing beyond your own coaching practice.

I find the results mindblowing - especially as it's on people who are steering the trajectory of one of the more promising (youth) communities of our time.  Even just the simple fact that people with so high opportunity cost on their time want to continue says a lot. In other words, if this is as good as it seems, one should prioritize providing this kind of coaching (or something similarly valuable) to all leaders within EA. 

Tee @ 2022-04-07T09:58 (+4)

In other words, if this is as good as it seems, one should prioritize providing this kind of coaching (or something similarly valuable) to all leaders within EA. 

I wouldn't disagree with this! Another way to say this, even if it's half as good as it seems, like if you slashed all of the metrics by which I calculated value here by 50% (e.g. quantitative monetary evaluations, productivity, # of people who had a notably good experience, # and quality of testimonials, # of people who continued on in a paid arrangement after the trial) it's still worth devoting far more attention and resources to this from within the community. 

SebastianSchmidt @ 2022-04-11T15:52 (+2)

Yes, I agree.

SebastianSchmidt @ 2022-04-05T09:45 (+8)

The bigger coaching landscape:

  1. I appreciate your emphasis on leaders and can clearly see how someone like you would be particularly well-positioned to doing this. However, I think it's worth high-lighting that there are other promising "audiences" who can benefit massively from coaching even though they're currently less impactful. In particular, I'm thinking about underutilized talent (there's so much out there!) as they have much lower opportunity cost on their time (and can more diligently carry out the actions and reflections) and are more flexible about certain worldviews, strategies, etc..
    1. A simple toymodel to describe this with tentative ballpark numbers (sorry for the numbers. We're all intrinsically valuable people capable of living our purpose for the greater good - I hope no one will be offended here):
      1. EA leaders might have an expected lifetime impact of 100 points and great medium-term coaching might increase their lifetime impact by 30% [5%-60%]. 100*1.3 (coaching effect)=130. I.e., 30 counter-factual impact points.
      2. Underutilized talent might currently have an expected lifetime impact of 5 points but can experience astonishing effects from great medium-term coaching and associated "growth time". Concretely, it might increase by 250% [50%-500%]. 5*3.5 =17.5. I.e., 12.5 additional impact points which is the counter-factual.
    2. These numbers are very rough and could easily be off by a factor of 5 in either direction but they do seem kind of reasonable to my first-hand experiences although it's probably more likely than not that they'll be overestimates. However, they broadly show that focusing on underutilized talent might be similarly promising to focusing on leaders although a group of coaches should seek to serve various groups.
    3. What do you think about this - in particular the numbers that I brought forth?
  2. How much unmet (known as well as unknown) demand do you think there currently are within the community? I.e., given the eagerness of the participants (as well as my personal experience) I'm inclined to think that virtually all EA leaders should have a coach.
Tee @ 2022-04-07T12:22 (+8)

it's worth high-lighting that there are other promising "audiences" who can benefit massively from coaching even though they're currently less impactful.


Couldn't agree more. There were a set of strategic and tactical reasons why I felt it would be more compelling to make the case with leaders first. It seemed to me like a more straightforward way to cleanly demonstrate value in multiple ways. Others might disagree. Curious about your take. 

As an example, in the case where a broader community-talent-enrichment-focused project needs to receive funding support, you first need funders to be able to approximate and properly appreciate the value of coaching. 

This is quite a bit more difficult if you're trying to project the value of future talent + (likely) getting lower valuations of the value of the coaching because early-career people have a different relationship to money & growth, not to mention the anecdotal anchoring that often obfuscates these decisions for funders that would likely work against trying to fundraise for a project like this. 

You'd basically need tastemakers and purse-holders to have the willingness to evaluate this, ability to evaluate this, come to agree with this reasoning, and/or have high in-network-derived trust of an individual, in order to have a shot at doing that. Also curious if you have a take on that. 

> What do you think about this - in particular the numbers that I brought forth? 

It seems flatly clear to me that investing in the development of individuals at earlier inflection points would be extraordinarily valuable. Add-em-up methods of approximating value is not my strong suit unfortunately (nor my preferred method of approximating things in certain contexts), so I probably don't have much to say on specifics of your BOTEC there

> How much unmet (known as well as unknown) demand do you think there currently are within the community? I.e., given the eagerness of the participants (as well as my personal experience) I'm inclined to think that virtually all EA leaders should have a coach.

I mention here that, strictly speaking, I don't know for sure. I'd say that certainly there's far more demand for professionalized support than we've clocked, and there's far more developmental needs that individuals have 'at the top' than people realize. Being nitpicky, I'm not so sure that ambient demand or perceived need for coaching is a perfect proxy for whom, how much, and how useful it would be, though it does tend to be that those who recognize that they need help are much more easily helped.