Many EA orgs say they place a lot of financial value on their previous hire. What does that mean, if anything? And why aren't they hiring faster?

By 80000_Hours @ 2018-10-12T18:55 (+24)

In our 2017 and 2018 surveys of EA organisation leaders, the median respondent reported being willing to forego very large amounts of additional donations to hold on to their recent hires - several hundred thousand dollars for a junior employee, and millions for a senior one. 

However, commenters on this forum and elsewhere have pointed to two observations that seem to be in tension with these figures:

How can this make sense?

One possibility is that the survey responses are wrong. We list weaknesses with the methodology in the article about the survey. But there are other possibilities.

Below we offer a potential explanation of what is going on. In the process we hope to clarify that the answers to the above survey question are far from a decisive consideration in favour of working at those organisations.

Most importantly, we asked the organisations to value typical recent hires. So this is a an ex post estimate of the value of recent hires, rather than an ex ante estimate of the value of future hires.

Future hires need to be searched for, selected, trained and come to be trusted. These are large additional costs, since they consume senior staff time. Even if you do everything right, there’s a good chance that new hires won’t work out. The organisations also have a limited number of management slots in which to add new staff. Read more in this great post by GiveWell. 

These factors mean both that existing staff are very valuable, but the expected returns of hiring new staff may not be high - and in particular, may not beat the returns of other activities.

What does this mean for whether you should try to work at one of these organisations? If you’ve already got an offer (or could easily get one because they’re already familiar with your work) then working there is potentially very impactful. It also means that it’s valuable to find out if you could get one of these roles, so long as it’s not too expensive to learn. But it doesn’t necessarily imply very much at all about the likelihood of additional jobs opening up at these organisations, or the competitiveness of opportunities.

We propose that these jobs be seen as great roles to aim for, but with a high risk of not working out. If you think you might be a good fit, then look for cheap ways to test them out, but do this with a solid back-up plan, so you can easily switch to something else if it doesn’t work out. If you do get an offer, and you’re motivated to take it, then it’s likely a top option.

More broadly, we recommend using the process here to generate the best options for you. Ultimately we should all be aiming for the best thing we can personally do.

Below, we go into some more depth on the reasons for the above.

 

--

If an organisation’s previous hire is really worth hundreds of thousands of dollars, let alone millions, shouldn’t they be snapping up more staff members by the pound? Here are some possible reasons that they aren’t:

The upshot of the above is that:

We think our articles about the survey and talent gaps haven’t highlighted the above as much as they should.

The numbers in the survey are most relevant to someone who is offered a job at one of these projects, and is deciding whether to do that, or earn to give, or do direct work elsewhere. Added: Though even then, insofar as they are only part way through the discovery and training process, the numbers will still be inflated.

One way around the bottleneck described above is suggesting that our readers go work for larger established organisations which have the capacity to absorb many more people simultaneously, thanks to more existing staff or scaleable training processes. Examples of this could be government agencies, existing foundations, or some forms of earning to give. That is something we will be trying to do more of over the next year.

 

undefined @ 2018-10-14T08:47 (+29)

My hunch is (as implied elsewhere) 'talent-constraint' with 'talent' not further specified is apt to mislead. My impression for longtermist orgs (I understand from Peter and others this may apply less to orgs without this as the predominant focus) is there are two broad classes, which imperfectly line up with 'senior' versus 'junior'.

The 'senior' class probably does fit (commensensically understood) 'talent-constraint', in that orgs or the wider ecosystem want to take everyone who clears a given bar. Yet these bars are high even when conditioned on the already able cohort of (longtermist/)EAs. It might be things like 'ready to run a research group', 'can manage operations for an org' (cf. Tara's and Tanya's podcasts), 'subject matter expertise/ability/track record'.

One common feature is that these people add little further load on current (limited) management capacity, either because they are managing others or are already 'up to speed' to contribute themselves without extensive training or supervision. (Aside: I suspect this is a under-emphasised bonus of 'value-aligned operations staff' - their tacit knowledge of the community/mission/wider ecosystem may permit looser management than bringing on able professionals 'from outside'.) From the perspective of the archetypal 'pluripotent EA' a few years out from undergrad, these are skills which are hard to develop and harder to demonstrate.

More 'junior' roles are those where the criteria are broader (at least in terms of legible ones: 'what it takes' to be a good generalist researcher may be similarly rare to 'what it takes' to be a good technical AI safety researcher, but more can easily 'rule themselves out' of the latter than the former), where 'upskilling' is a major objective, or where there's expectation of extensive 'hands-on' management.

There might be similarly convex returns to getting a slightly better top candidate (e.g. 'excellent versus very good' might be 3x rather than 1.3x). Regardless, there will not be enough positions for all the talented candidates available: even if someone at an org decided to spend their time only managing and training junior staff (and haste considerations might lead them to spending more of their time doing work themselves than investing in the 'next generation'), they can't manage dozens at a time.

I think confusing these two broad classes is an easy way of burning a lot of good people (cf. Denise's remarks). If Alice the 23-year-old management consultant might reason on current messaging, "EA jobs are much better for the world than management consultancy, and they're after good people - I seem to fit the bill, so I should switch career into this". She might then forsake her promising early career for an unedifying and unsuccessful period as 'EA perennial applicant', ending up worse than she was at the start. EA has a vocational quality to it - key it does not become a siren song.

There seem a few ways to do this better, as alluded to in prior discussions here and elsewhere:

0) If I'm right, it'd be worth communicating the 'person spec' for cases where (common-sense) talent constraint applies, and where we really would absorb basically as many as we could get (e.g. "We want philosophers to contribute to GPR, and we're after people who either already have a publication record in this area, or have signals of 'superstar' ability even conditioned on philosophy academia. If this is you, please get in touch.").

1) Concurrently, it'd be worth publicising typical applicants:place or similar measures of competition for hiring rounds in more junior roles to allow applicants to be better calibrated/emphasise the importance of plan B. (e.g. "We have early-career roles for people thinking of working as GPR researchers, which serves the purpose of talent identification and development. We generally look for XYZ. Applications for this are extremely competitive (~12:1). Other good first steps for people who want to work in this field are these"). {MIRI's research fellows page does a lot of this well}.

2) It would be good for there to be further work addressed to avoiding 'EA underemployment', as I would guess growth in strong candidates for EA roles will outstrip intra-EA opportunities. Some possibilities:

2.1) There are some areas I'd want to add to the longtermist portfolio which might be broadened into useful niches for people with comparative advantage in them (macrohistory, productivity coaching and nearby versions, EA-relevant bits of psychology, etc.) I don't think these are 'easier' than the existing 'hot' areas, but they are hard in different ways, and so broaden opportunities.

2.2) Another option would be 'pre-caching human capital' into areas which are plausible candidates for becoming important as time goes on. I imagine something like international relations turning out to be crucial (or, contrariwise, relatively unimportant), but it seems better rather than waiting for this to be figured out for instead people to coordinate and invest themselves across the portfolio of plausible candidates. (Easier said than done from the first person perspective, as such a strategy potentially involves making an uncertain bet with many years of one's career, and if it turns out to be a bust ex post the good ex ante EV may not be complete consolation).

2.3) There seem a lot of stakeholders where it would be good for EAs to enter due to the second-order benefits even if their direct work is of limited direct relevance (e.g. having more EAs in tech companies looks promising to me, even if they aren't doing AI safety). (Again, not easy from the first person-perspective).

2.4) A lot of skills for more 'senior' roles can and have been attained outside of the EA community. Grad school is often a good idea for researchers, and professional/management aptitude is often a transferable skill. So some of the options above can be seen as a holding-pattern/bet hedging approach: they hopefully make one a stronger applicant for such roles, but in the meanwhile one is doing useful things (and also potentially earning to give, although I think this should be a minor consideration for longtermist EAs given the field is increasingly flush with cash).

If the framing is changed to something like, "These positions are very valuable, but very competitive - it is definitely worth you applying (as you in expectation increase the quality of the appointed candidate, and the returns of a slightly better candidate are very high), don't bet the farm (or quit the day job) on your application - and if you don't get in, here's things you could do to slant your career to have a bigger impact", I'd hope the burn risk falls dramatically: in many fields there are lots of competitive oversubscribed positions which don't impose huge costs to unsuccessful applicants.

undefined @ 2018-10-12T20:26 (+17)

Since this looks like it was written partially in response to me, I’d like to reply. First, I appreciate the clarification. It is very helpful and I definitely agree with most of it.

It strikes me that the actual problem here is one of messaging. By getting EA orgs to list very large figures for their hires and talk about talent gaps writ large, you risk misleading EAs into thinking that they should be focusing on applying or upskilling for these jobs in particular, when the actual value of doing so may be less than it appears (though still potentially large). It seems like it would be far more informative to ask EAs to place figures on future hires or discuss more detail about how exactly they feel constrained or bottlenecked.

Also, if EA orgs are better informed that there actually are a lot of talented applicants out there, perhaps these orgs may invest more in figuring out how to productively bring on more people.

undefined @ 2018-10-12T22:02 (+16)

I appreciate this follow up as well. Peter and I seem to be of similar thinking here and in his comment on the other post, but just to add:

I don’t think the misunderstanding stems as much from the recent-hire valuations, as from terms like “talent bottleneck” and “talent constrained”. Especially “talent constrained” used alongside “funding constrained”. I could be mistaken, but it would seem odd to say you’re “funding constrained” but can’t use more funding at the moment. Whereas we are saying orgs are “talent constrained” but can’t make use of available talent. They evidently don’t function quite the same, so phrasing them in this matched sort of way invites erroneous comparison.

Similarly, I feel a “talent bottleneck” implies an insufficient supply of talent/applicants, which doesn’t seem to be the case. I guess it’s more that there’s insufficient talent actually working on the problems, but it’s not a matter of supply, so it’s more of a “hiring bottleneck” or an “organizational capacity bottleneck”.

EA orgs aren't so much constrained by a lack of available talent as they are constrained by their capacity to deploy additional talent

It seems like it would be far more informative to ask EAs to place figures on future hires

I had the same thought that it might be more informative to know EA leaders answers to something like, “I’d rather have $X in additional donations than my next ideal hire.” Agree that it’s difficult to know how a future hire will work out, but maybe there’s still something to be learned from the value they’d place on an additional ideal hire, as it wouldn’t necessarily be the same as holding on to a recent successful hire. I’m admittedly out of my depth here.

undefined @ 2018-10-12T22:14 (+10)

We have a forthcoming post on whether the expressions 'talent gap/bottleneck/constraint' are generating more heat than light and should be phased out in favour of more specific terms.

undefined @ 2018-10-12T21:20 (+9)

Hey Peter - it's partly for you, but also many other people who have the same questions.

I can comment on the choice of question in the survey as I'm the one who wrote it.

The reason we went for an ex post assessment rather than an ex ante one was that we thought people would be able to more reliably assess how they feel about a previous hire today, than remember how they felt about a previous hire in the past.

Asking people to remember what they thought before runs the risk that they will substitute a hard question (what they thought months ago before they knew how someone would work out) for an easy question (what do they think given what they know now). Then we'd get a similar answer, but it would look ex ante when it actually isn't.

It also seemed quite difficult for organisations to forecast how much they’ll value a typical hire in the future because, among other reasons, it’s difficult to anticipate how successful future searches will be.

In retrospect I think the framing we chose was probably a mistake, because the two assessments are much more different than most readers understand them to be. I agree with your suggestions for improvements and, indeed, we concluded our blog post with our plans to ask new questions next year or else interview a smaller number of people in more depth.

Hopefully a different approach next year will help us avoid this confusion going forward.

As for the article being misleading, we've:

i) Commented that these roles are hard to fill at the point when these figures are first mentioned.

ii) Explained the ex post, ex ante distinction in the relevant section, and now added a link to this post.

iii) Noted we don't have much confidence in the answers to that question and would not recommend that people update very much based on it.

undefined @ 2018-10-12T23:06 (+14)

Thanks for making the ex ante versus ex post distinction. But it makes me confused about the penultimate paragraph; if I am offered an job at an org and am comparing to earning to give, shouldn’t I be using the (currently unpublicised) ex ante numbers, not these ex post numbers?

The risks of bad personal fit, costs of senior staff time, costs of fast hiring in general, and time taken to build trust are all still in the future at the point, and don’t apply to the earning to give alternative. As far as I can tell, the only cost which has already been sunk at that point is the cost of evaluating me as a candidate. In my experience of working on the recruiting side of a non-EA org, this is far smaller than the other costs outlined, in particular the costs of training and building trust. I’m curious if the EA orgs feel differently.

In general though, I don’t think attempting to save these numbers by pointing out how hiring is subtly much more expensive than you would think interacts much with my objection to these numbers, since each additional reason you give me for why hiring is wildly expensive for EA orgs is yet another reason to prefer earning to give, precisely because it does not impose those costs! All these reasons simply mean the numbers should be lower than what you get from an ex post phrasing of the question, at least insofar as they are being used for what appears to be their intended purpose, namely comparison of options.

undefined @ 2018-10-12T23:30 (+4)

I wrote 'most relevant' to contrast that group with people who know they'll never be able to do direct work at the organisations surveyed, for whom those figures are largely irrelevant. As you say, they're still not quite right for that group because you're only part way through the process (I think by the time you're offered a job, about half the costs have been paid, though that depends on whether you had a trial period).

They would be even more relevant to someone considering quitting - but as they can talk to their colleagues, they probably wouldn't be using this survey.

The costs of hiring makes applying to join a project look worse, but it also makes earning to give to fund the salaries of new staff look worse too. If you're expecting to donate hoping the money will be used to fund new hires, it seems like it should cancel out.

The three ways around bearing those costs that I can see at first glance are i) earning to give to prevent a group from laying off staff for lack of funding, ii) earning to give to buy capital goods rather than hire people, iii) staying in your current direct work job rather than switching.

undefined @ 2018-10-13T07:41 (+4)

Since most of the EA orgs in question are heavily constrained in hiring by whatever level of growth they can manage or feel comfortable with (that’s kinda the whole point of the OP, right?), it would not generally be my assumption that additional funds would be used for extra hiring compared to the counterfactual. I grant that if that is the assumption, these effects seem to cancel.

Other ways not listed in your last paragraph.

-earning to give to allow orgs to raise salaries

-earning to give to fund regranting

-earning to give to fund things like targeted advertising (you may have intended to cover this category in ‘capital goods’, I’m not sure)

These things are much closer to my model of where extra funding to at least CEA and 80k in at least the past 18 months has gone, not into additional hiring.

undefined @ 2018-10-13T18:13 (+1)

Hey Alex,

A quick note on another aspect that I think Rob might have underemphasised:

Hiring could generate very high returns to the organisations but often not as high as other marginal activities.

Here's a simple model: If you think that many EA orgs spend about $1m per year and generate $10m per year of "value" (as rough orders of magnitude), then even if a marginal hire generates $1m "excess" value to the org, that's only a 10% increase in impact for that year. This is good but there could easily be even more important things for senior management to focus on.

This would explain why the orgs don't hire many people.

However, it would still be consistent with the idea that marginal hires are valuable and can have more impact by working at the org than by earning to give, since each generates $1m.

It would also mean conditional on an org running a hiring process, it's better for that process to go as well as possible and get the best possible applicants.

undefined @ 2018-10-13T21:37 (+4)

In that simple model, it appears to me that marginal hires are worth $1m minus the counterfactual use of senior staff time (hypothetically, and relevantly, suppose all the possible hires for a position decided to earn to give instead overnight. It would not be the case that the world was $1m worse off, because now senior staff are freed up to do other things). If there are in fact even more important things for senior management to focus on, this would be a negative number.

More realistically, we could assume orgs are prioritising marginal hiring correctly relative to their other activities (not obvious to me, but a reasonable outside view without delving into the org particulars I think), in which case the value of a marginal additional hire would simply be ~0.

So again, I appreciate the attempt to boil down the area of disagreement to the essentials, and even very largely agree with the essential models and descriptions you and Rob are using as they happen to match my experience working and recruiting for a different talent-constrained organisation, but feel like this kind of response is missing the point I'm trying to make, namely that these ex post numbers, even if accurate on their own terms, are not particularly relevant for comparison of options.

undefined @ 2018-10-18T20:45 (+1)

Hey Alex,

I agree with the thrust of your point: if hiring is very costly, then that's reason against working at EA orgs vs etg.

It still seems like there could be a world in which hiring is high return but often (but not always) lower return than some other marginal activities. This will mean both that (i) recent hires are valuable (ii) the orgs don't hire many people (iii) hiring is sometimes worthwhile. I was just trying to show how these could be consistent.

Going back to the overall issue of why the dollar figures are high, I don't think the "hiring is costly" point is the main thing going on, and think this post might have emphasised that too much. The post was more trying to answer the question of why the orgs don't hire more often despite the high dollar figures. The broader question of why the figures are high and what to do based on them is discussed briefly in the original article about the survey, but hasn't yet been tackled in depth.

With your first two paragraphs, I just want to step back and point out that things get pretty confusing when you include all opportunity costs. When you do that, the return of every action except the single best action is zero or negative. Being close to zero is actually good. It's probably less confusing to think in terms of a ranked list of actions that senior staff could take.

I also expect the orgs partially take account of the opportunity costs of staff time when reporting the dollar value figures, though it's hard to be sure. This is why next year we'll focus on in-depth interviews to better understand the figures rather than repeating the survey.

undefined @ 2018-10-19T08:21 (+10)

With your first two paragraphs, I just want to step back and point out that things get pretty confusing when you include all opportunity costs. When you do that, the return of every action except the single best action is zero or negative. Being close to zero is actually good. It's probably less confusing to think in terms of a ranked list of actions that senior staff could take.

True, but I haven't accounted for all the opportunity costs, just one of them, namely the 'senior staff time' opportunity cost. If you are in fact close to 0 after that cost alone (i.e. being in a situation where a new hire would use x time and generate $1m, but an alternative org-improvement action that could be taken internally would generate $950k), that isn't 'good', it's awful, because one of those actions incurs opportunity costs on the applicant side (namely, and at the risk of beating a dead horse, the cost of not earning to give), but the other does not.

So we could look at this as a ranked list of potential senior staff actions, but to do so usefully the ranking needs to be determined by numbers that account for all the costs and benefits to the wider world and only exclude senior staff time (i.e. use $1m minus opportunity cost to applicant minus salary minus financial cost of hiring process per successful hire etc.), not this gross $1m number.

Similarly, potential applicants to EA orgs making a ranked list of their options should include all costs and benefits that aren't tied to them, i.e. they should subtract senior staff time from the $1m number, if that hasn't been done for them already. Which is what I've in fact been recommending people do. But my experience is that people who haven't been directly involved in hiring during their career radically underestimate the cost of hiring, and many applicants fall into that camp, so for them to take account of this is not trivial. I mean, it's not trivial for the orgs either, but I do think it is relatively easier.

I also expect the orgs partially take account of the opportunity costs of staff time when reporting the dollar value figures, though it's hard to be sure. This is why next year we'll focus on in-depth interviews to better understand the figures rather than repeating the survey.

Given this conversation, I'm pretty skeptical of that? My experience with talking to EA org leaders is that if I beat this horse pretty hard they back down on inflated numbers or add caveats, but otherwise they say things that sound very like 'However, it would still be consistent with the idea that marginal hires are valuable and can have more impact by working at the org than by earning to give, since each generates $1m.', a statement it sounds like we now both agree is false or at least apt to mislead people who could earn to give for slightly less than $1m into thinking they should switch when they shouldn't.

For the benefit of third parties reading this thread, I have had conversations in this area with Ben and other org leaders in the past, and I actually think Ben thinks about these issues more clearly than almost anybody else I've spoken to. So the above paragraph should not be read as a criticism of him personally, rather a statement that 'if you can slip up and get this wrong, everybody else is definitely getting this wrong, and I speculate that you might be projecting a bit when you state they are getting it right'. The only thing Ben personally has done here is been kind enough to put the model in writing where I can more-easily poke holes in it.

undefined @ 2018-10-13T10:03 (+13)

The narrative that "EA is talent constrained" has had, as mentioned, some negative and unintended consequences. One more I'd like to add is that on this narrative the advised action "go work for a larger organization, like the government" might feel for many people as failure or "not living up to their goals." I'm afraid this leads to more value drift - people's values siding away from EA-aligned values - because they feel they're not "living EA" anyway.

undefined @ 2018-10-18T07:46 (+25)

The problem here is that people in the EA movement overtly associate being EA not with 'doing high-impact things' but with 'do EA-approved work, ideally at an EA org'.

It is not obvious to me how this is fixable. It doesn't help that recommendations change frequently, so that entering paths that were 'EA-approved' once aren't any longer. As Greg said, people won't want to risk that. It's unfortunate that we punish people for following previous recommendations. This also doesn't exactly incentivize people to follow current recommendations and leads to EAs being flakey, which is bad for long-term impact.

I think one thing that would be good for people is to have a better professional & do-gooding network outside of EA. If you are considering entering a profession, you can find dedicated people there and coordinate. You can also find other do-gooding communities. In both cases you can bring the moral motivations and the empirical standards to other aligned people.

undefined @ 2018-10-20T17:11 (+2)

... is to have a better professional & do-gooding network outside of EA.

+1

I've been pretty impressed by the level of professional support & general benevolence of people who work in non-EA domains.

(Most of my experience here is with the Bay Area tech scene & the payments antifraud scene; unsure how far this generalizes.)

undefined @ 2018-10-23T00:12 (+10)

This makes me feel more strongly that there should be a separate career advice organization focused on near term causes. (See here for my original comment proposing this idea.)

A near term career advice organization could do the following:

I think the proposed organization would actually complement 80,000 Hours by expanding the number of cause areas for which there's in-depth career advice and coaching; the two organizations could even establish a partnership where they refer people to each other as appropriate.

(As noted in my original comment, I think it's better to have a separate organization do this since a long-term focused organization understandably wants to focus its efforts on causes that are effective from its perspective.)

This approach could have various benefits including:

undefined @ 2018-10-25T00:41 (+3)

80,000 Hours would likely be supportive of another organisation specialising in global health or factory farming career advising. I'd prefer to divide up by problem areas, rather than long vs. short term. We plan to write more about this.

undefined @ 2018-10-25T01:51 (+5)

That would be great, Ben!

Worth also noting that The Charity Entrepreneurship Program seems like a great way to get involved with creating a new charity in global health or factory farming.

undefined @ 2018-10-13T06:49 (+8)

Recruiting is a thing and an EA org with the funds should be willing to consult with the best in the world if it's a one time cost to do so and you can do judgmental bootstrapping on their models and techniques.

undefined @ 2018-10-18T20:27 (+7)

Just FYI we're doing this at 80k as part of our new headhunting project, and have also sought advice on hiring in the past from people like the partners at YC.

undefined @ 2018-10-20T17:09 (+6)

+1 to this.

My general sense is that EA orgs should put more effort into ascertaining what current best-in-class practice is in the different domains they operate in, before they begin building out their own methodology. (weakly held)

undefined @ 2018-10-13T09:57 (+6)

A first thought I have is "if the current organizations cannot grow fast enough to use all the available talent, why not increase the growth rate of the movement?" That is, should we start more (paid) projects?

I think there is a case to be made that we should: there seem to be many small projects with capable people seeking funding. They are not as effective per employee as some of the core organizations, but the core organizations don't need the money. I don't hold this opinion strongly, and I'm curious what other people think.

undefined @ 2018-10-13T15:49 (+4)

Is it fair to summarize the thesis as: there is heaps of super-valuable talent out there but the main reason we can't cash it is that it can't be absorbed into existing managerial structures?

If so, then shouldn't we be advocating aggressively for absorbing the talent through greater funding of and more infrastructure for new EA orgs and EA contractor roles?

undefined @ 2018-10-13T19:01 (+1)

It's worth noting in the previous 80k post that many orgs consider hiring skilled managers to be a top priority. So I don't think it's that this isn't happening, just... well, it takes time.

undefined @ 2018-10-13T18:23 (+1)

No, I don't think it's that simple. I think it's sometimes true that the main issue is absorbing talent, but there are also situations when the top hire is much better than the second best, so generates a large excess value. This is because many of these roles require a very unusual skill-set.

I think the message to take away is more like "it's hard to infer what to do based on the survey figures".

Personally, I still think it would be very useful to find more talented people and for more people to consider applying to these roles; we just need to bear in mind that these roles require a very unusual skill-set, so people should always have a good back-up plan.

undefined @ 2018-10-13T22:37 (+13)

Personally, I still think it would be very useful to find more talented people and for more people to consider applying to these roles; we just need to bear in mind that these roles require a very unusual skill-set, so people should always have a good back-up plan.

I'm curious what your model of the % value increase in the top hire is when you, say, double current hiring pools. It needs to be high enough to offset the burnt value from people's investments in those application processes. This is not only expensive for individual applicants in the moment, but also carries the long term risk of demotivating people - and thereby having a counterfactually smaller hiring pool in future years.

EA seems to be already at the point where lots of applicants are frustrated and might value drift, thereby dropping out of the hiring pool. I am not keen on making this situation worse. It might cause permanent harm.

Do you agree there's a trade-off here? If so, I'm not sure whether our disagreement comes from different assessments of value increases in the top hire or burnt value in the hiring pool.

undefined @ 2018-10-14T18:14 (+4)

I just want to note that not every rejected application has been burnt value for me and most have actually been positive, especially in terms of things learned. In the ones I got far it has resulted in more rather than less motivation. In the case I had to do work-related tasks (write research proposal, or execute a sample of typical research) I learned a lot.

On the other hand, increasing the applicants:hired-ratio would mostly increase the proportion of people not getting far in the application process which is where least of the value positive factors are and most of the negative.

undefined @ 2018-10-14T20:47 (+2)

Oh I agree people will often learn useful things during application processes. I just think the opportunity cost can be very high, especially when processes take months and people have to wait to figure out whether they got into their top options. I also think those costs are especially high in the top applicants - they have to invest the most and might learn the most useful things, but they also lose the most due to higher opportunity costs.

And as you said, people who get filtered out early lose less time and other resources on application processes. But they might still feel negatively about it, especially given the messaging. Maybe their equally rejected friends feel just as bad, which in the future could dissuade other friends who might be potential top hires to even try.

undefined @ 2018-10-14T23:15 (+3)

Personally, I still think it would be very useful to find more talented people and for more people to consider applying to these roles (Ben)

It needs to be high enough to offset the burnt value from people's investments in those application processes. (Denise)

In theory, there's no real conflict between these two statements. Doubling the number of people who would consider applying for a post shouldn't impose a major cost on those potential applicants. At the same time, we could take steps to make it clearer to potential applicants what exactly the hiring criteria is, so we're not wasting people's time.

In fact, I think it would probably be ideal if we increased the number of people who consider applying for each EA job but decreased the number actually applying!

undefined @ 2018-10-18T20:25 (+2)

I agree there's a tradeoff. We're pretty unsure about how much to encourage people towards these roles at the margin.

We've seen cases of people on the other side who said they didn't apply for these roles since they assumed they were too competitive, but were found later and ended up doing really well. We've seen lots of cases of people who ended up getting the job but said they almost didn't apply for the same reason. I suspect we still miss a lot of great applicants.

To avoid this, we could encourage more people to apply, but that will result in more people getting demotivated when they don't succeed. Finding the ideal balancing point seems really hard.

I do agree with Khorton below, though, that we should try to find more "win-win" approaches, such as encouraging people to consider and find out if there might be a good role for them, and providing people with clearer criteria.

undefined @ 2018-10-14T08:59 (+1)

we just need to bear in mind that these roles require a very unusual skill-set, so people should always have a good back-up plan

Hmm, if EA work is valuable but the selection bar excludes most EAs, that could actually mean some/many of the following:

  • many people should just have a different plan A.
  • we need to get much better at selecting the best talent
  • we need to recruit much more selectively
  • EAs should have stronger backup plans
undefined @ 2018-10-13T07:17 (+4)

It seems like the reasons listed why organizations who value talent highly aren't hiring center on a growth constraint that can't be alleviated by money or talent. If there is this growth constraint, then doesn't it just mean we should focus elsewhere, i.e. by doing activities independent of these organizations? It seems like if organizations have slightly more room for talent than money, but ultimately little room for either, then their relative preference between the two shouldn't matter much, no?

Jon_Behar @ 2019-05-10T18:42 (+3)

I think this survey question is too hard to interpret to provide actionable information, and I’ve argued we should replace it with some alternative questions:

As 80K puts it, “Skill bottlenecks are a matter of degree” and these degrees can vary significantly depending on the specific skills in question. But we have little hard data to quantify skill gaps across various areas.
I suggest adding new questions to the next talent survey of EA organizations to help capture some of these nuances.[9] These questions will hopefully make it easier to answer questions like: Are EA organizations talent constrained? If so, which sorts of organizations and which sorts of talent? How large are these constraints? What can be done about them?
New questions (which should continue past surveys’ practice of asking the same questions about both a junior and senior hire and anonymizing organizational responses due to the sensitivity of the information involved):
● Generally speaking, how easy/difficult do you currently find it to fill roles at your organization? (Scale of 1 = Very difficult to 5 = Very easy)
● What do you pay current employees relative to what they could earn on open market, including jobs in the for-profit sector? (Multiple choice: More; about the same; 0-10% less; 11-20% less, etc)
● What do you pay current employees relative to what they could otherwise earn in the nonprofit sector (including all nonprofits not just EA organizations)? (Multiple choice: More; about the same; 0-10% less; 11-20% less, etc)
● What do you pay current employees relative to what they could otherwise earn at another EA organization? (Multiple choice: More; about the same; 0-10% less; 11-20% less, etc)
● How much do you plan to offer future hires relative to current employees in similar roles? (Multiple choice: More; Less; About the same. If someone answers “more”: Do you plan to increase salaries for existing employees? What factors into this decision?[10]
● Would any of the following steps be helpful in closing your organization’s talent gaps? (Rate the following options on a Scale of 1 =Not at all helpful to 5 = Extremely helpful):
○ Increasing salaries
○ Investing in recruiting
○ Increased publicity of position
○ Better access to interested candidates
○ Other (please describe)
This information is relatively easy to collect and interpret, corresponds to a real-world decision organizations make (how much to pay), is grounded in observable data (market wages), and is comparable across roles, organizations, geographies, and causes.[11] To mitigate the cost of data collection, I suggest abandoning survey questions about non-traditional labor metrics that seem to be producing noisy data and to generally be causing confusion.[12] If we collect compensation data and respondents indicate which types of roles they’re thinking about as junior and senior (which will vary across organizations), we’ll have a rich and actionable picture how EA compensation stacks up to the competition for various types of skills.