Is the community short of software engineers after all?

By Benjamin_Todd @ 2016-09-23T11:53 (+13)

It often seems like software engineering is the most over-represented career in the community. On this ground, at 80,000 Hours we've discouraged more people from going into the area, in order to increase the diversity of skills in the community. 

However, recently the following organisations have been trying to hire EA-aligned software engineers:

And I don't think any of these groups have found it particularly easy. 

Might this mean we're actually short of software engineers after all? It's a bit hard to tell at this point, but if these positions continue to be unfilled, then it'll look that way.

If we are short of engineers, what's the explanation? Some ideas:

 

Are you an engineer with over 2yr experience who's involved in effective altruism, and interested in switching to direct work? Get in touch with these organisations.

 

undefined @ 2016-09-24T02:08 (+21)

For those who don't know, I work as a data scientist / software engineer, have more than two years experience, identify as an EA, and donate a considerable portion of my income (right now living on ~$40K and donating ~$50K).

This post resonates with me because I often get annoyed that my earning to give job has little to none direct social impact and takes a lot of time away from my volunteering that I would otherwise like to do. However, when considering these direct work jobs, I usually end up not applying for some of the following reasons, in order from most feared to least feared:

(a) For personal reasons, I need to work in Chicago right now, but none of the other organizations are in Chicago. I would definitely consider remote work, but that sounds like it would make me really lonely as I have few friends outside of work and my girlfriend is long distance. My thought is that someone else who can live in the Bay would be a better fit.

(b) I'm concerned the jobs won't be technically challenging enough. My perception is that these jobs often involve maintaining WordPress sites or chaining Google Sheets together and don't involve making great technology. This motivation of mine exists alongside my EA motivations and is what keeps me interested in my for-profit job, but I'm afraid I'd lose it in a direct work job and I wouldn't be fulfilled from EA drive alone. My thought is that someone else who is less experienced or not motivated by technical challenge would be a better fit. I'd definitely be interested in any EA org that had a strong need for data science though, whether it be in generating predictions, creating product recommendations, classifying objects, etc.

(c) I'm concerned I don't have the relevant skills. These jobs seem very front-end focused or focused on making mobile apps, and my experience is in data science and back-end engineering. I'm not good at designing apps and would think that someone else more skilled at front-end design would be a better fit.

(d) I'm concerned these jobs don't offer sufficient salary. While I definitely identify with EA and want to donate a lot of money, I'm not super into sacrifice and would like to do things that, in Chicago, require $40K or more. I'd also like to save to eventually raise a family, buy a house, send kids to college, etc. My thought is that someone who can take a lower salary would be a better fit.

(e) I think that doing for-profit work will build better career capital that could launch an even more impactful career in the future, perhaps in tech entrepreneurship.

I don't know if (a)-(e) are actually true, but they're fears that keep me from exploring much further. I also think (a)-(e) is also based in the thought that there are many other EA software engineers who could easily be a better fit because of these limitations, but maybe that's not true? I didn't know any EA org other than 80K was struggling to hire software engineers, so that definitely updates me in that direction.

undefined @ 2016-09-24T11:04 (+8)

These concerns seem similar to other EA engineers we talked to about working at 80k. In particular, lots are more back-end focused rather than front-end, and many were worried about career capital.

It might be one of those cases where lots of people think they're not ideally suited to the role for reasons like this, but actually everyone else is reasoning in the same way. In reality, someone who doesn't seem well suited in absolute terms is relatively best positioned to do it. http://www.benkuhn.net/advantage

I think all these roles pay more than $40k, several more like $70, and Wave pays market rates.

I think some of the roles will be technically interesting. 80k isn't "building new technology" and mainly uses wordpress, but there's a lot of design challenges, and, based on our trials, it still seems very challenging to do everything quickly to a high level.

I think the career capital concerns might be overblown as I've written about before. https://80000hours.org/2015/11/working-at-effective-altruist-organisations-good-or-bad-for-career-capital/

undefined @ 2016-09-25T16:13 (+2)

Additional reason that applies to me and probably other EA engineers: Earning to Give lets your impact be more liquid and therefore better directed.

E2G lets you donate money to whichever organization in whichever cause area you think is best. Signing on to work at CEA means you think (impact at CEA) + (donating ?5-15k to Best Charity) is better than (donating 30-60k to Best Charity).

If you think CEA (or New Incentives, or Wave or whatever) is The Most Optimal Charity, easy decision. But it's not clear why the math would work out if you think X-risk, animal charities, or basic science is the right cause area... or even if you're into global health/poverty but think GiveWell is better at charity evaluation than you are.

undefined @ 2016-10-10T14:28 (+1)

This by the way is what certificates of impact are for, although it's not a practical suggestion right now because it's only been implemented at the toy level.

The idea is to create a system where your comparative advantage, in terms of knowledge and skills, is decoupled from your value system. Two people can be working for whichever org best needs their skills, even though the other best matches their values, and agree to swap impact with each other. (As well as the much more complex versions of that setup that would occur in real life).

undefined @ 2016-09-24T16:55 (+2)

In response to b, I think that's true for the 80k job. I decided not to apply for the 80k job because it was WordPress, which is horrible to work with and bad for career capital as a developer. Other developers I spoke to about it felt similarly.

But this isn't true of all of the jobs.

For example, the GiveDirectly advert says "GiveDirectly is looking for a full-stack developer who is ready to own, develop, and refine a broad portfolio of products, ranging from mobile and web applications to backend data integrations. As GiveDirectly’s only full-time technologist they will be responsible for developing solutions to the organization's most challenging technical problems, and owning the resolution from end to end."

When I unsuccessfully applied to Wave it similarly sounded like a standard backend web development job, not WordPress or tying together google sheets.

undefined @ 2016-09-24T17:25 (+6)

Just to clarify to future 80k engineers who are reading this - the current site is in WordPress so the job would involve some WordPress work, but we expect much work in the future won't be in Wordpress e.g. our career quiz has an angular JS front-end; if we build our own career tool then it would likely be in ruby on rails on a sub-domain.

undefined @ 2016-09-24T18:37 (+1)

I suppose (c) is a more valid concern than (b) then.

undefined @ 2016-09-24T04:31 (+7)

You need to consider base rates to be able to claim that there aren't enough EA software developers. Hiring good developers is hard, but hiring good anyone is hard. The question is, is it disproportionately hard relative to hiring for other jobs, or to software engineer hiring for non-EA companies?

Also relevant: I've looked at the job postings for some of the orgs you listed, and they generally want several years of experience. EAs are disproportionately in college or just out of college, which makes it harder to find experienced candidates.

undefined @ 2016-09-24T00:03 (+6)

Would hiring non-EA programmers be better because the counterfactuals are lower? It doesn't seem obvious to me that value-alignment of software engineers is as (or more) important as experience. On the other hand, it could build career capital for the EA developers (but their counterfactual job might be better for career capital, depending on what it is).

undefined @ 2016-10-03T20:54 (+2)

Value-alignment of the team is super important, so non-EA team members are risky, less motivated, and usually don't work out.

Also EA engineers are going to stick around in the movement, so if you train them up and get to know them the community will benefit in the future, whereas non-EAs will go elsewhere.

Finally, because you have to pay the non-EAs more, I think much of the benefits get used up.

Consider: Option 1: Hire a non-EA at $100k per year, who generates $200k of value -> surplus of $100k. If they hadn't worked for you, they would have donated $0.

OR

Option 2: Hire an EA at $50k per year, who generates $200k of value -> surplus of $150k. But, if they hadn't worked for you, they would have earned $100k and donated $50k. So the net surplus is $150k - $50k = $100k.

The surplus is the same in both cases.

undefined @ 2016-10-04T05:16 (+1)

Value-alignment of the team is super important, so non-EA team members are risky, less motivated, and usually don't work out.

I thought this comment on Michael's post was interesting. If it really is best practice for nonprofits to ignore how committed potential employees are and just hire on competence, it seems like we should take that pretty seriously.

BTW, it might be a mistake to conflate "value-aligned" with "currently thinks of themselves as an EA". For example, if you advertise an EA job on a regular old job board, and make it clear that you want to hire someone who's passionate about your cause area and you're offering a below-market salary, it seems like anyone you found through that process would have at least some degree of value alignment (assuming that they could be making substantially more money working for a less altruistic employer).

Also EA engineers are going to stick around in the movement, so if you train them up and get to know them the community will benefit in the future, whereas non-EAs will go elsewhere.

I'd think that the opposite logic applies--EA engineers are likely to stick around in the movement regardless of whether you hire them, whereas non-EA engineers might choose to become more involved through their experience working for you.

It seems like the value of information is really high here. Feels like at least one of the 5 organizations you mention should experiment with hiring non-EA engineers, if the talent crunch really is as bad as you say. Unfortunately I'm not that excited about working for any of the 5 organizations you mention, but I'd be happy to help them interview engineers to get an idea of how competent they are. (I've got some experience doing this.)

undefined @ 2016-10-09T11:37 (+4)

(I'm one of the founders of Wave.) Thanks for mentioning us!

We have indeed found it tricky to hire engineers. Our EA pedigree makes it easier though, since lots of people want to make an impact with their career; as does being a full distributed team so we can hire people from anywhere in the world. We're exclusively hiring via our networks right now and the EA community has been quite helpful for us. Most of the engineers we've hired had heard of EA before joining, although I think not all. I don't think many of them identify as explicitly EA (though a few do, myself included).

When we've rejected engineering candidates from the EA community, it's been mainly for technical reasons (not enough experience or velocity doing the things we do), followed by cultural homogeneity risk. To expand on the latter risk: we often reject people who seem "too EA" because we are afraid of importing too much cultural baggage from an existing community - it would feel outgroupish if you didn't know much about EA but there were all these EA memes flying around the slack channel. So far we've mostly avoided that.

That said, I don't think anyone should self-exclude from our hiring process for being too EA; we can decide that for ourselves. Get in touch if you might be interested.

undefined @ 2016-09-23T13:40 (+3)

I can’t seem to find CEA’s job posting. Does someone have a link?

(I’m mostly ETGing as software engineer at the moment because EAF is currently more more funding constrained, so I can better support it that way. I’ll be geographically bound to Berlin for a while longer.)

Edit: Oh, and a six years experience in software development, in case someone is keeping track of the numbers.

undefined @ 2016-09-23T21:25 (+1)

It's not formally advertised yet - it's the least pressing of those roles.

maximumpeaches @ 2021-06-01T22:39 (+2)

One factor nobody has mentioned is the lack of communication between these organizations and software engineers. On Reddit I see posts all the time with titles like "are there any orgs where I can have a meaningful career?", especially in the /r/cscareerquestions and /r/experienceddevs sub-forums. The people creating these posts have never heard of 80,000 hours or even the term "effective altruism".

I agree with other comments about how jobs might not match with programmer's desires for work that creates career capital (i.e. uses modern tech stack not wordpress), or low pay, or not being in their city. All of these are valid reasons, but I think they are only valid for the people considering the roles. There are a lot of people who are never considering these roles because they have no idea they exist. The other thing is that low pay, etc. is hard to fix, but getting the word out to more people about impactful jobs is easy.

I know in college I spent a lot of time searching job boards, trying to find a meaningful software engineering job. I never found one. Years later I stumble upon 80,000 hours (and a few other ways to find meaningful jobs) but the internet is a big place, and finding 80,000 hours is like finding a needle in a haystack in some ways.

(Another thing to consider is that 80,000 hours recently added the job board? I'm not sure what was there before the job board, but the website as a whole would've helped much less if it didn't have a job board. Currently the 80,000 hours website is kinda like "you probably shouldn't be a software engineer unless you went to a top 10 school and then here are some great AI safety roles..." but the job board does highlight opportunities for the non-1% of software engineers.)

I think overall there is a small proportion of software engineers who care about EA and the like, and so connecting them with the also small proportion of engineer jobs that enable EA is kind of statistically unlikely.

Anyways there are probably ways to make the jobs at 80,000 hours easier to access. I have some personal plans on my small scope, and maybe 80,000 hours can do I dunno more SEO or something.

undefined @ 2016-09-28T09:02 (+2)

FYI I applied to New Incentives with ~10 yrs experience on the 1st of September. Haven't heard back.

undefined @ 2016-10-03T20:56 (+3)

Startups value fit with the team very highly, so it's always very hard to predict who they'll be interested in. Programming skill / experience is only one component.

undefined @ 2016-09-23T20:47 (+2)

What %age of market rate are these jobs offering? I think 'people who are willing to sacrifice (e.g.) 30% of their potential salary are pretty rare independent of field.

undefined @ 2016-10-09T11:45 (+3)

At Wave, we offer market-ish for a startup ($90-180k for engineers depending on experience, plus equity). That said, several of the engineers we've hired took a substantial pay cut in nominal terms (usually from Google) to join us. Hopefully the equity will equalize or exceed the cut over time, but that's always a risk. That said, I'm skeptical of anyone who says that an engineer "needs" a 150k salary to be happy; the marginal value of each extra $10k seems to drop dramatically around $70k and maybe more like around $100-120k if you have a family.

undefined @ 2016-09-24T16:59 (+1)

Apart from 80k, do you know if the other organisations have had few applicants to these jobs or lots of applicants but no-one good enough?