Should you do an economics PhD (or master's)?

By david_reinstein @ 2022-04-19T20:04 (+62)

David Reinstein: all opinions are mine unless noted. Extensive input from Phil Trammell and an Anonymous Contributor (quoted extensively, henceforth “AC”). Thanks to Pete Wildeford and David Moss for feedback. I intend to continue to update and improve this post in situ (or linking out a ‘permanently updated' version).

Overview and some takeaways

Should I do an economics PhD (or master's)'? What do I need to learn to work at an EA org? How can I level up on this stuff and prove value?

These were the most frequent questions I got at the 2022 EAGx Boston conference. I mainly discussed this with undergraduate students, but also with people at career pivot points.^[1]

My overall view, epistemic basis/confidence, key points

Main ‘pros’: Much of EA is based in economics, and economics speaks to most of the important cause areas and debates in EA, as well as to the important empirical questions. Conditional on going for a PhD, I believe economics will be one of the stronger choices for the sort of people reading this post. A PhD in economics, and much of the associated training (over ~2 years of coursework and ~3-5 years of ‘writing’) helps you towards a range of career paths with potential for strong impact (and a comfortable life) both within and outside EA organizations. Being a PhD student in the right place and time (and mental state) can be very stimulating, productive, creative, and connection-building.^[2] You are typically given a lot of freedom in the research phase, as long as your work meets the general approval and framework of your advisor(s) and what the gatekeepers think is important, credible and ‘is economics’. Typically, you don’t have to pay for a PhD, you will get money to support yourself, and PhD stipends are often OK.

Important considerations:

• Economics is broad (in its methods and focus-area paths). Often differences in approaches among economists (pure theory, applied econometrics, macro, etc.) are greater than the difference between some economists and some (e.g.) political scientists or psychologists.
• Important impact paths that an economics PhD may help with include:
  • Applied work ‘informed by expertise and credibility',
  • Deep work formally/mathematically addressing fundamental questions of global priorities and social welfare,
  • Theoretical, computational, and empirical work considering markets and/or the global economy, informing (e.g.) animal welfare policies or the development of technology,
  • Empirical work assessing the impact of interventions, or considering assessing human behavior, choices, attitudes and preferences.
• There are a range of relevant career paths

1. Academia and academically-leaning think-tanks;
   1. doing EA-relevant research
   2. as well as potentially transforming academia and the scholarly debate ‘from the inside’,
2. Working in governments or NGOs (many require/prefer PhDs),
3. Working at EA-aligned organizations like Rethink Priorities, Global Priorities Institute, Open Philanthropy, maybe MIRI … note a lot of differences across these,
4. For-profit and entrepreneurial options; possibly impactful for out-of-the-bun thinkers/doers.

Main ‘cons and caveats’:

• For many/most paths you could learn most/all of the relevant skills and approaches, and background without getting a PhD,^[3]

• In some key areas economics might not be as strong or relevant as other fields (statistics and data science for robust empirical work and predictions, decision science and cognitive science for AI alignment work),

• The economics PhD program makes you jump some time-consuming hoops that are likely not going to be relevant to your applied career path,^[4]

• People around you will not mainly be value-aligned; beware value drift towards academic prestige and publications as a goal in itself (sometimes also towards earning money). Remember: your goals may not be your advisor’s/the professors' goals. And what you (and we in EA) think is important may not be what the field has traditionally found is important (or what they think others think).

TLDR on ‘Master's programs’: Some of these can be good (especially the ones tailored towards research) but many are cash grabs and ‘barely worth the paper they are printed on’. Caveat emptor; look at the outcomes, talk to the people who’ve done it recently. The ‘research targeted’ ones can be better.

Epistemic Basis/background

Moderately confident. My views below are very much shaped by my experience and; some aspects may be dated or shade towards my own particular journey (Berkeley, UK, etc). However, I have asked around a bit, especially on the (EA) Economics Grad School Advice Slack.

Background – David Reinstein (davidreinstein.org): My advice is based on my own background and perspectives. I am 45 years old.^[5] I went to Berkeley for my PhD, and I switched between topics, fields, and advisors a few times. After this I worked in the UK for 15 years as a ‘lecturer’ and ‘senior lecturer’ … roughly equivalent to Assistant/Associate Professor.^[6] In 2020 I took a job as a Senior Economist at Rethink Priorities.^[7]

Background - Anonymous Contributor: I am in my late 20s, but have not yet done an Econ PhD nor have I worked in the non-profit or gov sector. I recently finished (what I consider to be) a high-quality Econ Master's in the Washington DC area, and decided to pursue a PhD path partway through that. My advice is colored by my exposure to policymakers, and my relatively old age to be starting an Econ PhD. Both factors mean I value personal fit more and prestige less, compared to other aspiring economists. This is because policy work is medium- or low-prestige among academic economists, and also because I don’t want to spend too much more time building signals. (When young, it may be useful to spend time doing more predocs, taking more math classes, or reapplying to PhDs in a future year.)

Sidebar: The 80k advice is good as a first pass. However, I think more detail is needed

Why is the 80K advice not the end of story?

• It’s 7 years old. Some things have changed. My impression:
  • There are now more competitive requirements for admission (‘pre-docs’),
  • and better non-academic options (including at EA orgs) for people with econ/data/quant like skills, perhaps;
    • this cuts both ways because these are better options for people with PhDs but also the credential seems less necessary
• It’s largely very general conventional wisdom
  • e.g., it doesn’t specifically consider cause areas (animal welfare, AI alignment, cause prioritization and EA meta etc) or skills and types of work (data and statistics vs real analysis and economic theory)
  • It seems not very informed by insider perspectives.
• Some things seem a bit off … e.g.,
  • 1 out of 5 rating for earnings seems too low… in the short run, maybe, but in the medium the earnings are certainly pretty decent compared to many other paths.
• The “50% chance of going into academia” seems too high to me. Maybe half of people end up doing something involving some teaching/university affiliation, but probably not substantial research. Even at the top-10 PhD programs I’ve heard that less than half go into academia. And many people go into academia briefly but don’t get tenure/permanency.^[8]

Your Mileage May Vary: What are your goals? What proximate ways do you hope to achieve them? What do you like/are good at doing?

The ‘best advice’ (and the optimal choices) depends on...

• What are you trying to do with your life and career? What are your impact goals? (E.g., ‘reduce existential risk’)
• The ways you want to achieve those goals, through what activities and approaches (e.g., ‘by helping design better legislation in the US and internationally’)?
• What position you are in, and what are your skills, aptitudes, and things you want to build? (E.g., writing, pure math, coding and data, analytical argumentation, social skills)
• What are your opportunities (have you been admitted, do you have job experience, etc.)?

To give advice, you need to consider how to answer …^[9]

What causes and areas do you care about?

Areas of interest (Listed roughly in declining order of ‘how interested were the people who talked to me at EAGx’):

1. AI risk: fundamental/alignment research, AI governance
2. “Broad longtermism” and specific LT risks
3. Global health etc.
4. Improving institutional decision-making
5. Fundamental and theoretical EA epistemic and meta-issues (determining ‘what is of value, how can we know this/learn about this, how to consider moral uncertainty…’ GPI and philosophy-leaning stuff )
6. Outreach, Messaging, Community building, Political and social change
7. Animal welfare (maybe tied with the above)

Activities and aptitudes

How do you want to spend your time? What do you want to ‘get good at’?

1. “Pure research”:

   • Research to ‘understand the world’ or ‘understand deep theoretical puzzles’ for their own sake
   • Methodological work (esp. econometrics and some parts of micro theory) to learn about the world for its own sake and to provide tools for other economists
   • Pure math and theoretical modeling of optimization and decision theory, strategic interaction, markets and aggregation
   • EA/GPI-esque applications of the above, interacting with analytic philosophy (how to measure and consider value and welfare functions, etc.)

2. Applied economic theory:

   • How markets adjust in ‘partial equilibrium’, prices, etc., … often engaging with data. This ‘unglamorous’ field is highly relevant to animal welfare considerations
   • Macroeconomics, including growth; understanding banks, finance, and financial transactions
   • Considering specific policies and market failures
   • Mechanism design … e.g., applied to real-world auctions, prediction markets, public goods provision, etc.
   • Game theory: the ‘lessons and logic’ can be applied to the real-world^[10]

3. General ‘applied econometrics’:

   • Econometric and statistics/data-sci work where you don’t necessarily formally derive the properties of your estimators and don't come up with new estimators
   • Maybe the bulk of what economists do … relevant to global health and development, animal welfare, perhaps various sources of existential risk, analysis of institutions

4. Designing and running experiments and RCTs

   • Statistics (power calculations etc)
   • Considering confounds and psychological motivators
   • Microeconomics of incentives and mechanisms
   • “Making things happen online”
   • Making things happen in developing countries and with in-person interventions

5. ‘Informed literature review’ … assess and summarize, informed by economic ‘lessons and principles’, and by an understanding of empirical methods

   • Clear writing and communications
   • Reasoning transparency and analytical rigor

6. Applied ‘impact work’; sometimes used in policy econ academia; very relevant at EA orgs like RP

   • MonteCarlo Fermi ‘guesstimation’ (for CBA etc)
   • Practical forecasting
   • Finance, accounting, actuary-adjacent stuff (maybe various discounting and risk-adjustment calculations, cost of capital, dynamic optimization (?), asset valuation/CAPM etc)

My impression/memory of the “standard” Economics (academic, careerist, status/signaling seeking, Risk-averse, and perhaps backwards-looking) advice

Note: This ‘standard advice’ is not my advice; I agree with some points and disagree with others^[11]

The standard advice (perhaps geared for top students) I’ve heard is something like...

• Go to the top program you can get into

• Keep trying until you get into a top 5-10 program*^[12]

  • A non-top program (below 25 ranked or so) is probably not worth going to
  • But it may still be worth doing if you are not trying to get an academic job
  • But “no matter where you go, you will need to be one of the top students in the program to get an academic job.” – AC

• Don’t worry about your research interests now as these will change

• Take as much real analysis as you can; this will help you ace the microeconomics (theory) course and impress the faculty (professors)

  • But don’t do theory as your field, choose an empirical and applied field like health economics, where ‘the market is good in this field’

• Finding ‘an interesting question’, a ‘novel’ data set, and a clever ‘identification strategy’ are the keys to research success (i.e., a ‘top job market paper and publication’)

• Work as hard as you can to do exactly what your supervisor thinks is best … (but don’t bother them too much)

• “Just write” and, as you move towards your dissertation, try to present and submit your work to a top journal.

  • Never submit your work to a journal below the top-50

• Sacrifice everything to prepare for the job market and don’t take any risks. Apply to all the universities that might be interested in you, but don’t customize your applications.

• Apply to research-focused academic jobs, at universities and university-equivalent institutes and think tank (the Federal Reserve, World Bank)

Note: Some of this advice is good, but I don’t agree with all of this advice for reasons I’ll try to cover… and it may be somewhat bad advice for people interested in certain EA career paths.

Why that ‘baseline advice’ may not (and probably is not?) relevant to you, and may be risky

• Your goals are different

• Value drift, especially in academia … ‘getting top papers’ becomes the goal

• They may give backwards-looking advice … advice that worked in past and recent years but may not be good going forward. They may be unaware of the new world of opportunities, including in EA

• Much of the economics curriculum may be irrelevant to your interests (Real analysis? Macro? General equilibrium?) …

  • Trammell: I think this is true for basically any economist, not just one getting a PhD for EA reasons; everyone specializes in something.^[13]

• Economics tends to be conservative; they may not sign off on your research areas/interests.

  • They want you to ‘publish in top journals’ and become a professor

• In economics/academia, ‘top papers’ need to

  • ‘Flex’ to show how clever you are^[14]

  • Show relevance to existing/prior models and paradigms… the ‘deep parameters of economics and human behavior’

  • Perhaps* “motivate” its importance to existing (often US government) policy priorities and non-EA-aligned goals (like improving pension systems in rich countries)

    • *Trammell: “My strong impression is that the oft-lamented tension here doesn't really exist at all, at least on the current margin.”^[15]

The no-brainer cases: Who should ‘definitely go for a PhD’

1. I’m a professor-type

If…

• If you want to do ‘pure research’ → you almost definitely need to go into academia → you need a PhD
• If you want to teach economics at university … (same)
  • Remember that the teaching part of an economics PhD is, in part, training you for the above

2. I’m interested in impact, but the PhD is giving me everything I want …

If...

• Your PhD admission comes with ok/decent funding, with few strings attached,^[16]

  • Trammell: Even if it comes with no funding, it might be worth getting EA funding to do it instead of getting EA funding to do similar research outside a program... the latter at least comes with a credential at the end, and support from an advisor.

• The taught component seems relevant to you

  • usually real analysis-heavy microeconomics (the ‘hoop to jump through’), macro, econometrics, maybe econ. history in the first year …
  • choose 2-4 fields in the second year

• There are likely to be several professors you can work with… ideally willing to mentor you, generous with their time, and _willing to support and sign off on your intended research. _

  • AC: rare to know all of these before admission, especially since it's common for faculty to ignore you until you're in PhD Year 3. 3+ professors in fields/ methodologies / values you like is a good rule-of-thumb. 2 senior tenured professors may also work.”^[17] If either A or B hold, and you think you can do it → it’s probably worth getting the PhD.

Advice by field, interest; “case studies”

AI alignment research

I speculate that some of the methods of theoretical economics are not so relevant to important standard economics questions but actually highly relevant to AI alignment! (DR speculation; PT and AC have broadly similar thoughts)

Decision theory, axiomatic specifications of preference and choice: These were useful in considering the basic ‘existence proof’ for things like ‘could we think of a person as an optimizer’ and ‘when can markets yield optimal social outcomes’. But people don't act so rigidly, there are few ‘very general results’ (e.g., even ‘demand curves slope down’ is only under a bunch of conditions, and only applies to the theoretical Hicksian demand). A small amount of complexity can reverse important results derived from simple specifications

But computers _do _ behave rigidly, and they ‘do just what we tell them’. So it's very important to know the implications of what we mean when we are stating a set of preferences, and what choices this could lead to. We need to define it rigorously; otherwise computers will find ‘edge cases and loopholes’ and may make a lot of paperclips.

Game theory and mechanism design also seem to take on a new life when we consider the deep issues of alignment and developing adversarial systems to keep other computers in check.

This seems a very ripe area for research. But doing an econ PhD with this target comes with some caveats:

1. Much of the economics curriculum you will have to cover in the first 1-2 years (and maybe engage with for the run in your teaching and seminars) will be largely irrelevant to this target
2. It may be hard to find advisors and people supportive of this research within the profession and within your department. I know that some exist, but they are not completely willing to ‘come out’ in public.

Global health and development

AC:

The case for doing a PhD is strongest in the global health and development (GH&D) field. This is an area where you have numerous employers (J-PAL, World Bank, IMF, maybe Treasury or World Trade Organization for trade issues etc.) who respect an econ PhD, heavily reward (or even require) it for certain senior roles, and have lots of openings.

This is (probably) a good fit for empirically-minded folks and people who care about policy relevance. Theory is relatively de-emphasized in dev econ fields, and systemic theories of change are viewed with intense suspicion. A PhD is not necessarily the fastest track to gaining status or seniority, but it is useful for certain technical policy making roles

(DR) Still, even within GH&D, there are particular skills and focuses that are probably more relevant for EAs, and some institutions and researchers who are more sympathetic to these approaches than others. You can work at EA orgs like Rethink Priorities without necessarily having a PhD in this area. There are collection of key relevant skills and experience that partially overlap a GH&D PhD. These include data and statistical acumen, an understanding of impact evaluation, measures of value and social welfare (QALY/DALY etc.), clearly synthesizing literature with reasoning transparency, digesting technical work in other fields (especially medicine), the ability to derive practical quantitative insights, and Fermi/MonteCarlo guestimation.

At other EA organizations, particularly charities doing direct work (Lead Exposure Elimination Project etc.), the “working on interventions in the field” part of much GH&D PhD and RA work may also be relevant.

General long-termism, meta-EA, cause prioritization, deep philosophical issues in EA

You might be engaging in a range of issues and concerns of ‘what is important and why’ and ‘how do frameworks compare’ and ‘how should we expect to know things’. A broad background in economics may be important here, as well as some background in philosophy and other fields. But potentially, very strong undergraduate engagement could suffice.

The deep theory: For the majority of ‘GPI type stuff’ you will probably need technical depth in economic theory (e.g., microeconomics, preferences and choices, constrained optimization, risk and time preferences, market failures, welfare economics and social welfare functions, public choice, game theory, mechanism design, growth theory, finance theory) … or perhaps in philosophy or cognitive science or decision science or another adjacent field.

(As I mentioned for AI alignment/risk) I think that some of the most formal aspects of economic theory (such as ‘preference axioms’) take on new relevance when we consider deep and thorny issues of social preferences and welfare functions. E.g.,

• Non-utilitarian preference relations like 'lexicographic' ... seems particularly relevant to population ethics (it's OK to have 'non-continuity', perhaps)
• ‘Incomplete preferences’ and ‘non-transitive preferences’ seem bizarre when applied to individuals, but arise naturally in discussing social choices and acceptable social rules, and their implications

Animal welfare

I think academic economists often see ‘actually modeling particular economies and markets’ as prosaic and unsexy, unlikely to get those hot and juicy top publications. Empirical work in these areas is seen as ‘something industry people do.’ Applied theoretical results about systems of production, supply, and demand are seen as ‘fully explored and old-school’.

But I see an important new role for these questions, and this modeling, in considering the impact of specific policy changes on animal welfare. There may be situations in which ‘if we consider the sum of producer and consumer surplus, little can be said, and markets can be expected to work well’.

However, if we consider the externality of (various measures) of animal suffering and well-being, there may be a range of new results and approaches, and practical findings. Much of this may adapt methods used in environmental economics (which also takes externalities seriously), but other work may tread new paths. The interaction of inputs and supply and demand in multiple animal product (and agricultural) markets seems distinct.

(General and partial equilibrium models, and Implications of GE models for animal welfare interventions; a new set of value measurements and possible interventions; not just the 'market failures' approach.)

What to focus on if you are interested in animal welfare? I’m not sure. David Rhys-Bernard’s syllabus has some interesting articles in this section. But as I highlight above, I suspect that modeling markets, perhaps agricultural economics as a field, is particularly relevant. Work involving ‘consumer behavior (maybe IO and marketing) and behavioral economics fields also seem important, particularly in considering interventions to reduce animal-product consumption. (Further afield, there may also be some role for economists as behavioral/decision scientists in considering, measuring, and quantifying measures of sentience and capacity for pain and pleasure, to potentially adapt to moral weights.)

For this target area, finding an aligned program may be particularly important. I don’t think there are many Economists who do, or are interested in this stuff, although there are signs the field is growing. Jayson Lusk’s name comes up a lot, and I know a few others, such as Josh Tasoff, who have work/interests in this area. This upcoming Stanford conference seems particularly promising.

Do you need a PhD to work in EA aligned Animal Welfare research?

Probably not, but it could help. There seems to be a shortage of quantitative and data skill, or ability and willingness to make calibrated guesstimates in this area (see discussions of Animal Charity Evaluators’ decision making). However, this may reflect the deep underlying uncertainty about interventions themselves. The economist’s skills are indeed valued in this area (e.g., see Rethink Priorities recent hiring and job advertisements).

A Master's degree: sometimes a scam, sometimes worth it?

There are skills to learn and getting feedback is important.

But many MSc programs are money-spinners; they will admit anyone who has cash, even if they don’t speak the language, know little economics or math, etc. They often provide little value to students, and all students who can regurgitate a few bullet points will graduate with the degree. I (David Reinstein) know this from experience.

Even in these suboptimal programs, top students who proactively engage with the academic staff might be able to get a lot of attention and value.^[18] But on the whole, I would say it tends not to be worth it. It might be better to enroll in a PhD program and then consider dropping out after you complete the master's program part of it.^[19]

Trammell: Agreed that the big difference seems to be whether it's a "dissertation-prep" master's (often called an MRes) or not. For any EAs interested in the MPhil at Oxford, I can testify that it really is basically the first two years of a typical US econ PhD. The first year is maybe the same as what you'd get at Brown but with ~20% less material.

AC: Agree with Trammell. That said, at least in the US, these dissertation-prep ,aster’s are highly competitive, difficult, and expensive. Schools will charge a premium since attendees typically have an inflexible demand for attending a top 10 program. Think carefully about whether this is the best use of your time. If you’re good enough to get into one of these programs, you may be good enough to get a Predoctoral Research Assistant role instead. That would make you money, give you access to letters of recommendation, and give you some exploration value.

Exception to this exception, if you’re from a marginalized group or a developing country, you may qualify for some scholarships and PhD-prep programs designed for under-represented minorities. These are few and far between, but they are out there.

In more detail (AC)

Cautionary note: Even if a program had a reputation for being good, they may no longer be good or as good as they used to be. The incentive for a master's program is always to exploit reputation for profit. Quality can decrease fast, sometimes between years if the university is losing money. Best sign is asking someone who goes to the program today what they think about it, and also ask if they're heard any rumors about planned declines in standards. (More adjuncts, bigger class sizes, etc.)

DR: You might expect universities to want to maintain quality to preserve their reputation in the long term. But, working against that… 1. Many administrators are given short term incentives, 2. Master's programs often seem to be sustained income-generators in the long term, as a sort of degree mill. Universities are happy to let them cross-subsidize other things. They may expect that “it’s well known that universities run Master's for cash, so it shouldn’t affect the reputation of a university as a whole.” Perhaps the governments (and employers?) sending students to this even know the degree is of low value… but they use the credential as a way of signaling to outsiders, or maintaining a power status quo.

Upside note: Scrutinize the placements carefully. This is your next-best sign of program quality. Most programs won't publish anything. But even if they do, be skeptical. Say the program places 10 people into Great Organization. Is that 10 people per year? Is that 10 people across the entire history of the program? Would these 10 people have counterfactually gotten into the Great Organization already? Nearly all Master's programs report selectively, but some are more manipulative of their reports than others. Sometimes, universities will claim credit for placing someone who already works at the United Nations into the United Nations. Even if the placements seem legitimate, ask yourself if it's realistic for the median student to accomplish. If not, would you expect yourself to be a top-quartile / top 10% student?

Rough rule-of-thumb: If the university website shares employment outcomes and tuition info in a detailed and accessible way, then that's a promising sign. If you have to cross-reference multiple webpages and track down fine print to get this information, then that's a worrying sign.

Other paths to EA-economics-adjacent research; building skills and credibility

There is a wealth of online and written material to learn all of the aptitudes described above. There are also great Slack groups and support communities and the modern world enables all sorts of ways we can organize collaborative learning and knowledge bases.

(But see the caveat below about credentials.)

In fact, I think that in many areas you will do better by working through interactive web books and their coding examples and exercise than you would from a program involving sitting in lecture theaters. It should not be difficult to replicate the training you get in the first year or two of a PhD economics program through self-study, if you have the "fuel”, and some feedback. (In fact, I think we could and should develop a better curriculum for this that is more relevant to EA or the specific EA-linked fields described above. Some stabs at this here (see links within).)

What you need is:

1. Motivation and drive, supported by
   • The confidence that you are going in the right direction and what you are learning is valuable
   • Social support
2. Some feedback and setting you on the right path, particularly when you get stuck on certain questions
3. The opportunity to demonstrate what you have learned in a way that employers will believe you

The first of these could come through our great EA networks. The second is a bit challenging, but you could consider things like hiring a PhD student as a tutor, or engaging directly with online textbooks open (helping professors build these while getting your questions answered). And again the EA network can probably help you with this. There are a few EA fellowships and training things; more could and should be built to focus on more technical economics and related training.

For the third “proof of value” part, I suggest

1. Helping build online web material at the nexus of EA issues and advanced and intermediate econometric and methods … This could include online textbooks (esp. ‘Bookdowns’), Wiki resources, StackExchange, and perhaps some parts of LessWrong and the EA Forum (if we work to make these more rigorous)
2. Open peer-review and feedback on economics research
3. (Obviously) Doing research and assisting on research projects, with people at EA organizations, with academics, and on your own initiative. Effective Thesis may be able to help coordinate this, hopefully even for people not enrolled in a degree program

Credentialism caveat:

• AC: I would caveat that this only gets you the skill and not the credential. For traditional roles, particularly in government and some think tanks, the credential is important in and of itself. For EA roles and Silicon Valley and places that value self-learning, this is less important.
• DR: I agree; also in academia itself, obviously. Slight caveat to the caveat, is that the world is changing. If credentials are no longer an important signal of value, we might expect institutions to adjust to no longer require the same credentials.

Notes