Introducing Effective Self-Help
By Ben Williamson @ 2022-01-06T13:11 (+111)
Effective Self-Help (ESH) is a pilot initiative researching the most effective ways people can improve their wellbeing and productivity.
This post explains what Effective Self-Help is, what I hope the project can achieve, and why I believe this work may be highly impactful. As this is a long post, I suggest reading the Summary section below and then skipping to whatever sections seem of greatest interest.
Based on rough Fermi calculations made using Guesstimate, I estimate a mean impact per Effective Self-Help article of 180 hours of high-impact work and 4.2 QALYs. However, these numbers are very uncertain and best taken as an indication that ESH could be highly impactful and cost-effective, rather than a signal that it will be.
Thank you to the many people who have discussed ideas, critiqued drafts, and provided invaluable insights in recent weeks. This project has received initial funding from the EA Infrastructure Fund.
Summary
Effective Self-Help is currently an ongoing series of articles on relevant topics posted to the EA Forum. I intend to develop the project into a home for the best guidance on how individuals can improve their wellbeing and productivity. The project is currently solely the work of me, Ben Williamson, with feedback, comments, and ideas from lots of other lovely people involved in EA.
In the What is Effective Self-Help? section, I discuss how I think ESH can be unique as a self-help resource by combining several key components:
- Prioritisation: Highlighting the most effective interventions and how effective they are, rather than just listing interventions that may be effective.
- Research Quality: Providing more rigorous and well-rounded advice that combines the findings of scientific literature with more Bayesian forms of evidence.
- Presentation: Maximising engagement through curated recommendations and the use of digital media (video, audio, and interactive programmes).
- Breadth: Covering topics across the full breadth of wellbeing and productivity, rather than just specific domains of expertise.
- Practicality: A focus on enabling behaviour change by highlighting practical actions and tailoring content to maximise the success of habit adoption.
- Positive mental health: Looking at how to improve the wellbeing of people who are not struggling with mental health issues as well as those who are.
- Free Access: Avoiding any influence of paywalls, advertising, or affiliate marketing on the quality of advice provided and full access to advice.
Under Why Effective Self-Help could be highly impactful, I envisage three main routes to impact which are discussed in greater depth in the ‘Theory of Change’ sections. These are:
- Increasing the productivity of people doing high-impact work
- Helping a large audience make small improvements to their wellbeing
- Attracting new people to the EA community through an interest in self-improvement.
I created Guesstimate models for the first two of these routes to impact, producing best-guess estimates that ESH could add 180 hours of productive work and 4.2 QALYs per article. However, these results are highly uncertain and best interpreted as an indication that ESH has the potential to be highly impactful if things go well, rather than a strong endorsement that it will be.
Downside risks and arguments for ESH being ineffective discusses a range of ways in which Effective Self-Help could be of limited value or of active harm. The main downside risks discussed are that ESH could:
- Give advice that is of net harm by badly accounting for risks and side-effects
- Serve as a weak introduction to EA
- Make the EA community more insular.
The main ways listed for how that Effective Self-Help could be of limited effectiveness are:
- Struggling to make useful recommendations due to the low quality of evidence on many topics
- Limited popularity due to most readers preferring narrative-based self-help advice
- Individual differences in benefit washing out general differences in effectiveness
- Greater cost-effectiveness for interventions targeting other groups/ causes
- Finding that articles are a poor method of presenting recommendations compared to other formats.
In the Final notes & next steps section, I discuss current my current uncertainties with the project, including whether to focus on wellbeing or productivity exclusively, whether to prioritise interventions based on cost-effectiveness or only effect size, and how best to select which topics to research.
I also discuss the work I’m currently doing for Effective Self-Help and my vision for the project moving forward. This includes a brief outline of how I intend to expand the project in the coming months, including potentially hiring people to work on it with me. If you would be interested in helping with the project, the ‘Get Involved’ sub-section near the end explains more about what roles I am potentially looking to fill.
What is Effective Self-Help?
What is it currently?
An in-progress series of research articles on self-help topics.
Currently, ESH is an ongoing series of articles researching the most effective actions anyone can take to improve their wellbeing and productivity. So far, I’ve posted articles on Sleep: effective ways to improve it and Stress: practical interventions to reduce its impact on wellbeing, as well as a directory of podcast episodes on mental health topics.
Effective Self-Help aims to be comprehensive, gradually covering all aspects of improving wellbeing and productivity that are highly effective. Future articles are planned and/or underway for a wide range of topics, including:
- Maximising the effectiveness of behaviour change and habit formation
- Cost-effective ways to purchase greater productivity and wellbeing
- Increasing measures of subjective wellbeing
- Techniques for improving relationships with friends, family, and partners
- Improving attention
- Comparing the effectiveness of different meditation and mindfulness apps
- Key diet do’s and don’ts
- A cost-effectiveness analysis for bright light therapy
- An assessment of the most useful supplements and nootropics
What do I want it to become?
A one-stop shop for learning how to live better.
The big idea of Effective Self-Help is to provide comprehensive guidance on how best to improve your wellbeing, researching interventions across the full spectrum of wellbeing and productivity topics. My primary aim is to create a one-stop shop for self-help advice, helping people and organisations maximise the effectiveness of efforts to improve their wellbeing and/or productivity.
Topics will expand over time to include any area of wellbeing and productivity that appears important, from improving relationships to the relative effectiveness of different supplements. With a backbone of research articles, ESH will aim to present this information in the most useful and engaging format possible, such as through videos, podcasts, or interactive programmes.
I believe that high-quality self-help research can be highly impactful and of great benefit to both the EA community and the wider public. Within EA, curated guidance and training could be provided as part of EA retreats, intro fellowships, local group initiatives, and EA organisations’ staff wellbeing policies. For the general public, engaging and accessible web resources could produce incremental improvements to thousands of people’s wellbeing and productivity while attracting new people to the EA community.
What makes Effective Self-Help different?
Self-help is a huge industry. Additionally, even just within the EA community, there are already good projects and articles on self-help. What makes ESH different and valuable?
Below are the 7 key components that I believe make this project new and worthwhile. While there are existing projects[1] that meet some of these criteria, I am unaware of any that meet most or all. On that basis, I think Effective Self-Help can offer something distinctive and highly valuable, slowly outcompeting lower-quality projects by providing better, more trustworthy guidance in a superior format for free.
Prioritisation (Comparative ranking)
Many self-help articles provide lists of actions to take to improve a certain aspect of mental health and wellbeing. However, these articles tend to lack any transparent methodology for how actions have been selected, which might be the most effective, and by what margin.
In line with one of the cornerstone findings of EA, that some charities are >100x as effective than others, it seems likely to assume that some self-help interventions are >10x as effective than others for a given individual.
This makes the prioritisation and ranking of actions essential for directing people to the most effective improvements to their wellbeing and productivity. Considering the time constraints of many people doing high-impact work, prioritisation appears even more vital.
Research Quality (Rigour)
Projects like the EA Mental Health Navigator, and many of the popular articles written on mental health on the Forum, are compilations of existing resources and recommendations for different topics.
Effective Self-Help aims to produce better recommendations than are currently available through deeper reviews of the available scientific literature, a more consistent and rigorous research framework, and the use of a Bayesian approach to evidence and value.
In particular, current self-help literature often focuses on the statistical significance of study results when the focus should be directed to effect sizes. Much like hits-based giving, it is potentially far more valuable to pursue an intervention that could be very high impact but with uncertain evidence in its favour than something that we are near certain has only a small impact.
Effective Self-Help will rank interventions based on their average effect size, directing attention to how much something will help rather than just that it is of some help, and highlight the most effective interventions.
Though I am still uncertain of the best method for doing so, I intend for ESH’s recommendations to also account for the costs (e.g. time, money, risks) involved in making different changes and for ESH to use a wider basis of evidence (with appropriate weighting) than just available scientific studies to determine what may be most effective.
Presentation (Curation)
A substantial proportion of popular self-help guidance is in written form, either in books or on blogs. While much of this is useful, I think there is a general lack of curation. Recommendations are often buried partway through the text and clear, actionable advice can be difficult to find.
By curating recommendations and presenting them prominently, ESH could cause a much higher percentage of its readers to make changes based on the recommendations presented than the average self-help resource. The articles on sleep and stress provide good examples of trying to present key takeaways in this kind of quick and obvious way.
While written advice can be effective in presenting a clear argument, I think much greater benefit would come from the clear presentation of key conclusions and arguments in more engaging formats. I envisage the greatest benefit from presenting the research findings in non-text formats.
Videos, podcasts, online courses, and interactive programmes all could plausibly make the changes we recommend more engaging to learn about, easier to understand, and therefore easier to implement. Organisations like Clearer Thinking, The School of Life, and Waking Up provide good examples of efforts to provide valuable content in more engaging formats.
Breadth (Comprehensiveness)
While there is a staggering range of self-help literature, there are very few projects looking to cover all aspects of useful life advice/ guidance. I have started Effective Self-Help as a relatively limited research project to test its effectiveness, only covering a few relevant areas, but the long-term aim is to provide comprehensive self-help research, covering all plausibly impactful topics and interventions.
A singular location for high-quality guidance could save people substantial time, ending the need to trawl through resources and establish what is actually useful. It could also plausibly help direct people to the most effective recommendations based on their circumstances, with people finding the website through an interest in a certain topic but then discovering interventions that are also or even more useful for them.
Having a quick, screening quiz for prioritising recommendations prominently placed on the website could achieve this quite efficiently.
Practicality (Focused on enabling action)
As discussed above, a lot of good self-help advice is presented in a blog format. While this can be good for valuably changing the way we think about a topic, I think the greatest benefit comes from providing people with specific actions and changes they can make.
A focus on practical action helps to reduce the percentage of readers who will finish an article and agree with its conclusions but fail to make any beneficial changes in behaviour based on its ideas.
As part of this, particular emphasis will be given to providing support for implementing new behaviours and researching how to maximise people’s success in adopting new habits. Knowing what to do to improve something is not enough; ESH must ensure that people are best supported to follow through with effective action.
Positive Mental Health (Self-Help)
Existing projects in EA like the EA Mental Health Navigator primarily target members of the community who currently suffer from poor mental health. Effective Self-Help would be more unique as a resource by plausibly providing support relevant to everyone in the community.
Effective Self-Help aims to produce small changes in wellbeing and productivity that add up to a large effect for people who may already have relatively high wellbeing and productivity. In this way, the project roughly emulates the philosophy of marginal gains in elite sport, made famous by Dave Brailsford’s work for British Cycling and Team Sky.
Free Access (Paywall-free)
Current self-help resources can be influenced by a desire to sell a certain product, course, or book. The best advice, presented in the best way, is saved for those willing to pay for it as its producers depend on these revenue streams for their income. Even with the best of intentions, I think this financial incentive can compromise the quality of advice given, at least to an extent, such as when:
- A worse product is recommended because a company pays to sponsor it.
- Less time and effort is spent developing the free resources provided because the quality of paid resources has a bigger impact on the writer’s income.
- Clickbait titles and writing styles are used to increase audience growth and engagement at the expense of providing clear, high-quality advice.
- Email sign-ups are placed above helpful advice.
As a moonshot goal, the project would become the default location for wellbeing and productivity advice, continually updating guidance on the most effective interventions based on new research.
Why Effective Self-Help could be highly impactful
A general case
Self-help is something we all do all the time. Many of our everyday decisions - from what we eat, read and watch, to how we communicate and why we maintain certain habits - are motivated by a belief that these choices will improve our quality of life.
But if the general quality of information available to us is low, we are likely making sub-optimal decisions. With better knowledge, we could improve the choices we already make regarding how to maximise our wellbeing. Great self-help advice highlights actions we take that have a significant impact on our wellbeing but, with conscious thought and small changes, could be made substantially more beneficial.
Interest in better mental health resources for the EA community seems relatively high. 55% of respondents to the 2018 EA Mental Health survey agreed or strongly agreed that ‘offering mental health resources to its members’ is an effective use of the EA Community’s resources (vs. 12% who disagreed or strongly disagreed). This suggests research into effective interventions could benefit a large part of the community, especially if integrated into other projects such as fellowships, retreats, or consulting for EA organisations.
The following sub-sections discuss three theories of change for Effective Self-Help. As the project is still at an early stage and I have limited information upon which to weigh their relative value, I have not yet prioritised between these paths.
Theory of Change #1: Increasing number/quality of productive hours for EA Community members
Reasoning
Members of the community do high-impact work on the world’s most pressing problems. Improving their productivity increases the amount and quality of high-impact work on these problems by increasing the expected quality and quantity of work completed. Improving wellbeing could further increase productivity and produce a range of indirect benefits to work quality and quantity.
I envisage two main routes to impact in this way:
1. Incremental improvements for lots of people (marginal gains)
- Small increases in productivity can directly increase the amount of work people do per hour and the number of hours they work in total.
- Small improvements in many individuals’ wellbeing can increase productivity indirectly through:
- Fewer hours of work lost to mental/ physical illness
- Fewer hours of work lost to researching wellbeing/ health practices
- Improved networking from happier/ friendlier community dynamics
2. Profound improvements for a small % of people
- Some advice is high impact but only for the small minority of people it affects (e.g. sleep apnea). Large improvements for a minority of readers may:
- Reduce community burnout/ value drift
- Prevent major periods of lost work (e.g. avoid a depressive episode)
- Significantly increase the number of hours they can work and the quality of their work
As an aside, ‘high-impact’ work is an unhelpfully vague term that I am using to avoid getting into the weeds of the relative impact of different jobs. Roughly speaking, I am defining high-impact work as either earning to give, working for an EA organisation, or working on a high-priority cause (X-risks; animal welfare; etc.), though this likely has significant flaws as a framing.
Guesstimate model
To gain a better impression of the potential value of ESH in increasing the number and quality of productive hours for EA community members, I made a rough Fermi calculation through Guesstimate. This allows the user to input a value range rather than a fixed estimate for a given property, better accounting for the high uncertainty inherent in rough models like this one.
It is also worth noting that Guesstimate runs a new Monte Carlo simulation each time the page is refreshed, meaning the numbers below may be slightly higher or lower than what you see on the model.
According to this model, the average Effective Self-Help article will produce between 28 and 580 hours of additional high-impact work, with a mean estimate of 180 hours of productive work added. Please take a look at the model and the comments attached to each value to see how I produced these figures.
Looking at the sensitivity analysis for the above estimate, some key (mean average) figures from the model include:
- 2.5 recommended changes made per article
- 600 unique viewers per article
- A 6.9% increase in productivity per recommended change
As is evident from the wide range in hours of high-impact work added by each article, there is a lot of uncertainty in these estimates. This is a rough model and I have generally only briefly reviewed relevant evidence to produce a relatively informed range of estimates.
With more time and detailed consideration, these numbers could change significantly. The numbers are also based on my initial model of publishing articles and so do not account for how different and combined methods of presentation may affect the project’s success.
Given these points, I believe the most useful takeaway here is simply that the Guesstimate model demonstrates that ESH could be very high-impact if things go well with the project. At the most optimistic end of the estimates, each article could produce an additional hour of high-impact work for $4.5, thereby producing roughly six or more hours of high-impact work for every hour that could have been paid for directly.
Theory of Change #2: Improving wellbeing among the general public
Reasoning
Outside of the EA community, better self-help advice may also be a cheap way to incrementally improve global wellbeing if it were to reach a sufficiently large audience. This would plausibly occur through the same rough mechanisms as discussed above, combining incremental improvements for lots of people with large improvements for a small minority.
Guesstimate model
As with the previous theory of change, I have made a Guesstimate model to roughly quantify the potential value Effective Self-Help could produce. This model estimates the Quality-Adjusted Life Years (QALYs) that the average ESH article may add by cultivating a large audience among the general public and is therefore based on a larger readership than ESH currently has.
Some parameters in this model are the same as the previous one but with values adjusted to reflect potential differences in engagement between an EA audience and one comprised of the general public. Again, please take a look at the model and the comments attached to it to see the reasoning/ evidence behind the different estimates included.
Based on the model, the average Effective Self-Help article will add anywhere from 0.14 to 25 QALYs, with a mean score of 4.2 QALYs added per article. At the most optimistic end, this would result in a cost per QALY of $190[2]. Similarly to the previous model, the very large range in potential QALYs added makes it foolish to draw firm conclusions from these numbers.
At this stage, it remains plausible that with a large audience ESH could either be disappointingly ineffective at producing significant wellbeing improvement or impressively effective. Much of this current variance appears rooted in the eventual size of the audience (e.g. 5k vs. 50k) and the QALYs added per recommended change (which appeared highly uncertain from my very brief evidence review).
Theory of Change #3: Bringing new people into EA
As a final route to impact, a large and successful self-help website based on EA principles could plausibly attract many good new people to the community. Self-help is a very popular and widely-applicable topic, giving Effective Self-Help a huge potential audience.
80,000 Hours provides a useful example of an EA project that offers advice that is relevant and useful to the general public as well as EAs. Outside of personal contacts, 80,000 Hours has been the most common way people have first heard about EA since 2018. This suggests that providing people with a service that is beneficial to them for non-EA reasons, but underpinned by key ideas of effective altruism, may be a particularly effective way of engaging them with EA ideas.
As discussed previously, ESH would place a high bar on evidence quality, finding the most effective solutions, and using numerical estimates when uncertain - all key parts of the quantitative approach that underpins much of EA. In this way, the project could serve to sell the mindset of EA to people while they learn about topics they find more engaging and relevant.
With a small push, Effective Self-Help could direct its audience towards the rest of EA. Possible avenues for this include:
- An ‘About’ page highlighting the origins of the project within EA and the EA Forum
- A note acknowledging funding from the EA Infrastructure Fund at the top/ bottom of each page
- Links to relevant evidence and discussion of topics on the EA Forum or other EA sites.
- Prominently placed arguments highlighting how being able to make a large difference to other people’s lives with a little extra time/ money can be valuable motivation for making small improvements to personal wellbeing and productivity.
Downside risks and arguments for ESH being ineffective
Needless to say, I am enthusiastic about the potential impact of this project. However, this would not be a reasonable evaluation without at least a reasonable attempt at exploring why ESH might not be worth pursuing. The following section is split between downside risks (ways in which ESH could be counterfactually harmful) and arguments for ineffectiveness (reasons why ESH may be of low impact/ value).
Some of the arguments below appear important and could significantly compromise the value of the project. However, by taking reasonable steps to mitigate against them, these issues also seem solvable. Better arguments against the value of Effective Self-Help, whether in the comments or privately, are encouraged!
Downside risks
ESH gives advice that proves to be of net harm
Without sufficient care, ESH could provide advice that produces more harm than good. Assuming a reasonable minimum standard of research, I think it’s unlikely that ESH would provide advice that is directly harmful. More realistically, ESH could be of net harm in one of two ways:
- By providing advice that is of small benefit to most people but of large harm to a minority, with the negative effects outweighing the positive (e.g. recommending a supplement with more serious or prevalent side effects than expected)
- By diverting people away from actions that would have been more beneficial (e.g. self-help advice for treating/ reducing depression encourages some people not to seek therapy or medication when these would have been of far greater benefit).
Mitigating downside risks:
- Factoring in risks, such as the side effects of supplements, to recommendations (and perhaps generally overestimating their importance) should help minimise the first issue.
- Consistently highlighting important non self-help options and emphasising that recommendations are supplementary to things like therapy rather than replacements (e.g. like with this Scott Alexander article) should mitigate the second issue.
Serving as a weak introduction to EA
One way ESH could be impactful is by introducing new people to EA ideas, some of whom may then become actively involved in the EA community/ work in high-impact areas. However, ESH could be counterfactually harmful if it serves as a significantly worse introduction to EA than people may have otherwise received.
Specifically, Effective Self-Help could either:
- Give people a negative impression of EA through a bad/ skewed presentation of its ideas
- Be less persuasive than an alternative means of introduction to the community someone may have later been exposed to. This would mean some people are dissuaded from EA when they may have later been persuaded of it in ESH’s absence.
Mitigating downside risks:
- Consistent feedback from other (more senior) members of the EA community on how best to present EA through the lens of Effective Self-Help should mostly mitigate this.
- A self-help website has limited overlap with current alternative routes to discovering EA (e.g. 80,000 Hours; Future Perfect). Given this, it seems likely that the proportion of people dissuaded from EA by ESH who would have discovered the community elsewhere would be outweighed by people who become engaged in EA through ESH who would not otherwise have learnt of EA.
Increasing the insularity of the EA community
Creating a self-help resource within the EA community that relies primarily on common EA methods could reduce people’s exposure to new ideas and ways of thinking. EA individuals and organisations developing similar policies for wellbeing and productivity based on ESH’s work could reduce experimentation and mean useful insights from taking different approaches are lost.
Additionally, if I continued to produce most/ all of the recommendations ESH makes, any biases and limitations in my thinking could also increase ESH’s contribution to greater insularity in EA.
Mitigating downside risks:
- I think this is a smaller concern than the other two but to mitigate this, ESH could emphasise experimentation with its recommendations, make it easy for people to make suggestions for what to research/ recommend, and build a research team so my ideas do not have sole influence on what’s recommended.
Arguments against ESH being effective
There is not enough (good) evidence to be able to make useful recommendations
In researching the first two articles, I have found this to be somewhat, but not fatally, true. There are often only a handful of studies on the effectiveness of certain practices that meet basic thresholds (e.g. the inclusion of a control group). That said, there has generally been enough information to make a reasonable estimate of effectiveness so far.
This problem can also likely be further mitigated by continuing to improve methods of research and comparison, particularly by finding a consistent, and ideally relatively objective, way of including more Bayesian evidence.
Readers prefer fluffy, narrative-based self-help articles over those that are more practical and evidence-focused, limiting the project’s audience and impact
This seems quite plausible to me but I haven’t yet tested this. Engagement for our first post was higher than expected and suggestive of a large potential audience, but what appeals to EA Forum readers is likely to be quite different to what appeals to a more general audience. Producing content in more engaging formats (audio, video, etc.) may also help to counter this.
If this were a significant issue, it’s unlikely that ESH would still be cost-effective as an intervention targeted at improving the wellbeing of the general public (Theory of Change #2) as it would struggle to generate a sufficiently large audience. However, this would be unlikely to affect the project’s benefits to people in EA and similar communities, given the focus on practicality and evidence already apparent in these groups (Theory of Change #1).
The project could also still be highly valuable if the subset of the general public who do not prefer narrative-based articles are also those most likely to engage with EA, which seems a reasonable assumption. In this way, ESH may only cultivate a modest audience amongst the general public but still drive significant new engagement with EA through this smaller audience (Theory of Change #3).
Individual differences in benefit significantly outweigh the general differences in value between interventions
Put another way, prioritising interventions would be of little value because the large majority of effectiveness is determined by the circumstances, background, and behaviours of each reader. This seems relatively plausible in a milder form (individual differences have a significant, but not overwhelming effect) but my intuition is that it is unlikely that individual differences are large enough to make generic differences in effectiveness unimportant.
Providing multiple top recommendations and encouraging experimentation so people try multiple solutions both seem like somewhat effective solutions to this. Some form of a screening quiz for prioritising recommendations based on the individual could also be valuable in addressing this.
There are better formats and approaches to providing self-help advice than articles and a website
This seems likely to me but also not much of a concern. With good, evidence-based recommendations to provide, Effective Self-Help could pivot in multiple directions in finding the best way to disseminate this information. Classes, podcast episodes, ebooks, and Youtube videos all seem like possibly useful and achievable formats to repackage information into.
Interventions targeting other groups are likely to be much more cost-effective
This appears quite likely but only applies to Effective Self-Help as a means of improving global wellbeing (Theory of Change #2).
Self-help advice targeted at the general public is likely to have an audience that mostly comes from developed countries. As a general principle, interventions targeted at larger and more neglected groups (e.g. the global poor, farmed animals, or future generations) are likely to be more cost-effective, often by a significant margin. This could make ESH counterfactually harmful if it diverts work and/or funds away from more effective causes.
However, Effective Self-Help could still be highly worthwhile and cost-effective by focusing on improving productivity in the EA community (Theory of Change #1) and attracting a significant number of new people to EA (Theory of Change #3). In doing so, ESH could increase the amount of valuable work done to serve larger, more neglected groups/ causes and the number/quality of people working in these areas.
Final notes & next steps
Below are some brief final notes on what I’m uncertain about (feedback encouraged), the work I’m doing at the moment for Effective Self-Help, and how you can help!
Things I’m uncertain about
Wellbeing, productivity, or both?
At present, my concept for Effective Self-Help includes work on effective interventions for both wellbeing and productivity. However, there is a good case to be made that work on one of these topics is likely to be higher impact than work on the other. Which topic this is likely depends on the theory of change that dominates the project’s overall value.
In other words, if Effective Self-Help becomes a project aimed mostly at increasing the quality and quantity of ‘high-impact’ work, a focus on productivity is likely to be best. Alternatively, a focus on wellbeing could be more valuable if this proved a more popular draw to a general public audience and ESH delivered most of its impact from directing new people to the EA community.
As the project is still at a very early stage, I am currently hedging between the two options and researching interventions for both wellbeing and productivity. Given the uncertainty involved in estimates of impact, this appears to me as the wisest strategy in the near term.
However, I am interested in arguments for why ESH should focus only on either wellbeing or productivity. I expect that down the line, Effective Self-Help will concentrate mostly, or exclusively, on one or other topic rather than both.
Cost-effectiveness vs. effect size
The two articles I’ve published so far purposefully differ in the presentation of how effective different interventions are.
The sleep article prioritises interventions based on their cost-effectiveness, using ann adapted weighted factor model to consider how effective different practices are in relation to the time, money, effort, and risks involved in doing them. In contrast, the article on stress prioritises interventions based purely on their effect size, with the ‘costs’ of an intervention only listed in the article’s evidence table.
I am undecided on the best approach to take going forward as I can see good arguments on both sides. Ranking interventions in terms of cost-effectiveness makes ESH’s work more unique and practical. This could lead to the project having a higher impact by:
- Encouraging a greater percentage of readers to take up and maintain recommended changes
- Leading to a larger total audience as ESH stands out more relative to other good sources of self-help advice.
Alternatively, prioritising interventions based purely on their effect size could also make ESH more impactful by:
- Directing people to the interventions that produce the biggest changes, potentially incentivising people to take up more impactful recommendations
- Allowing each reader to make their own decisions on how important different costs are to them, meaning they may then find interventions that are a better individual fit that they are more likely to maintain.
How best to select topics
Up to this point, I have spent little time on prioritising topics, choosing to focus initially on doing research and developing the wider concept of the project. The process up to now has been a rough mix of my intuitions about what would be valuable, suggestions people have made to me, and brief reviews of studies and articles on wellbeing/ productivity.
This is likely to be quite a sub-optimal method of choosing topics and is therefore something I plan to change. I think a wider outreach for suggestions could be valuable, along with time spent doing meta-level research into how to prioritise between different aspects of wellbeing and productivity.
What I’m currently working on
Articles
I expect to publish multiple further articles in the next few weeks so if you are excited by the project then keep an eye out for those. The next post will be an outline of the most effective techniques and strategies for building habits, likely followed by a review of the most cost-effective ways to spend money increasing your productivity.
Other formats
I am keen to find as many avenues as possible for deriving value from the research this project produces. I intend to upload recordings of myself reading the articles currently published for ESH as an initial experiment with audio content.
I’ve also had discussions with some wonderful people about using the recommendations for talks, workshops, fellowships, and courses. If you have any ideas for how to best make use of this work, let me know.
How you can help
Get involved
Effective Self-Help is currently funded through a seed grant from the EA Infrastructure Fund. I plan to reapply for funding in the next few weeks, as well as reach out to other grant programs.
I hope to expand the project in the coming months and this will likely involve building a team of people to work with me on ESH. I am uncertain of what exactly this may look like but expect it may be determined in large part by the skills/ fit of people interested in getting involved. Roughly speaking, I would be excited to hear from anyone interested in the project with any of the following skills/ experience:
- Visual content: producing engaging digital media - ideally short, informative videos, and potentially social media content.
- Research: reviewing relevant scientific literature to compile effect sizes and build evidence tables (like this one).
- Writing: producing clean, engaging, well-structured writing that communicates large amounts of research in a concise manner.
- Web design: building/ maintaining an engaging, easy to navigate website capable of hosting a variety of media.
I may look to source work in these areas through employed staff, contractors, and/or volunteers. I am also open to the idea of someone joining the project as a ‘co-founder’ in effect, working alongside me across different areas and with a vision of the project’s development as a whole.
If you think you might be a good fit for getting involved in whatever capacity, you can send me a message, email, or set up a meeting with me.
Please have a low bar for getting in touch. This is an early-stage project so I’m looking for a level of experience that is closer to “I’ve made a couple of nice-looking videos for YouTube and know how it works” than “I have 5 years experience working in digital production”.
Feedback
I am hopeful Effective Self-Help may grow into something of substantial benefit to the EA community and beyond. I expect that there are flaws in the work I’ve done so far, the reasoning in this article, and the plans made for moving forward. If you have thoughts on these, please let me know.
I would also be highly appreciative of any offers of mentorship/ guidance, either with the uncertainties discussed above or more generally in how best to direct and manage a project like this.
Acknowledgements
A special thanks to Michael Aird for particularly detailed feedback on this piece and support with developing an initial concept for ESH, to Brendon Wong for many valuable and enthusiastic discussions on how best to build an effective self-help resource, to David Hartsbough for particularly insightful and enthusiastic feedback, and to Joseph Young for persistent encouragement to make this project happen.
- ^
Existing projects that are (somewhat) similar include Upgradable, Optimize.me, Overcome, Better.so, The School of Life and FoundmyFitness.
- ^
For some comparison, GiveDirectly is estimated to avert a DALY equivalent for roughly $860, the Against Malaria Foundation for $50, and the Deworm the World Initiative for $14 (Hillebrandt, 2021)
casebash @ 2022-01-06T16:14 (+12)
I'm very keen to see how this project goes.
I think it's certainly going to be something of a challenge given how many high-quality resources are out there.
Two thoughts:
• If this projects starts becoming influential within EA, then it would be worthwhile paying experts to comment and review the articles
• It might be worth running a survey to see which alternate resources EAs are most likely to utilize instead and use that as the bar you need to exceed
Ben Williamson @ 2022-01-07T12:44 (+6)
I appreciate these thoughts, good suggestions and I agree on both counts. Running a survey like that hadn't come to mind but is definitely something I'll do now (even just a basic initial version like asking it as a question post here on the Forum)
davidhartsough @ 2022-01-06T18:01 (+9)
Fantastic! I feel as though nearly any project founded with the basis of those 7 principles is bound to be pretty amazing.
I can't wait to see how you'll tackle these challenges and uncertainties. You've got great question along with a great idea.
I had a few thoughts pop up throughout the read, but I'll just stick to 2 to post in this comment:
#1.) I'm curious to hear what people in the EA forum think about the idea of ESH running its own research from time to time to help fill in any gaps or further test any ideas. If ESH is truly struggling to find good research on a particular topic, then could the ESH team conduct its own studies? For example, Clearer Thinking runs its own studies for almost every article it writes. Thoughts?
#2.) Since your feedback request is for potential flaws, I'll briefly mention a risk that I've seen in self-help that is adjacent to two of the points you mentioned in the "Downside risks" section ("ESH gives advice that proves to be of net harm" and "Individual differences in benefit significantly outweigh the general differences in value between interventions") ::
Some self-help interventions can be wonderful for resolving a personal problem for X% of people in a particular set of circumstances yet also exacerbate the problem for Y% of people with a particular factor that changes the conditions. (And without getting into details, I'll just say that the cascading consequences tend to result in a lot of suffering for the people in group Y.)
So how can this be prevented? In the section on "differences in value between interventions" you mentioned the idea of using "a screening quiz for prioritising recommendations based on the individual." I think that's a great idea. Maybe there are other solutions to come up with as well.
I just wanted to call this out since sometimes interventions aren't just a positive "difference in value" between people — sometimes they're helpful for most yet harmful to a few. And with any given medium/format for the resources ESH will provide, it will need to consider how to communicate this (if there is that risk of harm). In the form of an article, sometimes all you can do is empower the reader with the tools they need to evaluate their own conditions/circumstances to determine which intervention makes sense for them personally (or how to customize an intervention, or whether or not they should even consider the intervention for themselves at all). Beyond articles, it would be terrific to incorporate more interactive elements (like the screening idea) to not only improve the effectiveness of the ESH resources but also prevent potential harms/risks. (And I think these ideas fall nicely into "Practicality", "Comprehensiveness/Breadth", "Presentation", "Research Rigor", and [personalized] "Prioritization".)
(Overall, I'm in love with the idea and am surprised it didn't come into being 10 years ago. Thanks for the thorough introduction!)
casebash @ 2022-01-07T05:08 (+2)
For example, Clearer Thinking runs its own studies for almost every article it writes
Wouldn't that be extremely expensive?
Eddie Liu @ 2022-01-17T20:58 (+4)
Depending on the study design, it can actually be relatively cheap! Just use Amazon Mechanical Turk or Positly to get participants. Of course, some study designs would be 20x harder than others. I imagine that testing different self help interventions would be rather hard but smaller surveys would be really easy.
davidhartsough @ 2022-01-07T12:30 (+4)
Great question! I don't actually know. (Although I do know that Spark Wave, the parent organization, is also the "parent" of Positly [both founded by Spencer Greenberg], so they probably have a deal worked out haha. Who knows.)
MichaelA @ 2022-01-08T09:44 (+6)
FWIW, I'd guess that:
- It's rarely wise to do as many different types of things as Spencer Greenberg does and usually best to focus more, at least until you're ~excelling at each thing
- I think Spencer's approach is going very well for him, but that it's not good as standard advice for most people or even most entreprneurial EAs
- I think it's plausible it'd make sense for ESH to eventually sometimes do quite small, quick, simple, and cheap primary empirical research, like a small quick survey, as one small additional input into recommendations. But I think it's easy to underestimate how much survey work benefits from methodological expertise and effort, and I think that sort of work is very different from what's proposed in this post. So I'd guess it shouldn't be one of ESH's primary focuses or something ESH tries to get really good at (just in order to focus).
But those are just some quick thoughts.
Manon Gouiran @ 2022-04-05T11:02 (+6)
Effective Self-Help is looking for feedback! Hi everyone, we're looking to evaluate our work up to now researching the most effective wellbeing and productivity interventions. We'd really appreciate you filling out this 5-10 minute evaluation survey which will help us figure out how we can be of most help to the EA community.
You don't need to have read any of our reports up to now to fill out the survey so we'd love to hear your thoughts, even if you've never heard of us before! Thanks in advance for your help!
Max_Carpendale @ 2022-01-09T10:24 (+5)
Really looking forward to your articles!
It seems like you have a good strategy for producing high-quality articles (and particularly happy you'll be ordering by effect size).
Ben Williamson @ 2022-01-10T10:20 (+1)
Thanks!
PeterSlattery @ 2022-01-08T22:31 (+3)
Thanks for this, Ben. I am quite excited about your ideas and I like your work so far. I think that this is a very worthwhile project, and might be interested in helping in the future with creation or promotion once I get a few things off my plate.
I am very invested in self-help - probably more than most people (and possibly too much!). For instance, I have been tracking my happiness and many other variables daily since mid 2018 (see this post as an example). Over my life, I have tried hundreds of different things (e.g., supplements, technological and lifestyle interventions) for improving my productivity and health. I would therefore be hopeful that I could make some helpful contributions.
Some quick thoughts on building the project (I imagine that a few of these are already in your head):
- Building a community and getting regular engagement with the work seems very important. I'd make sure to post about this and the posts in all relevant Facebook groups and LessWrong etc.
- Related to that, I think that you should consider connecting and collaborating with key parties who have interdependent goals & similar incentives, e.g.,
- The EA coaching and self-help community
- Academic researchers interested in publishing in this area. READI might be interested in the future when we have more capacity. We might also be able to help with the review/evidence synthesis processes.
- Entrepreneurs/orgs like Spark Wave or Happier Lives Institutes
- I like the 'surveying people to understand what they want' idea proposed by casebash. Helps prioritise what you work on and also likely increases engagement with audience.
- I'd keep revising and updating work on sleep and productivity (maybe treat them as 'living reviews' updated biannually).
I won't commit to do anything more than make this comment now (I am already overinvested in projects!) but I wanted to quickly share ideas and signal my support and interest after reading it. I look forward to reading more posts!
Ben Williamson @ 2022-01-10T10:30 (+3)
Thank you! Feel free to reach out about getting involved at any point when you have more capacity. These are some great points that I've mostly started working on but there's (always) a lot more to do.
I have many more people I want to reach out to about the project and certainly I'm now working on building a wider community/ collaboration team for moving forward. And absolutely, I want each review to be a living document and for the recommendations to be consistently improved and updated over time. I think I'll add a note on the stress and sleep articles to reflect that.
davidhartsough @ 2022-01-12T13:05 (+4)
+1 to the "living reviews" idea! Love that Peter! Such a good goal to have the outputs be "consistently improved and updated over time".
PeterSlattery @ 2022-01-10T23:43 (+1)
Thanks Ben, I will!
markus_over @ 2022-01-09T10:12 (+3)
I think that you should consider connecting and collaborating with key parties who have interdependent goals & similar incentives
A small addition to your list would be this post about a study on a depression related intervention that I believe originated from within the EA community. Might well be worth contacting the author.
Jonas Moss @ 2022-03-09T10:44 (+2)
I am undecided on the best approach to take going forward as I can see good arguments on both sides. Ranking interventions in terms of cost-effectiveness makes ESH’s work more unique and practical.
Do both? And include brave, precise practical recommendations. Ranking solutions in terms of cost-effectiveness is riskier for you (you will be criticized more), but probably the most helpful for most of your audience. But effect sizes are nice to know too.
One of the greatest barriers to starting an important transformation is the practicalities (which of one 20+ group sessions in the gym should I sign up for?) Choice paralysis makes everything harder, it makes it hard just to start. For instance, I think that you bravely taking a stand on questions such as which group session I should go to (Zumba? Cardio combat? Indoor running?) would help me lot.
I like the following quote of Karolina Sarek:
> What about a situation where your audience really wants an answer? Maybe they’re a funder who is thinking about funding a new project or organization. If you, as the researcher, do not draw conclusions from your own research, in a way you're passing this responsibility through to your audience. And very often, the audience has less knowledge or time to explore all of the nuances of your research as a whole, and therefore could draw worse conclusions compared to those that you would.
So yeah, you're passing on the cost-effectiveness research to people who know much less about the subject than you.
markus_over @ 2022-01-09T10:06 (+2)
Interesting project! It reminds me a bit of Huberman lab, the existence and apparent popularity of which could be taken as an argument in favor of ESH to be worthwhile (although format, target audience and focus might of course differ quite a bit).
One thing I personally find very interesting is the point you mentioned as a counter argument: "Individual differences in benefit significantly outweigh the general differences in value between interventions" - in my opinion, this could even be viewed as quite the opposite: my impression is that in most easily digestible sources (such as pop-sci books, podcasts, blogs), this point is mostly ignored, and getting reliable information about this facet of health and well-being interventions would be great.
People very often speak of effect sizes as if they were a thing inherent to an intervention or substance, when actually they quite often seem to depend strongly on the person. An intervention with a strong positive effect size on only 10% of people could be much more exciting than an intervention with a weak effect on everybody. Even a large positive effect on small number of people combined with negative effects on the rest could be a very useful intervention, given you find out early enough whether it works for you or not. Getting some insight into the nature of variance of different interventions, if such data is available, could be really useful. It might of course be the case that most studies don't offer such insights, because it's impossible to tell whether the subset of participants that benefited from an intervention can be attributed to noise or not.
Ben Williamson @ 2022-01-10T10:23 (+2)
That's a great point! I definitely intend to review what evidence is available for the size of individual differences in benefit from a given intervention. As you point out, if there are ways of accounting for this in how to select/ prioritise interventions, that could be particularly useful.