EA Malaysia Cause Prioritisation Report (2021)

By Yi-Yang @ 2021-04-24T05:48 (+45)

Summary

• What kind of research is this?
  • We conducted a cause prioritisation research at a local level, specifically in Malaysia.
• Who is involved in this research?
  • Yi-Yang Chua, Lynn Tan, Chan Tzu Kit, and Kaartik Nagarajan.
• What did we hope to identify from this research?
  • The top cause areas in Malaysia.
  • Strategic insights on how we should build the EA community in Malaysia.
  • Career pathways that are the most impactful in Malaysia (excluding remote jobs advertised in 80,000 Hours).
• Who we think this research will benefit
  • Malaysians, and EA aligned Malaysians seeking to maximise impact.
  • Potentially other EAs who are hoping to conduct similar research
• What are the top cause areas in Malaysia, from most impactful to least?
  Note: that the following cause area rankings are not definitive and conclusive.
  • EA community building
  • Farmed animal welfare (land)
  • Mental health
  • AI risks
  • Financial literacy
  • Family planning
  • Farmed animal welfare (aquatic)
  • Improving diversity and inclusion

Disclaimer

We think it’s important to list the limitations of our research, so that readers can make better judgements on this research. 

• We don’t have a strong conclusion. We don’t think our findings should be taken very seriously, nor do we think our research process is a template that potential researchers should follow strictly. However, we do think our findings can be considered for future research or strategic decisions related to EA. If we were to conduct another 40 hours of research, we think that it’s very likely that the rankings will change somewhat - we estimate the following credences:
  • 80% credence that at least four cause areas will remain in the top eight.
  • 90% credence that at least one cause area will drop out of the top eight.
• We lacked experienced or trained researchers. Most of our on-the-go learning stemmed from reading Charity Entrepreneurship’s research methodologies. Lynn Tan, who was part of CE’s incubation programme in 2020, also helped out with the research.
• We lacked full-time researchers. All five of us were involved at a voluntary capacity for around 2 hours a week. As a result, we unintentionally spent longer than we intended conducting the research. The entire research process took 8 months - we started around July 2020 and concluded most of our findings by March 2021.
• Our findings may be outdated. Since there’s an approximate four month gap between us publishing this report and completing our literature review, we might have overlooked new evidence that appeared in between this duration.
• We conducted our research at a cause area level rather than at an intervention level. Since findings at such a level are very broad, it’s difficult to generalise conclusions on a specific cause area; moreover, conclusions may change dramatically at the intervention level of a specific cause area.
• We might not have sufficiently considered other metrics or cause areas that could be objectively important (i.e. unknown unknowns). For example, we didn’t include “advocating economic growth friendly policies” as a cause area, which is likely to have qualified  in the top eight cause areas.
• In areas which lacked objective estimates, we had different “rigid” subjective beliefs that led to significant differences in conclusions which were difficult to reconcile (i.e. we retained our initial scores) despite having thorough discussion during the recalibration stage.
• Some of the cause areas had final scores that were highly similar, which means some cause areas didn’t rank over the other significantly. For example, the difference between the cause area that rated first and the cause area that rated second was a mere 0.46%.
• Given that we did not detail the exact steps taken and the specific literature we reviewed to inform our decision, our analysis may be difficult to replicate or be checked by independent researchers.

Research process

We emulated a large part of our methodology based on Charity Entrepreneurship’s research process. However, we also reduced a large chunk of CE’s research process into a more manageable methodology due to a lack of capacity and research experience. Some key differences include:

• Conducting the research at a cause area level rather than at a intervention level
• Choosing only one method to sort the ideas, namely the weighted factor model
• No individual reports for the top cause areas

Here is our research process: 

1. Idea generation stage
   1. We brainstormed for ideas, which produced 43 cause areas.
2. Initial ranking stage
   1. We brainstormed for factors that we wanted to use for weighing the cause areas. Most of the factors can be found here.
   2. Each researcher weighed the factors using a 100-point distribution privately and independently; then aggregated the weightages using the median. The top three weighted factors, which were used for the initial ranking of cause areas, were neglectedness, cost-effectiveness, and logistical difficulty.
   3. We evaluated the 43 cause areas using the top three weighted factors and a 5-point Likert scale privately and independently; then aggregated the scores using the median. This evaluation was done without any additional research on any cause areas.
   4. We picked the top 12 cause areas from the initial 43 cause areas. We chose 12 mostly due to capacity constraints.
   5. We then combined the several cause areas into eight due to overlapping definitions.
3. Weighted factor model stage
   1. We chose 12 factors that we wanted to use for weighing the top eight cause areas, and grouped them into two categories.
      1. Overall impact: evidence base, cost-effectiveness, neglectedness, logistical difficulty, metric focus, flexibility, flow through effects, timing, and risk of negative or no impact
      2. Limiting factor: scale limit, and funding availability, and talent availability
   2. We weighed the 12 individual factors using a 100-point allocation privately and independently; then aggregated the weightages using the median. We conducted a brief discussion to recalibrate our weightages.
   3. We  weighed the two categories (overall impact and limiting factor) 60% and 40% respectively.
   4. We conducted a 1-hour literature review for each of the eight cause areas, and evaluated them the 12 factors on a 5-point Likert scale. This research and evaluation was done privately and independently.
   5. We then aggregated our individual scores using the median.
4. Recalibration stage
   1. We conducted four 1.5-hour long discussions for each cause area and factor that had more than a two point difference in scores (i.e. we didn’t have similar conclusions). These discussions were used for understanding new evidence and perspectives from one another, which were subsequently used to update our beliefs and recalibrate our scores.
   2. We then aggregated our individual scores using the median.

Findings

For more detailed scores, you can find them here.

Implications

• Based on the findings of this research, we are more confident that having some voluntary part-time capacity to continue with EA community building in Malaysia is likely the right choice. However, without more evidence, we are uncertain whether a paid full-time capacity will be a good choice.
• We are more confident in the impact and expected value of organising events or discussions around certain EA-related cause areas listed above .
• With regards to further research, we think it might be helpful to:
  • Conduct deeper research into EA-related cause areas listed above (e.g. EA community building, farmed animal welfare, mental health, and AI risks), in particular what specific interventions will be impactful and how impactful will they be.
  • Conduct shallow research into other cause areas that are potentially impactful but did not make it into the initial ranking stage.

Learnings

This research did not generate highly decision-relevant insights at this time. However, it did confirm our initial intuition (or increase our confidence) about which cause area is most important. 

We should have been more specific with our research goals in the initial exploration stage by forecasting the value of information that we expected this research to generate. We should have then compared and prioritised which projects to take on with our limited capacity. We suspect doing this initial exercise would have helped us make a more informed decision and set better expectations. 

EDIT from Yi-Yang: please do check out Jamie Harris's comment and my response below. I think it incapsulates learnings that I wasn't able to write down in our forum post submission. Basically, I wouldn't recommend using this methodology as the first iteration of local priorities research for groups.   

Contact

If you’re interested in discussing more about our findings, please comment below or reach out to us at our Facebook page. You can also reach out to us by filling up this form.