ImpactMatters was acquired by CharityNavigator; but it doesn't seem to have been incorporated, presented, or used in a great way. (Update/boost)

Some positive developments here:

Charities Working in High-Impact Causes Highly rated charities working in unusually high-impact causes, identified by the effective altruism community and our analysts

I'm not directly involved with this effort, but I do know that CN has a new person in charge of trying to make these impact ratings work, and they are trying to think very creatively about how to improve and add value. They need something they can do at scale and it's a tough job. I know this because their team has been reaching out to various people in the space (myself included) to help with brainstorming. So just to say, they are definitely not burying this effort. 

I think it's still unclear if CN will be able to turn this effort into something compelling, but I think it's worth waiting to see what their new team comes up with before passing final judgement. 

I think that intra-area comparisons may be ok, also due to acceptability by users, who may think traditionally about charity (for example, open Charity Navigator because they received an emotional appeal from a US-based charity), only more areas which relate to EA causes should be included and listed in a (strategic) combination with non-EA cause areas (more clicks) (maybe the impact units and top charity per impact unit should be listed alongside or so - $40,000 to train a guide dog for 1 person, $100 to restore eyesisght of 1 person, etc), and more areas which involve EA and non-EA charities should be included (e. g. providing meals to the hungry as opposed to meals in US homeless shelters - then, for example, Yemen Aid, which provides 6.4 meals/$1, can compete with $2/meal) - the 'at the risk of famine' and 'food insecure' distinction is lost by this grouping but this can be included in further description. This can be sold to Charity Navigator as greater total impact. Potentially some computing capacity of some persons in EA looking to volunteer, or in companies, such as Google, to improve the popularity of CN can come in a bundle? The risk is that Charity Navigator will lose funding influence share due to less captivating entertainment of users interested in charity assessment but ways for the opposite should be developed.

You should speak with Sanjay to speak with Elijah Goldberg or Tamsin Chen.

Podcasters can ask for a brief interview with Dean Karlan, for example to elaborate on ways of motivating charities, especially those in neglected cause and geographical areas with high potential, to develop solutions that support systemic change toward institutions that safeguard continuous improvement of living standards of an increasing number of individuals,  and test these solutions affordably. [For example, should they present data in a specific format to Charity Navigator to get high rating, or emulate and improve the approaches of top charities in order to gain further funding?] - the last time I checked (asked the Charity Navigator 'agent'), only US-based charities (including internationally operating ones who have only a foundation in the US) were evaluated, due to the use of tax statements, but the incorporation of international organizations' evaluation was considered. Elijah was also thinking of developing a simple questionnaire to assess impact.

Also, Northwestern students can inquire regarding a class project similar to the Pro bono student impact audit opportunity shared by ImpactMatters in 2020 to develop a rapid evaluation framework within select causes. This may already exist.

In terms of a thorough but more affordable and impactful evaluation, I was/am suggesting (linked in this post) that

• counterfactual beneficiaries' outcomes are estimated by increasingly more robust methods (organizations may be interested in accurate predictions due to estimates' records and later data validation)
• funders' alternative investments are considered (e. g. personal investment vs. funding of a comparable program)
• in-program's-absence caretakers' alternative investments are considered (e. g. into necessities vs. luxuries vs. 'independence' capital development)
• pro bono impact increase consultancy is offered
  • at least for charities which can increase their cost-effectiveness with relative facility, such as by geographical expansion or animal welfare considerations

I was also thinking about measuring subjective wellbeing changes via the visual analog scale method as the end goal. This, if given also to individuals (hypothesized for non-humans) who would benefit indirectly from an externality or another change, could address the issue of an organization addressing multiple issues on priority needs basis and thus having difficulties to focus on one or several impact metrics and may detect systemic change better.

The key would be to get organizations to fill a form, such as by automatically making an appealing annual report for them, offering discounts on evaluation and fundraising consultancy (such as narrating competitiveness due to higher impact), or the Charity Navigator listing.

To conclude,  the current rating works well, but important additionalities may be available.

David Moss:

There was some discussion of the original acquisition here.

Historically, Charity Navigator has been extremely hostile to effective altruism, as you probably know, so perhaps this isn't surprising

My response

Thank you, I had not seen Luke Freeman @givingwhatwecan's earlier post

That 2013 opinion piece/hit job is shocking. But that was 9 years ago or so.

I doubt CN would have acquired IM just to bury it; there might be some room for positive suasion here.

Moving some comments from the Shortform...

Aaron Gertler wrote:

I spent a few minutes looking at the impact feature, and I... will also go with "not satisfied".

From their review of Village Enterprise:

Impact & Results scores of livelihood support programs are based on income generated relative to cost. Programs receive an Impact & Results score of 100 if they increase income for a beneficiary by more than $1.50 for every $1 spent and a score of 75 if income increases by more than $0.85 for every $1 spent. If a nonprofit reports impact but doesn't meet the threshold for cost-effectiveness, it earns a score of 50.

My charitable interpretation is that the "$0.85" number is meant to represent one year's income, and to imply a higher number over time (e.g. you have new skills or a new business that boosts your income for years to come).

But I also think it's plausible that "$0.85" is meant to refer to the total increase, such that you could score "75" by running a program that, in your own estimation, helps people less than just giving them money.

(The "lowest score is 50" element puzzled me at first, but this page clarifies that you score "0" if CN can't find enough information to estimate your impact in the first place.)

Still, this is much better than the original CN setup, and I hope this is an early beta version with many improvements on the way.