Summary: GiveWell's list of top and standout charities is identical to that of 2014, although with special emphasis on the Against Malaria Foundation. Giving What We Can's recommendations have been static for several years. To estimate what rate of churn we should expect, I examine the age distribution of GiveWell (and Giving What We Can) recommended charities. For top international aid charities, typical age is a decade or less, possibly a result of focus on smaller organizations narrowly focused on interventions with recent funding gaps.
What does stability in GiveWell and Giving What We Can recommendations mean?
This year's list of top charity and standout charity recommendations from charity evaluator GiveWell are identical to those of 2014, although GiveWell has provided firmer guidance for AMF as the target for marginal donations after Good Ventures has filled the most urgent funding needs of the other recommended charities. Likewise, Giving What We Can's recommendations are the same as 2014 and 2013. What should we infer from this (tiny) bit of information?
On the one hand, improving charity evaluation (at least for a given set of criteria, which may be evaluated themselves) should have diminishing returns as it converges toward the truth, or at least reasonable estimates. If every charity were perfectly evaluated, then changes in rankings would occur only as the effectiveness and room for more funding of existing charities shifted, or new ones were added.
On the other hand, ratings might be stable because of insufficient work in finding new charities to recommend, before approaching an ideal level of evaluation. GiveWell has expanded its staff recently, but many senior staff have shifted effort towards the Open Philanthropy Project or training the new hires.
One indirect piece of evidence is the age distribution of recommended charities. If charities that meet the criteria for recommendation tend to be older, then there will be fewer new entrants that might qualify for recommendation in a given year, and we should expect more stability than if qualifying charities tend to be young. So I went over GiveWell and Giving What We Can recommendations from 2008-2015 to find median and mean ages of recommended charities and put them in a Google Sheet.
GiveWell recommendations (all)
For my first pass I looked at all charities recommended by GiveWell, including 'standout,' 'silver,' and U.S. domestic charities.
The earlier years are dragged up by the several U.S. charities, which tended to be older, and Partners in Healh and Population Services International. However, the core single-intervention charities GiveWell has focused on tend to be quite young.
GiveWell top international aid recommendations
GiveWell has used different systems for ranking its recommendations over the years, so there are some judgement calls in comparing its different recommendations. GiveWell's old recommendations are archived on its site, see 'past recommendations here. I interpreted the top international aid recommendations as:
Giving What We Can recommendations
Giving What We Can's list is similar to GiveWell's but has been more stable, focusing on a few main international aid charities, and has also aged with them:
Why are the recommended charities so young?
The charities recommended by GiveWell have tended to be narrowly focused on a specific, novel intervention that GiveWell considers to be highly effective and yet underutilized, and which are easy for it to understand. These criteria may naturally tend to weed out older organizations.
As bednet distributions and mass deworming continue to scale up funding gaps may be expected to decline. Much of the case for deworming is based on relatively recent (and still quite uncertain) research, so that there hasn't been time for old organizations to be based around it.
Older successful organizations may also tend to increase in size, but larger organizations with many distinct programs have been rejected because of the difficulty of parsing their operations.
This is also a small sample with lots of room for chance.
Conclusions
If the universe of potential top charities skews young, about a decade old, then we might expect to see something like 10% annual churn from that consideration alone. For the list of 8 top and standout charities I would hope to see more charities emerge soon, although it would be unsurprising for AMF to remain 'on top' longer. GiveWell's increasing money moved, especially through Good Ventures, may also accelerate change by shifting room for more funding and actively cultivating new charities with participation grants and funding of randomized controlled trials.
What does stability in GiveWell and Giving What We Can recommendations mean?
This year's list of top charity and standout charity recommendations from charity evaluator GiveWell are identical to those of 2014, although GiveWell has provided firmer guidance for AMF as the target for marginal donations after Good Ventures has filled the most urgent funding needs of the other recommended charities. Likewise, Giving What We Can's recommendations are the same as 2014 and 2013. What should we infer from this (tiny) bit of information?
On the one hand, improving charity evaluation (at least for a given set of criteria, which may be evaluated themselves) should have diminishing returns as it converges toward the truth, or at least reasonable estimates. If every charity were perfectly evaluated, then changes in rankings would occur only as the effectiveness and room for more funding of existing charities shifted, or new ones were added.
On the other hand, ratings might be stable because of insufficient work in finding new charities to recommend, before approaching an ideal level of evaluation. GiveWell has expanded its staff recently, but many senior staff have shifted effort towards the Open Philanthropy Project or training the new hires.
One indirect piece of evidence is the age distribution of recommended charities. If charities that meet the criteria for recommendation tend to be older, then there will be fewer new entrants that might qualify for recommendation in a given year, and we should expect more stability than if qualifying charities tend to be young. So I went over GiveWell and Giving What We Can recommendations from 2008-2015 to find median and mean ages of recommended charities and put them in a Google Sheet.
GiveWell recommendations (all)
For my first pass I looked at all charities recommended by GiveWell, including 'standout,' 'silver,' and U.S. domestic charities.
| 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | |
| Median age | 21 | 15 | 13 | 9 | 8 | 6 | 9.5 | 10.5 |
| Mean age | 23.2 | 17.4 | 15.125 | 10.75 | 7 | 7 | 11.25 | 12.25 |
The earlier years are dragged up by the several U.S. charities, which tended to be older, and Partners in Healh and Population Services International. However, the core single-intervention charities GiveWell has focused on tend to be quite young.
GiveWell top international aid recommendations
GiveWell has used different systems for ranking its recommendations over the years, so there are some judgement calls in comparing its different recommendations. GiveWell's old recommendations are archived on its site, see 'past recommendations here. I interpreted the top international aid recommendations as:
- 2008: Partners in Health and Population Services International
- 2009: VillageReach and Stop TB Partnership
- 2010: VillageReach
- 2011: AMF+SCI
- 2012: AMF, GiveDirectly (GD), SCI
- 2013: DtWI, GD, SCI
- 2014 and 2015: AMF, DtWI, GD, SCI
| 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | |
| Median age | 29.5 | 8.5 | 9.5 | 8 | 8 | 6 | 8.5 | 9.5 |
| Mean age | 29.5 | 8.5 | 11.66666667 | 8 | 7 | 7 | 8.5 | 9.5 |
Giving What We Can recommendations
Giving What We Can's list is similar to GiveWell's but has been more stable, focusing on a few main international aid charities, and has also aged with them:
| 2008 | 2009 | 2010 | 2011 | 2012 | 2013 | 2014 | 2015 | |
| Median age | 29.5 | 8.5 | 10 | 8 | 8 | 6 | 8.5 | 9.5 |
| Mean age | 29.5 | 8.5 | 10 | 8 | 7 | 7 | 8.5 | 9.5 |
Why are the recommended charities so young?
The charities recommended by GiveWell have tended to be narrowly focused on a specific, novel intervention that GiveWell considers to be highly effective and yet underutilized, and which are easy for it to understand. These criteria may naturally tend to weed out older organizations.
As bednet distributions and mass deworming continue to scale up funding gaps may be expected to decline. Much of the case for deworming is based on relatively recent (and still quite uncertain) research, so that there hasn't been time for old organizations to be based around it.
Older successful organizations may also tend to increase in size, but larger organizations with many distinct programs have been rejected because of the difficulty of parsing their operations.
This is also a small sample with lots of room for chance.
Conclusions
If the universe of potential top charities skews young, about a decade old, then we might expect to see something like 10% annual churn from that consideration alone. For the list of 8 top and standout charities I would hope to see more charities emerge soon, although it would be unsurprising for AMF to remain 'on top' longer. GiveWell's increasing money moved, especially through Good Ventures, may also accelerate change by shifting room for more funding and actively cultivating new charities with participation grants and funding of randomized controlled trials.
How are the mean and median ages different for 2010 if there was only one top recommendation?
ReplyDeleteMTGAP,
ReplyDeleteMy error, now fixed, thank you. Click through the spreadsheet link to see the charities and calculations.