2009 International Aid Research Process Overview
More recent version available
This is an older version of this page. There is a more recent version of this page available here.
This page summarizes the process we have followed to rate and recommend international aid organizations.
Table of Contents
- More recent version available
- Goal of the report
- Our process
- Key links in our report
Goal of the report
We seek to identify outstanding charities in the areas of developing-world health, economic empowerment, and education whose work is proven, cost-effective, and scalable (i.e., has room for more funding).
We do not claim that charities with these qualities are the only ones that should be funded, but rather that they are a good fit for individual donors, or people who are:
- Looking to give $250,000 or less in a year, and therefore unlikely to be able to fund personalized projects or hire their own staff (or philanthropic advice).
- Without personal/firsthand knowledge of particular charities' effectiveness - essentially starting with no or very limited knowledge of their options.
- Without the time or capacity to visit charities' operations on the ground, develop relationships with charities, etc. - looking to give effectively but also quickly.
Also see our blog post on small, unproven charities, which argues that such donors should focus on what is proven, cost-effective and scalable.
1. Literature review
We started our work in this area by reading and reviewing recent literature on the subject of international aid.
- We examined the publications lists of all relevant think tanks and public research organizations we could find, including Center for Global Development, Poverty Action Lab, Innovations for Poverty Action, World Institute for Development Economics Research, Overseas Development Institute, Kiel Institute for the World Economy, Brookings, Bureau for Research and Economic Analysis of Development, Development Research Institute. We read titles, abstracts and/or full publications depending on how relevant publications looked to our key questions and areas of focus (more below). We also read several books: The White Man's Burden by William Easterly, Making Poverty History by Jeffrey Sachs, The Bottom Billion by Paul Collier, Reinventing Foreign Aid eds. William Easterly and Nancy Birdsall, Making Aid Work by Abhijit Banerjee.
- We conducted systematic, in-depth examinations of several particularly relevant sources, including the Disease Control Priorities report, the Copenhagen Consensus, and others detailed here.
We initially were guided by a small set of general questions about international aid, which grew as we reviewed literature into a larger set of general questions, including questions about which approaches to aid seem most promising and/or have the strongest track records.
You can follow the progress of our literature review over time by reviewing the output to our public research email list between December 2008 and July 2009.
2. Identifying priority interventions
Believing health to be the area of international aid with the strongest track record, we have focused on identifying health interventions that are backed by strong expert recommendations and/or evidence bases (while also being relatively cost-effective), so that we can flag charities focusing on these interventions. Our page on priority interventions lists the interventions we have flagged as "priority" and describes our full process for identifying them.
3. Examining hundreds of charities
2009 research process
In 2009 (our first round of international research), we identified 510 organizations for investigation by searching tax records, databases and lists of relevant organizations, and considering all referrals (for more, see details of our process for finding charities, along with a list of all charities considered in 2009).
For each of these organizations, we examined the website and applied two heuristics to determine whether to investigate further. Any organization that, at the time we examined it, either clearly focused on one or more of our priority programs (see previous step) or publicly published substantial evidence regarding impact was flagged for further investigation (see next step).
The details of these heuristics, followed by a complete list of charities we examined in 2009 and which heuristics they have or have not passed, are available here. Further discussion of why we feel these heuristics are appropriate, and likely to identify promising charities with reasonable accuracy, is available at this blog post.
We contacted all examined charities have been contacted regarding their status, and we are open to reconsidering any charity that has come more into line with our criteria (we encourage charities to use our cause-specific submission form to alert us).
4. Examining promising charities more in depth
Any charity passing the heuristics detailed above has been investigated more in depth, to determine the extent to which it meets our criteria, and has been written up in an individualized charity review. Our full list of charities considered includes links to individualized reviews for any charities that qualified.
5. Identifying and ranking top charities
The most outstanding charities according to our criteria have been the subject of especially intensive investigations. They are ranked them on our list of top charities, which includes links to the reviews of these charities.
The above steps were taken between mid-2008 and mid-2009.
In mid-2010, we performed our first annual update of this report, consisting of:
- Reviewing all relevant charities we had newly become aware of.
- Checking citations and new research to ensure that the research on which our report was based was up to date.
- Revisiting recommended charities, seeking updates on their progress and funding needs.
We plan annual updates of this report. Some such updates may involve not only the above steps, but also evolution in our criteria for recommending charities.
Key links in our report
Conclusions and process
- Our top charities, including links to each top charity's full review.
- Our scope and criteria for this report. Our primary criteria are
- Strong evidence of positive impact
- Reasonable cost-effectiveness in terms of cost per life saved/significantly changed, although we use this concept with caution due to the difficulty of meaningful and precise estimates
- Room for more funding, an issue we believe does not receive enough attention (more at our 5-part blog series on the topic)
- Full list of charities considered, including whether they passed our initial screen (described above) and including links to individualized reviews when available.
- Discussions of highly celebrated charities that we don't recommend (and why we don't recommend them).
- Our priority interventions, including a description of the process by which we identified these programs. We flagged charities focusing on these programs for further investigation, although focusing on these programs was not a requirement (we also flagged and investigated any charity publishing substantial evidence regarding its impact)
- Summary of conversations with people with relevant expertise. (Transcripts/notes/recordings linked when available.)
- Notes from 2010 site visits to recommended charities in South Africa and Mozambique.
Our views on particular areas of developing-world charity
- Direct cash transfer
- Gifts of livestock
Our take on several debates particularly relevant to developing-world charity
- Aid's track record: we believe that aid has a murky track record overall, but has had some substantial successes in the area of health.
- The root causes of poverty: we see no clearly correct or consensus theory of the root causes of poverty, and favor interventions that help individuals concretely.
- The challenge of local ownership: we don't feel there is any known reliable formula for achieving local buy-in to an aid project, so we don't evaluate charities for their adherence to any particular set of practices along these lines; we prefer to evaluate them by demonstrated impact.
- Our take on the goal of sustainability: we prefer that an aid project be eventually sustainable without continued subsidies - all else equal - but due to the murky track record of goals along these lines, we look first and foremost for demonstrated impact.