# GiveDirectly Blogs

Exploring how to get real change for your dollar.
Updated: 38 min 54 sec ago

### Response to concerns about GiveWell’s spillovers analysis

Thu, 12/06/2018 - 14:02

Last week, we published an updated analysis on “spillover” effects of GiveDirectly‘s cash transfer program: i.e., effects that cash transfers may have on people who don’t receive cash transfers but who live nearby those who do receive cash transfers.1For more context on this topic, see our May 2018 blog post. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We concluded: “[O]ur best guess is that negative or positive spillover effects of cash are minimal on net.” (More)

Economist Berk Özler posted a series of tweets expressing concern over GiveWell’s research process for this report. We understood his major questions to be:

1. Why did GiveWell publish its analysis on spillover effects before a key study it relied on was public? Is this consistent with GiveWell’s commitment to transparency? Has GiveWell done this in other cases?
2. Why did GiveWell place little weight on some papers in its analysis of spillover effects?
3. Why did GiveWell’s analysis of spillovers focus on effects on consumption? Does this imply that GiveWell does not value effects on other outcomes?

These questions apply to GiveWell’s research process generally, not just our spillovers analysis, so the discussion below addresses topics such as:

• When do our recommendations rely on private information, and why?
• How do we decide on which evidence to review in our analyses of charities’ impact?
• How do we decide which outcomes to include in our cost-effectiveness analyses?

Finally, this feedback led us to realize a communication mistake we made: our initial report did not communicate as clearly as it should have that we were specifically estimating spillovers of GiveDirectly’s current program, not commenting on spillovers of cash transfers in general. We will now revise the report to clarify this.

Note: It may be difficult to follow some of the details of this post without having read our report on the spillover effects of GiveDirectly’s cash transfers.

Summary

In brief, our responses to Özler’s questions are:

• Why did GiveWell publish its analysis on spillover effects before a key paper it relied on was public? One of our major goals is to allocate money to charities as effectively as possible. Sometimes, research we learn about cannot yet be made public but we believe it should affect our recommendations. In these cases, we incorporate the private information into our recommendations and we are explicit about how it is affecting our views. We expect that private results may be more likely to change but nonetheless believe that they contain useful information; we believe ignoring such results because they are private would lead us to reach less accurate conclusions. For another recent example of an important conclusion that relied on private results, see our update on the preliminary (private) results from a study on No Lean Season, which was key to the decision to remove No Lean Season as a top charity in 2018. We discuss other examples below.
• Why did GiveWell place little weight on some papers in its analysis of spillover effects? In general, our analyses aim to estimate the impact of programs as implemented by particular charities. The goal of our spillovers analysis is to make our best guess about the size of spillover effects caused by GiveDirectly’s programs in Kenya, Uganda, and Rwanda. We are not trying to communicate an opinion on the size of spillover effects of cash transfers in other countries or in development economics more broadly. Therefore, our analysis places substantially more weight on studies that are most similar to GiveDirectly’s program on basic characteristics such as geographic location and program type. Correspondingly, we place little weight on papers that do not meet these criteria. However, we’d welcome additional information that would help us improve our future decisionmaking about which papers to put the most weight on in our analyses.
• Why did GiveWell’s analysis of spillovers focus on effects on consumption? Our cost-effectiveness models focus on key outcomes that we expect to drive the bulk of the welfare effects of a program. In the case of our spillovers analysis, we believe the two most relevant outcomes for estimating spillover effects on welfare are consumption and subjective well-being. We chose to focus on consumption effects in large part because (a) this is consistent with how we model the impacts of other programs, such as deworming, and (b) distinguishing effects on subjective well-being from effects on consumption in a way that avoids double-counting benefits was too complex to do in the time we had available. It is possible that additional work on subjective well-being measures would meaningfully change how we assess benefits of programs (for this program and potentially others). This is a question we plan to return to in the future.

As noted above, our current best guess is that negative or positive spillover effects of GiveDirectly’s cash transfers are minimal on net. However, we emphasize that our conclusion at this point is very tentative, and we hope to update our views next year if there is more public discussion or research on the areas of uncertainty highlighted in our analysis and/or if public debate about the studies covered in our report raises major issues we had not previously considered.

Details follow.

Why did GiveWell publish its analysis on spillover effects before a key paper it relied on was public?

In our analysis of the spillover effects of GiveDirectly’s cash transfer program, we place substantial weight on GiveDirectly’s “general equilibrium” (GE) study (as we noted we would do in May 2018,2“We plan to reassess the cash transfer evidence base and provide our updated conclusions in the next several months (by November 2018 at the latest). One reason that we do not plan to provide a comprehensive update sooner is that we expect upcoming midline results from GiveDirectly’s “general equilibrium” study, a large and high-quality study explicitly designed to estimate spillover effects, will play a major role in our conclusions. Results from this study are expected to be released in the next few months.” (More.) jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); prior to seeing the study’s results) because:

• it is the study with the largest sample size,
• its methodology was designed to estimate both across-village and within-village spillover effects, and
• it is a direct study of a version of GiveDirectly’s program.

The details of this study are currently private, though we were able to share the headline results and methodology when we published our report.

This represents one example of a general policy we follow, which is to be willing to compromise to some degree on transparency in order to use the best information available to us to improve the quality of our recommendations. More on the reasoning behind this policy:

• Since our recommendations affect the allocation of over $100 million each year, the value of improving our recommendations by factoring in the best information (even if private) can be high. Every November we publish updates to our recommended charities so that donors giving in December and January (when the bulk of charitable giving occurs) can act on the most up-to-date information. • We have ongoing communications with charities and researchers to learn about new information that could affect our recommendations. Private information (both positive and negative) has been important to our views on a number of occasions. Beyond the example of our spillovers analysis, early private results were key to our views on topics including: • No Lean Season in 2018 (negative result)3“In a preliminary analysis shared with GiveWell in September 2018, the researchers did not find evidence for a negative or positive impact on migration, and found no statistically significant impact on income and consumption.” (More.) jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); • Deworming in 2017 (positive result)4“We have seen preliminary, confidential results from a 15-year follow-up to Miguel and Kremer 2004. We are not yet able to discuss the results in detail, but they are broadly consistent with the findings from the 10-year follow-up analyzed in Baird et al. 2016.” (More.) jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); • Insecticide resistance in 2016 (modeling study)5“We have seen two modeling studies which model clinical malaria outcomes in areas with ITN coverage for different levels of resistance based on experimental hut trial data. Of these two studies, the most recent study we have seen is unpublished (it was shared with us privately), but we prefer it because the insecticide resistance data it draws from is more recent and more comprehensive.” (More.) jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); • Development Media International in 2015 (negative result)6“The preliminary endline results did not find any effect of DMI’s program on child mortality (it was powered to detect a reduction of 15% or more), and it found substantially less effect on behavior change than was found at midline. We cannot publicly discuss the details of the endline results we have seen, because they are not yet finalised and because the finalised results will be embargoed prior to publication, but we have informally incorporated the results into our view of DMI’s program effectiveness.” (More.) jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); • Living Goods in 2014 (positive result)7“The researchers have published an abstract on the study, and shared a more in-depth report with us. The more in-depth report is not yet cleared for publication because the authors are seeking publication in an academic journal.” (More.) jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); • Note that in all of the above cases we worked with the relevant researchers to get permission to publicly share basic information about the results we were relying on, as we did in the case of the GE study. • In all cases, we expected that full results would be made public in the future. Our understanding is that oftentimes early headline results from studies can be shared publicly while it may take substantially longer to publicly release full working papers because working papers are time-intensive to produce. We would be more hesitant to rely on a study that has been private for an unusually long period of time unless there were a good reason for it. • However, relying on private studies conflicts to some extent with our goal to be transparent. In particular, we believe two major downsides of our policy with respect to private information are (a) early private results are more likely to contain errors, and (b) we are not able to benefit from public scrutiny and discussion of the research. We would have ideally seen a robust public discussion of the GE study before we released our recommendations in November, but the timeline for the public release of GE study results did not allow that. We look forward to closely following the public debate in the future and plan to update our views based on what we learn. • Despite these limitations, we have generally found early, private results to be predictive of final, public results. This, combined with the fact that we believe private results have improved our recommendations on a number of occasions, leads us to believe that the benefits of our current policy on using private information outweigh the costs. A few other notes: • Although we provide a number of cases above in which we relied on private information, the vast majority of the key information we rely on for our charity recommendations is public. • When private information is shared with us that implies a positive update about a charity’s program, we try to be especially attentive about potential conflicts of interest. In this case, there is potential for concern because the GE study was co-authored by Paul Niehaus, Chairman of GiveDirectly. We chose not to substantially limit the weight we place on the GE study because (a) a detailed pre-analysis plan was submitted for this study, and (b) three of the four co-authors (Ted Miguel, Johannes Haushofer, and Michael Walker) do not have an affiliation with GiveDirectly. We have no reason to believe that GiveDirectly’s involvement altered the analysis undertaken. In addition, the GE study team informed us that Paul Niehaus recused himself from final decisions about what the team communicated to GiveWell. • When we published our report (about one week ago), we expected that some additional analysis from the GE study would be shared publicly soon (which we still expect). We do not yet have an exact date and do not know precisely what content will be shared (though we expect it to be similar to what was shared with us privately). Why did GiveWell place little weight on some papers in its analysis of spillover effects? Some general context on GiveWell’s research that we think is useful for understanding our approach in this case is: • We are typically estimating the impact of programs as implemented by particular charities, not aiming to publish formal meta-analyses about program areas as a whole. As noted above, we believe we should have communicated more clearly about this in our original report on spillovers and we will revise the report to clarify. • We focus our limited time on the research that we think is most likely to affect our decisions, so our style of analysis is often different from what is typically seen in academia. (We think the differences in the kind of work we do is captured well by a relevant Rachel Glennerster blog post.) Consistent with the above, the goal of our spillovers analysis was to make a best guess for the size of the spillover effect of GiveDirectly’s (GD’s) program in Kenya, Uganda, and Rwanda specifically.8This program provides$1,000 unconditional transfers and treats almost all households within target villages in Kenya and Uganda (though still treats only eligible households in Rwanda). jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We are not trying to communicate an opinion on the size of spillover effects of cash transfers in other countries or development economics more broadly. If we were trying to do the latter, we would have considered a much wider range of literature.

We expect that studies that are most similar to GD’s program on basic characteristics such as geographic location and program type will be most useful for predicting spillovers in the GD context. So, we prioritize looking at studies that 1) took place in sub-Saharan Africa, and 2) evaluate unconditional cash transfer programs (further explanation in footnote).9On (1): Our understanding is that the nature and size of spillover effects is likely to be highly dependent on the context studied, for example because the extent to which village economies are integrated might differ substantially across contexts (e.g. how close households are to larger markets outside of the village in which they live, how easily goods can be transported, etc.).
On (2): We expect that providing cash transfers conditional on behavioral choices is a fairly different intervention from providing unconditional cash transfers, and so may have different spillover effects. jQuery("#footnote_plugin_tooltip_9").tooltip({ tip: "#footnote_plugin_tooltip_text_9", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We would welcome additional engagement on this topic: that is, (a) to what extent should we believe that effects estimated in studies not meeting these criteria would apply to GD’s cash transfer programs, and (b) are there other criteria that we should have used?

A further factor that causes us to put more weight on the five studies we chose to review deeply is that they all study transfers distributed by GD, which we see as increasing their relevance to GD’s current work (though the specifics of the programs that were studied vary from GD’s current program). We believe that studies that do not meet the above criteria could affect our views on spillovers of GD’s program to some extent, but they would receive lower weight in our conclusions since they are less directly relevant to GD’s program.

We saw further review of studies that did not meet the above criteria as lower priority than a number of other analyses that we think would be more likely to shift our bottom line estimate of the spillovers of GD’s program. Even though we focused on the subset of studies most relevant to GD’s program, we were not able to combine their results to create a reasonable explicit model of spillover effects because we found that key questions were not answered by the available data (our attempt at an explicit model is in the following footnote).10We tried to create such an explicit model here (explanation here). jQuery("#footnote_plugin_tooltip_10").tooltip({ tip: "#footnote_plugin_tooltip_text_10", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); One fundamental challenge is that we are trying to apply estimates of “within-village” spillover effects to predict across-village spillover effects.11GiveDirectly treats almost all households within target villages in Kenya and Uganda (though still treats only eligible households in Rwanda). jQuery("#footnote_plugin_tooltip_11").tooltip({ tip: "#footnote_plugin_tooltip_text_11", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Additional complications are described here.

More on why we placed little weight on particular studies that Özler highlighted in his comments:12Note on terminology: In our spillovers analysis report, we talk about studies in terms of “inclusion” and “exclusion.” We may use the term “exclude” differently than it is sometimes used in, e.g., academic meta-analyses. When we say that we have excluded studies, we have typically lightly reviewed their results and placed little weight on them in our conclusions. We did not ignore them entirely, as may happen for papers excluded from an academic meta-analysis. To try to clarify this, in this blog post we have used the term “place little weight.” We will try to be attentive to this in future research that we publish. jQuery("#footnote_plugin_tooltip_12").tooltip({ tip: "#footnote_plugin_tooltip_text_12", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

• We placed little weight on the following papers in our initial analysis for the reasons given in parentheses: Angelucci & DiGiorgi 2009 (conditional transfers, study took place in Mexico), Cunha et al. 2017 (study took place in Mexico), Filmer et al. 2018 (conditional transfers, study took place in the Philippines), and Baird, de Hoop, and Özler 2013 (mix of conditional and unconditional transfers).
• In addition, the estimates of mental health effects on teenage schoolgirls in Baird, de Hoop, and Özler 2013 seem like they would be relatively less useful for predicting the impacts of spillovers from cash transfers given to households, particularly in villages where almost all households receive transfers as is often the case in GD’s program.13We expect that local spillover effects via psychological mechanisms are less likely to occur with the current spatial distribution of GD’s program. In GD’s program in Kenya and Uganda, almost all households are treated within its target villages. In addition, the majority of villages within a region are treated in a block. Baird, de Hoop, and Özler 2013 estimate spillover effects within enumeration areas (groups of several villages), and the authors believe that the “detrimental effects on the mental well-being of those randomly excluded from the program in intervention areas is consistent with the idea that an individual’s utility depends on her relative consumption (or income or status) within her peer group”, p.372. The spatial distribution of GD’s program in Kenya and Uganda makes it more likely that the majority of one’s local peer group receives the same treatment assignment. jQuery("#footnote_plugin_tooltip_13").tooltip({ tip: "#footnote_plugin_tooltip_text_13", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
Why did GiveWell’s analysis of spillovers focus on effects on consumption? Does this imply that GiveWell does not value effects on other outcomes?

Some general context on GiveWell’s research that we think is useful for understanding our approach in this case is:

• When modeling the cost-effectiveness of any program, there are typically a large number of outcomes that could be included in the model. In our analyses, we focus on the key outcomes that we expect to drive the bulk of the welfare effects of a program.
• For example, our core cost-effectiveness model primarily considers various programs’ effects on averting deaths and increasing consumption (either immediately or later in life). This means that, e.g., we do not include benefits of averting vision impairment in our cost-effectiveness model for vitamin A supplementation (in part because we expect those effects to be relatively small as a portion of the overall impact of the program).
• This does not mean that we think excluded outcomes are unimportant. We focus on the largest impacts of programs because (a) we think they are a good proxy for the overall impact of the relevant programs, and (b) having fewer outcomes simplifies our analysis, which leads to less potential for error, better comparability between programs, and a more manageable time investment in modeling.
• For a deeper assessment of which program impacts we include and exclude from our core cost-effectiveness model and why, see our model’s “Inclusion/exclusion” sheet.14We have not yet added it, but we plan to add “Subjective well-being” under the list of outcomes excluded in the “Cross-cutting / Structural” section of the sheet, since it may be relevant to all programs. jQuery("#footnote_plugin_tooltip_14").tooltip({ tip: "#footnote_plugin_tooltip_text_14", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We aim to include outcomes that can be justified by evidence, feasibly modeled, and are consistent with how we handle other program outcomes. We revisit our list of excluded outcomes periodically to assess whether such outcomes could lead to a major shift in our cost-effectiveness estimate for a particular program.

In our spillovers analysis, we applied the above principles to try to identify the key welfare effects. Among the main five studies we reviewed on spillovers, it seems like the two most relevant outcomes are consumption and subjective well-being. We chose to focus on consumption for the following reasons:

• Assessing the effects of cash transfers on consumption (rather than subjective well-being) is consistent with how we model the welfare effects of other programs that we think increase consumption on expectation, such as deworming.
• Distinguishing effects on subjective well-being from effects on consumption in order to avoid double-counting benefits was too complex to do in the time we had available. It seems intuitively likely that standards of living (proxied by consumption) affect subjective well-being. In the Haushofer and Shapiro studies and in the GE study, the spillover effects act in the same direction for both consumption and subjective well-being. We do not think it would be appropriate to simply add subjective well-being effects into our model over and above effects on consumption since that risks double-counting benefits.
• We do not have a strong argument that consumption is a more robust proxy for “true well-being” than subjective well-being, but given that consumption effects can be more easily compared across our programs we have chosen it as the default option at this point.

We hope to broadly revisit in the future whether we should be placing more weight on measures of subjective well-being across programs. It is possible that additional work on subjective well-being measures would meaningfully change how we assess benefits of programs (for this program and potentially others).

Examples of our questions about how to interpret subjective well-being effects in the cash spillovers literature include:

• In the Haushofer and Shapiro studies, how should we interpret each of the underlying components of the subjective well-being indices? For example, how does self-reported life satisfaction map onto utility versus self-reported happiness?
• In Haushofer, Reisinger, & Shapiro 2015, there is a statistically significant negative spillover effect on life-satisfaction, but there are no statistically significant effects on happiness, depression, stress, cortisol levels or the overall subjective well-being index (column (4) of Table 1). How should we interpret these findings?
Next steps
• We hope that there is more public discussion on some of the policy-relevant questions we highlighted in our report and on the other points of uncertainty highlighted throughout this post. Our conclusions on spillovers are very tentative and could be affected substantially by more analysis, so we would greatly appreciate any feedback or pointers to relevant work.15If you are aware of relevant analyses or studies that we have not covered here, please let us know at info@givewell.org. jQuery("#footnote_plugin_tooltip_15").tooltip({ tip: "#footnote_plugin_tooltip_text_15", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
• We are planning to follow up with Dr. Özler to better understand his views on spillover effects of cash transfers. We have appreciated his previous blog posts on this topic and want to ensure we are getting multiple perspectives on the relevant issues.

Notes   [ + ]

1. ↑ For more context on this topic, see our May 2018 blog post. 2. ↑ “We plan to reassess the cash transfer evidence base and provide our updated conclusions in the next several months (by November 2018 at the latest). One reason that we do not plan to provide a comprehensive update sooner is that we expect upcoming midline results from GiveDirectly’s “general equilibrium” study, a large and high-quality study explicitly designed to estimate spillover effects, will play a major role in our conclusions. Results from this study are expected to be released in the next few months.” (More.) 3. ↑ “In a preliminary analysis shared with GiveWell in September 2018, the researchers did not find evidence for a negative or positive impact on migration, and found no statistically significant impact on income and consumption.” (More.) 4. ↑ “We have seen preliminary, confidential results from a 15-year follow-up to Miguel and Kremer 2004. We are not yet able to discuss the results in detail, but they are broadly consistent with the findings from the 10-year follow-up analyzed in Baird et al. 2016.” (More.) 5. ↑ “We have seen two modeling studies which model clinical malaria outcomes in areas with ITN coverage for different levels of resistance based on experimental hut trial data. Of these two studies, the most recent study we have seen is unpublished (it was shared with us privately), but we prefer it because the insecticide resistance data it draws from is more recent and more comprehensive.” (More.) 6. ↑ “The preliminary endline results did not find any effect of DMI’s program on child mortality (it was powered to detect a reduction of 15% or more), and it found substantially less effect on behavior change than was found at midline. We cannot publicly discuss the details of the endline results we have seen, because they are not yet finalised and because the finalised results will be embargoed prior to publication, but we have informally incorporated the results into our view of DMI’s program effectiveness.” (More.) 7. ↑ “The researchers have published an abstract on the study, and shared a more in-depth report with us. The more in-depth report is not yet cleared for publication because the authors are seeking publication in an academic journal.” (More.) 8. ↑ This program provides $1,000 unconditional transfers and treats almost all households within target villages in Kenya and Uganda (though still treats only eligible households in Rwanda). 9. ↑ On (1): Our understanding is that the nature and size of spillover effects is likely to be highly dependent on the context studied, for example because the extent to which village economies are integrated might differ substantially across contexts (e.g. how close households are to larger markets outside of the village in which they live, how easily goods can be transported, etc.). On (2): We expect that providing cash transfers conditional on behavioral choices is a fairly different intervention from providing unconditional cash transfers, and so may have different spillover effects. 10. ↑ We tried to create such an explicit model here (explanation here). 11. ↑ GiveDirectly treats almost all households within target villages in Kenya and Uganda (though still treats only eligible households in Rwanda). 12. ↑ Note on terminology: In our spillovers analysis report, we talk about studies in terms of “inclusion” and “exclusion.” We may use the term “exclude” differently than it is sometimes used in, e.g., academic meta-analyses. When we say that we have excluded studies, we have typically lightly reviewed their results and placed little weight on them in our conclusions. We did not ignore them entirely, as may happen for papers excluded from an academic meta-analysis. To try to clarify this, in this blog post we have used the term “place little weight.” We will try to be attentive to this in future research that we publish. 13. ↑ We expect that local spillover effects via psychological mechanisms are less likely to occur with the current spatial distribution of GD’s program. In GD’s program in Kenya and Uganda, almost all households are treated within its target villages. In addition, the majority of villages within a region are treated in a block. Baird, de Hoop, and Özler 2013 estimate spillover effects within enumeration areas (groups of several villages), and the authors believe that the “detrimental effects on the mental well-being of those randomly excluded from the program in intervention areas is consistent with the idea that an individual’s utility depends on her relative consumption (or income or status) within her peer group”, p.372. The spatial distribution of GD’s program in Kenya and Uganda makes it more likely that the majority of one’s local peer group receives the same treatment assignment. 14. ↑ We have not yet added it, but we plan to add “Subjective well-being” under the list of outcomes excluded in the “Cross-cutting / Structural” section of the sheet, since it may be relevant to all programs. 15. ↑ If you are aware of relevant analyses or studies that we have not covered here, please let us know at info@givewell.org. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } } The post Response to concerns about GiveWell’s spillovers analysis appeared first on The GiveWell Blog. ### New research on cash transfers Fri, 05/04/2018 - 12:21 Summary • There has been a good deal of discussion recently about new research on the effects of cash transfers, beginning with a post by economist Berk Özler on the World Bank’s Development Impact blog. We have not yet fully reviewed the new research, but wanted to provide a preliminary update for our followers about our plans for reviewing this research and how it might affect our views of cash transfers, a program implemented by one of our top charities, GiveDirectly. • In brief, the new research suggests that cash transfers may be less effective than we previously believed in two ways. First, cash transfers may have substantial negative effects on non-recipients who live near recipients (“negative spillovers”). Second, the benefits of cash transfers may fade quickly. • We plan to reassess the cash transfer evidence base and provide our updated conclusions in the next several months (by November 2018 at the latest). One reason that we do not plan to provide a comprehensive update sooner is that we expect upcoming midline results from GiveDirectly’s “general equilibrium” study, a large and high-quality study explicitly designed to estimate spillover effects, will play a major role in our conclusions. Results from this study are expected to be released in the next few months. • Our best guess is that we will reduce our estimate of the cost-effectiveness of cash transfers to some extent, but will likely continue to recommend GiveDirectly. However, major updates to our current views, either in the negative or positive direction, seem possible. More detail below. Background GiveDirectly, one of our top charities, provides unconditional cash transfers to very poor households in Kenya, Uganda, and Rwanda. Several new studies have recently been released that assess the impact of unconditional cash transfers, including a three-year follow-up study (Haushofer and Shapiro 2018, henceforth referred to as “HS 2018”) on the impact of transfers that were provided by GiveDirectly. Berk Özler, a senior economist at the World Bank, summarized some of this research in two posts on the World Bank Development Impact blog (here and here), noting that the results imply that cash transfers may be less effective than proponents previously believed. In particular, Özler raises the concerns that cash may: 1. Have negative “spillovers”: i.e., negative effects on households that did not receive transfers but that live near recipient households. 2. Have quickly-fading benefits: i.e., the standard of living for recipient households may converge to be similar to non-recipient households within a few years of receiving transfers. Below, we discuss the topics of spillover effects and the duration of benefits of cash transfers in more detail, as well as some other considerations relevant to the effectiveness of cash transfers. In brief: • If substantial spillover effects exist, they have the potential to significantly affect our cost-effectiveness estimates for cash transfers. We are uncertain what we will conclude about spillover effects of cash transfers after deeply reviewing all relevant new literature, but we expect that upcoming midline results from GiveDirectly’s “general equilibrium” study will play a major role in our conclusions. Our best guess is that the general equilibrium study and other literature will not imply that GiveDirectly’s program has large negative spillovers, but we remain open to the possibility that we should substantially negatively update our views after reviewing the relevant literature. • Several new studies seem to find that cash may have little effect on recipients’ standard of living beyond the first year after receiving a transfer. Our best guess is that after reviewing the relevant research in more detail we will decrease our estimate of the cost-effectiveness of cash transfers to some extent. In the worst (unlikely) case, this factor could lead us to believe that cash is about 1.5-2x less cost-effective than we currently do. Spillovers Negative spillovers of cash transfers have the potential to lead us to majorly revise our estimates of the effects of cash; we currently assume that cash does not have major negative or positive spillover effects. At this point, we are uncertain what we will conclude about the likely spillover effects of cash after reviewing all relevant new literature, including GiveDirectly’s forthcoming “general equilibrium” study. Our best guess is that GiveDirectly’s current program does not have large spillover effects, but it seems plausible that we could ultimately conclude that cash either has meaningful negative spillovers or positive spillovers. We will not rehash the methodological details and estimated effect sizes of HS 2018 in this post. For a basic understanding of the findings and methodological issues, we recommend reading Özler’s posts, the Center for Global Development’s Justin Sandefur’s post, GiveDirectly’s latest post, or Haushofer and Shapiro’s response to Özler’s posts. The basic conclusions that we draw from this research are: • Under one interpretation of its findings, HS 2018 measures negative spillover effects that could outweigh the positive effects of cash transfers.1From Sandefur’s post: “Households who had been randomly selected to receive cash were much better off than their neighbors who didn’t. They had$400 more assets—roughly the size of the original transfer, with all figures from here on out in PPP terms—and about $47 higher consumption each month. It looked like an amazing success. “But when Haushofer and Shapiro compared the whole sample in these villages—half of whom had gotten cash, half of whom hadn’t—they looked no different than a random sample of households in control villages. In fact, their consumption was about$6 per month less ($211 versus$217 a month).

“There are basically two ways to resolve this paradox:

“1) Good data, bad news. Cash left recipients only modestly better off after three years (lifting them from $217 to$235 in monthly consumption), and instead hurt their neighbors (dragging them down from $217 to$188 in monthly consumption). Taking the data at face value, this is the most straightforward interpretation of the results.

“2) Bad data, good news. Alternatively, the $47 gap in consumption between recipients and their neighbors is driven by gains to the former not losses to the latter. The estimates of negative side-effects on neighbors are driven by comparisons with control villages where—if you get into the weeds of the paper—it appears sampling was done differently than in treatment villages. (In short, the$217 isn’t reliable.)” jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
• We do not yet have a strong view on how likely it is that the negative interpretation of HS 2018’s findings is correct. This would require having a deeper understanding of what we should believe about a number of key methodological issues in HS 2018 (see following footnote for two examples).2One methodological issue is how to deal with attrition, as discussed in Haushofer and Shapiro 2018, Pg. 9: “However, there is a statistically significant difference in attrition levels for households in control villages relative to households in treatment villages from endline 1 to endline 2: 6 percentage points more pure control households were not found at endline 2 relative to either group of households in treatment villages. In the analysis of across-village treatment effects and spillover effects we use Lee bounds to deal with this differential attrition; details are given below.”

Another potential issue as described by Özler’s post: “The short-term impacts in Haushofer and Shapiro (2016) were calculated using within-village comparisons, which was a big problem for an intervention with possibility of spillovers, on which the authors had to do a lot of work earlier (see section IV.B in that paper) and in the recent paper. They got around this problem by arguing that spillover effects were small and insignificant. Of course, then came the working paper on negative spillovers on psychological wellbeing mentioned above and now, the spillover effects look sustained and large and unfortunately negative on multiple domains three years post transfers.

“The authors estimated program impacts by comparing T [treatment group] to S [spillover group], instead of the standard comparison of T to C [control group], in the 2016 paper because of a study design complication: researchers randomly selected control villages, but did not collect baseline data in these villages. The lack of baseline data in the control group is not just a harmless omission, as in ‘we lose some power, no big deal.’ Because there were eligibility criteria for receiving cash, but households were sampled a year later, no one can say for certain if the households sampled in the pure control villages at follow-up are representative of the would-be eligible households at baseline.

“So, quite distressingly, we now have two choices to interpret the most recent findings:

“1) We either believe the integrity of the counterfactual group in the pure control villages, in which case the negative spillover effects are real, implying that total causal effects comparing treated and control villages are zero at best. Furthermore, there are no ITT [intention to treat] effects on longer-term welfare of the beneficiaries themselves – other than an increase in the level of assets owned. In this scenario, it is harder to retain confidence in the earlier published impact findings that were based on within-village comparisons – although it is possible to believe that the negative spillovers are a longer-term phenomenon that truly did not exist at the nine-month follow-up.

“2) Or, we find the pure control sample suspect, in which case we have an individually randomized intervention and need to assume away spillover effects to believe the ITT estimates.” jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); HS 2018 reports that the potential bias introduced by methodological issues may be able to explain much of the estimated spillover effects.3Haushofer and Shapiro 2018, Pgs. 24-25: “These results appear to differ from those found in the initial endline, where we found positive spillover effects on female empowerment, but no spillover effects on other dimensions. However, the present estimates are potentially affected by differential attrition from endline 1 to endline 2: as described above, the pure control group showed significantly greater attrition than both treatment and spillover households between these endlines. To assess the potential impact of attrition, we bound the spillover effects using Lee bounds (Table 8). This analysis suggests that differential attrition may account for several of these spillover effects. Specifically, for health, education, psychological well-being, and female empowerment, the Lee bounds confidence intervals include zero for all sample definitions. For asset holdings, revenue, and food security, they include zero in two of the three sample definitions. Only for expenditure do the Lee bounds confidence intervals exclude zero across all sample definitions. Thus, we find some evidence for spillover effects when using Lee bounds, although most of them are not significantly different from zero after bounding for differential attrition across treatment groups.” jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
• The mechanism for what may have caused large negative spillovers (if they exist) in HS 2018 is uncertain, though the authors provide some speculation (see footnote).4Haushofer and Shapiro 2018, Pg. 3: “We do not have conclusive evidence of the mechanism behind spillovers, but speculate it could be due to the sale of productive assets by spillover households to treatment households, which in turn reduces consumption among the spillover group. Though not always statistically different from zero, we do see suggestive evidence of negative spillover effects on the value of productive assets such as livestock, bicycles, motorbikes and appliances. We note that GiveDirectly’s current operating model is to provide transfers to all eligible recipients in each village (within village randomization was conducted only for the purpose of research), which may mitigate any negative spillover effects.” jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We would increase our credence in the existence of negative spillover effects if there were strong evidence for a particular mechanism.

One further factor that complicates application of HS 2018’s estimate of spillover effects is that GiveDirectly’s current program is substantially different from the version of its program that was studied in HS 2018. GiveDirectly now provides $1,000 transfers to almost all households in its target villages in Uganda and Kenya; the intervention studied by HS 2018 predominantly involved providing ~$287 transfers to about half of eligible (i.e., very poor) households within treatment villages, and HS 2018 measured spillover effects on eligible households that did not receive transfers.5See this section of our cash transfers intervention report. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); GiveDirectly asked us to note that it now defaults to village-level (instead of within-village) randomization for the studies it participates in, barring exceptional circumstances. Since GiveDirectly’s current program provides transfers to almost all households in its target villages, spillovers of its program may largely operate across villages rather than within villages. These changes to the program and the spillover population of interest may lead to substantial differences in estimated spillover effects.

Fortunately, GiveDirectly is running a large (~650 villages) randomized controlled trial of an intervention similar to its current program that is explicitly designed to estimate the spillover (or “general equilibrium”) effects of GiveDirectly’s program.6From the registration for “General Equilibrium Effects of Cash Transfers in Kenya”: “The study will take place across 653 villages in Western Kenya. Villages are randomly allocated to treatment or control status. In treatment villages, GiveDirectly enrolls and distributes cash transfers to households that meet its eligibility criteria. In order to generate additional spatial variation in treatment density, groups of villages are assigned to high or low saturation. In high saturation zones, 2/3 of villages are targeted for treatment, while in low saturation zones, 1/3 of villages are targeted for treatment. The randomized assignment to treatment status and the spatial variation in treatment intensity will be used to identify direct and spillover effects of cash transfers.”

Note that this study will evaluate a variant of GiveDirectly’s program that is different from its current program in that it will not provide transfers to almost all households in target villages. The study will estimate the spillover effects of cash transfers on ineligible (i.e., slightly wealthier) households in treatment villages, among other populations. Since GiveDirectly’s standard program now provides transfers to almost all households in its target villages, estimates of effects on ineligible households may need to be extrapolated to other populations of interest (e.g., households in non-target villages) to be most relevant to GiveDirectly’s current program. jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Midline results from this study are expected to be released in the next few months.

Since we expect GiveDirectly’s general equilibrium study to play a large role in our view of spillovers, we expect that we will not publish an overview of the cash spillovers literature until we’ve had a chance to review its results. However, we see the potential for negative spillover effects of cash as very concerning and it is a high-priority research question for us; we plan to publish a detailed update that incorporates HS 2018, previous evidence for negative spillovers (such as studies on inflation and happiness), the general equilibrium study, and any other relevant literature in time for our November 2018 top charity recommendations at the latest.

Duration of benefits

Several new studies seem to find that cash may have little effect on recipients’ standard of living beyond the first year after receiving a transfer. Our best guess is that after reviewing the relevant research in more detail we will decrease our estimate of the cost-effectiveness of cash to some extent. In the worst (unlikely) case, this could lead us to believe that cash is about 1.5-2x less cost-effective than we currently do.

In our current cost-effectiveness analysis for cash transfers, we mainly consider two types of benefits that households experience due to receiving a transfer:

1. Increases in short-term consumption (i.e., immediately after receiving the transfer, very poor households are able to spend money on goods such as food).
2. Increases in medium-term consumption (i.e., recipients may invest some of their cash transfer in ways that lead them to have a higher standard of living in the 1-20 years after first receiving the transfer).

Potential spillover effects aside, our cost-effectiveness estimate for cash has a fairly stable lower bound because we place substantial value on increasing short-term consumption for very poor people, and providing cash allows for more short-term consumption almost by definition. In particular:

• Our current estimates are consistent with assuming little medium-term benefit of cash transfers. We estimate that about 60% of a typical transfer is spent on short-term goods such as eating more food, and count this as about 40-60% of the benefits of the program.7For our estimate of the proportion of the benefits of cash transfers that come from short-term consumption increases, see row 30 of the “Cash” sheet in our 2018 cost-effectiveness model.

For our estimate of the proportion of transfers that is spent on short-term consumption, we rely on results from GiveDirectly’s randomized controlled trial, which shows investments of $505.94 (USD PPP) (within villages, or$601.88 across villages) on a transfer of $1,525 USD PPP, or about one-third of the total. See Pg. 117 here and Pg. 1 here for total transfer size. jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); If we were to instead assume that 100% of the transfer was spent on short-term consumption (i.e., none of it was invested), our estimate of the cost-effectiveness of cash would become about 10-30% worse.8See a version of our cost-effectiveness analysis in which we made this assumption here. The calculations in row 35 of the “Cash” tab show how assuming that 0% of the transfer is invested would affect staff members’ bottom line estimates. jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We think using the 100% short-term consumption estimate may be a reasonable and robust way to model the lower bound of effects of cash given various measurement challenges (discussed below). • Nevertheless, our previous estimates of the medium-term benefits of cash transfers may have been too optimistic. Based partially on a speculative model of the investment returns of iron roofs (a commonly-purchased asset for GiveDirectly recipients), most staff assumed that about 40% of a transfer will be invested, and that those investments will lead to roughly 10% greater consumption for 10-15 years.9See rows 5, 8, and 14, “Cash” sheet, 2018 Cost-Effectiveness Analysis – Version 1. jQuery("#footnote_plugin_tooltip_9").tooltip({ tip: "#footnote_plugin_tooltip_text_9", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Some new research discussed in Özler’s first post suggests that there may be little return on investment from cash transfers within 2-4 years after the transfer, though the new evidence is somewhat mixed (see footnote).10See this section of Özler’s post: “This new paper and Blattman’s (forthcoming) work mentioned above join a growing list of papers finding short-term impacts of unconditional cash transfers that fade away over time: Hicks et al. (2017), Brudevold et al. (2017), Baird et al. (2018, supplemental online materials). In fact, the final slide in Hicks et al. states: ‘Cash effects dissipate quickly, similar to Brudevold et al. (2017), but different to Blattman et al. (2014).’ If only they were presenting a couple of months later…” See also two other recent papers that find positive effects of cash transfers beyond the first year: Handa et al. 2018 and Parker and Vogl 2018. The latter finds intergenerational effects of a conditional cash transfer program in Mexico, so may be less relevant to GiveDirectly’s program. jQuery("#footnote_plugin_tooltip_10").tooltip({ tip: "#footnote_plugin_tooltip_text_10", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Additionally, under the negative interpretation of HS 2018’s results, it finds that cash transfers did not have positive consumption effects for recipients three years post-transfer, though it finds a ~40% increase in assets for treatment households (even in the negative interpretation).11Haushofer and Shapiro 2018, Abstract: “Comparing recipient households to non-recipients in distant villages, we find that transfer recipients have 40% more assets (USD 422 PPP) than control households three years after the transfer, equivalent to 60% of the initial transfer (USD 709 PPP).” Haushofer and Shapiro 2018, Pg. 28: “Since we have outcome data measured in the short run (~9 months after the beginning of the transfers) and in the long-run (˜3 years after the beginning of transfers), we test equality between short and long-run effects…Results are reported in Table 9. Focusing on the within-village treatment effects, we find no evidence for differential effects at endline 2 compared to endline 1, with the exception of assets, which show a significantly larger treatment effect at endline 2 than endline 1. However, this effect is largely driven by spillovers; for across-village treatment effects, we cannot reject equality of the endline 1 and endline 2 outcomes. This is true for all variables in the across-village treatment effects except for food security and psychological well-being, which show a smaller treatment effect at endline 2 compared to endline 1. Thus, we find some evidence for decreasing treatment effects over time, but for most outcome variables, the endline 1 and 2 outcomes are similar.” jQuery("#footnote_plugin_tooltip_11").tooltip({ tip: "#footnote_plugin_tooltip_text_11", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Note that any benefits from owning iron roofs were not factored in to the consumption estimates in HS 2018.12Haushofer and Shapiro 2018, pgs. 32-33: “Total consumption…Omitted: Durables expenditure, house expenditure (omission not pre-specified for endline 1 analysis)” jQuery("#footnote_plugin_tooltip_12").tooltip({ tip: "#footnote_plugin_tooltip_text_12", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); If we imagine the potential worst case scenario implied by these results and assume that the ~40% of a cash transfer that is invested has zero benefits, our cost-effectiveness estimate would get about 2x worse. Our best guess is that we’ll decrease our estimate for the medium-term effects of cash to some extent, though we’re unsure by how much. Challenging questions we’ll need to consider in order to arrive at a final estimate include: • If we continue to assume that about 40% of transfers are invested, and that those investments do not lead to any future gains in consumption, then we are effectively assuming that money spent on investments is wasted. Is this an accurate reflection of reality, i.e. are recipients failing to invest transfers in a beneficial manner? • Is our cost-effectiveness model using a reasonable framework for estimating recipients’ standard of living over time? Currently, we only estimate cash’s effects on consumption. However, assets such as iron roofs may provide an increase in standard of living for multiple years even if they do not raise consumption. How, if at all, should we factor this into our estimates? • GiveDirectly’s cash transfer program differs in many ways from other programs that have been the subject of impact evaluations. For example, GiveDirectly provides large, one-time transfers whereas many government cash transfers provide smaller ongoing support to poor families. How should we apply new literature on other kinds of cash programs to our estimates of the effects of GiveDirectly? Next steps We plan to assess all literature relevant to the impact of cash transfers and provide an update on our view on the nature of spillover effects, duration of benefits, and other relevant issues for our understanding of cash transfers and their cost-effectiveness in time for our November 2018 top charity recommendations at the latest. Notes [ + ] 1. ↑ From Sandefur’s post: “Households who had been randomly selected to receive cash were much better off than their neighbors who didn’t. They had$400 more assets—roughly the size of the original transfer, with all figures from here on out in PPP terms—and about $47 higher consumption each month. It looked like an amazing success. “But when Haushofer and Shapiro compared the whole sample in these villages—half of whom had gotten cash, half of whom hadn’t—they looked no different than a random sample of households in control villages. In fact, their consumption was about$6 per month less ($211 versus$217 a month).

“There are basically two ways to resolve this paradox:

“1) Good data, bad news. Cash left recipients only modestly better off after three years (lifting them from $217 to$235 in monthly consumption), and instead hurt their neighbors (dragging them down from $217 to$188 in monthly consumption). Taking the data at face value, this is the most straightforward interpretation of the results.

“2) Bad data, good news. Alternatively, the $47 gap in consumption between recipients and their neighbors is driven by gains to the former not losses to the latter. The estimates of negative side-effects on neighbors are driven by comparisons with control villages where—if you get into the weeds of the paper—it appears sampling was done differently than in treatment villages. (In short, the$217 isn’t reliable.)” 2. ↑ One methodological issue is how to deal with attrition, as discussed in Haushofer and Shapiro 2018, Pg. 9: “However, there is a statistically significant difference in attrition levels for households in control villages relative to households in treatment villages from endline 1 to endline 2: 6 percentage points more pure control households were not found at endline 2 relative to either group of households in treatment villages. In the analysis of across-village treatment effects and spillover effects we use Lee bounds to deal with this differential attrition; details are given below.”

Another potential issue as described by Özler’s post: “The short-term impacts in Haushofer and Shapiro (2016) were calculated using within-village comparisons, which was a big problem for an intervention with possibility of spillovers, on which the authors had to do a lot of work earlier (see section IV.B in that paper) and in the recent paper. They got around this problem by arguing that spillover effects were small and insignificant. Of course, then came the working paper on negative spillovers on psychological wellbeing mentioned above and now, the spillover effects look sustained and large and unfortunately negative on multiple domains three years post transfers.

“The authors estimated program impacts by comparing T [treatment group] to S [spillover group], instead of the standard comparison of T to C [control group], in the 2016 paper because of a study design complication: researchers randomly selected control villages, but did not collect baseline data in these villages. The lack of baseline data in the control group is not just a harmless omission, as in ‘we lose some power, no big deal.’ Because there were eligibility criteria for receiving cash, but households were sampled a year later, no one can say for certain if the households sampled in the pure control villages at follow-up are representative of the would-be eligible households at baseline.

“So, quite distressingly, we now have two choices to interpret the most recent findings:

“1) We either believe the integrity of the counterfactual group in the pure control villages, in which case the negative spillover effects are real, implying that total causal effects comparing treated and control villages are zero at best. Furthermore, there are no ITT [intention to treat] effects on longer-term welfare of the beneficiaries themselves – other than an increase in the level of assets owned. In this scenario, it is harder to retain confidence in the earlier published impact findings that were based on within-village comparisons – although it is possible to believe that the negative spillovers are a longer-term phenomenon that truly did not exist at the nine-month follow-up.

“2) Or, we find the pure control sample suspect, in which case we have an individually randomized intervention and need to assume away spillover effects to believe the ITT estimates.” 3. ↑ Haushofer and Shapiro 2018, Pgs. 24-25: “These results appear to differ from those found in the initial endline, where we found positive spillover effects on female empowerment, but no spillover effects on other dimensions. However, the present estimates are potentially affected by differential attrition from endline 1 to endline 2: as described above, the pure control group showed significantly greater attrition than both treatment and spillover households between these endlines. To assess the potential impact of attrition, we bound the spillover effects using Lee bounds (Table 8). This analysis suggests that differential attrition may account for several of these spillover effects. Specifically, for health, education, psychological well-being, and female empowerment, the Lee bounds confidence intervals include zero for all sample definitions. For asset holdings, revenue, and food security, they include zero in two of the three sample definitions. Only for expenditure do the Lee bounds confidence intervals exclude zero across all sample definitions. Thus, we find some evidence for spillover effects when using Lee bounds, although most of them are not significantly different from zero after bounding for differential attrition across treatment groups.” 4. ↑ Haushofer and Shapiro 2018, Pg. 3: “We do not have conclusive evidence of the mechanism behind spillovers, but speculate it could be due to the sale of productive assets by spillover households to treatment households, which in turn reduces consumption among the spillover group. Though not always statistically different from zero, we do see suggestive evidence of negative spillover effects on the value of productive assets such as livestock, bicycles, motorbikes and appliances. We note that GiveDirectly’s current operating model is to provide transfers to all eligible recipients in each village (within village randomization was conducted only for the purpose of research), which may mitigate any negative spillover effects.” 5. ↑ See this section of our cash transfers intervention report. 6. ↑ From the registration for “General Equilibrium Effects of Cash Transfers in Kenya”: “The study will take place across 653 villages in Western Kenya. Villages are randomly allocated to treatment or control status. In treatment villages, GiveDirectly enrolls and distributes cash transfers to households that meet its eligibility criteria. In order to generate additional spatial variation in treatment density, groups of villages are assigned to high or low saturation. In high saturation zones, 2/3 of villages are targeted for treatment, while in low saturation zones, 1/3 of villages are targeted for treatment. The randomized assignment to treatment status and the spatial variation in treatment intensity will be used to identify direct and spillover effects of cash transfers.”

Note that this study will evaluate a variant of GiveDirectly’s program that is different from its current program in that it will not provide transfers to almost all households in target villages. The study will estimate the spillover effects of cash transfers on ineligible (i.e., slightly wealthier) households in treatment villages, among other populations. Since GiveDirectly’s standard program now provides transfers to almost all households in its target villages, estimates of effects on ineligible households may need to be extrapolated to other populations of interest (e.g., households in non-target villages) to be most relevant to GiveDirectly’s current program. 7. ↑ For our estimate of the proportion of the benefits of cash transfers that come from short-term consumption increases, see row 30 of the “Cash” sheet in our 2018 cost-effectiveness model.

For our estimate of the proportion of transfers that is spent on short-term consumption, we rely on results from GiveDirectly’s randomized controlled trial, which shows investments of $505.94 (USD PPP) (within villages, or$601.88 across villages) on a transfer of $1,525 USD PPP, or about one-third of the total. See Pg. 117 here and Pg. 1 here for total transfer size. 8. ↑ See a version of our cost-effectiveness analysis in which we made this assumption here. The calculations in row 35 of the “Cash” tab show how assuming that 0% of the transfer is invested would affect staff members’ bottom line estimates. 9. ↑ See rows 5, 8, and 14, “Cash” sheet, 2018 Cost-Effectiveness Analysis – Version 1. 10. ↑ See this section of Özler’s post: “This new paper and Blattman’s (forthcoming) work mentioned above join a growing list of papers finding short-term impacts of unconditional cash transfers that fade away over time: Hicks et al. (2017), Brudevold et al. (2017), Baird et al. (2018, supplemental online materials). In fact, the final slide in Hicks et al. states: ‘Cash effects dissipate quickly, similar to Brudevold et al. (2017), but different to Blattman et al. (2014).’ If only they were presenting a couple of months later…” See also two other recent papers that find positive effects of cash transfers beyond the first year: Handa et al. 2018 and Parker and Vogl 2018. The latter finds intergenerational effects of a conditional cash transfer program in Mexico, so may be less relevant to GiveDirectly’s program. 11. ↑ Haushofer and Shapiro 2018, Abstract: “Comparing recipient households to non-recipients in distant villages, we find that transfer recipients have 40% more assets (USD 422 PPP) than control households three years after the transfer, equivalent to 60% of the initial transfer (USD 709 PPP).” Haushofer and Shapiro 2018, Pg. 28: “Since we have outcome data measured in the short run (~9 months after the beginning of the transfers) and in the long-run (˜3 years after the beginning of transfers), we test equality between short and long-run effects…Results are reported in Table 9. Focusing on the within-village treatment effects, we find no evidence for differential effects at endline 2 compared to endline 1, with the exception of assets, which show a significantly larger treatment effect at endline 2 than endline 1. However, this effect is largely driven by spillovers; for across-village treatment effects, we cannot reject equality of the endline 1 and endline 2 outcomes. This is true for all variables in the across-village treatment effects except for food security and psychological well-being, which show a smaller treatment effect at endline 2 compared to endline 1. Thus, we find some evidence for decreasing treatment effects over time, but for most outcome variables, the endline 1 and 2 outcomes are similar.” 12. ↑ Haushofer and Shapiro 2018, pgs. 32-33: “Total consumption…Omitted: Durables expenditure, house expenditure (omission not pre-specified for endline 1 analysis)” function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } } The post New research on cash transfers appeared first on The GiveWell Blog. ### Our top charities for giving season 2017 Mon, 11/27/2017 - 14:33 This year, we added two new top charities, Evidence Action’s No Lean Season program and Helen Keller International’s vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action’s Dispensers for Safe Water program to our list of standout charities. We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do. Our top charities and recommendations for donors, in brief Top charities We now have nine top charities. They are: Standout charities We also provide a list of standout charities. We believe they are implementing programs that are evidence-backed and may be extremely cost-effective. However, we do not feel as confident in the impact of these organizations as we do in our top charities. Conference call to discuss recommendations We are planning to hold a conference call at 1:30pm ET/10:30am PT on Thursday, November 30 to discuss our charity recommendations and answer your questions. If you’d like to join the call, please register using this online form. If you can’t make this date, but would be interested in joining another call at a later date, please indicate this on the registration form. Additional details and explanation Below, we provide: • An explanation of changes to our recommended charity list and of major changes to our review process in the past year that are not specific to any one organization. More • A discussion of our approach to determining how much funding charities can use effectively (“room for more funding”) and our ranking of charities’ funding gaps. More • Reasoning behind how we have ranked charities’ funding gaps. More • Details about each of our new top charities, including an overview of what we know about their work and our understanding of their funding needs. More • Details about each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year, and our understanding of their funding needs. More • A brief overview of each of our standout charities. More • The process we followed that led to these recommendations. More • An update on giving to support GiveWell’s operations versus giving to our top charities. More Major changes in the last 12 months Major changes to our recommended charities list and review process over the past year include: • Overall, we believe our top charities are able to absorb more funding than they could in previous years. This is a result both of recent additions to the top charities list with large funding gaps (particularly Malaria Consortium) as well as expansion by top charities that have been on the list for a longer time (particularly Deworm the World and AMF). We expect overall “room for more funding” to continue to expand as we gain more confidence in recently-added top charities and continue to add new top charities, particularly through GiveWell Incubation Grants, our program to grow the pipeline of potential future top charities and improve our understanding of our current top charities. • We added two new programs to our list of top charities: vitamin A supplementation (VAS) and seasonal migration subsidies. We have not previously recommended charities that work on these programs. We had considered VAS a priority program for a number of years but had not found an organization that was able to answer our key questions. While we have some remaining questions, we can now make a strong case for supporting HKI’s work on VAS. We initially supported No Lean Season through GiveWell’s Incubation Grants program. No Lean Season is the first organization we have added to our top charity list through our Incubation Grants program. • Last year, the charities we recommended on the margin were estimated to be about three times as cost-effective as unconditional cash transfers, the program implemented by top charity GiveDirectly. This year, we believe that the charities we are recommending on the margin are about six times as cost-effective as cash transfers. For the most part, this change was due to (a) a series of small adjustments to our cost-effectiveness model and (b) changes in which individuals contribute to the model and the values entered into the model by these and other contributors. We now feel fairly confident that there will be large amounts of room for more funding in this range. As more time has passed without identifying opportunities that are considerably more cost-effective than this, we have become more pessimistic about finding such opportunities. Our current best guess is that, if they exist, they will be in the area of policy advocacy in developing countries, on issues like lead regulation and tobacco taxation. We intend to do further research in those areas. • We made a significant change to our cost-effectiveness analysis to more formally incorporate adjustments for the way in which our top charities’ funding affects funding from other sources by (a) attracting more resources to the programs they work on (e.g., governments contributing staff time to support implementation of the programs) or (b) displacing resources that would have otherwise supported the programs. We will be writing more about this in a future post. • We continued to analyze the complex evidence base for deworming (treating intestinal parasites), the program implemented by four out of our nine top charities. At the end of 2016, David Roodman, a Senior Advisor to GiveWell, conducted a detailed review of the core evidence underlying our deworming recommendation (blog posts here and here). This year, we saw new follow-up results on the main study that leads us to recommend deworming, which continued to show similar long-term impacts of deworming on adult earnings as were estimated previously. Further investigation and updates based on new data led us to believe that two deworming studies (Croke 2014 and Bleakley 2007) no longer provide substantial support for the theory that deworming has long-term impacts. We plan to write more about this in the future. All together, this work led us to the same conclusion about deworming: that it is a reasonable bet to take based on its strong cost-effectiveness (which incorporates our uncertainty about the impact). Room for more funding analysis Types of funding gaps In the last two years, we used a framework of “capacity funding” and “execution levels” to compare funding gaps (unfilled funding needs) across charities. This framework was intended to capture whether funding would enable a charity to expand or grow in important ways and how likely it was, in our estimation, that each top charity would be constrained by funding in the next year. We developed this approach in response to a situation where we expected to direct more funding to several of our top charities than they would be able to use (commit or spend) in that year. We used capacity funding to describe opportunities to increase the amount of funding a charity might be able to absorb in the future (by, say, investing in expanding to a new location) and execution levels to describe the likelihood, down to the 5 percent level, that a charity would be able to make use of additional funding before encountering non-funding bottlenecks to their work. This year, because we have added new top charities and most of our other top charities have more room for more funding than in previous years, we expect that the funding we will direct to each organization will not reach the level where they will encounter significant non-funding bottlenecks. As a result, we have moved away from describing capacity funding and execution levels. Ranking funding gaps The first million dollars to a charity can have a very different impact from the 20th million dollars. Accordingly, we have created a ranking of individual funding gaps that accounts for our best guess of the impact of additional funds at each level. The below table lays out our ranking of funding gaps, up to$75.7 million in total funding. We expect Good Ventures to give $75 million to GiveWell’s top charities this year, so this table is our recommendation to Good Ventures, plus the allocation of funding that GiveWell holds to allocate at its discretion (currently$0.7 million). We then discuss our recommendation for all other donors.

The Open Philanthropy Project, which was incubated at GiveWell but is now a separate organization, plans to write more soon about the reasons for Good Ventures increasing its support of GiveWell top charities from $50 million last year to$75 million this year. In short, the amount was based on discussions about how to allocate funding across time and across cause areas. It was not set based on the total size of top charities’ funding gaps or the projection of what others would give.

Charity Description Amount (millions) All top charities Incentive grants: $2.5 million per charity 22.5 All standout charities Standout grants:$100,000 per charity 0.7 Deworm the World Funding gaps in India and Kenya over the next three years (including central costs) 3.0 Helen Keller International Funding gaps over three years in Burkina Faso, Mali, and Guinea—countries that have missed recent vitamin A campaigns due to lack of funding 4.7 No Lean Season Full funding gap over three years for implementing the program in Bangladesh 9.0 Deworm the World Three years of funding for a new program in Pakistan and reserves to protect against funding shortfalls in India 10.4 Malaria Consortium Part of the funding gap for SMC in Burkina Faso, Nigeria, and Chad over the next three years 25.4

In total, we are recommending that Good Ventures make the following grants:

• Malaria Consortium’s seasonal malaria chemoprevention program: $27.9 million • Evidence Action’s Deworm the World Initiative:$15.2 million. We are also recommending that GiveWell’s Board of Directors grant the $0.7 million in discretionary funds that we currently hold from the third quarter (from donors who selected to give to “Grants to recommended charities at GiveWell’s discretion” on our donation form) to Deworm the World, bringing the total to$15.9 million.
• Evidence Action’s No Lean Season program: $11.5 million • Helen Keller International’s vitamin A supplementation program:$7.2 million
• Schistosomiasis Control Initiative: $2.5 million • Against Malaria Foundation:$2.5 million
• Sightsavers’ deworming program: $2.5 million • END Fund’s deworming program:$2.5 million
• GiveDirectly: $2.5 million Our recommendation to donors For donors who are interested in directing funding to whichever recommended charity or charities GiveWell believes has the most pressing funding need at the time the funds are granted, we recommend giving to “Grants to recommended charities at GiveWell’s discretion.” These grants will respond to the greatest funding need we see; they may not match the recommended allocation outlined below. For donors (other than Good Ventures) who are interested in donating directly to our top charities, we recommend splitting your donation as follows: • 70 percent to the Against Malaria Foundation • 30 percent to the Schistosomiasis Control Initiative Why these recommendations? Our recommendations to donors, including Good Ventures, are based on: 1. Overall cost-effectiveness of the charity. Our cost-effectiveness model is a key input into our decision-making process, and large differences in modeled cost-effectiveness impact our recommendations. We try not to put significant weight on relatively small differences in cost-effectiveness according to the model because many inputs are highly uncertain. Our model this year found relatively small differences between many top charities, with Deworm the World at ~12 times as cost-effective as cash transfers, four top charities in the ~6-10x cash transfers range, and three top charities in the ~3-5x cash transfers range. We consider differences between charities implementing the same intervention or interventions that have similar inputs and output in the model more meaningful (e.g., malaria nets and seasonal malaria chemoprevention) than differences between charities implementing quite different interventions. We have completed a sensitivity analysis of our cost-effectiveness analysis to get a better sense for which parameters are most sensitive. We are more hesitant to consider differences in the cost-effectiveness as meaningful when they rely on very sensitive inputs. 2. Cost-effectiveness of particular funding opportunities. Charities’ work can vary significantly in cost-effectiveness across locations due to different costs, disease burdens, uptake in the targeted population, or probability that other funders would step in in GiveWell’s absence. While not a part of our formal cost-effectiveness model, we ran supplementary analyses of cost-effectiveness for some locations for which our top charities were seeking additional funding and considered the output as part of our prioritization of funding gaps. 3. Qualitative factors not captured in our cost-effectiveness model. The main factors we focused on were: • Proportion of the global funding need for the program that is filled. We expect that funders will generally (but imperfectly) select the areas where cost-effectiveness is higher first, leaving the areas with higher costs, lower disease burden, lower cultural acceptance of the program, etc. for last. We believe we have captured some of the consequences of this in our cost-effectiveness analysis. For example, we use national level disease burden estimates for the countries in which each charity has worked and/or plans to work; charities working in higher burden countries are therefore modelled as more cost-effective. But we do not use sub-national estimates to distinguish the highest priority regions within a country; if charities are filling the lowest priority funding gaps within a county, they will likely be less cost-effective than our model suggests. This was an important consideration in comparing AMF and Malaria Consortium. We estimate that ~80 percent of the global funding need for nets (the program AMF implements) has been filled, and ~35 percent of the global funding need for seasonal malaria chemoprevention (the program Malaria Consortium implements). • Our level of knowledge about the organization. We have recommended AMF, Deworm the World, SCI, and GiveDirectly for many years. We know less about Malaria Consortium and No Lean Season and the least about HKI. We seek to be somewhat conservative about recommending large amounts of funding to organizations where there is a relatively high chance that additional research could lead us to believe the program was less cost-effective than we previously thought. • Ease of communication with the organization. It is important to us that we are able to learn over time about the charities we recommend, to enable us to improve our decisions. The ability to communicate effectively with an organization is a key factor in our ability to learn from the organization’s experiences. • Ongoing monitoring and likelihood of detecting future problems. Evaluating an organization’s monitoring processes and results is an important part of our charity reviews and for the most part is not captured in our cost-effectiveness analysis. As with ease of communication, we have more confidence in recommending funds to an organization if we believe that we will learn about how successful its work has been. Summary of key considerations for top charities The table below summarizes the key considerations for our nine top charities. More detail is provided below, as well as in the charity reviews. Estimated cost-effectiveness (relative to cash transfers) Our level of knowledge about the organization Primary benefits of the intervention Ease of communication Ongoing monitoring and likelihood of detecting future problems Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation Other major considerations AMF ~6x High Deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars High proportion (~80%) of global gap for program is filled Malaria Consortium (SMC program) ~7x Moderate Under-5 deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars Relatively low proportion (~35%) of global gap for program is filled Helen Keller International (VAS program) ~9x Moderate Under-5 deaths averted Strong Moderate High: could absorb tens of millions of dollars Learning benefits Deworm the World ~12x High Possible increased income in adulthood Strong Strong Moderate: could absorb millions of dollars END Fund (deworming program) ~4x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars SCI ~10x High Possible increased income in adulthood Moderate Moderate High: could absorb tens of millions of dollars Sightsavers (deworming program) ~5x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars No Lean Season ~5x Moderate Immediate increase in consumption Strong Moderate Low: further funding would be used for different types of activities Potential upside GiveDirectly Baseline High Immediate increase in consumption and assets Strong Strong Very high: could absorb over 100 million dollars Reasons for this funding gap ranking Prioritization of funding that we have recommended to Good Ventures (we recommend Good Ventures fill the highest-priority funding needs first, to ensure these are funded): 1. We start by recommending that each top charity receive$2.5 million as an “incentive grant.” These grants are intended to be a major contribution to the charity’s work in recognition of the fact that they have met GiveWell’s criteria and have dedicated significant time to working with us to help us follow their progress and plans each year. We don’t want our top charity funding process to be winner-takes-all because we believe that charities would be less likely to want to participate in that case.

2. After incentive grants, we believe the next most valuable funding to provide is for Deworm the World’s work in Kenya and India over the next three years. Deworm the World’s work in Kenya and India is the most cost-effective opportunity we have found. We estimate that its work in Kenya is ~20x as cost-effective as cash transfers and in India is ~30x+ as cost-effective as cash transfers.
3. We rank providing funding to our two new top charities, Helen Keller International (HKI)’s VAS program and No Lean Season, next.

We estimate that HKI could use $7.2 million over three years to support VAS campaigns in countries with high child mortality rates that have recently missed campaigns due to lack of funds. HKI’s cost-effectiveness is at the high end of the range for top charities (~9x cash transfers). We believe HKI could absorb more than$7.2 million in additional funding for VAS effectively but that this $7.2 million gap is likely more cost-effective than HKI’s average cost-effectiveness. Also, because HKI is a new top charity of ours, we expect this first part of its gap to have significant learning benefits for us: by giving this money, we’ll be better positioned to follow HKI’s work and review its monitoring, which we believe will make it more likely that we have a more accurate estimate of its impact in future years. We decided to recommend funding all of No Lean Season’s funding gap in Bangladesh for the next three years. While No Lean Season’s cost-effectiveness is at the lower end of our top charities (~5x cash transfers), we see additional reasons to prioritize this gap. We believe No Lean Season is the top charity where there is the strongest case to be made for “upside”; our cost-effectiveness analysis may not capture the potential impact of scaling a new program that could lead to greater visibility and funding for a novel type of program. 4. We think the next highest priority funding to provide is$10.4 million to Deworm the World. This funding would support a new program in Pakistan and provide reserve funding for programs supported with restricted funds. We estimate that the program in Pakistan will be roughly ~7x as cost-effective as cash transfers, though this estimate is very sensitive to estimates of worm burdens in the locations where Deworm the World plans to work.

The reserve funding is intended to make it unlikely that the India program, which we believe is very highly cost-effective, will be interrupted—Deworm the World relies on restricted funding for this program and there is some chance that this funding will not be available in the future. It may use this GiveWell-directed funding for other opportunities if it is not needed to backstop restricted funding in India; we expect that it will have unfunded opportunities remaining in the next few years, particularly in Nigeria.

5. The last funding gap on our list of recommendations for Good Ventures is $23.6 million to Malaria Consortium for its work on SMC. When choosing which gap to recommend for the remainder of Good Ventures’$75 million, we focused on the remaining funding needs for Malaria Consortium’s SMC program, AMF, and SCI, which we believe to have the next highest-value gaps. Our cost-effectiveness model indicates that SCI is the most cost-effective of these three organizations (~10x cash transfers, compared with ~6-7x cash transfers for AMF and Malaria Consortium), but when the difference in modelled cost-effectiveness between two charities is relatively small, we also put significant weight on qualitative factors. We believe that AMF and Malaria Consortium are stronger on some qualitative factors, particularly the likelihood that we will be able to learn about the programs’ performance through the monitoring they conduct. Between AMF and Malaria Consortium, we have prioritized Malaria Consortium’s funding gap primarily due to the qualitative considerations discussed above around the proportion of the global funding need that is filled. After following Malaria Consortium for a second year, we believe that Malaria Consortium and AMF are comparable on other major qualitative factors, such as quality of ongoing monitoring and likelihood of detecting future problems.

The total amount we are recommending for Malaria Consortium’s SMC program represents a rough compromise between providing a high level of funding to a program that we prefer to the next funding gap on the list and not wanting to make too large of a bet on an organization that we have less experience with than some other top charities.

Prioritization for non-Good Ventures donors:

1. Our current recommendation for donors is to give to GiveWell for making grants to top charities at our discretion. Our goal is for SCI to receive $9 million, in addition to the$2.5 million incentive grant that we are recommending to Good Ventures, and AMF to receive the remainder of expected GiveWell-directed funding because AMF and SCI represent the next highest-value funding opportunities we see. Giving us funding to grant at our discretion allows GiveWell to better target this allocation, and to adapt if we learn new information about pressing, high-value funding needs at our top charities.
2. For donors who prefer to give directly to charities, we recommend giving 70 percent to AMF and 30 percent to SCI. These percentages are our best guess of what will achieve our target allocation given our projections of total donations driven by our recommendations.

This allocation comes from a belief that, at these margins, it is difficult to distinguish between the quality of AMF and SCI’s funding gaps. SCI has better modeled cost-effectiveness, while AMF appears to be better on several qualitative factors, including monitoring of program performance. We have roughly targeted a two-to-one ratio between the two.

Details on new top charities

Helen Keller International (HKI) for work on vitamin A supplementation

Our full review of HKI’s work on vitamin A supplementation is here.

Overview

HKI (http://www.hki.org/) is a large organization with multiple programs focused on reducing malnutrition and averting blindness and poor vision. Our review focuses on HKI’s work on vitamin A supplementation (VAS) and our recommendation is specific to its VAS program. HKI provides technical assistance, engages in advocacy, and contributes funding to government-run VAS programs.

There is strong evidence from many randomized controlled trials (RCTs) conducted in the 1980s and 1990s that VAS can substantially reduce child mortality, but weaker evidence on how effective VAS is in the places HKI would work with additional funding in the next few years. In particular, there is little available information on current rates of vitamin A deficiency in areas where HKI works. We have adjusted our cost-effectiveness analysis for our best guess of how much less effective VAS is today (~25 percent as effective as in the trials in the 1980s and 1990s); the intervention remains cost-effective with that adjustment.

We feel that the monitoring data that we have seen from HKI’s programs gives us limited information on HKI’s past performance, but demonstrates the types of data HKI is able to collect on program performance. We have requested that HKI collect this monitoring data of all programs funded with GiveWell-directed funds.

Overall, we have not yet investigated HKI at the same level of depth as some of our other top charities, which we have recommended for several years. We have reviewed documents from HKI, had a number of conversations with their staff, and spent three days meeting with HKI and observing a VAS campaign in Guinea. We have remaining questions about HKI’s work that we will seek more information on in the future, but overall we believe this program is, like our other top charities, an excellent giving opportunity.

Funding gap

We believe that HKI’s VAS work is highly likely to be constrained by funding next year. HKI has provided details of VAS programs that it could support with additional funding of up to about $41.4 million in 2018-2020. HKI appears to have limited prospects for funding these programs from other sources. Our understanding is that with additional funds, HKI would cause additional rounds of VAS to occur in some countries, while in other countries, HKI primarily aims to increase coverage rates in rounds of VAS that would take place regardless of its involvement. We have asked HKI to prioritize use of GiveWell-directed funding in countries where it expects to cause additional rounds of VAS to occur. HKI’s funding gap for countries that have recently missed VAS campaigns due to lack of funds is$7.2 million.

HKI’s VAS work was supported by the Canadian government in the past. That funding ended in 2016 and has not been renewed. Over the past year, several VAS campaigns have been skipped in countries HKI previously supported.

Evidence Action’s No Lean Season program

Our full review of No Lean Season is here.

Overview

No Lean Season (https://www.evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the season of income and food insecurity (‘lean season’) between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on a household member stating their intention to migrate to urban or other rural locations to seek short-term employment.

Several randomized controlled trials (RCTs) of subsidies to increase migration provide moderately strong evidence that such an intervention increases household income and consumption during the lean season. An additional RCT is ongoing. We estimate that No Lean Season is roughly five times as cost-effective as cash transfers (see our cost-effectiveness analysis).

Evidence Action has shared some details of its plans for monitoring No Lean Season in the future, but, as many of these plans have not been fully implemented, we have seen limited results. Therefore, there is some uncertainty as to whether No Lean Season will produce the data required to give us confidence that loans are appropriately targeted and reach their intended recipients in full; that recipients are not pressured into accepting loans; and that participants successfully migrate, find work, and are not exposed to major physical and other risks while migrating.

Funding gap

We expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further$3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Evidence Action is seeking funding beyond this level to allow it to build reserves for No Lean Season.

Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net (LLIN) distributions for protection against malaria in developing countries. AMF has conducted post-distribution surveys of all completed distributions to determine whether LLINs have reached their intended destinations and how long they remain in good condition. AMF’s post-distribution surveys have generally found positive results (with some exceptions); we believe they have some methodological limitations.

We estimate that AMF’s program is roughly six times as cost-effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

Prior to this year, we had seen results from AMF’s “post-distribution check ups” (PDCUs) from two countries, Malawi and the Democratic Republic of the Congo, and had significant uncertainties about the methodology used in each location. We have now also seen results from Ghana. We have more confidence in our understanding of AMF’s PDCUs than we did previously, though this work is ongoing. In particular, we commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings (see links). Further discussion of the strengths and weaknesses of PDCUs here.

In 2017, AMF signed relatively few new agreements to fund LLIN distributions and, as a result, has a balance of $58 million in uncommitted funds, or$35 million if distributions where AMF believes agreements are imminent are counted as committed. Our understanding is that many of AMF’s conversations with countries could not progress until decisions were made about how much Global Fund funding each country would allocate to LLIN distributions (as opposed to other malaria control efforts). This decision-making process extended into late 2017. Global Fund funding is allocated on three-year cycles and we do not expect this to continue to be a bottleneck for AMF in 2018.

Funding gap

We believe that AMF is very likely to be constrained by lack of funding. There is high uncertainty in the maximum amount of funding that AMF could use productively, though we expect the maximum to be much greater than what AMF is likely to receive. To fund all of the distributions that it is currently in detailed discussions about, AMF would need $50 million more than we project it will receive. The total funding gap for LLINs for 2018-2020 appears to be hundreds of millions of dollars. With additional funding, AMF’s top priorities would be to fund a portion of the next round of distributions, in 2018-2020, in each of the countries in which it has recently funded distributions. END Fund (for work on deworming) Our full review of the END Fund’s work on deworming is here. Background The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming. Slightly more than half of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list. We have seen limited monitoring results on the number of children reached in END Fund-supported programs. In 2016, the END Fund began requiring that surveys be conducted to determine whether its programs have reached a large proportion of children targeted; we have seen coverage surveys for (a non-random sample of) 35 percent of its 2016 deworming grant portfolio. These studies leave us with some remaining questions about the program’s impact. Important changes in the last 12 months We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas that the END Fund works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. We also saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible. Funding gap We believe the END Fund could substantially increase its deworming grantmaking with additional funds. We roughly estimate that there is gap of$18 million between the amount of funding the END Fund will have available for grants for deworming and the amount of funding it would need to make all of the potential grants it has identified. Sources of major uncertainty in this estimate include whether the END Fund will encounter non-funding bottlenecks in some of its identified and early-stage opportunities, the amount of funding it will receive from other sources, the proportion of funding it will allocate to deworming, and costs other than grants.

Evidence Action’s Deworm the World Initiative

Our full review of Deworm the World is here.

Background

Evidence Action’s Deworm the World (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates deworming programs. Its main countries of operation are India, Kenya, and Nigeria, and it is considering expanding to Pakistan.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World overall is the most cost-effective charity we have found. We estimate that it is ~12 times as cost-effective as cash transfers, but note that, due to differences in worm burdens and costs across countries, there is significant variation in cost-effectiveness across the countries in which it works. We estimate that its work to date in India has been more than 30 times as cost-effective as cash transfers, while its planned work in Nigeria is around three times as cost-effective as cash transfers (though this estimate is based on low-quality information).

Important changes in the last 12 months

We estimate that Deworm the World could absorb considerably more funding this year than we estimated last year, due to opportunities it has identified to expand its geographic reach. (More in the next section.)

The quality of the monitoring that we have seen from Deworm the World has remained high. To date, we have seen limited monitoring from Nigeria, which is a new addition to Deworm the World’s portfolio and is expected to become a major portion of its work in the future. This is of minor concern given the strong monitoring track record elsewhere and how new the program is in Nigeria.

Funding gap

We believe that Deworm the World is very likely to be constrained by funding. We expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive over the next three years. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities. With additional funding, Deworm the World would sustain its current work in Kenya and India, and would seek to expand its work in Nigeria and India to additional states and support the government in Pakistan to initiate a deworming program. GiveDirectly Our full review of GiveDirectly is here. Background GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82 percent overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people. We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. We believe that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected. It continues to experiment heavily, with the aim of improving how its own cash transfer programs are run as well as those of governments. It has recently started work on a universal basic income trial and has started partnering with major funders on evaluations of cash transfers in new geographies with the aim of influencing the broader international aid sector to use its funding more cost-effectively. We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost-effective than cash. Important changes in the last 12 months We had previously expressed reservations about GiveDirectly’s targeting strategy: that by excluding the least poor households in each village, the program might lead to negative reactions by non-recipients, increase costs per household reached, and exclude households that were still quite poor. In 2017, GiveDirectly largely switched to a “saturation” approach of making transfers to all households in selected villages. It will continue to use a targeted approach in Rwanda, where government regulations require such an approach, but the saturation approach will be used in Kenya and Uganda. In 2016, GiveDirectly built up its operations in Uganda and Kenya with the anticipation of revenue growth in 2017. Revenue growth has been slower than expected and GiveDirectly had to lay off some staff as a result. GiveDirectly launched its universal basic income project this month. In 2015, Good Ventures made a grant of$25 million to GiveDirectly on GiveWell’s recommendation. GiveDirectly’s goals for the grant were to expand its ability to raise funds from donors not influenced by GiveWell’s recommendation and to collaborate with large aid institutions or governments to address their questions about cash transfers. We expect to write more about the performance of the grant in the future, but, in short, our impression is that fundraising has progressed slower than expected and collaborative projects have progressed more quickly than expected.

Funding gap

We believe that GiveDirectly is highly likely to be constrained by funding next year. It expects to use additional funding primarily for standard cash transfers and for additional collaborative projects. For collaborative projects, GiveDirectly’s potential partners require it to contribute funding, which the partner matches (at a one-to-one ratio, minimum). These projects would largely be in countries GiveDirectly has not worked in before and many are at an early stage of discussion. We estimate that GiveDirectly could use more than $200 million in additional funding in 2018-2019. Malaria Consortium (for work on seasonal malaria chemoprevention) Our full review of Malaria Consortium’s seasonal malaria chemoprevention program is here. Background Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria. There is strong evidence that SMC substantially reduces cases of malaria. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately statistically powered to find an impact on mortality. Malaria Consortium and its partners have conducted studies in all of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, though past surveys have been conducted after four rounds of SMC (SMC is given in a maximum of four treatment courses at monthly intervals) and may be subject to error due to the inaccurate recall or recordkeeping. Starting in 2017, Malaria Consortium is conducting coverage surveys after each round of SMC, to reduce recall error. Important changes in the last 12 months We have increased our confidence in Malaria Consortium’s monitoring, though we have not yet seen all of the research that Malaria Consortium expected to share in 2017 (in particular, tracking of malaria cases and deaths over time in areas where Malaria Consortium works). Coverage survey results from 2016 were generally positive, with a couple of outliers. The change from conducting coverage surveys after four treatment cycles to conducting them after each cycle will increase our confidence in the results. Last year, we had only a rough estimate of how much additional funding Malaria Consortium could use productively. We have significantly improved our understanding of its room for more funding this year. Funding gap We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. It appears that there is a large remaining global need for additional funding for SMC programs and that Malaria Consortium is well-positioned to fill these gaps, if it has sufficient funding to do so. Malaria Consortium estimates that it could spend$28-30 million per year on SMC in each of the next three years and that this level of funding would largely fill the global funding gap for SMC, with the exception of Nigeria, where the scale of the gap would be beyond Malaria Consortium’s operational capacity in the short term.

It appears to have limited prospects for major funding from other sources. The major grant for Malaria Consortium’s work on SMC previously, from Unitaid, is ending and Malaria Consortium told us that it will not be renewed.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify partner countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies to determine whether its programs have reached a large proportion of children targeted. These studies cover (a non-random sample of) about 40 percent of treatments SCI reports having delivered over the past few years. The studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is less cost-effective than Deworm the World and more cost-effective than Sightsavers and the END Fund. Given the uncertainty in our cost-effectiveness model, we are hesitant to say that SCI is more cost-effective than AMF and Malaria Consortium, though taken literally, SCI is 1.5 times as cost-effective as AMF and Malaria Consortium (~10x cash transfers vs. ~6-7x cash transfers).

Important changes in the last 12 months

We continued to follow SCI’s progress in 2017 and there have not been many major changes to its work. As in the past, SCI shared monitoring of deworming coverage levels for a portion of its programs with us; there continue to be several SCI-supported countries for which we have not seen monitoring results. In the past, we have noted that we had low confidence in the accuracy of the financial information that SCI provided and that SCI made significant improvements to its financial systems in 2016; our remaining concerns about SCI’s financial management and reporting are fairly minor.

In 2017, SCI allocated nearly all available funding to programs in its 2017-2018 budget year. This was a large increase in spending over the previous budget year ($9.6 million in 2016-2017 compared with$22.5 million in 2017-2018), driven in large part by a large increase in GiveWell-directed funding ($3.7 million in 2015 compared with$16.6 million in 2016). We believe this decision was due in part to a miscommunication with GiveWell—in a conversation with SCI in early 2017, we recommended that they treat the funds like a multi-year grant because of the risk of large fluctuations in GiveWell-directed funding, but we did not emphasize this point. SCI told us that it plans to allocate future funding over multiple years, noting that its funding allocation decisions in 2016-2017 were due to the desire to avoid allowing drugs to expire as well as a misunderstanding with GiveWell about how the funding was intended to be used.

Funding gap

We estimate that SCI could productively use about $30 million more than it expects to receive to deliver treatments to school-aged children over the next three years. It could use almost three times this amount if it were to follow World Health Organization guidelines, which include treating many adults; we are not recommending funding to treat adults because we haven’t seen sufficient evidence on the impact of treating adults. The primary use of this funding, and SCI’s top priority, would be to sustain and expand work in current countries of operation. A smaller portion would be used to expand to up to four additional countries. Sightsavers (for work on deworming) Our full review of Sightsavers is here. Background Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and – more specifically – advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections. Sightsavers has shared surveys for some of its past NTD programs that measure whether these programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact. We have seen very limited results from Sightsavers’ deworming programs specifically. For GiveWell-supported programs, Sightsavers has told us it will conduct coverage surveys for each round of deworming; we have reviewed one of those surveys to date. Important changes in the last 12 months In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results. We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. Funding gap We believe that Sightsavers’ deworming work is likely to be constrained by funding next year. Sightsavers has provided details of deworming programs that it could fund with additional funding of up to about$6.4 million in 2018 and 2019. Sightsavers appears to have limited prospects for funding these programs from other sources. We believe it is likely that Sightsavers could absorb funding beyond this amount to extend programs to 2020 and/or seek out additional opportunities to fund deworming programs.

Of the $6.4 million,$2.8 million would be used to add deworming to existing NTD programs and 3.7 million would be used to fund NTD programs that would treat several NTDs in addition to schistosomiasis and STH. We will request that Sightsavers prioritize the first set of opportunities, because we believe they will likely be more cost-effective. Standout charities In addition to our top charities, we recognize standout charities—organizations that support programs that may be extremely cost-effective and are evidence-backed but for which we have less confidence in their impact than we do for our top charities. We have reviewed their work and feel these groups stand out from the vast majority of organizations we have considered in terms of the evidence base for the program they support, their transparency, and their potential cost-effectiveness. These organizations offer additional giving options for donors who feel highly aligned with their work. We’ve added one organization to the list this year: Evidence Action’s Dispensers for Safe Water. We don’t follow standout organizations as closely as we do our top charities. We generally have one or two calls per year with representatives from each group and publish notes on our conversations. We provide brief updates on these charities below. New addition to the standout list: • Evidence Action’s Dispensers for Safe Water. The Dispensers for Safe Water program provides chlorine dispensers for decontamination of drinking water to prevent diarrhea and associated deaths of young children. We believe that there is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death. Our rough cost-effectiveness analysis of Dispensers for Safe Water suggests that the program is in a similar range of cost-effectiveness as unconditional cash transfer programs. Our review of Dispensers for Safe Water is here. Organizations that have conducted randomized controlled trials of their programs: • Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its child survival media campaign in Burkina Faso and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15 percent or more) and data from health facilities finding an increase in facility visits. (The results have not yet been published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. DMI is conducting an RCT of its family planning radio campaign in Burkina Faso and it is planning work on early child development in Burkina Faso and child survival in Mozambique. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI is here and notes from our most recent conversation with DMI are here. • Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a RCT of its program and measured a 27 percent reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of GiveDirectly. It is conducting a second RCT of its program and results are expected in 2020. Living Goods recently expanded the number of family planning products it offers and is interested in expanding to a third country. Living Goods is scaling up its program and could scale up more quickly with additional funding. Our review of Living Goods is here and notes from our most recent conversation with Living Goods are here. Organizations working on micronutrient fortification: We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence. • Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost-effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. FFI’s recent work includes advocating for legislation to mandate that rice imported to West Africa is fortified with vitamins and minerals. Our full review is here and notes from our most recent conversation are here. • Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. GAIN has focused its recent USI work on Tanzania, Mozambique, Ethiopia, and Kenya, which it targeted based on relatively low levels of coverage of iodized salt and strong relationships with stakeholders. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN is here and notes from our most recent conversation are here. • Iodine Global Network (IGN). Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-directed funding has made up a large part of its funding in recent years. It expects to have data from before and after its recent work in Madagascar, Lebanon, and possibly Israel by the end of 2018; this data may provide additional evidence of IGN’s impact. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN is here and notes from our most recent conversation here. • Project Healthy Children (PHC)/Sanku. PHC/Sanku aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs and by enabling fortification among small-scale millers. PHC is scaling up its Sanku project, which equips small millers with a machine that enables them to fortify their flour with micronutrients; we have not done as much formal analysis of Sanku as of PHC’s core work on advocacy and technical assistance to countries to implement fortification. PHC/Sanku expects to be constrained by funding in the future. Our review of PHC/Sanku is here and notes from our more recent conversation are here. Our research process in 2017 We plan to detail the work we completed this year in a future post as part of our annual review process. A major focus of 2017 was improving our recommendations in future years, in particular through our work on GiveWell Incubation Grants and prioritizing promising programs for further investigation. Below, we highlight the key research that led to our current charity recommendations. This page describes our overall process. • Following existing top charities. We followed the progress and plans of each of our 2016 top charities. We had several conversations by phone with each organization, met in person at least once with each top charity (including a three-day visit to Rwanda and the Democratic Republic of the Congo with the END Fund), and reviewed documents they shared with us. • Identifying new top charities. • No Lean Season. We had recommended a series of Incubation Grants to No Lean Season beginning in 2014 and have followed its progress since then. This year, due to the scale at which No Lean Season was operating and the track record it had established, we decided that the No Lean Season program was at a stage of development where we could evaluate it as a potential top charity. In addition to extensive communications with No Lean Season staff over the phone and reviewing documents they shared with us, GiveWell staff spent five days visiting the program in Bangladesh. • Helen Keller International’s vitamin A supplementation program. Earlier this year, Research Analyst Chelsea Tabart began reaching out to organizations that might be a fit for our criteria, but with which we had limited or no previous contact with. As a result of that process, we reconnected with Helen Keller International (which we first considered as a potential top charity in 2007) and began to consider its vitamin A supplementation program as a potential top charity. In addition to extensive communications with HKI staff over the phone and reviewing documents they shared with us, GiveWell staff spent three days meeting with HKI staff in Guinea and observing a vitamin A supplementation program. • Completing intervention reports on obstetric fistula surgery and measles vaccination campaigns; completing interim intervention reports on SMS reminders for vaccination, Sayana® Press (an injectable contraceptive), oral rehydration solution, and antiretroviral therapy for HIV/AIDS; and expanding our interim intervention report on seasonal malaria chemoprevention to a full intervention report. • Staying up to date on the research for malaria nets, cash transfers, and deworming. We did not find major new research on cash transfers, nets, or deworming that affected our recommendation of GiveDirectly, AMF, or the organizations we recommend for their work on deworming. David Roodman published an in-depth review (parts 1 and 2) of the deworming studies that form the primary basis of our views on the impact of deworming (though much of this work was completed in 2016 and informed our top charity recommendations last year). • Making extensive updates to our cost-effectiveness model and publishing several updates to the model over the course of the year. We instituted a process to track and report publicly on updates to the model to reduce the possibility of errors and make our process more transparent. This year, staff members have also provided substantially more detail in our cost-effectiveness file about why they have chosen particular inputs. Giving to GiveWell vs. top charities GiveWell is currently in a stable financial position. We project that our revenue and our expenses will be approximately equal in the future. However, this projection forecasts some growth in the level of operating support we receive. In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations. We retain our “excess assets policy” to ensure that if we fundraise for our own operations beyond a certain level, we will grant the excess to our recommended charities. We cap the amount of operating support we ask Good Ventures to provide to GiveWell at 20 percent, for reasons described here. We thus ask that donors who use GiveWell’s research consider the following: • If you have supported GiveWell’s operations in the past, we ask that you maintain your support. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and to minimize staff time spent on fundraising for our operating expenses. • If you have not supported GiveWell’s operations in the past, we ask that you designate 10 percent of your donation to help fund GiveWell’s operations. This can be done by selecting the option to “Add 10% to help fund GiveWell’s operations” on our credit card donation form or letting us know how you would like to designate your funding when giving another way. We’re happy to answer questions in the comments below. Please also feel free to reach out directly with any questions. The post Our top charities for giving season 2017 appeared first on The GiveWell Blog. ### Are GiveWell’s top charities the best option for every donor? Wed, 06/21/2017 - 12:15 We’re sometimes asked whether we think GiveWell’s top charities are the “best,” in some absolute sense of the word, or whether we’d ever advise that a donor give to an opportunity outside of our recommendations. This post aims to clarify how GiveWell thinks about different giving options and their suitability for different types of donors. We believe that GiveWell’s top charities offer donors an outstanding opportunity to do a lot of good and are the best option for most donors. However, some donors—those with a very high degree of trust in a particular individual or organization to make this decision, donors with lots of time (in excess of 50 hours per year, and likely more) to consider their giving decision, or donors whose values point strongly toward a particular cause outside of the ones GiveWell covers—may find opportunities to have a greater impact per dollar than GiveWell’s top charities. Note that we think these characteristics are likely to be necessary, but not sufficient, for finding these types of opportunities; we still expect good giving to be hard, and spending, for example, 50 hours per year on research isn’t necessarily going to yield better opportunities. In this post, we describe relevant considerations for donors in greater detail. Giving to GiveWell’s top charities GiveWell was founded to serve donors with limited amounts of time to make giving decisions. GiveWell’s co-founders, Elie Hassenfeld and Holden Karnofsky, were in this situation when they started GiveWell as a side project in 2006. They found that determining where to give effectively was a full-time project and quit their jobs to start GiveWell in 2007. GiveWell’s top charity recommendations serve all donors. We rely on evidence and detail our rationale for making a recommendation publicly, so donors can vet our work; a strength of our recommendations is their falsifiability. We believe our top charity recommendations serve donors who want to give as effectively as possible and have only limited time to determine where to donate, and (prior to GiveWell) no trusted person or entity to outsource their thinking to, particularly well. Our criteria and recommendations were designed with this type of donor in mind: • Our top charities are largely uncontroversial and relatively straightforward ways to do a lot of good—for example, by providing direct aid such as insecticide-treated nets to prevent malaria and cash transfers to very poor households. There is room for debate on the evidence behind these interventions and their cost-effectiveness, but the basic case for them—and the fact that they are likely to do more good than harm—is subject to little debate, so a donor can feel fairly confident in these basics without needing to do their own research. • GiveWell publishes the full details of our charity analyses so that donors can review and vet our work, and so that donors with very limited time can trust that any major problems would likely be caught by others (with more time). • Because we lay out the entire case for the charities online, donors can spot-check any particular part of it to get a sense of whether we’re thinking reasonably about the issues that seem most salient to them. • Our top charities have room for more funding. In other words, we believe additional marginal donations to these organizations enable them to do more good. Our guess is that most donors that use GiveWell fit this profile (want to give as effectively as possible and have only limited time to determine where to donate, and no other trusted person or entity to outsource their thinking to). Below, we discuss alternative donor profiles: (1) Donors with limited time and a high amount of trust in a person or organization to inform their giving decisions This group of donors has limited time to spend on making a giving decision and has an organization or person (other than GiveWell or GiveWell staff) they personally trust to make or inform this decision. In this case, they may defer to that person or organization’s recommendations. (2) Donors with lots of time Donors with a lot of time to spend on giving decisions (50+ hours per year) may be able to find opportunities that GiveWell hasn’t. For example, a donor might know someone who is starting a charity and feel, based on their research, that supporting their project at an early stage might be a particularly leveraged way to do good. A donor with lots of time may also be very familiar with a particular cause and feel highly confident in a particular organization and its need for funding. These donors may want to compare alternative opportunities to GiveWell’s top charities. They may also want to actively vet GiveWell’s recommendations as part of their research process. Donors with lots of time may also wish to apply a different strategy to their giving. GiveWell largely recommends charities where sufficient evidence exists to make a fairly robust estimate of the expected value of a donation. Donors with much more time to spend (maybe even significantly more than 50 hours per year) thinking about where to give may want to take a “hits-based giving” approach—having a high tolerance for philanthropic risk, so long as the overall expected value is sufficiently high. This is the approach the Open Philanthropy Project, which was incubated at GiveWell, has taken, and we believe doing this well requires a lot of work, as the Open Philanthropy Project discussed in a blog post last year (emphasis original): Aim for deep understanding of the key issues, literatures, organizations, and people around a cause, either by putting in a great deal of work or by forming a high-trust relationship with someone else who can. If we [the Open Philanthropy Project] support projects that seem exciting and high-impact based on superficial understanding, we’re at high risk of being redundant with other funders. If we support projects that seem superficially exciting and high-impact, but aren’t being supported by others, then we risk being systematically biased toward projects that others have chosen not to support for good reasons. By contrast, we generally aim to support projects based on the excitement of trusted people who are at a world-class level of being well-informed, well-connected, and thoughtful in relevant ways. Achieving this is challenging. It means finding people who are (or can be) maximally well-informed about issues we’ll never have the time to engage with fully, and finding ways to form high-trust relationships with them. As with many other philanthropists, our basic framework for doing this is to choose focus areas and hire staff around those focus areas. In some cases, rather than hiring someone to specialize in a particular cause, we try to ensure that we have a generalist who puts a great deal of time and thought into an area. Either way, our staff aim to become well-networked and form their own high-trust relationships with the best-informed people in the field. I [Open Philanthropy Project Executive Director Holden Karnofsky] believe that the payoff of all of this work is the ability to identify ideas that are exciting for reasons that require unusual amounts of thought and knowledge to truly appreciate. (3) Donors with values that differ from GiveWell staff Donors who hold different values than the majority of GiveWell staff, or who place more weight on a particular cause outside of the causes covered by GiveWell, may find other giving opportunities to be more attractive for reasons beyond the time/trust framework articulated earlier in this post. For example, individuals who place a very high value on farm animal welfare may wish to give a large proportion of their donation, if not all of their donation, to organizations working in that cause. We’re happy to speak with you about giving decisions. If you’re not sure which considerations apply to you, please reach out. We’re always happy to talk through giving decisions. The post Are GiveWell’s top charities the best option for every donor? appeared first on The GiveWell Blog. ### Our updated top charities for giving season 2016 Mon, 11/28/2016 - 22:58 We have refreshed our top charity rankings and recommendations. We now have seven top charities: our four top charities from last year and three new additions. We have also added two new organizations to our list of charities that we think deserve special recognition (previously called “standout” charities). Instead of ranking organizations, we rank funding gaps, which take into account both charities’ overall quality and cost-effectiveness and what more funding would enable them to do. We also account for our expectation that Good Ventures, a foundation we work closely with, will provide significant support to our top charities (50 million in total). Our recommendation to donors is based on the relative value of remaining gaps once Good Ventures’ expected giving is taken into account. We believe that the remaining funding gaps offer donors outstanding opportunities to accomplish good with their donations.

Our top charities and recommendations for donors, in brief

Top charities

We are continuing to recommend the four top charities we did last year and have added three new top charities:

1. Against Malaria Foundation (AMF)
2. Schistosomiasis Control Initiative (SCI)
3. END Fund for work on deworming (added this year)
4. Malaria Consortium for work on seasonal malaria chemoprevention (added this year)
5. Sightsavers for work on deworming (added this year)
6. Deworm the World Initiative, led by Evidence Action
7. GiveDirectly

We have ranked our top charities based on what we see as the value of filling their remaining funding gaps. We do not feel a particular need for individuals to divide their allocation across all of the charities, since we are expecting Good Ventures will provide significant support to each. For those seeking our recommended allocation, we recommend giving 75% to the Against Malaria Foundation and 25% to the Schistosomiasis Control Initiative, which we believe to have the most valuable unfilled funding gaps.

Our recommendation takes into account the amount of funding we think Good Ventures will grant to our top charities, as well as accounting for charities’ existing cash on hand, and expected fundraising (before gifts from donors who follow our recommendations). We recommend charities according to how much good additional donations (beyond these sources of funds) can do.

Other Charities Worthy of Special Recognition

As with last year, we also provide a list of charities that we believe are worthy of recognition, though not at the same level (in terms of likely good accomplished per dollar) as our top charities (we previously called these organizations “standouts”). They are not ranked, and are listed in alphabetical order.

Below, we provide:

• An explanation of major changes in the past year that are not specific to any one charity. More
• A discussion of our approach to room for more funding and our ranking of charities’ funding gaps. More
• Summary of key considerations for top charities. More
• Detail on each of our new top charities, including an overview of what we know about their work and our understanding of each organization’s room for more funding. More
• Detail on each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year and our understanding of each organization’s room for more funding. More
• The process we followed that led to these recommendations. More
• A brief update on giving to support GiveWell’s operations vs. giving to our top charities. More

Conference call to discuss recommendations

We are planning to hold a conference call at 5:30pm ET/2:30pm PT on Thursday, December 1 to discuss our recommendations and answer questions.

If you’d like to join the call, please register using this online form. If you can’t make this date but would be interested in joining another call at a later date, please indicate this on the registration form.

Major changes in the last 12 months

Below, we summarize the major causes of changes to our recommendations (since last year).

Most important changes in the last year:

• We engaged with more new potential top charities this year than we have in several years (including both inviting organizations to participate in our process and responding to organizations that reached out to us). This work led to three additional top charities. We believe our new top charities are outstanding giving opportunities, though we note that we are relatively less confident in these organizations than in our other top charities—we have followed each of the top charities we are continuing to recommend for five or more years and have only began following the new organizations in the last year or two.
• Overall, our top charities have more room for more funding than they did last year. We now believe that AMF, SCI, Deworm the World, and GiveDirectly have strong track records of scaling their programs. Our new top charities add additional room for more funding and we believe that the END Fund and Malaria Consortium, in particular, could absorb large amounts of funding in the next year. We expect some high-value opportunities to go unfilled this year.
• Last year, we wrote about the tradeoff between Good Ventures accomplishing more short-term good by filling GiveWell’s top charities’ funding gaps and the long-term good of saving money for other opportunities (as well as the good of not crowding out other donors, who, by nature of their smaller scale of giving, may have fewer strong opportunities). Due to the growth of the Open Philanthropy Project this year and its increased expectation of the size and value of the opportunities it may have in the future, we expect Good Ventures to set a budget of $50 million for its contributions to GiveWell top charities. The Open Philanthropy Project plans to write more about this in a future post on its blog. Room for more funding analysis Types of funding gaps We’ve previously outlined how we categorize charities’ funding gaps into incentives, capacity-relevant funding, and execution levels 1, 2, and 3. In short: • Incentive funding: We seek to ensure that each top charity receives a significant amount of funding (and to a lesser extent, that charities worthy of special recognition receive funding as well). We think this is important for long-run incentives to encourage other organizations to seek to meet these criteria. This year, we are increasing the top charity incentive from$1 million to $2.5 million. • Capacity-relevant funding: Funding that we believe has the potential to create a significantly better giving opportunity in the future. With one exception, we don’t believe that any of our top charities have capacity-relevant gaps this year. We have designated the first$2 million of Sightsavers’ room for more funding as capacity-relevant because seeing results from a small number of Sightsavers deworming programs would significantly expand the evidence base for its deworming work and has the potential to lead us to want to support Sightsavers at a much higher level in the future (more).
• Execution funding: Funding that allows charities to implement more of their core programs. We separated this funding into three levels: level 1 is the amount at which we think there is a 50% chance that the charity will be bottlenecked by funding; level 2 is a 20% chance of being bottlenecked by funding, and level 3 is a 5% chance.

Ranking funding gaps

The first million dollars to a charity can have a very different impact from, e.g., the 20th millionth dollar. Accordingly, we have created a ranking of individual funding gaps that accounts for both (a) the quality of the charity and the good accomplished by its program per dollar, and (b) whether a given level of funding is capacity-relevant and whether it is highly or only marginally likely to be needed in the coming year.

The below table lays out our ranking of funding gaps. When gaps have the same “Priority,” this indicates that they are tied. When gaps are tied, we recommend filling them by giving each equal dollar amounts until one is filled, and then following the same procedure with the remaining tied gaps. See footnote for more.*

The table below includes the amount we expect Good Ventures to give to our top charities. For reasons the Open Philanthropy Project will lay out in another post, we expect that Good Ventures will cap its giving to GiveWell’s top charities this year at $50 million. We expect that Good Ventures will start with funding the highest-rated gaps and work its way down, in order to accomplish as much good as possible. Note that we do not always place a charity’s full execution level at the same rank and in some cases rank the first portion of a given charity’s execution level ahead of the remainder. This is because many of our top charities are relatively close to each other in terms of their estimated cost-effectiveness (and thus, the value of their execution funding). For reasons we’ve written about in the past, we believe it is inappropriate to put too much weight on relatively small differences in explicit cost-effectiveness estimates. Because we expect that there are diminishing returns to funding, we would guess that the cost-effectiveness of a charity’s funding gap falls as it receives more funding. Priority Charity Amount, in millions USD (of which, expected from Good Ventures*) Type Comment 1 Deworm the World$2.5 (all) Incentive – 1 SCI $2.5 (all) Incentive – 1 Sightsavers$2.5 (all) Incentive – 1 AMF $2.5 (all) Incentive – 1 GiveDirectly$2.5 (all) Incentive – 1 END Fund $2.5 (all) Incentive – 1 Malaria Consortium$2.5 (all) Incentive – 1 Other charities worthy of special recognition $1.5 (all) Incentive$250,000 each for six charities 3 SCI $6.5 (all) Fills rest of execution level 1 Highest cost-effectiveness of remaining level 1 gaps 4 AMF$8.5 (all) First part of execution level 1 Similar cost-effectiveness to END Fund and Sightsavers and greater understanding of the organization. Expect declining cost-effectiveness within Level 1, and see other benefits (incentives) to switching to END Fund and Sightsavers after this point. 5 END Fund $2.5 (all) Middle part of execution level 1 Given relatively limited knowledge of charity, capping total recommendation at$5 million 6 Sightsavers $0.5 (all) Fills rest of execution level 1 Similar cost-effectiveness to AMF and the END Fund 7 Deworm the World$2.0 (all) Fills execution level 2 Highest-ranked level 2 gap. Highest cost-effectiveness and confidence in organization 8 SCI $4.5 (all) First part of execution level 2 Highest cost-effectiveness of remaining level 2 gaps 9 Malaria Consortium$2.5 (all) Part of execution level 1 Given relatively limited knowledge of charity, capping total recommendation at $5 million 10 AMF$18.6 ($5.1) Part of execution level 1 Expect declining cost-effectiveness within level 1; ranked other gaps higher due to this and incentive effects 11 SCI$4.5 ($0) Fills execution level 2 Roughly expected to be more cost-effective than the remaining$49 million of AMF level 1

* Also includes $1 million that GiveWell holds for grants to top charities. More below. Summary of key considerations for top charities The table below summarizes the key considerations for our seven top charities. More detail is provided below as well as in the charity reviews. Consideration AMF Malaria Consortium Deworm the World END Fund SCI Sightsavers GiveDirectly Estimated cost-effectiveness (relative to cash transfers) ~4x ~4x ~10x ~4x ~8x ~5x Baseline Our level of knowledge about the organization High Relatively low High Relatively low High Relatively low High Primary benefits of the intervention Under-5 deaths averted and possible increased income in adulthood Possible increased income in adulthood Immediate increase in consumption and assets Ease of communication Moderate Strong Strong Strong Moderate Moderate Strongest Ongoing monitoring and likelihood of detecting future problems Moderate Moderate Strong Moderate Moderate Moderate Strongest Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation High: less than half of Execution Level 1 filled High: not quantified, but could likely use significantly more funding Low: Execution Levels 1 and 2 filled High: half of Execution Level 1 filled Moderate: Execution Level 1 and some of Level 2 filled Moderate: Execution Level 1 filled Very high: less than 15% of Execution Level 1 filled Our recommendation to donors If Good Ventures uses a budget of$50 million to top charities and follows our prioritization of funding gaps, it will make the following grants (in millions of dollars, rounded to one decimal place):

• AMF: $15.1 • Deworm the World:$4.5
• END Fund: $5.0 • GiveDirectly:$2.5
• Malaria Consortium: $5.0 • SCI:$13.5
• Sightsavers: $3.0 • Grants to other charities worthy of special recognition:$1.5

We also hold about $1 million that is restricted to granting out to top charities. We plan to use this to make a grant to AMF, which is the next funding gap on the list after the expected grants from Good Ventures. We estimate that non-Good Ventures donors will give approximately$27 million between now and the start of June 2017; we expect to refresh our recommendations to donors in mid-June. Of this, we expect $18 million will be allocated according to our recommendation for marginal donations, while$9 million will be given based on our top charity list—this $9 million is considered ‘expected funding’ for each charity and therefore subtracted from their room for more funding.$18 million spans two gaps in our prioritized list, so we are recommending that donors split their gift, with 75% going to AMF and 25% going to SCI, or give to GiveWell for making grants at our discretion and we will use the funds to fill in the next highest priority gaps.

Details on new top charities

Before this year, our top charity list had remained nearly the same for several years. This means that we have spent hundreds of hours talking to these groups, reading their documents, visiting their work in the field, and modeling their cost-effectiveness. We have spent considerably less time on our new top charities, particularly Malaria Consortium, and have not visited their work in the field (though we met with Sightsavers’ team in Ghana). We believe our new top charities are outstanding giving opportunities, though we think there is a higher risk that further investigation will lead to changes in our views about these groups.

Four of our top charities, including two new top charities, support programs that treat schistosomiasis and soil-transmitted helminthiasis (STH) (“deworming”). We estimate that SCI and Deworm the World’s deworming programs are more cost effective than mass bednet campaigns, but our estimates are subject to substantial uncertainty. For Sightsavers and END Fund, our greater uncertainty about cost per treatment and prevalence of infection in the areas where they work leads us to the conclusion that the cost-effectiveness of their work is on par with that of bednets. It’s important to note that we view deworming as high expected value, but this is due to a relatively low probability of very high impact. Our cost-effectiveness model implies that most staff members believe you should use a multiplier of less than 1% compared to the impact (increased income in adulthood) found in the original trials—this could be thought of as assigning some chance that deworming programs have no impact, and some chance that the impact exists but will be smaller than was measured in those trials. Full discussion in this blog post. Our 2016 cost-effectiveness analysis is here.

This year, David Roodman conducted an investigation into the evidence for deworming’s impact on long-term life outcomes. David will write more about this in a future post, but in short, we think the strength of the case for deworming is similar to last year’s, with some evidence looking weaker, new evidence that was shared with us in an early form this year being too preliminary to incorporate, and a key piece of evidence standing up to additional scrutiny.

END Fund (for work on deworming)

Our full review of END Fund is here.

Overview

The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming.

About 60% of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list.

We have not yet seen monitoring results on the number of children reached in END Fund-supported programs. The END Fund has instituted a requirement that grantees conduct coverage surveys and the first results will be available in early 2017. While we generally put little weight on plans for future monitoring, we feel that the END Fund’s commitment is unusually credible because surveys are already underway or upcoming in the next few months, we are familiar enough with the type of survey being used (from research on other deworming groups) that we were able to ask critical questions, and the END Fund provided specific answers to our questions.

We have more limited information on some questions for the END Fund than we do for the top charities we have recommended for several years. We do not have a robust cost per treatment figure, and also have limited information on infection prevalence and intensity.

Funding gap

We estimate that the END Fund could productively use between $10 million (50% confidence) and$22 million (5% confidence) in the next year to expand its work on deworming. By our estimation, about a third of this would be used to fund other NTD programs.

This estimate is based on (a) a list of deworming funding opportunities that the END Fund had identified as of October and its expectation of identifying additional opportunities over the course of the year (excluding opportunities to grant funding to Deworm the World, SCI, or Sightsavers, which we count in those organizations’ room for more funding); and (b) our rough estimate of how much funding the END Fund will raise. The END Fund is a fairly new organization whose revenue comes primarily from a small number of major donors so it is hard to predict how much funding it will raise.

The END Fund’s list of identified opportunities includes both programs that END Fund has supported in past years and opportunities to get new programs off the ground.

Sightsavers (for work on deworming)

Our full review of Sightsavers is here.

Overview

Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and, more specifically, advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections.

We believe that deworming is a highly cost-effective program and that there is moderately strong evidence that Sightsavers has succeeded in achieving fairly high coverage rates for some of its past NTD programs. We feel that the monitoring data we have from SCI and Deworm the World is somewhat stronger than what we have from Sightsavers—in particular, the coverage surveys that Sightsavers has done to date were on NTD programs that largely did not include deworming. Sightsavers plans to do annual coverage surveys on programs that are supported by GiveWell-influenced funding.

We have more limited information on some questions for Sightsavers than we do for the top charities we have recommended for several years. We do not have a robust cost-per-treatment figure, though the information we have suggests that it is in the same range as the cost-per-treatment figures for SCI and Deworm the World. We also have limited information on infection prevalence and intensity in the places Sightsavers works. This limits our ability to robustly compare Sightsavers’ cost effectiveness to other top charities, but our best guess is that the cost-effectiveness of the deworming charities we recommend is similar.

Funding gap

We believe Sightsavers could productively use or commit between $3.0 million (50% confidence) and$10.1 million (5% confidence) in funding restricted to programs with a deworming component in 2017.

This estimate is based on (a) a list of deworming funding opportunities that Sightsavers created for us; and (b) our understanding that Sightsavers would not allocate much unrestricted funding to these opportunities in the absence of GiveWell funding. It’s difficult to know whether other funders might step in to fund this work, but Sightsavers believes that is unlikely and deworming has not been a major priority for Sightsavers to date.

Sightsavers’ list of opportunities includes both adding deworming to existing NTD mass distribution programs and establishing new integrated NTD programs that would include deworming and spans work in Nigeria, Guinea-Bissau, Democratic Republic of Congo, Guinea, Cameroon, Cote d’Ivoire, and possibly South Sudan.

Malaria Consortium (for work on seasonal malaria chemoprevention)

Our full review of Malaria Consortium is here.

Overview

Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria.

The evidence for SMC appears strong (stronger than deworming and not quite as strong as bednets), but we have not yet examined the intervention at nearly the same level that we have for bednets, deworming, unconditional cash transfers, or other priority programs. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately powered to find an impact on mortality.

Malaria Consortium and its partners have conducted studies in most of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, but leave us with some remaining questions about the program’s impact.

Overall, we have more limited information on some questions for Malaria Consortium than we do for the top charities we have recommended for several years. We have remaining questions on cost per child per year and on offsetting effects from possible drug resistance and disease rebound.

Funding gap

We have not yet attempted to estimate Malaria Consortium’s maximum room for more funding. We would guess that Malaria Consortium could productively use at least an additional $30 million to scale up its SMC activities over the next three to four years. We have a general understanding of where additional funds would be used but have not yet asked for a high level of detail on potential bottlenecks to scaling up. We do not believe Malaria Consortium has substantial unrestricted funding available for scaling up its support of SMC programs and expect its restricted funding for SMC to remain steady or decrease in the next few years. Details on top charities we are continuing to recommend Against Malaria Foundation (AMF) Our full review of AMF is here. Background AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net distributions (for protection against malaria) in developing countries. There is strong evidence that distributing nets reduces child mortality and malaria cases. AMF provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity. We estimate that AMF’s program is roughly 4 times as cost effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others. Important changes in the last 12 months In 2016, AMF significantly increased the number and size of distributions it committed funding to. Prior to 2015, it had completed (large-scale) distributions in two countries, Malawi and Democratic Republic of Congo (DRC). In 2016, it completed a distribution in Ghana and committed to supporting distributions in an additional three countries, including an agreement to contribute$28 million to a campaign in Uganda, its largest agreement to date by far.

AMF has continued to collect and share information on its past large-scale distributions. This includes both data from registering households to receive nets (and, in some cases, data on the number of nets each household received) and follow-up surveys to determine whether nets are in place and in use. Our research in 2016 has led us to moderately weaken our assessment of the quality of AMF’s follow up surveys. In short, we learned that the surveys in Malawi have not used fully randomized selection of households and that the first two surveys in DRC were not reliable (full discussion in this blog post). We expect to see follow-up surveys from Ghana and DRC in the next few months that could expand AMF’s track record of collecting this type of data. We also learned that AMF has not been carrying out data audits in the way we believed it was (though this was not a major surprise as we had not asked AMF for details of the auditing process previously).

AMF has generally been communicative and open with us. We noted in our mid-year update that AMF had been slower to share documentation for some distributions; however, we haven’t had concerns about this in the second half of the year.

In August 2016, four GiveWell staff visited Ghana where an AMF-funded distribution had recently been completed. We met with AMF’s program manager, partner organizations, and government representatives and visited households in semi-urban and rural areas (notes and photos from our trip).

Our estimate of the cost-effectiveness of nets has fallen relative to cash transfers since our mid-year update. At that point, we estimated that nets were ~10x as cost-effective as cash transfers, and now we estimate that they are ~4x as cost-effective as cash transfers. This change was partially driven by changes in GiveWell staff’s judgments on the tradeoff between saving lives of children under five and improving lives (through increased income and consumption) in our model, and partially driven by AMF beginning to fund bed net distributions in countries with lower malaria burdens than Malawi or DRC.

Funding gap

AMF currently holds $17.8 million, and expects to commit$12.9 million of this soon. We estimate it will receive an additional $4 million by June 2017 ($2 million from donors not influenced by GiveWell and $2 million from donors who give based on our top charity list) that it could use for future distributions. Together, we expect that AMF will have about$9 million for new spending and commitments in 2017.

We estimate that AMF could productively use or commit between $87 million (50% confidence) and$200 million (5% confidence) in the next year. We arrived at this estimate from a rough estimate of the total Africa-wide funding gap for nets in the next three years (from the African Leaders Malaria Alliance)—estimated at $125 million per year. The estimate is rough in large part because the Global Fund to Fight AIDS, Tuberculosis and Malaria, the largest funder of LLINs, works on three-year cycles and has not yet determined how much funding it will allocate for LLINs for 2018-2020. We talked to people involved in country-level planning of mass net distributions and the Global Fund, who agreed with the general conclusion that there were likely to be large funding gaps in the next few years. In mid-2016, AMF had to put some plans on hold due to lack of funding. We now believe that AMF has a strong track record of finding distribution partners to work with and coming to agreements with governments, and we do not expect that to be a limiting factor for AMF. The main risks we see to AMF’s ability to scale are the possibility that funding from other funders is sufficient (since our estimate of the gap is quite rough), the likelihood that government actors have limited capacity for discussions with AMF during a year in which they are applying for Global Fund funding, AMF’s staff capacity to manage discussions with additional countries (it has only a few staff members), and whether gaps will be spread across many countries or located in difficult operating environments. We believe the probability of any specific one of these things impeding AMF’s progress is low. We believe there are differences in cost-effectiveness within execution level 1 and believe the value of filling the first part of AMF’s gap may be higher than additional funding at higher levels. This is because AMF’s priorities include committing to large distributions in the second half of 2019 and 2020, which increases the uncertainty about whether funding would have been available from another source. We and AMF have discussed a few possibilities for how AMF might fill funding gaps. AMF favors an approach where it purchases a large number of nets for a small number of countries. This approach has some advantages including efficiency for AMF and leverage in influencing how distributions are carried out. Our view is that the risk of displacing a large amount of funding from other funders using this approach outweighs the benefits. If AMF did displace a large amount of funding which would otherwise have gone to nets, that could make donations applied to these distributions considerably less cost-effective. More details on our assessment of AMF’s funding gap are in our full review. Deworm the World Initiative, led by Evidence Action Our full review of Deworm the World is here. Background Deworm the World (evidenceaction.org/#deworm-the-world), led by Evidence Action, advocates for, supports, and evaluates deworming programs. It has worked in India and Kenya for several years and has recently expanded to Nigeria, Vietnam, and Ethiopia. Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality. As noted above, we believe that Deworm the World is slightly more cost-effective than SCI, more cost-effective than AMF and the other deworming charities, and about 10 times as cost-effective as cash transfers. Important changes in the last 12 months Deworm the World has made somewhat slower progress than expected in expanding to new countries. In late 2015, Good Ventures, on GiveWell’s recommendation, made a grant of$10.8 million to Deworm the World to fund its execution level 1 and 2 gaps. Execution level 1 funding was to give Deworm the World sufficient resources to expand into Pakistan and another country. Deworm the World has funded a prevalence survey in Pakistan, which is a precursor to funding treatments in the country. It has not expanded into a further country that it was not already expecting to work in. As a result, we believe that Deworm the World has somewhat limited room for more funding this year.

Overall, we have more confidence in our understanding of Deworm the World and its parent organization Evidence Action’s spending, revenues, and financial position than we did in previous years. While trying to better understand this information this year, we found several errors. We are not fully confident that all errors have been corrected, though we are encouraged by the fact that we are now getting enough information to be able to spot inconsistencies. Evidence Action has been working to overhaul its financial system this year.

Our review of Deworm the World has focused on two countries, Kenya and India, where it has worked the longest. In 2016, we saw the first results of a program in another country (Vietnam), as well as continued high-quality monitoring from Kenya and India. The Vietnam results indicate that Deworm the World is using similar monitoring processes in new countries as it has in Kenya and India and that results in Vietnam have been reasonably strong.

Evidence Action hired Jeff Brown (formerly Interim CEO of the Global Innovation Fund) as CEO in 2015. Recently Evidence Action announced that he has resigned and has not yet been replaced. Our guess is this is unlikely to be disruptive to Deworm the World’s work; Grace Hollister remains Director of the Deworm the World Initiative.

Funding gap

We believe that there is a 50% chance that Deworm the World will be slightly constrained by funding in the next year and that additional funds would increase the chances that it is able to take advantage of any high-value opportunities it encounters. We estimate that if it received an additional $4.5 million its chances of being constrained by funding would be reduced to 20% and at$13.4 million in additional funding, this would be reduced to 5%.

In the next year, Deworm the World expects to expand its work in India and Nigeria and may have opportunities to begin treatments in Pakistan and Indonesia. It is also interested in using unrestricted funding to continue its work in Kenya, and puts a high priority on this program. Its work in Kenya has to date been funded primarily by the Children’s Investment Fund Foundation (CIFF) and this support is set to expire in mid 2017. It is unclear to us whether CIFF will continue providing funding for the program and, if so, for how long. Due to the possibility that Deworm the World unrestricted funding may displace funding from CIFF, and, to a lesser extent, the END Fund and other donors, we consider the opportunity to fund the Kenya program to be less cost-effective in expectation than it would be if we were confident in the size of the gap.

More details in our full review.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify recipient countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies in about two-thirds of the countries it works in to determine whether its programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is slightly less cost-effective than Deworm the World, more cost-effective than AMF and the other deworming charities, and about 8 times as cost-effective as cash transfers.

Important changes in the last 12 months

In past years, we’ve written that we had significant concerns about SCI’s financial reporting and financial management, and the clarity of our communication with SCI. In June, we wrote that we had learned of two substantial errors in SCI’s financial managment and reporting that began in 2015. We also noted that we thought that SCI’s financial management and financial reporting, as well as the clarity of its communication with us overall, had improved significantly. In the second half of the year, SCI communicated clearly with us about its plans for deworming programs next year and its room for more funding.

SCI reports that it has continued to scale up its deworming programs over the past year and that it plans to start up new deworming programs in two states in Nigeria before the end of its current budget year.

This year, SCI has shared a few more coverage surveys from deworming programs in Ethiopia, Madagascar, and Mozambique that found reasonably high coverage.

Professor Alan Fenwick, Founder and Director of SCI for over a decade, retired from his position this year, though will continue his involvement in fundraising and advocacy. The former Deputy Director, Wendy Harrison, is the new Director.

Funding gap

We estimate that SCI could productively use or commit a maximum of between $9.0 million (50% confidence) and$21.4 million (5% confidence) in additional unrestricted funding in its next budget year.

Its funding sources have been fairly steady in recent years with about half of its revenue in the form of restricted grants, particularly from the UK government’s Department for International Development (this grant runs through 2018), and half from unrestricted donations, a majority of which were driven by GiveWell’s recommendation. We estimate that SCI will have around $5.4 million in unrestricted funding available to allocate to its 2017-18 budget year (in addition to$6.5 million in restricted funding).

SCI has a strong track record of starting and scaling up programs in a large number of countries. SCI believes it could expand significantly with additional funding, reaching more people in the countries it works in and expanding to Nigeria and possibly Chad.

More details in our full review.

GiveDirectly

Our full review of GiveDirectly is here.

Background

GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82% overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people.

We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. It appears that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected (more). It continues to experiment heavily, with the aim of improving how its own and government cash transfer programs are run. It has recently started work on evaluations that benchmark programs against cash with the aim of influencing the broader international aid sector to use its funding more cost-effectively.

We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost effective than cash.

Important changes in the last 12 months

GiveDirectly has continued to scale up significantly, reaching a pace of delivering $21 million on an annual basis in the first part of 2016 and expecting to reach a pace of$50 million on an annual basis at the end of 2016. It has continued to share informative and detailed monitoring information with us. Given its strong and consistent monitoring in the past, we have taken a lighter-touch approach to evaluating its processes and results this year.

The big news for GiveDirectly this year was around partnerships and experimentation. It expanded into Rwanda (its third country) and launched a program to compare, with a randomized controlled trial, another aid program to cash transfers (details expected to be public next year). The program is being funded by a large institutional funder and Google.org. It expects to do additional “benchmarking” studies with the institutional funder, using funds from Good Ventures’ 2015 $25 million grant, over the next few years. It also began fundraising for and started a pilot of a universal basic income (UBI) guarantee—a program providing long-term, ongoing cash transfers sufficient for basic needs, which will be evaluated with a randomized controlled trial comparing the program to GiveDirectly’s standard lump sum transfers. The initial UBI program and study is expected to cost$30 million. We estimate that it is less cost-effective than GiveDirectly’s standard model, but it could have impact on policy makers that isn’t captured in our analysis.

We noted previously that Segovia, a for-profit technology company that develops software for cash transfer program implementers and which was started and is partially owned by GiveDirectly’s co-founders, would provide its software for free to GiveDirectly to avoid conflicts of interest. However, in 2016, after realizing that providing free services to GiveDirectly was too costly for Segovia (customizing the product for GiveDirectly required much more Segovia staff time than initially expected), the two organizations negotiated a new contract under which GiveDirectly will compensate Segovia for its services. GiveDirectly wrote about this decision here. GiveDirectly told us that it recused all people with ties to both organizations from this decision and evaluated alternatives to Segovia. Although we believe that there are possibilities for bias in this decision and in future decisions concerning Segovia, and we have not deeply vetted GiveDirectly’s connection with Segovia, overall we think GiveDirectly’s choices were reasonable. However, we believe that reasonable people might disagree with this opinion, which is in part based on our personal experience working closely with GiveDirectly’s staff for several years.

Funding gap

We believe that GiveDirectly is very likely to be constrained by funding next year. GiveDirectly has been rapidly building its capacity to enroll recipients and deliver funds, while some of its revenue has been redirected to its universal basic income guarantee program (either because of greater donor interest in that program or by GiveDirectly focusing its fundraising efforts on it).

We expect GiveDirectly to have about $20 million for standard cash transfers in its 2017 budget year. This includes raising about$15.8 million from non-GiveWell-influenced sources between now and halfway through its 2017 budget year (August 2017) and $4 million from donors who give because GiveDirectly is on GiveWell’s top charity list.$4 million is much less than GiveWell-influenced donors gave in the last year. This is because several large donors are supporting GiveDirectly’s universal basic income guarantee program this year and because one large donor gave a multi-year grant that we don’t expect to repeat this year.

GiveDirectly is currently on pace (with no additional hiring) to have four full teams operating its standard cash transfer model in 2017. To fully utilize four teams, it would need $28 million more than we expect it to raise. We accordingly expect that GiveDirectly will downsize somewhat in 2017, because we do not project it raising sufficient funds to fully utilize the increased capacity it has built to transfer money. Given recent growth, we believe that GiveDirectly could easily scale beyond four teams and we estimate that at$46 million more than we expect it to raise (66 million total for standard transfers), it would have a 50% chance of being constrained by funding. Other charities worthy of special recognition Last year, we recommended four organizations as “standouts.” This year we are calling this list “other charities worthy of special recognition.” We’ve added two organizations to the list: Food Fortification Initiative and Project Healthy Children. Although our recommendation to donors is to give to our top charities over these charities, they stand out from the vast majority of organizations we have considered in terms of the evidence base for their work and their transparency, and they offer additional giving options for donors who feel highly aligned with their work. We don’t follow these organizations as closely as we do our top charities. We generally have one or two calls per year with each group, publish notes on our conversations, and follow up on any major developments. We provide brief updates on these charities below: • Organizations that have conducted randomized controlled trials of their programs: • Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its program and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15% or more) and data from health facilities finding an increase in facility visits. (The results, because the trial was only completed in the last year, are not yet published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. We noted last year that DMI was planning to conduct another survey for the RCT in late 2016; it has decided not to move forward with this, but is interested in conducting new research studies in other countries, if it is able to raise the money to do so. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI, with conversation notes and documents from 2016, is here. • Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a randomized controlled trial of its program and measured a 27% reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of cash. Living Goods is scaling up its program and may need additional funding in the future, but has not yet been limited by funding. We published an update on Living Goods in mid-2016. Our 2014 review of Living Goods is here. • Organizations working on micronutrient fortification: We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence. • Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. Our full review is here. • Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN, published in 2016 based on research done in 2015, is here. • IGN. Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-influenced funding has made up a large part of its funding in the past year. This year, we published an update on our investigation into IGN’s work in select countries in 2015 and notes from our conversation with IGN to learn about its progress in 2016 and plans for 2017. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN, from 2014, is here. • Project Healthy Children (PHC). PHC aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs. Our review is preliminary and in particular we do not have a recent update on how PHC would use additional funding. Our review of PHC, published in 2016 but based on information collected in 2015, is here. Our research process in 2016 We plan to detail the work we completed this year in a future post as part of our annual review process. Much of this work, particularly our experimental work and work on prioritizing interventions for further investigation, is aimed at improving our recommendations in future years. Here we highlight the key research that led to our current recommendations. See our process page for our overall process. • As in previous years, we did intensive follow up with each of our top charities, including publishing updated reviews mid-year. We had several conversations by phone with each organization, met in person with Deworm the World, SCI, and AMF (over the course of a 4-day site visit to Ghana), and reviewed documents they shared with us. • In 2015 and 2016, we sought to expand top charity room for more funding and consider alternatives to our top charities by inviting other groups that work on deworming, bednet distributions, and micronutrient fortification to apply. This led to adding Sightsavers, the END Fund, Project Healthy Children, and Food Fortification Initiative to our lists this year. Episcopal Relief & Development’s NetsforLife® Program, Micronutrient Initiative, and Nothing but Nets declined to fully participate in our review process. • We completed intervention reports on voluntary medical male circumcision (VMMC) and cataract surgery. We asked VMMC groups PSI (declined to fully participate) and the Centre for HIV and AIDS Prevention Studies (pending) to apply. We had conversations with several charities working on cataract surgery and have not yet asked any to apply. • We did very preliminary investigations into a large number of interventions and prioritized a few for further work. This led to interim intervention reports on seasonal malaria chemoprevention (SMC), integrated community case management (iCCM) and ready-to-use therapeutic foods for treating severe acute malnutrition and recommending Malaria Consortium for its work on SMC. • We stayed up to date on the research for bednets, cash transfers, and deworming. We published a report on insecticide resistance and its implications for bednet programs. A blog post on our work on deworming is forthcoming. We did not find major new research on cash transfers that affected our recommendation of GiveDirectly. Giving to GiveWell vs. top charities GiveWell and the Open Philanthropy Project are planning to split into two organizations in the first half of 2017. The split means that it is likely that GiveWell will retain much of the assets of the previously larger organization while reducing its expenses. We think it’s fairly likely that our excess assets policy will be triggered and that we will grant out some unrestricted funds. Given that expectation, our recommendation to donors is: • If you have supported GiveWell’s operations in the past, we ask that you consider maintaining your support. It is fairly likely that these funds will be used this year for grants to top charities, but giving unrestricted signals your support for our operations and allows us to better project future revenue and make plans based on that. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and minimize staff time spent on fundraising. • If you have not supported GiveWell’s operations in the past, we ask that you consider checking the box on our donate form to add 10% to help fund GiveWell’s operations. In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations. Footnotes: * For example, if30 million were available to fund gaps of $10 million,$5 million, and $100 million, we would recommend allocating the funds so that the$10 million and $5 million gaps were fully filled and the$100 million gap received $15 million. The post Our updated top charities for giving season 2016 appeared first on The GiveWell Blog. ### Weighing organizational strength vs. estimated cost-effectiveness Thu, 07/14/2016 - 15:00 A major question we’ve asked ourselves internally over the last few years is how we should weigh organizational quality versus the value of the intervention that the organization is carrying out. In particular, is it better to recommend an organization we’re very impressed by and confident in that’s carrying out a good program, or better to recommend an organization we’re much less confident in that’s carrying out an exceptional program? This question has been most salient when deciding how to rank giving to GiveDirectly vs giving to the Schistosomiasis Control Initiative. GiveDirectly vs SCI GiveDirectly is an organization that we’re very impressed by and confident in, more so than any other charity we’ve come across in our history. Reasons for this: But, we estimate that marginal dollars to the program it implements — direct cash transfers — are significantly less cost-effective than bednets and deworming programs. Excluding organizational factors, our best guess is that deworming programs — which SCI supports — are roughly 5 times as cost-effective as cash transfers. As discussed further below, our cost effectiveness estimates are generally based on extremely limited information and are therefore extremely rough, so we are cautious in assigning too much weight to them. Despite the better cost-effectiveness of deworming, we’ve had significant issues with SCI as an organization. The two most important: • We originally relied on a set of studies showing dramatic drops in worm infection coinciding with SCI-run deworming programs to evaluate SCI’s track record; we later discovered flaws in the study methodology that led us to conclude that they did not demonstrate that SCI had a strong track record. We wrote about these flaws in 2013 and 2014. • We’ve seen limited and at times erroneous financial information from SCI over the years. We have seen some improvements in SCI’s financial reporting in 2016, but we still have some concerns, as detailed in our most recent report. More broadly, both of these cases are examples of general problems we’ve had communicating with SCI over the years. And we don’t believe SCI’s trajectory has generated evidence of overall impressiveness comparable to GiveDirectly’s, discussed above. Which should we recommend? One argument is that GiveWell should only recommend exceptional organizations, and so the issues we’ve seen with SCI should disqualify them. But, we think that the ~5x difference in cost-effectiveness is meaningful. There’s a large degree of uncertainty in our cost-effectiveness analyses, which is something we’ve written a lot about in the past, but this multiplier appears somewhat stable (it has persisted in this range over time, and currently is consistent with the individual estimates of many staff members), and a ~5x difference gives a fair amount of room for SCI to do more good even accounting both for possible errors in our analysis and for differences in organizational efficiency. A separate argument that we’ve made in the past is that great organizations have upside that goes beyond the value of conducting the specific program they’re implementing. For example, early funding to a great organization may have allow it to grow faster and increase the amount of money going to their program globally, either through proving the model or through their own fundraising. And GiveDirectly has shown some propensity for potentially innovative projects, as discussed above. We think that earlier funding to GiveDirectly had this benefit, but it’s less of a consideration now that GiveDirectly is a more mature organization. We believe this upside exists for what we’ve called “capacity-relevant” funding, which is the type of funding need that we consider to be most valuable when ranking the importance of marginal dollars to each of our top charities, and refers to funding gaps that we expect will allow organizations to grow in an outsized way in the future, for instance by going into a new country. Bottom line Our most recent recommendations ranked SCI’s funding gap higher than GiveDirectly’s due to SCI’s cost-effectiveness. We think that SCI is a strong organization overall, despite the issues we’ve noted, and we think that the “upside” for GiveDirectly is limited on the margin, so ultimately our estimated 5x multiplier looks meaningful enough to be determinative. We remain conflicted about this tradeoff and regularly debate it internally, and we think reasonable donors may disagree about which organization to support. The post Weighing organizational strength vs. estimated cost-effectiveness appeared first on The GiveWell Blog. ### Mid-year update to top charity recommendations Thu, 06/23/2016 - 17:25 This post provides an update on what we’ve learned about our top charities in the first half of 2016. We continue to recommend all four of our top charities. Our recommendation for donors seeking to directly follow our advice remains the same: we recommend they give to the Against Malaria Foundation (AMF), which we believe has the most valuable current funding gap. Below, we provide: • Updates on our view about AMF, which we consider the most important information we’ve learned in the last half-year (More) • Updates on other top charities (More) • A discussion of the reasoning behind our current recommendation to donors (More) Updates on AMF Background AMF (www.againstmalaria.com) provides funding for long-lasting insecticide-treated net distributions (for protection against malaria) in developing countries. There is strong evidence that distributing nets reduces child mortality and malaria cases. AMF has relatively strong reporting requirements for its distribution partners and provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity. Overall, AMF is the best giving opportunity we are currently aware of. That said, we have concerns about AMF’s recent monitoring and transparency that we plan to focus on in the second half of the year. Updates from the last six months We are more confident than we were before in AMF’s ability to successfully complete deals with most countries it engages with. Over the past few years, our key concern about AMF has been whether it would be able to effectively absorb additional funding and sign distribution agreements with governments and other partners. At the end of 2013, we stopped recommending AMF because we felt it did not require additional funding, and our end-of-year analyses in 2014 and 2015 discussed this issue in depth. In early 2016, AMF signed agreements to fund two large distributions (totaling$37 million) of insecticide-treated nets in countries it has not previously worked in. We now believe that AMF has effectively addressed this concern.

AMF is in discussions for several additional large distributions. AMF currently holds approximately $23.3 million, and we believe that it is very likely to have to slow its work if it receives less than an additional$11 million very quickly. It is possible that it could also use up to an additional (approximately) $18 million more during this calendar year. It may be more valuable to give to AMF now than it will be later this year or next year. AMF’s funding gap may be time-sensitive because: 1. AMF is in several discussions about distributions that would take place in 2017. It has told us that it needs to make decisions within a month or two about which discussions to pursue. We don’t have a clear sense for how long before a distribution AMF needs to be able to commit funding, and note that, for example, AMF committed in February 2016 to a distribution in Ghana taking place in June to August 2016. That said, it seems quite plausible that AMF needs to commit soon to distributions taking place in 2017. 2. We don’t know whether there will be large funding gaps for nets in 2018 and beyond. The price of nets has been decreasing and the size of grants from the two largest funders of nets, the Global Fund to fight AIDS, TB, and Malaria and the President’s Malaria Initiative, is not yet known. (The Global Fund is holding its replenishment conference in September, in which donor governments are asked to make three-year pledges, so we may know more before the end of the year.) It’s possible that these funders will fund all or nearly all of the net needs in countries other than those that are particularly hard to work in for 2018. If that happens, gifts to AMF in late 2016 could be less valuable than gifts in the next couple of months. (This could also mean that, if AMF fills gaps in 2017 that would have been filled by other funders in 2018, gifts now are less valuable than they have been in the past. We have added an adjustment for this to our cost-effectiveness analysis, but given the high degree of uncertainty, this could be a more important factor than we are currently adjusting for.) Notwithstanding the above, we have important questions about AMF that we plan to continue to investigate. None of these developments caused us to change our recommendation about giving to AMF, but they are important considerations for donors: 1. Monitoring data: We have new concerns about AMF’s monitoring of its distributions, particularly its post-distribution check-up (PDCU) surveys. These surveys are a key part of our confidence in the quality of AMF’s distributions. For Malawi, where most of the PDCUs completed to date have been done, our key concern is that villages that surveyors visit are not selected randomly, but are instead selected by hand by staff of the organization that both implements and monitors the distributions, which seems fairly likely to lead to bias in the results. We have also seen results from the first two PDCUs from DRC. We have not yet looked at the DRC results in-depth or discussed them with AMF, but there appear to be major problems in how the surveys were carried out (particularly a high percentage of internally inconsistent data – around 40%-50%) and, if we believe the remaining data, fairly high rates of missing or unhung nets (~20% at 6-months) and nets that deteriorated quickly (65% were in ‘very good’ or ‘good’ condition at 6-months). 2. Transparency: Recently, AMF has been slower to share documentation from some distributions. AMF has told us that it has this documentation and we are concerned that AMF is not being as transparent as it could be. We believe this documentation is important for monitoring the quality of AMF’s distributions; it includes PDCUs, results from re-surveying 5% of households in during pre-distribution registrations (AMF has told us that this is a standard part of its process, but we have not seen results from any distributions), and malaria case rate data from Malawi that AMF has told us it has on hand. AMF attributes the delays to lack of staff capacity. We plan to write more about monitoring and transparency in a future post. 3. Insecticide resistance: Insecticide resistance (defined broadly as “any ways in which populations of mosquitoes adapt to the presence of insecticide-treated nets (ITNs) in order to make them less effective”) is a major threat to the effectiveness of ITNs. Insecticide resistance seems to be fairly common across sub-Saharan Africa, and it seems that resistance is increasing. It remains difficult to quantify the impact of resistance, but our very rough best guess (methodology described in more detail below) is that ITNs are roughly one-third less effective in the areas where AMF is working than they would be in the absence of insecticide resistance. We continue to believe, despite resistance, ITNs remain a highly cost-effective intervention. See our full report for more detail. Other updates on AMF • To better understand whether AMF is providing nets that would not otherwise have been funded, we considered five cases where AMF considered funding a distribution and did not ultimately provide funding. We then looked at whether other funders stepped in and how long of a delay resulted from having to wait for other funders. We published the details here. In short, most distributions took place later than they would have if AMF had funded them (on average over a year), which probably means that the people were not protected with nets during that time. We feel that these case studies provide some evidence that nets that AMF buys do not simply displace nets from other funding sources. • We’ve noted in the past that the delays in AMF signing agreements for distributions may have been due to AMF’s hesitation about paying for the costs of a distribution other than the purchase price of nets. For the distributions that AMF has signed this year, AMF has agreed to pay for some non-net costs, particularly the costs of PDCUs. The Global Fund to fight AIDS, TB, and Malaria is paying for the other non-net costs of the distribution. AMF’s willingness to fund some of the non-net costs may have made it easier for it to sign distribution agreements and put funds to use more quickly. Updates on our other top charities Schistosomiasis Control Initiative (full report) Background SCI (www3.imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs (treating children for schistosomiasis and other intestinal parasites). SCI’s role has primarily been to identify recipient countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs. In past years, we’ve written that we had significant concerns about SCI’s financial reporting and financial management that meant we lacked high-quality, basic information about how SCI was spending funding and how much funding it had available to allocate to programs. We decided to focus our work in the first half of 2016 on this issue. We felt that seeing significant improvements in the quality of SCI’s finances was necessary for us to continue recommending SCI. We believe that deworming is a program backed by relatively strong evidence. We have reservations about the evidence, but we think the potential benefits are great enough, and costs low enough, to outweigh these reservations. SCI has conducted studies in about half of the countries it works in (including the countries with the largest programs) to determine whether its programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but have major methodological limitations. We have not asked SCI for monitoring results since last year. Updates from the last six months We published a separate blog post on our work on SCI so far this year. Our main takeaways: • SCI has begun producing higher-quality financial documents that allow us to learn some basic financial information about SCI. • We learned of two substantial errors in SCI’s financial management and reporting. 1) a July 2015 grant from GiveWell for about$333,000 was misallocated within Imperial College, which houses SCI, until we noticed it was missing from SCI’s revenue in March 2016; and (2) in 2015, SCI underreported how much funding it would have from other sources in 2016, leading us to overestimate its room for more funding by $1.5 million. • The clarity of our communication with SCI about its finances has improved, but there is still substantial room for further improvement. We feel that SCI has improved, but we would still rank our other top charities ahead of it in terms of our ability to communicate and understand their work. Given this situation, we continue to recommend SCI now and think that SCI is reasonably likely to retain its top charity status at the end of 2016. We plan, in the second half of 2016, to expand the scope of our research on SCI. We have not asked SCI for an update on its room for more funding (due to our focus on financial documents in the first half of the year). It’s our understanding that funds that SCI receives in the next six months will be allocated to work in 2017 and beyond. Because of this, we don’t believe that SCI has a pressing need for additional funds, though our guess is that it will have room for more funding when we next update our recommendations in November and that funds given before then will help fund gaps for the next budget year. GiveDirectly (full report) Background GiveDirectly (www.givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 83% overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people. We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. It appears that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected (more). It continues to experiment heavily. Updates from the last six months • GiveDirectly announced an initiative to test a “basic income guarantee” to provide long-term, ongoing cash transfers sufficient for basic needs. The cost-effectiveness of providing this form of cash transfers may be different from the one-time transfers GiveDirectly has made in the past. • GiveDirectly continues to have more room for more funding than we expect GiveWell-influenced donors to fill in the next six months. Its top priority is funding the basic income guarantee project. • In late 2015 and early 2016, when GiveDirectly began enrolling participants in Homa Bay county, Kenya, it experienced a high rate of people refusing to be enrolled in the program. The reason for this is not fully clear, though GiveDirectly believes in some cases local leaders advised people to not trust the program. While GiveDirectly has temporarily dealt with this setback by moving its operations to a different location in Homa Bay county, it is possible that similar future challenges could reduce GiveDirectly’s ability to commit as much as it currently projects. • GiveDirectly has reached an agreement with a major funder which provides a mechanism through which multiple benchmarking projects (projects comparing cash transfers to other types of aid programs) can be launched. The major funder may fund up to$15 million for four different benchmarking projects with GiveDirectly. GiveDirectly plans to make available up to $15 million of the grant it received from Good Ventures in 2015 to match funds committed by the major funder. GiveDirectly and its partner have not yet determined which aid programs will be evaluated or how the evaluations will be carried out. • We are reasonably confident that GiveDirectly could effectively use significantly more funding than we expect it to receive, including an additional$30 million for additional cash transfers in 2016, though scaling up to this size would require a major acceleration in the second half of the year. We have not asked GiveDirectly how funding above this amount would affect its activities and plans (because we think it is very unlikely that GiveDirectly will receive more than $30 million from GiveWell-influenced supporters before our next update in November). Deworm the World (full report) Background Deworm the World (www.evidenceaction.org/deworming), led by Evidence Action, advocates for, supports, and evaluates government-run school-based deworming programs (treating children for intestinal parasites). We believe that deworming is a program backed by relatively strong evidence. We have reservations about the evidence, but we think the potential benefits are great enough, and costs low enough, to outweigh these reservations. Deworm the World retains monitors whose reports indicate that the deworming programs it supports successfully deworm children. Updates from the last six months • We asked Deworm the World whether additional funding in the next six months would change its activities or plans. It told us that it does not expect funding to be the bottleneck to any work in that time. We’d guess that there is a very small chance that it will encounter an unexpected opportunity and be bottlenecked by funding before our next update in November. • Deworm the World appears to be making progress expanding to new countries. It has made a multi-year commitment to provide technical assistance and resources to Cross River state, Nigeria for its school-based deworming program (the first deworming is scheduled for the end of this month), and are undertaking a nationwide prevalence survey in Pakistan. • In the past, we have focused our review of Deworm the World on its work in India. We are in the process of learning more about its work in other locations, particularly Kenya. The monitoring we have seen from Kenya appears to be high quality. Summary of key considerations for top charities The table below summarizes the key considerations for our four top charities. With the exception of modest changes to room for more funding, our high-level view of our top charities, as summarized in the table below, is the same as at our last update in November 2015. Consideration AMF Deworm the World GiveDirectly SCI Program estimated cost-effectiveness (relative to cash transfers) ~10x ~10x Baseline ~5x Directness and robustness of the case for impact Strong Moderate Strongest Moderate Transparency and communication Strong Strong Strongest Weakest Ongoing monitoring and likelihood of detecting future problems Strong Strong Strongest Weakest Organizational track record of rolling out program Moderate Moderate Strong Strong Room for more funding High Limited High Likely moderate (not investigated) Reasoning behind our current recommendation to donors Our recommendation for donors seeking to directly follow our advice is to give to AMF, which we believe has the most valuable current funding gap. We believe AMF will likely have opportunities to fund distributions this year which it will not be able to fund without additional funding. Due to the excellent cost-effectiveness of AMF’s work, we consider this a highly valuable funding gap to fill. Our current estimate is that on average AMF saves a life for about every$3,500 that it spends; this is an increase from our November 2015 estimate and reflects changes to our cost-effectiveness model as well as some of our inputs into bed nets’ cost-effectiveness. As always, we advise against taking cost-effectiveness estimates literally and view them as highly uncertain.

The below table lays out our ranking of funding gaps for June to November 2016. The first million dollars to a charity can have a very different impact from, e.g., the 20th million dollars. Accordingly, our ranking of individual funding gaps accounts for both (a) the quality of the charity and the good accomplished by its program, per dollar, and (b) whether a given level of funding is highly or only marginally likely to be needed in the next six months.
We consider funding that allows a charity to implement more of its core program (without substantial benefits beyond the direct good accomplished by this program) to be “execution funding.” We’ve separated this funding into three levels:

• Level 1: the amount we expect a charity to need in the coming year. If a charity has less funding than this level, we think it is more likely than not that it will be bottlenecked (or unable to carry out its core program to the fullest extent) by funding in the coming year. For this mid-year update, we have focused on funds that are needed before our next update in November, with the exception of SCI where we believe funds will not affect its work until next year.
• Level 2: if a charity has this amount, we think there is an ~80% chance that it will not be bottlenecked by funding.
• Level 3: if a charity has this amount, we think there is a ~95% chance that it will not be bottlenecked by funding.

(Our rankings can also take into account whether a gap is “capacity-relevant” or providing an incentive to engage in our process. We do not currently believe that our top charities have capacity-relevant gaps and are not planning to make mid-year incentive grants, so we haven’t gone into detail on that here. More details on how we think about capacity-relevant and execution gaps in this post.)

Priority Charity Amount (millions) Type Description Comment 1 AMF $11.3 Execution level 1 Fund distributions in two countries that AMF is in discussions with but does not have sufficient funding for AMF is strongest overall 2 AMF$7.3 Execution level 2 Fund the next largest gap on the list of remaining 2016-17 gaps in African countries – 3 SCI $10.1 Execution level 1 Very rough because we haven’t discussed this with SCI; further gaps not estimated Not as strong as AMF in isolation, so ranked below for same type of gap 4 AMF$10.5 Execution level 3 Fund the final two AMF-relevant gaps on the list of remaining 2016-17 gaps in African countries – 5 GiveDirectly $22.2 Execution level 1 Basic income guarantee program and additional standard transfers Not as cost-effective as bednets or deworming, so lower priority 6 Deworm the World$6.0 Execution level 3 A rough guess at the funding needed to cover a 3-year deworming program in a new country Strong cost-effectiveness, but unlikely to need funds in the short-term 6 GiveDirectly $7.8 Execution level 2 Funding for additional structured projects; further gaps not estimated – We are not recommending that Good Ventures make grants to our top charities for this mid-year refresh. In November 2015, we recommended that Good Ventures fund 50% of our top charities’ highest-value funding gaps for the year and Good Ventures gave$44.4 million to our top four charities. We felt this approach resulted in Good Ventures funding its “fair share” while avoiding creating incentives for other donors to avoid the causes we’re interested in, which could lead to less overall funding for these causes in the long run. (More on this reasoning available here.)

The post Mid-year update to top charity recommendations appeared first on The GiveWell Blog.

### Our updated top charities for giving season 2015

Wed, 11/18/2015 - 14:06

We have refreshed our top charity rankings and recommendations. Our set of top charities and standouts is the same as last year’s, but we have introduced rankings and changed our recommended funding allocation, due to a variety of updates – particularly to our top charities’ room for more funding. In particular, we are recommending that Good Ventures, a foundation with which we work closely, support our top charities at a higher level than in previous years. This post includes our recommendations to Good Ventures, and gives our recommendations to individual donors after accounting for these grants.

Overall, we think the case for our top charities is stronger than in previous years, and room for more funding is greater.

Our top charities and recommendations for donors, in brief

Top charities

1. Against Malaria Foundation (AMF)
2. Schistosomiasis Control Initiative (SCI)
3. Deworm the World Initiative, led by Evidence Action
4. GiveDirectly

This year, we are ranking our top charities based on what we see as the value of filling their remaining funding gaps. Unlike in previous years, we do not feel a particular need for individuals to divide their allocation between the charities, since we are recommending that Good Ventures provide significant support to each. For those seeking our recommended allocation, we simply recommend giving to the top-ranked charity on the list, which is AMF.

Our recommendation takes the grants we are recommending to Good Ventures into account, as well as accounting for charities’ existing cash on hand and expected non-GiveWell-related fundraising, and recommends charities according to how much good additional donations (beyond these sources of funds) can do. (Otherwise, as explained below, Deworm the World would be ranked higher.) Thus, AMF’s #1 ranking is not based on its overall value as an organization, but based on the value of its remaining funding gap.

Standout charities

As with last year, we also provide a list of charities that we believe are strong standouts, though not at the same level (in terms of likely good accomplished per dollar) as our top charities. They are not ranked, and are listed in alphabetical order.

Below, we provide:

• An explanation of major changes in the past year that are not specific to any one charity. More
• A summary of our top charities’ relative strengths and weaknesses, and how we would rank them if room for more funding were not an issue. More
• A discussion of our refined approach to room for more funding. More
• The recommendations we are making to Good Ventures, and how we rank our top charities after taking these grants (and their impact on room for more funding) into account. More
• Detail on each of our top charities, including major changes over the past year, strengths and weaknesses for each, and our understanding of each organization’s room for more funding. More
• The process we followed that led to these recommendations. More
• A brief update on giving to support GiveWell’s operations vs. giving to our top charities. More

Conference call to discuss recommendations

We are planning to hold a conference call at 5:30pm ET/2:30pm PT on Tuesday, December 1st to discuss our recommendations and answer questions.

If you’d like to join the call, please register using this online form. If you can’t make this date but would be interested in joining another call at a later date, please indicate this on the registration form.

Major changes in the last 12 months

Below, we summarize the major causes of changes to our recommendations (since last year).

Overall, the case for our top charities is stronger than it was in past years. The Deworm the World Initiative shared new monitoring and evaluation materials with us, so we are more confident than we were a year ago that it is a strong organization implementing high-quality programs. In addition, the extra year of work we have seen from AMF and GiveDirectly bolsters our view that they will be able to utilize additional funding effectively.

Our top charities have increased room for more funding. Last year, we expected donors following our recommendations to fully fill the most critical funding gaps of our top charities (excluding GiveDirectly) because they had limited room for more funding: GiveDirectly had a total funding gap of ~$40 million and our other three top charities had a total gap of ~$18 million. This year, all of our top charities have more room for more funding. We believe that GiveDirectly could absorb more than $80 million and other top charities together could collectively utilize more than$100 million. We do not expect donors following our recommendations to fully fill these gaps.

We are recommending that Good Ventures make larger grants to top charities. For reasons we will be detailing in a future post, we are recommending that Good Ventures make substantial grants to our top charities this year, though not enough to close their funding gaps.

Continued refinement of the concept of “room for more funding.” We’ve tried to create a much more systematic and detailed room for more funding analysis, because the stakes of this analysis have become higher due to (a) increased room for more funding across the board and (b) increased interest from Good Ventures in providing major support.

In past years, we’ve discussed charities’ room for more funding as a single figure without distinguishing between (a) the amount the charity would spend in the next 12 months, (b) the amount the charity needs to prevent it from slowing its work due to lack of funds, and (c) funding that would be especially important to the organization’s development and success (a dual benefit) in addition to expanding implementation of its program. This year, we’ve made three changes to our room for more funding analysis:

• We’ve made (a) an assessment of whether additional funds merely allow a charity to implement its program (“execution”) or (b) whether additional funds would be especially important to the charity’s development and success as an organization (“capacity-relevant”). We also explicitly note the role of incentives for meeting GiveWell’s top-charity criteria in our recommendations (we seek to ensure that each top charity receives at least $1 million, to encourage other organizations to seek to meet these criteria). • We are explicitly assessing “execution”-related room for more funding based on our estimate of the probability that lack of funding will lead to a charity slowing its progress. We distinguish between Level 1, Level 2, and Level 3 “execution” funding gaps; a higher number means the money is less likely to be needed. • We are now ranking “funding gaps,” not just ranking charities, because the first million dollars to a charity can have a very different impact from, e.g., the 20th million dollars. For example, if Charity A accomplishes more good per dollar with its programs than Charity B, we would rank Charity A above Charity B for a given type of gap (we would rank Charity A’s “Execution Level 1” gap above Charity B’s), but we might rank Charity B’s “Execution Level 1” gap (the amount of funding it will likely need) above Charity A’s “Execution Level 3” gap (the amount of funding gap it might, but probably will not, need to carry out more of its programs in the coming year). We discuss these ideas in greater depth below. Summary of key considerations for top charities The table below summarizes the key considerations for our four top charities. More detail is provided below as well as in the charity reviews. Consideration AMF Deworm the World GiveDirectly SCI Program estimated cost-effectiveness (relative to cash transfers) ~10x ~10x Baseline ~5x Directness and robustness of the case for impact Strong Moderate Strongest Moderate Transparency and communication Strong Strong Strongest Weakest Ongoing monitoring and likelihood of detecting future problems Strong Strong Strongest Weakest Organizational track record of rolling out program Moderate Moderate Strong Strong Room for more funding, after accounting for grants we are recommending to Good Ventures (more below) Very high Limited Very high High Overall, our ranking of the charities with room for more funding issues set aside (just considering a hypothetical dollar spent by the charity on its programs, without the “capacity-relevant funding” and “incentives” issues discussed below) would be: 1. AMF and Deworm the World 3. SCI 4. GiveDirectly However, when we factor in room for more funding (including the impact of the grants we’re recommending to Good Ventures), the picture changes. More on this below. Room for more funding analysis Capacity-relevant funding and incentives Capacity-relevant funding: additional funding can sometimes be crucial for a charity’s development and success as an organization. For example, it can contribute to a charity’s ability to experiment, expand, and ultimately have greater room for more funding over the long run. It can also be important for a charity’s ability to raise funds from non-GiveWell donors, which can be an important source of long-term leverage and can put the organization in a stronger overall position. We think of this sort of funding gap as particularly important to fill, because it can make a big difference over the long run; in particular, it may substantially affect the long-term quality of our giving recommendations. “Capacity-relevant” funds can include (a) funds that are explicitly targeted at growth (e.g., funds to hire fundraising staff); (b) funds that enable a charity to expand into areas it hasn’t worked in before, which can lead to important learning about whether and how the charity can operate in the new location(s); and (c) funds that would be needed in order to avoid challenging contractions in a charity’s activities which could jeopardize the charity’s long-term growth and funding prospects. Some specific examples: • The grant that Good Ventures made to GiveDirectly earlier this year is capacity-relevant because it will be used for: (a) building a fundraising team that will aim to raise substantial donations from non-GiveWell donors, and (b) developing partnerships with bilateral donors and local governments to deliver cash transfers or to run experiments comparing standard aid programs to cash transfers. • Early funding that GiveDirectly received was capacity-relevant because it enabled GiveDirectly to rapidly grow from a small organization moving a few hundred thousand dollars per year to a much larger organization moving more than$10 million per year. If this funding hadn’t been forthcoming, GiveDirectly might be much smaller today and have much less room for more funding.
• We now think that some additional funding to AMF and Deworm the World will be capacity-relevant because each organization has only operated in a very small number of countries and new funding will enable each to enter new countries. This will allow them to learn how to operate there, and demonstrate that they can do so, increasing our willingness (and likely that of other donors) to recommend more to these organizations in the future.

It’s hard to draw sharp lines around capacity-relevant funding, and all funding likely has some effect on an organization’s development, but we have tried to identify and prioritize the funding gaps that seem especially relevant.

Execution funding allows charities to implement more of their core program but doesn’t appear to have substantial benefits beyond the direct good accomplished by this program. We’ve separated this funding into three levels:

• Level 1: the amount we expect a charity to need in the coming year. If a charity has less funding than this level, we think it is more likely than not that it will be bottlenecked (or unable to carry out its core program to the fullest extent) by funding in the coming year.
• Level 2: if a charity has this amount, we think there is an ~80% chance that it will not be bottlenecked by funding.
• Level 3: if a charity has this amount, we think there is a ~95% chance that it will not be bottlenecked by funding.

Incentives: we think it is important that charities we recommend get a substantial amount of funding due to being a GiveWell top charity, because this ensures that incentives are in place for charities (and potential charity founders) to seek to meet our criteria for top charities and thus increase the number of charities we recommend and the total room for more funding available, even when they don’t end up being ranked #1. We seek to ensure that each top charity gets at least $1 million as a result of our recommendation, and we consider this to be a high-priority goal of our recommendations. The charity-specific sections of this post discuss the reasoning behind the figures we’ve assigned to “capacity-relevant” and “Execution Level 1” gaps, but they do not provide the full details of how we arrived at these figures (and do not explicitly address the “Execution Level 2” and “Execution Level 3” gaps). We expect to add this analysis to our charity reviews in the coming weeks. Funding gaps The total (i.e., Capacity-relevant, Execution Levels 1, 2, and 3, and Incentive) funding gaps (in millions of dollars, rounded to one decimal place) for each of our top charities are: • AMF:$98.2
• Deworm the World: $19.0 • GiveDirectly:$84.0
• SCI: $26.3 However, for reasons described above, the first million dollars to a charity can have a very different impact from, e.g., the 20th million dollars. Accordingly, we have created a ranking of individual funding gaps that accounts for both (a) the quality of the charity and the good accomplished by its program, per dollar (as laid out above), and (b) whether a given level of funding is capacity-relevant and whether it is highly or only marginally likely to be needed in the coming year. The below table lays out our ranking of funding gaps. When gaps have the same “Priority,” this indicates that they are tied. The table below includes the amount we are recommending to Good Ventures. For reasons we will lay out in another post, we are recommending to Good Ventures a total of ~$44.4 million in grants to top charities. Having set that total, we are recommending that Good Ventures start with funding the highest-rated gaps and work its way down, in order to accomplish as much good as possible.

When gaps are tied, we recommend filling them by giving each equal dollar amounts until one is filled, and then following the same procedure with the remaining tied gaps. See footnote for more.*

Priority Charity Amount Type Recommendation to Good Ventures Comments 1 DtWI $7.6 Capacity-relevant$7.6 DtWI and AMF are strongest overall 1 AMF $6.5 Capacity-relevant$6.5 See above 1 GD $1.0 Incentive$1.0 Ensuring each top charity receives at least $1 million 1 SCI$1.0 Incentive $1.0 Ensuring each top charity receives at least$1 million 2 GD $8.8 Capacity-relevant$8.8 Not as cost-effective as bednets or deworming, so lower priority, but above non-capacity-relevant gaps 2 DtWI $3.2 Execution Level 2 / possibly capacity-relevant$3.2 Level 1 gap already filled via “capacity-relevant” gap. See footnote for more** 2 AMF $43.8 Execution Level 1$16.3 Exhausts remaining recommendations to Good Ventures 3 SCI $4.9 Execution Level 1 0 Not as strong as DtWI and AMF in isolation, so ranked below them for same type of gap 3 AMF$24.0 Execution Level 2 0 – 4 DtWI $8.2 Execution Level 3 0 – 4 AMF$24.0 Execution Level 3 0 – 4 SCI $11.6 Execution Level 2 0 – 5 GD$24.8 Execution Level 1 0 – 5 SCI $8.8 Execution Level 3 0 – 6 GD$20.9 Execution Level 2 0 – 7 GD $28.6 Execution Level 3 0 – Our recommendations to Good Ventures and others Summing the figures from the above table, we are recommending that Good Ventures make the following grants (in millions of dollars, rounded to one decimal place): • AMF:$22.8
• Deworm the World: $10.8 • GiveDirectly:$9.8
• SCI: $1 We also recommend that Good Ventures give$250,000 to each of our standout charities. These grants go to the outstanding organizations and create additional incentives for groups to try to obtain a GiveWell recommendation.

After these grants, AMF will require an additional ~$27.5 million to close its Execution Level 1 gap (i.e., to make it more likely than not that it is able to proceed without being bottlenecked due to lack of funding). We rank this gap higher than any of the other remaining funding gaps for our top charities, as laid out in the table above. We estimate that non-Good Ventures donors will give approximately$15 million between now and January 31, 2016. Because we do not expect AMF’s remaining ~$27.5 million Execution Level 1 funding gap to be fully filled, we rank it #1 and recommend that donors give to AMF. We rank the remaining charities for donors who are interested in having the greatest impact per dollar based on how highly their highest-rated remaining gap ranks in the table above. That results in the following rankings for individual donors: 1. AMF 2. SCI 3. Deworm the World Initiative 4. GiveDirectly Details on top charities We present information on our top charities in alphabetical order. Against Malaria Foundation (AMF) Our full review of AMF is here. Background AMF (www.againstmalaria.com) provides funding for long-lasting insecticide-treated net distributions (for protection against malaria) in developing countries. There is strong evidence that distributing nets reduces child mortality and malaria cases. AMF has relatively strong reporting requirements for its distribution partners and provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity. In 2011, AMF received a large amount of funding relative to what it had received historically, so it began to focus primarily on reaching agreements for large-scale net distributions (i.e., distributions on the order of hundreds of thousands of nets rather than tens of thousands of nets). In its early efforts to scale up, AMF struggled to finalize large-scale net distribution agreements. At the end of 2013, we announced that we planned not to recommend additional donations to AMF due to room for more funding-related issues (more detail in this blog post). In 2014, AMF committed most of its funds to several new distributions — some in Malawi, some in the Democratic Republic of the Congo (DRC) — and we recommended it as a top charity again. Important changes in the last 12 months Previously, our confidence in AMF’s ability to scale had been limited by the fact that it had only completed large-scale distributions with one partner (Concern Universal) in one country (Malawi). However, AMF carried out its largest distribution to date (~620,000 nets) with a new partner in the DRC in late 2014. We have not yet seen some key documentation from the large DRC distribution, but early indications suggest that the distribution generally went as planned, despite our concern that the DRC may have been an especially challenging place to work (more details here). We see this as a positive update that AMF will be able to carry out high-quality large-scale distributions in a variety of locations in the future. AMF has continued to collect and share follow-up information on its past large-scale distributions, and this information seems to support the notion that these distributions are high-quality (i.e., that nets are reaching the target population and are being used). We provide a summary of these reports in our review. Funding gap AMF currently holds$18.5 million, and we estimate it will receive an additional $1.6 million before January 31, 2016 (excluding donations influenced by GiveWell) that it could use for future distributions. AMF has told us that it has a pipeline of possible future net distributions that add up to roughly$100 million beyond what it currently holds (details in our review).

We believe that AMF’s progress would be slowed due to lack of funding were it to receive less than $50.3 million in additional funding (this is its total capacity-relevant and “Execution Level 1” gap as presented earlier in the post). In particular, we view the first additional$6.5 million that AMF would receive as capacity-relevant (and thus particularly valuable) because it would enable AMF to fund a distribution in a 5th country with a 5th partner, generating additional information about its ability to expand beyond the contexts in which it has worked to date. (Note that AMF already has funds on hand to enter its 3rd and 4th countries.)

We arrived at the capacity-relevant and Execution Level 1 figure by noting that AMF has $70.4 million worth of deals it is actively negotiating (5 deals in 4 countries) that it can only continue with if it holds the funds to do so. Subtracting the$20.1 million we expect to be available (the $18.5 million it currently holds plus the$1.6 million we expect it to receive in the coming months) leaves a $50.3 million funding gap. AMF failed to reach new distribution agreements in 2015; there is still significant uncertainty regarding AMF’s ability to finalize agreements with new partners and countries. Nevertheless, we see providing a large amount of additional funds to AMF as a reasonable bet, and see AMF as a very strong giving opportunity. We think it is possible that in November 2016 (when we next expect to complete a full refresh of our recommendations), we will recommend significantly less funding to AMF. We consider the funding we’re recommending to AMF now to be a good bet, but a risky one, because AMF currently has a relatively limited track record: it has worked with only two partners in two countries. Because of the lag between the time we provide funding and the time net distributions take place (often 2 years) and the additional lag caused by the time it takes to monitor distributions, we may not have additional information about whether or not AMF’s additional distributions were successful for 2-3 years. Next year, it is possible that we will choose to recommend significantly less funding to AMF while we wait for additional data to become available. There still appears to be a large global funding gap for bednets; a global bednet coordination group estimated that about 245 million additional nets would be needed in 2015-2017 (details in our review). Key considerations: • Program impact and cost-effectiveness. We estimate that bednets are ~10x as cost-effective as cash transfers. Our estimates are subject to substantial uncertainty. All of our cost-effectiveness analyses are available here. Our 2015 cost-effectiveness file is available here (.xlsx). • Directness and robustness of the case for impact. We believe that the connection between AMF receiving funds and those funds helping very poor individuals is less direct than GiveDirectly’s and more direct than SCI’s or Deworm the World’s. The uncertainty of our estimates is driven by a combination of AMF’s challenges historically disbursing the funds it receives and a general recognition that aid programs, even those as straightforward as bednets, carry significant risks of failure via ineffective use of nets, insecticide resistance, or other risks we don’t yet recognize relative to GiveDirectly’s program. AMF conducts extensive monitoring of its program; these results have generally indicated that people use the nets they receive. • Transparency and communication. AMF has been extremely communicative and open with us. We feel we have a better understanding of AMF than of SCI, and a similar level of knowledge about AMF as we have for Deworm the World, though our understanding is not as strong as our understanding of GiveDirectly. In particular, were something to go wrong in one of AMF’s distributions, we believe we would eventually find out (something we are not sure of in the case of SCI), but we believe our understanding would be less quick and complete than it would be for problems associated with GiveDirectly’s program (which has more of a track record of consistent intensive follow-up). • Risks: • We are not highly confident that AMF will be able to finalize additional distributions and do so quickly. AMF could struggle again to agree to distribution deals, leading to long delays before it spends funds. We view this as a relatively minor risk because the likely worst-case scenario is that AMF spends the funds slowly (or returns funds to donors). • We remain concerned about the possibility of resistance to the insecticides used in bednets. There don’t appear to be major updates on this front since our 2012 investigation into the matter; we take the lack of major news as a minor positive update. Our full review of AMF is here. Deworm the World Initiative, led by Evidence Action Our full review of Deworm the World is here. Background Deworm the World (www.evidenceaction.org/deworming), led by Evidence Action, advocates for, supports, and evaluates government-run school-based deworming programs (treating children for intestinal parasites). We believe that deworming is a program backed by relatively strong evidence. We have reservations about the evidence, but we think the potential benefits are great enough, and costs low enough, to outweigh these reservations. Deworm the World retains monitors whose reports indicate that the deworming programs it supports successfully deworm children. Important changes in the last 12 months In 2015, Deworm the World continued to support the scale-up and monitoring of deworming programs in India and Kenya. One of its notable activities this year was providing technical assistance to the Indian national government in support of India’s first national deworming day: a program in which the government provided assistance to Indian states to implement school-based deworming on a single day to encourage more states to implement the program. The first national deworming day took place in February 2015, and 12 states participated in the program (more details here). The quality of the monitoring that we saw from Deworm the World improved in 2015. Deworm the World continued to hire and train third-party monitors to directly observe deworming activities, and it slightly improved its estimates of how many children were treated. This information strongly suggests that the programs are generally operating as intended. More details in our review. Last year, Deworm the World stated to us that it could not use significant additional funding to scale up deworming programs. Deworm the World now believes that it has identified countries where it could use additional funds to support the scale-up of deworming programs, beginning with a potential program in Punjab province, Pakistan (more). (Deworm the World also plans to use funds it already holds or expects to receive to expand into Ethiopia and Nigeria.) Future donations to Deworm the World will likely be used outside of India, and in those cases governments may have less funding to support deworming. This may cause Deworm the World to pay a higher fraction of the overall cost of the program, making the potential for leverage of future donations more limited. Overall program costs may also be higher outside of India. More details in our review. A significant organizational update is that Alix Zwane stepped down as Executive Director of Evidence Action in August; she left to join the Global Innovation Fund as CEO. Evidence Action has since hired Jeff Brown (formerly Interim CEO of the Global Innovation Fund) as Executive Director. Grace Hollister remains Director of the Deworm the World Initiative. Overall, our impression is that Dr. Zwane has been a highly effective leader of Evidence Action and her departure risks disruptions that could lead to us changing our view of the organization, though we would guess that this will not be the case. In July, researchers published two new analyses of a key study regarding deworming (the most important piece of evidence we rely on), and the Cochrane Collaboration published an updated review of the evidence for mass deworming programs. The new papers did not change our overall assessment of the evidence on deworming. More in our blog post. Funding gap We believe that Deworm the World has significant opportunities to use additional funding to expand its program. We believe it may have opportunities to enter at least two more countries (in addition to Nigeria and Ethiopia, which it will be able to enter with funds it already has or expects to receive). We estimate its funding need using the two countries it is most likely to enter — Pakistan and Nepal — though note that in both cases, we see these as representative of the types of opportunities it may have, rather than the specific opportunities we expect it to take. Altogether, Deworm the World estimates that it would need$11.25 million to commit to fully funding three years of deworming programs in both countries. Because it holds (or expects to receive shortly) funding that will total $3.6 million, we estimate its funding gap for this work at$7.6 million.

Funding this gap is capacity-relevant, and is therefore a high priority, because we would like to see Deworm the World try to work in additional countries beyond India and Kenya, where it has worked historically. Next year, Deworm the World will also enter Nigeria and Ethiopia (with funding already available), so it will likely end the year having had some experience in five or more countries. This could substantially increase Deworm the World’s long-term room for more funding.

A complicating factor in thinking about Deworm the World’s funding gap is that Deworm the World is part of a larger organization, Evidence Action. Funding for Deworm the World may be fungible with funding for Evidence Action’s other activities, such as its Dispensers for Safe Water initiative (which we believe to be substantially less cost-effective than deworming). Because of this, it is difficult to determine Deworm the World’s true funding gap, and it is possible that some additional funds given to support Deworm the World could effectively lead to additional funds for a non-Deworm the World project. We understand that Evidence Action has received approximately $2.4 million in unrestricted funding over the past year. Fully funding Deworm the World could potentially cause Evidence Action to redirect some or all of these funds to its other programs. More details on all of the above are in our review. Key considerations: • Program impact and cost-effectiveness. We estimate that Deworm the World-associated deworming programs are ~10x as cost-effective as cash transfers. Our estimates are subject to substantial uncertainty. It’s important to note that we view deworming as high expected value, but this is due to a relatively low probability of very high impact. Most GiveWell staff members would agree that deworming programs are more likely than not to have very little or no impact, but there is some possibility that they have a very large impact. (Our cost-effectiveness model implies that most staff members believe there is at most a 1-2% chance that deworming programs conducted today have similar impacts to those directly implied by the randomized controlled trials on which we rely most heavily, which differed from modern-day deworming programs in a number of important ways.) Our 2015 cost-effectiveness file is available here (.xlsx). • Directness and robustness of the case for impact. Deworm the World doesn’t carry out deworming programs itself; it advocates for and provides technical assistance to governments implementing deworming programs, making direct assessments of its impact challenging. We have seen evidence that strongly suggests that Deworm the World-supported programs successfully deworm children. While we believe Deworm the World is impactful, our evidence is limited, and in addition, there is always a risk that future expansions will prove more difficult than past ones. • Transparency and communication. Deworm the World has been communicative and open with us. We believe that were something major to go wrong with Deworm the World’s work, we would be able to learn about it and report on it. • Risks: • Deworm the World is part of a larger organization, Evidence Action. It is possible that some additional funds given to support Deworm the World could effectively lead to additional funds for a non-Deworm the World project due to fungibility. Also, changes that affect Evidence Action (and its other programs) could indirectly impact Deworm the World. For example, if a major event occurs (either positive or negative) for Evidence Action, it is likely that it would reduce the time some staff could devote to Deworm the World. • Deworm the World is now largely raising funds to support programs that will be carried out under a different model in new countries, which makes it harder for us to predict future success based on historical results and may make it harder to understand and quantify Deworm the World’s impact even after the program is completed. Our full review of Deworm the World is here. GiveDirectly Our full review of GiveDirectly is here. Background GiveDirectly (www.givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 85% overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people. We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. It appears that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected (more). It continues to experiment heavily, to the point where every recipient is enrolled in a study or a campaign variation. Important changes in the last 12 months GiveDirectly continued to scale up significantly, utilizing most of the funding it received at the end of last year. It continued to share informative and detailed monitoring information with us. Overall, it grew its operations while maintaining the high quality of its program. In August, Good Ventures granted$25 million to GiveDirectly to support potentially high-upside opportunities, such as (a) building a fundraising team that will aim to raise substantial donations from non-GiveWell donors, and (b) developing partnerships with bilateral donors and local governments to deliver cash transfers or to run experiments comparing standard aid programs to cash transfers.

GiveDirectly’s increased efforts to network with potential government and donor partners have led to some results in 2015. For example, GiveDirectly will be implementing cash transfers in a randomized controlled trial in Rwanda that will be funded by a bilateral aid donor and Google. The study will test cash transfers against another still-to-be-chosen aid program. GiveDirectly is currently in several preliminary conversations with partners for similarly large projects in the future.

Funding gap

GiveDirectly believes it could move a total of ~$94 million to poor households in the year following March 1, 2016, for which it expects to have ~$12.6 million available by March 1. We have classified ~$34.5 million of this as the total “Execution Level 1,” capacity-relevant, and incentive funding gap (more on what this means above). We arrived at this figure by assuming that GiveDirectly could double its operations in Kenya (from ~$16.5 million/year to ~$33 million/year) and scale up to ~$12.1 million/year in Uganda. This would cost a total of ~$45.1 million, of which GiveDirectly already has ~$10.6 million on hand (ignoring $2 million that we exclude due to donor coordination issues), which results in a ~$34.5 million gap.

We’ve classified some of this as a “capacity-relevant” funding gap for our purposes (making it higher priority). First, we view the ~$12.1 million it would hope to spend in Uganda as capacity-relevant, in the sense that providing it could make a major difference to GiveDirectly’s long-term development. GiveDirectly told us that operating in Uganda is more challenging than in Kenya and that it expects to learn a significant amount as it grows. It is therefore planning to grow more slowly in Uganda than it did in Kenya. GiveDirectly made two arguments for Uganda being important for its long-term trajectory: 1. If GiveDirectly lost the ability to operate in Kenya, this would significantly diminish its ability to move funds out the door. Operating in Uganda is an important hedge against this risk. 2. Kenya is a particularly easy environment in which to operate because of the existence of M-PESA, a powerful and ubiquitous provider that enables GiveDirectly to transfer funds to recipients via mobile phones. The mobile payments network is significantly less developed outside of Kenya. As such, Uganda offers an important test case for operating in a more standard environment, which could be particularly valuable to GiveDirectly as it encourages aid agencies and country governments to expand direct cash assistance. It’s harder to estimate how much of the Kenya funding needs are properly classified as “capacity-relevant” (an important distinction for our purposes, as discussed above). We guess that were GiveDirectly to be operating at a level 50% its current size (such that it only spent ~$8.25 million/year in Kenya), it would be able to build capacity from that level to its current level (and beyond) as quickly as it did in its recent past. We therefore classify ~$8.25 million of the ~$16.5 million it hopes to spend in Kenya as “capacity-relevant” and ~$8.25 million as “execution.” We note that we are highly uncertain about these estimates and that were GiveDirectly to receive no additional funding, this would cause it to contract in Kenya and lay off some of its middle management, an action that would cause it to incur reasonably high costs; we think much more contraction than that would be significantly more challenging for GiveDirectly as an organization. Based on the above, and based on GiveDirectly’s existing available funds (with some adjustments for coordination issues, along the lines of this discussion from last year) we estimate that GiveDirectly has ~$9.8 million worth of unfunded opportunities that we ought to classify as capacity-relevant or incentive funding. (We arrive at this estimate based on: ~$20.35 million (total amount we classify as capacity-relevant from Kenya and Uganda) – ~$10.6 million (funds on hand, excluding donations we ignore due to coordination issues) = ~$9.75 million.) Longer-term, we expect to continue to view funding ~$8.25 million in Kenya as capacity-relevant support and would expect to consider future expansion in Uganda (up to the current level of Kenya, i.e., ~$16.5 million/year) capacity-relevant, as well. Once GiveDirectly reaches ~$16.5 million in Uganda and proves that it can operate at that level, we only expect to view ~$8.25 million as capacity-relevant and hope that it can raise funds from other sources to support its work. More details in our review. Key considerations: • Program impact and cost-effectiveness. Our best guess is that deworming or distributing bednets achieves ~10x times more humanitarian benefit per dollar donated than cash transfers. Our estimates are subject to substantial uncertainty. All of our cost-effectiveness analyses are available here. Our 2015 cost-effectiveness file is available here (.xlsx). • Directness and robustness of the case for impact. GiveDirectly collects and shares a significant amount of relevant information about its activities. The data it collects show that it successfully directs cash to very poor people, that recipients generally spend funds productively (sometimes on food, clothing, or school fees, other times on investments in a business or home infrastructure), and that it leads to very low levels of interpersonal conflict and tension. We are more confident in the impact of GiveDirectly’s work than in that of any of the other charities discussed in this post; we believe that cash transfers face a lower burden of proof than other interventions. • Transparency and communication. GiveDirectly has always communicated clearly and openly with us. It has tended to raise problems to us before we ask about them, and we generally believe that we have a very clear view of its operations. We feel more confident about our ability to keep track of future challenges than with any of the other charities discussed in this post. • Risks: • GiveDirectly has scaled (and hopes to continue to scale) quickly. Thus far, it has significantly increased the amount of money it can move with limited issues as a result. The case of staff fraud that GiveDirectly detected is one example of an issue possibly caused by its pace of scaling, but its response demonstrated the transparency and rigor we expect. Our full review of GiveDirectly is here. Schistosomiasis Control Initiative (SCI) Our full review of SCI is here. Background SCI (www3.imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs (treating children for schistosomiasis and other intestinal parasites). SCI’s role has primarily been to identify recipient countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs. Despite SCI sharing a number of spending reports with us, we do not feel we have a detailed and fully accurate picture of how SCI and the governments it supports have spent funds in the past. We don’t feel that SCI has ever purposefully been indirect with us, but we have often struggled to communicate effectively with SCI representatives. We still lack important and in some cases basic information about SCI’s finances, and we find this problematic. We believe that deworming is a program backed by relatively strong evidence. We have reservations about the evidence, but we think the potential benefits are great enough, and costs low enough, to outweigh these reservations. SCI has conducted studies in about half of the countries it works in (including the countries with the largest programs) to determine whether its programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but have some methodological limitations. Important changes in the last 12 months SCI reports that it has continued to scale up its deworming programs and that it has supported some programs in new countries, though we have limited monitoring information from these programs (e.g., we have not seen monitoring from its programs in Ethiopia, Sudan, Madagascar, and the DRC). This year, SCI has shared a few more coverage surveys that found reasonably high coverage of its programs. We have continued to have communication challenges with SCI. In particular: • We have a limited understanding of SCI’s work because we still lack important and basic information about how SCI spends money. SCI recognizes that its financial management system is disorganized, and some spending reports that SCI has sent us have contained errors. • We have struggled to gain a confident understanding of how SCI will use additional funds, and we cannot check how its funds were used after the fact because we lack information about its spending. In some cases, SCI has not spent additional funds as expected and it is unclear what caused the shift (more detail on one example in our August 2015 update). In July, researchers published two new analyses of a key study regarding deworming (the most important piece of evidence we rely on), and the Cochrane Collaboration published an updated review of the evidence for mass deworming programs. The new papers did not change our overall assessment of the evidence on deworming. More in our blog post. Funding gap SCI estimates that it would use the following amounts of unrestricted funding in each of the next three years (in millions of US dollars): • April 2016 – March 2017:$9.5
• April 2017 – March 2018: $13.6 • April 2018 – March 2019:$13.3

Our impression is that GiveWell-influenced donors contribute most of SCI’s unrestricted funds.

Our best guess is that, excluding the funds SCI may receive due to GiveWell’s recommendation, SCI will hold approximately $1.5 million in April 2016 that it could allocate to the above gaps. Also, after SCI set its fundraising targets, a funder committed$6 million over the next three years ($2 million per year) to deworming programs in Ethiopia, with which SCI is involved. Our best guess is that this funding reduces SCI’s “Execution Level 1” and incentive funding gap for the coming year from$9.5 million to $5.9 million. (We arrive at this estimate by subtracting ~$1.5 million and another $2 million from the total Level 1/incentive gap for the coming year). We do not classify any of this as “capacity-relevant” because we have little understanding of how it will be spent, and we do not expect to be able to understand how it was spent after the fact, either. More details on SCI’s funding gap are in our review. Key considerations: • Program impact and cost-effectiveness. Our best guess is that deworming programs implemented by SCI are ~5x as cost-effective as cash transfers. Our estimates are subject to substantial uncertainty. It’s important to note that we view deworming as high expected value, but this is due to a relatively low probability of very high impact. Most GiveWell staff members would agree that deworming programs are more likely than not to have very little or no impact, but there is some possibility that they have a very large impact. (Our cost-effectiveness model implies that most staff members believe there is at most a 1-2% chance that deworming programs conducted today have similar impacts to those directly implied by the randomized controlled trials on which we rely most heavily, which differed from modern-day deworming programs in a number of important ways.) Our 2015 cost-effectiveness file is available here (.xlsx). • Directness and robustness of the case for impact. SCI doesn’t carry out deworming programs itself; it advocates for and provides technical assistance to governments implementing deworming programs, making direct assessments of its impact challenging. We have seen some evidence demonstrating that SCI-supported programs successfully deworm children, though this evidence is relatively thin. Nevertheless, deworming is a relatively straightforward program, and we think it is likely (though far from certain) that SCI-supported deworming programs successfully deworm people. We have had difficulties communicating with SCI, which has reduced our ability to understand it. We have also spent significant time interviewing SCI staff and reviewing documents over the past 6 years and have found minor but not major concerns. • Transparency and communication. We don’t feel that SCI has ever purposefully been indirect with us, but we have often struggled to communicate effectively with SCI representatives. Specifically, (a) we had a major miscommunication with SCI about the meaning of its self-evaluations (more) and (b) although we have spent significant time with SCI, we remain unsure how SCI has spent funds and how much funding it has available (and we believe SCI itself does not have a clear understanding of this). Importantly, if there is a future unanticipated problem with SCI’s programs, we don’t feel confident that we will become aware of it. This contrasts with our other top charities, which we feel we have a strong ability to follow up on. • Risks: There are significantly more unknown risks with SCI than our other top charities due to our limited understanding of its activities. Our full review of SCI is here. Standouts As we did last year, we recommend four organizations as “standouts.” These charities score well on some of our criteria, but we are not confident enough in them to name them top charities. This year, we retain the same four standout organizations: Development Media International (DMI), the Global Alliance for Improved Nutrition’s Universal Salt Iodization program (GAIN-USI), the Iodine Global Network (IGN), and Living Goods. We followed all four of these charities in 2015, but have only published an updated review for DMI. We expect to publish updated reviews for GAIN-USI, IGN, and Living Goods in the near future. We provide brief updates on these charities below: • DMI. DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It is a standout because of its commitment to monitoring and the possibility that it is implementing a highly cost-effective program. DMI has recently completed a randomized controlled trial of its program. Last year, we had midline results from this trial, which generally looked promising.In November 2015, DMI privately shared preliminary endline results from the RCT. These results did not find any effect of DMI’s program on child mortality, and found substantially less effect on behavior change than was found in the midline results. We (understandably) cannot publicly discuss the details of the endline results we have seen, because they are not yet finalized and because the finalized results will be embargoed prior to publication. DMI believes that there were serious problems with endline data collection (note that we have not yet tried to independently assess this claim). With the support of the trial’s Independent Scientific Advisory Committee, DMI is planning to conduct another endline survey in late 2016, with results available in 2017.We are impressed by DMI’s openness with us about its results (and its willingness for us to share the high-level summary), and we hope to have discussions with DMI about how it might be able to work toward becoming a top charity in the future. Our full review of DMI is here. • GAIN-USI. GAIN’s Universal Salt Iodization (USI) program supports national salt iodization programs. There is strong evidence that salt iodization programs have a significant, positive effect on children’s cognitive development. GAIN-USI does not work directly to iodize salt; rather, it supports governments and private companies to do so, which could lead to leveraged impact of donations or to low impact, depending on its effectiveness. Last year, we wrote, “We tried but were unable to document a demonstrable track record of impact; we believe it may have had significant impacts, but we are unable to be confident in this with what we know now. More investigation next year could change this picture.” In 2015, we continued our assessment of GAIN, focusing on its work in India and Ethiopia, including a site visit to Ethiopia in July.Overall, we tried but were unable to establish clear evidence of GAIN successfully contributing to the impact of iodization programs. This is primarily due to (a) the difficulty in attributing impact to specific activities that GAIN carried out and (b) challenges we have had communicating with GAIN about its work. We have not yet completed our final report on GAIN but hope to publish it in the near future. We have published notes from some of the conversations that were part of this research and they are available here. Our 2014 review of GAIN is here. • IGN. Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization, and as with GAIN-USI, we tried but were unable to establish clear evidence of IGN successfully contributing to the impact of iodization programs. Unlike GAIN-USI, IGN is small, operating on a budget of approximately$0.5-$1 million per year, and relies heavily on volunteer time. We are planning to post an updated review in the near future. Our 2014 review of IGN is here. • Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a randomized controlled trial of its program and measured a 27% reduction in child mortality. We estimate that Living Goods saves a life for roughly each$10,000 it spends, approximately 3 times as much as our estimate for the cost per life saved of AMF’s program. We spoke with Living Goods and reviewed documents about their progress in 2015. We do not have major updates to report but are planning to post an updated review in the near future. Our 2014 review of Living Goods is here.

Our research process in 2015

This section describes the new work we did in 2015 to supplement our previous work on defining and identifying top charities. See the process page on our website for our overall process.

This year, we did not put a substantial amount of senior staff time into new top charities research work because (a) we were largely focused on building capacity, and (b) we reallocated a significant amount of capacity to the Open Philanthropy Project (see our post on our plans for 2015 for more details).

We focused the bulk of our research capacity for top charities work on staying up-to-date on our recommended charities. We also did an intensive evaluation of GAIN-USI, including a site visit (more details forthcoming).

We completed investigations of vitamin A supplementation and maternal and neonatal tetanus immunization campaigns. Both programs seem potentially competitive with our other priority programs, but we were not able to identify charities that worked on these programs that were willing to apply for a recommendation. We also made substantial progress on investigating several other programs, such as measles immunization, meningitis A vaccination, folic acid fortification, voluntary medical male circumcision for the prevention of HIV, and “Targeting the Ultra-Poor” (or “Ultra-Poor Graduation”) programs.

We stayed up to date on the research for bednets, cash transfers, and deworming.

We did not conduct an extensive search for new charities this year. We feel that we have a relatively good understanding of the existing charities that could potentially meet our criteria, based on past searches (see the process page on our website for more information). Instead, we solicited applications from organizations that we viewed as contenders for recommendations. A March post laid out which organizations we were hoping to investigate and why.

We did some initial research on several charities that we had not investigated before, but we did not complete the reviews in time for our 2015 recommendations. The organizations that we began investigating were:

We plan to complete these reviews in 2016.

Giving to GiveWell vs. top charities

We have grown significantly over the past few years and continue to raise funds to support our operations. This includes work on GiveWell’s top charities and the Open Philanthropy Project.

We plan to post an update on our funding situation before the end of the year.

The most up-to-date information available on this topic is linked from our June 2015 board meeting. The short story is that we are still seeking additional donations and encourage donors who feel they are sufficiently confident in our impact to give to us.

Footnotes:

* For example, if $30 million were available to fund gaps of$10 million, $5 million, and$100 million, we would recommend allocating the funds so that the $10 million and$5 million gaps were fully filled and the $100 million gap received$15 million.

This rule is material to the three gaps tied at priority level 2. It causes us to recommend that Good Ventures’ last $28.3 million to recommended charities is used to fully fill GiveDirectly’s$8.8 million capacity-relevant gap and Deworm the World’s $3.2 million Execution Level 2 (possible capacity-relevant) gap, but only fill$16.3 million of AMF’s Execution Level 1 gap.

** This gap can’t be cleanly classified because we think the funding is relatively unlikely to be needed, but if it is needed, it is likely to have capacity-relevant effects. Thus, it is technically classified as Execution Level 2, but we think it has similar value to Execution Level 1.

The post Our updated top charities for giving season 2015 appeared first on The GiveWell Blog.

### Good Ventures’ $25 million grant to GiveDirectly Mon, 08/03/2015 - 11:03 Good Ventures has granted$25 million to GiveDirectly for the support of GiveDirectly’s general operations. While the grant is unrestricted, we expect that GiveDirectly will most likely use this grant as follows:

• ~$6-9 million: to hire a marketing team to raise significantly more funding than GiveDirectly has raised in the past. • ~$16-19 million: to provide cash transfers to extremely low-income households, either using its standard operating model or as part of collaborations with large aid institutions or governments. These collaborations are intended to address questions that institutional funders have about cash transfers and could include running experiments comparing other aid programs against cash transfers.

We see this grant as an outstanding giving opportunity because:

• We believe that GiveDirectly is an exceptionally strong organization, and we see supporting its growth as an excellent opportunity.
• Providing funds to help GiveDirectly build its marketing team and arrange partnerships with large institutions could lead to future funding for cash transfers that dwarfs this initial investment.
• Funding GiveDirectly’s current model is one of the best giving opportunities we know of. A significant portion of this grant will directly fund GiveDirectly’s core model.

However, we see GiveDirectly’s plans as ambitious, so we would not be surprised if GiveDirectly fails to meet the full scope of its goals. More details on what we see as some of the risks to the success of this grant are below.

GiveDirectly continues to have significant room for more funding, so this grant is unlikely to have a direct impact on our end-of-year recommendations to donors.

Below, we go into more detail on:

For more background on GiveDirectly and its core operations, see our GiveDirectly review.

Process
In late 2014, Good Ventures expressed an interest in contributing significantly more to our top charities than it had in the past. It also asked whether additional grants could accelerate the growth of our top charities as organizations. We considered grants for all four of our top charities and ultimately prioritized discussions with GiveDirectly (more about our other top charities below).

Over the course of three months, we had four conversations with GiveDirectly about how it would use a large grant from Good Ventures. GiveDirectly put together a proposal that explained how it would use grants ranging in size from $23.5 million to$89 million. GiveWell and Good Ventures representatives discussed this proposal. GiveWell recommended that Good Ventures grant $25 million to GiveDirectly to support its general operations, and Good Ventures agreed. What will the grant be used for? The$25 million grant will primarily be used to support two activities:

• Fundraising: ~$6-9 million • Cash transfers (including partnerships and research): ~$16-19 million

More details on these activities are below.

Fundraising

GiveDirectly plans to spend $6-9 million of this grant over the next 3-4 years (spending ~$3.5 million over the next two years with the budget growing in later years) to hire a marketing team that will substantially increase GiveDirectly’s fundraising efforts. GiveDirectly expects these funds to cover a) hiring a marketing team that will develop a new fundraising strategy, and b) implementing the team’s strategy over the next 3-4 years. GiveDirectly’s long-term goal for its marketing team is to help GiveDirectly become an organization that raises and spends over $100 million per year with a reasonable fundraising cost-to-revenue ratio (more below). GiveDirectly’s first step under this grant will be to hire a new marketing director who will build and lead the new marketing team (including hiring 3-5 other staff members for the marketing team). GiveDirectly believes that it will be able to attract a top-talent marketing director because a) based on its research, the size of its marketing budget is competitive with other companies or non-profits where a top marketing director might be able to work, and b) it believes that having already secured the next 3-4 years of its marketing budget will make it easier to attract promising candidates. GiveDirectly intends for the marketing team to develop a fundraising strategy that will significantly expand GiveDirectly’s donor base. In particular, GiveDirectly aims to fundraise from donors who would not otherwise have given to GiveWell’s recommended charities. Tentatively, GiveDirectly believes the new fundraising strategy will focus on developing an improved web and mobile experience for donors. For example, the new web and mobile experience may enable donors to track and learn about the specific recipient to whom their donations were routed. Cash transfers (including partnerships and research) Approximately$16-19 million of Good Ventures’ grant will be used for cash transfers, either using GiveDirectly’s standard model or through partnerships with other institutional funders. In either case, GiveDirectly expects the delivery of the transfers to be accompanied by additional research. GiveDirectly told us that all of its standard transfers are now part of experimental evaluations that test the impact and design of its cash transfer programs. In collaborations with large aid institutions or governments, transfers could be used to a) run experiments with partners that compare the effectiveness of standard aid programs against cash transfers or b) help partners to experiment with delivering cash transfers.

In the past, GiveDirectly has found that it can more successfully arrange collaborative projects if it can co-fund the cash transfers in the project, so it requested funds from Good Ventures that could be used for this purpose.

Long-term, GiveDirectly hopes that such collaborative projects will encourage aid agencies and governments to deliver more cash transfers with their own funds. It also hopes that such projects may help to establish cash transfers as a standard “benchmark” program against which other global aid programs should be compared.

GiveDirectly plans to discuss partnerships with the following types of institutions:

• Donor aid agencies.
• Developing country governments (national and local). (For example, several governors in Kenya have already approached GiveDirectly about running cash transfer programs in their counties.)

How quickly will the funds be spent and when do we expect to see results?

In brief, we expect to be able to make a preliminary assessment of the success of this grant after the 2016 giving season (December 2016 – January 2017), at which point GiveDirectly expects that its fundraising strategy will be operational, that the cash transfers funded by this grant will be disbursed, and that it will have made some progress on arranging partnerships.

More details are below. We plan to follow up on each of the below expectations according to our standard timeline for charity updates.

Fundraising

As mentioned above, GiveDirectly expects the marketing team to spend its $6-9 million budget over the next 3-4 years. GiveDirectly has already begun to interview candidates to lead its marketing activities and expects to hire someone for the position by the end of 2015. It expects the rest of the marketing team (3-5 people) to be hired soon after the lead is on board. Many retail donors give significantly more during December, the time of year we refer to as “giving season.” We believe that giving season is the right period during which to assess GiveDirectly’s marketing team’s results. GiveDirectly expects its new marketing strategy to be fully operational by the 2016 giving season. After that giving season ends, GiveDirectly expects to have at least broken even on its fundraising expenses; i.e., it expects that the funds that it will have raised due to its new strategy will at least equal the money that it has spent on the strategy up to that point, and it hopes to do significantly better. Eventually, GiveDirectly hopes to be spending roughly$0.05 on fundraising for every $1 raised, which it believes would be a greater level of efficiency than the industry average. GiveDirectly expects that during the first several years of implementing its new fundraising strategy, it will spend closer to$0.10 – $0.20 per$1 raised.

We do not have a strong sense of what we should expect from a fundraising operation like GiveDirectly’s planned operation, but our rough expectation about its likely near-term efficiency is less optimistic than GiveDirectly’s. We would not be surprised if GiveDirectly managed only to cover its fundraising costs in these early years.

Cash transfers (and research)

GiveDirectly believes it will transfer most of the money allocated for standard cash transfers within the next year and a half (2015-2016). This seems like a reasonable expectation from our perspective. GiveDirectly has scaled up rapidly and has a track record of moving donations to recipients in a timely manner.

Partnerships

Of the three areas to which GiveDirectly is planning to allocate Good Ventures’ grant funds, this is the one for which we have the least well-defined expectations. GiveDirectly’s work on partnership projects has been preliminary, and we are uncertain how these projects will progress.

GiveDirectly has told us that it has promising prospects for partnerships, but we would not be surprised if it does not ultimately develop major partnerships with large institutions. However, if GiveDirectly is able to establish at least one major partnership, we would likely consider its partnerships work to be a success.

The large institutions that GiveDirectly would like to partner with tend to move relatively slowly, so we would not be surprised if these funds took a number of years to be disbursed. If GiveDirectly fails to find a partnership arrangement within a reasonable amount of time, we would expect it to allocate the funds earmarked for partnerships to other near-term uses.
What is the case for making the grant?

Our primary reasons for recommending this grant are:

• We believe that GiveDirectly is an exceptionally strong organization, and we see supporting its growth as an excellent giving opportunity. We felt comfortable recommending a grant of this size to GiveDirectly partly because we trust its staff’s views about the best use of additional funds and we expect that it will be able to competently execute on its plans. Some of the reasons that we have a positive view of GiveDirectly include:
• It has an impressive track record of scaling up as an organization: in its 2012 fiscal year, GiveDirectly transferred less than $0.5 million. In its 2014 fiscal year, it transferred$5 million, and incurred liabilities (future transfers for currently enrolled recipients) totaling $8.6 million (more in our full review of GiveDirectly and its financials). • It has always communicated clearly with us and has transparently discussed its mistakes (see a recent example here). We are confident that we will be able to follow up on and learn from the results of this grant, whether they are positive or negative. • We believe that the activities that this grant is funding have the potential to make a large, positive impact by increasing individual and institutional donors’ support for cash transfers. In particular, if GiveDirectly’s new fundraising strategy is successful, it could greatly improve the impact of a huge amount of donations. GiveDirectly may also be in a position to persuade institutional donors to fund cash transfers instead of less cost-effective activities. • GiveDirectly’s core model is one of the best opportunities we know of to use money to accomplish a great deal of good. It has significant room for more funding, and we believe that the money it transfers to very poor people and the research it conducts will have substantial impact. How did we settle on this grant size? GiveDirectly sent us a high-level summary of how it could use grants ranging in size from$23.5 million to $89 million. We decided to recommend a grant of$25 million primarily because a grant of this size would enable GiveDirectly to fully pursue what we saw as its two highest-leverage opportunities – implementing a new fundraising strategy and seeking partnerships with large aid institutions – while also providing GiveDirectly a large amount of funding to carry out its core activities of delivering cash transfers and running experiments on those transfers. GiveDirectly told us that it would plan to distribute any additional money it received beyond $23.5 million in roughly a proportion of 3:1:1 for cash:fundraising:partnerships, so most additional money would go to cash transfers. Also, we told GiveDirectly that Good Ventures would consider a grant in the range of approximately$20 million, and that initial figure likely played some role in anchoring GiveDirectly’s proposals.
What are the risks to the success of this grant?

The major risks to the success of this grant that we are aware of include:

• Possibility of fundraising failure: We do not have a strong sense of how likely GiveDirectly’s fundraising campaign is to succeed, but we would not be surprised if it failed to attract new donors at a desirable rate of efficiency. We see supporting GiveDirectly’s fundraising plans as a risky bet.
• Diversion of funds from other top charities: It is possible that GiveDirectly’s new fundraising campaigns could end up attracting donors who would have given to GiveWell’s other top charities if not for GiveDirectly’s campaigns. This may have the effect of diverting donors away from potentially more cost-effective giving opportunities. However, we see this as a relatively minor risk because GiveDirectly explicitly plans to target donors who would not otherwise become part of GiveWell’s audience.
• Possible slow disbursement of partnership funds: We think that the main risk associated with GiveDirectly’s partnerships plan is simply that the funds for partnership projects may not be used for a long period of time. In this situation, it may be that the funds would have been better spent on near-term opportunities to do good. However, this risk seems limited because we believe that GiveDirectly shares our desire for funds to be used in a timely fashion and will work to move these partnerships forward quickly.
• Too much focus on marketing: A small risk of the fundraising campaign is that GiveDirectly could eventually become too focused on marketing, at the cost of transparency and the quality of its operations. Based on our impression of the GiveDirectly team, we see this as relatively unlikely.

How does this grant affect GiveDirectly’s room for more funding?

We believe that GiveDirectly still has substantial room for more funding, and we continue to recommend donations to GiveDirectly.

Including this grant, GiveDirectly currently holds approximately $40 million that it plans to allocate to its core model. GiveDirectly believes that it has the capacity to transfer about$75 million over the next two years (2015 and 2016). If it received significantly more funding than $75 million, GiveDirectly believes it could hire an additional Field Director and move substantially more money. We view GiveDirectly’s estimates of its room for more funding as reliable due to its demonstrated ability to scale, and this estimate is consistent with our December 2014 analysis of GiveDirectly’s room for more funding. We plan to provide more details on our view of GiveDirectly’s room for more funding in our year-end refresh of our GiveDirectly review. Reflections on the potential benefits of sometimes being an “active” funder Previously, we’ve written about the distinction between passive and active funding and the strengths and weaknesses of each approach. Our experiences with GiveDirectly over the last few months demonstrate some potential benefits of sometimes being an active funder. In this case, it was only after Good Ventures encouraged us to ask our top charities about how they could use additional large amounts of funding that we had extensive discussions with GiveDirectly about possible growth plans. Our impression is that GiveDirectly’s vision for how it could use additional funds to grow as an organization sharpened over the course of these discussions as it became clear that Good Ventures was interested in making a substantial grant. Ultimately, these discussions revealed potentially high-impact giving opportunities that we would not have been aware of if we had not pursued the question of how GiveDirectly could use additional funding to accelerate its growth relatively actively. We had previous discussions with GiveDirectly’s leadership where we explicitly asked them a) whether there was anything else we could do to support their growth and b) whether it would potentially be valuable to provide additional funding for their fundraising team. In both cases, GiveDirectly answered that it did not see promising opportunities for additional funding. Our impression is that our persistent encouragement to think more deeply about how it would use significantly more funding, coupled with a strong interest from Good Ventures in providing such funding, motivated GiveDirectly to reflect on its needs and make the funding request detailed in this post. (Likewise, we would not have pursued these conversations with GiveDirectly as actively in the absence of Good Ventures’ persistent encouragement to do so.) What are Good Ventures’ future giving plans, and does this affect the expected room for more funding of GiveWell’s top charities? Below is our summary of Good Ventures’s stance, which Good Ventures has reviewed: Good Ventures is open to making other large grants (i.e., grants on a similar scale to this grant to GiveDirectly) to GiveWell’s top charities in the future, though it does not have any firm plans to do so at this time. We have explained to GiveDirectly that it should consider this to be a one-time grant and that it should not necessarily expect another large grant in the future. As discussed below, we plan to be continually thinking through the question of whether our other top charities might present comparably good cases for large grants. Good Ventures has no current plans to change its approach to making end-of-the-year grants to GiveWell’s top charities. Some of our followers may be wondering whether this grant implies that Good Ventures could fill all of our top charities’ funding gaps at some point soon. Good Ventures does not expect to close the funding gaps of GiveWell’s top charities in the foreseeable future. Good Ventures is still early in its exploration of higher-risk, potentially higher-expected value giving opportunities as part of the Open Philanthropy Project, and there are many possibilities for where its giving might eventually go. Good Ventures is interested in making large enough grants to top charities to take advantage of particularly exciting opportunities (such as those described in this post) and send a significant signal about the benefits to being a top charity, but fully closing each top charity’s funding gap would mean spending significantly more without significantly increasing these benefits. In addition, Good Ventures (as we do) sees it as a good thing for top charities to attract a large number of donations (and not just a large number of dollars); we believe this dynamic is better in terms of the signals it sends about the benefits of being a top charity, and in terms of the robustness of top charities’ financial situations. If Good Ventures were considering providing enough funding to fill our top charities’ funding gaps, it would let us know of its plans well in advance so that we would have time to alert our donors. If our top charities’ funding gaps were filled, we would find the next-highest priority funding gaps to recommend. Does Good Ventures plan to make large grants to GiveWell’s other top charities? When Good Ventures asked us about high-impact giving opportunities related to growing our top charities as organizations, we considered asking each of the top charities how they would use significantly more funding. We ultimately chose not to recommend large grants to other top charities for the time being. We explain the reasoning behind our decision for each of our other top charities below: • The Against Malaria Foundation (AMF) – At the same time that we first spoke with GiveDirectly about this grant, we also began discussions with AMF about a possible large grant. AMF believes that it has a large amount of room for more funding (roughly$50 million or more), but as we noted in our May 2015 update, we are hesitant to recommend that it receive a large amount of additional funding (i.e., on the order of tens of millions of dollars) until we can evaluate results from Kasaï Occidental, its first large-scale distribution outside of Malawi. It has recently posted reports from the Kasaï Occidental distribution (see “Distribution” section on this page), and we are currently reviewing them.
• Deworm the World Initiative, led by Evidence Action – We decided not to have a conversation with the Deworm the World Initiative about how it could grow with substantially more funding because, based on our recent analyses, we believed that the organization had limited room for more funding.
• Schistosomiasis Control Initiative (SCI) – We decided not to have a conversation with SCI about how it might be able to use a large grant because a) we felt that we would need to be able to have a better understanding of SCI’s past spending and monitoring before considering recommending a large grant, and b) we have often struggled to communicate effectively with SCI representatives in the past.

The post Good Ventures’ $25 million grant to GiveDirectly appeared first on The GiveWell Blog. ### Top charities’ room for more funding Fri, 04/03/2015 - 11:55 In December, we published targets for how much money we hoped to move to each of our top four charities, with the expectation of revisiting these targets mid-year: In past years, we’ve worked on an annual cycle, refreshing our recommendations each December. This year, because we anticipate closing (or nearly closing) the funding gaps of some of our top charities during giving season and moving a significant amount of money (~$5 million) after giving season before our next scheduled refresh, we plan to update our recommendations based solely on room for more funding in the middle of next year. We’re tentatively planning to do this on April 1st, the earliest we will realistically be able to post an update on charities’ ongoing funding needs that accounts for the funds they will receive over the next few months.

These targets were based on a guess that GiveWell-influenced donors would give $7.5 million to our top four charities in December 2014 to March 2015 (excluding Good Ventures and a$1 million gift to SCI from an individual that we knew about prior to setting the targets). Our actual money moved for this period was about $8.7 million to the top four charities, plus$0.4 million that we can allocate at our discretion and have not yet allocated.

Over the past couple of months, we have spoken with each of our top charities to get updates on how much funding they have received from GiveWell-influenced and other donors and their current room for more funding. In sum, the amounts that our top charities raised as a result of our recommendations were broadly consistent with what we expected and there have not been any significant updates to the charities’ room for more funding. Therefore, we are not revising our recommended allocation (for every $7.5 given,$5 to AMF, $1 to GiveDirectly,$1 to SCI, and $0.5 to Deworm the World) at this time. Summary for December 2014 to March 2015 (all figures in USD millions): Charity Target from individuals (Dec 2014) Max from individuals (Dec 2014) Actual from individuals Summary Against Malaria Foundation 5 5 4.5 Close to target Schistosomiasis Control Initiative 1 1 1.1 On target Deworm the World Initiative 0.5 1 0.7 Reached target but did not exceed max GiveDirectly 1 25 2.4 Reached target but did not exceed max Against Malaria Foundation (AMF) Donations to AMF from GiveWell-influenced donors were short of our target by about$0.5 million. AMF is currently in discussions about funding several large-scale bednet distributions. It is our understanding that the amount of funding AMF has available is a limiting factor on both how many nets it can provide to each distribution it is considering and on how many discussions it can pursue at one time.

We have written before about AMF’s lack of track record at signing agreements for and successfully completing large-scale distributions with partners other than Concern Universal in Malawi. In 2014, AMF signed its first agreement to fund a large-scale distribution with another partner in a different country: IMA World Health in the province of Kasaï Occidental in the Democratic Republic of the Congo (more). The Kasaï Occidental distribution was scheduled to be completed in late 2014. We have not yet seen results from this distribution, and AMF’s track record of completing and reporting on successful large-scale distributions remains limited. AMF expects to be able to share information from this distribution in the next few weeks.

We plan to continue recommending funds to AMF for now and to reassess AMF’s progress later in the year.

GiveDirectly

In December, we noted that GiveDirectly could likely absorb up to $25 million in funding from GiveWell-influenced individuals. We tracked$2.4 million to GiveDirectly from these individuals and it is possible that GiveWell influenced several million dollars more – between February 2014 and January 2015, GiveDirectly received several million dollars from individuals who did not provide information on how they learned about the organization. We continue to believe that GiveDirectly has substantial room for more funding.

Schistosomiasis Control Initiative (SCI)

In December we set a target of SCI receiving $1 million from GiveWell-influenced individual donors and set the max we aimed for SCI to receive from this group at the same amount. We estimate that SCI received about$1.1 million based on GiveWell’s recommendation.

We have fairly limited information on SCI’s room for more funding because (a) SCI recently began working with a new financial director and is in the process of reorganizing its financial system, and so has not yet been able to provide us with a comprehensive financial update; and (b) SCI held a meeting on March 24 to allocate unrestricted funds and sent us a report from that meeting recently, which we have not yet had time to review. We will be following up with SCI to learn more about its plans and funding needs.

We plan to continue recommending funds to SCI because (a) our room for more funding estimates for SCI are rough and we believe there is a reasonable chance that SCI has room for more funding; (b) we expect to learn more about SCI’s room for more funding in the next few months; and (c) we do not expect SCI to receive a large amount of funding due to our recommendation over the next few months (since most donors give in December).

Deworm the World Initiative, which is led by Evidence Action

In December we set a target of $0.5 million from GiveWell-influenced individual donors to Deworm the World and set the max we aimed for Deworm the World to receive from this group at$1 million. We estimate that Deworm the World received about $0.66 million based on GiveWell’s recommendation. It’s our understanding that Deworm the World may have opportunities over the next few years to support up to three deworming programs which could each cost several million dollars. We are in the process of following up with Deworm the World to learn more about how likely these programs are to require unrestricted funding from Deworm the World and when funding might become a bottleneck to moving forward with these programs. We plan to continue recommending funds to Deworm the World. The post Top charities’ room for more funding appeared first on The GiveWell Blog. ### Our updated top charities Mon, 12/01/2014 - 12:35 Our top charities are (in alphabetical order): We have recommended all four of these charities in the past. We have also included four additional organizations on our top charities page as standout charities. They are (in alphabetical order): In the case of ICCIDD, GAIN-USI, and DMI, we expect to learn substantially more in the coming years (both through further investigation and through further progress by the organizations); we see a strong possibility that these will become top-tier recommended charities in the future, and we can see reasons that impact-minded donors could choose to support them today. Ranking our top charities against each other is difficult and laden with judgment calls, particularly since: • Our cost-effectiveness analyses are non-robust, and reasonable people could reach a very wide variety of conclusions regarding which charity accomplishes the most good per dollar. • The charity we estimate as having the weakest cost-effectiveness (GiveDirectly) is also the one that we feel has the strongest organizational performance and the most direct, robust connection between donations and impact. • We do not currently feel highly confident in our cost-effectiveness estimates. We changed a number of inputs to our estimates recently. We did not have time to fully consider and vet them, and we plan to put more work into these estimates over the next few months. We do not expect our estimates to change significantly but given the fact that we have been updating them very recently, we would not be surprised if they do. We plan to publish a post soon detailing the major changes and most debatable assumptions in our current estimates. We consider the lateness of major revisions to this year’s estimates a shortcoming (and will be adding it to our mistakes page when we do our annual review). • This year we expect to influence a significant amount of donations. In some past years, we’ve been able to assume that each dollar of donations to an organization is about equally effective. This year, we could easily see one or more of our top charities reach the point of diminishing returns to additional donations and/or close its funding gap entirely. • We’ve been trying to predict and coordinate donations from Good Ventures, from individual donors, and from major donors who have given us private information about their plans. In so doing, we’ve run into game-theoretic challenges. If two donors are interested in funding the same organization, each has an incentive to downplay his/her interest in the hopes that the other will provide more of the funding. We’ve been trying to avoid reinforcing such incentives. We discuss how these considerations affected our targets below, and we plan to elaborate on this issue in a future post. • In past years, we’ve worked on an annual cycle, refreshing our recommendations each December. This year, because we anticipate closing (or nearly closing) the funding gaps of some of our top charities during giving season and moving a significant amount of money (~$5 million) after giving season before our next scheduled refresh, we plan to update our recommendations based solely on room for more funding in the middle of next year. We’re tentatively planning to do this on April 1st, the earliest we will realistically be able to post an update on charities’ ongoing funding needs that accounts for the funds they will receive over the next few months. This plan also raises questions about donor agency and coordination; we plan to discuss this in a future post.

We’ve tried to balance these considerations against each other and come up with an “ideal allocation” of the ~$7.5 million in estimated “money moved” we expect to influence (not counting grants from Good Ventures) over the next 4 months. Details are below. Based on this allocation, for any donors looking to give as we would, we recommend an allocation of$5 to AMF (67%), $1 to SCI (13%),$1 to GiveDirectly (13%) and $.50 to DtWI (7%) for every$7.50 given.

Good Ventures is planning to make grants of $5 million to each of AMF and GiveDirectly,$3 million to SCI, and $250,000 to DtWI. Good Ventures also plans to make grants of$250,000 to each of the standout organizations. We advised on these grants a few weeks ago, and did so while weighing our funding targets for each charity and forecasts of what other donors are likely to do; parts of our picture have since changed, and these grants do not represent the allocation we would advise donors to use nor do they reflect our views about the relative ranking of these organizations. We made sure to settle on and announce these grants before giving season so that no donor would have to grapple with questions about Good Ventures’s likely actions (more in our upcoming post on donor coordination), and Good Ventures will not be making additional grants to these charities in the near to medium future (6-12 months) unless there are substantive updates on things like evidence bases and capacity for absorbing money (i.e. Good Ventures will not be giving further simply in response to new information about donor behavior over the next 4 months).

Below we provide:

• Additional detail on each of these eight organizations, including (for past recommendations) major changes over the past year, strengths and weaknesses for each, and our understanding of each organization’s room for more funding (which forms the basis for our funding targets and recommended allocation). More
• The thinking behind our funding targets and recommended allocation. More
• The process we followed that led to these top charities. More
• Brief notes on giving now vs. giving later and giving to GiveWell vs. our top charities. More

Conference call to discuss our recommendationsWe are planning to hold a conference call at 5:30pm EST on Wednesday, December 3rd to discuss our recommendations and answer questions. If you’d like to join the call, please register using this online form. If you can’t make this date but would be interested in joining another call at a later date, please indicate this on the registration form.

Top charitiesWe present information on our top charities in alphabetical order.

Against Malaria Foundation (AMF)Our full review of AMF is here.

Important changes in the last 12 months

We named AMF our #1-ranked charity at the end of 2011. Over the next 2 years, AMF received more than $10 million on the basis of our recommendation but struggled to identify opportunities to use the funds it had received. At the end of 2013, we announced that we planned not to recommend additional donations to AMF until it committed the bulk of its current funds. This did not reflect a negative view of AMF; instead it reflected room for more funding related issues. More detail in this blog post. In 2014, AMF finalized several distributions in Malawi and the Democratic Republic of the Congo (DRC) with three different implementing partners (two of which account for the bulk of the nets to be distributed). In 2014, it committed approximately$8.4 million to distributions which will take place before January 1, 2016 (some of which have already begun) and now has $6.8 million available for future distributions.$1.7 million of this is committed to a distribution scheduled for 2017 (and could potentially be allocated to distributions taking place sooner). Excluding the 2017 distribution, AMF has committed approximately $11.2 million to distributions in its history. AMF continued to collect and share follow up information on its programs. We covered these reports in our August 2014 AMF update. Funding gap AMF requires access to funding in order to negotiate deals because it cannot initiate discussions with potential partners unless it is confident that it will have sufficient funding to support its future agreements. The funding it currently holds would enable it to fund approximately 3 distributions at a scale similar to what it has funded recently. AMF has told us that it has a pipeline of possible future net distributions that add up to$36 million (details in our review).

We see some reason for caution in thinking about AMF’s room for more funding. It has made strong progress on being able to negotiate distributions and commit funds. However, as of today there have only been two large-scale distributions that have moved forward far enough for data to be available. Both of these are significantly smaller than distributions AMF has recently or will soon fund, and both are in the same area with the same partner as each other. Some of the recently negotiated distributions could prove more challenging (since they are in DRC).

If AMF received an additional $10 million in total over the next 4 months, it would have about twice as much funding available as the total it committed to large-scale distributions in 2014. (As stated above, it committed$8.4 million to distributions taking place before 2017 and has $6.8 million available for further commitments.) If it received$25 million, it would have about 4 times that total. 2-4 times past distributions seems like a range that would allow AMF to do significantly more than it has in the past, without going so far beyond its past capacity as to raise serious scaling concerns.

We believe that $10 million total (the low end of that range), which means$5 million after the Good Ventures grant, is an appropriate target after which further donations are likely better off going to other charities.

Key considerations:

• Program impact and cost-effectiveness. Our best guess is that distributing bednets is in the same cost-effectiveness range as deworming programs and more cost-effective than cash transfers by a factor of 5-10. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for bednets is here (.xls), and the comparison to deworming, cash transfers and iodine is here (.xls).)
• Directness and robustness of the case for impact. We believe that the connection between AMF receiving funds and those funds helping very poor individuals is less direct than GiveDirectly’s and more direct than SCI’s or DtWI’s. The uncertainty of our estimates is driven by a combination of AMF’s challenges historically disbursing the funds it receives and a general recognition that aid programs, even those as straightforward as bednets, carry significant risks of failure via ineffective use of nets, insecticide resistance or other risks we don’t yet recognize relative to GiveDirectly’s program. AMF conducts extensive monitoring of its program; these results have generally indicated that people use the nets they receive.
• Transparency and communication. AMF has been extremely communicative and open with us. We feel we have a better understanding of AMF than SCI and worse than GiveDirectly. In particular, were something to go wrong in one of AMF’s distributions, we believe we would eventually find out (something we are not sure of in the case of SCI), but we believe our understanding would be less quick and complete than it would be for problems associated with GiveDirectly’s program (which has more of a track record of consistent intensive followup).
• Risks:
• Two of AMF’s recent distributions (and much of its future pipeline) will take place in the DRC. Our impression is that the DRC is a particularly difficult place to work, and it is possible that AMF’s distributions there will struggle or fail. We view this as a moderate risk.
• We are not highly confident that AMF will be able to finalize additional distributions and do so quickly. AMF could struggle again to agree to distribution deals, leading to long delays before it spends funds. We view this as a relatively minor risk because the likely worst case scenario is that AMF spends the funds slowly (or returns funds to donors).
• We remain concerned about the possibility of resistance to the insecticides used in bednets. There don’t appear to be major updates on this front since our 2012 investigation into the matter; we take the lack of major news as a minor positive update.

A note on how quickly we expect AMF to spend the funds it receives. AMF works by sourcing, evaluating and negotiating deals for net distributions. This process takes time and requires AMF to have significant access to funding – it cannot approach a country to begin negotiations unless it is confident that it will have sufficient funding to pay for the nets it offers. We would not be surprised if AMF fails to reach additional deals in the next 12 months. We do expect it to commit the majority of its available funds (that it will have as of this coming January) within the next 24 months. If AMF does not make much progress in committing funds in the next 12 months, we will adjust our recommendation for 2015 accordingly, possibly recommending a lower target level of funds or suspending the recommendation entirely (depending on the specifics of the situation).

Our full review of AMF is here.

Deworm the World Initiative, (DtWI), led by Evidence ActionOur full review of DtWI is here.

Important changes in the last 12 months

Dr. Kevin Croke released a new study of a randomized controlled trial of a deworming program showing large, long-term impacts from deworming programs (for more, see this blog post). This study is a significant positive update on the impacts of deworming and increased our confidence that deworming programs have significant long-term impacts.

DtWI spent the funds it received due to GiveWell’s recommendation largely as we anticipated; it now has some (though limited) room for more funding.

In 2014, two events affected DtWI’s projection of the additional funding it would require to scale up in India:

• The Children’s Investment Fund Foundation (CIFF), a major foundation that had supported DtWI’s programs in Kenya, agreed to a 6-year, $17.7 million grant to support DtWI’s expansion to additional states in India and technical assistance to the Government of India for a national deworming program. With these funds, DtWI does not require significant additional funding to support its India expansion. • The new Indian government expressed interest in conducting a single deworming day nationally with increased national attention and resources. Advocating for such a policy and assisting the national government in creating a plan became the major focus of DtWI’s India work in 2014, which both reduced the amount of time it was able to spend generating interest in heavy DtWI involvement in new states and also required little funding since there were few costs of that project aside from staff time. We see this as positive news regarding DtWI’s potential impact; it may simply reduce DtWI’s further need for funds from individual donors. Together, these changes led DtWI to the conclusion that funding is no longer the bottleneck to reaching more people in India. (More detail in this blog post.) Funding gap DtWI told us that it seeks$1.3 million over the next two years. We expect it to allocate approximately 30% of the additional funds it receives for work related to expanding school-based, mass deworming programs (including related operating and impact evaluation expenses) and will allocate other funds to priorities that are less directly connected to expanding and evaluating deworming programs (investigating ways to combine other evidence-based programs with deworming rollouts, supplementing a project supported by another funder).

Good Ventures has announced a $250,000 grant to DtWI, leaving it with$1.05 million in remaining room for more funding over the next two years. We would ideally like DtWI to receive an additional $500,000 (for a total of$750,000) to provide it with more than half of its two-year gap.

Key considerations:

• Program impact and cost-effectiveness. Our current calculations indicate that DtWI-associated deworming, when accounting for DtWI’s potential “leverage” in influencing government funds, has extremely strong cost-effectiveness, better than bednets and 10-20 times better than cash transfers. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for deworming, cash transfers and iodine is here (.xls).)
• Directness and robustness of the case for impact. DtWI doesn’t carry out deworming programs itself; it advocates for and provides technical assistance to governments implementing deworming programs, making direct assessments of its impact challenging. There are substantial potential advantages to supporting such an organization, as it may be able to have more impact per dollar by influencing government policy than by simply carrying out programs on its own, but this situation also complicates impact assessment. While we believe DtWI is impactful, our evidence is limited, and in addition, there is always a risk that future expansions will prove more difficult than past ones. In addition, DtWI is now largely raising funds to support research projects that are not directly connected to short-term implementation of deworming programs. We do not have a view about the value of these research projects.
• Transparency and communication. DtWI has been communicative and open with us. We have only recommended DtWI for one year and therefore have less history with it than AMF, GiveDirectly, or SCI, but we believe that were something to go wrong with DtWI’s work, we would be able to learn about it and report on it.
• Risks:
• DtWI is part of a larger organization, Evidence Action, so changes that affect Evidence Action (and its other programs) could indirectly impact DtWI. For example, if a major event occurs (either positive or negative) for Evidence Action, it is likely that it would reduce the time some staff could devote to DtWI.
• Most of DtWI’s funding is in the form of restricted funding from large, institutional funders. We are not sure how DtWI’s plans would change in response to a large funder offering it significant support to undertake a project not directly in line with its current plans.

Our full review of DtWI is here.

GiveDirectlyOur full review of GiveDirectly is here.

Important changes in the last 12 months

GiveDirectly continued to scale up significantly, utilizing most of the funding it received at the end of last year. It continued to share informative and detailed monitoring information with us. Overall, it grew its operations while maintaining high quality.

In June, three of its board members launched Segovia, a for-profit company aimed at improving the efficiency of cash transfer distributions in the developing world (see our blog post on Segovia for more information).

GiveDirectly is working with other researchers to begin a very large study on cash transfers and the impact they have on broader economic factors such as inflation and job growth. This study will include a long-term follow up component as well. GiveDirectly told us that the ideal sample size for this study, which is randomized at the village level, would require $15 million for cash transfers. Baseline data collection for the study began in August 2014. GiveDirectly has preregistered its plans for measurement and analysis (more information in our review). Funding gap GiveDirectly has scaled up significantly over the past year, spending (or committing to spend by enrolling recipients) approximately$13.6 million of the $17.4 million it received last year. (It also allocated an additional$1.8 million to other organizational costs.) It now believes that it could spend up to $40 million in a year. We believe this is a reasonable cap for GiveDirectly and would not hesitate to see it receive this amount. However, due to other charities’ significantly superior estimated cost-effectiveness, we are seeking larger total amounts for them. We hope that GiveDirectly will receive at least$1 million from individual donors (excluding Good Ventures) this giving season as a result of our recommendation.

Key considerations:

• Program impact and cost-effectiveness. Our best guess is that deworming or distributing bednets achieves 5-10 times more humanitarian benefit per dollar donated than cash transfers. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for deworming, cash transfers and iodine is here (.xls).)
• Directness and robustness of the case for impact. GiveDirectly collects and shares a significant amount of relevant information about its activities. The data it collects show that it successfully directs cash to very poor people, that recipients generally spend funds productively (sometimes on food, clothing, or school fees, other times on investments in a business or home infrastructure), and that it leads to very low levels of interpersonal conflict and tension. We are more confident in the impact of GiveDirectly’s work than in that of any of the other charities discussed in this post.
• Transparency and communication. GiveDirectly has always communicated clearly and openly with us. It has tended to raise problems to us before we ask about them, and we generally believe that we have a very clear view of its operations. We feel more confident about our ability to keep track of future challenges than with any of the other charities discussed in this post.
• Risks: GiveDirectly has scaled (and hopes to continue to scale) quickly. Thus far, it has significantly increased the amount of money it can move with limited issues as a result. The case of staff fraud that GiveDirectly detected is one example of an issue possibly caused by its pace of scaling, but its response demonstrated the transparency we expect.

Our full review of GiveDirectly is here.

Schistosomiasis Control Initiative (SCI)Our full review of SCI is here.

Important changes in the last 12 months

As discussed above regarding DtWI, Dr. Kevin Croke released a new study of a randomized controlled trial of a deworming program showing large, long-term impacts from deworming programs (for more, see this blog post). This study is a significant positive update on the impacts of deworming and increased our confidence that deworming programs have significant long-term impacts.

We continued our work revisiting SCI’s case for impact (detailed here). There appear to have been major problems with some, though not all, of the studies we had relied on (pre-2013) to assess SCI’s impact. SCI shared some additional monitoring information with us which supported the conclusion that its programs have generally succeeded, though these reports have significant limitations.

We also reviewed the papers of several academics who had previously been critical of SCI’s activities. We found little in this literature to change our views on SCI’s programs.

We spent significantly more time with SCI in 2014 (including a 3-day visit to its headquarters in London) than we had in previous years, aiming to improve our understanding of its operations and spending. The picture that emerged was more detailed though largely consistent with what we believed before. Specifically:

• We are less confident in our understanding of how SCI has spent unrestricted funds. At the end of 2013, we believed we had a relatively strong understanding of SCI’s unrestricted spending, but after spending additional time reviewing reports and discussing with SCI staff, we have more questions today than we did a year ago.
• We have better information about how SCI plans to use additional funds it receives and the constraints, besides funding, that SCI faces in utilizing additional funding (more in our review).

Funding gap

SCI told us that it has approximately $3.8 million worth of opportunities that it would be highly likely to undertake if it had the funding available. (Some of this would be spent in 2015 and some held for the following year to ensure programs can continue once started). It believes it could possibly absorb an additional$4.5 million (up to $8.3 million total) for opportunities that are more speculative. Overall, our best guess is that SCI will use up to approximately$6.3 million and, beyond that, would build up reserves.

Partly for reasons of donor coordination, we have set its target at $6.8 million total (more below). We hope that SCI will receive$1 million from individual donors (excluding Good Ventures) this giving season as a result of our recommendation.

Key considerations:

• Program impact and cost-effectiveness. Our best guess is that deworming is roughly as cost-effective as distributing bednets and more cost-effective than cash transfers by a factor of 5-10. Our estimates are subject to substantial uncertainty. (Note: all our cost-effectiveness analyses are available here. Our file for deworming, cash transfers and iodine is here (.xls).)
• Directness and robustness of the case for impact. We have seen some evidence demonstrating that SCI successfully deworms children, though this evidence is relatively thin. Nevertheless, deworming is a relatively straightforward program, and we think it is likely (though far from certain) that SCI is successfully deworming people. We have had difficulties communicating with SCI (see below), which has reduced our ability to understand it; we have also spent significant time interviewing SCI staff and reviewing documents over the past 5 years and have found minor but not major concerns.
• Transparency and communication. We have had consistent difficulties communicating with SCI. Specifically, (a) we had a major miscommunication with SCI about the meaning of its self-evaluations (more) and (b) although we have spent significant time with SCI, we remain unsure of how SCI has spent funds and how much funding it has available (and we believe SCI itself does not have a clear understanding of this). Importantly, if there is a future unanticipated problem with SCI’s programs, we don’t feel confident that we will become aware of it; this contrasts with AMF and GiveDirectly, both of which we feel we have a strong ability to follow up.
• Risks: There are significantly more unknown risks with SCI than our other top charities due to our limited understanding of its activities. We hope for SCI to have $6.8 million available, which is significantly more unrestricted funding than it has had available in the past. Our full review of SCI is here. SummaryThe table below summarizes the key considerations for our four top charities. Consideration AMF DtWI GiveDirectly SCI Program estimated cost-effectiveness (relative to cash transfers) 5-10x 10-20x 1x 5-10x (and possibly more) Directness and robustness of the case for impact Strong Weakest Strongest Moderate Transparency and communication Strong Strong Strongest Weakest Ongoing monitoring and likelihood of detecting future problems Strong Strong Strongest Weakest Organizational track record of rolling out program Moderate Moderate Strong Strong Room for more funding (more below) High Limited Very high Limited when accounting for all donors Note the absence of two criteria we have put weight on in years past: • Program evidence of effectiveness. With the new evidence about deworming, we think differences on this front are much reduced, though we still think net distribution and cash transfers have more robust cases than deworming. • Potential for innovation/upside. All of these organizations are fairly mature at this point, and we expect each to get significant revenue this giving season. Standouts Much of the work we did this year went into investigating potential new additions to our top charities list. The strongest contenders we found are discussed below. Ultimately, none of these made it into our top tier of recommendations, but that could easily change in the future. We believe that more investigative effort could result in a much better understanding of GAIN-USI (discussed below) and potentially a top-tier recommendation. Meanwhile, ICCIDD and DMI (also discussed below) do not have the track record we’d want to see for our top tier of recommendations, but in both cases we expect major developments in the next year. Specifically, ICCIDD will have a substantially larger working budget (due to GiveWell money moved), and DMI may have new data from its randomized controlled trial that could cause a significant upgrade in its status. These are all strong giving opportunities, and we’ve vetted them all relatively thoroughly. Two work on a program (universal salt iodization) that we believe has excellent cost-effectiveness and a strong evidence base, and the other two have recently released data from randomized evaluations of their own programs (something that is very rare among charities). We have thoroughly vetted each of these organizations, including site visits. And we can see arguments for supporting these organizations in lieu of our top charities this year, though we ultimately recommend our top charities above them. Below are some brief comments on each standout organization. Donors interested in learning more should read our full reviews of each organization. Development Media International (DMI) produces radio and television broadcasts in developing countries that encourage people to adopt improved health practices, such as exclusive breastfeeding of infants and seeking treatment for symptoms associated with fatal diseases. Its programs reach many people for relatively little money, so if its program successfully changes listeners’ behavior, it may be extremely cost-effective. It is in the midst of running a randomized controlled trial of its program; the midline results were released earlier this year, at which point we blogged about them. At midline, the study found moderate increases (relative to the control group) in self-reported health behaviors. Our attempt to estimate the likely mortality impact of these behaviors – when accounting for other concerns about the generalizability of the study – implied cost-effectiveness worse than AMF’s. This isn’t sufficient for a recommendation this year, as DMI has much less of a track record than our top charities. However, if endline results hit DMI’s targeted mortality impact, we would expect to adjust our estimate significantly, and DMI could become a top charity. DMI’s current budget is approximately$2.5 million; it has told us it expects to receive approximately $2.5-$4 million from existing funders in the next year and could absorb an additional $6-$7.5 million, which it would either use to supplement a program already broadcasting in a country or move into a new country, depending on how much it received.

Our cost-effectiveness analysis for DMI is here (.xls).

Our full review of DMI is here.

GAIN-USI. GAIN’s Universal Salt Iodization (USI) program supports salt iodization programs. There is strong evidence that salt iodization programs have a significant, positive effect on children’s cognitive development, and we consider the program to accomplish (very roughly speaking) comparable good per dollar to bednets and deworming (see our intervention report).

GAIN-USI does not work directly to iodize salt; rather, it supports governments and private companies to do so, which could lead to leveraged impact of donations or to diminished impact depending on its effectiveness. We tried but were unable to document a demonstrable track record of impact; we believe it may have had significant impacts, but we are unable to be confident in this with what we know now. More investigation next year could change this picture.

GAIN’s USI program was one of the recipients of a large, multi-year grant from the Bill and Melinda Gates Foundation. The grant ends in 2015 and has yet to be renewed; we are unsure of whether it will be.

Donors whose primary interest is supporting a strong intervention, and who are comfortable supporting a large and reputable organization whose role is to promote and support the intervention (but whose track record we cannot assess at this time), should strongly consider supporting GAIN’s USI program.

GAIN is a large organization running many programs, so donors should consider the possibility that funds restricted to GAIN’s USI program might effectively support its other efforts (more on this general concern here). GAIN told us that it has very little unrestricted funding, so it is unlikely to be able to reallocate funds from other programs to continue to support USI work. It is possible that resources that are shared across programs (such as some staff) could be shifted toward other programs if resources for USI increased, but we would guess that this effect would be small.

Our cost-effectiveness analysis for deworming, cash transfers and iodine is here (.xls).

Our full review of GAIN is here.

International Council for the Control of Iodine Deficiency Disorders Global Network (ICCIDD). Like GAIN-USI, ICCIDD supports (via advocacy and technical assistance rather than implementation) salt iodization, and as with GAIN-USI, we tried but were unable to establish a track record of successfully contributing to iodization programs. Unlike GAIN-USI, ICCIDD is small, operating on a budget of approximately half a million dollars per year, and relies heavily on volunteer time. We believe that additional funding in the range of a few hundred thousand dollars could have a significant positive impact on its operations.

Good Ventures has granted a total of $350,000 to ICCIDD this year ($100,000 as a participation grant and $250,000 with the grants announced today), and we would be happy to see ICCIDD receive a few hundred thousand dollars more, after which point we would be more hesitant as it would be more than doubling its budget. We hope that ICCIDD will use the additional funding to improve its capacity and potentially become a top charity in the future. Our cost-effectiveness analysis for deworming, cash transfers and iodine is here (.xls). Our full review of ICCIDD is here. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bed nets, clean cook stoves and solar lights. It completed a randomized controlled trial of its program and measured a 27% reduction in child mortality. We estimate that Living Goods saves a life for roughly each$10,000 it spends, approximately 3 times as much as our estimate for the cost per life saved of AMF’s program. Living Goods has been operating on a budget of $3 million per year and aims to scale up to operate on a budget of$10 million per year, of which it expects to receive approximately two-thirds from existing funders.

Our cost-effectiveness analysis for Living Goods is here (.xls).

Our full review of Living Goods is here.

Funding targets by charityIn order to give guidance to donors seeking to give as we would, we’ve come up with funding targets for each charity. These targets are based on “dividing up” $7.5 million in money moved, which is our best guess for how much individual donors will give based on our recommendations over the next 4 months. We are using the following principles in setting targets: • We’d like each top charity to receive a substantial amount of funding. When a charity receives substantial funding at our recommendation, it (a) gives that charity good reason to continue working with us, reporting to us, and helping us learn further about its activities; (b) gives that charity the opportunity to continue building its track record and demonstrating its capabilities, information we will use in future years; and (c) continues to reinforce the idea that GiveWell-recommended charities receive substantial funding – the main incentive charities have to participate in our process. • All else equal, we’d like stronger overall charities – defined as those that accomplish more good per dollar, taking all considerations into account – to receive more funding. • Each charity has a conceptual “maximum” past which we think donations would hit strongly diminishing returns. We aren’t allocating any “money moved” to a charity in excess of the max; beyond that point, we think the money is better spent supporting other top charities. We are also taking the announced Good Ventures grants into account. These grants were recommended using similar considerations, though some of our information has changed. Our targets are as follows. Note the distinction between “total max” (the most we’d be comfortable seeing a charity take in, at which point we would make an announcement), “total target” (the total amount we would like to see this charity take in, including Good Ventures grants and other donations), “target from individuals” (the amount we are seeking specifically from GiveWell-influenced individual over the next four months), and “max from individuals” (the most we’d be comfortable seeing a charity take in, taking into account what we know about other donors’ plans). • Against Malaria Foundation:$5 million target from individuals, $5 million max from individuals. As discussed in the section on AMF, our ideal amount for AMF to take in would be$10 million this giving season, and Good Ventures has already committed $5 million. We therefore target$5 million for AMF.
• Deworm the World Initiative: $0.5 million target from individuals,$1 million max from individuals. We think Deworm the World Initiative is an outstanding giving opportunity with limited room for more funding, as discussed above.
• Schistosomiasis Control Initiative: $1 million target from individuals,$1 million max from individuals. We believe SCI will end the giving season with $3 million from Good Ventures,$1 million from a major donor who discussed his plans with us, $1 million in donations that we expect to come from non-GiveWell-related sources (based on projections from past years rather than on knowledge of specific donors). We also believe it has$1 million in cash available for the $6.3-$8.3 million in opportunities we describe above. In total, then, SCI already can expect to have $6 million available, which would be around the maximum we’d recommend in isolation. However, our discussion with the possible$1 million donor has led us to set a higher overall “total target” than we would have otherwise, settling on a total target of $6.8 million. (We plan to elaborate on our thoughts about donor coordination and donor agency in a future post.) Since we are hoping for SCI to have a total of$6.8 million available for its activities, we are recommending $1 million in donations from GiveWell-influenced individuals this giving season. (We are rounding$0.8 million in estimated remaining gap to $1 million in recommended giving since these figures are not precise, and we see value in round numbers for our targets.) • GiveDirectly:$1 million target from individuals, $25 million max from individuals. We believe GiveDirectly could absorb up to$40 million total ($5 million from the Good Ventures grant,$10 million we expect it to receive from non-GiveWell-related sources already, and $25 million on top of that). However, our revised cost-effectiveness estimates (which we will discuss more in a future post) now classify cash transfers as significantly less cost-effective than bednet distribution and deworming, by a factor of around 5-10. In addition, the$5 million grant from Good Ventures and the funds we expect it to receive from elsewhere means that GiveDirectly will raise nearly as much in its next fiscal year as it did last year. Given that we anticipate moving roughly $7.5 million from individual donors in the next four months, we’d like to direct roughly$1 million of those donations to GiveDirectly. Note that GiveDirectly is, by a substantial amount, the organization we feel has performed best and most consistently in carrying out its intervention and providing quality data on the results, and people who are particularly skeptical of cost-effectiveness estimates are likely to find it the most appealing. We also are very excited about the future of GiveDirectly, in terms of its continuing ability to produce useful information via studies and its potential to grow and raise more from sources unconnected to GiveWell, though at this point we feel GiveDirectly is mature enough that further donations are not crucial in helping it toward this goal.

Summary table (all figures in USD millions):

Charity Total max (including all donations) Total target (including all donations) Donations committed or expected from Good Ventures and non-GiveWell sources Target from individuals Max from individuals Against Malaria Foundation 10 10 5 5 5 Schistosomiasis Control Initiative 6.8 6.8 6 1 1 Deworm the World Initiative 1.3 0.75 0.25 0.5 1 GiveDirectly 40 16 15 1 25

For donations beyond the ~$7.5 million total we’re projecting over the next four months, we think the decision of which charity to support would be particularly difficult. Of our top charities, only GiveDirectly would have clear room for more funding after receiving an amount in line with the above, but the others – and to a lesser extent, some of our standout charities – have significantly superior estimated cost-effectiveness according to our latest analyses. We will be continuing to stress-test and reflect on these analyses as we reflect on the question of how to modify our recommendations once the above targets are hit. Our research process in 2014This section describes the new work we did in 2014 to supplement our previous work on defining and identifying top charities. See the process page on our website for our overall process. This year, we completed an investigation of one new intervention (salt iodization). We made substantial progress on several others (maternal and neonatal tetanus immunization campaigns, mass drug administration for lymphatic filariasis, and vitamin A supplementation) but did not complete them. We also stayed up to date on the research for bednets, cash transfers and deworming and made a substantial update to our view on deworming, based on a new study by Kevin Croke. We did not conduct an extensive search for new charities this year. We feel that we have a relatively good understanding of the existing charities that could potentially meet our criteria, based on past searches (see the process page on our website for more information). Instead, we solicited applications from organizations that we viewed as contenders for recommendations. (Living Goods is an exception; it contacted us with the results from its randomized controlled trial.) A February post laid out which organizations we were hoping to investigate and why. In addition to the 4 standout charities, we also considered Nothing but Nets (a bednets organization that declined to participate in our process), Evidence Action’s Dispensers for Safe Water program (which is forthcoming), the Center for Neglected Tropical Disease and UNICEF’s maternal and neonatal tetanus program. In the case of the latter two, we ran out of time to complete the relevant intervention reports this year (due to prioritizing other work, which seemed more likely to lead to new recommendations) and plan to complete them in 2015. Brief notes on giving now vs. later and supporting GiveWell vs. top charitiesGiving now vs. giving later Last year, some staff members chose to save some of their charitable giving budget for future giving opportunities, and we discussed the considerations about giving now vs. later in this post. This year, we think the situation is a bit different, as AMF has returned to our top charities list, the case for both SCI and GiveDirectly has improved (due to new evidence on deworming and GiveDirectly’s strong performance in disbursing cash transfers), and we have extensively investigated possible other options. With these changes, we feel that (unlike last year) this year is an excellent year to give a substantial amount if you are interested primarily on our top charities work. We think our top charity recommendations are unlikely to improve a great deal (i.e. they’re unlikely to improve enough to make saving worthwhile) in the coming years. A couple considerations that might be relevant in weighing the decision to give now versus later: • Will the giving opportunities available in the future be better than the ones we have identified now? There are competing factors. On one hand, our research capacity has expanded significantly over the past 2 years, and this has given us the ability to research more opportunities both in our traditional, top charities work and the Open Philanthropy Project. On the other, the world is getting better and some of the best opportunities available today (e.g., deworming, bednets, salt iodization) may no longer be available 10 years from now. We now feel that we’ve investigated a large proportion of realistic short-to-medium-term contenders for top charity recommendations. If money moved ends up exceeding the ~$7.5 million we’re projecting over the next four months, a stronger case for waiting may emerge, as many of the strongest charities will be near what we think they can productively absorb in the short term (and our standout charities may become recommended next year, as discussed in the section on standouts).
• How much funding will be available in the future to the opportunities we identify? Our impression is that funding available for the opportunities we identify has and will continue to grow significantly. Good Ventures is a part of this, but we hope that other future, major philanthropists will consider supporting our recommendations as Good Ventures has.

Donors interested in supporting opportunities that come from the Open Philanthropy Project have a stronger case for saving to give later. Note that it could be several years before the Open Philanthropy Project has recommendations suitable for individual donors, and these recommendations will likely reflect a very different process, very different criteria, and a much higher tolerance for high-risk opportunities that are difficult to fully explain and defend in writing (though we will work hard to lay out the basic case).

Giving to GiveWell vs. our top charities

We have grown significantly over the past 2 years and continue to raise funds to support our operations. The funds we have received have enabled us to expand our staff. Without this increased capacity, we would not have been able to consider as many organizations as we did this year.

We plan to post an update soon about our budget situation. The most up to date information available is linked from our August board meeting. The short story is that we are still seeking additional donations. For the first time this year, our checkout form will ask donors to consider allocating 10% of their donation to our operating expenses. This option is not yet live on our website; we hope to implement this change in the next few weeks.

The post Our updated top charities appeared first on The GiveWell Blog.

### Update on GiveDirectly

Fri, 06/20/2014 - 15:27

Three members of GiveDirectly‘s board of directors (Paul Niehaus, Michael Faye, and Chris Hughes) are planning to start a for-profit technology company, Segovia, aimed at improving the efficiency of cash transfer distributions in the developing world. Segovia plans to sell software to developing-country governments for use in implementing their cash transfer programs.

This development was announced today (though we have been aware of and discussing it with GiveDirectly for some time). Some discussion is available at today’s post on the Development Channel blog.

GiveDirectly and Segovia will work out of the same office space in New York City.

Dr. Niehaus, who has been our primary contact at GiveDirectly and has unofficially played the role of GiveDirectly’s full-time Executive Director, will continue to devote significant time to GiveDirectly and serve as its President with primary responsibility for GiveDirectly. He will be co-employed by Segovia and has told us that he may spend up to 20% of his time on Segovia. Dr. Faye will become Segovia’s president. (Previously, both Dr. Niehaus and Dr. Faye have had full-time jobs outside of GiveDirectly, though they have had substantial responsibilities at GiveDirectly.)

We think this development is simultaneously a potentially very positive one broadly – bringing the possibility of greatly leveraged positive impact on the world – and one that raises new issues and risks for GiveDirectly and its donors.

We think these issues and risks (discussed further below) are noteworthy but ultimately similar in magnitude to, or smaller than, similar risks that exist for our other present and past recommended charities. We plan to continue recommending GiveDirectly as a top charity and continue to see it as an outstanding giving opportunity.

Note that we have discussed all of these issues with Dr. Niehaus and Dr. Faye – they have reviewed a draft of this post – and we believe they are aware of all of the issues we discuss below.

This post focuses on the following:

• What costs and benefits does this decision pose for GiveDirectly right now?
• What additional issues could arise in the future, particularly potential conflicts of interest between Segovia and GiveDirectly?
• Why have Dr. Niehaus, Dr. Faye, and Mr. Hughes decided to serve developing country governments and why are they using a for-profit-company structure?
• What effect will this have on our recommendation of GiveDirectly?

We have not tried to formulate a view on Segovia’s possible impact because this does not seem directly relevant to GiveWell or our donors. Based on what Dr. Niehaus and Dr. Faye have told us, we believe it’s plausible that given (a) the amount of money governments transfer to recipients and (b) the amount of money that may be lost by those programs due to negligence and/or corruption, Segovia could be very impactful and may represent some of the “upside” we hoped to see from GiveDirectly.

What costs and benefits does this decision pose for GiveDirectly right now?

We discuss several potential negative impacts Segovia could have on GiveDirectly; we also discuss potential positive impacts.

1. What impact will Segovia have on key staff’s time allocation to GiveDirectly?
2. Will Segovia’s existence affect the intensity with which GiveDirectly leadership work to maximize GiveDirectly’s impact?
3. Will Segovia directly affect GiveDirectly’s ability to absorb and distribute funds to recipients?
4. Will the general public react negatively to this announcement in a way that affects GiveDirectly’s ability to raise funds or otherwise distracts it from its core work?
5. What benefits might Segovia have for GiveDirectly?

What impact will Segovia have on key staff’s time allocation to GiveDirectly?

Dr. Niehaus and Dr. Faye told us that they expect the following changes to staff time allocations due to Segovia:

• Paul Niehaus, GiveDirectly’s President, had previously been splitting his time between GiveDirectly and his academic position at University of California at San Diego. Pending the university’s approval, he hopes to take a one-year leave of absence from his academic position to enable co-employment at GiveDirectly and Segovia. During this one-year leave of absence, he expects that the total amount of time he devotes to GiveDirectly will increase slightly and that he will spend a maximum of 20% of his time on Segovia.
• Michael Faye, Segovia’s president and a member of GiveDirectly’s Board of Directors, had previously worked at a management consulting firm but spent significant time on GiveDirectly. He has now taken a leave of absence from his job and intends to spend the vast majority of his time on Segovia while still offering time to GiveDirectly. He expects the time he spends on GiveDirectly to increase. More on this below.
• Melissa Harpool, Outreach Coordinator, will split her time between GiveDirectly and Segovia. Her current primary role is managing schedules, and the people whose schedules she manages will now be splitting time between Segovia and GiveDirectly. She had previously been full-time at GiveDirectly.

Dr. Niehaus and Dr. Faye told us that relevant staff track their time allocation to projects and will be able to share whether or not they have hit the targets described above.

Will Segovia affect the intensity with which GiveDirectly leadership work to maximize GiveDirectly’s impact?

Dr. Niehaus told us that he retains his ambitions for and commitment to GiveDirectly’s long term impact, but splitting attention between two organizations is difficult, especially when both are growing rapidly and likely to face significant obstacles.

It is plausible that given GiveDirectly’s and Segovia’s overlapping leadership, staff and office space, those involved with both might see Segovia as the more exciting opportunity. We believe that this could lead to reduced ambition or it could reduce the quality of the mental effort GiveDirectly’s leadership dedicates to maximizing GiveDirectly’s impact.

Will Segovia directly affect GiveDirectly’s ability to absorb and distribute funds to recipients?

Assuming that GiveDirectly staff meets the time targets described above, we don’t think Segovia will have a direct impact on GiveDirectly’s ability to absorb and distribute funds to recipients.

Will GiveDirectly receive a negative response from the general public that affects its ability to raise funds or otherwise distracts it from its core work?

We continue to see GiveDirectly as an outstanding giving opportunity and plan to continue recommending it to donors. That said, we are not confident about how others will react and remain concerned about the impact that the general public’s reaction might have on GiveDirectly’s future fundraising prospects.

Dr. Niehaus and Dr. Faye told us that they have attempted to reduce the likelihood that the response is negative by speaking at length with media in advance of the announcement so that stories written about their decision present a reasonable perspective on this new development. They have also communicated with their major donors and report that they have not encountered negative reactions.

What benefits will Segovia provide for GiveDirectly?

Potential benefits include:

• GiveDirectly will receive an equity stake in Segovia, which could result in GiveDirectly’s receiving additional funding in the future. The size of the stake is not yet determined. Dr. Niehaus, Dr. Faye, and Mr. Hughes are currently discussing the size of this stake with potential investors.
• The technology Segovia is planning to develop would likely be helpful to GiveDirectly. Segovia would give this technology to GiveDirectly without charge.
• As discussed above, Paul Niehaus has been based in San Diego and the rest of GiveDirectly staff is in New York. Dr. Faye has been employed full-time at a management consulting firm. Dr. Niehaus will be spending half his time in New York and hopes to take leave from his academic position, and Dr. Faye will now be working full-time out of the same office. Dr. Niehaus’s co-location with the rest of GiveDirectly staff will likely improve his ability to manage other staff. Dr. Faye’s co-location with Dr. Niehaus and other GiveDirectly staff may also increase his contribution to GiveDirectly. (Dr. Faye has told us that the time he has spent on GiveDirectly has increased since he took leave of absence from his job.)
• Mr. Hughes intends to significantly increase his work on advocating for cash transfers. This should benefit both Segovia and GiveDirectly.

What additional issues could arise in the future, particularly potential conflicts of interests between Segovia and GiveDirectly?

There may be cases where GiveDirectly has to consider actions that would maximize its impact but might harm Segovia’s interests. GiveDirectly board members (Paul Niehaus, Michael Faye, and Chris Hughes) will hold equity stakes in Segovia, so their financial interests could come into conflict with their roles as Directors of GiveDirectly. We see the following possible conflicts of interest:

• GiveDirectly’s board members’ financial interest in Segovia could lead them to use GiveDirectly as a means to promote Segovia. This could be via using Segovia’s software even if it’s not well suited to GiveDirecty’s needs, or otherwise using contacts/meetings that might take place due to GiveDirectly (e.g., government, academic or media contacts) to promote Segovia’s offering.
• Segovia will also have (a) investors and (b) staff who hold significant financial stakes in Segovia, which could lead to conflicts between maximizing profit and maximizing impact.
• If Segovia were bidding on a contract with a particular government, would GiveDirectly avoid offering its service in the same area/to the same government so that Segovia would have an easier path to a sale?

We have spent significant time with Paul Niehaus and some time with Michael Faye and Chris Hughes over the past few years, and we believe they have good intentions.

In addition, Dr. Niehaus, Dr. Faye, and Mr. Hughes hope to identify investors whose primary motivation is social impact, and believe that choosing investors wisely is a priority. They have also told us that they plan to expand GiveDirectly’s board to 6-7 directors, 3-4 of whom have no overlap with Segovia. Dr. Niehaus told us that overlapping directors would recuse themselves from votes that involve conflicts.

Why have Dr. Niehaus, Dr. Faye, and Mr. Hughes decided to serve developing country governments and why are they using a for-profit-company structure?

Dr. Niehaus and Dr. Faye believe that Segovia’s product is one that governments will want to purchase, and the product will have significant social impact. They have had a long-standing interest in working directly with governments.

Dr. Niehaus and Dr. Faye told us of their hope that GiveDirectly would work with government-run cash transfer programs in November 2013. We discuss this possibility in our review of GiveDirectly, relying on a summary of a conversation we had with them at the time.

Dr. Niehaus and Dr. Faye told us recently that they had initially hoped governments would transfer funds directly to/through GiveDirectly. The developing-country governments that GiveDirectly spoke with preferred technology to fully outsourcing implementation, saying that they already had a significant number of individuals employed to implement their cash transfer programs. Instead, governments asked for software that could improve their operations, which Segovia now aims to provide.

GiveDirectly still believes it will have opportunities to implement government programs, but Dr. Niehaus and Dr. Faye have come to the conclusion that there will be many more cases where governments want technology alone.

Dr. Niehaus and Dr. Faye pointed us to a World Economic Forum report estimating that developing-country governments distribute $400 billion in transfers each year. Dr. Niehaus and Dr. Faye have also told us that data showing rates of leakage of 50% or more are not uncommon in large public-sector transfer programs (i.e., the amount that never reaches the intended recipients). (More information about these sources in this footnote.) They believe that governments will see that purchasing Segovia’s product will save them money by allowing them to transfer more money to recipients at lower overall cost. We find the above explanation of Segovia’s potential impact plausible but have not tried to vet it as we don’t think our take on it has direct relevance to GiveDirectly or the donors who use our research. We have the impression that the belief that Segovia could have great social impact is the primary driver of Dr. Niehaus’s, Dr. Faye’s, and Mr. Hughes’ desire to start Segovia. Why has GiveDirectly settled on this corporate structure as opposed to another structure? Dr. Niehaus, Dr. Faye, and Mr. Hughes had initially expected to undertake this project as part of GiveDirectly’s existing non-profit structure but told us that they decided on the structure of a for-profit, independent company for three reasons: 1. Recruiting. We spoke with the recruiting firm that GiveDirectly retained for this search, and the person who led the search told us that recruiting top technology talent was slow. In some cases, the engineers GiveDirectly contacted were not interested in working for a non-profit. Even when GiveDirectly offered compensation packages competitive with for-profit companies, some engineers balked when they saw the negative attention that the media and donors give to high salaries in the non-profit sector. Dr. Niehaus, Dr. Faye, and Mr. Hughes place high priority on recruiting the very best possible talent, so while they feel they could have reasonable success recruiting as a non-profit, they see the improved recruiting prospects associated with a for-profit to be a major consideration. 2. Investment. GiveDirectly told us that there are investors who would support Segovia as a for-profit entity but would not be interested in supporting GiveDirectly, the non-profit. 3. Legal advice. GiveDirectly received legal advice that an independent for-profit company is the most straightforward way to avoid jeopardizing GiveDirectly’s tax exempt status. What effect will this have on our recommendation of GiveDirectly? We do not expect the existence of Segovia to change our recommendation of GiveDirectly. We expect GiveDirectly to continue to successfully distribute cash to very poor individuals in the developing world, and believe that the issues and risks described above are smaller than, or at worst similar in importance to, those that exist with all of our other recommended charities. We will continue to follow GiveDirectly closely and report on its progress. We have written previously about the “upside” we saw in GiveDirectly. We think that Segovia may be one example of that “upside” — Dr. Niehaus and Dr. Faye, partly through their work on GiveDirectly, saw an opportunity for significant social impact and are now pursuing it. However, we think the attention they will now pay to Segovia likely diminishes the upside of future donations to GiveDirectly. Footnote: On the World Economic Forum report described above, Dr. Niehaus wrote, “I have some questions about the methodology but believe the basic message that it is big and has problems.” On the leakage rates, he wrote, “India’s two largest social programs are the employment scheme (NREGS) and ration scheme (TPDS). For NREGS, the best nationally representative leakage estimate is by Imbert and Papp (published in R. Khera, editor, The battle for employment guarantee. Oxford University Press, 2011) who estimate that between 44% and 58% of participation reported in official figures is fictitious. This likely understates leakage in dollar figures since people who do work are often underpaid, but nationally representative data on earnings are not to the best of my knowledge available. For TPDS, the most recent nationally representative figures I know of are from the 2004-2005 NSS and are discussed in work by Svedberg in EPW who reports a national average estimate of 54% leakage of grains intended for the poor.” The post Update on GiveDirectly appeared first on The GiveWell Blog. ### A journalist visits GiveDirectly villages in Kenya Wed, 05/28/2014 - 13:16 In February, Jacob Kushner, a journalist living in Kenya, contacted us. We have long been interested in seeing more substantive coverage of philanthropy, so we were excited to talk to him. As a pilot project, Mr. Kushner decided to visit villages in which GiveDirectly had distributed some of its earliest cash transfers. We spoke with Mr. Kushner several times to offer thoughts and feedback, but we encouraged him to write about whatever he found (positive or negative about GiveDirectly). We also put him in touch with GiveDirectly to confirm that staff there were amenable to this project. Mr. Kushner completed his trip in April, and his full article follows. He also shared his full interview notes with us which we’ve posted here. We’ve summarized what we took away from his article here. Carolina Toth, Manager, People and Partnerships at GiveDirectly responds here. When giving out cash to the poor, what happens when some are left behind? A closer look at whether GiveDirectly’s cash transfers stoke community tension in Western Kenya By Jacob Kushner For several years now, the charity GiveDirectly has experimented with different ways of deciding who among Western Kenya’s rural poor should receive cash transfers. It’s an important consideration, because$1,000 means a lot to the families that receive it—and it can mean a lot of disappointment to the families that don’t. Last month I traveled to Western Kenya to speak with both lots, and I found that the discrepancy did not go unnoticed in their communities.

To date, GiveDirectly has undergone five different transfer programs in Siaya over the past three years, with different metrics for selecting recipients. I interviewed recipients from three of those cohorts:

• The Google Cohort (approximately 850 ‘thatch-roof only’ recipients whose transfers were completed in October 2013)
• The 200k Cohort (approximately 200 ‘thatch-roof only’ recipients whose transfers were completed in January 2013)
• The 2M cohort (approximately 2,000 recipients divided into ‘thatch-roof only’ villages and ‘saturation’ villages (in which nearly everyone is eligible) who have received one major transfer and will receive the second and final one in July 2014).

In a follow-up to a randomized controlled trial, GiveDirectly asked residents if they’d heard any complaints about GiveDirectly in their community. Sixty-four percent of respondents in Siaya County answered “yes,” as did 48 percent of those in the “Google” cohort (in Rareida it was 28 percent).

Fewer than 6 percent of respondents in all four groups said shouting or angry arguments had ensued because of the transfers, and fewer than 4 percent said they’d experienced crime, theft or violence or felt threatened as a result. Virtually no one said they’d argued with family members over how to spend the money, and no more than 7 percent in any group said their village elder had approached them asking for money.

Carolina Toth, Kenya Field Director for GiveDirectly, explained the results of a series of informal community group meetings in which GiveDirectly led residents in a discussion of who should be eligible for transfers.

Sixty-two percent of respondents in thatch-only villages said they’d heard complaints relating to ineligible households, compared with 46 percent in saturation villages. Thirty percent of those in thatch-only villages said they’d heard complaints about different criteria being used across different villages, compared with only 4 percent in saturation villages.

GiveDirectly concluded that the strongest takeaway from the discussions is that poorer ‘thatched’ households are more deserving but also that certain households that have mabati or permanent houses are deserving of the transfers as well. When asked about their own villages, residents preferred the saturation method. When asked about other villages, they preferred thatch-only. No one thought it would be “bad” if cash were given to some wealthier households.

Because recipients in saturation villages have yet to receive their second transfer (due in July), it’s too early to draw definite conclusions. But this and other previous reports leave several question unanswered:

To the extent that community tension may result in the wake of cash disbursements, how does that tension actually unfold? Who are the parties and what are some examples? Most importantly, what do non-recipients in those communities think about the fairness of the selection process? Do they feel stigmatized for not having received the money, and how does their perception of whether animosity resulted from the cash transfers compare with those of the recipients’ themselves?

In April I made a reporting trip to Siaya County to interview recipient and non-recipients in the communities where GiveDirectly has made those disbursements. Over three days I interviewed 15 people, asking whether they were happy with GiveDirectly’s selection process and whether any tension arose in their communities as a result of it.

I interviewed some recipients from each of the three cohorts and also interviewed recipients and in both the ‘saturation’ and ‘thatch’ divisions of the 2M cohort. I interviewed four non-recipients, at least one in each of the three cohorts.

My interviews seemed to reflect many of the conclusions of the RCT and subsequent follow up interviews and meetings. No one reported intra-family arguments about how to spend the money or being coerced by a spouse or family member to spend it in a particular way. Only one recipient said he’d originally disagreed with his spouse but that they eventually came to a mutual agreement. No one reported theft or that their own money had gone to waste in any way.

But 12 of the 15 respondents did indicate that some amount of tension had fostered in their community as a factor of some people having received money while others did not. By far the most tangible conflict mentioned to me occurred in the 200k cohort in the village of Koga.

There, the village elder did not receive a cash transfer. He was, however, consulted by GiveDirectly staff to assist in a tour of the boundaries of the village so GiveDirectly could identify eligible households, for which he was given a small token payment as compensation for his time. But in the words of one recipient there, “there was a scandal.” The elder “had conspired (to enlist) some households that were outside the area and had better houses, with the understanding that they would give him some money.”

GiveDirectly staff say the elder seems to have directed residents who lived in tin-roof houses to “squat” in vacant thatch roofed houses in order to receive the money. Subsequently, the assistant chief, with the support of the other village council members, dismissed the elder from his position.

When I spoke with the elder, he confirmed that he had misrepresented certain households in the village so they would be enrolled in the program. He justified that decision saying, “I was the village elder and I was working for the (entire) community.”

He said tension resulted when the initial disbursements were made and some families, including his own, were left out.

“I felt degraded by my community members. They were laughing at me that I didn’t receive any help even though I was the leader of the community. I was so humiliated.” He said the incident led him to ‘resign’ after more than 35 years of serving as an elder in Koga (he is 62 years old).

The second most tangible takeaway was the resentment and frustration expressed by the four non-recipients I interviewed. One woman in a “saturation” village was visibly angry as she described how she was not selected because the living room in her tin roof house is cemented, even though her other rooms are not. Another Koga man said he was cheated out of a transfer:

￼￼“The time the GiveDirectly team was working in the village, they came to my home ￼but at that time I was grazing cattle outside the compound and I saw them in my ￼sister-in-law’s house. I was curious. But due to how relations within households go ￼sour, my sister told the GiveDirectly team that I had left and was never around.”

￼Despite an appeal he said he made to GiveDirectly field staff, this man did not ￼receive a transfer. He says his economic situation is similar to that of the other recipients:

￼“I live in a house like this—(a) grass thatch house. I have children in school and I struggle to pay their fees. Some of my children for lack of funds have to be supported by my relatives in other areas, in Nairobi. I have only two cattle.”

GiveDirectly staff pointed out that “targeting” is a universal problem in development aid. Other methods used to select recipients—such as letting communities vote on who should receive, or requiring people to go to some lengths to prove they are indeed quite economically poor-off—have major drawbacks: Cronyism, and excessive bureaucracy and burdens, respectively. As an alternative, GiveDirectly employs another common method that uses easy-to-observe characteristics such as roof style to judge how wealthy or poor a household is. According to GiveDirectly’s own research, less than 5 percent of people in the 2M cohort villages complained, legitimately or otherwise, of being unfairly excluded. (￼In comparison, a recent study ￼￼of the Kenya Hunger Safety Net Program found an exclusion error rate of 46 ￼percent).

￼The man in Koga who says he was unfairly excluded also expressed sympathy for ￼the Koga village elder. “I would not be happy with what has happened to him, ￼because the feeling he has now at losing his job is the same feeling I have at not ￼getting the money. I feel bad for him because I am also going through some pain.”

￼The man also aired some critiques as to how some people in the community spent their money.

￼“I saw some beneficiaries, the way they misbehaved when they got the money, and ￼that made me feel it is important that recipients receive training on how to spend it. For example there are people who wasted it on drinking sprees, and others bought ￼items that they didn’t understand how they would maintain. For example, one bought a motorbike and used it for a few months, but now it is unused and has not ￼really helped him.”

￼Indeed, several interviewees mentioned the need for training to accompany the transfer process. GiveDirectly currently does not provide training or advise ￼recipients as to how they should spend their money. GiveDirectly does, however, provide a brochure that lists different possible categories of expenditure such as home construction, business, and farming. GiveDirectly is considering experiments ￼in which brochures also list the average returns that previous beneficiaries earned on each category of investment.

After completing the interviews, I asked Carolina Toth, the GiveDirectly field director, what she made of it all. I asked Toth what she thought about the village elder scandal in Koga—that a man who had served as elder for 35 years lost that position not because he violated a community custom, but simply a rule imposed by GiveDirectly.

“The village elder more often than not is one of the richer members of the community,” Toth said. As to his “previous feelings of entitlement to benefit from whatever is happening … I don’t think that’s an expectation we want to uphold.”

Toth and I also discussed the consequences for individuals who are excluded in a community where most residents receive the cash.

“It’s definitely a psychological event in their live,” Toth said. “But we know from the (randomized controlled trial) that there are huge spillover effects to the people who didn’t receive.”

When I asked Toth about the man who says he missed out on the transfer because his sister-in-law misinformed the GiveDirectly staff that he was not living in the village, Toth said it’s certainly true that some people get left out by mistake. But she said such cases are rare. As to the woman with the cemented living room who didn’t receive cash even though the rest of her home is not yet cemented, Toth said the GiveDirectly field staff can only make decisions based upon what they see—and that the distinction between a cemented house and a non-cemented house is not always entirely clear under such circumstances.

The vast majority of people who aren’t selected, said Toth, are skipped because they come from a marginally higher socioeconomic standing to whom the money would be less useful.

“What is the value of \$250 given to a family that’s richer? Wouldn’t that be more valuable in the hands of people who are really poor?” Toth asked. “We have a mission of giving to the extreme poor, so by excluding some people who are not in the extreme poor, you are able to reach more extreme poor.”

Ultimately, the question any cash transfer implementer must decide is, “Is the possibility that community tension may result from a non-universal disbursement so great or concerning that transfers should be made to all residents in a village despite the opportunity cost that fewer, even poorer people in other villages will not receive any cash?”

Thus far GiveDirectly has answered that question in the negative. With certain exceptions (such as allowing communities to nominate a pre-determined number of otherwise unqualified people for the disbursements) and with increased nuance (by considering more advanced criteria than simply thatch versus tin roofs and indoor plastering), GiveDirectly intends to continue excluding those residents who do not qualify as the poorest of the poor.

Jacob Kushner is a journalist based in Nairobi. He reports on foreign aid and investment in Africa, human rights and the extractives sector.

The post A journalist visits GiveDirectly villages in Kenya appeared first on The GiveWell Blog.