AMF Blogs

Our recommendations for giving in 2022

1 year 3 months ago

We wrote back in July that we expected to be funding-constrained this year. That remains true as we approach the end of the year, putting us in the unusual position of leaving impact on the table.

We've set a goal of raising $600 million in 2022, but our research team has identified $900 million in highly cost-effective funding gaps. That leaves $300 million in funding gaps unfilled. By donating this year, you can help us not only meet but exceed our goal—and say yes to more excellent opportunities to save and improve lives.

Additionally, our giving guidance for donors has changed this year. For the first time, our top recommendation is to give to our new All Grants Fund, which we allocate to any need that meets our cost-effectiveness bar. We think it's the best bet for donors who want to support the most promising opportunities we've found to help people, regardless of program or location. And it reflects our current views on how we can best meet our goal of maximizing global well-being—by taking advantage of every path to impact, whether that's funding top charities, seeding and scaling newer programs, or funding research.

Read More

The post Our recommendations for giving in 2022 appeared first on The GiveWell Blog.

Miranda Kaplan

Our recommendations for giving in 2021

2 years 3 months ago

You can have a remarkable impact by supporting cost-effective, evidence-based charities.

Just looking at the approximately $100 million GiveWell had discretion to grant in 2020—a subset of all the money we directed to the charities we recommend—the impact of our donors is impressive. We estimate these grants will:

  • Save more than 24,000 lives
  • Treat over 6 million children with a full course of antimalarial medication
  • Provide vitamin A supplementation to over 8.6 million children
  • Deliver over 4.4 million long-lasting insecticide-treated nets (LLINs) to protect against malaria
  • Vaccinate 118,000 children
  • Treat over 11.4 million children for parasitic worms

Read More

The post Our recommendations for giving in 2021 appeared first on The GiveWell Blog.

Ben Bateman

Initial thoughts on malaria vaccine approval

2 years 5 months ago

The World Health Organization (WHO) recently recommended the widespread use of the malaria vaccine RTS,S/AS01 for children. It provides an additional, effective tool to fight malaria. This is great news!

We've been following this vaccine's development for years and, in the last few months, have been speaking with organizations involved in its development and potential wider rollout.

Our work on RTS,S (and other malaria vaccines) is ongoing, and we might significantly update our views in the near future. But because we've been following its progress, we're sharing some initial thoughts.

Read More

The post Initial thoughts on malaria vaccine approval appeared first on The GiveWell Blog.

Julie Faller

Our recommendations for giving in 2020

3 years 3 months ago

You can have a major, positive impact today by choosing to support organizations backed by strong evidence: our top charities.

We recommend the nonprofits that offer the most impact per dollar we're aware of. In fact, we estimate that you can save a life by donating $3,000-$5,000 to our top recommendation.[1]

If you're a longtime donor, you'll recognize most of this year's top charities. You may even wonder why our list hasn't changed much. However, a tremendous amount of research—truly thousands of hours—has been done to ensure that these organizations continue to meet our high standards. And although there are many familiar names, one is entirely new: New Incentives.

We're proud to share our recommendations and grateful to you for considering supporting them. We hope you'll read on!

Read More

The post Our recommendations for giving in 2020 appeared first on The GiveWell Blog.

Catherine Hollander

Maximum Impact Fund update: Q1 and Q2 2020

3 years 5 months ago

Thanks to our donors, we have disbursed $23.3 million in flexible funding to our top charities this year. This generous, flexible support is worthy of celebration!

This post focuses on our decision to grant $15.3 million to the Against Malaria Foundation (AMF), which includes the $11.7 million that donors gave to "Grants to recommended charities at GiveWell's discretion" in the first half of 2020.[1]

AMF supports the distribution of insecticide-treated nets in areas with high rates of malaria. The nets stop mosquitoes from biting and spreading the disease. We estimate our donors' support for AMF will collectively save over 3,000 lives, mostly of young children in the Democratic Republic of the Congo (DRC) and Guinea.[2] Without this grant, we think net distributions in DRC and Guinea would have been delayed.

We believe that AMF was the highest-impact choice for this grant. We chose AMF after assessing the effect of the COVID-19 pandemic on our top charities, the urgency of our top charities' funding needs, and our estimates of their impact per dollar. We're grateful for GiveWell donors' trust in providing flexible funding to fill this need.

Read More

The post Maximum Impact Fund update: Q1 and Q2 2020 appeared first on The GiveWell Blog.

Catherine Hollander

Why we’re excited to fund charities’ work a few years in the future

3 years 6 months ago

We recently spoke with someone who wanted to donate to a GiveWell top charity. They were interested in getting the funding "out the door" and to program participants as quickly as possible.

But our top choice for funding today is Malaria Consortium's seasonal malaria chemoprevention program—for work it expects to complete in 2022.[1] The potential donor was puzzled. Shouldn't we prioritize an organization that needs the money sooner?

We often recommend donations today that support programs a few years from now. This probably diverges from many people's intuitions about getting funding out the door as soon as possible.

Read More

The post Why we’re excited to fund charities’ work a few years in the future appeared first on The GiveWell Blog.

Catherine Hollander

GiveWell donors supported more than direct delivery: AMF and new net research

3 years 10 months ago

Supporters of the Against Malaria Foundation in recent years may have had even more impact than they expected.

The Against Malaria Foundation (AMF) is a GiveWell top charity because we believe its program to distribute insecticide-treated nets prevents people from dying of malaria. AMF-supported net distributions are incredibly cost-effective; we estimate that a $2,000-3,000 donation averts one death.[1] AMF's work is important in and of itself to fund.

Not all AMF donations, however, just support typical net distributions. In recent years, AMF supported research on a new type of insecticide-treated net, the piperonyl butoxide (PBO) net. This research itself (i.e., researcher time, equipment, and administrative costs) was funded by a small number of AMF donors who explicitly agreed to support it. The research was conducted on nets that were contributed by a broad group of AMF donors.

Preliminary data suggest that PBO nets are more effective at preventing malaria than standard insecticide-treated nets in areas where mosquitoes have developed insecticide resistance. We think it is likely that AMF sped up the completion of a large-scale, high-quality study of these new nets.

We're thrilled to recommend charities that contribute research in the fields in which they operate. AMF doesn't just prevent deaths from malaria by distributing nets—it has improved our and others' understanding of which nets can work best in the future. This post is to share this story with our donors, whose contributions make this work possible.

Summary

In this post, we'll discuss:

  • Insecticide resistance and the potential of PBO nets. (More)
  • AMF's role in PBO net research. (More)

Read More

The post GiveWell donors supported more than direct delivery: AMF and new net research appeared first on The GiveWell Blog.

Catherine Hollander

Announcing our 2019 top charities

4 years 3 months ago

We're excited to announce our top charities for 2019. After thousands of hours of vetting and review, eight charities stood out as excellent.

These charities work on evidence-backed and impactful health and poverty alleviation programs serving people in the poorest parts of the world. We've identified specific opportunities for our top charities to use an additional $75 million in donations to save 33,000 lives, $30 million to treat 36 million children for parasitic worm infections, and $450 million to provide unconditional cash transfers to 375,000 extremely low-income individuals. Our expectation is that our top charities can effectively use even more funding than that—that's just a starting point.

Our 2019 recommendation: "Grants to recommended charities at GiveWell's discretion"

Our top recommendation for donors giving in 2019 is to give to "Grants to recommended charities at GiveWell's discretion." We will grant these funds each quarter to the top charity or charities where we believe they will have the greatest impact.

The top charity we model as having the highest impact per additional dollar can change throughout the year. To inform our understanding, we ask our top charities to provide us with updated information on an ongoing basis. For example, a top charity may share that it has found new opportunities for impact, such as the potential to work in a new country with a significant need for its program.

In addition, top charities typically receive funding from GiveWell donors and other sources on an ongoing basis. We update our expectations of how much additional funding charities need each quarter by incorporating funding they have received since our last allocation of "Grants to recommended charities at GiveWell's discretion."

Summary
  • Our 2019 top charities (More)
  • How we prioritize our top charities' funding needs (More)
  • New information we learned in 2019 (More)
  • Giving to GiveWell's operations (More)
  • Tips for donating efficiently (More)
  • Questions? (More)
  • More information on our top charities and 2019 review process (More)

Read More

The post Announcing our 2019 top charities appeared first on The GiveWell Blog.

Catherine Hollander

Allocation of discretionary funds from Q2 2019

4 years 6 months ago

In the second quarter of 2019, donors gave a combined $2.3 million to GiveWell for granting to recommended charities at our discretion. We greatly appreciate this support, which enables us to direct funding where we believe it can be used most effectively. We grant this funding to one or more of our top charities each quarter.

We decided to allocate all $2.3 million to the Against Malaria Foundation (AMF). AMF is a GiveWell top charity that provides support for the distribution of long-lasting insecticide-treated nets to prevent malaria. AMF has been named a GiveWell top charity seven times. We chose to allocate the second-quarter funding to AMF because we believe AMF has a highly cost-effective and time-sensitive opportunity to spend it.

Our bottom line

We continue to recommend that donors giving to GiveWell choose the option on our donation form for "grants to recommended charities at GiveWell’s discretion" so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to AMF, which we believe could use additional funding for highly cost-effective work, even after receiving the $2.3 million in funding mentioned above.

Read More

The post Allocation of discretionary funds from Q2 2019 appeared first on The GiveWell Blog.

Catherine Hollander

Allocation of discretionary funds from Q1 2019

4 years 9 months ago

In the first quarter of 2019, donors gave a combined $4.7 million for granting to recommended charities at our discretion.

We really appreciate the generosity of our supporters in making it possible for us to regularly allocate funding to the top charity or charities that we believe can best use additional funding. Thank you!

In this post, we discuss our decision to allocate this $4.7 million to the Against Malaria Foundation (AMF), as well as the process we followed to arrive at this decision.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to AMF, which we believe could use additional funding for highly cost-effective work, even after receiving the $4.7 million in funding mentioned above.

Our bottom line

As we did last quarter, we focused our efforts on deciding between allocating funding to Malaria Consortium vs. AMF. We currently believe that AMF has a more time sensitive funding need than Malaria Consortium, and our best guess is that it will have equivalent impact per dollar to Malaria Consortium. This led us to allocate funding to AMF.

Read More

The post Allocation of discretionary funds from Q1 2019 appeared first on The GiveWell Blog.

Isabel Arjmand

Allocation of discretionary funds from Q2 2018

5 years 6 months ago

In April to June 2018, we received $1.2 million in funding for making grants at our discretion. In addition, GiveWell's Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. In this post we discuss:

  • The decision to allocate the $4.1 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
  • Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.
  • Why we have allocated unrestricted funds to making grants to recommended charities.

Read More

The post Allocation of discretionary funds from Q2 2018 appeared first on The GiveWell Blog.

Natalie Crispin

Why we don’t use subnational malaria mortality estimates in our cost-effectiveness models

5 years 6 months ago
Summary

We recently completed a small project to determine whether using subnational baseline malaria mortality estimates would make a difference to our estimates of the cost-effectiveness of two of our top charities, the Against Malaria Foundation and Malaria Consortium. We ultimately decided not to include these adjustments because they added complexity to our models and would require frequent updating, while only making a small difference (a 3-4% improvement) to our bottom line.

Though this post is on a fairly narrow topic, we believe this example illustrates the principles we use to make decisions about what to include in our cost-effectiveness model.

Background

Two of our top charities—the Against Malaria Foundation (AMF) and Malaria Consortium’s seasonal malaria chemoprevention program—implement programs to prevent malaria, a leading killer of people in low- and middle-income countries.

One of the core reasons we recommend AMF and Malaria Consortium is their cost-effectiveness: how much impact they have (e.g., cases of malaria prevented, malaria deaths averted) with the funds they receive. Our estimates of charities’ cost-effectiveness isn’t just helpful to us in determining which charities should be GiveWell top charities; we also rely on these estimates to guide our decisions about how to allocate funding between our top charities.

Our cost-effectiveness estimates for AMF and Malaria Consortium use country-wide data on malaria mortality and malaria incidence in the places that both organizations work.1In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); However, neither organization serves a whole country—rather, they operate in sub-national regions—so the use of country-level estimates could cause us to either underestimate or overestimate their cost-effectiveness. If, for example, these programs are focused in the areas of the country with the highest malaria burden, using the average burden for the country would lead us to underestimate their cost-effectiveness. So, we completed a project to determine how much of an impact using subnational estimates would have, to consider whether we ought to incorporate this information into our cost-effectiveness analysis.

How we estimated the impact of subnational malaria incidence

AMF distributes insecticide-treated nets to prevent malaria; Malaria Consortium’s seasonal malaria chemoprevention (SMC) program provides preventive anti-malarial drugs. We used estimates of subnational malaria incidence from the Malaria Atlas Project (MAP) to see if regions covered by nets or eligible for SMC had higher or lower incidence than the average in the country in which they are located.2We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We focused on all areas covered by nets or eligible for SMC (rather than those covered by our top charities, specifically) for two reasons:

  1. Our understanding is that when our top charities contribute resources to a country’s net distribution or SMC programs, the marginal region covered by these additional resources is not necessarily the same as the region to which these resources are assigned (because these resources are fungible with other resources within the national programs).3A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
  2. Our aim is to estimate the cost-effectiveness of funds donated to these organizations in the future. The subnational region where AMF has worked in the past has not historically been a good indicator of the region where it will work in future.
Results for net distributions in countries where AMF works

We looked at geographical variation in malaria incidence in countries where AMF works, weighting each region by the number of nets it currently receives.4We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average net delivered in the countries in which AMF works is hung in an area with 0-9% higher malaria incidence than the average in that country, and the weighted average adjustment to AMF’s cost-effectiveness would be 3% (in other words, AMF becomes 3% more cost-effective if we incorporate subnational estimates).5See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Zambia +9% Uganda +4% Ghana +4% Democratic Republic of the Congo +1% Togo +1% Malawi +0% Results for SMC in countries where Malaria Consortium works

We looked at six countries comprising >95% of Malaria Consortium’s SMC spending and compared malaria incidence in districts eligible for SMC with the country-wide average.6“The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });7The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average region eligible for SMC in countries where Malaria Consortium works has -2% to 17% higher malaria incidence than the average in that country. The weighted average adjustment to Malaria Consortium’s cost-effectiveness would be 4%.8See Cell C126. jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Commentary Guinea +17% Conakry, the capital, is ineligible for SMC and has low incidence. Nigeria +12% SMC appears to be targeted in the north, where malaria incidence is slightly higher. Niger +2% The majority of the population is either covered or planned to be covered from 2019. Burkina Faso 0% All districts are eligible. Mali 0% All districts are eligible. Chad -2% The four regions with very low malaria incidence (Borkou, Tibesti, Ennedi Est and Ouest) aren’t eligible for SMC, but are sparsely populated. What we concluded

We decided not to include these adjustments in our cost-effectiveness analysis because they increased complexity, without substantially affecting the bottom line.

When we decide whether to include adjustments in our model in general, we use a framework that first takes our best guess of the likely effect size and then rates each of the remaining question on a three-point scale.

Score9We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. jQuery("#footnote_plugin_tooltip_9").tooltip({ tip: "#footnote_plugin_tooltip_text_9", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Commentary Best guess of effect size 3-4% Can it be objectively justified? 3/3 While we have not investigated the MAP data in detail, we would guess that after further investigation, we would conclude it provides a reasonable approximation of subnational malaria incidence.10You can read more about MAP’s methodology in this paper. jQuery("#footnote_plugin_tooltip_10").tooltip({ tip: "#footnote_plugin_tooltip_text_10", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); How easily can it be modelled? 3/3 The methodology is clear and simple. Is it consistent with our other cost-effectiveness analyses? 2/3 We could include subnational adjustments for both of our top charities that implement malaria-prevention programs, but we believe it is unlikely there would be sufficient data to do the same for prevalence of worms or vitamin A deficiency (the focus of five of our other seven top charities).

Even though these adjustments can be objectively justified and are fairly easy to model, the bottom-line difference they make to our cost-effectiveness estimates is insufficient to warrant the (moderate) increase in the complexity of our models. These adjustments would also introduce an inconsistency between our methodologies for top charities. As a result, we are not planning to incorporate subnational adjustments at this time.

When would we revisit this conclusion?

We will revisit using subnational malaria mortality estimates if AMF or Malaria Consortium start working in countries where it would make a large difference to the bottom line. We would include subnational adjustments if AMF contributed nets in any of these countries: Djibouti (+500% adjustment), South Africa (+259%), and Swaziland (+126%), where malaria is endemic in some parts of the country but not others. We would also consider subnational adjustments if AMF contributed nets in Namibia (+25%), Kenya (+23%), Madagascar (+14%), or Rwanda (+10%).11The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_11").tooltip({ tip: "#footnote_plugin_tooltip_text_11", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We will investigate whether subnational adjustments would make a substantial difference if Malaria Consortium enters additional countries; at this time, we do not have details on which regions are eligible for SMC in countries in which Malaria Consortium is not currently operating.12We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. jQuery("#footnote_plugin_tooltip_12").tooltip({ tip: "#footnote_plugin_tooltip_text_12", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

You can read the internal emails discussing our decision process here.

Notes   [ + ]

1. ↑ In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. 2. ↑ We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. 3. ↑ A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. 4. ↑ We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. 5. ↑ See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. 6. ↑ “The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. 7, 11. ↑ The data and calculations are in this spreadsheet. 8. ↑ See Cell C126. 9. ↑ We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. 10. ↑ You can read more about MAP’s methodology in this paper. 12. ↑ We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Why we don’t use subnational malaria mortality estimates in our cost-effectiveness models appeared first on The GiveWell Blog.

James Snowden (GiveWell)

Allocation of discretionary funds from Q1 2018

5 years 9 months ago

In the first quarter of 2018, we received $2.96 million in funding for making grants at our discretion. In this post we discuss:

  • The decision to allocate the $2.96 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
  • Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.

Allocation of discretionary funds

The allocation of 70 percent of the funds to AMF and 30 percent to SCI follows the recommendation we have made, and continue to make, to donors. For more discussion on this allocation, see our blog post about allocating discretionary funds from the previous quarter.

We also considered the following possibilities for this quarter:

Helen Keller International (HKI) for stopgap funding in one additional country

We discussed this possibility in our blog post about allocating discretionary funds from the previous quarter. After further discussing this possibility with HKI, our understanding is that (a) the amount of funding needed to fill this gap will likely be small relative to the amount of GiveWell-directed funding that HKI currently holds, and (b) we will have limited additional information in time for this decision round that we could use to compare this new use of funding to HKI’s other planned uses of funding. We will continue discussing this opportunity with HKI and may allocate funding to it in the future. Our current expectation is that we will ask HKI to make the tradeoff between allocating the GiveWell-directed funding it holds to this new opportunity and continuing to hold the funds. Holding the funds gives the current programs more runway (originally designed to fund three years) and gives HKI more flexibility to fund highly cost-effective, unanticipated opportunities in the future. We believe that HKI is currently in a better position to assess cost-effectiveness of the opportunities it has than we are, while we will seek to maximize cost-effectiveness in the longer run by assessing HKI’s track record of cost-effectiveness and comparing that to the cost-effectiveness of other top charities.

We remain open to the possibility that HKI will share information with us that will lead us to conclude that this new opportunity is a better use of funds than our current recommendation of 70 percent to AMF and 30 percent to SCI. In that case, we would allocate funds from the next quarter to fill this funding gap (and could accelerate the timeline on that decision if it were helpful to HKI).

Evidence Action’s Deworm the World Initiative for funding gaps in India and Nigeria

We spoke with Deworm the World about two new funding gaps it has due to unexpected costs in its existing programs in India and Nigeria.

In India, the cost overruns total $166,000. Deworm the World has the option of drawing down a reserve of $5.5 million (from funds donated on GiveWell’s recommendation). The reserve was intended to backstop funds that were expected but not fully confirmed from another funder. Given the small size of the gap relative to the available reserves, our preference is for Deworm the World to use that funding and for us to consider recommending further reserves as part of our end-of-year review of our top charities’ room for more funding.

In Nigeria, there is a funding gap of $1.7 million in the states that Deworm the World is currently operating in. Previous budgets assumed annual treatment for all children, and Deworm the World has since become aware of the existence of areas where worm prevalence is high enough that twice per year treatment is recommended. Our best guess is that AMF and SCI are more cost-effective than Deworm the World’s Nigeria program (see discussion in this post). It is possible that because additional funding would go to support additional treatments in states where programs already operate, the cost to deliver these marginal treatments would be lower. We don’t currently have enough data to analyze whether that would significantly change the cost-effectiveness in this case.

Deworm the World also continues to have a funding gap for expansion to other states in Nigeria. We wrote about this opportunity in our previous post on allocating discretionary funding.

Malaria Consortium for seasonal malaria chemoprevention (SMC)

We continue to see a case for directing additional funding to Malaria Consortium for SMC, as we did last quarter. Our views on this program have not changed. For further discussion, see our previous post on allocating discretionary funding.

What is our recommendation to donors?

We continue to recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we are continuing to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact. The reasons for this recommendation are the same as in our previous post on allocating discretionary funding.

The post Allocation of discretionary funds from Q1 2018 appeared first on The GiveWell Blog.

Natalie Crispin

Allocation of discretionary funds from Q4 2017

5 years 11 months ago

In the fourth quarter of 2017, we received $5.6 million in funding for making grants at our discretion. In this post we discuss:

  • The decision to allocate the $5.6 million to the Schistosomiasis Control Initiative (SCI).
  • Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.

We noted in November that we would use funds received for making grants at our discretion to fill the next highest priority funding gaps among our top charities. We also noted that our best guess at the time was that we would give 70 percent to the Against Malaria Foundation (AMF) and 30 percent to SCI.

Based on information received since November, described below, we allocated the $5.6 million to SCI, rather than dividing these funds between AMF and SCI, as previously expected. GiveWell’s Executive Director, Elie Hassenfeld, the fund advisor on the Effective Altruism Fund for Global Health and Development, also recommended that the fund grant out the $1.5 million that it held to SCI.

Update on AMF

AMF has been somewhat slower to make commitments to fund distributions of insecticide-treated nets than we expected and our best guess is that its currently available funding will be sufficient to fund all distributions that it is likely to commit to before our next major round of funding allocations in November. Notwithstanding that fact, we continue to believe that AMF has room for more funding. Additional funds would reduce the risk that AMF’s progress will be slowed if it is able to sign several major agreements in the next few months, which, while somewhat unlikely in our estimation, remains a possibility.

We wrote in November 2017:

Progress at signing new agreements was slow in 2017, leaving AMF with a large amount of funds on hand. We attribute this to the fact that countries spent much of 2017 applying for Global Fund funding and decisions about how much funding would be allocated to LLIN distributions for 2018-2020 and what the funding gaps would be for LLINs were being finalized in many countries as of October 2017. AMF noted that it did not commit to funding distributions earlier in part because GiveWell had asked AMF not to make funding commitments until the size of funding gaps were known.

Our expectation had been that the last couple months of 2017 and first months of 2018 would be a period in which AMF would commit a significant portion of its available funding to help fill these gaps because we expected countries to have more visibility into their funding gaps following finalization of Global Fund commitments around October 2017. This has not been the case. AMF recently told us that most of the countries that it was in discussions with did not have visibility into their funding gaps until December 2017, and in some cases it has taken longer than that. In making the decision regarding the fourth quarter discretionary funds, we relied on a document from AMF detailing its signed and potential agreements as of early February. The document noted that AMF had committed to one new distribution since October, in Ghana in 2018. This distribution will cost about $8 million. (We have since learned that AMF has also committed to additional distributions in Papua New Guinea in 2019 and 2020, costing $5.2 million and signed in November 2017, and in Malawi in 2018, costing $10.1 million and signed in mid February.)

AMF’s pipeline of potential future distributions includes both repeat distributions with partners and in countries it has worked with in the past and distributions with new potential partners. AMF has decided to move somewhat slowly with both types of partners. In the case of repeat partners, for several distributions, AMF is waiting to verify that the partner is able to deliver all requested data from distributions that took place in 2017 (and the monitoring that follows each distribution) before agreeing to fund the next round of nets to be delivered in 2020. These decisions seem very reasonable to us, but do result in a short-term decrease in the amount of funding we expect AMF to be able to absorb. When it is ready to do so, AMF could potentially commit up to $50 million to distributions in this category. For the largest potential new partnership that AMF is considering, there are some concerns about in-country capacity and AMF expects to to commit to a smaller-scale distribution (with an estimated cost of $5 million) with the partner and assess the results of that distribution before committing to a larger-scale distribution. AMF is also considering two additional opportunities to commit $5 to $7 million each to distributions with new partners. It could potentially commit tens of millions of dollars to one or more of these countries in future rounds if the initial engagements go well. AMF is also in several early stage conversations about potential distributions with new partners.

According to the document that we relied on for this decision, AMF held $64 million in uncommitted funds, of which $15 million was set aside for “agreement imminent” distributions, leaving $49 million “available to allocate.” Accounting for the additional agreements for Papua New Guinea and Malawi noted above, we estimate that AMF had $49 million in uncommitted funds and $45 million available to allocate as of late February.

The combination of somewhat slower progress in signing distributions than expected and our updated understanding of AMF’s pipeline led us to conclude that AMF continues to have room for more funding, but that SCI’s funding needs were more urgent. Our best guess was that the $5.6 million from GiveWell discretionary funds and $1.5 million from the Effective Altruism Fund would have a greater impact if allocated to SCI.

Update on SCI

In November, we recommended that donors give 30 percent to SCI because SCI had additional room for more funding to sustain its work in its current countries of operation and would need to scale down without additional funding. SCI recently confirmed to us that it would need to cut budgets if it did not receive additional funds before setting its annual budget for April 2018 to March 2019 in March 2018. With AMF having a less urgent funding need than previously expected, we concluded that the best use of the fourth quarter discretionary funds would be to allocate them to SCI.

It is also the case that in the last few months of 2017 SCI received less funding than we projected, both from donors influenced by GiveWell’s research and other donors.

We believe that SCI will continue to have room for more funding after the two grants totaling about $7 million. Recently, SCI sent us an early version of a budget for its 2018-19 budget year. It includes funding requests from each country program, estimates of country program requests in cases where the country has not yet submitted a request, and estimates of SCI spending on central costs and research costs. We estimate that, assuming the same budget in each of the next three years, SCI’s funding gap for that period, after receiving the grants discussed above, is about $9 million. SCI could likely absorb funding beyond that level, as the budget does not include opportunities it has to expand to additional countries. It also assumes that SCI’s other major funders will continue their support at the same level, and some of this funding may be in doubt. We note that about 13 percent of treatments that would be delivered at this scale would be for adults (discussion of this here).

Other possibilities that we decided against

Helen Keller International (HKI) for stopgap funding in one additional country

In December, Good Ventures, on GiveWell’s recommendation, provided HKI with funding for vitamin A supplementation (VAS) programs in Burkina Faso, Mali, and Guinea. Since then, HKI has learned about an unanticipated funding gap for VAS in another country. As a result, a planned VAS distribution in September may not reach national scale and/or may not include deworming (as is common for VAS campaigns). We are in ongoing conversations with HKI about either HKI allocating some of the Good Ventures funding to this country, or GiveWell providing additional funding to cover the gap. We plan to consider this funding opportunity when allocating discretionary funds from the first quarter of 2018. We expect to hold more than enough in discretionary funds (received in the first quarter of 2018) to fill the potential gap and HKI has told us that more information about the gap will be available in time for that decision. (We grant out funds from the previous quarter about two months after the end of that quarter, after we have fully checked the accuracy of our data and the size of grants).

Evidence Action’s Deworm the World for Nigeria

The grant that Good Ventures made to Evidence Action for Deworm the World in December 2017, based on our recommendation, did not include sufficient funds to fund expansion of Deworm the World’s work in Nigeria. Deworm the World sought funding for this work and we prioritized other charities’ funding gaps ahead of this work because we modeled the cost-effectiveness of this work as being lower. We noted in November, “its planned work in Nigeria is around three times as cost-effective as cash transfers (though this estimate is based on low-quality information).” We continue to think that AMF and SCI’s marginal uses of funding are likely more cost-effective than Deworm the World’s potential work in Nigeria, but this conclusion is highly dependent on a model that incorporates many highly uncertain values.

Malaria Consortium for seasonal malaria chemoprevention (SMC)

Our recommendation of Malaria Consortium has resulted in about $30 million in funding for its SMC program since November; however, we believe that there will still be a large funding gap for the program over the next three years. We decided against providing additional funding to Malaria Consortium at this time because of worries about increasing our already very large bet on a program that’s relatively new to us. We are not opposed to increasing this funding level in the future but on balance believe that granting additional funds to SCI is a stronger option at current levels. We’d also note that we’d expect additional funding at this time to go to funding SMC in 2019 and beyond (given the time needed to order drugs and plan programs for the 2018 SMC season) and there is some uncertainty as to the size of the funding gap for SMC in 2019. The program is in a scale-up phase globally and other major funders may increase their contributions to SMC starting in 2019.

What is our recommendation to donors?

We continue to recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we are continuing to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.

As part of the process we went through to decide where to allocate these funds, we also discussed whether we should update our recommendation for donors who prefer to give directly to our top charities. We ultimately decided that because updating that recommended allocation is a difficult and time-consuming process because of the additional research and internal discussions involved and because, relatively speaking, few dollars follow this recommendation outside of giving season, we plan to update that allocation only once each year (in November) unless we believe our previously recommended allocation is clearly suboptimal.

In this case, we believe that the $7 million in grants to SCI roughly brings the situation back in line with where it was in November, with AMF and SCI having the next most impactful funding gaps and it being difficult to distinguish on the margin between the quality of AMF and SCI’s funding gaps. SCI has better modeled cost-effectiveness, while AMF appears to be better on several qualitative factors, including monitoring of program performance.

The post Allocation of discretionary funds from Q4 2017 appeared first on The GiveWell Blog.

Natalie Crispin

Revisiting the evidence on malaria eradication in the Americas

6 years 2 months ago
Summary
  • Two of GiveWell’s top charities fight malaria in sub-Saharan Africa.
  • GiveWell’s valuations of these charities place some weight on research by Hoyt Bleakley on the impacts of malaria eradication efforts in the American South in the 1920s and in Brazil, Colombia, and Mexico in the 1950s.
  • I reviewed the Bleakley study and mostly support its key findings: the campaigns to eradicate malaria from Brazil, Colombia, and Mexico, and perhaps the American South as well, were followed by accelerated income gains for people whose childhood exposure to the disease was reduced. The timing of these events is compatible with the theory that rolling back malaria increased prosperity. Full write-up here.
Introduction

I blogged three weeks ago about having reviewed and reanalyzed Hoyt Bleakley’s study of the effort in the 1910s to rid the American South of hookworm disease (not malaria). That study, published in 2007, seems to show that the children who benefited from the campaign attended school more and went on to earn more as adults.

For GiveWell, Bleakley’s 2010 study is to malaria parasites as his 2007 study is to intestinal worms. Like the 2007 paper, the 2010 one looks back at large-scale, 20th-century eradication campaigns in order to estimate impacts on schooling and adult income. It too produces encouraging results. And it has influenced GiveWell’s recommendations of certain charities—the Against Malaria Foundation and Malaria Consortium’s seasonal malaria chemoprevention program.

Because GiveWell had already invested in replicating and reanalyzing Bleakley (2007), and because the two papers overlap in data and method, I decided to do the same for Bleakley (2010). And here the parallel between the two papers breaks down: having run the evidence through my analytical sieve, my confidence that eradicating malaria boosted income is substantially higher than my confidence that eradicating hookworm did. I’m a bit less sure that it did so in the United States than in Brazil, Colombia, and Mexico; but the Latin American experience is probably more relevant for the places in which our recommended charities work.

This post will walk through the results. For details, see the new working paper. Because my malaria reanalysis shares so much with the hookworm one, I have written this post as if you read the last one. If you haven’t, please do that now.

How the malaria analysis differs from the hookworm one

Having just emphasized the commonality between Bleakley’s hookworm and malaria eradication studies—and my reanalyses thereof—in order to orient you, I should explain how the two differ:

  • The hookworm study is set exclusively in the American South, while the malaria study looks at efforts in four countries. In the United States in the 1920s, no doubt inspired by the previous decade’s success against hookworm, the Rockefeller Foundation and the U.S. Public Health Service promoted a large-scale program to drain swamps and spray larvicides, which cut malaria mortality in the South by 60%. Then in the 1950s, with the discovery of DDT, the World Health Organization led a worldwide campaign against the disease. Partly because of data availability, Bleakley (2010) studies the consequences in Brazil, Colombia, and Mexico.1Bleakley (2010) also chose these countries because they had malarial and non-malarial regions, allowing comparisons. See Bleakley (2010), note 6. For sample maps see this. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
  • Where the hookworm study groups data two ways—first by place of residence to study short-term effects, then by place of birth to study long-term effects—the malaria study does only the latter.
  • I pre-registered my analysis plan for the malaria study with the Open Science Framework and hewed to it. While I did not allow the plan to bind my actions, it serves to disclose which analytical tactics I settled on before I touched the data and could know what results they would produce.2Actually we registered a plan for the hookworm study too, but the malaria plan was better informed—and better followed—precisely because it came on the heels of the similar hookworm reanalysis. For brevity, I skipped this theme in my blog post. I did write about it in the hookworm working paper. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
  • The Bleakley malaria paper appeared in a journal published by the American Economic Association (AEA), which requires its authors to post data and computer code on the AEA website. This aided replication and reanalysis. Unfortunately, as appears to be the norm among AEA journals, the Bleakley (2010) data and code only reproduce the paper’s tables, not the graphs that in this case I see as central.
  • For Brazil, Colombia, and Mexico, I mostly relied on that publicly posted data for the crucial information on which regions within a country had the most malaria, rather than trying to construct those variables from old maps and books in Spanish and Portuguese. I also relied on the public data for geographic control variables. I think it can be valuable to go back to primary sources, but for the time being at least, this step looked too time-consuming. I did update and expand the Latin outcome data, on such things as literacy and income, because it is already conveniently digitized in IPUMS International. And I reconstructed all the U.S. data from primary sources, simply by copying what we assembled for the hookworm reanalysis.
Results

In showing you what I found, I’ll follow nearly the same narrative as in my previous post’s section on the “long-term impact on earnings.” To start, here is a key graph from the Bleakley (2010) paper—or really four graphs. In each country’s graph, as with the hookworm graphs, each dot shows the association between historical disease burden in a state (or municipio) and the average income in adulthood of people born there in a given year. In all but Colombia, the leftmost dots line up with the negative range on the vertical axis, meaning that, initially, coming from a historically malarial area stunted one’s income. For example, some of the early U.S. dots are around –0.1 on the vertical axis, which means that being native to swampy Mississippi instead of arid Wyoming cut one’s adult earnings by about 10%.3For cross-country comparability, Bleakley (2010) normalizes the malaria mortality and ecology indexes so that the 5th- and 95th-percentile geographic units—Wyoming and Mississippi in the U.S. case—score 0 and 1. Income proxies are taken in logs. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); The dots later rise, suggesting that the liability of coming from malarial areas faded, and even reversed. In Colombia, the dots start around zero but also then rise.

As in the hookworm study, here, Bleakley (2010) superimposes on the dots the step-like contour representing how malaria eradication is expected to play out in the data. The steps reach their full height when the campaigns are taken to have started—1920 in the United States and 1957 in the Latin nations. All babies born after these points were alike in that they grew up fully in the post–eradication campaign world. The step contours begin their rises 18 years earlier, when the first babies were born who would benefit from eradication at least a bit by their 18th birthdays.4These graphs incorporate all of Bleakley’s control variables. In my hookworm post, I began both results sections with “basic” graphs that did not include all the controls, imitating Bleakley (2007). In contrast, all the Bleakley (2010) graphs incorporate full controls. So I do the same. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Next is my closest replication of the key Bleakley (2010) graphs. These use Bleakley’s data, as posted, but not Bleakley’s computer code, since that was not posted:

The next version adds the latest rounds of census data from the Latin nations and the newer, larger samples from old census rounds for the United States. It also redefines childhood as lasting 21 instead of 18 years, because I discovered that the Bleakley (2010) code uses 18 but the text uses 21. That budges the first dashed lines back by three years:

I avoided superimposing step contours on these data points because I worried that it would trick the brain into thinking that the contours fit the data better than they do. But whether the step contour fits the plots above is exactly what you should ask yourself now. Does it seem as if the dots rise, or rise more, between each pair of vertical, dashed lines? I could see the answer being “yes” for all but Mexico. And that could be a fingerprint of malaria eradication.

I ask that question more formally in the next quartet, fitting line segments to successive ranges of the data. The dots in the four graphs are the same as above, but I’ve taken away the grey confidence intervals for readability. The p values in the lower-left of each pane speak to whether any upward or downward bends at the allowed kink points are statistically significant, i.e., hard to ascribe to chance alone. Where the p values are low—and they mostly are, even in Mexico—they favor the Bleakley (2010) reading that rolling back malaria raised incomes.

In Brazil, Colombia, and Mexico, this statistical test is fairly confident that red lines bend upward at the first kinks (p = 0.00 for Brazil and Colombia and 0.07 for Mexico). That is: in high-malaria areas, relative to low-malaria areas, as the first babies were born who could benefit in childhood from eradication, future incomes rose. The test is less confident for the United States, where the first allowed kink, in 1899, gets a high-ish p value of 0.39. However, the U.S. trend clearly bends upward—just earlier than predicted by the Bleakley (2010) theory. That might mean that the Bleakley (2010) theory is slightly wrong: maybe when it came to impacts on future earnings, malaria exposure continued to matter into one’s twenties, at least in the United States 100 years ago. Then, people born in the South even a bit before 1899 (the date of the first U.S. kink point) would have benefited from the eventual campaign against malaria; and that first kink should be moved to the left, where it would match the data better and produce a lower p value. Or perhaps that high p value of 0.39 signifies that the Bleakley (2010) model is completely wrong for the United States, and that forces other than malaria eradication drove the South’s catch-up on income.

Now, in addition to the four measures of income studied above–one for each country—the Bleakley (2010) paper looks at eight other outcomes. Six are literacy and years of schooling completed, tracked separately in Brazil, Colombia, and Mexico. In addition, there is, for Brazil, earned income—as distinct from total income (“earned” meaning earned through work). And there is, for the United States, Duncan’s Socioeconomic Index (SEI), which blends the occupational income score, explained in my last post, with information about a person’s education level. Your Duncan’s SEI is highest if you hold what is typically a high-paying job (as with the occupational income score) and you have a lot of education.

The first public version of the Bleakley study makes graphs for the additional eight outcomes too. But the final, journal-published version drops them, perhaps to save space. Since for me, the graphs are so central, I generated my own graphs for the other eight outcomes:

These figures hand us a mixed bag. In the United States, the trend on Duncan’s index appears to bend as predicted at the first allowed kink (p = 0.04) but not the second. Seemingly, relative income gains continued in the South well after malaria eradication could cause them. In Brazil, while relative progress on earned income slows when expected (second kink, p = 0.04), it does not appear to accelerate when expected (first kink), perhaps owing to small samples in the early years. In none of the Latin countries does relative progress on adult literacy or years of schooling slow with much statistical significance at the expected time (second kink points in bottom six graphs). The trend bends in all six at the first kink point, and with statistical significance—but the wrong way in Mexico.

In fact, the mixed bag partly corroborates Bleakley (2010), which also questions whether rolling back malaria increased schooling. The new results depart from Bleakley (2010) in also questioning the benefit for literacy. And they cast some doubt on the income impact in the United States. In both the U.S. plots—in the upper-left of the last two sets of graphs above—it’s clear that the income gap between the South and the rest narrowed over many decades. It’s less clear that it did so with a rhythm attributable to the malaria eradication effort of the 1920s.

Conclusion

For me, this reanalysis triggers a modest update to my understanding of the impacts of malaria prevention. With regard to adult income in Latin America, and perhaps the United States, the Bleakley (2010) theory withstands reexamination. It holds up less well for literacy, but this is not very surprising given that Bleakley (2010) also did not find clear impacts on schooling.

I wouldn’t say that my confirmation proves that malaria eradication campaigns in the Americas boosted income in the way that a large-scale randomized study might. But then neither, if you read him closely, does Bleakley. Rather, the evidence “indicates” impact. The theory that malaria eradication in the Americas increased earnings fits pretty well to the data we have. And that is probably about as much certainty as we can expect from this historical analysis.

Much of the data and code for this study are here (2 GB). Because of IPUMS licensing limitations, the download leaves out the census data for Brazil, Colombia, and Mexico. The included “read me” file explains how to obtain this data. The full write-up is here.

Notes   [ + ]

1. ↑ Bleakley (2010) also chose these countries because they had malarial and non-malarial regions, allowing comparisons. See Bleakley (2010), note 6. For sample maps see this. 2. ↑ Actually we registered a plan for the hookworm study too, but the malaria plan was better informed—and better followed—precisely because it came on the heels of the similar hookworm reanalysis. For brevity, I skipped this theme in my blog post. I did write about it in the hookworm working paper. 3. ↑ For cross-country comparability, Bleakley (2010) normalizes the malaria mortality and ecology indexes so that the 5th- and 95th-percentile geographic units—Wyoming and Mississippi in the U.S. case—score 0 and 1. Income proxies are taken in logs. 4. ↑ These graphs incorporate all of Bleakley’s control variables. In my hookworm post, I began both results sections with “basic” graphs that did not include all the controls, imitating Bleakley (2007). In contrast, all the Bleakley (2010) graphs incorporate full controls. So I do the same. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Revisiting the evidence on malaria eradication in the Americas appeared first on The GiveWell Blog.

David Roodman

Our top charities for giving season 2017

6 years 3 months ago

This year, we added two new top charities, Evidence Action’s No Lean Season program and Helen Keller International’s vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action’s Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Our top charities and recommendations for donors, in brief

Top charities

We now have nine top charities. They are:

Standout charities

We also provide a list of standout charities. We believe they are implementing programs that are evidence-backed and may be extremely cost-effective. However, we do not feel as confident in the impact of these organizations as we do in our top charities.

Conference call to discuss recommendations

We are planning to hold a conference call at 1:30pm ET/10:30am PT on Thursday, November 30 to discuss our charity recommendations and answer your questions.

If you’d like to join the call, please register using this online form. If you can’t make this date, but would be interested in joining another call at a later date, please indicate this on the registration form.

Additional details and explanation

Below, we provide:

  • An explanation of changes to our recommended charity list and of major changes to our review process in the past year that are not specific to any one organization. More
  • A discussion of our approach to determining how much funding charities can use effectively (“room for more funding”) and our ranking of charities’ funding gaps. More
  • Reasoning behind how we have ranked charities’ funding gaps. More
  • Details about each of our new top charities, including an overview of what we know about their work and our understanding of their funding needs. More
  • Details about each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year, and our understanding of their funding needs. More
  • A brief overview of each of our standout charities. More
  • The process we followed that led to these recommendations. More
  • An update on giving to support GiveWell’s operations versus giving to our top charities. More
Major changes in the last 12 months

Major changes to our recommended charities list and review process over the past year include:

  • Overall, we believe our top charities are able to absorb more funding than they could in previous years. This is a result both of recent additions to the top charities list with large funding gaps (particularly Malaria Consortium) as well as expansion by top charities that have been on the list for a longer time (particularly Deworm the World and AMF).

    We expect overall “room for more funding” to continue to expand as we gain more confidence in recently-added top charities and continue to add new top charities, particularly through GiveWell Incubation Grants, our program to grow the pipeline of potential future top charities and improve our understanding of our current top charities.

  • We added two new programs to our list of top charities: vitamin A supplementation (VAS) and seasonal migration subsidies. We have not previously recommended charities that work on these programs.

    We had considered VAS a priority program for a number of years but had not found an organization that was able to answer our key questions. While we have some remaining questions, we can now make a strong case for supporting HKI’s work on VAS.

    We initially supported No Lean Season through GiveWell’s Incubation Grants program. No Lean Season is the first organization we have added to our top charity list through our Incubation Grants program.

  • Last year, the charities we recommended on the margin were estimated to be about three times as cost-effective as unconditional cash transfers, the program implemented by top charity GiveDirectly. This year, we believe that the charities we are recommending on the margin are about six times as cost-effective as cash transfers. For the most part, this change was due to (a) a series of small adjustments to our cost-effectiveness model and (b) changes in which individuals contribute to the model and the values entered into the model by these and other contributors.

    We now feel fairly confident that there will be large amounts of room for more funding in this range. As more time has passed without identifying opportunities that are considerably more cost-effective than this, we have become more pessimistic about finding such opportunities. Our current best guess is that, if they exist, they will be in the area of policy advocacy in developing countries, on issues like lead regulation and tobacco taxation. We intend to do further research in those areas.

  • We made a significant change to our cost-effectiveness analysis to more formally incorporate adjustments for the way in which our top charities’ funding affects funding from other sources by (a) attracting more resources to the programs they work on (e.g., governments contributing staff time to support implementation of the programs) or (b) displacing resources that would have otherwise supported the programs. We will be writing more about this in a future post.

  • We continued to analyze the complex evidence base for deworming (treating intestinal parasites), the program implemented by four out of our nine top charities.

    At the end of 2016, David Roodman, a Senior Advisor to GiveWell, conducted a detailed review of the core evidence underlying our deworming recommendation (blog posts here and here).

    This year, we saw new follow-up results on the main study that leads us to recommend deworming, which continued to show similar long-term impacts of deworming on adult earnings as were estimated previously.

    Further investigation and updates based on new data led us to believe that two deworming studies (Croke 2014 and Bleakley 2007) no longer provide substantial support for the theory that deworming has long-term impacts. We plan to write more about this in the future. All together, this work led us to the same conclusion about deworming: that it is a reasonable bet to take based on its strong cost-effectiveness (which incorporates our uncertainty about the impact).

Room for more funding analysis

Types of funding gaps

In the last two years, we used a framework of “capacity funding” and “execution levels” to compare funding gaps (unfilled funding needs) across charities. This framework was intended to capture whether funding would enable a charity to expand or grow in important ways and how likely it was, in our estimation, that each top charity would be constrained by funding in the next year.

We developed this approach in response to a situation where we expected to direct more funding to several of our top charities than they would be able to use (commit or spend) in that year. We used capacity funding to describe opportunities to increase the amount of funding a charity might be able to absorb in the future (by, say, investing in expanding to a new location) and execution levels to describe the likelihood, down to the 5 percent level, that a charity would be able to make use of additional funding before encountering non-funding bottlenecks to their work.

This year, because we have added new top charities and most of our other top charities have more room for more funding than in previous years, we expect that the funding we will direct to each organization will not reach the level where they will encounter significant non-funding bottlenecks. As a result, we have moved away from describing capacity funding and execution levels.

Ranking funding gaps

The first million dollars to a charity can have a very different impact from the 20th million dollars. Accordingly, we have created a ranking of individual funding gaps that accounts for our best guess of the impact of additional funds at each level.

The below table lays out our ranking of funding gaps, up to $75.7 million in total funding. We expect Good Ventures to give $75 million to GiveWell’s top charities this year, so this table is our recommendation to Good Ventures, plus the allocation of funding that GiveWell holds to allocate at its discretion (currently $0.7 million). We then discuss our recommendation for all other donors.

The Open Philanthropy Project, which was incubated at GiveWell but is now a separate organization, plans to write more soon about the reasons for Good Ventures increasing its support of GiveWell top charities from $50 million last year to $75 million this year. In short, the amount was based on discussions about how to allocate funding across time and across cause areas. It was not set based on the total size of top charities’ funding gaps or the projection of what others would give.

Charity Description Amount (millions) All top charities Incentive grants: $2.5 million per charity 22.5 All standout charities Standout grants: $100,000 per charity 0.7 Deworm the World Funding gaps in India and Kenya over the next three years (including central costs) 3.0 Helen Keller International Funding gaps over three years in Burkina Faso, Mali, and Guinea—countries that have missed recent vitamin A campaigns due to lack of funding 4.7 No Lean Season Full funding gap over three years for implementing the program in Bangladesh 9.0 Deworm the World Three years of funding for a new program in Pakistan and reserves to protect against funding shortfalls in India 10.4 Malaria Consortium Part of the funding gap for SMC in Burkina Faso, Nigeria, and Chad over the next three years 25.4

In total, we are recommending that Good Ventures make the following grants:

  • Malaria Consortium’s seasonal malaria chemoprevention program: $27.9 million
  • Evidence Action’s Deworm the World Initiative: $15.2 million. We are also recommending that GiveWell’s Board of Directors grant the $0.7 million in discretionary funds that we currently hold from the third quarter (from donors who selected to give to “Grants to recommended charities at GiveWell’s discretion” on our donation form) to Deworm the World, bringing the total to $15.9 million.
  • Evidence Action’s No Lean Season program: $11.5 million
  • Helen Keller International’s vitamin A supplementation program: $7.2 million
  • Schistosomiasis Control Initiative: $2.5 million
  • Against Malaria Foundation: $2.5 million
  • Sightsavers’ deworming program: $2.5 million
  • END Fund’s deworming program: $2.5 million
  • GiveDirectly: $2.5 million

Our recommendation to donors

For donors who are interested in directing funding to whichever recommended charity or charities GiveWell believes has the most pressing funding need at the time the funds are granted, we recommend giving to “Grants to recommended charities at GiveWell’s discretion.” These grants will respond to the greatest funding need we see; they may not match the recommended allocation outlined below.

For donors (other than Good Ventures) who are interested in donating directly to our top charities, we recommend splitting your donation as follows:

  • 70 percent to the Against Malaria Foundation
  • 30 percent to the Schistosomiasis Control Initiative
Why these recommendations?

Our recommendations to donors, including Good Ventures, are based on:

  1. Overall cost-effectiveness of the charity. Our cost-effectiveness model is a key input into our decision-making process, and large differences in modeled cost-effectiveness impact our recommendations. We try not to put significant weight on relatively small differences in cost-effectiveness according to the model because many inputs are highly uncertain.

    Our model this year found relatively small differences between many top charities, with Deworm the World at ~12 times as cost-effective as cash transfers, four top charities in the ~6-10x cash transfers range, and three top charities in the ~3-5x cash transfers range. We consider differences between charities implementing the same intervention or interventions that have similar inputs and output in the model more meaningful (e.g., malaria nets and seasonal malaria chemoprevention) than differences between charities implementing quite different interventions.

    We have completed a sensitivity analysis of our cost-effectiveness analysis to get a better sense for which parameters are most sensitive. We are more hesitant to consider differences in the cost-effectiveness as meaningful when they rely on very sensitive inputs.

  2. Cost-effectiveness of particular funding opportunities. Charities’ work can vary significantly in cost-effectiveness across locations due to different costs, disease burdens, uptake in the targeted population, or probability that other funders would step in in GiveWell’s absence. While not a part of our formal cost-effectiveness model, we ran supplementary analyses of cost-effectiveness for some locations for which our top charities were seeking additional funding and considered the output as part of our prioritization of funding gaps.
  3. Qualitative factors not captured in our cost-effectiveness model. The main factors we focused on were:
    • Proportion of the global funding need for the program that is filled. We expect that funders will generally (but imperfectly) select the areas where cost-effectiveness is higher first, leaving the areas with higher costs, lower disease burden, lower cultural acceptance of the program, etc. for last. We believe we have captured some of the consequences of this in our cost-effectiveness analysis. For example, we use national level disease burden estimates for the countries in which each charity has worked and/or plans to work; charities working in higher burden countries are therefore modelled as more cost-effective. But we do not use sub-national estimates to distinguish the highest priority regions within a country; if charities are filling the lowest priority funding gaps within a county, they will likely be less cost-effective than our model suggests. This was an important consideration in comparing AMF and Malaria Consortium. We estimate that ~80 percent of the global funding need for nets (the program AMF implements) has been filled, and ~35 percent of the global funding need for seasonal malaria chemoprevention (the program Malaria Consortium implements).
    • Our level of knowledge about the organization. We have recommended AMF, Deworm the World, SCI, and GiveDirectly for many years. We know less about Malaria Consortium and No Lean Season and the least about HKI. We seek to be somewhat conservative about recommending large amounts of funding to organizations where there is a relatively high chance that additional research could lead us to believe the program was less cost-effective than we previously thought.
    • Ease of communication with the organization. It is important to us that we are able to learn over time about the charities we recommend, to enable us to improve our decisions. The ability to communicate effectively with an organization is a key factor in our ability to learn from the organization’s experiences.
    • Ongoing monitoring and likelihood of detecting future problems. Evaluating an organization’s monitoring processes and results is an important part of our charity reviews and for the most part is not captured in our cost-effectiveness analysis. As with ease of communication, we have more confidence in recommending funds to an organization if we believe that we will learn about how successful its work has been.

Summary of key considerations for top charities

The table below summarizes the key considerations for our nine top charities. More detail is provided below, as well as in the charity reviews.

Estimated cost-effectiveness (relative to cash transfers) Our level of knowledge about the organization Primary benefits of the intervention Ease of communication Ongoing monitoring and likelihood of detecting future problems Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation Other major considerations AMF ~6x High Deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars High proportion (~80%) of global gap for program is filled Malaria Consortium (SMC program) ~7x Moderate Under-5 deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars Relatively low proportion (~35%) of global gap for program is filled Helen Keller International (VAS program) ~9x Moderate Under-5 deaths averted Strong Moderate High: could absorb tens of millions of dollars Learning benefits Deworm the World ~12x High Possible increased income in adulthood Strong Strong Moderate: could absorb millions of dollars END Fund (deworming program) ~4x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars SCI ~10x High Possible increased income in adulthood Moderate Moderate High: could absorb tens of millions of dollars Sightsavers (deworming program) ~5x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars No Lean Season ~5x Moderate Immediate increase in consumption Strong Moderate Low: further funding would be used for different types of activities Potential upside GiveDirectly Baseline High Immediate increase in consumption and assets Strong Strong Very high: could absorb over 100 million dollars

Reasons for this funding gap ranking

Prioritization of funding that we have recommended to Good Ventures (we recommend Good Ventures fill the highest-priority funding needs first, to ensure these are funded):

  1. We start by recommending that each top charity receive $2.5 million as an “incentive grant.” These grants are intended to be a major contribution to the charity’s work in recognition of the fact that they have met GiveWell’s criteria and have dedicated significant time to working with us to help us follow their progress and plans each year. We don’t want our top charity funding process to be winner-takes-all because we believe that charities would be less likely to want to participate in that case.
  2. After incentive grants, we believe the next most valuable funding to provide is for Deworm the World’s work in Kenya and India over the next three years. Deworm the World’s work in Kenya and India is the most cost-effective opportunity we have found. We estimate that its work in Kenya is ~20x as cost-effective as cash transfers and in India is ~30x+ as cost-effective as cash transfers.
  3. We rank providing funding to our two new top charities, Helen Keller International (HKI)’s VAS program and No Lean Season, next.

    We estimate that HKI could use $7.2 million over three years to support VAS campaigns in countries with high child mortality rates that have recently missed campaigns due to lack of funds. HKI’s cost-effectiveness is at the high end of the range for top charities (~9x cash transfers). We believe HKI could absorb more than $7.2 million in additional funding for VAS effectively but that this $7.2 million gap is likely more cost-effective than HKI’s average cost-effectiveness. Also, because HKI is a new top charity of ours, we expect this first part of its gap to have significant learning benefits for us: by giving this money, we’ll be better positioned to follow HKI’s work and review its monitoring, which we believe will make it more likely that we have a more accurate estimate of its impact in future years.

    We decided to recommend funding all of No Lean Season’s funding gap in Bangladesh for the next three years. While No Lean Season’s cost-effectiveness is at the lower end of our top charities (~5x cash transfers), we see additional reasons to prioritize this gap. We believe No Lean Season is the top charity where there is the strongest case to be made for “upside”; our cost-effectiveness analysis may not capture the potential impact of scaling a new program that could lead to greater visibility and funding for a novel type of program.

  4. We think the next highest priority funding to provide is $10.4 million to Deworm the World. This funding would support a new program in Pakistan and provide reserve funding for programs supported with restricted funds. We estimate that the program in Pakistan will be roughly ~7x as cost-effective as cash transfers, though this estimate is very sensitive to estimates of worm burdens in the locations where Deworm the World plans to work.

    The reserve funding is intended to make it unlikely that the India program, which we believe is very highly cost-effective, will be interrupted—Deworm the World relies on restricted funding for this program and there is some chance that this funding will not be available in the future. It may use this GiveWell-directed funding for other opportunities if it is not needed to backstop restricted funding in India; we expect that it will have unfunded opportunities remaining in the next few years, particularly in Nigeria.

  5. The last funding gap on our list of recommendations for Good Ventures is $23.6 million to Malaria Consortium for its work on SMC. When choosing which gap to recommend for the remainder of Good Ventures’ $75 million, we focused on the remaining funding needs for Malaria Consortium’s SMC program, AMF, and SCI, which we believe to have the next highest-value gaps. Our cost-effectiveness model indicates that SCI is the most cost-effective of these three organizations (~10x cash transfers, compared with ~6-7x cash transfers for AMF and Malaria Consortium), but when the difference in modelled cost-effectiveness between two charities is relatively small, we also put significant weight on qualitative factors. We believe that AMF and Malaria Consortium are stronger on some qualitative factors, particularly the likelihood that we will be able to learn about the programs’ performance through the monitoring they conduct. Between AMF and Malaria Consortium, we have prioritized Malaria Consortium’s funding gap primarily due to the qualitative considerations discussed above around the proportion of the global funding need that is filled. After following Malaria Consortium for a second year, we believe that Malaria Consortium and AMF are comparable on other major qualitative factors, such as quality of ongoing monitoring and likelihood of detecting future problems.

    The total amount we are recommending for Malaria Consortium’s SMC program represents a rough compromise between providing a high level of funding to a program that we prefer to the next funding gap on the list and not wanting to make too large of a bet on an organization that we have less experience with than some other top charities.

Prioritization for non-Good Ventures donors:

  1. Our current recommendation for donors is to give to GiveWell for making grants to top charities at our discretion. Our goal is for SCI to receive $9 million, in addition to the $2.5 million incentive grant that we are recommending to Good Ventures, and AMF to receive the remainder of expected GiveWell-directed funding because AMF and SCI represent the next highest-value funding opportunities we see. Giving us funding to grant at our discretion allows GiveWell to better target this allocation, and to adapt if we learn new information about pressing, high-value funding needs at our top charities.
  2. For donors who prefer to give directly to charities, we recommend giving 70 percent to AMF and 30 percent to SCI. These percentages are our best guess of what will achieve our target allocation given our projections of total donations driven by our recommendations.

    This allocation comes from a belief that, at these margins, it is difficult to distinguish between the quality of AMF and SCI’s funding gaps. SCI has better modeled cost-effectiveness, while AMF appears to be better on several qualitative factors, including monitoring of program performance. We have roughly targeted a two-to-one ratio between the two.

Details on new top charities

Helen Keller International (HKI) for work on vitamin A supplementation

Our full review of HKI’s work on vitamin A supplementation is here.

Overview

HKI (http://www.hki.org/) is a large organization with multiple programs focused on reducing malnutrition and averting blindness and poor vision. Our review focuses on HKI’s work on vitamin A supplementation (VAS) and our recommendation is specific to its VAS program. HKI provides technical assistance, engages in advocacy, and contributes funding to government-run VAS programs.

There is strong evidence from many randomized controlled trials (RCTs) conducted in the 1980s and 1990s that VAS can substantially reduce child mortality, but weaker evidence on how effective VAS is in the places HKI would work with additional funding in the next few years. In particular, there is little available information on current rates of vitamin A deficiency in areas where HKI works. We have adjusted our cost-effectiveness analysis for our best guess of how much less effective VAS is today (~25 percent as effective as in the trials in the 1980s and 1990s); the intervention remains cost-effective with that adjustment.

We feel that the monitoring data that we have seen from HKI’s programs gives us limited information on HKI’s past performance, but demonstrates the types of data HKI is able to collect on program performance. We have requested that HKI collect this monitoring data of all programs funded with GiveWell-directed funds.

Overall, we have not yet investigated HKI at the same level of depth as some of our other top charities, which we have recommended for several years. We have reviewed documents from HKI, had a number of conversations with their staff, and spent three days meeting with HKI and observing a VAS campaign in Guinea. We have remaining questions about HKI’s work that we will seek more information on in the future, but overall we believe this program is, like our other top charities, an excellent giving opportunity.

Funding gap

We believe that HKI’s VAS work is highly likely to be constrained by funding next year. HKI has provided details of VAS programs that it could support with additional funding of up to about $41.4 million in 2018-2020. HKI appears to have limited prospects for funding these programs from other sources.

Our understanding is that with additional funds, HKI would cause additional rounds of VAS to occur in some countries, while in other countries, HKI primarily aims to increase coverage rates in rounds of VAS that would take place regardless of its involvement. We have asked HKI to prioritize use of GiveWell-directed funding in countries where it expects to cause additional rounds of VAS to occur. HKI’s funding gap for countries that have recently missed VAS campaigns due to lack of funds is $7.2 million.

HKI’s VAS work was supported by the Canadian government in the past. That funding ended in 2016 and has not been renewed. Over the past year, several VAS campaigns have been skipped in countries HKI previously supported.

Evidence Action’s No Lean Season program

Our full review of No Lean Season is here.

Overview

No Lean Season (https://www.evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the season of income and food insecurity (‘lean season’) between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on a household member stating their intention to migrate to urban or other rural locations to seek short-term employment.

Several randomized controlled trials (RCTs) of subsidies to increase migration provide moderately strong evidence that such an intervention increases household income and consumption during the lean season. An additional RCT is ongoing. We estimate that No Lean Season is roughly five times as cost-effective as cash transfers (see our cost-effectiveness analysis).

Evidence Action has shared some details of its plans for monitoring No Lean Season in the future, but, as many of these plans have not been fully implemented, we have seen limited results. Therefore, there is some uncertainty as to whether No Lean Season will produce the data required to give us confidence that loans are appropriately targeted and reach their intended recipients in full; that recipients are not pressured into accepting loans; and that participants successfully migrate, find work, and are not exposed to major physical and other risks while migrating.

Funding gap

We expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Evidence Action is seeking funding beyond this level to allow it to build reserves for No Lean Season.

Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net (LLIN) distributions for protection against malaria in developing countries. AMF has conducted post-distribution surveys of all completed distributions to determine whether LLINs have reached their intended destinations and how long they remain in good condition. AMF’s post-distribution surveys have generally found positive results (with some exceptions); we believe they have some methodological limitations.

We estimate that AMF’s program is roughly six times as cost-effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

Prior to this year, we had seen results from AMF’s “post-distribution check ups” (PDCUs) from two countries, Malawi and the Democratic Republic of the Congo, and had significant uncertainties about the methodology used in each location. We have now also seen results from Ghana. We have more confidence in our understanding of AMF’s PDCUs than we did previously, though this work is ongoing. In particular, we commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings (see links). Further discussion of the strengths and weaknesses of PDCUs here.

In 2017, AMF signed relatively few new agreements to fund LLIN distributions and, as a result, has a balance of $58 million in uncommitted funds, or $35 million if distributions where AMF believes agreements are imminent are counted as committed. Our understanding is that many of AMF’s conversations with countries could not progress until decisions were made about how much Global Fund funding each country would allocate to LLIN distributions (as opposed to other malaria control efforts). This decision-making process extended into late 2017. Global Fund funding is allocated on three-year cycles and we do not expect this to continue to be a bottleneck for AMF in 2018.

Funding gap

We believe that AMF is very likely to be constrained by lack of funding. There is high uncertainty in the maximum amount of funding that AMF could use productively, though we expect the maximum to be much greater than what AMF is likely to receive. To fund all of the distributions that it is currently in detailed discussions about, AMF would need $50 million more than we project it will receive. The total funding gap for LLINs for 2018-2020 appears to be hundreds of millions of dollars.

With additional funding, AMF’s top priorities would be to fund a portion of the next round of distributions, in 2018-2020, in each of the countries in which it has recently funded distributions.

END Fund (for work on deworming)

Our full review of the END Fund’s work on deworming is here.

Background

The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming.

Slightly more than half of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list.

We have seen limited monitoring results on the number of children reached in END Fund-supported programs. In 2016, the END Fund began requiring that surveys be conducted to determine whether its programs have reached a large proportion of children targeted; we have seen coverage surveys for (a non-random sample of) 35 percent of its 2016 deworming grant portfolio. These studies leave us with some remaining questions about the program’s impact.

Important changes in the last 12 months

We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas that the END Fund works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. We also saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible.

Funding gap

We believe the END Fund could substantially increase its deworming grantmaking with additional funds. We roughly estimate that there is gap of $18 million between the amount of funding the END Fund will have available for grants for deworming and the amount of funding it would need to make all of the potential grants it has identified. Sources of major uncertainty in this estimate include whether the END Fund will encounter non-funding bottlenecks in some of its identified and early-stage opportunities, the amount of funding it will receive from other sources, the proportion of funding it will allocate to deworming, and costs other than grants.

Evidence Action’s Deworm the World Initiative

Our full review of Deworm the World is here.

Background

Evidence Action’s Deworm the World (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates deworming programs. Its main countries of operation are India, Kenya, and Nigeria, and it is considering expanding to Pakistan.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World overall is the most cost-effective charity we have found. We estimate that it is ~12 times as cost-effective as cash transfers, but note that, due to differences in worm burdens and costs across countries, there is significant variation in cost-effectiveness across the countries in which it works. We estimate that its work to date in India has been more than 30 times as cost-effective as cash transfers, while its planned work in Nigeria is around three times as cost-effective as cash transfers (though this estimate is based on low-quality information).

Important changes in the last 12 months

We estimate that Deworm the World could absorb considerably more funding this year than we estimated last year, due to opportunities it has identified to expand its geographic reach. (More in the next section.)

The quality of the monitoring that we have seen from Deworm the World has remained high. To date, we have seen limited monitoring from Nigeria, which is a new addition to Deworm the World’s portfolio and is expected to become a major portion of its work in the future. This is of minor concern given the strong monitoring track record elsewhere and how new the program is in Nigeria.

Funding gap

We believe that Deworm the World is very likely to be constrained by funding. We expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive over the next three years. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities.

With additional funding, Deworm the World would sustain its current work in Kenya and India, and would seek to expand its work in Nigeria and India to additional states and support the government in Pakistan to initiate a deworming program.

GiveDirectly

Our full review of GiveDirectly is here.

Background

GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82 percent overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people.

We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. We believe that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected. It continues to experiment heavily, with the aim of improving how its own cash transfer programs are run as well as those of governments. It has recently started work on a universal basic income trial and has started partnering with major funders on evaluations of cash transfers in new geographies with the aim of influencing the broader international aid sector to use its funding more cost-effectively.

We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost-effective than cash.

Important changes in the last 12 months

We had previously expressed reservations about GiveDirectly’s targeting strategy: that by excluding the least poor households in each village, the program might lead to negative reactions by non-recipients, increase costs per household reached, and exclude households that were still quite poor. In 2017, GiveDirectly largely switched to a “saturation” approach of making transfers to all households in selected villages. It will continue to use a targeted approach in Rwanda, where government regulations require such an approach, but the saturation approach will be used in Kenya and Uganda.

In 2016, GiveDirectly built up its operations in Uganda and Kenya with the anticipation of revenue growth in 2017. Revenue growth has been slower than expected and GiveDirectly had to lay off some staff as a result.

GiveDirectly launched its universal basic income project this month.

In 2015, Good Ventures made a grant of $25 million to GiveDirectly on GiveWell’s recommendation. GiveDirectly’s goals for the grant were to expand its ability to raise funds from donors not influenced by GiveWell’s recommendation and to collaborate with large aid institutions or governments to address their questions about cash transfers. We expect to write more about the performance of the grant in the future, but, in short, our impression is that fundraising has progressed slower than expected and collaborative projects have progressed more quickly than expected.

Funding gap

We believe that GiveDirectly is highly likely to be constrained by funding next year. It expects to use additional funding primarily for standard cash transfers and for additional collaborative projects. For collaborative projects, GiveDirectly’s potential partners require it to contribute funding, which the partner matches (at a one-to-one ratio, minimum). These projects would largely be in countries GiveDirectly has not worked in before and many are at an early stage of discussion. We estimate that GiveDirectly could use more than $200 million in additional funding in 2018-2019.

Malaria Consortium (for work on seasonal malaria chemoprevention)

Our full review of Malaria Consortium’s seasonal malaria chemoprevention program is here.

Background

Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria.

There is strong evidence that SMC substantially reduces cases of malaria. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately statistically powered to find an impact on mortality.

Malaria Consortium and its partners have conducted studies in all of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, though past surveys have been conducted after four rounds of SMC (SMC is given in a maximum of four treatment courses at monthly intervals) and may be subject to error due to the inaccurate recall or recordkeeping. Starting in 2017, Malaria Consortium is conducting coverage surveys after each round of SMC, to reduce recall error.

Important changes in the last 12 months

We have increased our confidence in Malaria Consortium’s monitoring, though we have not yet seen all of the research that Malaria Consortium expected to share in 2017 (in particular, tracking of malaria cases and deaths over time in areas where Malaria Consortium works). Coverage survey results from 2016 were generally positive, with a couple of outliers. The change from conducting coverage surveys after four treatment cycles to conducting them after each cycle will increase our confidence in the results.

Last year, we had only a rough estimate of how much additional funding Malaria Consortium could use productively. We have significantly improved our understanding of its room for more funding this year.

Funding gap

We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. It appears that there is a large remaining global need for additional funding for SMC programs and that Malaria Consortium is well-positioned to fill these gaps, if it has sufficient funding to do so.

Malaria Consortium estimates that it could spend $28-30 million per year on SMC in each of the next three years and that this level of funding would largely fill the global funding gap for SMC, with the exception of Nigeria, where the scale of the gap would be beyond Malaria Consortium’s operational capacity in the short term.

It appears to have limited prospects for major funding from other sources. The major grant for Malaria Consortium’s work on SMC previously, from Unitaid, is ending and Malaria Consortium told us that it will not be renewed.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify partner countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies to determine whether its programs have reached a large proportion of children targeted. These studies cover (a non-random sample of) about 40 percent of treatments SCI reports having delivered over the past few years. The studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is less cost-effective than Deworm the World and more cost-effective than Sightsavers and the END Fund. Given the uncertainty in our cost-effectiveness model, we are hesitant to say that SCI is more cost-effective than AMF and Malaria Consortium, though taken literally, SCI is 1.5 times as cost-effective as AMF and Malaria Consortium (~10x cash transfers vs. ~6-7x cash transfers).

Important changes in the last 12 months

We continued to follow SCI’s progress in 2017 and there have not been many major changes to its work. As in the past, SCI shared monitoring of deworming coverage levels for a portion of its programs with us; there continue to be several SCI-supported countries for which we have not seen monitoring results. In the past, we have noted that we had low confidence in the accuracy of the financial information that SCI provided and that SCI made significant improvements to its financial systems in 2016; our remaining concerns about SCI’s financial management and reporting are fairly minor.

In 2017, SCI allocated nearly all available funding to programs in its 2017-2018 budget year. This was a large increase in spending over the previous budget year ($9.6 million in 2016-2017 compared with $22.5 million in 2017-2018), driven in large part by a large increase in GiveWell-directed funding ($3.7 million in 2015 compared with $16.6 million in 2016). We believe this decision was due in part to a miscommunication with GiveWell—in a conversation with SCI in early 2017, we recommended that they treat the funds like a multi-year grant because of the risk of large fluctuations in GiveWell-directed funding, but we did not emphasize this point. SCI told us that it plans to allocate future funding over multiple years, noting that its funding allocation decisions in 2016-2017 were due to the desire to avoid allowing drugs to expire as well as a misunderstanding with GiveWell about how the funding was intended to be used.

Funding gap

We estimate that SCI could productively use about $30 million more than it expects to receive to deliver treatments to school-aged children over the next three years. It could use almost three times this amount if it were to follow World Health Organization guidelines, which include treating many adults; we are not recommending funding to treat adults because we haven’t seen sufficient evidence on the impact of treating adults.

The primary use of this funding, and SCI’s top priority, would be to sustain and expand work in current countries of operation. A smaller portion would be used to expand to up to four additional countries.

Sightsavers (for work on deworming)

Our full review of Sightsavers is here.

Background

Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and – more specifically – advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections.

Sightsavers has shared surveys for some of its past NTD programs that measure whether these programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact. We have seen very limited results from Sightsavers’ deworming programs specifically. For GiveWell-supported programs, Sightsavers has told us it will conduct coverage surveys for each round of deworming; we have reviewed one of those surveys to date.

Important changes in the last 12 months

In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results.

We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years.

Funding gap

We believe that Sightsavers’ deworming work is likely to be constrained by funding next year. Sightsavers has provided details of deworming programs that it could fund with additional funding of up to about $6.4 million in 2018 and 2019. Sightsavers appears to have limited prospects for funding these programs from other sources. We believe it is likely that Sightsavers could absorb funding beyond this amount to extend programs to 2020 and/or seek out additional opportunities to fund deworming programs.

Of the $6.4 million, $2.8 million would be used to add deworming to existing NTD programs and $3.7 million would be used to fund NTD programs that would treat several NTDs in addition to schistosomiasis and STH. We will request that Sightsavers prioritize the first set of opportunities, because we believe they will likely be more cost-effective.

Standout charities

In addition to our top charities, we recognize standout charities—organizations that support programs that may be extremely cost-effective and are evidence-backed but for which we have less confidence in their impact than we do for our top charities. We have reviewed their work and feel these groups stand out from the vast majority of organizations we have considered in terms of the evidence base for the program they support, their transparency, and their potential cost-effectiveness. These organizations offer additional giving options for donors who feel highly aligned with their work.

We’ve added one organization to the list this year: Evidence Action’s Dispensers for Safe Water.

We don’t follow standout organizations as closely as we do our top charities. We generally have one or two calls per year with representatives from each group and publish notes on our conversations. We provide brief updates on these charities below.

New addition to the standout list:

  • Evidence Action’s Dispensers for Safe Water. The Dispensers for Safe Water program provides chlorine dispensers for decontamination of drinking water to prevent diarrhea and associated deaths of young children. We believe that there is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death. Our rough cost-effectiveness analysis of Dispensers for Safe Water suggests that the program is in a similar range of cost-effectiveness as unconditional cash transfer programs. Our review of Dispensers for Safe Water is here.

Organizations that have conducted randomized controlled trials of their programs:

  • Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its child survival media campaign in Burkina Faso and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15 percent or more) and data from health facilities finding an increase in facility visits. (The results have not yet been published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. DMI is conducting an RCT of its family planning radio campaign in Burkina Faso and it is planning work on early child development in Burkina Faso and child survival in Mozambique. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI is here and notes from our most recent conversation with DMI are here.
  • Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a RCT of its program and measured a 27 percent reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of GiveDirectly. It is conducting a second RCT of its program and results are expected in 2020. Living Goods recently expanded the number of family planning products it offers and is interested in expanding to a third country. Living Goods is scaling up its program and could scale up more quickly with additional funding. Our review of Living Goods is here and notes from our most recent conversation with Living Goods are here.

Organizations working on micronutrient fortification:

We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence.

  • Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost-effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. FFI’s recent work includes advocating for legislation to mandate that rice imported to West Africa is fortified with vitamins and minerals. Our full review is here and notes from our most recent conversation are here.
  • Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. GAIN has focused its recent USI work on Tanzania, Mozambique, Ethiopia, and Kenya, which it targeted based on relatively low levels of coverage of iodized salt and strong relationships with stakeholders. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN is here and notes from our most recent conversation are here.
  • Iodine Global Network (IGN). Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-directed funding has made up a large part of its funding in recent years. It expects to have data from before and after its recent work in Madagascar, Lebanon, and possibly Israel by the end of 2018; this data may provide additional evidence of IGN’s impact. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN is here and notes from our most recent conversation here.
  • Project Healthy Children (PHC)/Sanku. PHC/Sanku aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs and by enabling fortification among small-scale millers. PHC is scaling up its Sanku project, which equips small millers with a machine that enables them to fortify their flour with micronutrients; we have not done as much formal analysis of Sanku as of PHC’s core work on advocacy and technical assistance to countries to implement fortification. PHC/Sanku expects to be constrained by funding in the future. Our review of PHC/Sanku is here and notes from our more recent conversation are here.
Our research process in 2017

We plan to detail the work we completed this year in a future post as part of our annual review process. A major focus of 2017 was improving our recommendations in future years, in particular through our work on GiveWell Incubation Grants and prioritizing promising programs for further investigation.

Below, we highlight the key research that led to our current charity recommendations. This page describes our overall process.

  • Following existing top charities. We followed the progress and plans of each of our 2016 top charities. We had several conversations by phone with each organization, met in person at least once with each top charity (including a three-day visit to Rwanda and the Democratic Republic of the Congo with the END Fund), and reviewed documents they shared with us.
  • Identifying new top charities.
    • No Lean Season. We had recommended a series of Incubation Grants to No Lean Season beginning in 2014 and have followed its progress since then. This year, due to the scale at which No Lean Season was operating and the track record it had established, we decided that the No Lean Season program was at a stage of development where we could evaluate it as a potential top charity. In addition to extensive communications with No Lean Season staff over the phone and reviewing documents they shared with us, GiveWell staff spent five days visiting the program in Bangladesh.
    • Helen Keller International’s vitamin A supplementation program. Earlier this year, Research Analyst Chelsea Tabart began reaching out to organizations that might be a fit for our criteria, but with which we had limited or no previous contact with. As a result of that process, we reconnected with Helen Keller International (which we first considered as a potential top charity in 2007) and began to consider its vitamin A supplementation program as a potential top charity. In addition to extensive communications with HKI staff over the phone and reviewing documents they shared with us, GiveWell staff spent three days meeting with HKI staff in Guinea and observing a vitamin A supplementation program.
  • Completing intervention reports on obstetric fistula surgery and measles vaccination campaigns; completing interim intervention reports on SMS reminders for vaccination, Sayana® Press (an injectable contraceptive), oral rehydration solution, and antiretroviral therapy for HIV/AIDS; and expanding our interim intervention report on seasonal malaria chemoprevention to a full intervention report.
  • Staying up to date on the research for malaria nets, cash transfers, and deworming. We did not find major new research on cash transfers, nets, or deworming that affected our recommendation of GiveDirectly, AMF, or the organizations we recommend for their work on deworming. David Roodman published an in-depth review (parts 1 and 2) of the deworming studies that form the primary basis of our views on the impact of deworming (though much of this work was completed in 2016 and informed our top charity recommendations last year).
  • Making extensive updates to our cost-effectiveness model and publishing several updates to the model over the course of the year. We instituted a process to track and report publicly on updates to the model to reduce the possibility of errors and make our process more transparent. This year, staff members have also provided substantially more detail in our cost-effectiveness file about why they have chosen particular inputs.
Giving to GiveWell vs. top charities

GiveWell is currently in a stable financial position. We project that our revenue and our expenses will be approximately equal in the future. However, this projection forecasts some growth in the level of operating support we receive.

In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations. We retain our “excess assets policy” to ensure that if we fundraise for our own operations beyond a certain level, we will grant the excess to our recommended charities.

We cap the amount of operating support we ask Good Ventures to provide to GiveWell at 20 percent, for reasons described here. We thus ask that donors who use GiveWell’s research consider the following:

  • If you have supported GiveWell’s operations in the past, we ask that you maintain your support. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and to minimize staff time spent on fundraising for our operating expenses.
  • If you have not supported GiveWell’s operations in the past, we ask that you designate 10 percent of your donation to help fund GiveWell’s operations. This can be done by selecting the option to “Add 10% to help fund GiveWell’s operations” on our credit card donation form or letting us know how you would like to designate your funding when giving another way.

We’re happy to answer questions in the comments below. Please also feel free to reach out directly with any questions.

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

Natalie Crispin

Are GiveWell’s top charities the best option for every donor?

6 years 8 months ago

We’re sometimes asked whether we think GiveWell’s top charities are the “best,” in some absolute sense of the word, or whether we’d ever advise that a donor give to an opportunity outside of our recommendations. This post aims to clarify how GiveWell thinks about different giving options and their suitability for different types of donors.

We believe that GiveWell’s top charities offer donors an outstanding opportunity to do a lot of good and are the best option for most donors. However, some donors—those with a very high degree of trust in a particular individual or organization to make this decision, donors with lots of time (in excess of 50 hours per year, and likely more) to consider their giving decision, or donors whose values point strongly toward a particular cause outside of the ones GiveWell covers—may find opportunities to have a greater impact per dollar than GiveWell’s top charities. Note that we think these characteristics are likely to be necessary, but not sufficient, for finding these types of opportunities; we still expect good giving to be hard, and spending, for example, 50 hours per year on research isn’t necessarily going to yield better opportunities.

In this post, we describe relevant considerations for donors in greater detail.

Giving to GiveWell’s top charities

GiveWell was founded to serve donors with limited amounts of time to make giving decisions. GiveWell’s co-founders, Elie Hassenfeld and Holden Karnofsky, were in this situation when they started GiveWell as a side project in 2006. They found that determining where to give effectively was a full-time project and quit their jobs to start GiveWell in 2007.

GiveWell’s top charity recommendations serve all donors. We rely on evidence and detail our rationale for making a recommendation publicly, so donors can vet our work; a strength of our recommendations is their falsifiability. We believe our top charity recommendations serve donors who want to give as effectively as possible and have only limited time to determine where to donate, and (prior to GiveWell) no trusted person or entity to outsource their thinking to, particularly well. Our criteria and recommendations were designed with this type of donor in mind:

  • Our top charities are largely uncontroversial and relatively straightforward ways to do a lot of good—for example, by providing direct aid such as insecticide-treated nets to prevent malaria and cash transfers to very poor households. There is room for debate on the evidence behind these interventions and their cost-effectiveness, but the basic case for them—and the fact that they are likely to do more good than harm—is subject to little debate, so a donor can feel fairly confident in these basics without needing to do their own research.
  • GiveWell publishes the full details of our charity analyses so that donors can review and vet our work, and so that donors with very limited time can trust that any major problems would likely be caught by others (with more time).
  • Because we lay out the entire case for the charities online, donors can spot-check any particular part of it to get a sense of whether we’re thinking reasonably about the issues that seem most salient to them.
  • Our top charities have room for more funding. In other words, we believe additional marginal donations to these organizations enable them to do more good.

Our guess is that most donors that use GiveWell fit this profile (want to give as effectively as possible and have only limited time to determine where to donate, and no other trusted person or entity to outsource their thinking to).

Below, we discuss alternative donor profiles:

(1) Donors with limited time and a high amount of trust in a person or organization to inform their giving decisions

This group of donors has limited time to spend on making a giving decision and has an organization or person (other than GiveWell or GiveWell staff) they personally trust to make or inform this decision. In this case, they may defer to that person or organization’s recommendations.

(2) Donors with lots of time

Donors with a lot of time to spend on giving decisions (50+ hours per year) may be able to find opportunities that GiveWell hasn’t. For example, a donor might know someone who is starting a charity and feel, based on their research, that supporting their project at an early stage might be a particularly leveraged way to do good. A donor with lots of time may also be very familiar with a particular cause and feel highly confident in a particular organization and its need for funding. These donors may want to compare alternative opportunities to GiveWell’s top charities. They may also want to actively vet GiveWell’s recommendations as part of their research process.

Donors with lots of time may also wish to apply a different strategy to their giving. GiveWell largely recommends charities where sufficient evidence exists to make a fairly robust estimate of the expected value of a donation. Donors with much more time to spend (maybe even significantly more than 50 hours per year) thinking about where to give may want to take a “hits-based giving” approach—having a high tolerance for philanthropic risk, so long as the overall expected value is sufficiently high. This is the approach the Open Philanthropy Project, which was incubated at GiveWell, has taken, and we believe doing this well requires a lot of work, as the Open Philanthropy Project discussed in a blog post last year (emphasis original):

Aim for deep understanding of the key issues, literatures, organizations, and people around a cause, either by putting in a great deal of work or by forming a high-trust relationship with someone else who can. If we [the Open Philanthropy Project] support projects that seem exciting and high-impact based on superficial understanding, we’re at high risk of being redundant with other funders. If we support projects that seem superficially exciting and high-impact, but aren’t being supported by others, then we risk being systematically biased toward projects that others have chosen not to support for good reasons. By contrast, we generally aim to support projects based on the excitement of trusted people who are at a world-class level of being well-informed, well-connected, and thoughtful in relevant ways.

Achieving this is challenging. It means finding people who are (or can be) maximally well-informed about issues we’ll never have the time to engage with fully, and finding ways to form high-trust relationships with them. As with many other philanthropists, our basic framework for doing this is to choose focus areas and hire staff around those focus areas. In some cases, rather than hiring someone to specialize in a particular cause, we try to ensure that we have a generalist who puts a great deal of time and thought into an area. Either way, our staff aim to become well-networked and form their own high-trust relationships with the best-informed people in the field.

I [Open Philanthropy Project Executive Director Holden Karnofsky] believe that the payoff of all of this work is the ability to identify ideas that are exciting for reasons that require unusual amounts of thought and knowledge to truly appreciate.

(3) Donors with values that differ from GiveWell staff

Donors who hold different values than the majority of GiveWell staff, or who place more weight on a particular cause outside of the causes covered by GiveWell, may find other giving opportunities to be more attractive for reasons beyond the time/trust framework articulated earlier in this post. For example, individuals who place a very high value on farm animal welfare may wish to give a large proportion of their donation, if not all of their donation, to organizations working in that cause.

We’re happy to speak with you about giving decisions.

If you’re not sure which considerations apply to you, please reach out. We’re always happy to talk through giving decisions.

The post Are GiveWell’s top charities the best option for every donor? appeared first on The GiveWell Blog.

Catherine

AMF and Population Ethics

7 years 3 months ago

One important input we consider in our top charity recommendations is our quantitative cost-effectiveness estimate of each charity: an estimate of how much good the charity accomplishes per dollar. We have written before about the many challenges of constructing and interpreting such cost-effectiveness estimates. One such challenge is the problem of how to assign a numerical “score” to various good outcomes such as averted deaths or increased incomes, so that they can all be compared on the same scale. This involves answering questions such as:

  • What would I do if I had a choice between doubling the income of three individuals in extreme poverty for a year and averting the death of a child under the age of 5?
  • How many deaths of children under the age of 5 would I need to avert to accomplish as much good as averting the death of one adult?

GiveWell staff members enter their estimates for such values-based comparisons in our cost-effectiveness analysis (CEA). Along with empirical estimates of program costs and outcomes, this forms the basis for the numerical comparisons we make between our top charities.

Recently, a post on the Effective Altruism Forum raised concerns about our recommendation of AMF specifically. It argues that the value of averting deaths for children under five depends on one’s view of population ethics – a branch of philosophy that asks questions like “Is it good to avert a death if it has no long-run impact on the total population?” – and that some approaches to population ethics would imply a substantial discount to our cost-effectiveness estimates. We’ve chosen to respond to this argument at length because we think it is interesting, serves as a good example of the thorny issues we grapple with in estimating cost-effectiveness, and gives the opportunity to explain some aspects of our 2016 cost-effectiveness model that are not widely understood.

In this post, I will

  • Explain basic concepts in population ethics and how they inform the way people think about the value of averting death
  • Summarize arguments from the post mentioned above, which argues that people with certain views on population ethics should substantially discount our cost-effectiveness estimate of the Against Malaria Foundation (AMF) to better reflect their values
  • Walk through the reasoning behind our current estimate of AMF’s cost-effectiveness and explain why I believe it’s compatible with most plausible accounts of population ethics, so discounts as aggressive as those suggested in the linked post are likely inappropriate.

Population ethics and the value of averting death

Population ethics is a branch of philosophy which outlines some major considerations that influence how people value averting deaths. It is defined in the link as “the theory of when one state of affairs is better than another, where the states of affairs may differ over the number of people who ever live.” Population ethics deals with questions such as:

  • Can it be morally good or bad to create new people who would not have otherwise lived, or is creating people always morally neutral (assuming they do not affect people who are already living)?
  • Is the badness of death lessened if someone else will be born to “replace” the person who died?
  • What makes (premature) death bad? Is it because the individual misses out on the years of happy life they would have otherwise lived? Is it because they had a strong preference to live that was violated? Is it because surviving loved ones will grieve? Is it some combination of these?
  • Is death worse at some ages than at others?

A population ethics stance is a set of beliefs that inform how to compare different states of the world by answering these and similar questions. Staff members’ implicit or explicit stance on population ethics guides the way they quantify the value of averting death in the CEA. This in turn can greatly affect how the cost-effectiveness of life-saving charities such as the Against Malaria Foundation (AMF) and Malaria Consortium’s seasonal malaria chemoprevention program compares to the cost-effectiveness of life-improving charities such as GiveDirectly and charities implementing mass deworming programs.

In a recent post on the Effective Altruism Forum, Michael Plant argues that AMF is unlikely to be highly cost-effective under four plausible population ethics stances:

  • Total hedonic utilitarianism: The stance that the best state of the world is the one with the most total happiness or fulfillment. Under total hedonic utilitarianism, averting someone’s death is only good to the extent that it results in more overall happiness experienced in the world (regardless of who experiences it). Thus, if a child dies, and as a result the family has another child they would not otherwise have had, the badness of the child’s death comes only from the gap between their death and the new child’s birth and the grief and other negative effects experienced by family members. (This is assuming the child who died and the new child who is born as a result would live similarly happy lives.) This counterintuitive result is referred to as “the replacement problem” in this post. Michael argues that a total utilitarian should substantially discount GiveWell’s cost-effectiveness estimate for AMF to account for the replacement problem.
  • The deprivation stance: According to the deprivation stance, the badness of death is equal to the number of years of life that the individual who died misses out on as a result of their death: in other words, their life expectancy at the time of death. Unlike in total hedonic utilitarianism, this badness is not lessened even if someone else is immediately born as a direct result of the death. Michael argues that this is the point of view which is most favorable to AMF, but those who hold this view should still be more interested in other interventions, such as life extension.
  • The time-relative interest stance: Quoting from Michael’s original post,

    [The time-relative interest account] holds the badness of death depends, roughly, on the extent to which it frustrates the person’s interests in continuing to live. This captures the [intuition] many people have that it’s much more important to save a 20-year old than a 1-minute old foetus because, in essence, that 1-minute old foetus hasn’t developed enough to miss out on life.

    Note: the time-relative interest stance, unlike the previous two stances on population ethics, doesn’t suggest a simple formula that would capture most of the “badness” of death if all the empirical facts were known. Michael argues that those who subscribe to the time-relative interest stance should “reduce [GiveWell’s] estimate of AMF’s effectiveness by however much [they] discount child deaths compared to adult ones.”

  • Epicureanism: This stance holds that death in itself cannot be bad for the individual who died, because as soon as they die, they cease to have morally relevant interests. Thus, a given death is only bad due to suffering in the process of death, and the grief and other negative effects it has on survivors. Epicureanism also does not immediately suggest a simple formula that would estimate the badness of death given all the empirical facts.

I believe Michael suggests discounts on our cost-effectiveness estimate of AMF that would be too aggressive for most people’s value systems. I explain this further in the rest of the post, but I want to first make a general point: if you are concerned your stance on population ethics does not align with the GiveWell median, I believe it would be better to download an editable copy of GiveWell’s CEA and input your own values for rows 7, 53, 63, and 64 on the “Parameters” sheet than to attempt to multiply GiveWell’s bottom-line cost-effectiveness estimate by some factor to account for expected differences in population ethics. This is because the cost-effectiveness model is complex and in parts unintuitive, and I don’t believe it will be straightforward to guess how your stance on population ethics differs from the implicit stance of the median GiveWell staff member.

Summary of considerations against discounting our cost-effectiveness estimate

Here are the key reasons why I believe most people should likely not reduce GiveWell’s stated cost-effectiveness estimate of AMF as much as Michael suggests:

  • GiveWell’s 2016 cost-effectiveness model suggests that the median GiveWell staff member believes that averting the death of a young child averts ~8 disability-adjusted life-years, or DALYs (“Parameters”, C14), whereas last year the GiveWell median opinion was ~35 DALYs averted, as Michael states in the post. While GiveWell does not take an official stance on population ethics as an organization, I believe this change is a result of some staff members leaning away from an explicit “discounted age-weighted expected years of life lost” model of value to a more complex and less precise model of value that does not map intuitively to the concept of DALYs as used in health economics. (More.)
  • GiveWell’s current cost-effectiveness estimate does not predict that a $3500 donation to AMF will, on expectation, prevent the death of one young child. It predicts that a $3500 donation to AMF will on expectation cause a combination of outcomes that are equivalent in value to saving the life of one young child, according to the median staff member. (More.) According to staff median estimates, the benefits of AMF are driven
    • ~37% by preventing deaths of children under the age of 5
    • ~36% by preventing deaths of people age 5 and over
    • ~27% by providing future financial benefits due to improved young-childhood development, similar to deworming charities.
  • It is not clear that the “replacement problem” described in the original post fully applies to AMF, and if it does, it’s not clear that conservative staff estimates of value (see A) don’t already account for it in the 2016 model. (More). Because of this, a total utilitarian may end up concluding that AMF is more cost-effective than we suggest. (More.)
  • I don’t believe it’s obvious than even an Epicurean would consider AMF to be dramatically less cost-effective than the GiveWell CEA suggests — although it is likely that they would discount our cost-effectiveness estimate somewhat. (More.)

I provide more detail on these four points in the rest of the post. I’m going to reference specific cell numbers and sheets in our November 2016 CEA throughout.

DALYs are a flexible, non-literal measure in our CEA

From Michael’s original post:

[GiveWell claims that saving a child’s life is worth] 35 ‘QALYs’ (Quality-Adjusted Life Years), which is [a] more technical way of saying it creates 35 years of healthy life for the beneficiary.

As mentioned above, the new median value is 8 DALYs per under-5 life saved. (See the “YLL per Death” sheet in our CEA for examples of calculations that take a more explicit deprivation or time-relative interest stance, which were more common in the past.)

More generally, in the context of our CEA, I don’t think it makes sense to treat “DALYs” as straightforwardly denoting “years of life lost by the person with the disease + years lived with disability due to the disease.” Particularly in 2016, staff members interpreted the “DALYs averted per death of an under-5 averted—AMF” parameter (“Parameters”, B63) as an opportunity to quantify how holistically “bad” the death of a young child is. This interpretation can take into account broader considerations such as:

  • For total utilitarians, concerns of replacement
  • Harm caused by parental grief
  • Potential economic harm caused by sickness and death
  • Time-relative interest considerations which weigh young children less highly than older people
  • Considerations around the loss of the child’s individual identity and the child’s desire for life

Note: I’m not sure what considerations different staff members actually did take into account; I’m just observing that our interpretation of the DALY unit is broad enough to allow for a range of such considerations.

Because DALYs have been somewhat divorced from their rigid, concrete meaning, I think comparing how a given staff member filled in row 7, row 53, row 63, and row 64 in the “Parameters” sheet (all of which are value inputs) is more informative for understanding their values than looking at any one of those inputs in isolation. For example, staff member Sophie included comments on her inputs in O7, O63, and O64.

Also because of this ambiguity, if you want to make adjustments for your beliefs on population ethics, I believe it would be better to download an editable copy of GiveWell’s CEA and input your own values for rows 7, 53, 63, and 64 in the “Parameters” sheet than to attempt to discount GiveWell’s bottom-line cost-effectiveness estimate by some factor to account for expected differences in population ethics.

In particular, I think that with certain reasonable assumptions about replacement rate (calculated here), a total utilitarian or someone with a time-relative interest account of value may end up concluding that AMF is more cost-effective than our estimates suggest (see below).

Over 60% of AMF’s expected benefits are not driven by saving young children

From Michael’s original post:

GiveWell estimate[s], although this is not to be taken too seriously, [that] $3,500 to AMF saves a child’s life.

However, the cost-effectiveness estimate for AMF produced by our staff median parameters (B74) is around $3400 per young life saved-equivalent, not $3400 per young life saved.

That is, our cost-effectiveness estimate does not predict that if you give $3400 to AMF you will, on expectation, prevent the death of one young child. It predicts that if you give $3400 to AMF, you will on expectation cause a combination of outcomes that, according to the values of the median GiveWell staff member, are morally equivalent to saving the life of one young child. Specifically, the expected benefits of AMF are split between:

  • Preventing the deaths of children under 5
  • Preventing the deaths of people 5 or older
  • Improving the expected future income of young children, similar to deworming

Note that benefits in the 2015 model were also not exclusively driven by preventing deaths of children under the age of 5: cell M20 in the “GW medians” sheet of the 2015-2016 CEA implies that the median staff member believed that one year of bed net coverage was almost as effective as one year of deworming for improving expected future income of children. The primary new addition this year is accounting for the deaths of older people.

Example calculation

This section walks through the math of how we achieved our “cost per young life saved”-equivalent figure; please consider this section especially optional.

According to our median cost-effectiveness estimates (all cell numbers taken from the “Bed Nets” sheet of our CEA):

  • For every ~$9,161 spent, on expectation one marginal death of a child under the age of 5 is averted (cell B55). Each under-5 death prevented gets a weight of one “young life equivalent” unit. According to the median GiveWell staff member, averting the death of a child under 5 averts about 8 DALYs (“Bed Nets”, B57).
  • For every ~$37,391 spent, on expectation one marginal death of a person age 5 or over is averted (multiply cells B61 and B63 to get the ratio of 5-or-over deaths averted / under-5 death averted, and then divide the cost per under-5 death above by this value). According to the median GiveWell staff member, each 5-or-over death prevented gets a weight of 4 “young life equivalent” units (“Bed Nets”, B62).
  • For every ~$500 spent, on expectation you have produced financial utility equivalent to increasing an individual’s ln(consumption) by one unit for one year (“Bed Nets”, B69, multiplied by 500)—which means ~doubling their income for a year. According to the median GiveWell staff member, averting 1 DALY is equivalent to increasing ln(consumption) by one unit for three years (“Bed Nets”, B72). Combining that with the value of 8 DALYs per death of a young child above, this means that each unit increase of ln(income) gets a weight of 1 / 24 “young life equivalent” units.

This means that a $37,391 donation to AMF would, in expectation:

  • Prevent the deaths of ~4.08 children under the age of 5, i.e. ~4.08 young-life equivalent units
  • Prevent the death of ~1 person over the age of 5, i.e. ~4 young-life equivalent units.
  • Have a financial benefit equivalent to increasing the ln(income) of 37.391 * 2 = 74.782 people by one unit, i.e. 3.12 young-life equivalent units.

Putting that together we have $37,391 / (4.08 + 4 + 3.12) = $3338 per young life saved-equivalent. (Note: this is not exactly equivalent to the value in “Results”, R4, likely because of a combination of rounding and small adjustments that are not accounted for here.)

Do parents have additional children to replace those who die at a young age?

In 2014, GiveWell commissioned David Roodman to write a report on the possible causal link between mortality and fertility, which is linked in Michael’s post:

By GiveWell’s own estimates, the effect of AMF is that it leaves total population numbers largely unchanged. I call this the ‘replacement problem’ for total utilitarians because, in these replacement cases, they can’t say there’s much (or any) value in saving lives apart from the effects of bereavement on the parents.

However, I think the picture presented in David Roodman’s analysis is more nuanced than this. From the conclusion of that report:

As mentioned at the outset, we should expect that where fertility is most controlled, typically indicated by total fertility of about 2 births/woman or less, that the volitional replacement effect is large—that for every child’s life saved, parents avert one birth. That births/woman averaged 2.7 in developing countries as a whole in 2005–10, and that the number has probably fallen more since, suggest that most couples today are engaging in family planning. Meanwhile, where the fertility transition does not yet appear to have occurred the replacement effect is likely much smaller. The studies I find most informative tend to corroborate this theory, indicating near-full replacement among a group of relatively affluent countries; partial replacement in a context where fertility had begun to decline but still had far to go (Uttar Pradesh); and no replacement in an area of continuing high fertility (Northern Ghana).

I spent a little bit of time trying to come up with a reasonable range of estimates for the fertility replacement rate in the areas AMF operates in, primarily informed by whether it seems like those areas are converging to a fertility rate of 2 births per woman. This should definitely not be considered the final word on the complicated topic of replacement rates, nor should it be considered an official GiveWell estimate. However, I found it to be helpful as a personal exercise, and it may be valuable to some people to see my reasoning:

According to GiveWell’s latest update on top charities, marginal funds from GiveWell-influenced donors will go toward AMF’s Execution Level 1 gap. We believe at Execution Level 1, AMF will likely use marginal funds to do more work in the places it had worked previously: primarily Malawi, Ghana, and Uganda. According to this tool, estimates for fertility 2010-2015 for these countries were:

  • ~5.25 lifetime births per woman in Malawi
  • ~5.91 births per woman in Uganda
  • ~4.25 births per woman in Ghana

(To get the above estimates, I chose the following options in the linked tool, in order: “Total fertility (children per woman)”, “Malawi, Ghana, Uganda”, “2010-2015”.)

I also calculated the average fertility rate across these three countries (weighted by population) to be approximately 5.2 in this spreadsheet (see sheet “AMF Countries”).

Furthermore, according to the United Nations’ probabilistic predictions as shown in this tool, it seems none of these three countries are likely to converge to two births per woman until past 2050. Based on David’s analysis, this would imply that family planning is not pervasive in the countries where AMF will likely operate over the next couple of years, and so the death of a child would lead to significantly less than one additional expected birth in the family. After a very quick scan of the table in the Conclusions section of the mortality-fertility report, one study struck out as potentially most informative for estimating replacement rates in such countries: Bhalotra and van Soest 2008.

Bhalotra and van Soest 2008 was a study in Uttar Pradesh, India, which used data from 1963-1999 to estimate that the death of a child under one month old is followed by 0.37-0.52 extra births. In the “India” sheet of my spreadsheet, I calculate that average fertility over the period of 1960-2000 was approximately 4.78. Assuming fertility in Uttar Pradesh 1963-1999 was similar, it seems the average fertility today in Ghana, Malawi, and Uganda is slightly higher than in the population studied in Bhalotra and van Soest 2008 (5.2 vs 4.78). This implies the expected replacement rate should potentially be even lower than 0.37-0.52.

I want to emphasize that these calculations are extremely rough, but they support my general impression that it is not clear that replacement is close to 1 in countries where AMF is likely to use its marginal funds in 2016.

A total utilitarian may consider AMF more cost-effective than GiveWell does

From Michael’s original post:

I should note it’s not particularly important what the exact replacement ratio is. If it turns out AMF causes parents to have 0.5 fewer children for every 1 life it saves, the total utilitarian should still [halve] AMF’s effectiveness.

This doesn’t seem obvious to me; there are a couple of flavors of total utilitarian I could imagine, and in general they don’t seem to assign a substantially lower value of DALYs averted per death of a young child averted than the GiveWell median, even assuming a 75% replacement rate (which is significantly higher than my best guess). The values used below come from the results of modifying the following values on the “YLL per death” sheet of our CEA: “Discount rate” (C6), “Age-weight parameter beta” (C7), and life expectancy at age 5 (I8 and J8).

  • You could count additional years of life created without discounting over time or weighting by age. After the deprivation stance, this is perhaps the view of population ethics that is most favorable to AMF. Assuming a life expectancy of ~65 years at age 5, preventing the death of a five-year-old in Malawi, Ghana, or Uganda would avert 65 * 0.25 = ~16 DALYs if there is a 75% replacement rate.
  • You could discount by time and weight by age. This is the sub-type of the total utilitarian viewpoint that is least favorable to AMF. Suppose the “Discount rate” is set to 0.03 and the “Age-weight parameter beta” is set to 0.04 (giving the highest weight to a year of experience as a 25 year old and a lower weight to a year of experience as a young child). If life expectancy at age five is still 65, then causing an additional five-year-old to exist would produce the equivalent ~36 years of healthy life for a 25 year old. Adjusting for replacement, preventing the death of a five-year-old in Malawi, Ghana, or Uganda would avert 36 * 0.25 = 9 DALYs, similar to the GiveWell median of 8.
  • You could have an in-between view (for example, weighting by age without discounting or vice versa), which would produce in-between estimates for DALYs per young death prevented.

It seems that because GiveWell staff members’ median estimates for the value of averting the death of a young child are already relatively conservative from a pure total utilitarian standpoint, it may be too aggressive for most total utilitarians to discount further due to the replacement problem, unless they believe replacement rates to be close to 1.

I believe broadly similar considerations would apply to certain kinds of time-relative interest viewpoints.

An Epicurean may still believe AMF produces substantial value per dollar

From Michael’s original post:

The fourth option is the Epicurean view, named after Greek philosopher Epicurus. It holds that there’s nothing good about creating someone and that death doesn’t harm anyone: once someone is dead, there is no them for anything to be bad. Obviously the process of dying can be painful. The point Epicureans make is that nothing is good or bad for you once you’re dead. On this account, the badness of death consists only in the suffering felt by the living.

For Epicureans, the value of their $3,500 donation to AMF is that it stops a family from having to grieve for a lost child.

I think there are significant costs for survivors beyond the emotional cost of grief associated with the deaths prevented by AMF:

  • For the deaths of children under the age of 5: if parents “replace” their lost child by having another, we must include in the DALY burden estimate:
    • The monetary costs of having a new baby and raising that baby to the age the deceased child was at the time of death, which may significantly impact the consumption or savings of a poor family
    • The strain on the mother’s health and productivity associated with the course of a normal pregnancy
    • The expected harm due to the possibility of serious complications in pregnancy — if this results in the mother’s death, it could result in serious permanent harm to her surviving dependents (see the next bullet point)

    The above costs are higher the higher you believe the replacement rate is.

  • For the deaths of people over the age of 5, particularly if they are parents or heads of households, we must account for:
    • The loss of the productive income they provide to the family
    • Other harm caused to dependents due to the loss of their care and guidance — it’s plausible this is very long-lasting
  • For all malaria deaths, we must account for the costs of seeking medical care to attempt to prevent the death and the costs of funeral rites, both of which might be a large strain on a poor family’s income

In the comments section of the original post, Michael also suggests that it’s implausible that grief alone could impose a significant welfare cost here:

[Replacement] doesn’t say anything about the parents. Total utils should account for that too, but note how much of the value of the intervention replacement removes. You thought you were giving a child 35+ years of life and preventing parental suffering, but now you’re just (in effect) doing the [latter]. If parental suffering is equivalent to taking away 1 year of happy life away from each parent (IMO, v unlikely), then AMF is equivalent to 2 happy years rather than 37+.

I run through some calculations here http://www.plantinghappiness.co.uk/the-questionable-importance-of-saving-lives/

It’s not clear to me that the non-tangible costs to the parents should be assigned a value of less than 1 DALY total, particularly under the broad and flexible conception of DALYs that is used in the GiveWell CEA. I think there are two plausible ways we could try to quantify this from the parents’ perspective:

  • Preference utilitarianism: How many years of their lives would parents trade away to prevent their young child from dying?
  • Hedonic utilitarianism: How long do parents grieve after the death of their young child, and how intensely on average do they experience this grief?

If you have a preference utilitarian theory of value, then it seems plausible that averting the death of a child could be equivalent to averting multiple DALYs to a parent. One parent has replied to the linked comment saying they would likely trade multiple years of their life to prevent the death of their two-year-old child, and this doesn’t strike me as a very unusual sentiment. Parents have also been known to sacrifice their life or take large risks to save their child.

However, Michael’s theory of value appears to be hedonic utilitarianism, as explained in this comment:

I’m thinking hedonically and am leaning on the literature on hedonic adaptation….few [life] events have a long term impact on happiness, either positive or negative.

I am not familiar enough with the literature on bereavement, subjective well-being and hedonic adaptation to have an informed view on how long parents typically grieve for the death of a young child, or a good sense of how intense the subjective experience of grief is. I find it plausible that the negative effects on parents’ subjective well-being could be relatively moderate and short-lived, and I also find it plausible that these effects could be extreme and long-lasting.

My colleague Luke Muehlhauser studied the literature on subjective well-being several months ago, so I asked him for his impressions. He replied:

It’s hard to say. First, the strongest designs used for studies of subjective well-being (SWB) and life events are panel studies (for a review see Luhmann et al. 2012), which makes causal inference quite tricky, even given recent econometric innovations. Second, the outcome measures typically used in SWB studies are not as well-validated as (e.g.) patient-reported outcomes used in health care (PROMIS) or the measures typically used in educational testing, especially for use across cultures and over long periods of time (as in studies of SWB and life events).

That said, if we cross our fingers and hope that the available panel studies are very roughly capturing what’s going on, we can make some guesses. I haven’t seen panel studies on SWB and the loss of a child, but perhaps we should expect the SWB effects to be similar as with the loss of a spouse, or perhaps somewhat smaller than that, especially in areas with a high rate of under-5 mortality. The Luhmann et al. meta-analysis of prospective panel studies on SWB and the loss of a spouse says (p. 605), roughly, that loss of a spouse is indeed quite bad for SWB initially, that pre-event levels of “cognitive well-being” (cognitively-assessed life satisfaction) are typically achieved within a couple years, and that adaptation is surprisingly rapid for “affective well-being” (feelings of happiness/sadness), with pre-event levels achieved within a couple months. So loss of a spouse is bad, but (according to Luhmann et al.) less bad than (e.g.) unemployment. That said, I should add that I don’t personally trust these underlying studies, nor Luhmann et al.’s method of combining them.

All told, I would guess an Epicurean would likely choose a lower value for “DALYs averted per death of an under-5 averted” than the GiveWell median of 8 (“Parameters”, C63), but I am unsure whether it would be a dramatic downward adjustment, particularly if the Epicurean places relatively more weight on parents’ stated or revealed preferences compared to their subjective experiences, or if they are relatively skeptical of academic research on subjective well-being. For example, my value of 4 DALYs per under-5 death averted (“Parameters”, E63) seems within the realm of plausibility for an Epicurean. This along with my other inputs suggests that I should believe AMF is approximately as cost-effective as GiveDirectly (“Results”, D12).

Conclusion

If you have strong beliefs about population ethics, and are interested in donating to organizations serving the global poor that meet our criteria, I think it would be valuable to download an editable copy of our CEA, override cells C7, C53, C63, and C64 in the “Parameters” sheet with your own values, and then view column R in the “Results” sheet to see what charity cost-effectiveness estimates and rankings that would imply. I’ve outlined some considerations that may apply to total utilitarians and Epicureans above.

If you don’t have a strong stance on population ethics and are wondering what the original critique should imply about whether your giving decisions, two main things are worth keeping in mind:

  • According to the median GiveWell staff member, over half of the benefits of AMF are driven by saving the life of people aged 5 and older or by improving future incomes for young children, rather than saving the lives of children under 5 (“Bed Nets”, B78-80). If you otherwise agreed with the median staff member’s values but believed that averting a young child’s death averts ~0 DALYs, AMF’s cost-effectiveness would be reduced ~37%. If you believed that averting death in general has ~0 value but agreed with the median staff member’s empirical and moral beliefs about improving future incomes, AMF’s cost-effectiveness would be reduced ~73%.
  • The median staff member’s estimate of 8 DALYs averted per young death averted appears to be within the realm of plausibility for multiple stances on population ethics, including total utilitarianism (where it seems to be more on the low end) and Epicureanism (where it seems to be on the high end). I would guess that it is also within the range of many interpretations of the time-relative interest theory of value.

The post AMF and Population Ethics appeared first on The GiveWell Blog.

Ajeya Cotra

Our updated top charities for giving season 2016

7 years 3 months ago

We have refreshed our top charity rankings and recommendations. We now have seven top charities: our four top charities from last year and three new additions. We have also added two new organizations to our list of charities that we think deserve special recognition (previously called “standout” charities).

Instead of ranking organizations, we rank funding gaps, which take into account both charities’ overall quality and cost-effectiveness and what more funding would enable them to do. We also account for our expectation that Good Ventures, a foundation we work closely with, will provide significant support to our top charities ($50 million in total). Our recommendation to donors is based on the relative value of remaining gaps once Good Ventures’ expected giving is taken into account. We believe that the remaining funding gaps offer donors outstanding opportunities to accomplish good with their donations.

Our top charities and recommendations for donors, in brief

Top charities

We are continuing to recommend the four top charities we did last year and have added three new top charities:

  1. Against Malaria Foundation (AMF)
  2. Schistosomiasis Control Initiative (SCI)
  3. END Fund for work on deworming (added this year)
  4. Malaria Consortium for work on seasonal malaria chemoprevention (added this year)
  5. Sightsavers for work on deworming (added this year)
  6. Deworm the World Initiative, led by Evidence Action
  7. GiveDirectly

We have ranked our top charities based on what we see as the value of filling their remaining funding gaps. We do not feel a particular need for individuals to divide their allocation across all of the charities, since we are expecting Good Ventures will provide significant support to each. For those seeking our recommended allocation, we recommend giving 75% to the Against Malaria Foundation and 25% to the Schistosomiasis Control Initiative, which we believe to have the most valuable unfilled funding gaps.

Our recommendation takes into account the amount of funding we think Good Ventures will grant to our top charities, as well as accounting for charities’ existing cash on hand, and expected fundraising (before gifts from donors who follow our recommendations). We recommend charities according to how much good additional donations (beyond these sources of funds) can do.

Other Charities Worthy of Special Recognition

As with last year, we also provide a list of charities that we believe are worthy of recognition, though not at the same level (in terms of likely good accomplished per dollar) as our top charities (we previously called these organizations “standouts”). They are not ranked, and are listed in alphabetical order.

Below, we provide:

  • An explanation of major changes in the past year that are not specific to any one charity. More
  • A discussion of our approach to room for more funding and our ranking of charities’ funding gaps. More
  • Summary of key considerations for top charities. More
  • Detail on each of our new top charities, including an overview of what we know about their work and our understanding of each organization’s room for more funding. More
  • Detail on each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year and our understanding of each organization’s room for more funding. More
  • The process we followed that led to these recommendations. More
  • A brief update on giving to support GiveWell’s operations vs. giving to our top charities. More


Conference call to discuss recommendations

We are planning to hold a conference call at 5:30pm ET/2:30pm PT on Thursday, December 1 to discuss our recommendations and answer questions.

If you’d like to join the call, please register using this online form. If you can’t make this date but would be interested in joining another call at a later date, please indicate this on the registration form.

Major changes in the last 12 months

Below, we summarize the major causes of changes to our recommendations (since last year).

Most important changes in the last year:

  • We engaged with more new potential top charities this year than we have in several years (including both inviting organizations to participate in our process and responding to organizations that reached out to us). This work led to three additional top charities. We believe our new top charities are outstanding giving opportunities, though we note that we are relatively less confident in these organizations than in our other top charities—we have followed each of the top charities we are continuing to recommend for five or more years and have only began following the new organizations in the last year or two.
  • Overall, our top charities have more room for more funding than they did last year. We now believe that AMF, SCI, Deworm the World, and GiveDirectly have strong track records of scaling their programs. Our new top charities add additional room for more funding and we believe that the END Fund and Malaria Consortium, in particular, could absorb large amounts of funding in the next year. We expect some high-value opportunities to go unfilled this year.
  • Last year, we wrote about the tradeoff between Good Ventures accomplishing more short-term good by filling GiveWell’s top charities’ funding gaps and the long-term good of saving money for other opportunities (as well as the good of not crowding out other donors, who, by nature of their smaller scale of giving, may have fewer strong opportunities). Due to the growth of the Open Philanthropy Project this year and its increased expectation of the size and value of the opportunities it may have in the future, we expect Good Ventures to set a budget of $50 million for its contributions to GiveWell top charities. The Open Philanthropy Project plans to write more about this in a future post on its blog.

Room for more funding analysis


Types of funding gaps

We’ve previously outlined how we categorize charities’ funding gaps into incentives, capacity-relevant funding, and execution levels 1, 2, and 3. In short:

  • Incentive funding: We seek to ensure that each top charity receives a significant amount of funding (and to a lesser extent, that charities worthy of special recognition receive funding as well). We think this is important for long-run incentives to encourage other organizations to seek to meet these criteria. This year, we are increasing the top charity incentive from $1 million to $2.5 million.
  • Capacity-relevant funding: Funding that we believe has the potential to create a significantly better giving opportunity in the future. With one exception, we don’t believe that any of our top charities have capacity-relevant gaps this year. We have designated the first $2 million of Sightsavers’ room for more funding as capacity-relevant because seeing results from a small number of Sightsavers deworming programs would significantly expand the evidence base for its deworming work and has the potential to lead us to want to support Sightsavers at a much higher level in the future (more).
  • Execution funding: Funding that allows charities to implement more of their core programs. We separated this funding into three levels: level 1 is the amount at which we think there is a 50% chance that the charity will be bottlenecked by funding; level 2 is a 20% chance of being bottlenecked by funding, and level 3 is a 5% chance.

Ranking funding gaps

The first million dollars to a charity can have a very different impact from, e.g., the 20th millionth dollar. Accordingly, we have created a ranking of individual funding gaps that accounts for both (a) the quality of the charity and the good accomplished by its program per dollar, and (b) whether a given level of funding is capacity-relevant and whether it is highly or only marginally likely to be needed in the coming year.

The below table lays out our ranking of funding gaps. When gaps have the same “Priority,” this indicates that they are tied. When gaps are tied, we recommend filling them by giving each equal dollar amounts until one is filled, and then following the same procedure with the remaining tied gaps. See footnote for more.*

The table below includes the amount we expect Good Ventures to give to our top charities. For reasons the Open Philanthropy Project will lay out in another post, we expect that Good Ventures will cap its giving to GiveWell’s top charities this year at $50 million. We expect that Good Ventures will start with funding the highest-rated gaps and work its way down, in order to accomplish as much good as possible.

Note that we do not always place a charity’s full execution level at the same rank and in some cases rank the first portion of a given charity’s execution level ahead of the remainder. This is because many of our top charities are relatively close to each other in terms of their estimated cost-effectiveness (and thus, the value of their execution funding). For reasons we’ve written about in the past, we believe it is inappropriate to put too much weight on relatively small differences in explicit cost-effectiveness estimates. Because we expect that there are diminishing returns to funding, we would guess that the cost-effectiveness of a charity’s funding gap falls as it receives more funding.

Priority Charity Amount, in millions USD (of which, expected from Good Ventures*) Type Comment 1 Deworm the World $2.5 (all) Incentive – 1 SCI $2.5 (all) Incentive – 1 Sightsavers $2.5 (all) Incentive – 1 AMF $2.5 (all) Incentive – 1 GiveDirectly $2.5 (all) Incentive – 1 END Fund $2.5 (all) Incentive – 1 Malaria Consortium $2.5 (all) Incentive – 1 Other charities worthy of special recognition $1.5 (all) Incentive $250,000 each for six charities 3 SCI $6.5 (all) Fills rest of execution level 1 Highest cost-effectiveness of remaining level 1 gaps 4 AMF $8.5 (all) First part of execution level 1 Similar cost-effectiveness to END Fund and Sightsavers and greater understanding of the organization. Expect declining cost-effectiveness within Level 1, and see other benefits (incentives) to switching to END Fund and Sightsavers after this point. 5 END Fund $2.5 (all) Middle part of execution level 1 Given relatively limited knowledge of charity, capping total recommendation at $5 million 6 Sightsavers $0.5 (all) Fills rest of execution level 1 Similar cost-effectiveness to AMF and the END Fund 7 Deworm the World $2.0 (all) Fills execution level 2 Highest-ranked level 2 gap. Highest cost-effectiveness and confidence in organization 8 SCI $4.5 (all) First part of execution level 2 Highest cost-effectiveness of remaining level 2 gaps 9 Malaria Consortium $2.5 (all) Part of execution level 1 Given relatively limited knowledge of charity, capping total recommendation at $5 million 10 AMF $18.6 ($5.1) Part of execution level 1 Expect declining cost-effectiveness within level 1; ranked other gaps higher due to this and incentive effects 11 SCI $4.5 ($0) Fills execution level 2 Roughly expected to be more cost-effective than the remaining $49 million of AMF level 1

* Also includes $1 million that GiveWell holds for grants to top charities. More below.

Summary of key considerations for top charities

The table below summarizes the key considerations for our seven top charities. More detail is provided below as well as in the charity reviews.

Consideration AMF Malaria Consortium Deworm the World END Fund SCI Sightsavers GiveDirectly Estimated cost-effectiveness (relative to cash transfers) ~4x ~4x ~10x ~4x ~8x ~5x Baseline Our level of knowledge about the organization High Relatively low High Relatively low High Relatively low High Primary benefits of the intervention Under-5 deaths averted and possible increased income in adulthood Possible increased income in adulthood Immediate increase in consumption and assets Ease of communication Moderate Strong Strong Strong Moderate Moderate Strongest Ongoing monitoring and likelihood of detecting future problems Moderate Moderate Strong Moderate Moderate Moderate Strongest Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation High: less than half of Execution Level 1 filled High: not quantified, but could likely use significantly more funding Low: Execution Levels 1 and 2 filled High: half of Execution Level 1 filled Moderate: Execution Level 1 and some of Level 2 filled Moderate: Execution Level 1 filled Very high: less than 15% of Execution Level 1 filled

Our recommendation to donors

If Good Ventures uses a budget of $50 million to top charities and follows our prioritization of funding gaps, it will make the following grants (in millions of dollars, rounded to one decimal place):

  • AMF: $15.1
  • Deworm the World: $4.5
  • END Fund: $5.0
  • GiveDirectly: $2.5
  • Malaria Consortium: $5.0
  • SCI: $13.5
  • Sightsavers: $3.0
  • Grants to other charities worthy of special recognition: $1.5

We also hold about $1 million that is restricted to granting out to top charities. We plan to use this to make a grant to AMF, which is the next funding gap on the list after the expected grants from Good Ventures.

We estimate that non-Good Ventures donors will give approximately $27 million between now and the start of June 2017; we expect to refresh our recommendations to donors in mid-June. Of this, we expect $18 million will be allocated according to our recommendation for marginal donations, while $9 million will be given based on our top charity list—this $9 million is considered ‘expected funding’ for each charity and therefore subtracted from their room for more funding.

$18 million spans two gaps in our prioritized list, so we are recommending that donors split their gift, with 75% going to AMF and 25% going to SCI, or give to GiveWell for making grants at our discretion and we will use the funds to fill in the next highest priority gaps.

Details on new top charities

Before this year, our top charity list had remained nearly the same for several years. This means that we have spent hundreds of hours talking to these groups, reading their documents, visiting their work in the field, and modeling their cost-effectiveness. We have spent considerably less time on our new top charities, particularly Malaria Consortium, and have not visited their work in the field (though we met with Sightsavers’ team in Ghana). We believe our new top charities are outstanding giving opportunities, though we think there is a higher risk that further investigation will lead to changes in our views about these groups.

A note about deworming

Four of our top charities, including two new top charities, support programs that treat schistosomiasis and soil-transmitted helminthiasis (STH) (“deworming”). We estimate that SCI and Deworm the World’s deworming programs are more cost effective than mass bednet campaigns, but our estimates are subject to substantial uncertainty. For Sightsavers and END Fund, our greater uncertainty about cost per treatment and prevalence of infection in the areas where they work leads us to the conclusion that the cost-effectiveness of their work is on par with that of bednets. It’s important to note that we view deworming as high expected value, but this is due to a relatively low probability of very high impact. Our cost-effectiveness model implies that most staff members believe you should use a multiplier of less than 1% compared to the impact (increased income in adulthood) found in the original trials—this could be thought of as assigning some chance that deworming programs have no impact, and some chance that the impact exists but will be smaller than was measured in those trials. Full discussion in this blog post. Our 2016 cost-effectiveness analysis is here.

This year, David Roodman conducted an investigation into the evidence for deworming’s impact on long-term life outcomes. David will write more about this in a future post, but in short, we think the strength of the case for deworming is similar to last year’s, with some evidence looking weaker, new evidence that was shared with us in an early form this year being too preliminary to incorporate, and a key piece of evidence standing up to additional scrutiny.

END Fund (for work on deworming)

Our full review of END Fund is here.

Overview

The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming.

About 60% of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list.

We have not yet seen monitoring results on the number of children reached in END Fund-supported programs. The END Fund has instituted a requirement that grantees conduct coverage surveys and the first results will be available in early 2017. While we generally put little weight on plans for future monitoring, we feel that the END Fund’s commitment is unusually credible because surveys are already underway or upcoming in the next few months, we are familiar enough with the type of survey being used (from research on other deworming groups) that we were able to ask critical questions, and the END Fund provided specific answers to our questions.

We have more limited information on some questions for the END Fund than we do for the top charities we have recommended for several years. We do not have a robust cost per treatment figure, and also have limited information on infection prevalence and intensity.

Funding gap

We estimate that the END Fund could productively use between $10 million (50% confidence) and $22 million (5% confidence) in the next year to expand its work on deworming. By our estimation, about a third of this would be used to fund other NTD programs.

This estimate is based on (a) a list of deworming funding opportunities that the END Fund had identified as of October and its expectation of identifying additional opportunities over the course of the year (excluding opportunities to grant funding to Deworm the World, SCI, or Sightsavers, which we count in those organizations’ room for more funding); and (b) our rough estimate of how much funding the END Fund will raise. The END Fund is a fairly new organization whose revenue comes primarily from a small number of major donors so it is hard to predict how much funding it will raise.

The END Fund’s list of identified opportunities includes both programs that END Fund has supported in past years and opportunities to get new programs off the ground.

Sightsavers (for work on deworming)

Our full review of Sightsavers is here.

Overview

Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and, more specifically, advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections.

We believe that deworming is a highly cost-effective program and that there is moderately strong evidence that Sightsavers has succeeded in achieving fairly high coverage rates for some of its past NTD programs. We feel that the monitoring data we have from SCI and Deworm the World is somewhat stronger than what we have from Sightsavers—in particular, the coverage surveys that Sightsavers has done to date were on NTD programs that largely did not include deworming. Sightsavers plans to do annual coverage surveys on programs that are supported by GiveWell-influenced funding.

We have more limited information on some questions for Sightsavers than we do for the top charities we have recommended for several years. We do not have a robust cost-per-treatment figure, though the information we have suggests that it is in the same range as the cost-per-treatment figures for SCI and Deworm the World. We also have limited information on infection prevalence and intensity in the places Sightsavers works. This limits our ability to robustly compare Sightsavers’ cost effectiveness to other top charities, but our best guess is that the cost-effectiveness of the deworming charities we recommend is similar.

Funding gap

We believe Sightsavers could productively use or commit between $3.0 million (50% confidence) and $10.1 million (5% confidence) in funding restricted to programs with a deworming component in 2017.

This estimate is based on (a) a list of deworming funding opportunities that Sightsavers created for us; and (b) our understanding that Sightsavers would not allocate much unrestricted funding to these opportunities in the absence of GiveWell funding. It’s difficult to know whether other funders might step in to fund this work, but Sightsavers believes that is unlikely and deworming has not been a major priority for Sightsavers to date.

Sightsavers’ list of opportunities includes both adding deworming to existing NTD mass distribution programs and establishing new integrated NTD programs that would include deworming and spans work in Nigeria, Guinea-Bissau, Democratic Republic of Congo, Guinea, Cameroon, Cote d’Ivoire, and possibly South Sudan.

Malaria Consortium (for work on seasonal malaria chemoprevention)

Our full review of Malaria Consortium is here.

Overview

Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria.

The evidence for SMC appears strong (stronger than deworming and not quite as strong as bednets), but we have not yet examined the intervention at nearly the same level that we have for bednets, deworming, unconditional cash transfers, or other priority programs. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately powered to find an impact on mortality.

Malaria Consortium and its partners have conducted studies in most of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, but leave us with some remaining questions about the program’s impact.

Overall, we have more limited information on some questions for Malaria Consortium than we do for the top charities we have recommended for several years. We have remaining questions on cost per child per year and on offsetting effects from possible drug resistance and disease rebound.

Funding gap

We have not yet attempted to estimate Malaria Consortium’s maximum room for more funding. We would guess that Malaria Consortium could productively use at least an additional $30 million to scale up its SMC activities over the next three to four years. We have a general understanding of where additional funds would be used but have not yet asked for a high level of detail on potential bottlenecks to scaling up.

We do not believe Malaria Consortium has substantial unrestricted funding available for scaling up its support of SMC programs and expect its restricted funding for SMC to remain steady or decrease in the next few years.

Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net distributions (for protection against malaria) in developing countries. There is strong evidence that distributing nets reduces child mortality and malaria cases.

AMF provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity.

We estimate that AMF’s program is roughly 4 times as cost effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

In 2016, AMF significantly increased the number and size of distributions it committed funding to. Prior to 2015, it had completed (large-scale) distributions in two countries, Malawi and Democratic Republic of Congo (DRC). In 2016, it completed a distribution in Ghana and committed to supporting distributions in an additional three countries, including an agreement to contribute $28 million to a campaign in Uganda, its largest agreement to date by far.

AMF has continued to collect and share information on its past large-scale distributions. This includes both data from registering households to receive nets (and, in some cases, data on the number of nets each household received) and follow-up surveys to determine whether nets are in place and in use. Our research in 2016 has led us to moderately weaken our assessment of the quality of AMF’s follow up surveys. In short, we learned that the surveys in Malawi have not used fully randomized selection of households and that the first two surveys in DRC were not reliable (full discussion in this blog post). We expect to see follow-up surveys from Ghana and DRC in the next few months that could expand AMF’s track record of collecting this type of data. We also learned that AMF has not been carrying out data audits in the way we believed it was (though this was not a major surprise as we had not asked AMF for details of the auditing process previously).

AMF has generally been communicative and open with us. We noted in our mid-year update that AMF had been slower to share documentation for some distributions; however, we haven’t had concerns about this in the second half of the year.

In August 2016, four GiveWell staff visited Ghana where an AMF-funded distribution had recently been completed. We met with AMF’s program manager, partner organizations, and government representatives and visited households in semi-urban and rural areas (notes and photos from our trip).

Our estimate of the cost-effectiveness of nets has fallen relative to cash transfers since our mid-year update. At that point, we estimated that nets were ~10x as cost-effective as cash transfers, and now we estimate that they are ~4x as cost-effective as cash transfers. This change was partially driven by changes in GiveWell staff’s judgments on the tradeoff between saving lives of children under five and improving lives (through increased income and consumption) in our model, and partially driven by AMF beginning to fund bed net distributions in countries with lower malaria burdens than Malawi or DRC.

Funding gap

AMF currently holds $17.8 million, and expects to commit $12.9 million of this soon. We estimate it will receive an additional $4 million by June 2017 ($2 million from donors not influenced by GiveWell and $2 million from donors who give based on our top charity list) that it could use for future distributions. Together, we expect that AMF will have about $9 million for new spending and commitments in 2017.

We estimate that AMF could productively use or commit between $87 million (50% confidence) and $200 million (5% confidence) in the next year. We arrived at this estimate from a rough estimate of the total Africa-wide funding gap for nets in the next three years (from the African Leaders Malaria Alliance)—estimated at $125 million per year. The estimate is rough in large part because the Global Fund to Fight AIDS, Tuberculosis and Malaria, the largest funder of LLINs, works on three-year cycles and has not yet determined how much funding it will allocate for LLINs for 2018-2020. We talked to people involved in country-level planning of mass net distributions and the Global Fund, who agreed with the general conclusion that there were likely to be large funding gaps in the next few years. In mid-2016, AMF had to put some plans on hold due to lack of funding.

We now believe that AMF has a strong track record of finding distribution partners to work with and coming to agreements with governments, and we do not expect that to be a limiting factor for AMF. The main risks we see to AMF’s ability to scale are the possibility that funding from other funders is sufficient (since our estimate of the gap is quite rough), the likelihood that government actors have limited capacity for discussions with AMF during a year in which they are applying for Global Fund funding, AMF’s staff capacity to manage discussions with additional countries (it has only a few staff members), and whether gaps will be spread across many countries or located in difficult operating environments. We believe the probability of any specific one of these things impeding AMF’s progress is low.

We believe there are differences in cost-effectiveness within execution level 1 and believe the value of filling the first part of AMF’s gap may be higher than additional funding at higher levels. This is because AMF’s priorities include committing to large distributions in the second half of 2019 and 2020, which increases the uncertainty about whether funding would have been available from another source.

We and AMF have discussed a few possibilities for how AMF might fill funding gaps. AMF favors an approach where it purchases a large number of nets for a small number of countries. This approach has some advantages including efficiency for AMF and leverage in influencing how distributions are carried out. Our view is that the risk of displacing a large amount of funding from other funders using this approach outweighs the benefits. If AMF did displace a large amount of funding which would otherwise have gone to nets, that could make donations applied to these distributions considerably less cost-effective. More details on our assessment of AMF’s funding gap are in our full review.

Deworm the World Initiative, led by Evidence Action

Our full review of Deworm the World is here.

Background

Deworm the World (evidenceaction.org/#deworm-the-world), led by Evidence Action, advocates for, supports, and evaluates deworming programs. It has worked in India and Kenya for several years and has recently expanded to Nigeria, Vietnam, and Ethiopia.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World is slightly more cost-effective than SCI, more cost-effective than AMF and the other deworming charities, and about 10 times as cost-effective as cash transfers.

Important changes in the last 12 months

Deworm the World has made somewhat slower progress than expected in expanding to new countries. In late 2015, Good Ventures, on GiveWell’s recommendation, made a grant of $10.8 million to Deworm the World to fund its execution level 1 and 2 gaps. Execution level 1 funding was to give Deworm the World sufficient resources to expand into Pakistan and another country. Deworm the World has funded a prevalence survey in Pakistan, which is a precursor to funding treatments in the country. It has not expanded into a further country that it was not already expecting to work in. As a result, we believe that Deworm the World has somewhat limited room for more funding this year.

Overall, we have more confidence in our understanding of Deworm the World and its parent organization Evidence Action’s spending, revenues, and financial position than we did in previous years. While trying to better understand this information this year, we found several errors. We are not fully confident that all errors have been corrected, though we are encouraged by the fact that we are now getting enough information to be able to spot inconsistencies. Evidence Action has been working to overhaul its financial system this year.

Our review of Deworm the World has focused on two countries, Kenya and India, where it has worked the longest. In 2016, we saw the first results of a program in another country (Vietnam), as well as continued high-quality monitoring from Kenya and India. The Vietnam results indicate that Deworm the World is using similar monitoring processes in new countries as it has in Kenya and India and that results in Vietnam have been reasonably strong.

Evidence Action hired Jeff Brown (formerly Interim CEO of the Global Innovation Fund) as CEO in 2015. Recently Evidence Action announced that he has resigned and has not yet been replaced. Our guess is this is unlikely to be disruptive to Deworm the World’s work; Grace Hollister remains Director of the Deworm the World Initiative.

Funding gap

We believe that there is a 50% chance that Deworm the World will be slightly constrained by funding in the next year and that additional funds would increase the chances that it is able to take advantage of any high-value opportunities it encounters. We estimate that if it received an additional $4.5 million its chances of being constrained by funding would be reduced to 20% and at $13.4 million in additional funding, this would be reduced to 5%.

In the next year, Deworm the World expects to expand its work in India and Nigeria and may have opportunities to begin treatments in Pakistan and Indonesia. It is also interested in using unrestricted funding to continue its work in Kenya, and puts a high priority on this program. Its work in Kenya has to date been funded primarily by the Children’s Investment Fund Foundation (CIFF) and this support is set to expire in mid 2017. It is unclear to us whether CIFF will continue providing funding for the program and, if so, for how long. Due to the possibility that Deworm the World unrestricted funding may displace funding from CIFF, and, to a lesser extent, the END Fund and other donors, we consider the opportunity to fund the Kenya program to be less cost-effective in expectation than it would be if we were confident in the size of the gap.

More details in our full review.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify recipient countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies in about two-thirds of the countries it works in to determine whether its programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is slightly less cost-effective than Deworm the World, more cost-effective than AMF and the other deworming charities, and about 8 times as cost-effective as cash transfers.

Important changes in the last 12 months

In past years, we’ve written that we had significant concerns about SCI’s financial reporting and financial management, and the clarity of our communication with SCI. In June, we wrote that we had learned of two substantial errors in SCI’s financial managment and reporting that began in 2015. We also noted that we thought that SCI’s financial management and financial reporting, as well as the clarity of its communication with us overall, had improved significantly. In the second half of the year, SCI communicated clearly with us about its plans for deworming programs next year and its room for more funding.

SCI reports that it has continued to scale up its deworming programs over the past year and that it plans to start up new deworming programs in two states in Nigeria before the end of its current budget year.

This year, SCI has shared a few more coverage surveys from deworming programs in Ethiopia, Madagascar, and Mozambique that found reasonably high coverage.

Professor Alan Fenwick, Founder and Director of SCI for over a decade, retired from his position this year, though will continue his involvement in fundraising and advocacy. The former Deputy Director, Wendy Harrison, is the new Director.

Funding gap

We estimate that SCI could productively use or commit a maximum of between $9.0 million (50% confidence) and $21.4 million (5% confidence) in additional unrestricted funding in its next budget year.

Its funding sources have been fairly steady in recent years with about half of its revenue in the form of restricted grants, particularly from the UK government’s Department for International Development (this grant runs through 2018), and half from unrestricted donations, a majority of which were driven by GiveWell’s recommendation. We estimate that SCI will have around $5.4 million in unrestricted funding available to allocate to its 2017-18 budget year (in addition to $6.5 million in restricted funding).

SCI has a strong track record of starting and scaling up programs in a large number of countries. SCI believes it could expand significantly with additional funding, reaching more people in the countries it works in and expanding to Nigeria and possibly Chad.

More details in our full review.

GiveDirectly

Our full review of GiveDirectly is here.

Background

GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82% overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people.

We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. It appears that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected (more). It continues to experiment heavily, with the aim of improving how its own and government cash transfer programs are run. It has recently started work on evaluations that benchmark programs against cash with the aim of influencing the broader international aid sector to use its funding more cost-effectively.

We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost effective than cash.

Important changes in the last 12 months

GiveDirectly has continued to scale up significantly, reaching a pace of delivering $21 million on an annual basis in the first part of 2016 and expecting to reach a pace of $50 million on an annual basis at the end of 2016. It has continued to share informative and detailed monitoring information with us. Given its strong and consistent monitoring in the past, we have taken a lighter-touch approach to evaluating its processes and results this year.

The big news for GiveDirectly this year was around partnerships and experimentation. It expanded into Rwanda (its third country) and launched a program to compare, with a randomized controlled trial, another aid program to cash transfers (details expected to be public next year). The program is being funded by a large institutional funder and Google.org. It expects to do additional “benchmarking” studies with the institutional funder, using funds from Good Ventures’ 2015 $25 million grant, over the next few years.

It also began fundraising for and started a pilot of a universal basic income (UBI) guarantee—a program providing long-term, ongoing cash transfers sufficient for basic needs, which will be evaluated with a randomized controlled trial comparing the program to GiveDirectly’s standard lump sum transfers. The initial UBI program and study is expected to cost $30 million. We estimate that it is less cost-effective than GiveDirectly’s standard model, but it could have impact on policy makers that isn’t captured in our analysis.

We noted previously that Segovia, a for-profit technology company that develops software for cash transfer program implementers and which was started and is partially owned by GiveDirectly’s co-founders, would provide its software for free to GiveDirectly to avoid conflicts of interest. However, in 2016, after realizing that providing free services to GiveDirectly was too costly for Segovia (customizing the product for GiveDirectly required much more Segovia staff time than initially expected), the two organizations negotiated a new contract under which GiveDirectly will compensate Segovia for its services. GiveDirectly wrote about this decision here. GiveDirectly told us that it recused all people with ties to both organizations from this decision and evaluated alternatives to Segovia. Although we believe that there are possibilities for bias in this decision and in future decisions concerning Segovia, and we have not deeply vetted GiveDirectly’s connection with Segovia, overall we think GiveDirectly’s choices were reasonable. However, we believe that reasonable people might disagree with this opinion, which is in part based on our personal experience working closely with GiveDirectly’s staff for several years.

Funding gap

We believe that GiveDirectly is very likely to be constrained by funding next year. GiveDirectly has been rapidly building its capacity to enroll recipients and deliver funds, while some of its revenue has been redirected to its universal basic income guarantee program (either because of greater donor interest in that program or by GiveDirectly focusing its fundraising efforts on it).

We expect GiveDirectly to have about $20 million for standard cash transfers in its 2017 budget year. This includes raising about $15.8 million from non-GiveWell-influenced sources between now and halfway through its 2017 budget year (August 2017) and $4 million from donors who give because GiveDirectly is on GiveWell’s top charity list. $4 million is much less than GiveWell-influenced donors gave in the last year. This is because several large donors are supporting GiveDirectly’s universal basic income guarantee program this year and because one large donor gave a multi-year grant that we don’t expect to repeat this year.

GiveDirectly is currently on pace (with no additional hiring) to have four full teams operating its standard cash transfer model in 2017. To fully utilize four teams, it would need $28 million more than we expect it to raise. We accordingly expect that GiveDirectly will downsize somewhat in 2017, because we do not project it raising sufficient funds to fully utilize the increased capacity it has built to transfer money. Given recent growth, we believe that GiveDirectly could easily scale beyond four teams and we estimate that at $46 million more than we expect it to raise ($66 million total for standard transfers), it would have a 50% chance of being constrained by funding.

Other charities worthy of special recognition

Last year, we recommended four organizations as “standouts.” This year we are calling this list “other charities worthy of special recognition.” We’ve added two organizations to the list: Food Fortification Initiative and Project Healthy Children. Although our recommendation to donors is to give to our top charities over these charities, they stand out from the vast majority of organizations we have considered in terms of the evidence base for their work and their transparency, and they offer additional giving options for donors who feel highly aligned with their work.

We don’t follow these organizations as closely as we do our top charities. We generally have one or two calls per year with each group, publish notes on our conversations, and follow up on any major developments.

We provide brief updates on these charities below:

  • Organizations that have conducted randomized controlled trials of their programs:
    • Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its program and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15% or more) and data from health facilities finding an increase in facility visits. (The results, because the trial was only completed in the last year, are not yet published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. We noted last year that DMI was planning to conduct another survey for the RCT in late 2016; it has decided not to move forward with this, but is interested in conducting new research studies in other countries, if it is able to raise the money to do so. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI, with conversation notes and documents from 2016, is here.
    • Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a randomized controlled trial of its program and measured a 27% reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of cash. Living Goods is scaling up its program and may need additional funding in the future, but has not yet been limited by funding. We published an update on Living Goods in mid-2016. Our 2014 review of Living Goods is here.
  • Organizations working on micronutrient fortification: We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence.
    • Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. Our full review is here.
    • Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN, published in 2016 based on research done in 2015, is here.
    • IGN. Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-influenced funding has made up a large part of its funding in the past year. This year, we published an update on our investigation into IGN’s work in select countries in 2015 and notes from our conversation with IGN to learn about its progress in 2016 and plans for 2017. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN, from 2014, is here.
    • Project Healthy Children (PHC). PHC aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs. Our review is preliminary and in particular we do not have a recent update on how PHC would use additional funding. Our review of PHC, published in 2016 but based on information collected in 2015, is here.

Our research process in 2016

We plan to detail the work we completed this year in a future post as part of our annual review process. Much of this work, particularly our experimental work and work on prioritizing interventions for further investigation, is aimed at improving our recommendations in future years. Here we highlight the key research that led to our current recommendations. See our process page for our overall process.

  • As in previous years, we did intensive follow up with each of our top charities, including publishing updated reviews mid-year. We had several conversations by phone with each organization, met in person with Deworm the World, SCI, and AMF (over the course of a 4-day site visit to Ghana), and reviewed documents they shared with us.
  • In 2015 and 2016, we sought to expand top charity room for more funding and consider alternatives to our top charities by inviting other groups that work on deworming, bednet distributions, and micronutrient fortification to apply. This led to adding Sightsavers, the END Fund, Project Healthy Children, and Food Fortification Initiative to our lists this year. Episcopal Relief & Development’s NetsforLife® Program, Micronutrient Initiative, and Nothing but Nets declined to fully participate in our review process.
  • We completed intervention reports on voluntary medical male circumcision (VMMC) and cataract surgery. We asked VMMC groups PSI (declined to fully participate) and the Centre for HIV and AIDS Prevention Studies (pending) to apply. We had conversations with several charities working on cataract surgery and have not yet asked any to apply.
  • We did very preliminary investigations into a large number of interventions and prioritized a few for further work. This led to interim intervention reports on seasonal malaria chemoprevention (SMC), integrated community case management (iCCM) and ready-to-use therapeutic foods for treating severe acute malnutrition and recommending Malaria Consortium for its work on SMC.
  • We stayed up to date on the research for bednets, cash transfers, and deworming. We published a report on insecticide resistance and its implications for bednet programs. A blog post on our work on deworming is forthcoming. We did not find major new research on cash transfers that affected our recommendation of GiveDirectly.

Giving to GiveWell vs. top charities

GiveWell and the Open Philanthropy Project are planning to split into two organizations in the first half of 2017. The split means that it is likely that GiveWell will retain much of the assets of the previously larger organization while reducing its expenses. We think it’s fairly likely that our excess assets policy will be triggered and that we will grant out some unrestricted funds. Given that expectation, our recommendation to donors is:

  • If you have supported GiveWell’s operations in the past, we ask that you consider maintaining your support. It is fairly likely that these funds will be used this year for grants to top charities, but giving unrestricted signals your support for our operations and allows us to better project future revenue and make plans based on that. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and minimize staff time spent on fundraising.
  • If you have not supported GiveWell’s operations in the past, we ask that you consider checking the box on our donate form to add 10% to help fund GiveWell’s operations. In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations.

Footnotes:

* For example, if $30 million were available to fund gaps of $10 million, $5 million, and $100 million, we would recommend allocating the funds so that the $10 million and $5 million gaps were fully filled and the $100 million gap received $15 million.

The post Our updated top charities for giving season 2016 appeared first on The GiveWell Blog.

Natalie Crispin

Updates on AMF’s transparency and monitoring

7 years 5 months ago

In our mid-year update, we continued to recommend that donors give to the Against Malaria Foundation (AMF), and we wrote that we believe AMF has the most valuable current funding gap among our top charities. We also briefly wrote about some new concerns we have about AMF based on our research from the first half of 2016.

This post describes our new concerns about AMF’s transparency and monitoring in more depth. We continue to believe that AMF stands out among bed net organizations, and among charities generally, for its transparency and the quality of its program monitoring. AMF makes substantial amounts of useful information on the impact of its programs—far more than the norm—publicly available on its website, and has generally appeared to value transparency as much as any organization we’ve encountered. But our research on AMF in 2016 has led us to moderately weaken our view that AMF stands out for these qualities. In short, this is because:

  • The first two post-distribution check-up surveys following AMF’s bed net distribution in Kasaï-Occidental, Democratic Republic of the Congo (DRC) were poorly implemented. AMF told us that it agrees that the surveys were poorly implemented and is working to improve data quality in future surveys.
  • We learned that the methodology for selecting communities and households for post-distribution check-up surveys in Malawi is less rigorous than we had previously thought.
  • AMF was slower to share information with us in 2016 than we would have liked. Unfortunately, we aren’t fully confident about what caused this to happen. We believe that AMF misunderstood the type of information we would value seeing, and this may have caused some (but not all) of this issue.

These updates somewhat lower our confidence in AMF’s track record of distributing bed nets that are used effectively (i.e., present, hanging, and in good condition) over the long term and in its commitment to transparency; however, this is only a moderate update, and we don’t think what we’ve learned is significant enough to outweigh AMF’s strengths. We continue to recommend AMF and believe that it has the highest-value current funding gap of any of our top charities.

Going forward, we plan to continue to learn more about AMF’s transparency and monitoring through reviewing the results of additional post-distribution surveys and continued communication with AMF.

Background on AMF’s strengths and evidence of impact

We recommend AMF because there is strong evidence that mass distribution of long-lasting insecticide-treated bed nets reduces child mortality and is cost-effective, and because of AMF’s strengths as an organization: a standout commitment to transparency and self-evaluation, substantial room for more funding to deliver additional bed nets, and relatively strong evidence overall on whether bed nets reach intended destinations and are used over the long term.

In particular, AMF requires that its distribution partners implement post-distribution check-up surveys every six months among a sample (around 5%) of recipient households for 2.5 years following a bed net distribution, and has publicly shared the results of these surveys from several of its bed net distributions in Malawi. It’s our understanding that AMF is quite unusual in this regard—other organizations that fund bed nets do not typically require post-distribution check-up surveys to monitor bed net usage over time, and do not publicly share monitoring data as AMF does.

Evidence of the impact of AMF’s bed net distribution in Kasaï-Occidental, DRC

AMF has sent us reports and data from two post-distribution check-up surveys (from eight months and twelve months after the distribution) from Kasaï-Occidental, DRC.

Donors may not realize that AMF has a short track record of major distributions. It has historically primarily worked with Concern Universal in Malawi, so these are the first surveys we’ve seen from large-scale AMF bed net distributions outside of Malawi. AMF’s post-distribution check-up surveys are intended to provide evidence on how many AMF nets are used effectively over the long-term, but we (and AMF) believe that these surveys in Kasaï-Occidental, DRC were poorly implemented (details in footnote).[1]

Due to the extent of the implementation issues, we don’t think the post-distribution check-up surveys provide a reliable estimate of the proportion of bed nets distributed in Kasaï-Occidental, DRC used effectively over the long term. (Note that AMF earlier provided a distribution report, registration data, and photos and videos as evidence that bed nets originally reached intended destinations.) It seems plausible to us that the reported rates of nets in-use (hung over a sleeping space) from the AMF distribution in the 8-month post-distribution check-up survey (~80%) and the 12-month post-distribution check-up survey (64-69%) are either substantial overestimates or substantial underestimates.

Non-random sampling in post-distribution surveys in Malawi

This year, we learned that Concern Universal, AMF’s distribution partner in Malawi, does not use a completely random process to select participants for post-distribution surveys. We have received some conflicting information from AMF and Concern Universal on the specifics of how the selection process deviates from pure randomization, so we aren’t confident that we fully understand how the selection process works in practice (details in footnote).[2]

Our earlier understanding was that Concern Universal randomly selected villages and households for post-distribution surveys without any adjustments.

We are now concerned that the results from post-distribution surveys from Malawi could be somewhat biased estimates of the long-term impact of AMF’s distributions (though we wouldn’t guess that the effect of the bias on the results would be very large, since AMF and Concern Universal described selection processes that seem likely to produce reasonably representative samples).

AMF told us that it may reconsider its requirements for random selection of participants in future post-distribution surveys and invited us to make suggestions for improvement.

AMF’s transparency and communication

Although we still believe that AMF stands out among bed net organizations for its commitment to transparency, AMF has recently been less transparent with us than we’ve come to expect.

In early 2016, we requested several documents from AMF (including the 8-month and 12-month post distribution surveys from Kasaï-Occidental, DRC, malaria case rate data from clinics in Malawi, and audits of household registration data from Malawi), which AMF told us it had available and would share once it had the capacity to review and edit them. Although we eventually received reports and data from the two DRC post-distribution surveys in June, we still haven’t seen the other documents we requested. AMF responded to these concerns here.

We are concerned that AMF did not tell us about the poor implementation of the first two Kasaï-Occidental, DRC surveys earlier, and that we only recently learned about the details of Concern Universal’s adjustments to random sampling for post-distribution surveys in Malawi. AMF told us it agrees that it should have communicated more clearly with us about these two issues and believes that it did not because it misunderstood the type of information we would value seeing. We are not confident that this fully explains AMF’s lack of transparency.

What we hope to learn going forward

AMF’s track record of providing evidence of impact on its bed net distributions outside of Malawi is currently very limited. Our impression is that DRC is a difficult country for charities to work in; we’re uncertain whether the methodological issues with the first two surveys from Kasaï-Occidental were due to the difficulty of working in DRC specifically, to more general issues with AMF starting programs in new countries and working with new implementing partners, or to the relatively poor performance of an implementing partner.

AMF has told us that it expects the implementation of future post-distribution surveys in DRC to improve, and that it has made several changes to its practices in response to the issues discussed above, including:

  • Hiring a Program Director, Shaun Walsh, whose primary job is to work in-country with distribution partners on planning, executing, and monitoring bed net distributions.
  • Requiring more detailed budgets and plans from distribution partners for upcoming post-distribution surveys in Ghana, Uganda, and Togo.
  • Focusing on improving timeliness of reporting on distributions and post-distribution surveys.

We plan to communicate closely with AMF on its upcoming post-distribution surveys, and update our views on AMF’s track record outside of Malawi when more survey results are available.

Notes
[1]
AMF’s reports on the surveys indicate that:

  1. It seems likely that different data collectors interpreted ambiguously-worded questions differently for both the 8-month and 12-month surveys. “Number of nets available” (translated from French) was variously interpreted as the number of nets hung, the number of nets hung plus the number of nets present but not hung, or the number of nets present but not hung. This led to internally inconsistent data (e.g. different numbers of nets reported for a single household for different survey questions) for a large proportion of households (42% in the 8-month post-distribution survey and around half in the 12-month post-distribution survey). AMF excluded households with internally inconsistent data from its analysis of the proportion of nets from the distribution still in use.
  2. AMF addressed this issue by re-writing survey questions after the 8-month survey, but the corrected survey questions were not put onto the data collectors’ smartphones before the 12-month survey.
  3. Household members sometimes reported inaccurate information to data collectors when survey questions were asked outside of a home. Data collectors later confirmed that the information was inaccurate (e.g. the household owned more bed nets than reported) by direct observation inside the home, but were not able to correct the data already entered into their smartphones.
  4. Data collectors did not distinguish between nets from the late 2014 AMF distribution and bed nets from other sources. AMF notes that the average level of previously-owned nets was around 2.5% so this would not have materially influenced the results of the post-distribution survey.

[2]

  • AMF told us:
    Concern Universal selects villages for post-distribution surveys in each health center catchment area where AMF nets were distributed. Concern Universal divides each health center catchment area into three “bands:” a short distance, medium distance, and far distance away from the health center. In each band, Concern Universal randomly selects between 25% and 50% of the villages. In each of those villages, Concern Universal randomly selects around 20% of the households.
  • In April 2016, we spoke with a representative of Concern Universal, who told us that, in addition to the stratification of villages by geographic location described by AMF, that villages selected in one post-distribution survey are excluded from being selected for the following post-distribution survey.

The post Updates on AMF’s transparency and monitoring appeared first on The GiveWell Blog.

Andrew Martin
Checked
7 minutes 17 seconds ago
Exploring how to get real change for your dollar.