Aggregator

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our June 2018 open thread here.

The post September 2018 open thread appeared first on The GiveWell Blog.

In April to June 2018, we received $1.2 million in funding for making grants at our discretion. In addition, GiveWell’s Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. In this post we discuss:

• The decision to allocate the $4.1 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.
• Why we have allocated unrestricted funds to making grants to recommended charities.

Allocation of discretionary funds

The allocation of 70 percent of the funds to AMF and 30 percent to SCI follows the recommendation we have made, and continue to make, to donors. For more discussion on this allocation, see our blog post about allocating discretionary funds from the fourth quarter of 2017.

We ask each top charity to provide details of how they will use additional funding each year, as part of our process to update our “room for more funding” summary for each top charity. This year, we have asked for this information by the end of July. We also ask each of our top charities to let us know if they encounter unexpected funding gaps at other times of year. We have not learned of new funding gaps in the last quarter.

What is our recommendation to donors?

We continue to recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we are continuing to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact. The reasons for this recommendation are the same as in our Q4 2017 post on allocating discretionary funding.

We will complete a full analysis of our top charities’ funding gaps and cost-effectiveness by November and expect to update our recommendation to donors at that time.

Why we have allocated unrestricted funds to making grants to recommended charities

In June, GiveWell’s Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. We generally use unrestricted funds to support GiveWell’s operating costs. The decision was made to grant out some of the unrestricted funds we hold in accordance with two policies:

• Our “excess assets” policy specifies that once we surpass a certain level of unrestricted assets, we grant out the excess rather than continue to hold it ourselves. We reviewed our unrestricted asset holdings and projected revenue and expenses for 2018-2020 and concluded that we held $1.8 million more than was required to give us a stable, predictable financial situation (details of how this rule is applied are at the previous link). The Board voted to irrevocably restrict this amount to making grants to recommended charities. Note that we continue to need ongoing donor support for our operations. This decision incorporates our projections for future donations.
• In order to limit the risks of relying too heavily on any single source of revenue, we cap the amount of funding that we will use from one source to support our operating costs at 20% of our projected annual expenses. In early 2018, we received a donation of $2.1 million in unrestricted funds. Our operating expense budget for 2018 is $4.9 million. Therefore, the Board voted to retain $1.0 million to support operating costs in 2018 and irrevocably restrict $1.1 million to making grants to recommended charities.

The post Allocation of discretionary funds from Q2 2018 appeared first on The GiveWell Blog.

In April to June 2018, we received $1.2 million in funding for making grants at our discretion. In addition, GiveWell's Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. In this post we discuss:

• The decision to allocate the $4.1 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.
• Why we have allocated unrestricted funds to making grants to recommended charities.

Read More

The post Allocation of discretionary funds from Q2 2018 appeared first on The GiveWell Blog.

In April to June 2018, we received $1.2 million in funding for making grants at our discretion. In addition, GiveWell's Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. In this post we discuss:

• The decision to allocate the $4.1 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.
• Why we have allocated unrestricted funds to making grants to recommended charities.

Read More

The post Allocation of discretionary funds from Q2 2018 appeared first on The GiveWell Blog.

In April to June 2018, we received $1.2 million in funding for making grants at our discretion. In addition, GiveWell’s Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. In this post we discuss:

• The decision to allocate the $4.1 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.
• Why we have allocated unrestricted funds to making grants to recommended charities.

Allocation of discretionary funds

The allocation of 70 percent of the funds to AMF and 30 percent to SCI follows the recommendation we have made, and continue to make, to donors. For more discussion on this allocation, see our blog post about allocating discretionary funds from the fourth quarter of 2017.

We ask each top charity to provide details of how they will use additional funding each year, as part of our process to update our “room for more funding” summary for each top charity. This year, we have asked for this information by the end of July. We also ask each of our top charities to let us know if they encounter unexpected funding gaps at other times of year. We have not learned of new funding gaps in the last quarter.

What is our recommendation to donors?

We continue to recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we are continuing to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact. The reasons for this recommendation are the same as in our Q4 2017 post on allocating discretionary funding.

We will complete a full analysis of our top charities’ funding gaps and cost-effectiveness by November and expect to update our recommendation to donors at that time.

Why we have allocated unrestricted funds to making grants to recommended charities

In June, GiveWell’s Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. We generally use unrestricted funds to support GiveWell’s operating costs. The decision was made to grant out some of the unrestricted funds we hold in accordance with two policies:

• Our “excess assets” policy specifies that once we surpass a certain level of unrestricted assets, we grant out the excess rather than continue to hold it ourselves. We reviewed our unrestricted asset holdings and projected revenue and expenses for 2018-2020 and concluded that we held $1.8 million more than was required to give us a stable, predictable financial situation (details of how this rule is applied are at the previous link). The Board voted to irrevocably restrict this amount to making grants to recommended charities. Note that we continue to need ongoing donor support for our operations. This decision incorporates our projections for future donations.
• In order to limit the risks of relying too heavily on any single source of revenue, we cap the amount of funding that we will use from one source to support our operating costs at 20% of our projected annual expenses. In early 2018, we received a donation of $2.1 million in unrestricted funds. Our operating expense budget for 2018 is $4.9 million. Therefore, the Board voted to retain $1.0 million to support operating costs in 2018 and irrevocably restrict $1.1 million to making grants to recommended charities.

The post Allocation of discretionary funds from Q2 2018 appeared first on The GiveWell Blog.

In April to June 2018, we received $1.2 million in funding for making grants at our discretion. In addition, GiveWell’s Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. In this post we discuss:

• The decision to allocate the $4.1 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.
• Why we have allocated unrestricted funds to making grants to recommended charities.

Allocation of discretionary funds

The allocation of 70 percent of the funds to AMF and 30 percent to SCI follows the recommendation we have made, and continue to make, to donors. For more discussion on this allocation, see our blog post about allocating discretionary funds from the fourth quarter of 2017.

We ask each top charity to provide details of how they will use additional funding each year, as part of our process to update our “room for more funding” summary for each top charity. This year, we have asked for this information by the end of July. We also ask each of our top charities to let us know if they encounter unexpected funding gaps at other times of year. We have not learned of new funding gaps in the last quarter.

What is our recommendation to donors?

We continue to recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we are continuing to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact. The reasons for this recommendation are the same as in our Q4 2017 post on allocating discretionary funding.

We will complete a full analysis of our top charities’ funding gaps and cost-effectiveness by November and expect to update our recommendation to donors at that time.

Why we have allocated unrestricted funds to making grants to recommended charities

In June, GiveWell’s Board of Directors voted to allocate $2.9 million in unrestricted funds to making grants to recommended charities. We generally use unrestricted funds to support GiveWell’s operating costs. The decision was made to grant out some of the unrestricted funds we hold in accordance with two policies:

• Our “excess assets” policy specifies that once we surpass a certain level of unrestricted assets, we grant out the excess rather than continue to hold it ourselves. We reviewed our unrestricted asset holdings and projected revenue and expenses for 2018-2020 and concluded that we held $1.8 million more than was required to give us a stable, predictable financial situation (details of how this rule is applied are at the previous link). The Board voted to irrevocably restrict this amount to making grants to recommended charities. Note that we continue to need ongoing donor support for our operations. This decision incorporates our projections for future donations.
• In order to limit the risks of relying too heavily on any single source of revenue, we cap the amount of funding that we will use from one source to support our operating costs at 20% of our projected annual expenses. In early 2018, we received a donation of $2.1 million in unrestricted funds. Our operating expense budget for 2018 is $4.9 million. Therefore, the Board voted to retain $1.0 million to support operating costs in 2018 and irrevocably restrict $1.1 million to making grants to recommended charities.

The post Allocation of discretionary funds from Q2 2018 appeared first on The GiveWell Blog.

Summary

We recently completed a small project to determine whether using subnational baseline malaria mortality estimates would make a difference to our estimates of the cost-effectiveness of two of our top charities, the Against Malaria Foundation and Malaria Consortium. We ultimately decided not to include these adjustments because they added complexity to our models and would require frequent updating, while only making a small difference (a 3-4% improvement) to our bottom line.

Though this post is on a fairly narrow topic, we believe this example illustrates the principles we use to make decisions about what to include in our cost-effectiveness model.

Background

Two of our top charities—the Against Malaria Foundation (AMF) and Malaria Consortium’s seasonal malaria chemoprevention program—implement programs to prevent malaria, a leading killer of people in low- and middle-income countries.

One of the core reasons we recommend AMF and Malaria Consortium is their cost-effectiveness: how much impact they have (e.g., cases of malaria prevented, malaria deaths averted) with the funds they receive. Our estimates of charities’ cost-effectiveness isn’t just helpful to us in determining which charities should be GiveWell top charities; we also rely on these estimates to guide our decisions about how to allocate funding between our top charities.

Our cost-effectiveness estimates for AMF and Malaria Consortium use country-wide data on malaria mortality and malaria incidence in the places that both organizations work.1In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); However, neither organization serves a whole country—rather, they operate in sub-national regions—so the use of country-level estimates could cause us to either underestimate or overestimate their cost-effectiveness. If, for example, these programs are focused in the areas of the country with the highest malaria burden, using the average burden for the country would lead us to underestimate their cost-effectiveness. So, we completed a project to determine how much of an impact using subnational estimates would have, to consider whether we ought to incorporate this information into our cost-effectiveness analysis.

How we estimated the impact of subnational malaria incidence

AMF distributes insecticide-treated nets to prevent malaria; Malaria Consortium’s seasonal malaria chemoprevention (SMC) program provides preventive anti-malarial drugs. We used estimates of subnational malaria incidence from the Malaria Atlas Project (MAP) to see if regions covered by nets or eligible for SMC had higher or lower incidence than the average in the country in which they are located.2We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We focused on all areas covered by nets or eligible for SMC (rather than those covered by our top charities, specifically) for two reasons:

1. Our understanding is that when our top charities contribute resources to a country’s net distribution or SMC programs, the marginal region covered by these additional resources is not necessarily the same as the region to which these resources are assigned (because these resources are fungible with other resources within the national programs).3A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
2. Our aim is to estimate the cost-effectiveness of funds donated to these organizations in the future. The subnational region where AMF has worked in the past has not historically been a good indicator of the region where it will work in future.

Results for net distributions in countries where AMF works

We looked at geographical variation in malaria incidence in countries where AMF works, weighting each region by the number of nets it currently receives.4We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average net delivered in the countries in which AMF works is hung in an area with 0-9% higher malaria incidence than the average in that country, and the weighted average adjustment to AMF’s cost-effectiveness would be 3% (in other words, AMF becomes 3% more cost-effective if we incorporate subnational estimates).5See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Zambia +9% Uganda +4% Ghana +4% Democratic Republic of the Congo +1% Togo +1% Malawi +0% Results for SMC in countries where Malaria Consortium works

We looked at six countries comprising >95% of Malaria Consortium’s SMC spending and compared malaria incidence in districts eligible for SMC with the country-wide average.6“The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });7The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average region eligible for SMC in countries where Malaria Consortium works has -2% to 17% higher malaria incidence than the average in that country. The weighted average adjustment to Malaria Consortium’s cost-effectiveness would be 4%.8See Cell C126. jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Commentary Guinea +17% Conakry, the capital, is ineligible for SMC and has low incidence. Nigeria +12% SMC appears to be targeted in the north, where malaria incidence is slightly higher. Niger +2% The majority of the population is either covered or planned to be covered from 2019. Burkina Faso 0% All districts are eligible. Mali 0% All districts are eligible. Chad -2% The four regions with very low malaria incidence (Borkou, Tibesti, Ennedi Est and Ouest) aren’t eligible for SMC, but are sparsely populated. What we concluded

We decided not to include these adjustments in our cost-effectiveness analysis because they increased complexity, without substantially affecting the bottom line.

When we decide whether to include adjustments in our model in general, we use a framework that first takes our best guess of the likely effect size and then rates each of the remaining question on a three-point scale.

Score9We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. jQuery("#footnote_plugin_tooltip_9").tooltip({ tip: "#footnote_plugin_tooltip_text_9", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Commentary Best guess of effect size 3-4% Can it be objectively justified? 3/3 While we have not investigated the MAP data in detail, we would guess that after further investigation, we would conclude it provides a reasonable approximation of subnational malaria incidence.10You can read more about MAP’s methodology in this paper. jQuery("#footnote_plugin_tooltip_10").tooltip({ tip: "#footnote_plugin_tooltip_text_10", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); How easily can it be modelled? 3/3 The methodology is clear and simple. Is it consistent with our other cost-effectiveness analyses? 2/3 We could include subnational adjustments for both of our top charities that implement malaria-prevention programs, but we believe it is unlikely there would be sufficient data to do the same for prevalence of worms or vitamin A deficiency (the focus of five of our other seven top charities).

Even though these adjustments can be objectively justified and are fairly easy to model, the bottom-line difference they make to our cost-effectiveness estimates is insufficient to warrant the (moderate) increase in the complexity of our models. These adjustments would also introduce an inconsistency between our methodologies for top charities. As a result, we are not planning to incorporate subnational adjustments at this time.

When would we revisit this conclusion?

We will revisit using subnational malaria mortality estimates if AMF or Malaria Consortium start working in countries where it would make a large difference to the bottom line. We would include subnational adjustments if AMF contributed nets in any of these countries: Djibouti (+500% adjustment), South Africa (+259%), and Swaziland (+126%), where malaria is endemic in some parts of the country but not others. We would also consider subnational adjustments if AMF contributed nets in Namibia (+25%), Kenya (+23%), Madagascar (+14%), or Rwanda (+10%).11The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_11").tooltip({ tip: "#footnote_plugin_tooltip_text_11", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We will investigate whether subnational adjustments would make a substantial difference if Malaria Consortium enters additional countries; at this time, we do not have details on which regions are eligible for SMC in countries in which Malaria Consortium is not currently operating.12We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. jQuery("#footnote_plugin_tooltip_12").tooltip({ tip: "#footnote_plugin_tooltip_text_12", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

You can read the internal emails discussing our decision process here.

Notes   [ + ]

1. ↑ In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. 2. ↑ We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. 3. ↑ A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. 4. ↑ We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. 5. ↑ See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. 6. ↑ “The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. 7, 11. ↑ The data and calculations are in this spreadsheet. 8. ↑ See Cell C126. 9. ↑ We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. 10. ↑ You can read more about MAP’s methodology in this paper. 12. ↑ We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Why we don’t use subnational malaria mortality estimates in our cost-effectiveness models appeared first on The GiveWell Blog.

Summary

We recently completed a small project to determine whether using subnational baseline malaria mortality estimates would make a difference to our estimates of the cost-effectiveness of two of our top charities, the Against Malaria Foundation and Malaria Consortium. We ultimately decided not to include these adjustments because they added complexity to our models and would require frequent updating, while only making a small difference (a 3-4% improvement) to our bottom line.

Though this post is on a fairly narrow topic, we believe this example illustrates the principles we use to make decisions about what to include in our cost-effectiveness model.

Background

Two of our top charities—the Against Malaria Foundation (AMF) and Malaria Consortium’s seasonal malaria chemoprevention program—implement programs to prevent malaria, a leading killer of people in low- and middle-income countries.

One of the core reasons we recommend AMF and Malaria Consortium is their cost-effectiveness: how much impact they have (e.g., cases of malaria prevented, malaria deaths averted) with the funds they receive. Our estimates of charities’ cost-effectiveness isn’t just helpful to us in determining which charities should be GiveWell top charities; we also rely on these estimates to guide our decisions about how to allocate funding between our top charities.

Our cost-effectiveness estimates for AMF and Malaria Consortium use country-wide data on malaria mortality and malaria incidence in the places that both organizations work.1In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); However, neither organization serves a whole country—rather, they operate in sub-national regions—so the use of country-level estimates could cause us to either underestimate or overestimate their cost-effectiveness. If, for example, these programs are focused in the areas of the country with the highest malaria burden, using the average burden for the country would lead us to underestimate their cost-effectiveness. So, we completed a project to determine how much of an impact using subnational estimates would have, to consider whether we ought to incorporate this information into our cost-effectiveness analysis.

How we estimated the impact of subnational malaria incidence

AMF distributes insecticide-treated nets to prevent malaria; Malaria Consortium’s seasonal malaria chemoprevention (SMC) program provides preventive anti-malarial drugs. We used estimates of subnational malaria incidence from the Malaria Atlas Project (MAP) to see if regions covered by nets or eligible for SMC had higher or lower incidence than the average in the country in which they are located.2We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We focused on all areas covered by nets or eligible for SMC (rather than those covered by our top charities, specifically) for two reasons:

1. Our understanding is that when our top charities contribute resources to a country’s net distribution or SMC programs, the marginal region covered by these additional resources is not necessarily the same as the region to which these resources are assigned (because these resources are fungible with other resources within the national programs).3A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
2. Our aim is to estimate the cost-effectiveness of funds donated to these organizations in the future. The subnational region where AMF has worked in the past has not historically been a good indicator of the region where it will work in future.

Results for net distributions in countries where AMF works

We looked at geographical variation in malaria incidence in countries where AMF works, weighting each region by the number of nets it currently receives.4We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average net delivered in the countries in which AMF works is hung in an area with 0-9% higher malaria incidence than the average in that country, and the weighted average adjustment to AMF’s cost-effectiveness would be 3% (in other words, AMF becomes 3% more cost-effective if we incorporate subnational estimates).5See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Zambia +9% Uganda +4% Ghana +4% Democratic Republic of the Congo +1% Togo +1% Malawi +0% Results for SMC in countries where Malaria Consortium works

We looked at six countries comprising >95% of Malaria Consortium’s SMC spending and compared malaria incidence in districts eligible for SMC with the country-wide average.6“The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });7The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average region eligible for SMC in countries where Malaria Consortium works has -2% to 17% higher malaria incidence than the average in that country. The weighted average adjustment to Malaria Consortium’s cost-effectiveness would be 4%.8See Cell C126. jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Commentary Guinea +17% Conakry, the capital, is ineligible for SMC and has low incidence. Nigeria +12% SMC appears to be targeted in the north, where malaria incidence is slightly higher. Niger +2% The majority of the population is either covered or planned to be covered from 2019. Burkina Faso 0% All districts are eligible. Mali 0% All districts are eligible. Chad -2% The four regions with very low malaria incidence (Borkou, Tibesti, Ennedi Est and Ouest) aren’t eligible for SMC, but are sparsely populated. What we concluded

We decided not to include these adjustments in our cost-effectiveness analysis because they increased complexity, without substantially affecting the bottom line.

When we decide whether to include adjustments in our model in general, we use a framework that first takes our best guess of the likely effect size and then rates each of the remaining question on a three-point scale.

Score9We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. jQuery("#footnote_plugin_tooltip_9").tooltip({ tip: "#footnote_plugin_tooltip_text_9", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Commentary Best guess of effect size 3-4% Can it be objectively justified? 3/3 While we have not investigated the MAP data in detail, we would guess that after further investigation, we would conclude it provides a reasonable approximation of subnational malaria incidence.10You can read more about MAP’s methodology in this paper. jQuery("#footnote_plugin_tooltip_10").tooltip({ tip: "#footnote_plugin_tooltip_text_10", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); How easily can it be modelled? 3/3 The methodology is clear and simple. Is it consistent with our other cost-effectiveness analyses? 2/3 We could include subnational adjustments for both of our top charities that implement malaria-prevention programs, but we believe it is unlikely there would be sufficient data to do the same for prevalence of worms or vitamin A deficiency (the focus of five of our other seven top charities).

Even though these adjustments can be objectively justified and are fairly easy to model, the bottom-line difference they make to our cost-effectiveness estimates is insufficient to warrant the (moderate) increase in the complexity of our models. These adjustments would also introduce an inconsistency between our methodologies for top charities. As a result, we are not planning to incorporate subnational adjustments at this time.

When would we revisit this conclusion?

We will revisit using subnational malaria mortality estimates if AMF or Malaria Consortium start working in countries where it would make a large difference to the bottom line. We would include subnational adjustments if AMF contributed nets in any of these countries: Djibouti (+500% adjustment), South Africa (+259%), and Swaziland (+126%), where malaria is endemic in some parts of the country but not others. We would also consider subnational adjustments if AMF contributed nets in Namibia (+25%), Kenya (+23%), Madagascar (+14%), or Rwanda (+10%).11The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_11").tooltip({ tip: "#footnote_plugin_tooltip_text_11", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We will investigate whether subnational adjustments would make a substantial difference if Malaria Consortium enters additional countries; at this time, we do not have details on which regions are eligible for SMC in countries in which Malaria Consortium is not currently operating.12We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. jQuery("#footnote_plugin_tooltip_12").tooltip({ tip: "#footnote_plugin_tooltip_text_12", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

You can read the internal emails discussing our decision process here.

Notes   [ + ]

1. ↑ In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. 2. ↑ We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. 3. ↑ A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. 4. ↑ We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. 5. ↑ See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. 6. ↑ “The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. 7, 11. ↑ The data and calculations are in this spreadsheet. 8. ↑ See Cell C126. 9. ↑ We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. 10. ↑ You can read more about MAP’s methodology in this paper. 12. ↑ We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Why we don’t use subnational malaria mortality estimates in our cost-effectiveness models appeared first on The GiveWell Blog.

Summary

We recently completed a small project to determine whether using subnational baseline malaria mortality estimates would make a difference to our estimates of the cost-effectiveness of two of our top charities, the Against Malaria Foundation and Malaria Consortium. We ultimately decided not to include these adjustments because they added complexity to our models and would require frequent updating, while only making a small difference (a 3-4% improvement) to our bottom line.

Though this post is on a fairly narrow topic, we believe this example illustrates the principles we use to make decisions about what to include in our cost-effectiveness model.

Background

Two of our top charities—the Against Malaria Foundation (AMF) and Malaria Consortium’s seasonal malaria chemoprevention program—implement programs to prevent malaria, a leading killer of people in low- and middle-income countries.

One of the core reasons we recommend AMF and Malaria Consortium is their cost-effectiveness: how much impact they have (e.g., cases of malaria prevented, malaria deaths averted) with the funds they receive. Our estimates of charities’ cost-effectiveness isn’t just helpful to us in determining which charities should be GiveWell top charities; we also rely on these estimates to guide our decisions about how to allocate funding between our top charities.

Our cost-effectiveness estimates for AMF and Malaria Consortium use country-wide data on malaria mortality and malaria incidence in the places that both organizations work.1In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); However, neither organization serves a whole country—rather, they operate in sub-national regions—so the use of country-level estimates could cause us to either underestimate or overestimate their cost-effectiveness. If, for example, these programs are focused in the areas of the country with the highest malaria burden, using the average burden for the country would lead us to underestimate their cost-effectiveness. So, we completed a project to determine how much of an impact using subnational estimates would have, to consider whether we ought to incorporate this information into our cost-effectiveness analysis.

How we estimated the impact of subnational malaria incidence

AMF distributes insecticide-treated nets to prevent malaria; Malaria Consortium’s seasonal malaria chemoprevention (SMC) program provides preventive anti-malarial drugs. We used estimates of subnational malaria incidence from the Malaria Atlas Project (MAP) to see if regions covered by nets or eligible for SMC had higher or lower incidence than the average in the country in which they are located.2We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We focused on all areas covered by nets or eligible for SMC (rather than those covered by our top charities, specifically) for two reasons:

1. Our understanding is that when our top charities contribute resources to a country’s net distribution or SMC programs, the marginal region covered by these additional resources is not necessarily the same as the region to which these resources are assigned (because these resources are fungible with other resources within the national programs).3A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });
2. Our aim is to estimate the cost-effectiveness of funds donated to these organizations in the future. The subnational region where AMF has worked in the past has not historically been a good indicator of the region where it will work in future.

Results for net distributions in countries where AMF works

We looked at geographical variation in malaria incidence in countries where AMF works, weighting each region by the number of nets it currently receives.4We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average net delivered in the countries in which AMF works is hung in an area with 0-9% higher malaria incidence than the average in that country, and the weighted average adjustment to AMF’s cost-effectiveness would be 3% (in other words, AMF becomes 3% more cost-effective if we incorporate subnational estimates).5See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Zambia +9% Uganda +4% Ghana +4% Democratic Republic of the Congo +1% Togo +1% Malawi +0% Results for SMC in countries where Malaria Consortium works

We looked at six countries comprising >95% of Malaria Consortium’s SMC spending and compared malaria incidence in districts eligible for SMC with the country-wide average.6“The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });7The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

The average region eligible for SMC in countries where Malaria Consortium works has -2% to 17% higher malaria incidence than the average in that country. The weighted average adjustment to Malaria Consortium’s cost-effectiveness would be 4%.8See Cell C126. jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

Country Adjustment Commentary Guinea +17% Conakry, the capital, is ineligible for SMC and has low incidence. Nigeria +12% SMC appears to be targeted in the north, where malaria incidence is slightly higher. Niger +2% The majority of the population is either covered or planned to be covered from 2019. Burkina Faso 0% All districts are eligible. Mali 0% All districts are eligible. Chad -2% The four regions with very low malaria incidence (Borkou, Tibesti, Ennedi Est and Ouest) aren’t eligible for SMC, but are sparsely populated. What we concluded

We decided not to include these adjustments in our cost-effectiveness analysis because they increased complexity, without substantially affecting the bottom line.

When we decide whether to include adjustments in our model in general, we use a framework that first takes our best guess of the likely effect size and then rates each of the remaining question on a three-point scale.

Score9We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. jQuery("#footnote_plugin_tooltip_9").tooltip({ tip: "#footnote_plugin_tooltip_text_9", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); Commentary Best guess of effect size 3-4% Can it be objectively justified? 3/3 While we have not investigated the MAP data in detail, we would guess that after further investigation, we would conclude it provides a reasonable approximation of subnational malaria incidence.10You can read more about MAP’s methodology in this paper. jQuery("#footnote_plugin_tooltip_10").tooltip({ tip: "#footnote_plugin_tooltip_text_10", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); How easily can it be modelled? 3/3 The methodology is clear and simple. Is it consistent with our other cost-effectiveness analyses? 2/3 We could include subnational adjustments for both of our top charities that implement malaria-prevention programs, but we believe it is unlikely there would be sufficient data to do the same for prevalence of worms or vitamin A deficiency (the focus of five of our other seven top charities).

Even though these adjustments can be objectively justified and are fairly easy to model, the bottom-line difference they make to our cost-effectiveness estimates is insufficient to warrant the (moderate) increase in the complexity of our models. These adjustments would also introduce an inconsistency between our methodologies for top charities. As a result, we are not planning to incorporate subnational adjustments at this time.

When would we revisit this conclusion?

We will revisit using subnational malaria mortality estimates if AMF or Malaria Consortium start working in countries where it would make a large difference to the bottom line. We would include subnational adjustments if AMF contributed nets in any of these countries: Djibouti (+500% adjustment), South Africa (+259%), and Swaziland (+126%), where malaria is endemic in some parts of the country but not others. We would also consider subnational adjustments if AMF contributed nets in Namibia (+25%), Kenya (+23%), Madagascar (+14%), or Rwanda (+10%).11The data and calculations are in this spreadsheet. jQuery("#footnote_plugin_tooltip_11").tooltip({ tip: "#footnote_plugin_tooltip_text_11", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We will investigate whether subnational adjustments would make a substantial difference if Malaria Consortium enters additional countries; at this time, we do not have details on which regions are eligible for SMC in countries in which Malaria Consortium is not currently operating.12We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. jQuery("#footnote_plugin_tooltip_12").tooltip({ tip: "#footnote_plugin_tooltip_text_12", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

You can read the internal emails discussing our decision process here.

Notes   [ + ]

1. ↑ In both cases, we rely on reports by Cochrane, an organization that produces systematic reviews and other synthesized research to inform decision-makers. For AMF, we use a decline in all-cause mortality, because the Cochrane review of anti-malarial bed net distributions reports the effect in terms of a reduction in all-cause mortality. For Malaria Consortium, we use a decline in malaria mortality (proxied by a decline in malaria incidence), as the Cochrane review of seasonal malaria chemoprevention reports the effect in terms of a reduction in malaria incidence, but not all-cause mortality. See our cost-effectiveness analysis for more details. 2. ↑ We assume that the regional distribution of malaria incidence is a reasonable proxy for the regional distribution of malaria mortality. 3. ↑ A limitation of this analysis is it does not account for the possibility that AMF and Malaria Consortium are causing locations that are higher priority or lower priority than the average location already covered by nets or eligible for SMC to be covered on the margin. We do not explicitly include estimates of the marginal region funded in our cost-effectiveness analysis because we often have limited information about which regions would be covered with marginal additional funds. 4. ↑ We assume that where nets have been delivered in the past is a good proxy for where new nets will be delivered in the future. The data and calculations are in this spreadsheet. 5. ↑ See Cell J114. We did not include Papua New Guinea (where AMF funds some nets) in this analysis, as MAP only covers countries in Africa. 6. ↑ “The suitability of an area for SMC is determined by the seasonal pattern of rainfall, malaria transmission and the burden of malaria. SMC is recommended for deployment in areas: (i) where more than 60% of the annual incidence of malaria occurs within 4 months (ii) where there are measures of disease burden consistent with a high burden of malaria in children (incidence ≥ 10 cases of malaria among every 100 children during the transmission season) (iii) where SP and AQ [the drugs used to treat children] retain their antimalarial efficacy.” WHO SMC field guide (2013), Pg 8. 7, 11. ↑ The data and calculations are in this spreadsheet. 8. ↑ See Cell C126. 9. ↑ We use these scores as a qualitative guide to help us think through what to include in our cost-effectiveness analysis. You can see the rubric we use to assign scores in this spreadsheet. 10. ↑ You can read more about MAP’s methodology in this paper. 12. ↑ We have not yet prioritized getting details on which regions are eligible for SMC in countries in which Malaria Consortium does not currently work, as this would likely impose a substantial time cost on Malaria Consortium. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Why we don’t use subnational malaria mortality estimates in our cost-effectiveness models appeared first on The GiveWell Blog.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2017 metrics report, which reviews what we know about how our research impacted donors. Please note:

• We report on “metrics years” that run from February through January; for example, our 2017 data cover February 1, 2017 through January 31, 2018.
• We differentiate between our traditional charity recommendations and the work of the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
• More context on the relationships between GiveWell, Good Ventures, and the Open Philanthropy Project can be found here.

Summary of influence: In 2017, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2017, we tracked $117.5 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our nine top charities received the majority of our money moved. Our seven standout charities received a total of $1.8 million.

Money moved by size of donor: In 2017, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2017, 90% of our money moved (excluding Good Ventures) came from 20% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 29% year-over-year to 23,049 in 2017. This included 14,653 donors who gave for the first time. Among all donors who gave in the previous year, about 42% gave again in 2017, up from about 35% who gave again in 2016.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2015. Of larger donors (those who gave $10,000 or more in either of the last two years), about 73% who gave in 2016 gave again in 2017.

GiveWell’s expenses: GiveWell’s total operating expenses in 2017 were $4.6 million. Our expenses decreased from about $5.5 million in 2016 due to the Open Philanthropy Project becoming a separate organization in June 2017. We estimate that 67% of our total expenses ($3.1 million) supported our traditional top charity work and about 33% supported the Open Philanthropy Project. In 2016, we estimated that expenses for our traditional top charity work were about $2.0 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.7 million in unrestricted funding (which we use to support our operations) in 2017, compared to $5.6 million in 2016. Our major institutional supporters and the six largest individual donors contributed about 49% of GiveWell’s operational funding in 2017.

Web traffic: The number of unique visitors to our website remained flat in 2017 compared to 2016 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2017 appeared first on The GiveWell Blog.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2017 metrics report, which reviews what we know about how our research impacted donors. Please note:

• We report on “metrics years” that run from February through January; for example, our 2017 data cover February 1, 2017 through January 31, 2018.
• We differentiate between our traditional charity recommendations and the work of the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
• More context on the relationships between GiveWell, Good Ventures, and the Open Philanthropy Project can be found here.

Summary of influence: In 2017, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2017, we tracked $117.5 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our nine top charities received the majority of our money moved. Our seven standout charities received a total of $1.8 million.

Money moved by size of donor: In 2017, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2017, 90% of our money moved (excluding Good Ventures) came from 20% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 29% year-over-year to 23,049 in 2017. This included 14,653 donors who gave for the first time. Among all donors who gave in the previous year, about 42% gave again in 2017, up from about 35% who gave again in 2016.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2015. Of larger donors (those who gave $10,000 or more in either of the last two years), about 73% who gave in 2016 gave again in 2017.

GiveWell’s expenses: GiveWell’s total operating expenses in 2017 were $4.6 million. Our expenses decreased from about $5.5 million in 2016 due to the Open Philanthropy Project becoming a separate organization in June 2017. We estimate that 67% of our total expenses ($3.1 million) supported our traditional top charity work and about 33% supported the Open Philanthropy Project. In 2016, we estimated that expenses for our traditional top charity work were about $2.0 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.7 million in unrestricted funding (which we use to support our operations) in 2017, compared to $5.6 million in 2016. Our major institutional supporters and the six largest individual donors contributed about 49% of GiveWell’s operational funding in 2017.

Web traffic: The number of unique visitors to our website remained flat in 2017 compared to 2016 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2017 appeared first on The GiveWell Blog.

GiveWell is dedicated to finding outstanding giving opportunities and publishing the full details of our analysis. In addition to evaluations of other charities, we publish substantial evaluation of our own work. This post lays out highlights from our 2017 metrics report, which reviews what we know about how our research impacted donors. Please note:

• We report on “metrics years” that run from February through January; for example, our 2017 data cover February 1, 2017 through January 31, 2018.
• We differentiate between our traditional charity recommendations and the work of the Open Philanthropy Project, which became a separate organization in 2017 and whose work we exclude from this report.
• More context on the relationships between GiveWell, Good Ventures, and the Open Philanthropy Project can be found here.

Summary of influence: In 2017, GiveWell influenced charitable giving in several ways. The following table summarizes our understanding of this influence.

Headline money moved: In 2017, we tracked $117.5 million in money moved to our recommended charities. Our money moved only includes donations that we are confident were influenced by our recommendations.

Money moved by charity: Our nine top charities received the majority of our money moved. Our seven standout charities received a total of $1.8 million.

Money moved by size of donor: In 2017, the number of donors and amount donated increased across each donor size category, with the notable exception of donations from donors giving $1,000,000 or more. In 2017, 90% of our money moved (excluding Good Ventures) came from 20% of our donors, who gave $1,000 or more.

Donor retention: The total number of donors who gave to our recommended charities or to GiveWell unrestricted increased about 29% year-over-year to 23,049 in 2017. This included 14,653 donors who gave for the first time. Among all donors who gave in the previous year, about 42% gave again in 2017, up from about 35% who gave again in 2016.

Our retention was stronger among donors who gave larger amounts or who first gave to our recommendations prior to 2015. Of larger donors (those who gave $10,000 or more in either of the last two years), about 73% who gave in 2016 gave again in 2017.

GiveWell’s expenses: GiveWell’s total operating expenses in 2017 were $4.6 million. Our expenses decreased from about $5.5 million in 2016 due to the Open Philanthropy Project becoming a separate organization in June 2017. We estimate that 67% of our total expenses ($3.1 million) supported our traditional top charity work and about 33% supported the Open Philanthropy Project. In 2016, we estimated that expenses for our traditional top charity work were about $2.0 million.

Donations supporting GiveWell’s operations: GiveWell raised $5.7 million in unrestricted funding (which we use to support our operations) in 2017, compared to $5.6 million in 2016. Our major institutional supporters and the six largest individual donors contributed about 49% of GiveWell’s operational funding in 2017.

Web traffic: The number of unique visitors to our website remained flat in 2017 compared to 2016 (when excluding visitors driven by AdWords, Google’s online advertising product).

For more detail, see our full metrics report (PDF).

The post GiveWell’s money moved and web traffic in 2017 appeared first on The GiveWell Blog.

We’ve added the Georgetown University Initiative on Innovation, Development, and Evaluation (gui2de)'s Zusha! Road Safety Campaign (from here on, "Zusha!") as a standout charity; see our full review here. Standout charities do not meet all of our criteria to be a GiveWell top charity, but we believe they stand out from the vast majority of organizations we have considered. See more information about our standout charities here.

Read More

The post Announcing Zusha! as a standout charity appeared first on The GiveWell Blog.

We’ve added the Georgetown University Initiative on Innovation, Development, and Evaluation gui2de‘s Zusha! Road Safety Campaign (from here on, “Zusha!”) as a standout charity; see our full review here. Standout charities do not meet all of our criteria to be a GiveWell top charity, but we believe they stand out from the vast majority of organizations we have considered. See more information about our standout charities here.

Zusha! is a campaign intended to reduce road accidents. Zusha! supports distribution of stickers to public service vehicles encouraging passengers to speak up and urge drivers to drive more safely. We provided a GiveWell Incubation Grant to Zusha! in January 2017 and discussed it in a February 2017 blog post.

For more information, see our full review. Interested donors can give to Zusha! by clicking “Donate” on that page.

The post Announcing Zusha! as a standout charity appeared first on The GiveWell Blog.

We’ve added the Georgetown University Initiative on Innovation, Development, and Evaluation gui2de‘s Zusha! Road Safety Campaign (from here on, “Zusha!”) as a standout charity; see our full review here. Standout charities do not meet all of our criteria to be a GiveWell top charity, but we believe they stand out from the vast majority of organizations we have considered. See more information about our standout charities here.

Zusha! is a campaign intended to reduce road accidents. Zusha! supports distribution of stickers to public service vehicles encouraging passengers to speak up and urge drivers to drive more safely. We provided a GiveWell Incubation Grant to Zusha! in January 2017 and discussed it in a February 2017 blog post.

For more information, see our full review. Interested donors can give to Zusha! by clicking “Donate” on that page.

The post Announcing Zusha! as a standout charity appeared first on The GiveWell Blog.

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our March 2018 open thread here.

The post June 2018 open thread appeared first on The GiveWell Blog.

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our March 2018 open thread here.

The post June 2018 open thread appeared first on The GiveWell Blog.

In the first quarter of 2018, we received $2.96 million in funding for making grants at our discretion. In this post we discuss:

• The decision to allocate the $2.96 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.

Allocation of discretionary funds

The allocation of 70 percent of the funds to AMF and 30 percent to SCI follows the recommendation we have made, and continue to make, to donors. For more discussion on this allocation, see our blog post about allocating discretionary funds from the previous quarter.

We also considered the following possibilities for this quarter:

Helen Keller International (HKI) for stopgap funding in one additional country

We discussed this possibility in our blog post about allocating discretionary funds from the previous quarter. After further discussing this possibility with HKI, our understanding is that (a) the amount of funding needed to fill this gap will likely be small relative to the amount of GiveWell-directed funding that HKI currently holds, and (b) we will have limited additional information in time for this decision round that we could use to compare this new use of funding to HKI’s other planned uses of funding. We will continue discussing this opportunity with HKI and may allocate funding to it in the future. Our current expectation is that we will ask HKI to make the tradeoff between allocating the GiveWell-directed funding it holds to this new opportunity and continuing to hold the funds. Holding the funds gives the current programs more runway (originally designed to fund three years) and gives HKI more flexibility to fund highly cost-effective, unanticipated opportunities in the future. We believe that HKI is currently in a better position to assess cost-effectiveness of the opportunities it has than we are, while we will seek to maximize cost-effectiveness in the longer run by assessing HKI’s track record of cost-effectiveness and comparing that to the cost-effectiveness of other top charities.

We remain open to the possibility that HKI will share information with us that will lead us to conclude that this new opportunity is a better use of funds than our current recommendation of 70 percent to AMF and 30 percent to SCI. In that case, we would allocate funds from the next quarter to fill this funding gap (and could accelerate the timeline on that decision if it were helpful to HKI).

Evidence Action’s Deworm the World Initiative for funding gaps in India and Nigeria

We spoke with Deworm the World about two new funding gaps it has due to unexpected costs in its existing programs in India and Nigeria.

In India, the cost overruns total $166,000. Deworm the World has the option of drawing down a reserve of $5.5 million (from funds donated on GiveWell’s recommendation). The reserve was intended to backstop funds that were expected but not fully confirmed from another funder. Given the small size of the gap relative to the available reserves, our preference is for Deworm the World to use that funding and for us to consider recommending further reserves as part of our end-of-year review of our top charities’ room for more funding.

In Nigeria, there is a funding gap of $1.7 million in the states that Deworm the World is currently operating in. Previous budgets assumed annual treatment for all children, and Deworm the World has since become aware of the existence of areas where worm prevalence is high enough that twice per year treatment is recommended. Our best guess is that AMF and SCI are more cost-effective than Deworm the World’s Nigeria program (see discussion in this post). It is possible that because additional funding would go to support additional treatments in states where programs already operate, the cost to deliver these marginal treatments would be lower. We don’t currently have enough data to analyze whether that would significantly change the cost-effectiveness in this case.

Deworm the World also continues to have a funding gap for expansion to other states in Nigeria. We wrote about this opportunity in our previous post on allocating discretionary funding.

Malaria Consortium for seasonal malaria chemoprevention (SMC)

We continue to see a case for directing additional funding to Malaria Consortium for SMC, as we did last quarter. Our views on this program have not changed. For further discussion, see our previous post on allocating discretionary funding.

What is our recommendation to donors?

We continue to recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we are continuing to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact. The reasons for this recommendation are the same as in our previous post on allocating discretionary funding.

The post Allocation of discretionary funds from Q1 2018 appeared first on The GiveWell Blog.

In the first quarter of 2018, we received $2.96 million in funding for making grants at our discretion. In this post we discuss:

• The decision to allocate the $2.96 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.

Read More

The post Allocation of discretionary funds from Q1 2018 appeared first on The GiveWell Blog.

In the first quarter of 2018, we received $2.96 million in funding for making grants at our discretion. In this post we discuss:

• The decision to allocate the $2.96 million to the Against Malaria Foundation (AMF) (70 percent) and the Schistosomiasis Control Initiative (SCI) (30 percent).
• Our recommendation that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we continue to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact.

Allocation of discretionary funds

The allocation of 70 percent of the funds to AMF and 30 percent to SCI follows the recommendation we have made, and continue to make, to donors. For more discussion on this allocation, see our blog post about allocating discretionary funds from the previous quarter.

We also considered the following possibilities for this quarter:

Helen Keller International (HKI) for stopgap funding in one additional country

We discussed this possibility in our blog post about allocating discretionary funds from the previous quarter. After further discussing this possibility with HKI, our understanding is that (a) the amount of funding needed to fill this gap will likely be small relative to the amount of GiveWell-directed funding that HKI currently holds, and (b) we will have limited additional information in time for this decision round that we could use to compare this new use of funding to HKI’s other planned uses of funding. We will continue discussing this opportunity with HKI and may allocate funding to it in the future. Our current expectation is that we will ask HKI to make the tradeoff between allocating the GiveWell-directed funding it holds to this new opportunity and continuing to hold the funds. Holding the funds gives the current programs more runway (originally designed to fund three years) and gives HKI more flexibility to fund highly cost-effective, unanticipated opportunities in the future. We believe that HKI is currently in a better position to assess cost-effectiveness of the opportunities it has than we are, while we will seek to maximize cost-effectiveness in the longer run by assessing HKI’s track record of cost-effectiveness and comparing that to the cost-effectiveness of other top charities.

We remain open to the possibility that HKI will share information with us that will lead us to conclude that this new opportunity is a better use of funds than our current recommendation of 70 percent to AMF and 30 percent to SCI. In that case, we would allocate funds from the next quarter to fill this funding gap (and could accelerate the timeline on that decision if it were helpful to HKI).

Evidence Action’s Deworm the World Initiative for funding gaps in India and Nigeria

We spoke with Deworm the World about two new funding gaps it has due to unexpected costs in its existing programs in India and Nigeria.

In India, the cost overruns total $166,000. Deworm the World has the option of drawing down a reserve of $5.5 million (from funds donated on GiveWell’s recommendation). The reserve was intended to backstop funds that were expected but not fully confirmed from another funder. Given the small size of the gap relative to the available reserves, our preference is for Deworm the World to use that funding and for us to consider recommending further reserves as part of our end-of-year review of our top charities’ room for more funding.

In Nigeria, there is a funding gap of $1.7 million in the states that Deworm the World is currently operating in. Previous budgets assumed annual treatment for all children, and Deworm the World has since become aware of the existence of areas where worm prevalence is high enough that twice per year treatment is recommended. Our best guess is that AMF and SCI are more cost-effective than Deworm the World’s Nigeria program (see discussion in this post). It is possible that because additional funding would go to support additional treatments in states where programs already operate, the cost to deliver these marginal treatments would be lower. We don’t currently have enough data to analyze whether that would significantly change the cost-effectiveness in this case.

Deworm the World also continues to have a funding gap for expansion to other states in Nigeria. We wrote about this opportunity in our previous post on allocating discretionary funding.

Malaria Consortium for seasonal malaria chemoprevention (SMC)

We continue to see a case for directing additional funding to Malaria Consortium for SMC, as we did last quarter. Our views on this program have not changed. For further discussion, see our previous post on allocating discretionary funding.

What is our recommendation to donors?

We continue to recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we are continuing to recommend giving 70 percent of your donation to AMF and 30 percent to SCI to maximize your impact. The reasons for this recommendation are the same as in our previous post on allocating discretionary funding.

The post Allocation of discretionary funds from Q1 2018 appeared first on The GiveWell Blog.