Aggregator

In the second quarter of 2019, donors gave a combined $2.3 million to GiveWell for granting to recommended charities at our discretion. We greatly appreciate this support, which enables us to direct funding where we believe it can be used most effectively. We grant this funding to one or more of our top charities each quarter.

We decided to allocate all $2.3 million to the Against Malaria Foundation (AMF). AMF is a GiveWell top charity that provides support for the distribution of long-lasting insecticide-treated nets to prevent malaria. AMF has been named a GiveWell top charity seven times. We chose to allocate the second-quarter funding to AMF because we believe AMF has a highly cost-effective and time-sensitive opportunity to spend it.

Our bottom line

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to AMF, which we believe could use additional funding for highly cost-effective work, even after receiving the $2.3 million in funding mentioned above.

Summary

In this post, we discuss:

• what AMF will do with additional funding. (More)
• other possibilities we considered. (More)
• our process for deciding where to allocate funds. (More)

What will AMF do with additional funding?

AMF told us that it will use additional funding to support a distribution of nets scheduled for 2020 in the Democratic Republic of the Congo (DRC). Distributions are often delayed by a few months. Our best guess is that these nets will be delivered in late 2020 or in 2021.

DRC has a higher malaria burden than most of the other countries where AMF supports distributions. We model AMF’s work in DRC to be more than 1.5 times as cost-effective as AMF’s past work, on average—we estimate that a donation of roughly $2,000 to support work in DRC will avert a death, compared to $3,600 for AMF’s work overall.1You can see our calculations by making a copy of our 2019 version 4 cost-effectiveness model; this will enable you to edit the sheet and change the values in the drop-down menu as described below:
1. Our estimate of the cost-effectiveness of AMF’s work in general: Go to the “Nets” tab. In cell B125, you’ll see the “Median cost per death averted (after accounting for leverage and funging)” for AMF. The value is $3,554.
2. Go to the “Country selection” tab, change the value for the Against Malaria Foundation in cell B7 from “Overall” (which includes all countries AMF works in) to “DRC” on the drop-down menu.
3. Our estimate of the cost-effectiveness of AMF’s work in DRC: Go back to the “Nets” tab. The value in cell B125 is now $2,072.
4. $3,554 divided by $2,072 = ~1.7.

jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

We consider this to be the most promising funding need among our top charities, in terms of timeliness and cost-effectiveness. Our process for comparing top charities’ needs each quarter is described in greater detail below.

Open questions and uncertainties

Although we see this as a very promising opportunity, we are somewhat unsure how AMF will actually allocate the funding it receives. AMF’s role in net distributions is to:

1. identify countries with funding gaps (funding needs that aren’t otherwise expected to be met) for nets;
2. find distribution partners (in-country non-profit organizations or government agencies) to carry out the distributions;
3. purchase nets; and
4. work with distribution partners to monitor the distribution and use of nets.

While AMF has told us that it will allocate additional funding to DRC, it is possible that AMF will deviate from its funding plans in the face of changing circumstances, primarily changes in the status of discussions with governments and changes in the amount of funding it has available to allocate. The most common changes in AMF’s plans in recent years have been (a) delays in distributions, often due to governments taking longer to sign agreements than AMF had originally estimated, and (b) changes in the quantity of nets purchased by AMF due to larger population numbers being found during registration than the government had estimated at earlier stages of planning.

According to AMF, the total funding gap in DRC over the next two years (2020-2021) is $55 million. In addition to its plans to fund work in DRC, AMF currently holds $39 million to fund distributions in three other countries. Although AMF is in discussions about funding these distributions, it has not yet signed formal agreements to do so. If any of the discussions fall through, we expect AMF to reallocate the funding it has set aside. In addition, AMF raised $38 million in 2018,2More specifically, this is AMF’s total revenue between February 1, 2018 and January 31, 2019, which is GiveWell’s 2018 “metrics year.” jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); and we estimate that AMF will continue to raise at least half of that amount annually, independent of whether GiveWell allocates additional discretionary funding to AMF. This suggests that AMF will raise enough funding in the next year to substantially reduce the size of the funding gap in DRC, though the timing of when funding is received may affect the timing of distributions. If AMF fully fills the DRC funding gap, it seems intuitively likely that there would be other bottlenecks that might impede its progress, such as ability to find partner organizations with the capacity to implement the distributions and fulfill AMF’s reporting and monitoring requirements. We do not know where or when AMF would choose to fund nets if it had more funding than it could allocate to DRC in 2019 to 2021.

We incorporate our uncertainty about where AMF will use additional funding into our cost-effectiveness estimate of its work. When we made our first-quarter discretionary funding allocation, which also went to AMF, we modeled an 87 percent chance of AMF’s additional funding supporting nets in DRC.3This model has not been published. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); As we considered where to grant second-quarter discretionary funding, we made a minor downward adjustment to 75 percent due to AMF’s continued lack of signed agreements with other countries and thus our greater uncertainty over how funds will be spent. Even with this uncertainty incorporated, we model AMF’s funding gap in DRC as a highly cost-effective opportunity.

Other possibilities we considered

Malaria Consortium’s seasonal malaria chemoprevention (SMC) program

When we granted discretionary funding we received in the first quarter, we focused on AMF and another top charity, Malaria Consortium’s SMC program, as the most promising recipients. Our decision centered on our comparison of the two organizations:

• Against Malaria Foundation
  • We modeled additional funding to AMF as more cost-effective than additional funding to Malaria Consortium’s SMC program. Our best guess, which we did not subject to our formal internal review process, was that AMF was 38 percent more cost-effective than Malaria Consortium’s SMC program.
  • We viewed AMF’s funding gap in DRC as time-sensitive because our expectation is that AMF receiving funding now will allow it to distribute nets sooner than if it receives the same amount of funding later this year.
• Malaria Consortium’s SMC program
  • We viewed Malaria Consortium’s SMC program as likely to have more overall impact per dollar based on unmodeled qualitative factors described in “Principle 2” here.
  • We did not expect that directing additional funding to Malaria Consortium would influence its spending on 2019 and 2020 programs—in other words, we didn’t see providing funding to Malaria Consortium as being particularly time-sensitive.

Weighing these factors, we ultimately chose AMF over Malaria Consortium based on its somewhat higher modeled cost-effectiveness and more time-sensitive funding need.

We now model additional funding to AMF as roughly 33 percent more cost-effective than additional funding to Malaria Consortium’s SMC program, as a result of adjusting the chance of additional funding supporting nets in DRC from 87 percent to 75 percent. We have not received any new information to update us on the time sensitivity of Malaria Consortium’s funding needs, and we continue to view Malaria Consortium as stronger than AMF on unmodeled qualitative factors.

We don’t view the comparison of the two organizations as meaningfully different than in the previous quarter, and we thus chose to prioritize AMF over Malaria Consortium again.

Other top charities

As far as we know, our six other top charities have not had any major changes in their funding needs or cost-effectiveness since March. We did not update our cost-effectiveness model since making our last quarterly allocation decision, nor did we receive any updates on our top charities’ room for more funding, beyond the $4.7 million in first-quarter discretionary funds that we allocated to AMF.

Process for deciding where to allocate funds

We follow the principles described in this blog post when deciding between funding opportunities.

We ask our top charities to alert us throughout the year if they learn of any new funding opportunities that we should consider in our discretionary funding decisions. None of our top charities informed us of such an opportunity for second-quarter funding.

With no new funding opportunities presented to us, we returned to our first-quarter funding recipient, AMF. When we granted first-quarter funding to AMF, we noted that AMF had a time-sensitive and cost-effective funding opportunity in DRC and a funding gap that was much larger than we were able to fill. As we considered where to allocate second-quarter funding, we asked AMF for information to help us assess whether that continued to be true. We asked AMF about its progress in signing net-distribution agreements, its ability to absorb additional funding for work in DRC, and whether additional funding sent to AMF in the next few months would contribute to filling the funding gap in DRC.

Notes   [ + ]

1. ↑ You can see our calculations by making a copy of our 2019 version 4 cost-effectiveness model; this will enable you to edit the sheet and change the values in the drop-down menu as described below:
1. Our estimate of the cost-effectiveness of AMF’s work in general: Go to the “Nets” tab. In cell B125, you’ll see the “Median cost per death averted (after accounting for leverage and funging)” for AMF. The value is $3,554.
2. Go to the “Country selection” tab, change the value for the Against Malaria Foundation in cell B7 from “Overall” (which includes all countries AMF works in) to “DRC” on the drop-down menu.
3. Our estimate of the cost-effectiveness of AMF’s work in DRC: Go back to the “Nets” tab. The value in cell B125 is now $2,072.
4. $3,554 divided by $2,072 = ~1.7.

2. ↑ More specifically, this is AMF’s total revenue between February 1, 2018 and January 31, 2019, which is GiveWell’s 2018 “metrics year.” 3. ↑ This model has not been published. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Allocation of discretionary funds from Q2 2019 appeared first on The GiveWell Blog.

One of our top priorities is to increase the amount of money we direct to our recommendations. As part of our effort to do this, we’re planning to try new kinds of communication. We hope to reach people who haven’t heard of or connected with GiveWell in the past, and to increase retention of our current donors by making the experience of donating through GiveWell more compelling.

We are experimenting on our homepage and in emails with using images and making our cost-effectiveness estimates more prominent. Our goal is to improve people’s connection to our work without compromising the accuracy of what we share.

There are potential downsides to this approach. We expect to balance our goal of communicating in a way that is emotionally compelling with our commitment to honesty and not misleading donors or overstating the case for our recommendations.

We’re not planning a major overhaul of GiveWell’s website or other communications in the near term, and we are unsure if we will make major changes in the future. Most of GiveWell’s communications will look as they always have. Our hope in the coming months is to learn whether there are new ways we can communicate about our work to increase our impact. We’re writing this post to share with you the context behind these experiments.

Summary

In this post, we discuss:

• Our communication experiments. (More)
• Challenges and potential downsides of our approach. (More)
• How you can help us improve. (More)

Our communication experiments

We’re initially experimenting with using images and emphasizing cost-effectiveness information in our communications. We selected these experiments based on our intuition, our understanding of best practices in the nonprofit sector, and the feedback we’ve received from GiveWell’s donors and others.

Over the years, we’ve heard from a number of our supporters that they wish GiveWell had more emotionally-oriented content so that they could more easily share GiveWell with their peers or feel more connected to their own gifts. We also understand that most charity fundraisers make emotional appeals tied to specific individuals or projects. Fundraisers may use cost figures to promote their causes, although these figures can be misleading (for example, claiming you can save a child’s life by donating $0.50—we estimate that even a very cost-effective program requires closer to $2,400 to avert the death of a child).

Taking these considerations into account, we want to move in the direction of sharing content that people can easily connect to without sacrificing honesty and accuracy. We plan to make these changes gradually and to see what works before committing to a long-term path. We’re starting by making the following changes to our homepage and certain email content we share:

1. Adding images. We believe we can create a closer connection to our top charities by showing pictures of the work they do—either to illustrate how the program is carried out or to show the people they have helped. We want to do this respectfully.

2. Featuring our cost-effectiveness figures more prominently. Although we don’t advise taking our cost-effectiveness estimates literally, we do think they are one of the best ways we can communicate about the rough magnitude of expected impact of donations to our recommended charities.

   A few years ago, we decided not to feature our cost-effectiveness estimates prominently on our website. We had seen people using our estimates to make claims about the precise cost to save a life that lost the nuances of our analysis; it seemed they were understandably misinterpreting concrete numbers as conveying more certainty than we have. After seeing this happen repeatedly, we chose to deemphasize these figures. We continued to publish them but did not feature them prominently.

   Over the past few years, we have incorporated more factors into our cost-effectiveness model and increased the amount of weight we place on its outputs in our reviews (see the contrast between our 2014 cost-effectiveness model versus our latest one). We thus see our cost-effectiveness estimates as important and informative.

   We also think they offer a compelling motivation to donate. We aim to share these estimates in such a way that it’s reasonably easy for anyone who wants to dig into the numbers to understand all of the nuances involved.

We’ve chosen two places to run our initial experiments with using images and emphasizing cost-effectiveness estimates: GiveWell’s homepage and certain email content.

Homepage updates

Our intuition is that someone spending a few minutes on GiveWell’s homepage would not come away with a clear understanding of what GiveWell does or an emotional connection to our work. We hypothesize that adding images, illustrations, and cost-effectiveness estimates will help new visitors better understand and connect to GiveWell’s work. We are also planning to link to a citations page that provides sources for the calculations we use on the homepage and enables readers to easily access the details of our research if they want to vet or understand our claims.

We plan to test two new versions of the homepage this summer; you can see preliminary versions here and here. We’ll be testing these pages against each other and our current homepage. We expect to update our homepage pending the results of this experiment—we’ll likely be looking at visits to our top charities page from the homepage, the bounce rate on the homepage (the percentage of visitors who leave the page without going to other parts of our website), and the duration of time spent on the homepage.

Impact emails

We hypothesize that drawing clearer connections between our donors’ support and what it enables charities to achieve will increase retention of GiveWell’s donors. We think doing so will make the experience of giving through GiveWell more meaningful and memorable. One way we think we can do this is by reporting to donors what we expect the impact of their gifts to be.

Information about impact has always been available on our website via our cost-effectiveness model, but it has neither been linked to individual donation amounts nor sent directly to donors. Up until recently, if a GiveWell donor was interested in the impact of their gift, they’d have to track down the relevant part of our cost-effectiveness model and do their own calculation.

Now, we’re experimenting with more proactively sharing this information. In the fall, we sent an email to a group of our donors who gave to the Against Malaria Foundation (AMF), a GiveWell top charity that distributes insecticide-treated nets to prevent malaria. This email explained how AMF works, using photographs of net distributions, and included our best estimate of the impact each individual’s donation would have, in terms of the nets purchased and deaths averted. You can see an example of this email here. We ended this experiment after a few weeks due to a technical glitch.

This year, we piloted sending an email to donors who supported “Grants to recommended charities at GiveWell’s discretion.” We grant these discretionary funds each quarter to the GiveWell top charity or charities that we believe have the most pressing funding needs. When we made these grants in 2019, we sent an email to donors who contributed to the discretionary funds. The email announced where we chose to grant the discretionary funds and why, along with a description of the charity that received the funds—including images of its work—and a calculation of each donor’s expected impact based on our cost-effectiveness analysis. You can see an example of this email here.

Anecdotally, these emails have been positively received. Over a dozen recipients of the “Grants to recommended charities” emails have contacted us (unprompted) to let us know they appreciated the information. We do not yet feel confident in extrapolating the impact of these emails on donor retention, as most donors give on an annual basis. We plan to continue sending these emails each quarter when we decide where to grant discretionary funds and to assess over the long term whether they impact donor retention.

Challenges and potential downsides

A major challenge we face with this project is striking the right balance between communicating clearly, creating a connection to our work, and honoring our values. We anticipate that:

1. the use of images could fail to treat beneficiary populations with the respect they deserve (for a discussion of some simplistic narratives about the relationship between donors and beneficiaries, see this blog post). We plan to be particularly careful about our selection of images and avoid depictions that do not respect the dignity of our beneficiaries.
2. the use of images might make our marketing harder to distinguish from typical charity outreach.
3. the use of cost-effectiveness figures may make it harder to distinguish GiveWell’s carefulness (the hundreds of hours our researchers collectively spend per year on cost-effectiveness analysis) from charities’ often unjustified claims about cost per impact.
4. the use of cost-effectiveness figures may cause donors to take these estimates literally rather than as a rough sense of the magnitude of expected impact of donations to our recommended charities. To mitigate this and item (3) above, we plan to make links to the detailed analysis behind our cost-effectiveness figures readily available.

The upside of moving more money to top charities and increasing donors’ engagement with our work seems worth tackling this challenge and its potential downsides.

How you can help us

We’re planning to move relatively slowly in this direction and adjust our actions based on the feedback we receive. If you have feedback about how our new communications are changing your view of the GiveWell brand (positively or negatively), please let us know by emailing info@givewell.org.

We’re excited to be on this new path and hopeful it will lead to more funding for our top charities.

The post Experiments in GiveWell communication appeared first on The GiveWell Blog.

One of our top priorities is to increase the amount of money we direct to our recommendations. As part of our effort to do this, we’re planning to try new kinds of communication. We hope to reach people who haven’t heard of or connected with GiveWell in the past, and to increase retention of our current donors by making the experience of donating through GiveWell more compelling.

We are experimenting on our homepage and in emails with using images and making our cost-effectiveness estimates more prominent. Our goal is to improve people’s connection to our work without compromising the accuracy of what we share.

There are potential downsides to this approach. We expect to balance our goal of communicating in a way that is emotionally compelling with our commitment to honesty and not misleading donors or overstating the case for our recommendations.

We’re not planning a major overhaul of GiveWell’s website or other communications in the near term, and we are unsure if we will make major changes in the future. Most of GiveWell’s communications will look as they always have. Our hope in the coming months is to learn whether there are new ways we can communicate about our work to increase our impact. We’re writing this post to share with you the context behind these experiments.

Summary

In this post, we discuss:

• Our communication experiments. (More)
• Challenges and potential downsides of our approach. (More)
• How you can help us improve. (More)

Our communication experiments

We’re initially experimenting with using images and emphasizing cost-effectiveness information in our communications. We selected these experiments based on our intuition, our understanding of best practices in the nonprofit sector, and the feedback we’ve received from GiveWell’s donors and others.

Over the years, we’ve heard from a number of our supporters that they wish GiveWell had more emotionally-oriented content so that they could more easily share GiveWell with their peers or feel more connected to their own gifts. We also understand that most charity fundraisers make emotional appeals tied to specific individuals or projects. Fundraisers may use cost figures to promote their causes, although these figures can be misleading (for example, claiming you can save a child’s life by donating $0.50—we estimate that even a very cost-effective program requires closer to $2,400 to avert the death of a child).

Taking these considerations into account, we want to move in the direction of sharing content that people can easily connect to without sacrificing honesty and accuracy. We plan to make these changes gradually and to see what works before committing to a long-term path. We’re starting by making the following changes to our homepage and certain email content we share:

1. Adding images. We believe we can create a closer connection to our top charities by showing pictures of the work they do—either to illustrate how the program is carried out or to show the people they have helped. We want to do this respectfully.

2. Featuring our cost-effectiveness figures more prominently. Although we don’t advise taking our cost-effectiveness estimates literally, we do think they are one of the best ways we can communicate about the rough magnitude of expected impact of donations to our recommended charities.

   A few years ago, we decided not to feature our cost-effectiveness estimates prominently on our website. We had seen people using our estimates to make claims about the precise cost to save a life that lost the nuances of our analysis; it seemed they were understandably misinterpreting concrete numbers as conveying more certainty than we have. After seeing this happen repeatedly, we chose to deemphasize these figures. We continued to publish them but did not feature them prominently.

   Over the past few years, we have incorporated more factors into our cost-effectiveness model and increased the amount of weight we place on its outputs in our reviews (see the contrast between our 2014 cost-effectiveness model versus our latest one). We thus see our cost-effectiveness estimates as important and informative.

   We also think they offer a compelling motivation to donate. We aim to share these estimates in such a way that it’s reasonably easy for anyone who wants to dig into the numbers to understand all of the nuances involved.

We’ve chosen two places to run our initial experiments with using images and emphasizing cost-effectiveness estimates: GiveWell’s homepage and certain email content.

Homepage updates

Our intuition is that someone spending a few minutes on GiveWell’s homepage would not come away with a clear understanding of what GiveWell does or an emotional connection to our work. We hypothesize that adding images, illustrations, and cost-effectiveness estimates will help new visitors better understand and connect to GiveWell’s work. We are also planning to link to a citations page that provides sources for the calculations we use on the homepage and enables readers to easily access the details of our research if they want to vet or understand our claims.

We plan to test two new versions of the homepage this summer; you can see preliminary versions here and here. We’ll be testing these pages against each other and our current homepage. We expect to update our homepage pending the results of this experiment—we’ll likely be looking at visits to our top charities page from the homepage, the bounce rate on the homepage (the percentage of visitors who leave the page without going to other parts of our website), and the duration of time spent on the homepage.

Impact emails

We hypothesize that drawing clearer connections between our donors’ support and what it enables charities to achieve will increase retention of GiveWell’s donors. We think doing so will make the experience of giving through GiveWell more meaningful and memorable. One way we think we can do this is by reporting to donors what we expect the impact of their gifts to be.

Information about impact has always been available on our website via our cost-effectiveness model, but it has neither been linked to individual donation amounts nor sent directly to donors. Up until recently, if a GiveWell donor was interested in the impact of their gift, they’d have to track down the relevant part of our cost-effectiveness model and do their own calculation.

Now, we’re experimenting with more proactively sharing this information. In the fall, we sent an email to a group of our donors who gave to the Against Malaria Foundation (AMF), a GiveWell top charity that distributes insecticide-treated nets to prevent malaria. This email explained how AMF works, using photographs of net distributions, and included our best estimate of the impact each individual’s donation would have, in terms of the nets purchased and deaths averted. You can see an example of this email here. We ended this experiment after a few weeks due to a technical glitch.

This year, we piloted sending an email to donors who supported “Grants to recommended charities at GiveWell’s discretion.” We grant these discretionary funds each quarter to the GiveWell top charity or charities that we believe have the most pressing funding needs. When we made these grants in 2019, we sent an email to donors who contributed to the discretionary funds. The email announced where we chose to grant the discretionary funds and why, along with a description of the charity that received the funds—including images of its work—and a calculation of each donor’s expected impact based on our cost-effectiveness analysis. You can see an example of this email here.

Anecdotally, these emails have been positively received. Over a dozen recipients of the “Grants to recommended charities” emails have contacted us (unprompted) to let us know they appreciated the information. We do not yet feel confident in extrapolating the impact of these emails on donor retention, as most donors give on an annual basis. We plan to continue sending these emails each quarter when we decide where to grant discretionary funds and to assess over the long term whether they impact donor retention.

Challenges and potential downsides

A major challenge we face with this project is striking the right balance between communicating clearly, creating a connection to our work, and honoring our values. We anticipate that:

1. the use of images could fail to treat beneficiary populations with the respect they deserve (for a discussion of some simplistic narratives about the relationship between donors and beneficiaries, see this blog post). We plan to be particularly careful about our selection of images and avoid depictions that do not respect the dignity of our beneficiaries.
2. the use of images might make our marketing harder to distinguish from typical charity outreach.
3. the use of cost-effectiveness figures may make it harder to distinguish GiveWell’s carefulness (the hundreds of hours our researchers collectively spend per year on cost-effectiveness analysis) from charities’ often unjustified claims about cost per impact.
4. the use of cost-effectiveness figures may cause donors to take these estimates literally rather than as a rough sense of the magnitude of expected impact of donations to our recommended charities. To mitigate this and item (3) above, we plan to make links to the detailed analysis behind our cost-effectiveness figures readily available.

The upside of moving more money to top charities and increasing donors’ engagement with our work seems worth tackling this challenge and its potential downsides.

How you can help us

We’re planning to move relatively slowly in this direction and adjust our actions based on the feedback we receive. If you have feedback about how our new communications are changing your view of the GiveWell brand (positively or negatively), please let us know by emailing info@givewell.org.

We’re excited to be on this new path and hopeful it will lead to more funding for our top charities.

The post Experiments in GiveWell communication appeared first on The GiveWell Blog.

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our March 2019 open thread here.

The post June 2019 open thread appeared first on The GiveWell Blog.

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our March 2019 open thread here.

The post June 2019 open thread appeared first on The GiveWell Blog.

In the first quarter of 2019, donors gave a combined $4.7 million for granting to recommended charities at our discretion.

We really appreciate the generosity of our supporters in making it possible for us to regularly allocate funding to the top charity or charities that we believe can best use additional funding. Thank you!

In this post, we discuss our decision to allocate this $4.7 million to the Against Malaria Foundation (AMF), as well as the process we followed to arrive at this decision.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to AMF, which we believe could use additional funding for highly cost-effective work, even after receiving the $4.7 million in funding mentioned above.

Our bottom line

As we did last quarter, we focused our efforts on deciding between allocating funding to Malaria Consortium vs. AMF. We currently believe that AMF has a more time sensitive funding need than Malaria Consortium, and our best guess is that it will have equivalent impact per dollar to Malaria Consortium. This led us to allocate funding to AMF.

What changed since last quarter?

In March 2019, we modeled AMF as somewhat more cost-effective than Malaria Consortium’s seasonal malaria chemoprevention (SMC) program, believed that both organizations had time-sensitive funding opportunities, but believed that Malaria Consortium would have more overall impact per dollar, when taking into account unmodeled qualitative factors (see “Principle 2” here). For this round of grantmaking, we updated slightly positively on AMF’s cost-effectiveness and believed that AMF had a time-sensitive funding opportunity while Malaria Consortium did not. These factors were sufficient to tip the balance in favor of allocating this funding to AMF.

What AMF will do with additional funding

AMF expects to allocate all funding that it receives in the near future toward distributing insecticide-treated nets in the Democratic Republic of Congo (DRC) in 2020.

The vast majority of the funding AMF currently has on hand is set aside for distributions in a series of other countries (we are aware of which countries but have been asked not to name them while discussions are continuing) in 2020, and in DRC in 2019 and 2020; this funding totals $57 million. It has made verbal agreements with those countries for those distributions, but has not yet signed contracts to commit the funding. There is a chance that one or more of these agreements will fall through, which could change when or how AMF uses the additional funding it receives now; we think the risk is real but that AMF’s other options (particularly to put more funding into DRC distributions in 2020-2021) are good and so we don’t see this as a major concern. According to AMF, agreements that have reached this stage with countries AMF has worked with before (as it has with each of these four countries) have not fallen through in the past. AMF also has an additional $5.5 million in uncommitted funds on hand, which it plans to allocate to DRC for 2020 distributions.

AMF estimates it could use up to an additional $12.0 million in DRC in 2020, after the $4.7 million we are granting to it, and up to an additional $36.8 million in DRC in 2021.

Our process

Our process for making this granting decision was less intensive than the process we used for the funds we received in the last quarter of 2018. We focused on making some updates to the information we had relied on last quarter, including:

1. Estimating Malaria Consortium’s room for more funding for SMC, in light of its receiving the $10.1 million in discretionary funds we granted last quarter.
2. Speaking with AMF and Malaria Consortium to discuss how additional funds would be used. We did not speak with other top charities for updates, as we believed, based on our work in late 2018 and earlier this year, that either Malaria Consortium or AMF could most cost-effectively use additional funding.
3. Updating our estimates of the cost-effectiveness of additional funds to AMF and Malaria Consortium. We applied the same changes as are discussed in the first footnote in this blog post to the latest version of our published cost-effectiveness model, which was updated in March to correct for an error in our model of insecticide-treated nets. These updates to the published model have not been published or vetted, and so are more likely to contain errors than our published cost-effectiveness model. We took this shortcut at this time to enable us to pursue other research work and because modest changes in the cost-effectiveness analysis (as a result of minor errors) would not have changed our conclusion here. We thoroughly revisit our comparisons between top charities once per year for our annual recommendations refresh in November; we are also developing a process to make it easier to update our published model throughout the year to reflect new information about how top charities would spend additional funding. The difference between our unvetted best guess and our published estimate for AMF is largely due to modeling the majority of marginal funding as going to DRC (which has a higher malaria burden that most of the other countries where AMF operates).

We ultimately relied on the same six principles as are described in this blog post.

Updates since March 2019

We wrote in March 2019 about our decision to allocate the funding we received in the fourth quarter of 2018 to Malaria Consortium’s SMC program. Below, we discuss updates to our understanding of the value of allocating marginal funds to AMF and Malaria Consortium’s SMC program since March 2019.

AMF

We identified and corrected an error in our cost-effectiveness analysis of insecticide-treated nets, which increased the estimated cost-effectiveness of AMF. As a result of this update, our (unvetted and unpublished) best guess is that marginal funding to AMF is approximately 40% more cost-effective than marginal funding to Malaria Consortium’s SMC program; in March, our best guess was that marginal funding to AMF was approximately 20% more cost-effective.

Malaria Consortium’s SMC program

Our understanding is that additional funding received at this point in time would be unlikely to influence Malaria Consortium’s spending on 2019 and 2020 programs, and that decisions regarding 2021 spending will likely be made in early 2020, after we have made our (larger) year-end funding recommendations.

This is a change from the previous quarter: due to our allocation of $10.1 million to Malaria Consortium last quarter, Malaria Consortium currently has enough funding to fund its work on SMC through 2020, even if it were to expand to the maximum scale it is considering reaching in 2020. (Our calculations, which were reviewed by Malaria Consortium, are available here.)

We expect that Malaria Consortium will have a large funding gap for SMC work in 2021, and we may fill some of that funding gap later in 2019.

The post Allocation of discretionary funds from Q1 2019 appeared first on The GiveWell Blog.

In the first quarter of 2019, donors gave a combined $4.7 million for granting to recommended charities at our discretion.

We really appreciate the generosity of our supporters in making it possible for us to regularly allocate funding to the top charity or charities that we believe can best use additional funding. Thank you!

In this post, we discuss our decision to allocate this $4.7 million to the Against Malaria Foundation (AMF), as well as the process we followed to arrive at this decision.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to AMF, which we believe could use additional funding for highly cost-effective work, even after receiving the $4.7 million in funding mentioned above.

Our bottom line

As we did last quarter, we focused our efforts on deciding between allocating funding to Malaria Consortium vs. AMF. We currently believe that AMF has a more time sensitive funding need than Malaria Consortium, and our best guess is that it will have equivalent impact per dollar to Malaria Consortium. This led us to allocate funding to AMF.

Read More

The post Allocation of discretionary funds from Q1 2019 appeared first on The GiveWell Blog.

In the first quarter of 2019, donors gave a combined $4.7 million for granting to recommended charities at our discretion.

We really appreciate the generosity of our supporters in making it possible for us to regularly allocate funding to the top charity or charities that we believe can best use additional funding. Thank you!

In this post, we discuss our decision to allocate this $4.7 million to the Against Malaria Foundation (AMF), as well as the process we followed to arrive at this decision.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to AMF, which we believe could use additional funding for highly cost-effective work, even after receiving the $4.7 million in funding mentioned above.

Our bottom line

As we did last quarter, we focused our efforts on deciding between allocating funding to Malaria Consortium vs. AMF. We currently believe that AMF has a more time sensitive funding need than Malaria Consortium, and our best guess is that it will have equivalent impact per dollar to Malaria Consortium. This led us to allocate funding to AMF.

Read More

The post Allocation of discretionary funds from Q1 2019 appeared first on The GiveWell Blog.

In the first quarter of 2019, donors gave a combined $4.7 million for granting to recommended charities at our discretion.

We really appreciate the generosity of our supporters in making it possible for us to regularly allocate funding to the top charity or charities that we believe can best use additional funding. Thank you!

In this post, we discuss our decision to allocate this $4.7 million to the Against Malaria Foundation (AMF), as well as the process we followed to arrive at this decision.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to AMF, which we believe could use additional funding for highly cost-effective work, even after receiving the $4.7 million in funding mentioned above.

Our bottom line

As we did last quarter, we focused our efforts on deciding between allocating funding to Malaria Consortium vs. AMF. We currently believe that AMF has a more time sensitive funding need than Malaria Consortium, and our best guess is that it will have equivalent impact per dollar to Malaria Consortium. This led us to allocate funding to AMF.

What changed since last quarter?

In March 2019, we modeled AMF as somewhat more cost-effective than Malaria Consortium’s seasonal malaria chemoprevention (SMC) program, believed that both organizations had time-sensitive funding opportunities, but believed that Malaria Consortium would have more overall impact per dollar, when taking into account unmodeled qualitative factors (see “Principle 2” here). For this round of grantmaking, we updated slightly positively on AMF’s cost-effectiveness and believed that AMF had a time-sensitive funding opportunity while Malaria Consortium did not. These factors were sufficient to tip the balance in favor of allocating this funding to AMF.

What AMF will do with additional funding

AMF expects to allocate all funding that it receives in the near future toward distributing insecticide-treated nets in the Democratic Republic of Congo (DRC) in 2020.

The vast majority of the funding AMF currently has on hand is set aside for distributions in a series of other countries (we are aware of which countries but have been asked not to name them while discussions are continuing) in 2020, and in DRC in 2019 and 2020; this funding totals $57 million. It has made verbal agreements with those countries for those distributions, but has not yet signed contracts to commit the funding. There is a chance that one or more of these agreements will fall through, which could change when or how AMF uses the additional funding it receives now; we think the risk is real but that AMF’s other options (particularly to put more funding into DRC distributions in 2020-2021) are good and so we don’t see this as a major concern. According to AMF, agreements that have reached this stage with countries AMF has worked with before (as it has with each of these four countries) have not fallen through in the past. AMF also has an additional $5.5 million in uncommitted funds on hand, which it plans to allocate to DRC for 2020 distributions.

AMF estimates it could use up to an additional $12.0 million in DRC in 2020, after the $4.7 million we are granting to it, and up to an additional $36.8 million in DRC in 2021.

Our process

Our process for making this granting decision was less intensive than the process we used for the funds we received in the last quarter of 2018. We focused on making some updates to the information we had relied on last quarter, including:

1. Estimating Malaria Consortium’s room for more funding for SMC, in light of its receiving the $10.1 million in discretionary funds we granted last quarter.
2. Speaking with AMF and Malaria Consortium to discuss how additional funds would be used. We did not speak with other top charities for updates, as we believed, based on our work in late 2018 and earlier this year, that either Malaria Consortium or AMF could most cost-effectively use additional funding.
3. Updating our estimates of the cost-effectiveness of additional funds to AMF and Malaria Consortium. We applied the same changes as are discussed in the first footnote in this blog post to the latest version of our published cost-effectiveness model, which was updated in March to correct for an error in our model of insecticide-treated nets. These updates to the published model have not been published or vetted, and so are more likely to contain errors than our published cost-effectiveness model. We took this shortcut at this time to enable us to pursue other research work and because modest changes in the cost-effectiveness analysis (as a result of minor errors) would not have changed our conclusion here. We thoroughly revisit our comparisons between top charities once per year for our annual recommendations refresh in November; we are also developing a process to make it easier to update our published model throughout the year to reflect new information about how top charities would spend additional funding. The difference between our unvetted best guess and our published estimate for AMF is largely due to modeling the majority of marginal funding as going to DRC (which has a higher malaria burden that most of the other countries where AMF operates).

We ultimately relied on the same six principles as are described in this blog post.

Updates since March 2019

We wrote in March 2019 about our decision to allocate the funding we received in the fourth quarter of 2018 to Malaria Consortium’s SMC program. Below, we discuss updates to our understanding of the value of allocating marginal funds to AMF and Malaria Consortium’s SMC program since March 2019.

AMF

We identified and corrected an error in our cost-effectiveness analysis of insecticide-treated nets, which increased the estimated cost-effectiveness of AMF. As a result of this update, our (unvetted and unpublished) best guess is that marginal funding to AMF is approximately 40% more cost-effective than marginal funding to Malaria Consortium’s SMC program; in March, our best guess was that marginal funding to AMF was approximately 20% more cost-effective.

Malaria Consortium’s SMC program

Our understanding is that additional funding received at this point in time would be unlikely to influence Malaria Consortium’s spending on 2019 and 2020 programs, and that decisions regarding 2021 spending will likely be made in early 2020, after we have made our (larger) year-end funding recommendations.

This is a change from the previous quarter: due to our allocation of $10.1 million to Malaria Consortium last quarter, Malaria Consortium currently has enough funding to fund its work on SMC through 2020, even if it were to expand to the maximum scale it is considering reaching in 2020. (Our calculations, which were reviewed by Malaria Consortium, are available here.)

We expect that Malaria Consortium will have a large funding gap for SMC work in 2021, and we may fill some of that funding gap later in 2019.

The post Allocation of discretionary funds from Q1 2019 appeared first on The GiveWell Blog.

This post discusses a set of issues with Evidence Action’s No Lean Season program. No Lean Season is a former GiveWell top charity and GiveWell Incubation Grant recipient. It is now shutting down. Evidence Action discusses its decision in this blog post.

Here, we share a significant amount of detail about this decision and the factors that contributed. Proactively sharing detailed information about a charity’s shortcomings may be unusual, but it is core to GiveWell’s mission. We are dedicated to transparency about our recommendations—the good and the bad.

Evidence Action has reviewed this post, and we’ve discussed our thinking at length with its senior leadership; however, the views expressed are our own. We have been impressed with Evidence Action’s commitment to transparency and continue to support its other work. These updates have not substantially changed our view of Evidence Action; we expect large programs to experience problems, to a certain extent. We believe Evidence Action responded to these problems responsibly, although we have several open questions.

Summary

Evidence Action is shutting down No Lean Season, a former GiveWell top charity that distributed no-interest subsidies to support seasonal migration in Bangladesh.

As we have discussed previously, a study of the No Lean Season program in 2017 found disappointing results; this led to our removal of No Lean Season, in agreement with Evidence Action, from our list of top charities.

In early 2019, Evidence Action’s senior leadership received allegations that a junior employee of the government agency in Bangladesh responsible for approving the No Lean Season program allegedly forged the government approval, allegedly in collaboration with an employee of the program’s implementing partner. The government agency allegedly later asked the implementing partner for a bribe to grant approval of the program. Senior leadership at Evidence Action then began an investigation that was largely unsuccessful in its attempts to learn more due to lack of full cooperation from the implementing partner. Evidence Action terminated its relationship with the implementing partner as a result. Evidence Action’s senior leadership also found that some Evidence Action program staff who worked directly with the partner did not fully cooperate with its investigation and had violated internal Evidence Action policy.

Evidence Action decided to shut down No Lean Season because the cost of finding and supporting a new implementing partner was too high, given the disappointing 2017 study results.

Separately, Evidence Action also informed us of a tragic accident involving migrants from households that had received No Lean Season subsidies. We do not believe this contributed to the decision to shut down No Lean Season, but we are sharing it in this post as the investigation into this accident recently concluded.

We will provide additional information on the following in this blog post:

• We outline below the factors contributing to Evidence Action’s decision to shut down its No Lean Season program (More):
  • The disappointing 2017 study of the program at scale. (More)
  • Evidence Action’s termination of its partnership earlier this year with the organization implementing the program in Bangladesh. After learning of the alleged improprieties (referenced above) in February 2019, senior leadership at Evidence Action began an investigation, conducted by external, independent legal counsel. Given the seriousness of the original allegations, Evidence Action also terminated its contract with the partner. The implementing partner largely refused to cooperate with the investigation, and as a result Evidence Action will not reengage with this partner in the future. (More)
• In the course of its investigation, senior leadership at Evidence Action found evidence of an approximately $400 payment by an Evidence Action program staff member that violated its internal policies, as well as contradictory and potentially misleading statements made by some program staff members to investigators. This finding was not material in the decision to shut down the program, as the implementing partner’s lack of cooperation was already known at that point. (More)
• We summarize the findings of the investigation into the accident involving migrants whose families had received subsidies from the program. (More)
• We do not see any of the above issues as a significant update on Evidence Action as an organization. We expect challenges when working in international development, and think senior leadership at Evidence Action responded responsibly to address these challenges. We do retain open questions about Evidence Action’s selection of implementing partners and its process for hiring and evaluating staff. Finally, we and Evidence Action agree that it should continue to strengthen its financial controls going forward. (More)
• Evidence Action expects No Lean Season to have some funding remaining after the program fully closes out. We expect to ask Evidence Action to redirect the remaining funding it received from GiveWell for No Lean Season to Evidence Action’s Deworm the World Initiative, a GiveWell top charity. We (and Evidence Action) will also take into account donors’ preferences for reallocating this funding; we provide instructions for donors who supported No Lean Season to communicate their preferences to us below. (More)

Shutting down No Lean Season

Mixed evidence of impact for program

We removed our top-charity recommendation of No Lean Season after reviewing the results of a large randomized controlled trial (RCT) of its program during the 2017 “lean season.” Evidence Action agreed with GiveWell’s decision.

The study found that the program did not increase rates of migration in 2017, the first year in which the program was implemented at scale. This implied that, at a minimum, the impact of the program was sensitive to details of implementation, and, potentially, the program was not effective at scale. In either case, the results reduced our and Evidence Action’s expectations about the program’s future cost-effectiveness. As of late 2018, Evidence Action had stopped seeking donations for the program but was continuing to operate it and collect additional data on its impact. Evidence Action ran another large trial of the program in 2018 (for which data collection is ongoing in 2019), and we planned to reassess No Lean Season as a potential top charity in 2019, upon receiving the results from that trial.

Investigation into allegations against Evidence Action’s implementing partner

Termination of partnership with implementing partner

In early 2019, Evidence Action terminated its partnership with its partner in Bangladesh; this partner bore primary responsibility for implementing the program. Evidence Action reports that the termination of its implementing partnership accelerated consideration of shutting down the program, rather than waiting for results from the 2018 trial of the program, and tipped the balance in favor of ending it.

According to multiple Evidence Action program staff (and reported to GiveWell by the law firm that subsequently led an investigation into what happened), the implementing partner told these Evidence Action program staff in February 2019 that it had discovered that the government licenses for it to operate the program had been improperly granted, and that there was a possibility that bribes were paid to government officials to obtain the improper licenses. According to these Evidence Action program staff, the implementing partner said that it had approached the government agency to rectify the issue with the licenses and a high-level official asked for a bribe to issue the licenses. The Evidence Action program staff said that the implementing partner had asked Evidence Action for authorization to pay the bribe. The Evidence Action program staff reported this immediately to senior leadership at Evidence Action, who shared it with us and other major donors to the program soon after.

Evidence Action terminated its contract with the implementing partner as a result, shortly after learning of this event. Senior leadership at Evidence Action hired DLA Piper, a global law firm, to lead an investigation into the issue. DLA Piper told us that it was not able to learn significantly more about the circumstances around the allegedly improperly granted licenses because the implementing partner did not cooperate with the investigation; the implementing partner provided only incomplete financial records and declined to participate in interviews. What we know about these circumstances, therefore, is based on DLA Piper’s interviews with the Evidence Action program staff who were in the February meeting; these staff were consistent in their reports that the implementing partner said that a government official had requested a bribe, but they were inconsistent on some other details.

Payment in violation of internal policies and contradictory and potentially misleading statements by Evidence Action program staff

In the course of its investigation, DLA Piper found that some Evidence Action program staff made contradictory and potentially misleading statements to the investigators, including statements about a payment one of them made in 2018 (of approximately $400) that went against Evidence Action policy.

According to the investigation findings, these staff members had learned that the implementing partner’s application to a government agency for licenses to operate the program in 2018 had been questioned and two of the staff discussed hiring a consultant to help with obtaining the licenses. The staff members requested approval from Evidence Action’s finance team to make a payment to hire a consultant; Evidence Action’s finance team responded that such consultant payments were not allowed by Evidence Action policy. The staff members later submitted a reimbursement request to Evidence Action for a cash payment for the same purpose. It is unclear whether a consultant was hired or how the funding was used. Evidence Action’s finance department sent the reimbursement. Senior leadership at Evidence Action has told us that the reimbursement should not have been sent and that its financial oversight practices were not adequate to detect this payment; it expects to make some changes to its controls and finance staff training as a result of this experience.

We are unsure what to make of these findings. The investigation only established that these staff received reimbursement for a use of funds they had been informed was not permissible under Evidence Action policy, and that they made misleading statements to the investigators. There are missing pieces in the story that we are unable to account for (due to the implementing partner’s lack of cooperation with the investigation and conflicting and potentially misleading statements from some Evidence Action program staff). Senior leadership at Evidence Action has taken what we see to be appropriate corrective action in its staffing; we have chosen not to discuss details about individuals.[1]

We understand that charities may on occasion experience malfeasance by staff, and we believe Evidence Action could not have prevented all possible scenarios where malfeasance might occur. We were disappointed to learn that Evidence Action staff appear to have made a payment in violation of Evidence Action policy and that this payment was approved; we are also disappointed that these staff made misleading statements to DLA Piper. We also believe that senior leadership at Evidence Action responded to the issues described above in a timely, thorough manner by investigating what occurred.

While it is disappointing to learn of improper behavior at one of our top charities, our ultimate focus is whether the program accomplishes good in the world.[2] Of course, if we learned of wrongdoing that was not responsibly handled, and/or reflected a large and serious gap in internal controls, that could lead us to remove a top charity from our list. But broadly speaking, we would not expect to be aware of every instance of fraud (nor do we believe it would be cost-effective for most organizations to put in place controls that would absolutely prevent all malfeasance).[3]

From discussions with senior leadership at Evidence Action, we do not believe the improper behavior by some Evidence Action program staff contributed to the decision to shut down the program, though it was uncovered as part of the investigation into the implementing partner.

Decision to end the program

In light of the 2017 RCT results, the need to find a new implementing partner, and the costs of doing so, Evidence Action decided to terminate the No Lean Season program.

We agree with Evidence Action’s decision. This is a nuanced position: we agree with the decision to shut down the program, but we do not believe that the program is “bad” or “ineffective.” Instead, we believe that continued investment in No Lean Season—taking into account all of the challenges this would involve—is unlikely to be one of the best opportunities we or Evidence Action have to cost-effectively save or improve lives. We discuss how the 2017 study changed our assessment of No Lean Season’s cost-effectiveness here.

The program was completed for the 2018-2019 season, with the exception of some subsidy repayment at the end of the season. Data collection and analysis for the 2018 RCT is ongoing and will be completed; we plan to write about the results once they are available.

Cumilla accident

A few weeks before the conversation that led to the termination of Evidence Action’s partnership with its implementing partner, there was an accident involving migrants from households that had received No Lean Season subsidies. We were saddened to hear of this tragic event. This tragedy did not trigger the shutdown of the program, but the investigation that Evidence Action conducted into the circumstances surrounding the accident recently concluded, so we include it here for completeness.

According to news reports[4] (and reported to us by Evidence Action), in January 2019, a coal-laden truck struck a shed at a brick kiln in Cumilla District in Bangladesh. The shed collapsed, killing 13 individuals sleeping inside. Five of the individuals were from households that had received migration subsidies from No Lean Season. Four of those were between the ages of 15 and 17. It is No Lean Season’s policy to only provide subsidies to individuals over the age of 18 and No Lean Season had a number of protocols in place to enforce this policy[5]; senior leadership at Evidence Action told us that it believes that the teenagers did not receive subsidies directly. The teenagers may have migrated independently of the program or individuals from their households who were over the age of 18 may have accepted the subsidies and given them to the teenagers to use to migrate.

Senior leadership at Evidence Action hired investigators to look into the circumstances around the use of No Lean Season subsidies by underage individuals. The investigators conducted a limited investigation (it was limited, at least in part, because the implementing partner did not provide requested documentation or interviews) and concluded that Evidence Action had “robust safeguards” in place for preventing underage migration, including checking birth dates on personal documents before issuing loans. We don’t see good reason to think that the program systematically increases overall risks of this type of accident.

Has our view of Evidence Action changed?

Taken together, the updates have raised questions about Evidence Action (see below). In general, malfeasance at a charity or its implementing partner could lead us to change our opinion of a charity. However, the details of this case have not led us to significantly reduce our confidence in Evidence Action. The decision to scale up No Lean Season was reasonable: high-quality evidence from when the program was operated at a small scale indicated that it had the potential to be cost-effective. Evidence Action decided to scale up based on that evidence, and ran another high-quality study to test the program’s impact at scale. While there were ethical and managerial lapses by some Evidence Action program staff and its implementing partner, as well as a failure of financial controls to catch an improper payment, we broadly believe that senior leadership at Evidence Action responded quickly, with transparency, and responsibly when the issues were uncovered, both to rectify the lapses and to consider how it might improve and prevent such lapses in the future. Overall, our high-level view of Evidence Action is very similar to what it was before we learned of these developments.

We’ve written before that we see Evidence Action as a group we are highly aligned with and that we are excited to support its growth and development (see, for example, here). We have recommended GiveWell Incubation Grants to support Evidence Action’s operations as well as to support its work to develop potential new GiveWell top charities, and we count two of its programs, the Deworm the World Initiative and Dispensers for Safe Water as a top and standout charity, respectively.

We expect large programs to experience problems, to some extent. We think senior leadership at Evidence Action took quick, thorough action to address the situation by launching an investigation and sharing updates with its major funders, as well as terminating its work with its implementing partner and taking corrective action with program staff who did not comply with Evidence Action’s policies.

We have the following open questions about Evidence Action deriving from these developments:

• Evidence Action’s selection of implementing partners. Should Evidence Action do more to vet its partners? Evidence Action has told us that it plans to make changes to its vetting practices as a result of this experience.
• Evidence Action’s processes for hiring and evaluating staff. Evidence Action staff members violated its financial policies. How should Evidence Action improve its processes for hiring and evaluating staff?
• Evidence Action financial oversight. While we think prevention of all malfeasance would be challenging (and may not be the best use of resources), are there cost-effective ways to reduce the likelihood of funds being misused in the future? What changes to its financial controls should Evidence Action implement?

We plan to continue discussions with Evidence Action to better understand its work in these areas.

What will happen with unused funds?

Over the course of operating No Lean Season, Evidence Action received funding earmarked for this program specifically and spent down a portion of that funding. Evidence Action expects to have funding remaining that is designated for No Lean Season when it has fully closed out its work on the program. A large portion of the remaining funding is from Good Ventures, a large foundation with which we work closely, which donated to No Lean Season as a result of GiveWell’s recommendation.

We expect to ask Evidence Action to redirect the remaining funding it received from GiveWell (donations made through our website or to GiveWell via check, wire transfer, or other means), including funding from Good Ventures, for No Lean Season to Evidence Action’s Deworm the World Initiative, which is a GiveWell top charity. If you made a donation to support No Lean Season and prefer that your donation (less the portion of total revenue that No Lean Season has spent) go to an Evidence Action program other than Deworm the World, please contact us at donations@givewell.org by July 31. We will also be emailing donors whose contact information we have. By default, if we don’t hear anything, funding will be directed to Deworm the World.

Note that by “remaining funding,” we mean the original donation size multiplied by the percentage of total revenue for No Lean Season that will remain when the program is fully closed out. We then expect to recommend that Good Ventures reduce its next annual grant to Deworm the World (assuming Deworm the World remains a GiveWell top charity, which we expect it to) in December by the same amount, so that Deworm the World does not receive more GiveWell-directed funding in 2019 than it would have in the absence of No Lean Season’s remaining funds.

We will post an update on our blog about how remaining GiveWell-directed funding for No Lean Season was reallocated.

Evidence Action’s implementing partner holds some program funding, primarily the grant from Evidence Action to the implementing partner for the subsidies for migrants that was collected for the 2018 implementation of the program. It is unclear whether it will return these funds to Evidence Action; it has not yet agreed to do so.

Notes

1. Evidence Action has told us that there is complexity in discussing in general terms the situations with the staff members involved because of the differences in the structure of their employment across several legal jurisdictions.

2. We generally agree with the “results not receipts” approach advocated by the Center for Global Development in this paper.

3. We did learn about two cases of staff fraud at GiveDirectly, another GiveWell top charity, in the past. You can read more in our review of GiveDirectly’s work here.

4. News reports of the accident are available here and here (among other places).

5. From Evidence Action’s blog: “The investigation found that the safeguards we had in place were robust, though ultimately could not fully eliminate the risk of an adult recipient choosing to pass their cash transport subsidy to a teenager in his place, contrary to program rules and protocols. These protocols were multilayered, and included verbally informing subsidy recipients of the condition that migrants must be at least 18 years of age; requiring subsidy recipients to sign or thumbprint an acknowledgement that both recipients and migrants (where different individuals) must be at least 18 years of age; reviewing national identification cards to verify that the subsidy recipient and any person that the recipient says plans to migrate from the household is at least age 18; and utilizing mobile data collection software that is programmed to prohibit field staff from including individuals reporting to be under the age of 18, in order to prevent accidental enrollment.”

The post Evidence Action is shutting down No Lean Season appeared first on The GiveWell Blog.

This post discusses a set of issues with Evidence Action’s No Lean Season program. No Lean Season is a former GiveWell top charity and GiveWell Incubation Grant recipient. It is now shutting down. Evidence Action discusses its decision in this blog post.

Here, we share a significant amount of detail about this decision and the factors that contributed. Proactively sharing detailed information about a charity’s shortcomings may be unusual, but it is core to GiveWell’s mission. We are dedicated to transparency about our recommendations—the good and the bad.

Evidence Action has reviewed this post, and we’ve discussed our thinking at length with its senior leadership; however, the views expressed are our own. We have been impressed with Evidence Action’s commitment to transparency and continue to support its other work. These updates have not substantially changed our view of Evidence Action; we expect large programs to experience problems, to a certain extent. We believe Evidence Action responded to these problems responsibly, although we have several open questions.

Summary

Evidence Action is shutting down No Lean Season, a former GiveWell top charity that distributed no-interest subsidies to support seasonal migration in Bangladesh.

As we have discussed previously, a study of the No Lean Season program in 2017 found disappointing results; this led to our removal of No Lean Season, in agreement with Evidence Action, from our list of top charities.

In early 2019, Evidence Action’s senior leadership received allegations that a junior employee of the government agency in Bangladesh responsible for approving the No Lean Season program allegedly forged the government approval, allegedly in collaboration with an employee of the program’s implementing partner. The government agency allegedly later asked the implementing partner for a bribe to grant approval of the program. Senior leadership at Evidence Action then began an investigation that was largely unsuccessful in its attempts to learn more due to lack of full cooperation from the implementing partner. Evidence Action terminated its relationship with the implementing partner as a result. Evidence Action’s senior leadership also found that some Evidence Action program staff who worked directly with the partner did not fully cooperate with its investigation and had violated internal Evidence Action policy.

Evidence Action decided to shut down No Lean Season because the cost of finding and supporting a new implementing partner was too high, given the disappointing 2017 study results.

Separately, Evidence Action also informed us of a tragic accident involving migrants from households that had received No Lean Season subsidies. We do not believe this contributed to the decision to shut down No Lean Season, but we are sharing it in this post as the investigation into this accident recently concluded.

We will provide additional information on the following in this blog post:

• We outline below the factors contributing to Evidence Action’s decision to shut down its No Lean Season program (More):
  • The disappointing 2017 study of the program at scale. (More)
  • Evidence Action’s termination of its partnership earlier this year with the organization implementing the program in Bangladesh. After learning of the alleged improprieties (referenced above) in February 2019, senior leadership at Evidence Action began an investigation, conducted by external, independent legal counsel. Given the seriousness of the original allegations, Evidence Action also terminated its contract with the partner. The implementing partner largely refused to cooperate with the investigation, and as a result Evidence Action will not reengage with this partner in the future. (More)
• In the course of its investigation, senior leadership at Evidence Action found evidence of an approximately $400 payment by an Evidence Action program staff member that violated its internal policies, as well as contradictory and potentially misleading statements made by some program staff members to investigators. This finding was not material in the decision to shut down the program, as the implementing partner’s lack of cooperation was already known at that point. (More)
• We summarize the findings of the investigation into the accident involving migrants whose families had received subsidies from the program. (More)
• We do not see any of the above issues as a significant update on Evidence Action as an organization. We expect challenges when working in international development, and think senior leadership at Evidence Action responded responsibly to address these challenges. We do retain open questions about Evidence Action’s selection of implementing partners and its process for hiring and evaluating staff. Finally, we and Evidence Action agree that it should continue to strengthen its financial controls going forward. (More)
• Evidence Action expects No Lean Season to have some funding remaining after the program fully closes out. We expect to ask Evidence Action to redirect the remaining funding it received from GiveWell for No Lean Season to Evidence Action’s Deworm the World Initiative, a GiveWell top charity. We (and Evidence Action) will also take into account donors’ preferences for reallocating this funding; we provide instructions for donors who supported No Lean Season to communicate their preferences to us below. (More)

Shutting down No Lean Season

Mixed evidence of impact for program

We removed our top-charity recommendation of No Lean Season after reviewing the results of a large randomized controlled trial (RCT) of its program during the 2017 “lean season.” Evidence Action agreed with GiveWell’s decision.

The study found that the program did not increase rates of migration in 2017, the first year in which the program was implemented at scale. This implied that, at a minimum, the impact of the program was sensitive to details of implementation, and, potentially, the program was not effective at scale. In either case, the results reduced our and Evidence Action’s expectations about the program’s future cost-effectiveness. As of late 2018, Evidence Action had stopped seeking donations for the program but was continuing to operate it and collect additional data on its impact. Evidence Action ran another large trial of the program in 2018 (for which data collection is ongoing in 2019), and we planned to reassess No Lean Season as a potential top charity in 2019, upon receiving the results from that trial.

Investigation into allegations against Evidence Action’s implementing partner

Termination of partnership with implementing partner

In early 2019, Evidence Action terminated its partnership with its partner in Bangladesh; this partner bore primary responsibility for implementing the program. Evidence Action reports that the termination of its implementing partnership accelerated consideration of shutting down the program, rather than waiting for results from the 2018 trial of the program, and tipped the balance in favor of ending it.

According to multiple Evidence Action program staff (and reported to GiveWell by the law firm that subsequently led an investigation into what happened), the implementing partner told these Evidence Action program staff in February 2019 that it had discovered that the government licenses for it to operate the program had been improperly granted, and that there was a possibility that bribes were paid to government officials to obtain the improper licenses. According to these Evidence Action program staff, the implementing partner said that it had approached the government agency to rectify the issue with the licenses and a high-level official asked for a bribe to issue the licenses. The Evidence Action program staff said that the implementing partner had asked Evidence Action for authorization to pay the bribe. The Evidence Action program staff reported this immediately to senior leadership at Evidence Action, who shared it with us and other major donors to the program soon after.

Evidence Action terminated its contract with the implementing partner as a result, shortly after learning of this event. Senior leadership at Evidence Action hired DLA Piper, a global law firm, to lead an investigation into the issue. DLA Piper told us that it was not able to learn significantly more about the circumstances around the allegedly improperly granted licenses because the implementing partner did not cooperate with the investigation; the implementing partner provided only incomplete financial records and declined to participate in interviews. What we know about these circumstances, therefore, is based on DLA Piper’s interviews with the Evidence Action program staff who were in the February meeting; these staff were consistent in their reports that the implementing partner said that a government official had requested a bribe, but they were inconsistent on some other details.

Payment in violation of internal policies and contradictory and potentially misleading statements by Evidence Action program staff

In the course of its investigation, DLA Piper found that some Evidence Action program staff made contradictory and potentially misleading statements to the investigators, including statements about a payment one of them made in 2018 (of approximately $400) that went against Evidence Action policy.

According to the investigation findings, these staff members had learned that the implementing partner’s application to a government agency for licenses to operate the program in 2018 had been questioned and two of the staff discussed hiring a consultant to help with obtaining the licenses. The staff members requested approval from Evidence Action’s finance team to make a payment to hire a consultant; Evidence Action’s finance team responded that such consultant payments were not allowed by Evidence Action policy. The staff members later submitted a reimbursement request to Evidence Action for a cash payment for the same purpose. It is unclear whether a consultant was hired or how the funding was used. Evidence Action’s finance department sent the reimbursement. Senior leadership at Evidence Action has told us that the reimbursement should not have been sent and that its financial oversight practices were not adequate to detect this payment; it expects to make some changes to its controls and finance staff training as a result of this experience.

We are unsure what to make of these findings. The investigation only established that these staff received reimbursement for a use of funds they had been informed was not permissible under Evidence Action policy, and that they made misleading statements to the investigators. There are missing pieces in the story that we are unable to account for (due to the implementing partner’s lack of cooperation with the investigation and conflicting and potentially misleading statements from some Evidence Action program staff). Senior leadership at Evidence Action has taken what we see to be appropriate corrective action in its staffing; we have chosen not to discuss details about individuals.[1]

We understand that charities may on occasion experience malfeasance by staff, and we believe Evidence Action could not have prevented all possible scenarios where malfeasance might occur. We were disappointed to learn that Evidence Action staff appear to have made a payment in violation of Evidence Action policy and that this payment was approved; we are also disappointed that these staff made misleading statements to DLA Piper. We also believe that senior leadership at Evidence Action responded to the issues described above in a timely, thorough manner by investigating what occurred.

While it is disappointing to learn of improper behavior at one of our top charities, our ultimate focus is whether the program accomplishes good in the world.[2] Of course, if we learned of wrongdoing that was not responsibly handled, and/or reflected a large and serious gap in internal controls, that could lead us to remove a top charity from our list. But broadly speaking, we would not expect to be aware of every instance of fraud (nor do we believe it would be cost-effective for most organizations to put in place controls that would absolutely prevent all malfeasance).[3]

From discussions with senior leadership at Evidence Action, we do not believe the improper behavior by some Evidence Action program staff contributed to the decision to shut down the program, though it was uncovered as part of the investigation into the implementing partner.

Decision to end the program

In light of the 2017 RCT results, the need to find a new implementing partner, and the costs of doing so, Evidence Action decided to terminate the No Lean Season program.

We agree with Evidence Action’s decision. This is a nuanced position: we agree with the decision to shut down the program, but we do not believe that the program is “bad” or “ineffective.” Instead, we believe that continued investment in No Lean Season—taking into account all of the challenges this would involve—is unlikely to be one of the best opportunities we or Evidence Action have to cost-effectively save or improve lives. We discuss how the 2017 study changed our assessment of No Lean Season’s cost-effectiveness here.

The program was completed for the 2018-2019 season, with the exception of some subsidy repayment at the end of the season. Data collection and analysis for the 2018 RCT is ongoing and will be completed; we plan to write about the results once they are available.

Cumilla accident

A few weeks before the conversation that led to the termination of Evidence Action’s partnership with its implementing partner, there was an accident involving migrants from households that had received No Lean Season subsidies. We were saddened to hear of this tragic event. This tragedy did not trigger the shutdown of the program, but the investigation that Evidence Action conducted into the circumstances surrounding the accident recently concluded, so we include it here for completeness.

According to news reports[4] (and reported to us by Evidence Action), in January 2019, a coal-laden truck struck a shed at a brick kiln in Cumilla District in Bangladesh. The shed collapsed, killing 13 individuals sleeping inside. Five of the individuals were from households that had received migration subsidies from No Lean Season. Four of those were between the ages of 15 and 17. It is No Lean Season’s policy to only provide subsidies to individuals over the age of 18 and No Lean Season had a number of protocols in place to enforce this policy[5]; senior leadership at Evidence Action told us that it believes that the teenagers did not receive subsidies directly. The teenagers may have migrated independently of the program or individuals from their households who were over the age of 18 may have accepted the subsidies and given them to the teenagers to use to migrate.

Senior leadership at Evidence Action hired investigators to look into the circumstances around the use of No Lean Season subsidies by underage individuals. The investigators conducted a limited investigation (it was limited, at least in part, because the implementing partner did not provide requested documentation or interviews) and concluded that Evidence Action had “robust safeguards” in place for preventing underage migration, including checking birth dates on personal documents before issuing loans. We don’t see good reason to think that the program systematically increases overall risks of this type of accident.

Has our view of Evidence Action changed?

Taken together, the updates have raised questions about Evidence Action (see below). In general, malfeasance at a charity or its implementing partner could lead us to change our opinion of a charity. However, the details of this case have not led us to significantly reduce our confidence in Evidence Action. The decision to scale up No Lean Season was reasonable: high-quality evidence from when the program was operated at a small scale indicated that it had the potential to be cost-effective. Evidence Action decided to scale up based on that evidence, and ran another high-quality study to test the program’s impact at scale. While there were ethical and managerial lapses by some Evidence Action program staff and its implementing partner, as well as a failure of financial controls to catch an improper payment, we broadly believe that senior leadership at Evidence Action responded quickly, with transparency, and responsibly when the issues were uncovered, both to rectify the lapses and to consider how it might improve and prevent such lapses in the future. Overall, our high-level view of Evidence Action is very similar to what it was before we learned of these developments.

We’ve written before that we see Evidence Action as a group we are highly aligned with and that we are excited to support its growth and development (see, for example, here). We have recommended GiveWell Incubation Grants to support Evidence Action’s operations as well as to support its work to develop potential new GiveWell top charities, and we count two of its programs, the Deworm the World Initiative and Dispensers for Safe Water as a top and standout charity, respectively.

We expect large programs to experience problems, to some extent. We think senior leadership at Evidence Action took quick, thorough action to address the situation by launching an investigation and sharing updates with its major funders, as well as terminating its work with its implementing partner and taking corrective action with program staff who did not comply with Evidence Action’s policies.

We have the following open questions about Evidence Action deriving from these developments:

• Evidence Action’s selection of implementing partners. Should Evidence Action do more to vet its partners? Evidence Action has told us that it plans to make changes to its vetting practices as a result of this experience.
• Evidence Action’s processes for hiring and evaluating staff. Evidence Action staff members violated its financial policies. How should Evidence Action improve its processes for hiring and evaluating staff?
• Evidence Action financial oversight. While we think prevention of all malfeasance would be challenging (and may not be the best use of resources), are there cost-effective ways to reduce the likelihood of funds being misused in the future? What changes to its financial controls should Evidence Action implement?

We plan to continue discussions with Evidence Action to better understand its work in these areas.

What will happen with unused funds?

Over the course of operating No Lean Season, Evidence Action received funding earmarked for this program specifically and spent down a portion of that funding. Evidence Action expects to have funding remaining that is designated for No Lean Season when it has fully closed out its work on the program. A large portion of the remaining funding is from Good Ventures, a large foundation with which we work closely, which donated to No Lean Season as a result of GiveWell’s recommendation.

We expect to ask Evidence Action to redirect the remaining funding it received from GiveWell (donations made through our website or to GiveWell via check, wire transfer, or other means), including funding from Good Ventures, for No Lean Season to Evidence Action’s Deworm the World Initiative, which is a GiveWell top charity. If you made a donation to support No Lean Season and prefer that your donation (less the portion of total revenue that No Lean Season has spent) go to an Evidence Action program other than Deworm the World, please contact us at donations@givewell.org by July 31. We will also be emailing donors whose contact information we have. By default, if we don’t hear anything, funding will be directed to Deworm the World.

Note that by “remaining funding,” we mean the original donation size multiplied by the percentage of total revenue for No Lean Season that will remain when the program is fully closed out. We then expect to recommend that Good Ventures reduce its next annual grant to Deworm the World (assuming Deworm the World remains a GiveWell top charity, which we expect it to) in December by the same amount, so that Deworm the World does not receive more GiveWell-directed funding in 2019 than it would have in the absence of No Lean Season’s remaining funds.

We will post an update on our blog about how remaining GiveWell-directed funding for No Lean Season was reallocated.

Evidence Action’s implementing partner holds some program funding, primarily the grant from Evidence Action to the implementing partner for the subsidies for migrants that was collected for the 2018 implementation of the program. It is unclear whether it will return these funds to Evidence Action; it has not yet agreed to do so.

Notes

1. Evidence Action has told us that there is complexity in discussing in general terms the situations with the staff members involved because of the differences in the structure of their employment across several legal jurisdictions.

2. We generally agree with the “results not receipts” approach advocated by the Center for Global Development in this paper.

3. We did learn about two cases of staff fraud at GiveDirectly, another GiveWell top charity, in the past. You can read more in our review of GiveDirectly’s work here.

4. News reports of the accident are available here and here (among other places).

5. From Evidence Action’s blog: “The investigation found that the safeguards we had in place were robust, though ultimately could not fully eliminate the risk of an adult recipient choosing to pass their cash transport subsidy to a teenager in his place, contrary to program rules and protocols. These protocols were multilayered, and included verbally informing subsidy recipients of the condition that migrants must be at least 18 years of age; requiring subsidy recipients to sign or thumbprint an acknowledgement that both recipients and migrants (where different individuals) must be at least 18 years of age; reviewing national identification cards to verify that the subsidy recipient and any person that the recipient says plans to migrate from the household is at least age 18; and utilizing mobile data collection software that is programmed to prohibit field staff from including individuals reporting to be under the age of 18, in order to prevent accidental enrollment.”

The post Evidence Action is shutting down No Lean Season appeared first on The GiveWell Blog.

Our top priorities this year support our goals to (a) increase the impact per dollar of the funds we direct and (b) increase our money moved. In 2019, we are focused on:

• Building research capacity. (More)
• Experimenting with approaches to outreach to find ones that we can scalably use to drive additional money moved. (More)
• Exploring new areas of research. (More)
• Improving GiveWell’s organizational strength. (More)
• Ongoing research. (More)

Building research capacity

We announced earlier this year our plans to hire researchers at three levels of seniority, listed here from most junior to most senior: Research Analyst, Senior Research Analyst, and Senior Fellow. Our goal is to have 3-5 signed offer letters in hand from new research staff by the end of 2019.

We’re hoping that additional research capacity will enable us to expand the scope of GiveWell’s research, with the aim of finding opportunities that are more cost-effective than our current top charities. We’re planning to roughly double the size of the research team over the next few years.

Outreach experimentation

We plan to expand our outreach to current and potential donors going forward, with the aim of increasing the amount of money we direct to our recommended charities. As part of this work, we recently hired Stephanie Stojanovic as our first Major Gifts Officer. Our goal is to decide by the end of the 1st quarter of 2020 whether to scale our staff capacity further in the area of major gifts, based on Stephanie’s initial work.[1]

We’re also planning to conduct experiments in 2019 related to how we message about our work to reach more people. These experiments could include work on search engine optimization and building landing pages that aim to communicate what GiveWell does and why it’s valuable, among other possibilities. We expect to have results by early 2020, as the bulk of donations we receive are made in December.

Finally, we’re planning to search for a VP of Marketing to oversee work across outreach domains (including major gifts, donor retention, advertising, marketing, and written communications). We guess there is a 50 percent chance we make a hire for this role in 2019.

Exploring new areas of research

As mentioned above, we’re in the early stages of expanding the scope of GiveWell’s research. We plan to look into several new areas in 2019, including public health regulation and possible paths to support government aid agencies.

This work is new for GiveWell, and as noted in our 2018 review post, we failed to make as much progress as we hoped in 2018 on our work on public health regulation. In 2019, we’re aiming to get substantially closer to the point where we have the staff structure to support grantmaking in new areas, though given how early we are in this work, we don’t yet have concrete goals we’re confident that we’ll achieve.

A stretch goal for 2019 is to settle on a structure that we believe will support grantmaking in public health regulation and begin recommending grants in that area.

We also plan to continue our investigation into possible paths to support government aid agencies; in particular, we plan to complete an investigation into an opportunity to do so in the area of results-based financing.

Improving GiveWell’s organizational strength

We expect to need additional operations capacity to maintain critical functions as GiveWell grows and to improve the organization going forward. We plan to hire one Operations Associate this year to assist with general operations needs, such as improving HR practices.

We plan to hire many new staff over the coming years. In preparation, we plan to improve our procedures and information for recruiting, vetting, and onboarding staff to GiveWell this year, such as by improving inclusive recruitment practices and updating the substantive content of our onboarding activities. We also plan to improve our systems for soliciting feedback from staff about how GiveWell can improve as an organization, in order to give management better insight into how things are going.

To accommodate our planned expansion, we plan to move to a new office that better suits our expected size and staff requirements.

Ongoing research

We have a number of ongoing research projects, detailed here. These include:

• Completing a full draft of qualitative assessments of our top charities. In theory, we aim to maximize one thing with our top charity recommendations—total improvement in well-being per dollar spent—and this is what our cost-effectiveness estimates intend to capture. In practice, there are costs and benefits that we do not observe and are not estimated in our models, and so we allow for qualitative adjustments to affect our recommendations. We’re in the process of laying out a framework for qualitatively assessing relative organizational strength.
• Updating key inputs into our cost-effectiveness estimates, such as:
  • How we use vitamin A deficiency data.
  • Using new malaria prevalence and child mortality data.
  • Using new data to update our estimates of costs incurred and target population reached for five of our top charities: Malaria Consortium’s seasonal malaria chemoprevention program, the Against Malaria Foundation, Helen Keller International’s vitamin A supplementation program, Sightsavers’ deworming program, and Evidence Action’s Deworm the World Initiative.
  • Better understanding the counterfactual to the work Evidence Action’s Deworm the World Initiative has done in India. This goal is one we hope to achieve if we have time, but is not critical to our assessment.

Conclusion

The concrete goals we aim to achieve in 2019 follow. We plan to revisit this list in early 2020 to assess our progress relative to our expectations, and to publish a blog post accounting for our work:

• Building research capacity
  • Have 3-5 signed offer letters in hand from new research staff.
• Outreach experimentation
  • Decide whether to scale up Major Gifts work.
  • Conduct experiments related to messaging about our work to reach more people.
  • Complete search for a VP of Marketing.
• Exploring new areas of research
  • Look into several new areas, including public health regulation and possible paths to support government aid agencies. Get substantially closer to the point where we have the staff structure to support grantmaking in new areas.
  • Stretch goal: Settle on a structure that we believe will support grantmaking in public health regulation and begin recommending grants in that area.
• Improving GiveWell’s organizational strength
  • Hire one Operations Associate.
  • Improve the staff onboarding process at GiveWell.
  • Improve systems for soliciting feedback from staff.
  • Move to a new office.
• Ongoing research
  • Our full list of concrete research goals for 2019 is in this document.

Notes

1. The majority of donations in support of GiveWell’s recommended charities are made in the fourth quarter of the year, and we generally don’t have a clear sense of the total amount given to GiveWell directly until the first quarter of the following year (and the second quarter for direct-to-charity donations that are reported to us), so we think this is the right time frame on which to assess major gifts work.

The post GiveWell’s plans for 2019 appeared first on The GiveWell Blog.

Our top priorities this year support our goals to (a) increase the impact per dollar of the funds we direct and (b) increase our money moved. In 2019, we are focused on:

• Building research capacity. (More)
• Experimenting with approaches to outreach to find ones that we can scalably use to drive additional money moved. (More)
• Exploring new areas of research. (More)
• Improving GiveWell's organizational strength. (More)
• Ongoing research. (More)

Read More

The post GiveWell’s plans for 2019 appeared first on The GiveWell Blog.

Our top priorities this year support our goals to (a) increase the impact per dollar of the funds we direct and (b) increase our money moved. In 2019, we are focused on:

• Building research capacity. (More)
• Experimenting with approaches to outreach to find ones that we can scalably use to drive additional money moved. (More)
• Exploring new areas of research. (More)
• Improving GiveWell’s organizational strength. (More)
• Ongoing research. (More)

Building research capacity

We announced earlier this year our plans to hire researchers at three levels of seniority, listed here from most junior to most senior: Research Analyst, Senior Research Analyst, and Senior Fellow. Our goal is to have 3-5 signed offer letters in hand from new research staff by the end of 2019.

We’re hoping that additional research capacity will enable us to expand the scope of GiveWell’s research, with the aim of finding opportunities that are more cost-effective than our current top charities. We’re planning to roughly double the size of the research team over the next few years.

Outreach experimentation

We plan to expand our outreach to current and potential donors going forward, with the aim of increasing the amount of money we direct to our recommended charities. As part of this work, we recently hired Stephanie Stojanovic as our first Major Gifts Officer. Our goal is to decide by the end of the 1st quarter of 2020 whether to scale our staff capacity further in the area of major gifts, based on Stephanie’s initial work.[1]

We’re also planning to conduct experiments in 2019 related to how we message about our work to reach more people. These experiments could include work on search engine optimization and building landing pages that aim to communicate what GiveWell does and why it’s valuable, among other possibilities. We expect to have results by early 2020, as the bulk of donations we receive are made in December.

Finally, we’re planning to search for a VP of Marketing to oversee work across outreach domains (including major gifts, donor retention, advertising, marketing, and written communications). We guess there is a 50 percent chance we make a hire for this role in 2019.

Exploring new areas of research

As mentioned above, we’re in the early stages of expanding the scope of GiveWell’s research. We plan to look into several new areas in 2019, including public health regulation and possible paths to support government aid agencies.

This work is new for GiveWell, and as noted in our 2018 review post, we failed to make as much progress as we hoped in 2018 on our work on public health regulation. In 2019, we’re aiming to get substantially closer to the point where we have the staff structure to support grantmaking in new areas, though given how early we are in this work, we don’t yet have concrete goals we’re confident that we’ll achieve.

A stretch goal for 2019 is to settle on a structure that we believe will support grantmaking in public health regulation and begin recommending grants in that area.

We also plan to continue our investigation into possible paths to support government aid agencies; in particular, we plan to complete an investigation into an opportunity to do so in the area of results-based financing.

Improving GiveWell’s organizational strength

We expect to need additional operations capacity to maintain critical functions as GiveWell grows and to improve the organization going forward. We plan to hire one Operations Associate this year to assist with general operations needs, such as improving HR practices.

We plan to hire many new staff over the coming years. In preparation, we plan to improve our procedures and information for recruiting, vetting, and onboarding staff to GiveWell this year, such as by improving inclusive recruitment practices and updating the substantive content of our onboarding activities. We also plan to improve our systems for soliciting feedback from staff about how GiveWell can improve as an organization, in order to give management better insight into how things are going.

To accommodate our planned expansion, we plan to move to a new office that better suits our expected size and staff requirements.

Ongoing research

We have a number of ongoing research projects, detailed here. These include:

• Completing a full draft of qualitative assessments of our top charities. In theory, we aim to maximize one thing with our top charity recommendations—total improvement in well-being per dollar spent—and this is what our cost-effectiveness estimates intend to capture. In practice, there are costs and benefits that we do not observe and are not estimated in our models, and so we allow for qualitative adjustments to affect our recommendations. We’re in the process of laying out a framework for qualitatively assessing relative organizational strength.
• Updating key inputs into our cost-effectiveness estimates, such as:
  • How we use vitamin A deficiency data.
  • Using new malaria prevalence and child mortality data.
  • Using new data to update our estimates of costs incurred and target population reached for five of our top charities: Malaria Consortium’s seasonal malaria chemoprevention program, the Against Malaria Foundation, Helen Keller International’s vitamin A supplementation program, Sightsavers’ deworming program, and Evidence Action’s Deworm the World Initiative.
  • Better understanding the counterfactual to the work Evidence Action’s Deworm the World Initiative has done in India. This goal is one we hope to achieve if we have time, but is not critical to our assessment.

Conclusion

The concrete goals we aim to achieve in 2019 follow. We plan to revisit this list in early 2020 to assess our progress relative to our expectations, and to publish a blog post accounting for our work:

• Building research capacity
  • Have 3-5 signed offer letters in hand from new research staff.
• Outreach experimentation
  • Decide whether to scale up Major Gifts work.
  • Conduct experiments related to messaging about our work to reach more people.
  • Complete search for a VP of Marketing.
• Exploring new areas of research
  • Look into several new areas, including public health regulation and possible paths to support government aid agencies. Get substantially closer to the point where we have the staff structure to support grantmaking in new areas.
  • Stretch goal: Settle on a structure that we believe will support grantmaking in public health regulation and begin recommending grants in that area.
• Improving GiveWell’s organizational strength
  • Hire one Operations Associate.
  • Improve the staff onboarding process at GiveWell.
  • Improve systems for soliciting feedback from staff.
  • Move to a new office.
• Ongoing research
  • Our full list of concrete research goals for 2019 is in this document.

Notes

1. The majority of donations in support of GiveWell’s recommended charities are made in the fourth quarter of the year, and we generally don’t have a clear sense of the total amount given to GiveWell directly until the first quarter of the following year (and the second quarter for direct-to-charity donations that are reported to us), so we think this is the right time frame on which to assess major gifts work.

The post GiveWell’s plans for 2019 appeared first on The GiveWell Blog.

2018 was a successful year for GiveWell. We achieved most of our goals and our money moved (donations made to our recommended charities due our research) increased significantly.

Each year, we look back at the goals we set the previous year and reflect on how our progress compared to our expectations.

This post will briefly discuss our key achievements and failures in 2018. We describe in detail our progress on the goals we outlined in 2018 here.

In 2018, we:

• Directed an estimated $65 million in donations to our top charities, not including the contributions of Good Ventures, a large foundation with which we work closely.
• Added senior hires in operations and outreach: a Director of Operations (Whitney Shinkle) and Head of Growth (Ben Bateman). We expect Whitney and Ben to make major contributions to our work in these domains.
• Continued to improve and expand our core research product, completing new intervention reports, deepening our analysis for several key inputs into our cost-effectiveness model, and providing more transparent explanations for how we decided to allocate funds between top charities.

Key achievements

Donations made to top charities as a result of our research

We currently estimate that the amount of money we directed to our top charities in 2018 was more than $65 million, not including the contributions from Good Ventures, a large foundation with which we work closely. This represents an increase of more than $20 million over 2017. The increase largely came from two multi-million dollar donations from donors who had supported GiveWell and/or our recommended charities in the past.

We plan to publish a full report on our 2018 donations and web traffic shortly.

Outreach and operations

We made two key senior hires in 2018: (1) Whitney Shinkle, who joined us in April as our new Director of Operations, and (2) Ben Bateman, who joined us in June as our first-ever Head of Growth.

We expect Whitney and Ben to play critical roles in laying the foundation to increase the amount of funding we can direct to our top charities. Whitney’s team, for example, is responsible for processing donations to our recommended charities, and for preparing GiveWell to increase the size of its staff. Ben is leading experiments to evaluate different ways we might increase the amount of funding we direct to our top charities via marketing and outreach.

Full details of our performance against our 2018 outreach and operations goals are here.

Research

We completed several projects that improved the quality of our cost-effectiveness estimates and how we write about them, and that we believe led to better decisions about where to allocate funds. For example, we made a major change to how we calculate worm intensity in the areas where our top charities work.

We also improved our transparency about these decisions, breaking our blog posts announcing our top charities into component parts to make them easier to follow (see 1, 2, and 3) and delving into more detail on our principles and funding gap analyses.

We published five new intervention reports, two of which were on the evidence for community-based management of acute malnutrition and syphilis screening and treatment during pregnancy, and recommended five new GiveWell Incubation Grants and two grant renewals. Two of our new grants supported Evidence Action Beta’s incubator and J-PAL’s Innovation in Government Initiative, respectively.

Full details of our performance on our 2018 research goals are here.

Key failures

Outreach and operations

We took a number of steps to improve our outreach to GiveWell’s existing donors. We had hoped this would lead to material improvements in retention of our donors as well as the amount of funding we were able to direct to our top charities from our donors. We haven’t completed a careful assessment of this work, but our belief at this point is that the steps we took last year are unlikely to have had a significant impact on donor retention.

Research

We made relatively little progress in exploring new areas of research (i.e., policy-oriented causes).

Additional information

This page has more details on our progress toward the goals we laid out in early 2018.

We plan to publish a post soon detailing our plans for 2019.

The post Review of GiveWell’s work in 2018 appeared first on The GiveWell Blog.

2018 was a successful year for GiveWell. We achieved most of our goals and our money moved (donations made to our recommended charities due our research) increased significantly.

Each year, we look back at the goals we set the previous year and reflect on how our progress compared to our expectations.

This post will briefly discuss our key achievements and failures in 2018. We describe in detail our progress on the goals we outlined in 2018 here.

In 2018, we:

• Directed an estimated $65 million in donations to our top charities, not including the contributions of Good Ventures, a large foundation with which we work closely.
• Added senior hires in operations and outreach: a Director of Operations (Whitney Shinkle) and Head of Growth (Ben Bateman). We expect Whitney and Ben to make major contributions to our work in these domains.
• Continued to improve and expand our core research product, completing new intervention reports, deepening our analysis for several key inputs into our cost-effectiveness model, and providing more transparent explanations for how we decided to allocate funds between top charities.

Read More

The post Review of GiveWell’s work in 2018 appeared first on The GiveWell Blog.

2018 was a successful year for GiveWell. We achieved most of our goals and our money moved (donations made to our recommended charities due our research) increased significantly.

Each year, we look back at the goals we set the previous year and reflect on how our progress compared to our expectations.

This post will briefly discuss our key achievements and failures in 2018. We describe in detail our progress on the goals we outlined in 2018 here.

In 2018, we:

• Directed an estimated $65 million in donations to our top charities, not including the contributions of Good Ventures, a large foundation with which we work closely.
• Added senior hires in operations and outreach: a Director of Operations (Whitney Shinkle) and Head of Growth (Ben Bateman). We expect Whitney and Ben to make major contributions to our work in these domains.
• Continued to improve and expand our core research product, completing new intervention reports, deepening our analysis for several key inputs into our cost-effectiveness model, and providing more transparent explanations for how we decided to allocate funds between top charities.

Key achievements

Donations made to top charities as a result of our research

We currently estimate that the amount of money we directed to our top charities in 2018 was more than $65 million, not including the contributions from Good Ventures, a large foundation with which we work closely. This represents an increase of more than $20 million over 2017. The increase largely came from two multi-million dollar donations from donors who had supported GiveWell and/or our recommended charities in the past.

We plan to publish a full report on our 2018 donations and web traffic shortly.

Outreach and operations

We made two key senior hires in 2018: (1) Whitney Shinkle, who joined us in April as our new Director of Operations, and (2) Ben Bateman, who joined us in June as our first-ever Head of Growth.

We expect Whitney and Ben to play critical roles in laying the foundation to increase the amount of funding we can direct to our top charities. Whitney’s team, for example, is responsible for processing donations to our recommended charities, and for preparing GiveWell to increase the size of its staff. Ben is leading experiments to evaluate different ways we might increase the amount of funding we direct to our top charities via marketing and outreach.

Full details of our performance against our 2018 outreach and operations goals are here.

Research

We completed several projects that improved the quality of our cost-effectiveness estimates and how we write about them, and that we believe led to better decisions about where to allocate funds. For example, we made a major change to how we calculate worm intensity in the areas where our top charities work.

We also improved our transparency about these decisions, breaking our blog posts announcing our top charities into component parts to make them easier to follow (see 1, 2, and 3) and delving into more detail on our principles and funding gap analyses.

We published five new intervention reports, two of which were on the evidence for community-based management of acute malnutrition and syphilis screening and treatment during pregnancy, and recommended five new GiveWell Incubation Grants and two grant renewals. Two of our new grants supported Evidence Action Beta’s incubator and J-PAL’s Innovation in Government Initiative, respectively.

Full details of our performance on our 2018 research goals are here.

Key failures

Outreach and operations

We took a number of steps to improve our outreach to GiveWell’s existing donors. We had hoped this would lead to material improvements in retention of our donors as well as the amount of funding we were able to direct to our top charities from our donors. We haven’t completed a careful assessment of this work, but our belief at this point is that the steps we took last year are unlikely to have had a significant impact on donor retention.

Research

We made relatively little progress in exploring new areas of research (i.e., policy-oriented causes).

Additional information

This page has more details on our progress toward the goals we laid out in early 2018.

We plan to publish a post soon detailing our plans for 2019.

The post Review of GiveWell’s work in 2018 appeared first on The GiveWell Blog.

In the fourth quarter of 2018, donors gave a combined $7.6 million in funding to GiveWell for making grants at our discretion. In this post, we discuss the process we used to decide how to allocate this $7.6 million, as well as an additional $0.8 million designated for grants at GiveWell’s discretion held by the Centre for Effective Altruism and $1.7 million in the EA Fund for Global Health and Development (which is managed by GiveWell Executive Director Elie Hassenfeld), for a total of $10.1 million in funding. We’re so grateful to have a community of supporters that relies on our work and is open to allowing us to allocate funding to the top charity or charities we believe need it most.

We noted in November 2018 that we would use funds received for making grants at our discretion to fill the next highest priority funding gaps among our top charities. At the time, we wrote:

If we had additional funds to allocate now, the most likely recipient would be Malaria Consortium to scale up its work providing seasonal malaria chemoprevention.

Based on our analysis in 2018 as well as updates we have received from our top charities since that time, we have decided to allocate this $10.1 million in funding to Malaria Consortium’s seasonal malaria chemoprevention (SMC) program. The SMC program consists of treating children with a course of preventive antimalarial drugs during the time of year when malaria transmission is greatest.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to Malaria Consortium’s seasonal malaria chemoprevention program, which we believe could use additional funding for highly cost-effective work, even after receiving the $10.1 million in funding mentioned above.

What Malaria Consortium will do with additional funding

We wrote in detail about Malaria Consortium’s room for additional funding for its SMC program as of November 2018 here. We also spoke with Malaria Consortium for an update in early 2019. Our understanding of what Malaria Consortium will do with additional funding for its SMC program (including this $10.1 million), in order of priority, is as follows:

1. Contribute to filling a potential funding gap in Burkina Faso, the existence of which depends on the actions of other funders. If the gap materializes, filling it could require up to $3 million in addition to the $5 million that Malaria Consortium expects to have remaining on hand after what’s currently budgeted for 2019 and 2020.
2. Scale up further in Nigeria and Chad in 2020. Our impression is that, given drug production constraints and the length of time needed to plan for the implementation of a campaign, receiving additional funding now rather than in late 2019 (when we plan to make our next recommendation to Good Ventures to fund top charities) increases the likelihood that Malaria Consortium can use the funding for 2020 programs.
3. Fund the continuation of programs into 2021. Malaria Consortium has received enough funding to maintain its programs through 2020, but has not allocated funding to maintain programs beyond 2020. To maintain the 2019 program scale in 2021, Malaria Consortium would require an additional $14.8 million in funding, assuming no unbudgeted costs (e.g., additional scale-up) are incurred before then. Our impression is that there is little difference between receiving funding now and in late 2019 in terms of Malaria Consortium’s ability to use it to fund 2021 programs.

Overview of our decision-making process

In early 2019, we checked in with each of our top charities that seemed like plausible recipients of this funding, based on our assessment of their funding needs in late 2018. In general, these check-ins indicated that there weren’t updates in the marginal funding opportunities at our top charities. More details follow in the rest of this post. We refer below to “funding gaps,” which we use to describe the amount of additional funding that we believe could be used effectively (the gap between what charities could use and what they have on hand).

After considering each funding opportunity, we came to believe that the two most promising funding gaps are Malaria Consortium’s for SMC and the Against Malaria Foundation’s. The Against Malaria Foundation (AMF), which distributes insecticide-treated nets to prevent malaria, currently has the opportunity to fund nets in the Democratic Republic of Congo (DRC); we expect a high level of cost-effectiveness for this opportunity due to high malaria rates in DRC.

We discuss the comparison between these two funding opportunities in the next section. We followed the six principles described in this post in deciding between these two opportunities and ultimately decided to grant these funds to Malaria Consortium’s SMC program.

Comparing Malaria Consortium and AMF

What AMF would do with additional funding

In February 2019, AMF told us it had $62.8 million in uncommitted funds, which it plans to commit to a few 2020 net distributions (these are not yet formal commitments—as of February, AMF had not yet signed agreements with government partners to fund these distributions). AMF told us that if it had additional funding at this time, it would allocate those funds toward closing the gap in funding for nets in DRC for 2020. AMF has also shared more detailed information with us about its plans for the funds it holds and its negotiations with country governments; that information is confidential at this time. AMF reports that the total need for funding in DRC for a universal coverage campaign across eight provinces is between $35 million and $45 million.

Comparison using our principles

Principle 1: Put significant weight on our cost-effectiveness estimates.

We estimate that Malaria Consortium’s SMC program and AMF are similar in cost-effectiveness but that AMF is somewhat more cost-effective on the margin.

The most recent version of our published cost-effectiveness model at the time we made this decision (2019 version 2) estimates that Malaria Consortium is 8.5 times as cost-effective as unconditional cash transfers (“8.5x cash” for short) and AMF’s work in DRC is 10x cash (calculated by making a copy of the spreadsheet and selecting DRC in the “Country selection” tab for AMF).

Our best guess of the cost-effectiveness of these two opportunities incorporates several additional adjustments. See this footnote for details.1We adjust for our guess about how factors that are not formally modeled would change the results. For details, see column AB of this spreadsheet, sheet “Consolidated funding gaps.” This adjustment replicates what we did to arrive at our recommendations at the end of 2018. (More in this blog post.)

For both AMF and Malaria Consortium, we update the country-specific malaria mortality data to be more recent (2017 instead of 2016 figures). For Malaria Consortium, we correct what we believe to be an error in our model (which makes a roughly 5% difference in the final cost-effectiveness estimate), and we have also used an updated method (compared to what we used previously) to account for the fact that the age range of children targeted for SMC differs slightly from the age ranges given in the available age-specific mortality data (3 to 12 months vs. 1 to 12 months). We plan to incorporate these changes into the published model in the future.

For AMF we make several additional adjustments:

– We use DRC-specific cost data and adjustment for insecticide resistance. Our published cost-effectiveness model uses average data for these two parameters when a specific country is selected in the “Country selection” tab.

– We adjust the lifespan of a net downward by 10% for DRC. This is a rough guess based on findings from AMF’s past monitoring in DRC that suggested that nets wore out more quickly than in other locations where AMF has funded nets.

– We use a smaller fungibility adjustment than we do for other countries to capture the lower probability (compared to other countries where AMF operates) that DRC would reallocate funding that it receives from the Global Fund to Fight AIDS, Tuberculosis and Malaria to cover part of the funding gap for nets if AMF did not fund the distribution. Our understanding from conversations with AMF and the Global Fund is that DRC is relatively underfunded by the Global Fund, due to caps on how much it can spend in a single country and DRC’s large malaria burden, and so our guess is that there is less scope for reallocating funds from other malaria interventions to nets.

– We model most marginal funding as going to DRC, with some funding going to other countries. We do so firstly because we believe having additional funding on hand may lead AMF to commit more funding to other countries than it otherwise might, and secondly because of the possibility of AMF deciding not to commit additional funding or to cap the amount it provides to DRC if it has concerns about the quality of the 2019 distributions it is funding in DRC.

– We adjust AMF’s cost-effectiveness downward by 5% to account for the fact we recently learned that AMF has skipped some post-distribution surveys, leading us to update our estimate of potential misappropriation given missing monitoring results (see this spreadsheet).
jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

With these updates, our best guess of the cost-effectiveness of these two opportunities is that additional funding to Malaria Consortium is 8.3x cash and to AMF is 10.0x cash, implying that AMF is 21% more cost-effective.

This estimate has not yet been vetted, so is more likely to contain errors than our published cost-effectiveness model. To enable us to pursue other research work throughout the year, we thoroughly revisit our comparisons between top charities once per year for our annual recommendations refresh in November. When making recommendations at other times of year, we ask ourselves “Have there been any major changes that should lead us to reconsider what we concluded last November?” In this case, we adjusted some of the inputs into our cost-effectiveness model to reflect what we have learned since November and found that the results were broadly similar to our published model. At this level of difference in estimated cost-effectiveness, which is small in relation to the uncertainty in the model, we are inclined to put substantial weight on the other principles discussed below, and particularly on Principle 2.

We are also somewhat concerned that funding AMF may create an incentive for AMF to prioritize less cost-effective spending opportunities over more cost-effective ones, thus reducing AMF’s overall cost-effectiveness in the long run. We estimate that the three other countries AMF is in negotiations with are less cost-effective places to work than DRC. If we were to provide funding to AMF for work in DRC, we could be indicating that a “gaming” strategy—in which an organization tells us that marginal funds would go to a more cost-effective opportunity because its funds on hand have been allocated to less cost-effective opportunities—results in additional funding beyond what it would receive if it allocated funding to more cost-effective opportunities first. We don’t want to create an incentive for organizations to prioritize funding less cost-effective opportunities ahead of more cost-effective ones. We haven’t estimated the potential impact of this factor quantitatively.

Principle 2: Consider additional information about an organization that we have not explicitly modeled.

While we incorporate many subjective factors into our cost-effectiveness models, there are additional costs and benefits that we believe may affect the true cost-effectiveness and that we do not believe are adequately captured by our models. Such uncaptured factors might include, for example: information that charities have and we lack about how to best to allocate funding among different locations; beneficiary experiences with the program that affect how much they benefit from it; and the degree to which charities have indirect impact through conducting research, acting as leaders in their fields, or bringing in new sources of funding.

As we generally do not have the opportunity to observe or measure these costs and benefits directly, we consider them qualitatively through proxies. Such proxies include: our perception of how thoughtfully charities answer our questions; whether they are transparent about mistakes they make; how successful they have been in meeting operational goals (such as hiring, geographic expansion, and instituting new technical systems); whether they conduct and publish research; the frequency of errors in the information they share with us; and whether they meet agreed-upon timelines for sharing information.

We plan to write more about factors that we consider outside of our CEA model in the next few months, as well as assessments of each of our top charities on the proxies we use.

Overall, we assess Malaria Consortium as consistently stronger on the above qualitative proxies than AMF.  Both organizations stand out from the vast majority of organizations we have considered for their transparency about both positive and negative results and their track record of collecting information about how their programs are performing. They have both spent a large number of hours over several years (for Malaria Consortium) or over a decade (for AMF) responding to our questions and document requests. This comparison is a relative one, and one that we have not fully justified publicly (but plan to shortly). Based on our experiences working with both organizations, we believe that Malaria Consortium has shown signs of having stronger organizational management.

Principle 3: Assess charities’ funding gaps at the margin, i.e., where they would spend additional funding, where possible.

We’ve accounted for what Malaria Consortium and AMF are likely to do with marginal funding in our cost-effectiveness estimates, above.

Principle 4: Default towards not imposing restrictions on charity spending.

On this principle, there’s no difference between the two opportunities. Funding provided by GiveWell to either program would not be restricted.

Principle 5: Fund on a three-year horizon, unless we are particularly uncertain whether we will want to continue recommending a program in the future.

On this principle, there’s no difference between the two opportunities.

Principle 6: Ensure charities are incentivized to engage with our process.

This principle favors Malaria Consortium, which has consistently provided requested information that aids us in understanding and evaluating their program. AMF has more often been delayed or inconsistent in providing the information we’ve requested.

Other options we decided against (our other six top charities)

Schistosomiasis Control Initiative

The Schistosomiasis Control Initiative (SCI)’s room for additional funding is highly dependent on how much funding it receives from the UK’s Department for International Development (DFID) over the next three years. As of the time we were making this decision, we had not yet received an update on the level of funding that DFID plans to provide. More information is available in our review.

Helen Keller International’s vitamin A supplementation program

Helen Keller International (HKI) told us that it plans to use the funding it has already received for vitamin A supplementation as we expected: to continue its work in Mali, Burkina Faso, Guinea, and Côte d’Ivoire and to restart work in Niger. With additional funding it would prioritize work in:

• Kenya, where it could spend about $2 million over three years.
• Cameroon, where it could spend about $4.2 million over three years.
• Nigeria, where it could spend $0.6 million to conduct a study of the impact of technical assistance work.
• DRC, where it could spend about $9 million to reopen a country office and fund vitamin A supplementation over three years.

In November 2018, we estimated that these opportunities were less cost-effective than Malaria Consortium’s SMC program.2For HKI’s programs, see this spreadsheet, sheet “Consolidated funding gaps,” column AB. For Malaria Consortium’s overall SMC program, see same spreadsheet, sheet “Cost-effectiveness results,” row 6. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We did not revisit those calculations as part of the quarterly allocation process.

Evidence Action’s Deworm the World Initiative

Deworm the World has told us that it plans to follow the prioritization laid out in our recommendation to Good Ventures. That prioritization leaves the following opportunities unfunded:

• Extending its funding runway beyond 2020 to 2021.
• Holding sufficient funding for 2020 programming in India that is currently supported by other funders.
• Improving financial stability via increased reserves.
• Expanding to new locations (two states in India and one state in Nigeria).

At the end of 2018, we estimated that these opportunities were 15.0x cash on average; however, that average was largely driven by the opportunity to expand to two new states in India, which is relatively low priority for Deworm the World because it is prioritizing financial stability over further expansion. With that in mind, we prefer to allocate funding to Malaria Consortium.

Sightsavers’ deworming program

Sightsavers indicated to us that it plans to follow the funding priorities it presented in 2018, with the exception of one area where there is no longer room for more funding. As a result of that change, Sightsavers has sufficient funding for all remaining opportunities to fund deworming that it currently has capacity to implement.

END Fund’s deworming program

We didn’t ask the END Fund for an update on its funding needs in early 2019, as we didn’t expect that an update would lead us to allocate discretionary funding to its deworming program. More context for this decision is available here.

GiveDirectly

We didn’t ask GiveDirectly for an update on its funding needs in early 2019, as we didn’t expect that an update would lead us to allocate discretionary funding to its work. More context for this decision is available here.

Notes   [ + ]

1. ↑ We adjust for our guess about how factors that are not formally modeled would change the results. For details, see column AB of this spreadsheet, sheet “Consolidated funding gaps.” This adjustment replicates what we did to arrive at our recommendations at the end of 2018. (More in this blog post.)

For both AMF and Malaria Consortium, we update the country-specific malaria mortality data to be more recent (2017 instead of 2016 figures). For Malaria Consortium, we correct what we believe to be an error in our model (which makes a roughly 5% difference in the final cost-effectiveness estimate), and we have also used an updated method (compared to what we used previously) to account for the fact that the age range of children targeted for SMC differs slightly from the age ranges given in the available age-specific mortality data (3 to 12 months vs. 1 to 12 months). We plan to incorporate these changes into the published model in the future.

For AMF we make several additional adjustments:

– We use DRC-specific cost data and adjustment for insecticide resistance. Our published cost-effectiveness model uses average data for these two parameters when a specific country is selected in the “Country selection” tab.

– We adjust the lifespan of a net downward by 10% for DRC. This is a rough guess based on findings from AMF’s past monitoring in DRC that suggested that nets wore out more quickly than in other locations where AMF has funded nets.

– We use a smaller fungibility adjustment than we do for other countries to capture the lower probability (compared to other countries where AMF operates) that DRC would reallocate funding that it receives from the Global Fund to Fight AIDS, Tuberculosis and Malaria to cover part of the funding gap for nets if AMF did not fund the distribution. Our understanding from conversations with AMF and the Global Fund is that DRC is relatively underfunded by the Global Fund, due to caps on how much it can spend in a single country and DRC’s large malaria burden, and so our guess is that there is less scope for reallocating funds from other malaria interventions to nets.

– We model most marginal funding as going to DRC, with some funding going to other countries. We do so firstly because we believe having additional funding on hand may lead AMF to commit more funding to other countries than it otherwise might, and secondly because of the possibility of AMF deciding not to commit additional funding or to cap the amount it provides to DRC if it has concerns about the quality of the 2019 distributions it is funding in DRC.

– We adjust AMF’s cost-effectiveness downward by 5% to account for the fact we recently learned that AMF has skipped some post-distribution surveys, leading us to update our estimate of potential misappropriation given missing monitoring results (see this spreadsheet).
2. ↑ For HKI’s programs, see this spreadsheet, sheet “Consolidated funding gaps,” column AB. For Malaria Consortium’s overall SMC program, see same spreadsheet, sheet “Cost-effectiveness results,” row 6. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Allocation of discretionary funds from Q4 2018 appeared first on The GiveWell Blog.

In the fourth quarter of 2018, donors gave a combined $7.6 million in funding to GiveWell for making grants at our discretion. In this post, we discuss the process we used to decide how to allocate this $7.6 million, as well as an additional $0.8 million designated for grants at GiveWell’s discretion held by the Centre for Effective Altruism and $1.7 million in the EA Fund for Global Health and Development (which is managed by GiveWell Executive Director Elie Hassenfeld), for a total of $10.1 million in funding. We’re so grateful to have a community of supporters that relies on our work and is open to allowing us to allocate funding to the top charity or charities we believe need it most.

We noted in November 2018 that we would use funds received for making grants at our discretion to fill the next highest priority funding gaps among our top charities. At the time, we wrote:

If we had additional funds to allocate now, the most likely recipient would be Malaria Consortium to scale up its work providing seasonal malaria chemoprevention.

Based on our analysis in 2018 as well as updates we have received from our top charities since that time, we have decided to allocate this $10.1 million in funding to Malaria Consortium’s seasonal malaria chemoprevention (SMC) program. The SMC program consists of treating children with a course of preventive antimalarial drugs during the time of year when malaria transmission is greatest.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to Malaria Consortium’s seasonal malaria chemoprevention program, which we believe could use additional funding for highly cost-effective work, even after receiving the $10.1 million in funding mentioned above.

What Malaria Consortium will do with additional funding

We wrote in detail about Malaria Consortium’s room for additional funding for its SMC program as of November 2018 here. We also spoke with Malaria Consortium for an update in early 2019. Our understanding of what Malaria Consortium will do with additional funding for its SMC program (including this $10.1 million), in order of priority, is as follows:

1. Contribute to filling a potential funding gap in Burkina Faso, the existence of which depends on the actions of other funders. If the gap materializes, filling it could require up to $3 million in addition to the $5 million that Malaria Consortium expects to have remaining on hand after what’s currently budgeted for 2019 and 2020.
2. Scale up further in Nigeria and Chad in 2020. Our impression is that, given drug production constraints and the length of time needed to plan for the implementation of a campaign, receiving additional funding now rather than in late 2019 (when we plan to make our next recommendation to Good Ventures to fund top charities) increases the likelihood that Malaria Consortium can use the funding for 2020 programs.
3. Fund the continuation of programs into 2021. Malaria Consortium has received enough funding to maintain its programs through 2020, but has not allocated funding to maintain programs beyond 2020. To maintain the 2019 program scale in 2021, Malaria Consortium would require an additional $14.8 million in funding, assuming no unbudgeted costs (e.g., additional scale-up) are incurred before then. Our impression is that there is little difference between receiving funding now and in late 2019 in terms of Malaria Consortium’s ability to use it to fund 2021 programs.

Overview of our decision-making process

In early 2019, we checked in with each of our top charities that seemed like plausible recipients of this funding, based on our assessment of their funding needs in late 2018. In general, these check-ins indicated that there weren’t updates in the marginal funding opportunities at our top charities. More details follow in the rest of this post. We refer below to “funding gaps,” which we use to describe the amount of additional funding that we believe could be used effectively (the gap between what charities could use and what they have on hand).

After considering each funding opportunity, we came to believe that the two most promising funding gaps are Malaria Consortium’s for SMC and the Against Malaria Foundation’s. The Against Malaria Foundation (AMF), which distributes insecticide-treated nets to prevent malaria, currently has the opportunity to fund nets in the Democratic Republic of Congo (DRC); we expect a high level of cost-effectiveness for this opportunity due to high malaria rates in DRC.

We discuss the comparison between these two funding opportunities in the next section. We followed the six principles described in this post in deciding between these two opportunities and ultimately decided to grant these funds to Malaria Consortium’s SMC program.

Comparing Malaria Consortium and AMF

What AMF would do with additional funding

In February 2019, AMF told us it had $62.8 million in uncommitted funds, which it plans to commit to a few 2020 net distributions (these are not yet formal commitments—as of February, AMF had not yet signed agreements with government partners to fund these distributions). AMF told us that if it had additional funding at this time, it would allocate those funds toward closing the gap in funding for nets in DRC for 2020. AMF has also shared more detailed information with us about its plans for the funds it holds and its negotiations with country governments; that information is confidential at this time. AMF reports that the total need for funding in DRC for a universal coverage campaign across eight provinces is between $35 million and $45 million.

Comparison using our principles

Principle 1: Put significant weight on our cost-effectiveness estimates.

We estimate that Malaria Consortium’s SMC program and AMF are similar in cost-effectiveness but that AMF is somewhat more cost-effective on the margin.

The most recent version of our published cost-effectiveness model at the time we made this decision (2019 version 2) estimates that Malaria Consortium is 8.5 times as cost-effective as unconditional cash transfers (“8.5x cash” for short) and AMF’s work in DRC is 10x cash (calculated by making a copy of the spreadsheet and selecting DRC in the “Country selection” tab for AMF).

Our best guess of the cost-effectiveness of these two opportunities incorporates several additional adjustments. See this footnote for details.1We adjust for our guess about how factors that are not formally modeled would change the results. For details, see column AB of this spreadsheet, sheet “Consolidated funding gaps.” This adjustment replicates what we did to arrive at our recommendations at the end of 2018. (More in this blog post.)

For both AMF and Malaria Consortium, we update the country-specific malaria mortality data to be more recent (2017 instead of 2016 figures). For Malaria Consortium, we correct what we believe to be an error in our model (which makes a roughly 5% difference in the final cost-effectiveness estimate), and we have also used an updated method (compared to what we used previously) to account for the fact that the age range of children targeted for SMC differs slightly from the age ranges given in the available age-specific mortality data (3 to 12 months vs. 1 to 12 months). We plan to incorporate these changes into the published model in the future.

For AMF we make several additional adjustments:

– We use DRC-specific cost data and adjustment for insecticide resistance. Our published cost-effectiveness model uses average data for these two parameters when a specific country is selected in the “Country selection” tab.

– We adjust the lifespan of a net downward by 10% for DRC. This is a rough guess based on findings from AMF’s past monitoring in DRC that suggested that nets wore out more quickly than in other locations where AMF has funded nets.

– We use a smaller fungibility adjustment than we do for other countries to capture the lower probability (compared to other countries where AMF operates) that DRC would reallocate funding that it receives from the Global Fund to Fight AIDS, Tuberculosis and Malaria to cover part of the funding gap for nets if AMF did not fund the distribution. Our understanding from conversations with AMF and the Global Fund is that DRC is relatively underfunded by the Global Fund, due to caps on how much it can spend in a single country and DRC’s large malaria burden, and so our guess is that there is less scope for reallocating funds from other malaria interventions to nets.

– We model most marginal funding as going to DRC, with some funding going to other countries. We do so firstly because we believe having additional funding on hand may lead AMF to commit more funding to other countries than it otherwise might, and secondly because of the possibility of AMF deciding not to commit additional funding or to cap the amount it provides to DRC if it has concerns about the quality of the 2019 distributions it is funding in DRC.

– We adjust AMF’s cost-effectiveness downward by 5% to account for the fact we recently learned that AMF has skipped some post-distribution surveys, leading us to update our estimate of potential misappropriation given missing monitoring results (see this spreadsheet).
jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

With these updates, our best guess of the cost-effectiveness of these two opportunities is that additional funding to Malaria Consortium is 8.3x cash and to AMF is 10.0x cash, implying that AMF is 21% more cost-effective.

This estimate has not yet been vetted, so is more likely to contain errors than our published cost-effectiveness model. To enable us to pursue other research work throughout the year, we thoroughly revisit our comparisons between top charities once per year for our annual recommendations refresh in November. When making recommendations at other times of year, we ask ourselves “Have there been any major changes that should lead us to reconsider what we concluded last November?” In this case, we adjusted some of the inputs into our cost-effectiveness model to reflect what we have learned since November and found that the results were broadly similar to our published model. At this level of difference in estimated cost-effectiveness, which is small in relation to the uncertainty in the model, we are inclined to put substantial weight on the other principles discussed below, and particularly on Principle 2.

We are also somewhat concerned that funding AMF may create an incentive for AMF to prioritize less cost-effective spending opportunities over more cost-effective ones, thus reducing AMF’s overall cost-effectiveness in the long run. We estimate that the three other countries AMF is in negotiations with are less cost-effective places to work than DRC. If we were to provide funding to AMF for work in DRC, we could be indicating that a “gaming” strategy—in which an organization tells us that marginal funds would go to a more cost-effective opportunity because its funds on hand have been allocated to less cost-effective opportunities—results in additional funding beyond what it would receive if it allocated funding to more cost-effective opportunities first. We don’t want to create an incentive for organizations to prioritize funding less cost-effective opportunities ahead of more cost-effective ones. We haven’t estimated the potential impact of this factor quantitatively.

Principle 2: Consider additional information about an organization that we have not explicitly modeled.

While we incorporate many subjective factors into our cost-effectiveness models, there are additional costs and benefits that we believe may affect the true cost-effectiveness and that we do not believe are adequately captured by our models. Such uncaptured factors might include, for example: information that charities have and we lack about how to best to allocate funding among different locations; beneficiary experiences with the program that affect how much they benefit from it; and the degree to which charities have indirect impact through conducting research, acting as leaders in their fields, or bringing in new sources of funding.

As we generally do not have the opportunity to observe or measure these costs and benefits directly, we consider them qualitatively through proxies. Such proxies include: our perception of how thoughtfully charities answer our questions; whether they are transparent about mistakes they make; how successful they have been in meeting operational goals (such as hiring, geographic expansion, and instituting new technical systems); whether they conduct and publish research; the frequency of errors in the information they share with us; and whether they meet agreed-upon timelines for sharing information.

We plan to write more about factors that we consider outside of our CEA model in the next few months, as well as assessments of each of our top charities on the proxies we use.

Overall, we assess Malaria Consortium as consistently stronger on the above qualitative proxies than AMF.  Both organizations stand out from the vast majority of organizations we have considered for their transparency about both positive and negative results and their track record of collecting information about how their programs are performing. They have both spent a large number of hours over several years (for Malaria Consortium) or over a decade (for AMF) responding to our questions and document requests. This comparison is a relative one, and one that we have not fully justified publicly (but plan to shortly). Based on our experiences working with both organizations, we believe that Malaria Consortium has shown signs of having stronger organizational management.

Principle 3: Assess charities’ funding gaps at the margin, i.e., where they would spend additional funding, where possible.

We’ve accounted for what Malaria Consortium and AMF are likely to do with marginal funding in our cost-effectiveness estimates, above.

Principle 4: Default towards not imposing restrictions on charity spending.

On this principle, there’s no difference between the two opportunities. Funding provided by GiveWell to either program would not be restricted.

Principle 5: Fund on a three-year horizon, unless we are particularly uncertain whether we will want to continue recommending a program in the future.

On this principle, there’s no difference between the two opportunities.

Principle 6: Ensure charities are incentivized to engage with our process.

This principle favors Malaria Consortium, which has consistently provided requested information that aids us in understanding and evaluating their program. AMF has more often been delayed or inconsistent in providing the information we’ve requested.

Other options we decided against (our other six top charities)

Schistosomiasis Control Initiative

The Schistosomiasis Control Initiative (SCI)’s room for additional funding is highly dependent on how much funding it receives from the UK’s Department for International Development (DFID) over the next three years. As of the time we were making this decision, we had not yet received an update on the level of funding that DFID plans to provide. More information is available in our review.

Helen Keller International’s vitamin A supplementation program

Helen Keller International (HKI) told us that it plans to use the funding it has already received for vitamin A supplementation as we expected: to continue its work in Mali, Burkina Faso, Guinea, and Côte d’Ivoire and to restart work in Niger. With additional funding it would prioritize work in:

• Kenya, where it could spend about $2 million over three years.
• Cameroon, where it could spend about $4.2 million over three years.
• Nigeria, where it could spend $0.6 million to conduct a study of the impact of technical assistance work.
• DRC, where it could spend about $9 million to reopen a country office and fund vitamin A supplementation over three years.

In November 2018, we estimated that these opportunities were less cost-effective than Malaria Consortium’s SMC program.2For HKI’s programs, see this spreadsheet, sheet “Consolidated funding gaps,” column AB. For Malaria Consortium’s overall SMC program, see same spreadsheet, sheet “Cost-effectiveness results,” row 6. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We did not revisit those calculations as part of the quarterly allocation process.

Evidence Action’s Deworm the World Initiative

Deworm the World has told us that it plans to follow the prioritization laid out in our recommendation to Good Ventures. That prioritization leaves the following opportunities unfunded:

• Extending its funding runway beyond 2020 to 2021.
• Holding sufficient funding for 2020 programming in India that is currently supported by other funders.
• Improving financial stability via increased reserves.
• Expanding to new locations (two states in India and one state in Nigeria).

At the end of 2018, we estimated that these opportunities were 15.0x cash on average; however, that average was largely driven by the opportunity to expand to two new states in India, which is relatively low priority for Deworm the World because it is prioritizing financial stability over further expansion. With that in mind, we prefer to allocate funding to Malaria Consortium.

Sightsavers’ deworming program

Sightsavers indicated to us that it plans to follow the funding priorities it presented in 2018, with the exception of one area where there is no longer room for more funding. As a result of that change, Sightsavers has sufficient funding for all remaining opportunities to fund deworming that it currently has capacity to implement.

END Fund’s deworming program

We didn’t ask the END Fund for an update on its funding needs in early 2019, as we didn’t expect that an update would lead us to allocate discretionary funding to its deworming program. More context for this decision is available here.

GiveDirectly

We didn’t ask GiveDirectly for an update on its funding needs in early 2019, as we didn’t expect that an update would lead us to allocate discretionary funding to its work. More context for this decision is available here.

Notes   [ + ]

1. ↑ We adjust for our guess about how factors that are not formally modeled would change the results. For details, see column AB of this spreadsheet, sheet “Consolidated funding gaps.” This adjustment replicates what we did to arrive at our recommendations at the end of 2018. (More in this blog post.)

For both AMF and Malaria Consortium, we update the country-specific malaria mortality data to be more recent (2017 instead of 2016 figures). For Malaria Consortium, we correct what we believe to be an error in our model (which makes a roughly 5% difference in the final cost-effectiveness estimate), and we have also used an updated method (compared to what we used previously) to account for the fact that the age range of children targeted for SMC differs slightly from the age ranges given in the available age-specific mortality data (3 to 12 months vs. 1 to 12 months). We plan to incorporate these changes into the published model in the future.

For AMF we make several additional adjustments:

– We use DRC-specific cost data and adjustment for insecticide resistance. Our published cost-effectiveness model uses average data for these two parameters when a specific country is selected in the “Country selection” tab.

– We adjust the lifespan of a net downward by 10% for DRC. This is a rough guess based on findings from AMF’s past monitoring in DRC that suggested that nets wore out more quickly than in other locations where AMF has funded nets.

– We use a smaller fungibility adjustment than we do for other countries to capture the lower probability (compared to other countries where AMF operates) that DRC would reallocate funding that it receives from the Global Fund to Fight AIDS, Tuberculosis and Malaria to cover part of the funding gap for nets if AMF did not fund the distribution. Our understanding from conversations with AMF and the Global Fund is that DRC is relatively underfunded by the Global Fund, due to caps on how much it can spend in a single country and DRC’s large malaria burden, and so our guess is that there is less scope for reallocating funds from other malaria interventions to nets.

– We model most marginal funding as going to DRC, with some funding going to other countries. We do so firstly because we believe having additional funding on hand may lead AMF to commit more funding to other countries than it otherwise might, and secondly because of the possibility of AMF deciding not to commit additional funding or to cap the amount it provides to DRC if it has concerns about the quality of the 2019 distributions it is funding in DRC.

– We adjust AMF’s cost-effectiveness downward by 5% to account for the fact we recently learned that AMF has skipped some post-distribution surveys, leading us to update our estimate of potential misappropriation given missing monitoring results (see this spreadsheet).
2. ↑ For HKI’s programs, see this spreadsheet, sheet “Consolidated funding gaps,” column AB. For Malaria Consortium’s overall SMC program, see same spreadsheet, sheet “Cost-effectiveness results,” row 6. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Allocation of discretionary funds from Q4 2018 appeared first on The GiveWell Blog.

In the fourth quarter of 2018, donors gave a combined $7.6 million in funding to GiveWell for making grants at our discretion. In this post, we discuss the process we used to decide how to allocate this $7.6 million, as well as an additional $0.8 million designated for grants at GiveWell’s discretion held by the Centre for Effective Altruism and $1.7 million in the EA Fund for Global Health and Development (which is managed by GiveWell Executive Director Elie Hassenfeld), for a total of $10.1 million in funding. We’re so grateful to have a community of supporters that relies on our work and is open to allowing us to allocate funding to the top charity or charities we believe need it most.

We noted in November 2018 that we would use funds received for making grants at our discretion to fill the next highest priority funding gaps among our top charities. At the time, we wrote:

If we had additional funds to allocate now, the most likely recipient would be Malaria Consortium to scale up its work providing seasonal malaria chemoprevention.

Based on our analysis in 2018 as well as updates we have received from our top charities since that time, we have decided to allocate this $10.1 million in funding to Malaria Consortium’s seasonal malaria chemoprevention (SMC) program. The SMC program consists of treating children with a course of preventive antimalarial drugs during the time of year when malaria transmission is greatest.

We continue to recommend that donors giving to GiveWell choose the option on our donation form for “grants to recommended charities at GiveWell’s discretion” so that we can direct the funding to the top charity or charities with the most pressing funding needs. For donors who prefer to give to a specific charity, we note that if we had additional funds to allocate at this time, we would very likely allocate them to Malaria Consortium’s seasonal malaria chemoprevention program, which we believe could use additional funding for highly cost-effective work, even after receiving the $10.1 million in funding mentioned above.

What Malaria Consortium will do with additional funding

We wrote in detail about Malaria Consortium’s room for additional funding for its SMC program as of November 2018 here. We also spoke with Malaria Consortium for an update in early 2019. Our understanding of what Malaria Consortium will do with additional funding for its SMC program (including this $10.1 million), in order of priority, is as follows:

1. Contribute to filling a potential funding gap in Burkina Faso, the existence of which depends on the actions of other funders. If the gap materializes, filling it could require up to $3 million in addition to the $5 million that Malaria Consortium expects to have remaining on hand after what’s currently budgeted for 2019 and 2020.
2. Scale up further in Nigeria and Chad in 2020. Our impression is that, given drug production constraints and the length of time needed to plan for the implementation of a campaign, receiving additional funding now rather than in late 2019 (when we plan to make our next recommendation to Good Ventures to fund top charities) increases the likelihood that Malaria Consortium can use the funding for 2020 programs.
3. Fund the continuation of programs into 2021. Malaria Consortium has received enough funding to maintain its programs through 2020, but has not allocated funding to maintain programs beyond 2020. To maintain the 2019 program scale in 2021, Malaria Consortium would require an additional $14.8 million in funding, assuming no unbudgeted costs (e.g., additional scale-up) are incurred before then. Our impression is that there is little difference between receiving funding now and in late 2019 in terms of Malaria Consortium’s ability to use it to fund 2021 programs.

Overview of our decision-making process

In early 2019, we checked in with each of our top charities that seemed like plausible recipients of this funding, based on our assessment of their funding needs in late 2018. In general, these check-ins indicated that there weren’t updates in the marginal funding opportunities at our top charities. More details follow in the rest of this post. We refer below to “funding gaps,” which we use to describe the amount of additional funding that we believe could be used effectively (the gap between what charities could use and what they have on hand).

After considering each funding opportunity, we came to believe that the two most promising funding gaps are Malaria Consortium’s for SMC and the Against Malaria Foundation’s. The Against Malaria Foundation (AMF), which distributes insecticide-treated nets to prevent malaria, currently has the opportunity to fund nets in the Democratic Republic of Congo (DRC); we expect a high level of cost-effectiveness for this opportunity due to high malaria rates in DRC.

We discuss the comparison between these two funding opportunities in the next section. We followed the six principles described in this post in deciding between these two opportunities and ultimately decided to grant these funds to Malaria Consortium’s SMC program.

Comparing Malaria Consortium and AMF

What AMF would do with additional funding

In February 2019, AMF told us it had $62.8 million in uncommitted funds, which it plans to commit to a few 2020 net distributions (these are not yet formal commitments—as of February, AMF had not yet signed agreements with government partners to fund these distributions). AMF told us that if it had additional funding at this time, it would allocate those funds toward closing the gap in funding for nets in DRC for 2020. AMF has also shared more detailed information with us about its plans for the funds it holds and its negotiations with country governments; that information is confidential at this time. AMF reports that the total need for funding in DRC for a universal coverage campaign across eight provinces is between $35 million and $45 million.

Comparison using our principles

Principle 1: Put significant weight on our cost-effectiveness estimates.

We estimate that Malaria Consortium’s SMC program and AMF are similar in cost-effectiveness but that AMF is somewhat more cost-effective on the margin.

The most recent version of our published cost-effectiveness model at the time we made this decision (2019 version 2) estimates that Malaria Consortium is 8.5 times as cost-effective as unconditional cash transfers (“8.5x cash” for short) and AMF’s work in DRC is 10x cash (calculated by making a copy of the spreadsheet and selecting DRC in the “Country selection” tab for AMF).

Our best guess of the cost-effectiveness of these two opportunities incorporates several additional adjustments. See this footnote for details.1We adjust for our guess about how factors that are not formally modeled would change the results. For details, see column AB of this spreadsheet, sheet “Consolidated funding gaps.” This adjustment replicates what we did to arrive at our recommendations at the end of 2018. (More in this blog post.)

For both AMF and Malaria Consortium, we update the country-specific malaria mortality data to be more recent (2017 instead of 2016 figures). For Malaria Consortium, we correct what we believe to be an error in our model (which makes a roughly 5% difference in the final cost-effectiveness estimate), and we have also used an updated method (compared to what we used previously) to account for the fact that the age range of children targeted for SMC differs slightly from the age ranges given in the available age-specific mortality data (3 to 12 months vs. 1 to 12 months). We plan to incorporate these changes into the published model in the future.

For AMF we make several additional adjustments:

– We use DRC-specific cost data and adjustment for insecticide resistance. Our published cost-effectiveness model uses average data for these two parameters when a specific country is selected in the “Country selection” tab.

– We adjust the lifespan of a net downward by 10% for DRC. This is a rough guess based on findings from AMF’s past monitoring in DRC that suggested that nets wore out more quickly than in other locations where AMF has funded nets.

– We use a smaller fungibility adjustment than we do for other countries to capture the lower probability (compared to other countries where AMF operates) that DRC would reallocate funding that it receives from the Global Fund to Fight AIDS, Tuberculosis and Malaria to cover part of the funding gap for nets if AMF did not fund the distribution. Our understanding from conversations with AMF and the Global Fund is that DRC is relatively underfunded by the Global Fund, due to caps on how much it can spend in a single country and DRC’s large malaria burden, and so our guess is that there is less scope for reallocating funds from other malaria interventions to nets.

– We model most marginal funding as going to DRC, with some funding going to other countries. We do so firstly because we believe having additional funding on hand may lead AMF to commit more funding to other countries than it otherwise might, and secondly because of the possibility of AMF deciding not to commit additional funding or to cap the amount it provides to DRC if it has concerns about the quality of the 2019 distributions it is funding in DRC.

– We adjust AMF’s cost-effectiveness downward by 5% to account for the fact we recently learned that AMF has skipped some post-distribution surveys, leading us to update our estimate of potential misappropriation given missing monitoring results (see this spreadsheet).
jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

With these updates, our best guess of the cost-effectiveness of these two opportunities is that additional funding to Malaria Consortium is 8.3x cash and to AMF is 10.0x cash, implying that AMF is 21% more cost-effective.

This estimate has not yet been vetted, so is more likely to contain errors than our published cost-effectiveness model. To enable us to pursue other research work throughout the year, we thoroughly revisit our comparisons between top charities once per year for our annual recommendations refresh in November. When making recommendations at other times of year, we ask ourselves “Have there been any major changes that should lead us to reconsider what we concluded last November?” In this case, we adjusted some of the inputs into our cost-effectiveness model to reflect what we have learned since November and found that the results were broadly similar to our published model. At this level of difference in estimated cost-effectiveness, which is small in relation to the uncertainty in the model, we are inclined to put substantial weight on the other principles discussed below, and particularly on Principle 2.

We are also somewhat concerned that funding AMF may create an incentive for AMF to prioritize less cost-effective spending opportunities over more cost-effective ones, thus reducing AMF’s overall cost-effectiveness in the long run. We estimate that the three other countries AMF is in negotiations with are less cost-effective places to work than DRC. If we were to provide funding to AMF for work in DRC, we could be indicating that a “gaming” strategy—in which an organization tells us that marginal funds would go to a more cost-effective opportunity because its funds on hand have been allocated to less cost-effective opportunities—results in additional funding beyond what it would receive if it allocated funding to more cost-effective opportunities first. We don’t want to create an incentive for organizations to prioritize funding less cost-effective opportunities ahead of more cost-effective ones. We haven’t estimated the potential impact of this factor quantitatively.

Principle 2: Consider additional information about an organization that we have not explicitly modeled.

While we incorporate many subjective factors into our cost-effectiveness models, there are additional costs and benefits that we believe may affect the true cost-effectiveness and that we do not believe are adequately captured by our models. Such uncaptured factors might include, for example: information that charities have and we lack about how to best to allocate funding among different locations; beneficiary experiences with the program that affect how much they benefit from it; and the degree to which charities have indirect impact through conducting research, acting as leaders in their fields, or bringing in new sources of funding.

As we generally do not have the opportunity to observe or measure these costs and benefits directly, we consider them qualitatively through proxies. Such proxies include: our perception of how thoughtfully charities answer our questions; whether they are transparent about mistakes they make; how successful they have been in meeting operational goals (such as hiring, geographic expansion, and instituting new technical systems); whether they conduct and publish research; the frequency of errors in the information they share with us; and whether they meet agreed-upon timelines for sharing information.

We plan to write more about factors that we consider outside of our CEA model in the next few months, as well as assessments of each of our top charities on the proxies we use.

Overall, we assess Malaria Consortium as consistently stronger on the above qualitative proxies than AMF.  Both organizations stand out from the vast majority of organizations we have considered for their transparency about both positive and negative results and their track record of collecting information about how their programs are performing. They have both spent a large number of hours over several years (for Malaria Consortium) or over a decade (for AMF) responding to our questions and document requests. This comparison is a relative one, and one that we have not fully justified publicly (but plan to shortly). Based on our experiences working with both organizations, we believe that Malaria Consortium has shown signs of having stronger organizational management.

Principle 3: Assess charities’ funding gaps at the margin, i.e., where they would spend additional funding, where possible.

We’ve accounted for what Malaria Consortium and AMF are likely to do with marginal funding in our cost-effectiveness estimates, above.

Principle 4: Default towards not imposing restrictions on charity spending.

On this principle, there’s no difference between the two opportunities. Funding provided by GiveWell to either program would not be restricted.

Principle 5: Fund on a three-year horizon, unless we are particularly uncertain whether we will want to continue recommending a program in the future.

On this principle, there’s no difference between the two opportunities.

Principle 6: Ensure charities are incentivized to engage with our process.

This principle favors Malaria Consortium, which has consistently provided requested information that aids us in understanding and evaluating their program. AMF has more often been delayed or inconsistent in providing the information we’ve requested.

Other options we decided against (our other six top charities)

Schistosomiasis Control Initiative

The Schistosomiasis Control Initiative (SCI)’s room for additional funding is highly dependent on how much funding it receives from the UK’s Department for International Development (DFID) over the next three years. As of the time we were making this decision, we had not yet received an update on the level of funding that DFID plans to provide. More information is available in our review.

Helen Keller International’s vitamin A supplementation program

Helen Keller International (HKI) told us that it plans to use the funding it has already received for vitamin A supplementation as we expected: to continue its work in Mali, Burkina Faso, Guinea, and Côte d’Ivoire and to restart work in Niger. With additional funding it would prioritize work in:

• Kenya, where it could spend about $2 million over three years.
• Cameroon, where it could spend about $4.2 million over three years.
• Nigeria, where it could spend $0.6 million to conduct a study of the impact of technical assistance work.
• DRC, where it could spend about $9 million to reopen a country office and fund vitamin A supplementation over three years.

In November 2018, we estimated that these opportunities were less cost-effective than Malaria Consortium’s SMC program.2For HKI’s programs, see this spreadsheet, sheet “Consolidated funding gaps,” column AB. For Malaria Consortium’s overall SMC program, see same spreadsheet, sheet “Cost-effectiveness results,” row 6. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); We did not revisit those calculations as part of the quarterly allocation process.

Evidence Action’s Deworm the World Initiative

Deworm the World has told us that it plans to follow the prioritization laid out in our recommendation to Good Ventures. That prioritization leaves the following opportunities unfunded:

• Extending its funding runway beyond 2020 to 2021.
• Holding sufficient funding for 2020 programming in India that is currently supported by other funders.
• Improving financial stability via increased reserves.
• Expanding to new locations (two states in India and one state in Nigeria).

At the end of 2018, we estimated that these opportunities were 15.0x cash on average; however, that average was largely driven by the opportunity to expand to two new states in India, which is relatively low priority for Deworm the World because it is prioritizing financial stability over further expansion. With that in mind, we prefer to allocate funding to Malaria Consortium.

Sightsavers’ deworming program

Sightsavers indicated to us that it plans to follow the funding priorities it presented in 2018, with the exception of one area where there is no longer room for more funding. As a result of that change, Sightsavers has sufficient funding for all remaining opportunities to fund deworming that it currently has capacity to implement.

END Fund’s deworming program

We didn’t ask the END Fund for an update on its funding needs in early 2019, as we didn’t expect that an update would lead us to allocate discretionary funding to its deworming program. More context for this decision is available here.

GiveDirectly

We didn’t ask GiveDirectly for an update on its funding needs in early 2019, as we didn’t expect that an update would lead us to allocate discretionary funding to its work. More context for this decision is available here.

Notes   [ + ]

1. ↑ We adjust for our guess about how factors that are not formally modeled would change the results. For details, see column AB of this spreadsheet, sheet “Consolidated funding gaps.” This adjustment replicates what we did to arrive at our recommendations at the end of 2018. (More in this blog post.)

For both AMF and Malaria Consortium, we update the country-specific malaria mortality data to be more recent (2017 instead of 2016 figures). For Malaria Consortium, we correct what we believe to be an error in our model (which makes a roughly 5% difference in the final cost-effectiveness estimate), and we have also used an updated method (compared to what we used previously) to account for the fact that the age range of children targeted for SMC differs slightly from the age ranges given in the available age-specific mortality data (3 to 12 months vs. 1 to 12 months). We plan to incorporate these changes into the published model in the future.

For AMF we make several additional adjustments:

– We use DRC-specific cost data and adjustment for insecticide resistance. Our published cost-effectiveness model uses average data for these two parameters when a specific country is selected in the “Country selection” tab.

– We adjust the lifespan of a net downward by 10% for DRC. This is a rough guess based on findings from AMF’s past monitoring in DRC that suggested that nets wore out more quickly than in other locations where AMF has funded nets.

– We use a smaller fungibility adjustment than we do for other countries to capture the lower probability (compared to other countries where AMF operates) that DRC would reallocate funding that it receives from the Global Fund to Fight AIDS, Tuberculosis and Malaria to cover part of the funding gap for nets if AMF did not fund the distribution. Our understanding from conversations with AMF and the Global Fund is that DRC is relatively underfunded by the Global Fund, due to caps on how much it can spend in a single country and DRC’s large malaria burden, and so our guess is that there is less scope for reallocating funds from other malaria interventions to nets.

– We model most marginal funding as going to DRC, with some funding going to other countries. We do so firstly because we believe having additional funding on hand may lead AMF to commit more funding to other countries than it otherwise might, and secondly because of the possibility of AMF deciding not to commit additional funding or to cap the amount it provides to DRC if it has concerns about the quality of the 2019 distributions it is funding in DRC.

– We adjust AMF’s cost-effectiveness downward by 5% to account for the fact we recently learned that AMF has skipped some post-distribution surveys, leading us to update our estimate of potential misappropriation given missing monitoring results (see this spreadsheet).
2. ↑ For HKI’s programs, see this spreadsheet, sheet “Consolidated funding gaps,” column AB. For Malaria Consortium’s overall SMC program, see same spreadsheet, sheet “Cost-effectiveness results,” row 6. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Allocation of discretionary funds from Q4 2018 appeared first on The GiveWell Blog.