We have published a more recent review of this organization. See our most recent report on AMF.
Donate
The Against Malaria Foundation (AMF) is one of our top-rated charities, and an organization that we feel offers donors an outstanding opportunity to accomplish good with their donations.
More information: What is our evaluation process?
Published: June 2016
Summary
What do they do? AMF provides funding for long-lasting insecticide-treated net (LLIN) distributions (for protection against malaria) in developing countries.
Does it work? There is strong evidence that distributing nets reduces child mortality and malaria cases. AMF has relatively strong reporting requirements for its distribution partners and provides a level of public disclosure and tracking of distributions that we have not seen from any other net distribution charity. AMF's post-distribution surveys have generally found positive results, but have some methodological limitations.
What do you get for your dollar? We estimate that the cost to purchase and distribute an AMF-funded net is $4.68 in Malawi and, very roughly, $6.06 in DRC (the two countries that AMF has worked most extensively in). We also very roughly estimate that the cost per net in distributions AMF is considering funding is $4.47. The numbers of malaria cases prevented and lives saved are a function of a number of difficult to estimate factors, which we discuss in detail below.
Is there room for more funding? We estimate that AMF is fairly likely to be able to commit an additional $11.3 million to distributions in 2016-2017 (beyond the $22.6 million it has on hand) over the next six months. In addition to the $11.3 million, it may be able to use up to $18 million more for additional 2016-2017 distributions. It could use substantially more to commit to 2018-2019 distributions, but these funds may displace funding from other funders.
AMF is recommended because of its:
- Focus on a program with excellent evidence of effectiveness and cost-effectiveness.
- Relatively strong processes for ensuring that nets reach their intended recipients and monitoring whether they are used over the long-term.
- Standout transparency and commitment to self-evaluation.
- Room for more funding – we believe AMF will be able to use additional funds to deliver additional nets.
Our main concerns:
- AMF has completed only seven large distributions, the type it aims to focus on in the future. Additionally, it has only worked with two distribution partners on its large distributions. Its track record of collecting the data it seeks from these distributions is good so far but limited due to the small number of distributions. In particular, we have seen only one follow up survey from distributions outside of Malawi, and this survey appears to be of lower quality than previous surveys from Malawi. Many of the distributions AMF hopes to fund are in countries where it has never worked, which may present unforeseen challenges.
- Recently, AMF has been slower to share documentation from some distributions. AMF has told us that it has this documentation and we are concerned that AMF is not being as transparent as it could be. We believe this documentation is important for monitoring the quality of AMF's distributions. AMF attributes the delays to lack of staff capacity. See response from AMF to this concern below.
- The best evidence for nets was collected before they were widely used and there is some evidence that mosquitoes have since adapted to the insecticide used in nets, possibly making them less effective. It seems that insecticide resistance is a growing concern, but it remains difficult to quantify the impact of resistance. There are several studies in progress that may help shed light on this within the next year. We discuss this issue in more detail in our page on this topic.
Table of Contents
- Summary
- Our review process
- A few key terms for this review
- What do they do?
-
Does it work?
- Are LLINs targeted at people who do not already have them?
- Do LLINs reach the intended destination?
- Are LLINs targeted at areas with high rates of malaria?
- Do those who receive the LLINs install them in their homes properly? Do those who receive the LLINs utilize them consistently over the long term?
- Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF?
- Possible negative or offsetting impact
- What do you get for your dollar?
- Is there room for more funding?
- AMF as an organization
- Sources
Our review process
We began reviewing AMF in 2009. Our review has consisted of:
- Reviewing AMF's public records for each of its net distributions and other documents AMF has shared with us.
- Extensive communication, including several meetings at AMF's London headquarters, with AMF Founder Rob Mather to discuss AMF's methods and funding needs.
- A visit to AMF's distribution partner organization, Concern Universal, in Malawi in October 2011 (notes and photos from this visit). We spoke with Concern Universal by phone in April 2016.
- Reviewing materials from AMF’s distributions in Malawi and the Democratic Republic of the Congo (DRC), including pre-distribution registration data, distribution reports, and data from post-distribution follow up surveys.
- Conversations with Peter Sherratt, AMF's Executive Chairman; Don de Savigny, a member of AMF's Malaria Advisory Group, and other individuals (who requested to remain anonymous) familiar with AMF's work and its attempts to finalize distributions.
- Conversations with Melanie Renshaw of the African Leaders Malaria Alliance and Marcy Erskine of the International Federation of the Red Cross about funding needs for nets.
All content on AMF, including past reviews, updates, blog posts and conversation notes, is available here.
A few key terms for this review
- A net refers to any bed net intended to block mosquitoes, whether treated with insecticide or not.
- An ITN is an insecticide-treated net. As discussed at our write-up on ITNs, distribution of ITNs has been rigorously studied and associated with declines in under-5 mortality and overall malaria cases.
- A LLIN is a long-lasting insecticide-treated net, intended to serve as an effective ITN for several years (details at our write-up on ITNs). The nets distributed by AMF are LLINs,1 and (as noted in our write-up on large-scale ITN distributions) most large-scale donor-financed malaria control today utilizes LLINs.
What do they do?
AMF provides long-lasting insecticide-treated nets (for protection against malaria) in bulk to other organizations, which then distribute the nets in developing countries.
Prior to November 2011, AMF contributed LLINs to small-scale distributions. Since then, it has de-emphasized small-scale distributions and it expects larger-scale distributions to be the primary way in which it distributes LLINs in the future (more below).
As of May 2016, AMF had completed seven large-scale distributions (and one small-scale emergency net distribution after flooding in Malawi) with two distribution partners for a total of 2.39 million LLINs distributed.2 An additional distribution, of 705,468 LLINs, is currently underway.3
AMF has signed agreements to fund two distributions in the second half of 2016 and early 2017 (approximately 2.69 million LLINs in Ghana, and approximately 10.74 million LLINs in Uganda), and an agreement to fund 1.48 million LLINs in 2018 in Malawi.4
Below, we provide a summary of AMF's medium and large-scale distributions.5 A more detailed summary of AMF's distributions can be found in our Summary of AMF Distributions spreadsheet.
| Location | Distribution partner | Number of LLINs | Cost to AMF (millions USD)6 | Timing |
|---|---|---|---|---|
| Ntcheu, Malawi | Concern Universal | 268,420 | $1.13 | Completed: December 2011-March 2012 |
| Balaka, Malawi | Concern Universal | 154,230 | $0.66 | Completed: October-December 2013 |
| Dedza, Malawi | Concern Universal | 290,770 | $1.11 | Completed: September-October 20147 |
| Kasaï Occidental, DRC | IMA World Health | 630,532 | $1.81 | Completed: August-November 2014 |
| Distribution in flood-affected districts, Malawi | Concern Universal and others8 | 80,000 | $0.30 | Completed: February-March 2015 |
| Dowa, Malawi | Concern Universal | 396,900 | $1.45 | Completed: March-May 2015 |
| Ntcheu, Malawi | Concern Universal | 335,000 | $1.20 | Completed: October 2015-March 20169 |
| Balaka, Malawi | Concern Universal | 235,000 | $0.86 | Completed: November-March 201610 |
| Total completed | 2,390,852 | $8.52 | ||
| Nord Ubangi, DRC | IMA World Health | 705,468 | $2.06 | Underway: October 2015-April 201611 |
| Total underway | 705,468 | $2.06 | ||
| Greater Accra Region, Ghana | Episcopal Relief & Development / government | 1,158,200 | $3.98 | Planned: June-July 2016 |
| Upper West Region, Ghana | Episcopal Relief & Development / government | 437,200 | $1.19 | Planned: June-July 2016 |
| Northern Region, Ghana | Episcopal Relief & Development / government | 1,093,700 | $2.14 | Planned: June-July 2016 |
| Western and Eastern Regions, Uganda | NMCP and other organizations | 10,743,120 | $29.66 | Planned: August 2016 - March 2017 |
| Dedza, Malawi | Concern Universal | 430,000 | $1.29 | Planned: July-August 2018 |
| Dowa, Malawi | Concern Universal | 435,000 | $1.61 | Planned: August-September 2018 |
| Ntcheu, Malawi | Concern Universal | 365,000 | $1.35 | Planned: September-October 2018 |
| Balaka, Malawi | Concern Universal | 250,000 | $0.93 | Planned: October-November 2018 |
| Total planned | 14,912,220 | $42.15 | ||
| Overall total | 18,008,540 | $52.73 |
The following table shows AMF's total expenditure, categorized into purchases of bed nets, spending on running the organization, and spending on other non-net costs (such as funding post-distribution check-ups, which are described below). We include spending since FY 2012 because this is when AMF shifted to its current model of larger-scale distributions.
| Category | FY 2012 to April 2016 spending | FY 2012 to April 2016 ratio |
|---|---|---|
| Net costs | $12.64 million | 82% |
| Other costs (non-net, non-organizational) |
$2.16 million | 14% |
| Organizational costs | $0.61 million | 4% |
| Total spending | $15.41 million | - |
AMF's role
AMF's role in LLIN distributions is to:13
- Identify countries with funding gaps for LLINs.
- Find distribution partners to carry out LLIN distributions and agree on expectations for the distribution, including who pays for costs other than the purchase price of LLINs and what information will be collected and shared with AMF.14
- Purchase LLINs and have them shipped to the distribution partner.
- Work with distribution partners to collect reports on the distribution and follow-up surveys. AMF posts these reports on its website.
Process for distributions
Selecting locations for distributions
When selecting locations for future distributions, AMF told us it consults a series of sources as it believes there is no one reliable resource with up-to-date information to determine where nets are needed. Sources it consults include the Alliance for Malaria Prevention's (AMP's) list of countries with significant net gaps, other malaria control funders, in-country technical advisors, the relevant National Malaria Control Program (NMCP), implementing organizations, and the African Leaders Malaria Alliance.15 AMF told us that it has been receiving more funding requests since it started funding larger distributions.16
As AMF investigates countries with existing net gaps, it also looks into organizations working within those countries that could serve as distribution partners.17 AMF's distribution partners must have the capacity and willingness to carry out a distribution that meets AMF's requirements (more below).18 Before deciding to work with Concern Universal for the Ntcheu 2012 distribution, AMF requested a distribution proposal, including information on malaria risk in the distribution area (including data on reported malaria cases), other net distributions in the area, coordination with government, and how the distribution would be carried out.19 We have seen similar distribution proposals, which include information on malaria risk and other sources of nets in the area, for distributions in Balaka in 2013, Dedza in 2014, and Kasaï Occidental in 2014.
AMF now requires that distribution partners submit a planning document instead of a distribution proposal. Planning documents are available for the Dowa 2015 distribution, the Balaka 2015 distribution, and the Ntcheu 2015 distribution. We have not seen planning documents or distribution proposals for the Nord Ubangi 2015 distribution (full list in Summary of AMF Distributions spreadsheet).20 We have not seen distribution proposals for the upcoming AMF distributions in Ghana or Uganda, though we have seen signed distribution agreements for these distributions.21
Requirements for distribution partners
AMF aims to work with distribution partners who will agree to the following requirements (see AMF Summary features of an AMF distribution for a brief overview of the requirements). We detail what evidence we have seen that past distributions have complied with these requirements in our Summary of AMF Distributions spreadsheet.
- Pre-distribution registration: AMF requires that its distribution partners carry out a pre-distribution registration survey (PDRS) in all areas that LLINs will be distributed to establish the number of sleeping spaces and LLINs in a condition such that they have at least one year of remaining use. (If such data already exist, AMF and the partner discuss whether the data is reliable and up-to-date, or whether a new PDRS should be conducted.)22 We have seen results from all of AMF's completed large-scale distributions in Malawi: full data from its Ntcheu 2012 distribution and sample results from its Balaka 2013, Dedza 2014, and Dowa 2015 distributions.23 We have also seen full data from AMF's distribution with IMA in Kasaï Occidental (this data was collected during the distribution rather than prior to it – more).24
- Distribution monitoring: An additional requirement is to provide AMF with photos and videos from the distribution and a report on the distribution, including problems encountered.25 AMF's distribution partner, Concern Universal, provided photos and/or videos for all but 2 of the 14 “sublocations” for its Ntcheu 2012 distribution.26 AMF has told us that it does not expect photos and/or videos from the Balaka 2013 or Dedza 2014 distributions because Concern Universal lost the photo files; however, we have heard conflicting messages about this documentation (see footnote).27 IMA World Health, AMF's distribution partner for the Kasaï Occidental 2014 distribution, sent AMF photos and/or video footage for 6 of the 8 "sublocations" of the distribution.28 AMF expects to receive photos and video from future distributions. AMF notes that it places little weight on the photos when assessing whether or not a distribution was conducted appropriately.29 Concern Universal has provided detailed distribution reports for three of the four distributions it has completed, as well as detailed weekly reports (which AMF does not require) for all four distributions.30 IMA has provided a detailed distribution report and a technology report for the Kasaï Occidental 2014 distribution.31
- Post-distribution check ups (PDCUs): AMF requires partners to conduct follow-up surveys in 5% of households at 6-month intervals for 2.5 years after a distribution.32 We summarize which PDCUs have been completed and whether they were completed on time in this spreadsheet (see "PDCUs" sheet). In short, four of the five expected PDCUs were completed for the Ntcheu 2012 distribution, three of the four expected PDCUs for the Balaka 2013 distribution have occurred (though it is unclear if the data from one of the three PDCUs is complete), and the two expected PDCUs from the Dedza 2014 distribution have occurred (though it is unclear if the data from one of the PDCUs is complete). PDCUs were also expected to take place for the Kasaï Occidental 2014 distribution in May 2015 (6-month) and November 2015 (12-month), as well as for the distribution in flood-affected districts in Malawi three months after the distribution, which also would have been May 2015 (AMF later told us this was delayed to late 2015); we have not seen results from these PDCUs.33
- Malaria case rate data: Previously, AMF expected to collect data on malaria case rates from the regions in which it funded LLIN distributions: in the distribution agreements for the Ntcheu 2011-2012, Balaka 2013, and Dedza 2014 distributions, AMF asked Concern Universal to provide monthly malaria case rate data from before and after the distributions from all health centers in the three districts.34
AMF has told us that Concern Universal has provided this data, but AMF has not yet published it.35
AMF told us that it has not made publication a high priority because it does not believe that the data is of high enough quality to reliably indicate trends in the malaria case rates.36
We have repeatedly asked AMF to share the unpublished data with us, but have not yet received it.
AMF has not included a requirement to collect and share malaria case rate data in its agreement with its distribution partner in DRC, IMA World Health.37 Malaria case rate data collection is not mentioned in the distribution agreements for upcoming distributions in Uganda and Ghana, though our understanding is that the details of these distributions are still being decided.38 We do not believe that the fact that AMF collects malaria case rate data is a consideration in AMF’s favor, and do not plan to continue to track AMF's progress in collecting malaria case rate data.39
- Non-net costs: Prior to the Balaka 2013 distribution, AMF asked all distribution partners to use their own funds or to find another funder for all non-net costs of the distribution.40
More recently, it has paid for some of these costs in certain distributions. AMF told us that it considers funding non-net costs in cases where (a) non-net costs are not covered by other partners, and (b) AMF feels confident that its distribution partners will manage and report on spending well.41
AMF has paid for all non-net costs for the Balaka 2013, Dedza 2014, and Dowa 2015 distributions and has agreed to pay for all non-net costs for the upcoming Malawi distributions in 2015, 2016, and 2018.42
It has also paid for the costs of the 24- and 33-month post-distribution check-ups for the Ntcheu 2012 distribution, and the shipping costs for the Nord Ubangi 2015 and flood-affected districts distributions.43
For its planned 2016 Ghana and Uganda distributions, AMF has agreed to pay for "non-standard" non-net costs, including PDCUs and, in Uganda, the cost of an "independent assessor [who will] determine whether nets are distributed to beneficiaries in the quantities listed." The Global Fund is paying for the "standard" non-net costs for the Ghana distributions and the government is paying for these costs in Uganda. "Standard" costs include shipping to the country, clearance charges, in-country transportation, pre-distribution planning and registration, and distribution costs.44
It is our understanding that AMF aims to primarily fund distributions where other partners cover non-net costs, which could limit its capacity to distribute nets, though AMF has shown flexibility on this issue.45
AMF looks for partners who, in addition to agreeing to the above requirements, are clear and responsive in their communication with AMF.
Other activities
AMF occasionally supports malaria control activities beyond the direct distribution of LLINs. For example:
- AMF plans to fund research on insecticide resistance that will be carried out in conjunction with AMF distributions. One study would be run by the Liverpool School of Tropical Medicine during the Nord Ubangi distribution in Equateur, DRC.46 AMF sent us a research proposal for the study which we have not yet reviewed.47 AMF may also fund research on the effectiveness of a new type of net in conjunction with its Uganda 2016 distribution.
- AMF is in the process of creating and co-funding a "Malaria Control Unit" (MCU) in Malawi with Concern Universal.48 The MCU will consist of 12-14 permanent staff members who will work on a variety of malaria control projects: improving malaria case rate data collection practices, monitoring the levels of malaria prevention and treatment supplies at local health centers, developing efficient methods to keep net coverage rates high in between mass distribution campaigns, and more.49 The MCU is also intended to assist with AMF and Concern Universal's distributions in Malawi.50 AMF has committed approximately $636,000 over three years to this project.51
- AMF has encouraged Concern Universal staff to attend national malaria control strategy meetings in Malawi to share AMF's processes and results in other districts. As a result, AMF has told us that some of its monitoring practices were being adopted for a large national distribution (approximately 7 million nets) in 18 of Malawi's 28 districts in March to May 2016.52 The practices being implemented include a) using sleeping spaces data to calculate the number of nets needed (as opposed to population data), b) spot-checking 5% of the registration data collected before the distribution, and c) putting summary data (at the village or health center level) into electronic form.53 AMF told us in February 2015 that it expects to be able to see the data collected during this distribution.54
Does it work?
On a separate page, we discuss the general evidence behind distributions of LLINs. We conclude that there is strong evidence that these distributions can be expected to reduce child mortality and malaria cases.
When evaluating the effectiveness of an LLIN distribution organization, we seek to answer the following questions:
- Are LLINs targeted at people who do not already have them? AMF typically requires partners to complete pre-distribution surveys to determine the number of nets required, if reliable and recent survey data is not available. We are not fully satisfied with the pre-distribution surveys' ability to ensure that people aren't under-reporting their existing LLIN ownership in order to get more LLINs, but believe that reasonable measures are being taken to address this issue.
- Do the LLINs reach the intended destinations? Distribution partners are asked to provide pictures and videos of LLINs being handed out (a requirement that has been complied with in two of five distributions where photos were expected), and report on how the distribution went, including problems encountered. Post-distribution surveys provide additional evidence that LLINs reached their destinations.
- Are LLINs targeted at areas with high rates of malaria? AMF seeks out distribution partners in countries that are known to have high rates of malaria.
- Do those who receive the LLINs install them in their homes properly? Do they utilize them consistently over the long term? AMF requires partners to conduct follow-up surveys at 6-month intervals, and for a period of 2.5 years. AMF has published documentation showing that its partner Concern Universal has generally complied with this requirement.55 We discuss some methodological limitations to these surveys below. We have little data to compare these results to, but they seem consistent with high, proper usage of nets for an extended period after distribution.56 AMF shared data from the first follow-up survey from DRC shortly before publication of this review; we have not analyzed it in depth but note that there appear to have been major weaknesses in the methodology and execution of the survey.
- Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF? The limited evidence we have suggests that donations to AMF increase the total number of LLINs distributed.
Details follow.
Are LLINs targeted at people who do not already have them?
We believe that the steps AMF and its partners are taking to minimize the ability of community members to request more nets than they need are reasonable.
AMF requires that distribution partners conduct household-level pre-distribution registration surveys (if recent, reliable household-level data is not available) to determine the number of existing LLINs with at least one year of remaining use and the number of nets required in each household.57 This process includes door-to-door surveys in the targeted district by government health workers.
Concern Universal told us that in Malawi (the location for which we have the most details on how the process has worked), the health workers enter each household, ask about who is living in the household and visually inspect sleeping spaces and existing nets. Supervisors revisit 5% of the households. We have requested but have not received data from the supervisors' visits. Concern Universal aims to include every household in the targeted areas in the registration process; we are unsure if in practice some villages or households that are located outside of villages may be missed and not receive nets. Concern Universal also conducts village meetings in which its staff read off household names and the number of LLINs that have been allocated to each household; households then indicate if any errors have been made. We observed one such village verification meeting during our visit to Malawi in October 2011.58 See footnote for more detail on the pre-distribution process used in Malawi.59
We have seen registration survey results from four of AMF's completed large-scale distributions in Malawi: full data from its Ntcheu 2012 distribution and sample results from its Balaka 2013, Dedza 2014, and Dowa 2015 distributions.60 We have not seen registration survey results from AMF's 2015 distributions in Ntcheu and Balaka.
We have not yet discussed details of the pre-registration process with AMF's other partners. For IMA's Kasaï Occidental distribution, we discuss what we know about the process from reports IMA has shared here. The process appears to be largely similar to Concern Universal's process for going house-to-house to determine net need, though it did not include supervisor re-visits or village verification meetings (in part because nets were distributed during the same visit in which net need was determined). Due to logistical issues encountered in Kasaï Occidental,61 IMA and AMF have planned to implement a pre-distribution registration survey for the distribution currently taking place in Nord Ubangi, DRC.62 We don't know whether there will be 5% spot checks or other verification methods used in Nord Ubangi.
We don't know if 5% spot checks are planned for AMF's upcoming distributions in Ghana.63 5% spot checks are planned for the upcoming Uganda distribution.64
We believe it is possible that some households may over-report the number of LLINs they need. Cases of households hiding nets in order to receive extra LLINs have been observed in campaigns in Senegal and Nigeria.65
Do LLINs reach the intended destination?
AMF requires that partners provide three types of documentation from during and after distributions:66
- Photos and videos for each "sublocation" of the distribution
- Distribution reports, detailing the activities completed during the distribution and problems encountered
- Post-distribution check ups every six months during the 2.5 years following the distribution, which include the number of AMF nets that were found in surveyed households
Photos and videos of distributions
Concern Universal provided photos and/or videos for all but 2 of the 14 “sublocations” in Ntcheu in which it distributed LLINs in 2012, although there are a fairly small number of photos for each sublocation (around 15 photos per sublocation).67 It has not yet provided photos and/or video for the other large distributions it has completed in Malawi (Balaka 2013, Dedza 2014, Dowa 2015, Balaka 2015, and Ntcheu 2015).68 IMA World Health has provided photos and/or video for 6 of the 8 "sublocations" of the Kasaï Occidental 2014 distribution, although there are only a fairly small number of photos for each sublocation (around 15 photos per sublocation).69
Because photos are missing for many recent distributions and there are fairly small numbers of photos available for other distributions, we do not find this alone to be convincing evidence that the nets reached their intended destinations.
AMF notes that it does not now consider photo and video footage part of its monitoring and evaluation activities that aim to assess whether a distribution has achieved its goals.
Distribution reports
During its Ntcheu 2012, Balaka 2013, Dedza 2014, and Dowa 2015 distributions, Concern Universal provided weekly reports on the number of LLINs distributed, the regions which were covered, and problems encountered in reaching particular villages or individuals.70 As we report in our March 2012 update, the Ntcheu 2012 weekly reports note challenges including attempted thefts, double registrations and logistical problems.71 Concern Universal provided a similar level of detail on challenges encountered in the Balaka 2013,72 and Dedza 2014 distributions,73 and the first half of the Dowa 2015 distribution.74 These reports increase our confidence that Concern Universal is aware of potential problems and has a system in place to address them.75
Concern Universal has provided weekly reports of its own accord (AMF does not require such reports). AMF does ask partners to provide summary reports on how distributions have gone after they are completed; we have seen such reports for the Ntcheu 2012, Balaka 2013, Dedza 2014, and Kasaï Occidental 2014 distributions, and we expect to see such reports from future distributions.76 The summary distribution reports vary in their level of detail, but all include an overview of the procedures followed during the distribution, a breakdown of the number of nets distributed into sublocations or villages, and details on challenges encountered.77 We believe that these detailed reports provide reasonable evidence that the distributions occurred and nets were sent to the intended locations.
Post-distribution check-up reports
Post-distribution surveys, discussed below, provide a further check on whether LLINs have reached their intended destinations. The surveys that have been completed to date have found relatively high coverage rates; the three completed 6-month surveys showed coverage rates of 85-90%, while a completed 8-month survey showed a coverage rate of 82% (see this summary of results). The surveys also find that most of the nets observed are from AMF.78 We believe this provides reasonable evidence that a distribution occurred and nets reached their intended locations, though we note that the surveys have some methodological limitations, including that they are conducted by the same organization that carried out the distribution, which may have an incentive to bias the results, and do not use random sampling (more detail below).
Other evidence
For the Kasaï Occidental 2014 distribution, AMF's distribution partner, IMA World Health, piloted the use of smartphones to record household data, including GPS coordinates, for registration and LLIN distribution.79 We have seen one image using the GPS data collected that appears to show a pin for each household where nets were distributed.80 AMF has also sent us detailed GPS data that shows the GPS coordinates for each household visited.81
Are LLINs targeted at areas with high rates of malaria?
At the highest level, AMF appears to exclusively target countries with known malaria risk.82 Since 2012, AMF’s large-scale distributions have occurred in the DRC or Malawi.83 The World Health Organization (WHO) reports that both the DRC and Malawi are in the top 23 countries contributing to the number of global malaria cases and deaths; based on 2013 data, the WHO estimated that both the DRC and Malawi had malaria death rates of between 50 and 99 deaths per 100,000 people.84 The WHO estimates that Ghana and Uganda, countries where AMF has agreed to fund large-scale distributions in 2016-2017, and two of the three additional countries AMF is considering funding distributions in as of May 2016, also have malaria death rates of between 50 to 99 deaths per 100,000 people.85
In its distribution proposal for the Ntcheu 2012 and Balaka 2013 distributions, Concern Universal reported the number of malaria cases in the proposed distribution area over a twelve month period preceding the distributions.86 We have not seen similar data for Concern Universal's Dedza 2014 or Dowa 2015 distributions, for the areas in the DRC where AMF has provided or will provide LLINs, or for Ghana or Uganda, countries where AMF has signed agreements to fund large-scale distributions.
Based on our understanding of how nets are allocated within countries, we would guess that looking at the malaria rates in the specific districts AMF works in is somewhat misleading and that AMF's funding should largely be considered to be adding to the total number of LLINs that are available for distribution in a country.
Do those who receive the LLINs install them in their homes properly? Do those who receive the LLINs utilize them consistently over the long term?
AMF requires partners to conduct follow-up surveys at 6-month intervals (plus or minus a month) for a period of 2.5 years, or until the next community-wide net distribution in the same area, to determine whether LLINs are present, are hung, and what condition they are in.87 Results are available from:
- 6-, 15-, 24-, and 33-month surveys following the Ntcheu 2012 distribution.
- 6-, 14-, and 19-month surveys following the Balaka 2013 distribution.88
- 8- and 14-month surveys following the Dedza 2014 distribution.89
- The 6-month survey from the Dowa 2015 distribution.90
Details of the methodology used in these surveys are available in our August 2014 update (see the description of the Ntcheu 15- and 24-month surveys) and August 2015 update (see the descriptions of the Ntcheu 33-, Balaka 14-, and Dedza 8-month surveys) on AMF. We discuss the methodology of the Balaka 6-month survey in this footnote.91 We also spoke with Concern Universal about the process used for PDCUs and summarize the process here.
We believe the methodology used has several limitations. The surveys were conducted by the same organization that conducted the distribution and which may have an incentive to bias the results. Villages are not selected randomly and it is not clear whether households within a village are selected randomly.92 We note that we have not seen technical details at the level of a high-quality academic study on how the surveys were carried out,93 and we therefore have not been able to fully vet the results.
AMF also shared data from the first follow-up survey for Kasaï Occidental shortly before publication of this review; we have not analyzed it in depth but note that there appear to have been major weaknesses in the methodology and execution of the survey that are likely to limit the usefulness of the results.
Results from follow-up surveys
We have summarized the results from all available PDCUs in this spreadsheet (data sources in footnote).94 As an example, the following table gives results from the Ntcheu 2012 distribution PDCUs.
| Ntcheu 2012: 6-month | Ntcheu 2012: 15-month | Ntcheu 2012: 24-month | Ntcheu 2012: 33-month | |
|---|---|---|---|---|
| # of households surveyed | 7,689 | 9,250 | 8,939 | 9,500 |
| # of AMF nets received by surveyed households | 15,814 | 18,580 | 18,448 | 20,688 |
| % of nets surveyed of total distributed | 6% | 7% | 7% | 8% |
| % of nets hung | 90% | 85% | 81% | 52% |
| % of nets missing | 1% | 3% | 4% | 3% |
| % nets in "very good" condition | 99% | 89% | 49% | 13% |
| % of nets in "good" condition | 1% | 6% | 28% | 23% |
| % of nets in "viable" condition | 0% | 2% | 15% | 19% |
| % of nets worn out | 0% | 3% | 8% | 45% |
| % of nets used correctly | 98% | 98% | 90% | 98% |
| % of people covered by nets | 90% | 84% | 79% | 56% |
Definitions:
- Nets hung: AMF told us that interviewers are asked to observe whether nets are hung by entering interviewees' houses (rather than simply asking interviewees if they are hung). Concern Universal told us that PDCU surveyors use their judgment to assess whether nets are being hung properly when they are interviewing recipients and inspecting their nets and sleeping spaces.95 It is our understanding that none of the nets included in the "nets hung" category are "worn out" (i.e., all are in "viable", "good", or "very good" condition).96
- "Very good" condition: LLIN has fewer than 2 holes of less than 2 cm in size.97
- "Good" condition: LLIN has fewer than 10 small holes.98
- "Viable" condition: LLIN has more than 10 small holes or has one large hole.99
- "Worn out": Survey reports from Concern Universal did not provide a definition of this term. AMF has told us in the past that an LLIN is considered worn out if it has multiple large holes and the LLIN is unrepairable, such that it would not provide protection against mosquitos.100
- Used correctly: According to a survey form from 2012, interviewers are asked to "ask the householder to demonstrate how the nets are used at night" and to then select yes or no to the question "are the nets being used correctly."101 In 2014, AMF told us that the procedure had not changed in more recent surveys.102 Note that some of the nets included in this category may not be AMF nets and that it is possible that some "used correctly" nets are also "worn out" nets.103 In this blog post, we discuss the possibility of bed nets being used for other purposes than covering sleeping spaces (e.g. being used as fishing nets). We have not seen evidence that AMF-distributed nets are being misused at a meaningful scale, and the PDCU results we have seen to date indicate that, at least in Malawi, a large majority of AMF-distributed nets are being used appropriately.
- People covered by nets: We are unsure of how to interpret this metric. We previously believed that this included all people who sleep under a properly hung and not worn out net. However, the data for some post-distribution check-ups is not consistent with this interpretation.104
The percentage of people covered by nets in AMF's post-distribution check-ups is roughly in line with the net usage rates of the trials of bed net efficacy documented in our page on Long Lasting Insecticide Treated Nets, in which small-scale studies of bed net efficacy had net usage rates generally in the 60%-80% range.105
The "decay model" we use to anticipate the lifespan of LLINs assumes that 92% of LLINs are functional and in use for the first year after a distribution, 80% of nets are functional and in use for the second year, and 50% of nets are functional and in use for the third year.106 AMF's post-distribution check ups for the Ntcheu 2012 distribution generally seem to match the decay model:107
- 6 months after the distribution, 90% of nets were hung and 1% were worn out or missing (the model predicts 92% of nets should be functional and in use).
- 15 and 24 months after the distribution, 84% and 79% of nets were hung and 6% and 12% were worn out or missing, respectively (the model predicts 80% of nets should be functional and in use).
- 33 months after the distribution, 52% of nets were hung and 48% were worn out or missing (the model predicts 50% should be functional and in use).
LLINs appear to have "decayed" more quickly in Balaka, Dedza, and Dowa.108
- In Balaka, 70% of nets were in "very good" condition at 6 months compared with 99% in "very good" condition at 6 months in Ntcheu, and 14% were worn out at 14 months compared with 3% in Ntcheu at 15 months.
- Dedza has a significantly lower percentage of nets in "very good" condition at 8 months than Balaka or Ntcheu had at 6 months (57% compared to 70% and 99%, respectively). At 14 months, Dedza had approximately the same percentage of "very good" nets as Balaka at 14 months, which is substantially lower than the percentage of "very good" nets in Ntcheu at 15 months (33% and 35%, compared with 89%, respectively).
- At 6 months 77% of nets in Dowa were in "very good" condition (compared to 99% in "very good" condition at 6 months in Ntcheu).
It is not clear to us how to compare the net quality and net usage rates found in AMF's post-distribution check ups to the assumptions in the decay model, in part because it is not clear whether the definition of a "functional and in use" net in the decay model depends on the type of metrics that AMF's data provides. We may do further analysis to more directly compare the model and AMF's data in the future.
Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF?
For the Ntcheu 2012 distribution in Malawi, AMF received data from the National Malaria Control Program on population figures and LLINs already distributed by district, as well as LLINs that had been committed to the country by other funders. AMF estimated, in September 2011, that an additional 909,586 nets were needed to achieve universal coverage in the country,109 of which AMF contributed 268,240.110 We have not done a similar analysis for later distributions.
On a separate page, we discuss some cases where AMF was in discussions to fund a distribution, but ultimately did not. In most of these cases, the net gap AMF was in discussions to fill persisted for six or more months after AMF's discussions closed, and, in two out of the five cases we looked at, gaps persisted for long periods (18 months and ~3 years, respectively). In most cases, the gap was eventually filled by another funder. As far as we can tell, during the time between AMF withdrawing from discussions and another funder stepping in, the populations targeted for distributions did not receive nets and likely were inadequately protected from malaria.
In addition, across Africa, there are substantial funding gaps for LLINs (more below) and because our impression from following AMF's progress over time is that, due to AMF's more limited funding and, perhaps, greater data requirements, governments often seek funding first from larger funders (particularly the Global Fund to fight AIDS, TB, and Malaria) and then may ask AMF to fill gaps. However, we note that countries are sometimes able to choose how they allocate Global Fund grants among malaria interventions (including LLINs, treatment, and diagnosis), so the availability of funding for LLINs from AMF could cause countries to allocate less Global Fund resources to LLINs.111
Possible negative or offsetting impact
- Do donated nets displace government health funding? This could be a concern if government funding that otherwise would have been spent on LLINs is spent on other, less worthwhile budget items. We have little sense of how important a concern this is in AMF's case. In the case of the 2012 distribution in Malawi, all anticipated nets were expected from international donors, and the total supply appeared to be still insufficient for country-wide universal coverage.112 Our rough understanding is that AMF is operating in countries with substantial net gaps that the country government would not have been able to fill with its own funding.113
- Do nets donated through AMF displace funding from other global health funders? It is possible that areas covered by AMF distributions would otherwise be covered by distributions supported by other malaria control funders, such as the Global Fund or the President's Malaria Initiative. AMF has told us that it is likely that areas it targets would eventually have been covered by other funders, though likely after a substantial delay. AMF has also argued that its distributions are more likely to achieve a higher level of coverage and better monitoring quality than other distributions.114 We do not have a good sense of how much displacement of other funder money is caused by AMF distributions, or of whether the displaced funds go towards bed net distributions in other places, or to other forms of malaria control (such as rapid diagnostic kits or antimalarial drugs). On a separate page, we discuss some cases where AMF was in discussions to fund a distribution, but ultimately did not. In most of these cases, the net gap AMF was in discussions to fill persisted for six or more months after AMF's discussions closed. The persistence of these gaps leads us to believe that historically, AMF has targeted gaps that would not have immediately been otherwise filled, though would have eventually been filled by other funders.
- Will insecticide-treated nets continue to be effective? As discussed in our report on insecticide-treated nets, there is strong evidence for the effectiveness of this intervention; however, the best evidence for the intervention was collected before LLINs were widely used and there is some evidence that mosquitoes have since adapted to the insecticide used in LLINs, possibly making them less effective. We have reviewed the evidence on the state of insecticide resistance. We concluded, "Broadly, it seems that insecticide resistance is a larger concern now than it was when we last thoroughly evaluated the evidence in 2012, but it remains difficult to quantify the impact of resistance. Our very rough best guess (methodology described in more detail below) is that ITNs are roughly one-third less effective in the areas where AMF is working than they would be in the absence of insecticide resistance. ITNs remain a highly cost-effective intervention after incorporating this discount."
- Do free LLIN distributions distort incentives for recipients or distort local markets for nets? As discussed in our report on insecticide-treated nets, we feel that there is a reasonably strong case for distributing LLINs freely rather than selling them at market (or even below-market) prices. We also think that the benefits of distributing LLINs freely to a population likely outweigh the negative consequences of distortion in local net markets, though we have not factored these potentially negative consequences into our cost-effectiveness analysis of AMF.
- Could distribution of LLINs be inequitable and unfair, causing problems in the targeted communities? We feel that AMF's process for determining needs for LLINs at the household level is fairly well-suited to ensuring that LLINs are distributed equitably. We have some concerns about whether AMF's process succeeds at identifying all villages or households located outside of villages.
- Does AMF divert skilled labor from other areas? In Malawi, net distributions have been conducted by low-level government health staff in partnership with the staff of AMF's partner NGO.115 AMF's partner in Malawi told us in 2012 that government health staff are normally involved in activities such as disseminating health-related information, reporting on levels of stunting and disease, carrying out immunization campaigns, and providing nutrition support.116 We do not know the extent to which net distribution reduces their ability to complete other duties, though we note that net distributions are generally completed within a few days in each local area.117 Diversion of skilled labor may be more of a concern in the DRC where 22 senior district health staff were employed as Field Supervisors for the Kasaï Occidental distribution.118 We have not seen the details of how AMF's upcoming distributions in Ghana and Uganda are being administered, so we do not know whether displacement of skilled labor will be a substantial concern for those distributions.
What do you get for your dollar?
Cost per LLIN distributed
Estimate based on data from AMF
Using budgets and estimates provided by AMF, we estimate that the total cost to purchase, distribute and follow up on the distribution of an AMF-funded LLIN is about $4.68 in Malawi and roughly $6.06 in DRC. The very roughly estimated average cost for future distributions that AMF is considering is $4.47 per net.119
Note that we prefer to include all costs incurred to carry out a project, not just those that the charity in question pays for itself. We believe that this gives the best view of what it costs to achieve a particular impact (such as saving a life) and also avoids the lack of clarity and complications of leverage in charity. Our estimate of the cost to distribute an AMF-funded LLIN aims to include both costs AMF pays and costs paid by others.
Full details in GiveWell estimate of AMF cost per net (May 2016).
The estimate for Malawi uses:
- The purchase price per LLIN, calculated from AMF's total LLIN budget and the number of LLINs distributed.
- Actual costs for most of the Balaka 2013 and Dedza 2014 distributions, as reported by Concern Universal, and budgeted costs for most of the PDCUs. It also includes budgeted costs for the Dowa 2015, Ntcheu 2015, and Balaka 2015 distributions.120 AMF told us that the non-net budgets include all monetary costs incurred by Concern Universal, as well as the use of vehicles and managerial staff time;121 it may exclude non-monetary costs such as the use of office space.
- An estimate of non-monetary costs incurred by Concern Universal and local governments, based on estimates provided in 2012 by Concern Universal for its distribution in Ntcheu province.122
- An estimate of AMF's organizational costs per net, including an estimate of the value of donated services and volunteer time. Because these costs generally do not vary with the number of LLINs distributed and remain roughly constant on a per year basis, the per LLIN estimate is highly dependent on the time period used and the number of LLINs distributed in that period. We estimate AMF's organizational costs as $0.50 per LLIN for fiscal years 2012-2016.123
For DRC, the estimate is rougher. We constructed the estimate similarly, although the information we have for DRC is likely of lower quality.124 The estimate for the DRC is based on:
- The purchase price per LLIN, calculated from AMF's total LLIN budget and the number of LLINs distributed.
- Actual costs for the Kasaï Occidental 2014 distribution and budgeted costs for the Nord Ubangi 2015 distribution.
- An estimate of post-distribution check up costs for the Nord Ubangi distribution. The estimate we have seen for post-distribution check up costs for Kasaï Occidental is not yet complete, so we have relied on the Nord Ubangi estimate.125
- We have used the same estimate of AMF organizational costs and non-monetary and government costs as we used for Malawi.
The higher cost per net in the DRC may be explained by a considerably more difficult operating environment in DRC,126 or our data for DRC may simply be of lower quality.
Our estimates of the cost per net for future distributions that AMF is considering are also rough; they are based on:
- The purchase price per LLIN, based on what AMF told us it expects the cost to be.
- AMF's estimate of the non-net costs associated with pre-distribution surveys, distribution days, and post-distribution check-ups for each upcoming distribution.127 These estimates are consistent with what we would expect based on costs in Malawi and DRC.128
- An estimate of AMF's organizational costs per net in the future, 129 including a very large (10x) increase in salaries in FY 2017. AMF is considering a large increase in number staff and has had only one salaried employee to date.130 We have very roughly guessed at what this might mean for AMF's salary costs (and have not tried to refine this estimate because of its very small effect on the cost per net). We estimate AMF's organizational costs costs will be about $0.12 per net in the future, compared with $0.50 in our Malawi and DRC estimates for FY 2012-2016. Our estimate is highly dependent on our assumption about the number of nets AMF will deliver in FY 2017.131
Global estimate
The Roll Back Malaria Partnership, which estimates funding gaps for LLINs across all African countries,132 uses an estimate of $2.70 to purchase a LLIN and $2.50 for all non-net costs of a distribution, for a total of $5.20 per LLIN.133 This assumption is used in cases where country-specific data is not available. Note that we have not seen details about where this estimate comes from, so we put little weight on it. The Co-Chair of the Roll Back Malaria Harmonization Working Group told us that non-net costs can vary significantly by country and that non-net costs in DRC may be more than $5 per LLIN.134 We have little information on how reliable the Roll Back Malaria Partnership's estimate is likely to be. Broadly, this estimate is consistent with the data we have seen from AMF. We have used estimates based on data from AMF in our cost per life saved calculations, because we have more information on their reliability. We do not believe our estimate of AMF's cost-per-net ought to be compared directly to the Roll Back Malaria Partnership's estimate, given how uncertain we are about the latter.
Cost per life saved
Using $4.47 as the average cost per net for countries that AMF is considering future distributions in, we estimate the cost per child life saved through an AMF-funded LLIN distribution at about $3,500.135
Note that our cost-effectiveness analyses are simple models that do not take into account a number of factors. For example, our model does not include other potential benefits of LLINs, such as non-fatal cases of malaria prevented, prevention of deaths in age groups other than under-5 year olds, or prevention of other mosquito-borne diseases. It also does not include possible offsetting impacts or other harms. We do include possible developmental impacts on children who sleep under an LLIN: we estimate that sleeping under an LLIN provides the same developmental impacts that a deworming pill provides. However, most of the benefit provided by LLINs is in the lives that LLINs save, not in their impact on development.
There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.
The full details of our cost-effectiveness analysis are at our report on mass distribution of LLINs.
Is there room for more funding?
We believe that AMF could use significantly more funds than it expects to receive.
In short:
- Estimated maximum: We estimate that AMF is fairly likely to be able to commit an additional $11.3 million to distributions in 2016-2017 (beyond the funds it has on hand) by December 2016. In addition to the $11.3 million, it may be able to use up to $18 million more for additional 2016-2017 distributions. It could use substantially more to commit to 2018-2019 distributions, but these funds may displace funding from other funders.
- Cash on hand: As of May 2016, AMF held $22.6 million that it could allocate to additional distributions.
- Other sources of funds: Historically, the vast majority of AMF's funding has been due to GiveWell's recommendation. More recently, AMF has been in conversations with a few potential major donors. We do not know what the size and timing of these potential gifts would be; our impression is that they are unlikely to significantly change AMF's funding gap.
- Past spending: In the first half of 2016, AMF signed agreements to fund large distributions in Ghana and Uganda taking place in 2016-17. These agreements have lessened our concerns about AMF's ability to productively use large amounts of money; its track record of signing major distributions is still fairly limited.
Details follow.
Available and expected funds
As of June 2016, AMF held approximately $61.4 million. Of this, $38.1 million was committed to three future distributions and two other projects,136 leaving $23.3 million in funds it could allocate to future distributions.137
Historically, the vast majority of AMF's funding has been due to GiveWell's recommendation.138 In 2015,139 AMF received approximately $5.2 million that it did not attribute to GiveWell.140 This is significantly more than we expected and we believe GiveWell may have influenced a large portion (perhaps 80%) of these donations. As a rough guess, we expect AMF to receive a similar amount from non-GiveWell sources in 2016.141
Expected maximum
The African Leaders Malaria Alliance (ALMA) estimates funding gaps for LLINs across all African countries. In 2014, 2015, March 2016, and June 2016 we asked Melanie Renshaw of ALMA for updates on the size of the funding gap. On all occasions she told us that there remained significant gaps in funding for LLINs.142
According to a gap analysis Dr. Renshaw shared with us in June 2016, approximately 41 million additional LLINs are still required in Sub-Saharan Africa for 2016 (down from 66 million in April 2016), and approximately 69 million additional LLINs will be needed in 2017 (down from 99 million in April 2016).143 75% of the gap for 2016-2017 is for nets for Nigeria.
We estimate that global funding gap that is relevant to AMF is roughly 11.6 million nets for 2016-2017, or about $52.4 million at $4.47 per net ($5.47 for PBO nets144 ; see above for cost per net calculations). This is the gap that we believe there is some chance AMF could address before December 2016 (when we plan to next update this estimate) if it had the funding to do so. This estimate is based on the following assumptions:
- Exclude Nigeria and Republic of the Congo. Our understanding from several sources is that the situation in Nigeria is complex and funding is not the primary constraint to conducting a high-quality distribution. Similarly, the Global Fund declined to fund nets in the Republic of the Congo because of poor performance on a previous malaria grant.145
- Carry over 90% of the gap from 2016. While most of the 2016 gap will carry over into 2017, the portion that would have been distributed through routine distribution channels (such as antenatal care) will not carry over.146
- For countries in AMF's pipeline, use the total that AMF is considering allocating to the distribution divided by $4.47 per net (or $5.47 per net for PBO nets, which cost $1 more) to estimate the AMF-relevant gap for the country.147
- Exclude Malawi. AMF has done extensive work in Malawi and is planning to fund later distributions there in 2018, so we assume that if it were the case that Malawi had a gap for 2016-2017, AMF would attempt to fill it. In other words, we'd guess that lack of funding is not the constraint to AMF filling a gap in Malawi.
- Exclude gaps for less than 1 million nets. AMF told us that it prefers not to initiate new partnerships in countries that require a small number of nets.148
The gaps for 2018 are much larger (approximately 200 million nets in total),149 but the estimates do not yet account for funding from the largest funders, which will be allocated later.150
GiveWell's prioritization of AMF's funding gaps
We have broken down our top charities' funding gaps and ranked them based on:
- Capacity relevance: how important the funding is for the charity's development and future success.
- Execution relevance: how likely it is that the charity's activities will be constrained if it does not receive the funding.
We believe that "capacity-relevant" gaps are the most important to fill, and "execution"-related gaps vary in importance. More explanation of this model is in this blog post.
We don't consider any of AMF's current funding gaps to be capacity-relevant.
We consider the funding gaps to be "execution" gaps and assign them a level (1, 2 or 3) based on how likely we believe it is that AMF would be constrained by funding (rather than other factors, such as inability to find additional distribution partners) if it is unable to fill the funding gap. Level 1 is a 50% chance of funding being the constraint, level 2 is a 20% chance, and level 3 is a 5% chance. We have estimated these gaps through December 2016, when we next plan to update the estimates. These judgments are rough and largely based on intuitions formed from following AMF over several years.
| Size of gap (millions USD) | Explanation | |
|---|---|---|
| Execution Level 1 | 11.3 | AMF is in discussions with three countries for 2017 distributions that it does not have sufficient funding for.151 This funding would allow AMF to fully fund these distributions. |
| Execution Level 2 | 7.3 | Funding for the next largest country gap |
| Execution Level 3 | 10.5 | Funding for the remaining two large 2016-2017 country gaps |
We also note that AMF has been constrained by limited staff capacity in 2016.152 If AMF remains capacity constrained going forward, capacity, not funding, may be the limiting factor on AMF's ability to fund more distributions. AMF has told us that it plans to hire more staff, possibility going from two to six central staff by the end of 2016 (which is likely to reduce AMF's staff capacity in the short-term as it focuses on hiring and training).153
AMF has told us that it believes committing to distributions over a year in advance results in higher-quality distributions.154 We do not weigh this consideration as highly as AMF does.
Pipeline of potential distributions
We rely on the total AMF-relevant LLIN gap for 2016-2017 distributions, as discussed above, to estimate AMF's room for more funding, given the uncertainty in AMF's pipeline and our uncertainty about the value of committing funds to distributions for 2018-2019 at this time. Here we summarize AMF's current pipeline for context and comparison.
AMF has told us that, as of June 2016, it is in discussions about potential distributions in seven countries, several of which it has not completed a distribution in before.155
AMF gave us two scenarios for its funding pipeline – one in which AMF would co-fund distributions with a funding partner (thus paying only for nets and "non-standard" non-net costs, particularly post-distribution check-up surveys) and one in which AMF would pay for all costs associated with the distributions it funds. We believe the co-funding scenario to be more likely.156
To fund all of these distributions, assuming that AMF pays only for nets and non-standard non-net costs, AMF estimates that it would need approximately $178 million total (in addition to current cash on hand), of which $163.7 million would be for distributions taking place in 2018-2019. Assuming that AMF pays for all distribution costs, AMF estimates that it would need approximately $300 million total (in addition to current cash on hand).157
AMF's pipeline has been changing quickly as its discussions have progressed. For example, between March and June 2016, AMF added two countries, removed a country, and tripled the number of nets it is considering funding in one country.158
We believe it is unlikely that all of the above discussions would be successful even if AMF had sufficient funding to move forward with all of them, and we note that, in the past, negotiations have often taken significantly longer than AMF has estimated. However, it is our impression that discussions around the Uganda 2016-17 distribution of 10.7 million nets took only eight weeks from initial contact to an agreement being signed.159
Rate of funds moved
In November 2013, we wrote in this blog post that we believed AMF did not have room for more funding because it had raised over $10.6 million due to GiveWell's recommendation since 2011 and had not spent the bulk of these funds. In 2014, AMF completed and signed agreements for several large distributions, increasing our confidence that it could productively use the donated funds it held.160 In 2015, AMF did not sign any new distribution agreements for the near future, although it signed agreements for four large distributions in Malawi in 2018.161
In January 2016, AMF signed an agreement to fund a distribution of 2.7 million nets in Ghana in June to July 2016.162 In April 2016, AMF signed an agreement to fund a distribution of 10.7 million nets in Uganda in August 2016 to March 2017.163 This is the largest distribution AMF has signed an agreement for to date.
There is still uncertainty as to AMF's ability to find new partners and countries to work with. To find a sufficiently large distribution often requires negotiating with the national malaria control programs of countries in sub-Saharan Africa, which we perceive to have some discretion in which funders they work with, and which we perceive to be choosing funders based on a variety of factors including size and reporting requirements.164 In the past, AMF has been able to fund only a relatively small piece of a given country's distribution, but has maintained substantial reporting requirements. This dynamic may create fundamental reasons for governments to prefer partnerships with other funders. We believe that this is the most important and fundamental explanation for AMF's difficulties finalizing distributions in the past.
We believe that other factors may have also played a role in AMF's past difficulties finalizing distributions. We discuss this topic in detail in this blog post. AMF believes that the factors that have played the largest role in delaying recent distributions include a) the reluctance of a funder AMF has been working closely with to pay for post-distribution check ups and b) the fact that AMF has not previously partnered with the countries it was in discussions with.165 It is our understanding that there have been two cases in which AMF could have likely secured a deal more quickly if it had agreed to fund post-distribution check ups (PDCUs); we are not sure if AMF's decision to push for a different funder to take on those costs was optimal.166 AMF has agreed to fund PDCU costs for the distributions it signed in early 2016.167
Effect of additional funds
AMF has told us that the number of opportunities it considers at one time is limited by the funds it holds and that the primary effect of having additional funds is to increase the size of the distributions it considers, as well as increasing the number of distributions it considers.168 The type of opportunities it will consider depends on how much funding it has available.169
In 2014, AMF told us that it had had several conversations about potential distributions that did not move forward because AMF was unable to consider a distribution of sufficient size.170 We do not have more details on these conversations, particularly on whether other issues contributed to the discussions not going forward. We do not know if proposed distribution size contributed at all to delays in signing distributions in 2015; the discussions in 2015 were for larger distributions than AMF previously funded.
What portion of all net distributions are funded by AMF?
In 2015, approximately 177.8 million nets were distributed in Sub-Saharan Africa. 61% of these nets were funded by the Global Fund, 23% of these nets were funded by the President's Malaria Initiative, and 5% were funded by UNICEF.171
In 2015, AMF funded approximately 1% of the nets distributed in Sub-Saharan Africa (note that this is less than 1% of the funding required for these nets, because AMF does not always pay for non-net costs).172 If AMF were to spend all of its unallocated cash on hand on 2016 net distributions (which we consider unlikely), and other funders provided the same number of nets in 2016 as they did in 2015, AMF would fund approximately 11% of all nets distributed in Sub-Saharan Africa in 2016.173
AMF as an organization
We believe the Against Malaria Foundation to be an exceptionally effective organization:
- Track record: We feel AMF has built up a strong track record of raising money, finding smaller distribution partners, and getting partners to report information publicly at an unusual level. More recently, AMF has begun to demonstrate its ability to implement its model at significantly larger scale.
- Communication: AMF has always communicated extremely clearly and directly with us and given thoughtful answers to our critical questions.
- Self-evaluation: AMF has invested heavily in self-evaluation, going above and beyond what’s usual for data collection on bed net distributions: it has demonstrated a commitment to collecting long-term net usage data.
- Transparency: AMF has generally appeared to value transparency as much as any organization we’ve encountered. It hasn’t just shared information with us; it publishes significant amounts of useful information publicly on its own, far more than the norm. Recently, we feel that AMF has not been as transparent as it could be. In early 2016, we requested several documents from AMF that AMF told us it had available. AMF has repeatedly delayed sending us these documents, explaining that it did not have the capacity to process these documents for sending. As of this writing (in May 2016), we have not received the requested documents from AMF.
Response from AMF
In this update review GiveWell has said AMF has recently been less transparent in some areas. They are right and we accept this criticism. It is not, however, a change in our policy and we remain committed to being a highly transparent organisation. We have not sent GiveWell some net monitoring information on DRC in a timely fashion due to it taking longer for us to receive the required data, it requiring more validating than we expected and this coinciding with a busy time at AMF in which we were finalising several agreements. All of the information critiqued here will be made public, as normal, and we are in the process of completing that now. This is an explanation not an excuse and GiveWell’s criticism is fair and we will seek to avoid this happening again as timely delivery of information is as important as its delivery.
However, we see some potential room for improvement:
- AMF has not been as aggressive and ambitious about growth and capacity as it could have been. It operated for most of its 11 years with only two full-time staff members; it does not appear to us to have built as robust a fundraising operation as it could. More recently, AMF has taken modest steps to increase its capacity. Since 2014 AMF's executive chairman has dedicated half his time to AMF and in February 2016, AMF hired a full-time program director.174 AMF has informed us that it is considering hiring more staff in 2016.175
More on how we think about evaluating organizations at our 2012 blog post.
Sources
- 1
"How will the money raised be spent? On bed nets (mosquito nets). Specifically, long-lasting insecticidal nets (LLINs)." AMF Frequently Asked Questions.
- 2
See our Summary of AMF Distributions spreadsheet. Adding up the "# LLINs" column for the Ntcheu District 2012, Balaka District 2013, Dedza District 2014, Kasaï Occidental 2014, Dowa District 2015, Balaka District 2015, and Ntcheu District 2015 distributions yields 2,390,852 nets. Spreadsheet sourced from AMF Distributions.
- 3
See our Summary of AMF Distributions spreadsheet. The Nord Ubangi 2015 distribution is 705,468 nets. Spreadsheet sourced from AMF Distributions.
- 4
See our Summary of AMF Distributions spreadsheet. Spreadsheet sourced from AMF Distributions
- Adding up the "# LLINs" column for Northern Region 2016, Upper West Region 2016, and Greater Accra Region 2016 results in 2,689,100 nets.
- Adding up the "# LLINs" column for Balaka 2018, Ntcheu 2018, Dowa 2018 and Dedza 2018 results in 1,480,000 nets.
- 5
- 6Excludes non-net costs that AMF pays for for some future distributions (in cases where we do not have estimates for these costs). More details in this spreadsheet.
- 7A mop-up distribution for an additional 45,000 nets that were still needed after the initial distribution was completed in May 2015. More here.
- 8AMF worked with Concern Universal, Médecins Sans Frontières (MSF) Belgium, and GOAL Global, and others. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 9In February 2016, AMF told us that the Ntcheu distribution had been delayed Rob Mather and Peter Sherratt, conversation with GiveWell, February 11, 2016. As of April 2016, the distribution was recorded as "completed" on AMF's website. AMF Distributions
- 10In February 2016, AMF told us that it was awaiting a report from "its recently completed distribution in Balaka, Malawi" Rob Mather and Peter Sherratt, conversation with GiveWell, February 11, 2016. As of April 2016, the distribution was recorded as "completed" on AMF's website. AMF Distributions
- 11As of April 2016, the distribution was still denoted as "Being distributed" on AMF's website. AMF Distributions
- 12
For underlying calculations, see this spreadsheet. Sourced from AMF Financial information.
- 13
This understanding is based on many conversations with AMF, and from following AMF's progress over time.
- 14
Note that the process of agreeing on expectations for the distribution often involves additional actors. For example, sometimes the distribution partner is not paying for non-net costs, in which case the non-net costs funder might also be part of the discussions about expectations. Additionally, AMF works closely with countries' National Malaria Control Programmes (NMCPs); NMCPs must approve the net distributions and often are willing to use government resources (e.g., community health workers) to assist with a net distribution.
- 15
- "This is a list of the countries with known gaps and where there are significant contiguous areas without nets, or a significant percentage required, and for which the estimate of need is believed to be reasonably accurate. It does not include countries where there are gaps, typically up to 40% of what the nation needs, but they are spread more uniformly across the country and would therefore require an ‘in-fill campaign’. An in-fill campaign is different from a so-called ‘universal coverage campaign’ because the percentage installed base of nets is higher in the former case and so a pre-distribution registration survey (PDRS) is an absolute requirement to ensure an efficient allocation of nets. Our methodology would lend itself to these campaigns if the relevant National Malaria Control Programme (NMCP) were to embrace a detailed PDRS. The list does not include, in our view, other countries where the need has not yet been quantified. Given there are many countries with needs estimated, we have not chosen to seek out other countries in need of nets. Our assumption is groups like AMP will be a source of reporting on additional countries as quantified needs emerge." Rob Mather, AMF Founder, email to GiveWell, August 8, 2012.
- In September 2015, we checked in with AMF about its process for determining in which countries it works. Rob Mather noted that AMF continues to review malaria prevalence data (where that data exists), although maintains a level of skepticism about that data given that it can be unreliable. Even so, AMF feels comfortable drawing conclusions about which countries have high malaria mortality burdens based on the data it sees and the conversations that it engages in. AMF checks the following sources to keep up-to-date on which countries have a significant malaria burden:
- The Alliance for Malaria Prevention, which sends out a weekly email with malaria-related data and information.
- The African Leaders Malaria Alliance (ALMA), which AMF has conversations with on a quarterly basis.
- In-country partners, who frequently attend malaria task force meetings and have recent news
- Members of malaria advisory groups
- Other connections
Before AMF decides to approach a country to offer funding for nets, it has many conversations to confirm the level of need that country has with the other actors that are working on malaria in the country. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- In early 2016, AMF described a location-selection process similar to that described previously: "AMF learns about net gaps and receives funding requests through its network in the malaria control community, particularly through the Alliance for Malaria Prevention and the African Leaders Malaria Alliance." Rob Mather and Peter Sherratt, conversation with GiveWell, February 28, 2016
- 16
"As it becomes involved in larger distributions, AMF is receiving a growing number of funding requests. As its funding increases, AMF aims to make more strategic investments by engaging in the planning cycles of countries where it has strong connections and experience." Rob Mather and Peter Sherratt, conversation with GiveWell, February 28, 2016
- 17
Rob Mather, AMF Founder, conversation with GiveWell, July 19, 2012
- 18
Rob Mather, AMF Founder, email to GiveWell, August 8, 2012
- 19
@AMF LLIN Distribution Proposal Form@. For the Malawi distribution proposal, see Concern Universal Ntcheu 2012 distribution proposal
- 20
"[GiveWell]: I believe you said on our September call that AMF now requires a planning document to be created with each distribution partner before a distribution. Is this correct? It’s possible I misinterpreted.
[AMF]: You are correct. We now require a detailed planning document. See recent Ntcheu and Balaka distributions. This has replaced the previously requested Distribution Proposal which we see as less useful now given: a) we now put in place legal agreement documents, b) each distribution page on the website covers the main points of the distribution, and c) what we seek (generically) in a distribution is covered elsewhere on the website."
Rob Mather, AMF Founder, email to GiveWell, November 9, 2015
- 21
- 22
Rob Mather, AMF Founder, conversation with GiveWell, July 19, 2012.
- 23
- 24
IMA World Health, Kasaï Occidental 2014 distribution data
- 25
- 26
- 27
Over the last few years, we have heard conflicting messages from AMF about this documentation:
- AMF previously told us that Concern Universal had forgotten to take photos for the Balaka 2013 and Dedza 2014 distributions.
- "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
[AMF:] No, CU did not take photos of this one. They are aware of this slip and there will be photos and video footage from Dedza. Photos and videos are also included in the agreement for the DRC-Kasaï distribution." Rob Mather, AMF Founder, email to GiveWell, May 22, 2014. - Originally, AMF did not think that photos were taken of the Dedza 2014 distribution. Rob Mather, AMF Founder, email to GiveWell, June 19, 2015
- "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
- In September 2015, AMF informed us that photos were taken. We are not sure what the cause of this confusion was. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- In November 2015, AMF told us that photos were taken, but the files were lost: "For Balaka and Dedza we have [no] photos. CU have told us they lost those files. Procedures are in place to ensure this does not happen again." Rob Mather, AMF Founder, email to GiveWell, November 9, 2015
- AMF previously told us that Concern Universal had forgotten to take photos for the Balaka 2013 and Dedza 2014 distributions.
- 28
- 29
- AMF told us that future distributions would have the same requirements for photos and video as have past distributions. Rob Mather and Peter Sherratt, conversation with GiveWell, November 6, 2014.
- Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- AMF noted that photos were more important when AMF funded smaller distributions - for example, AMF would require a photo of each village sign where nets were distributed as an extra check that nets arrived at their intended destinations. Rob Mather, AMF Founder, conversation with GiveWell, November 10, 2015
- 30
Concern Universal uses the following process for its distributions:- After the PDRS, Concern Universal creates a distribution plan, which identifies net distribution points, assigns villages to these points, and, based off the PDRS data, allocates a number of nets to be delivered to each point.
- Concern Universal staff then meet with traditional authorities and community leaders to explain the distribution plan and set a date for the distribution day. Community leaders give input so that the distribution day does not conflict with other events.
- The distribution day date is communicated to village members via word of mouth.
- On distribution day, bed nets are brought to the distribution points. Village members walk (or sometimes organize transit in a motor vehicle) to the distribution point. Each distribution point services 2-5 villages, and people may walk up to 5 miles to the distribution site.
- A roll call is taken, and as household names are called, their members approach to receive nets. Recipients sign or fingerprint in distribution books (printed in duplicate) to confirm receipt of their nets.
- Once the distribution is complete, each village leader receives a copy of their distribution book, and Concern Universal retains the other copy.
- Usually, some villages will not show up at the distribution point on distribution day, most likely due to a funeral that conflicted with the distribution day date. There are also cases of individual households not showing up.
- After the main phase of the distribution has been carried out, Concern Universal staff assess which villages were registered but did not show up. It then organizes a mop-up distribution to reach these communities. The mop-up distribution follows a similar procedure to the main-phase distribution; it can take place at the same distribution sites or at different sites.
Nelson Coelho, conversation with GiveWell, April 15, 2016
- 31
- IMA World Health, Kasaï Occidental 2014 distribution report
- IMA also produced a technology report on the use of smartphones in the distribution. IMA World Health, Kasaï Occidental 2014 technology report
- 32
- "Post-distribution Check-Ups (PDCUs) are carried out every 6 months for 2.5 years (so 5 checks) with 5% of households in each Health Area selected at random and visited unannounced. These data are put in electronic form." AMF Summary features of an AMF distribution
- PDCUs every six months for 2.5 years are included in the distribution agreements AMF signed for its upcoming distributions in Ghana and Uganda. Ghana: AMF Ghana 2016 distribution agreement; Uganda: AMF Uganda 2016 distribution agreement
- AMF used to require that distribution partners conduct follow-ups every 6 months for up to 3 or 4 years.
- "[AMF:] Post-Distribution Surveys (see Note 3). Please confirm you will carry out Post-Distribution Surveys (PDSs) every 6 months post-distribution for a period of up to four years to assess the level of net usage (hang-up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households.
[Concern Universal:] I can confirm Concern Universal and Dedza District Council will conduct a Post-Distribution Survey every six months following the distribution." Concern Universal Balaka 2013 distribution proposal, pg. 2 - Concern Universal Dedza 2014 distribution proposal, pg. 2 contains the same language.
- IMA World Health agreed to conduct surveys every six months for 3 year post-distribution. IMA World Health and AMF distribution agreement for Kasaï Occidental 2014, pgs. 3-4 and IMA World Health and AMF distribution agreement for Nord Ubangi 2015, pgs. 3-4
- "[AMF:] Post-Distribution Surveys (see Note 3). Please confirm you will carry out Post-Distribution Surveys (PDSs) every 6 months post-distribution for a period of up to four years to assess the level of net usage (hang-up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households.
- Concern Universal uses the following procedure for its PDCUs:
- From a list of all villages that received nets in a distribution, Concern Universal's project manager arbitrarily picks 5% of the villages to carry out PDCUs in. For each PDCU, the project manager tries to not select villages that are geographically close to each other or villages that have been visited in previous PDCUs. (Note that this selection procedure is not random and may be subject to bias.)
- For each selected village, the project manager arbitrarily selects 25 households to be surveyed during the PDCU. (Note that this selection procedure does not appear to be random and may be subject to bias.)
- Concern Universal coordinates with government health officials to ensure that the PDCU does not conflict with other scheduled activities.
- On the dates of the PDCU, a supervisor or HSA from the local health facility accompanies the enumerator (a contracted employee of Concern Universal) to the village being surveyed. The health facility supervisor (or HSA) and enumerator meet with the village leader to discuss the purpose of their visit. The health facility staffer and enumerator arrive at the village unannounced to avoid knowledge of the PDCU influencing beneficiary behavior.
- The village leader (or a representative of the village leader) then accompanies the enumerator and health facility staffer to each household selected for the PDCU. The village leader knows the location of each selected household, and introduces the household head to the PDCU team.
- The household head gives consent for the PDCU team to enter the household and perform its check. If a household head refuses, the enumerator informs the project manager and the project manager selects another household. If the household head is away when the PDCU team arrives, the enumerator will return later in the day to perform the check. Household heads are frequently away when the PDCU team arrives, often because they are doing fieldwork far from their home.
- During the PDCU check, the enumerator:
- assesses the number of regularly used sleeping spaces.
- asks how many people are in the household, and how many have had blood-diagnosed malaria in the past month.
- checks the brand of the bed nets. Nets are recorded either as 'Permanet,' 'Olyset' (the two brands AMF distributes), or 'other.'
- records whether the net was distributed by an AMF-funded distribution. AMF-funded nets have a tag with the AMF logo and the date of the distribution.
- assesses the net condition. Nets are assessed as 'very good,' 'okay,' or 'poor.'
- A net in very good condition has two or fewer holes of less than 2 cm.
- A net in okay condition has fewer than 10 small holes (less than 2 cm).
- A net in poor condition has more than 10 small holes, or one large hole (greater than 10 cm).
Net quality is assessed by physical examination by the enumerator. If a net is not hung, the net is unfolded and physically assessed.
- asks who sleeps under each net, categorized by children under 5, children ages 6-18, pregnant women, or other people.
- asks how many nets were originally distributed to the household during the AMF distribution, and also notes how many of these nets are not present, or worn out.
- records how many nets in the household are hung, and how many are not hung.
- assesses whether nets are being used correctly. For this assessment, the enumerator makes a judgment call about whether the nets being are used properly. Often, nets will not be hung during the day due to limited space, but still may be used properly at night.
Nelson Coelho, conversation with GiveWell, April 15, 2016
- 33
- AMF told us in February 2015 that the 6-month PDCU for the Kasaï Occidental 2014 distribution was scheduled for May 2015. Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015. Note that since the distribution ended in October 2014, the 6-month mark (at which PDCUs are intended to occur) would have been April 2015.
- Generally, PDCUs are expected to occur six months after a distribution. AMF told us that the first Kasaï Occidental 2014 PDCU was completed in June and July 2015, 8 months after the initial distribution. AMF also told us that it has received the PDCU data in mid-August 2015. Rob Mather, AMF Founder, email to GiveWell, September 9, 2015. As of April 2016, we had not seen this data.
- AMF told us that the 3-month PDCU for the flood-affected districts in Malawi had been rescheduled to be a 6-month PDCU and that it would occur in late 2015. Rob Mather, AMF Founder, email to GiveWell, October 12, 2015. As of April 2016, no data from this PDCU has been published, and we do not know if it has been carried out.
- 34
- We discuss case rate data from Ntcheu in our February 2013 update on AMF.
- "[AMF:] Please confirm you are able to provide monthly malaria case rate data going back at least 12 months for each health centre/clinic in the distribution area and will continue to provide monthly data for a period of four years post‐distribution. This ensures we understand pre‐distribution malaria levels and can monitor them post‐distribution.
[Concern Universal:] Malaria case rate information for each of the 16 Health Centres is available from the Balaka District Health Office using the Malawi Government’s national Health Management Information System (HMIS). This information will be sent to AMF separately." Concern Universal Balaka 2013 distribution proposal, pg. 2 - Concern Universal Dedza 2014 distribution proposal, pg. 2, contains the same language as above
- 35
- Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015
- AMF has asked for malaria case rate data from Concern Universal for 12 months preceding and 4 years following the distributions in Malawi. We discuss case rate data from Ntcheu in our February 2013 update on AMF.
- "[AMF:] Please confirm you are able to provide monthly malaria case rate data going back at least 12 months for each health centre/clinic in the distribution area and will continue to provide monthly data for a period of four years post‐distribution. This ensures we understand pre‐distribution malaria levels and can monitor them post‐distribution.
- [Concern Universal:] Malaria case rate information for each of the 16 Health Centres is available from the Balaka District Health Office using the Malawi Government’s national Health Management Information System (HMIS). This information will be sent to AMF separately."
Concern Universal Balaka 2013 distribution proposal, pg. 2. Concern Universal Dedza 2014 distribution proposal, pg. 2 contains the same language.
- AMF shared data from Ntcheu through through November 2012. AMF page on Ntcheu 2012 distribution. We do not believe this data is high quality. In August 2013, it told us that it planned to publish an audit of the data quality by the end of September 2013: “AMF has requested additional data from local health centers and is in the process of
reviewing it. It is planning to publish the results of this audit by the end of September.
At this point, it still believes that the corrected data will show a substantial (~50%+)
reduction in malaria cases in Ntcheu after its distribution there but is still assessing the
level of accuracy and therefore reliability of the malaria case rate data from Ntcheu.” Rob Mather, AMF Founder, conversation with GiveWell, August 15, 2013. - Later, AMF told us that it expected to post data in early 2014. See our August 2014 update on AMF
- In November 2014, AMF told us that it expected to share more recent data by the end of January 2015. AMF told us it has been waiting to collect data through to September 2014 to be able to review data for a full year and compare it with the prior year due to changes in the way data is recorded in the health centers. Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.
- As of April 2016, we have not seen case rate data from these distributions.
- 36
Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
- 37
- Discussions with IMA made it clear that the quality of the case rate data from the majority of health centers was considered unreliable. IMA told AMF that it believes there will be some health centers across the distribution area that can provide historic and future malaria case rate data that is reliable, and that it might be able to provide this data to AMF in the future.
- Rob Mather, AMF Founder, email to GiveWell, November 26, 2014
- 38
- 39
We've tracked the progress of AMF's collection and publication of malaria case rate data for several years. AMF told us several times that it was close to publishing data. Most recently, in June 2015, AMF told us that it currently has malaria case rate data, but has not prioritized preparing the data for publication. AMF also told us that the data is likely to be poor quality, and that it will put little to no weight on the results. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
- 40
Rob Mather, AMF Founder, conversation with GiveWell, July 19, 2012.
- 41
Rob Mather, AMF Founder, email to GiveWell, November 26, 2014.
- 42
- 43
- 44
- Rob Mather, AMF Founder, email to GiveWell, January 28, 2016
- "The Ghana distribution is now going ahead: AMF is funding the costs of the nets and AMF’s additional monitoring costs, and the Global Fund will fund other non-net costs." Rob Mather and Peter Sherratt, conversation with GiveWell, February 11, 2016
- "The non-net costs will be funded by the Ghana Malaria Global Fund Grant. These costs include those for shipping to Ghana, clearance, in-country transport, pre-distribution, distribution." AMF Ghana 2016 distribution agreement..
- "The non-net costs will be funded by the Uganda Ministry of Health which may use funding from its Roll Back Malaria Partners. These costs include those for shipping to Uganda, clearance, in-country transport, pre-distribution, and distribution." AMF Uganda 2016 distribution agreement, pg 2.
- Discussion of independent assessor is from AMF Uganda 2016 distribution agreement, pg 4.
- "Post-Distribution Check-Ups… AMF will fund an NGO to run the process." AMF Uganda 2016 distribution agreement, pg 5.
- 45
- "AMF funding non-net costs can remove the final barrier to a distribution proceeding and avoid potential lengthy delays, of up to a year, given the ideal timing of a distribution relative to other events, for example a rainy season." AMF page on non-net costs
- AMF has told us that the administrative burden of managing the financials for certain non-net costs (such as post-distribution check ups) are greater for AMF than for partners that have staff in-country. We aren't entirely sure why this is the case, although it seems plausible to us that having staff in-country would be useful for this purpose. AMF mentioned that in-country staff are useful for a) determining if budgets are correct, b) monitoring how money is spent, and c) dealing with small changes that a distribution partner might want to make to the budget. AMF also noted that many of its partners already have financial teams in-country who know how to manage international cashflows; AMF believes it makes sense to make use of these experienced teams. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 46
- Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015
- This research will (a) establish baseline rates of mosquito susceptibility to the insecticide used on most LLINs and (b) compare the effectiveness of LLINs used in most distributions and a newer type of LLIN that is more expensive but may be more effective at killing mosquitoes. Rob Mather and Peter Sherratt, conversation with GiveWell, November 6, 2014
- 47
AMF sent us a draft research proposal for the study in early 2015; some details are in our May 2015 update. AMF sent us what we believe was the final version of the research proposal in June 2015; we have not yet reviewed the final version. AMF insecticide research proposal from the London School of Tropical Medicine
- 48
"We are close to concluding discussion with Concern Universal for AMF and CU to co-fund a Malaria Unit (a larger team of people than currently in place dedicated to malaria control related work) in Malawi led by a senior CU Manager and employing 12 Malawians (10 additional hires)." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 49
"The aim of this unit will be to provide more resource to allow 1. improved efficiency in managing the four contracted three-yearly distributions and associated work and 2. allow us to together innovate and develop additional malaria control support for the four districts and the NMCP, specifically: i) intended close liaising with Health Centre re malaria case data and elements related to the monitoring/recording of malaria data i.e. stock levels of RDTKs, rubber gloves and other diagnosis equipment; qualified staff able to test for malaria, presence at the clinic so diagnosis can happen; systems, capacities etc) ii) research (data), discussion and involvement in ways of ensuring 80% sleeping space coverage throughout the three year net-distribution cycle, including investigation of ‘injection strategy’ net distributions involving mini-mass distributions at two years post-distribution and in the subsequent distribution cycle at one-year post distribution." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 50
The distributions and post-distribution check-ups occur frequently enough that having a trained, consistent staff to manage them is worthwhile. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 51
- $636,000 was allocated to the Malaria [Control] Unit in AMF funds status (June 2016), of which $100,000 was already paid. This allocation is about $200,000 higher than the amount AMF mentioned in our September 2015 conversation.
- "This would be an initial three year commitment from AMF of US$413k...Savings made on non-net costs (with the Malaria Unit funded, some costs included in non-net cost budgets would already be covered) we estimate to be of the order of US$70k, and perhaps as high as US$95k per distribution of which there are four in a three year cycle i.e. US$280-380k in total... If all progresses well we may be able to publish plans in the next six weeks once budgets have been agreed and assuming approval from the Malawi NMCP. We do not anticipate problems, although there is a work visa issue to resolve." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- AMF Malaria Unit draft budget
- 52
Over the last several years, AMF believes that its high net hang-up rates have begun to catch the attention of a number of groups in Malawi, including the NMCP, local organizations that distribute nets, and large international NGOs. AMF also told us that it encouraged Concern Universal to take a more active role in a task force on malaria control in Malawi. This meant that as interest in learning from AMF and Concern Universal grew, Concern Universal was available to explain AMF’s methodologies at the malaria task force meetings, and it was eventually decided to implement some of AMF’s practices for the upcoming distribution. Rob Mather and Peter Sherratt, conversation with GiveWell, June 2, 2015
- 53
Rob Mather, AMF Founder, email to GiveWell, October 12, 2015
- 54
"Although AMF won’t be managing this process, it is interested to observe how multiple districts will carry out distribution according to this particular data-driven approach. AMF believes that it will be allowed access to that data, either directly or via its in-country partners." Rob Mather and Peter Sherratt, conversation with GiveWell, February 6, 2015
- 55
- Fully published: Ntcheu 2012 distribution 6-, 12-, 24-, and 33-month PDCUs, Balaka 2013 distribution 6- and 12-month PDCUs, Dedza 2014 distribution 6-month PDCU.
- Skipped: Ntcheu 2012 distribution 18-month PDCU
- Data published, report not published: Dowa 2015 distribution 6-month PDCU, Balaka 2013 distribution 18-month PDCU, Dedza 2014 distribution 12-month PDCU.
- Not published: Balaka 2013 distribution 24-month PDCU, flood-affected districts emergency distribution PDCU.
See our summary of all post-distribution check-up (PDCU) statuses here (see “PDCUs” sheet).
- 56
See our summary of post-distribution check-up (PDCU) results (the follow-up surveys) here.
- 57
- Rob Mather, AMF Founder, conversation with GiveWell, July 19, 2012.
- "[GiveWell]: We know that AMF does not distribute bed nets to recipients if they already have an LLIN in good condition, but does AMF distribute bed nets to people if they have an untreated bed net or an ITN in good condition (instead of an LLIN)?
[AMF]: Yes. If it is not an LLIN it is not counted in any pre-distribution assessment of net need."
Rob Mather, AMF Founder, email to GiveWell, September 16, 2015
- 58
- A representative of Concern Universal told us that Concern Universal performs this data verification for its distributions in Malawi. See this footnote for detail. Nelson Coelho, conversation with GiveWell, April 15, 2016
- @GiveWell Notes from Site Visit with Concern Universal in Malawi (October 2011)@
- 59
In Malawi, the pre-distribution registration survey (PDRS) has consisted of the following steps:- Concern Universal collects demographic data from the district health office, including the number of households in the region and the number of people per household.
- Concern Universal briefs the HSAs, who will do the door-to-door surveying, about the PDRS.
- There are no detailed maps or geographical information system data for rural Malawi, so Concern Universal and HSAs rely on traditional authorities and community leaders to locate all the households in the targeted villages.
- HSAs go door-to-door to register all households in the targeted villages. A village leader accompanies the HSAs during this process. During household registration:
- The village leader and HSAs introduce themselves to the head of the household and explain why they are there.
- The HSA obtains permission to enter the household from the household head. (The difficulty of obtaining this permission varies based on the local culture. Local leaders can lend credibility to the HSA when asking for permission to enter a household.)
- Once in the household, the HSA collects information including the name of the head of the household, the number of over-5 and under-5 people in the household, the number of sleeping spaces, and the number of viable nets owned by the household. (The HSA asks household members for demographic information, and assesses the number of sleeping spaces and number of nets via visual inspection; 'viable' nets are nets that are less than 18 months old with fewer than 10 small holes and no large holes.)
- Concern Universal conducts "105%" data verification during the PDRS. HSAs are responsible for registering 100% of households in a village. In addition, supervisors independently register 5% of households in each village during the PDRS. This data is later compared to the HSA-collected data to check for consistency.
Concern Universal staff and government health officials supervise the registration.
- The data from the registration is entered into a Microsoft Access database, then printed out into registration books.
- After the registration, Concern Universal performs a data verification by assembling a few villages in each catchment area and taking a roll call of the household names in the registration book, and the number of nets that household is scheduled to receive. Verification is conducted for all villages. If someone claims to have been missed during this process, or claims that their number of scheduled nets is incorrect, then this is confirmed by a village leader, and, if confirmed, the correction is made.
Nelson Coelho, conversation with GiveWell, April 15, 2016
- 60
- 61
- "IMA obtained population data for the year 2012 from the Ministry of Health and applied the MOH’s estimate for population growth of 3% per annum." IMA World Health, Kasaï Occidental 2014 distribution report, pg. 6
- "There were many occasions where the number of nets needed for a community was overestimated. This was discovered in the registration and hang up because of diligent work done. Lesson learned[:] Better population estimates are needed and 1.8 people per nets should be the ratio always used." IMA World Health, Kasaï Occidental 2014 distribution report, pg. 46
- In cases where too few nets are sent to a location, AMF's distribution partner typically performs a "mop up" exercise in which it orders extra nets and distributes them several months later to the households that didn't receive the nets the first time around. Rob Mather, AMF Founder, conversation with GiveWell, February 24, 2015
- "If there are a significant number of nets left over – more than a few hundred for example, the nets are moved to the next distribution. An example would be the 5,770 so nets left over in Balaka (the need was assessed as 149,500 and we had moved in 160,000 nets to ensure we had enough nets to cover any additional need and the final number of nets distributed after mop-up was 154,230) which were moved up to Dedza and distributed as part of that distribution. The number of nets ordered and moved in to Dedza reflected that 5,700. If the number of nets left over is in the hundreds, they are passed to the District Health Officer (DHO) to be included a nets for routine distribution. Sometimes there has been a small, several hundred net shortfall and that gap has in the past been filled by nets in zone from other sources." Rob Mather, AMF Founder, email to GiveWell, September 16, 2015
- 62
"Differences [between the Nord Ubangi distribution and the Kasaï Occidental 2014 distribution]
1. The North Ubangi distribution will be a two-phase distribution – a distinct pre-distribution phase and a subsequent distribution phase. In West Kasaï establishing net need per household and distributing of nets was contemporaneous.
2. Distribution of nets will be done from distribution points rather than at the household
3. The ODK registration form has been re-designed to better capture the required data, to eliminate some types of error and to reduce the chance of others.
4. Operational lessons learned from the West Kasaï distribution will be implemented.
5. An experienced, full time technology manager has been employed by IMA meaning there will be more extensive and easier liaising with AMF on data elements of the distribution. We have met the new technology manger and have spent a week working with him and are impressed.Similarities [between the Nord Ubangi distribution and the Kasaï Occidental 2014 distribution]
1. The same smartphone-based data collection technology will be used as in West Kasaï.Comment
IMA has agreed there are benefits to conducting a separate, earlier pre-distribution phase during which accurate household-level net need is established. IMA are less able to estimate accurately population and net need in North Ubangi compared to what they felt was possible in West Kasaï.IMA believes point distributions rather than at-household distribution will be more cost-effective." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- 63
5% spot-checks are not mentioned in the signed distribution agreement. AMF Ghana 2016 distribution agreement
- 64
"The MoH-NMCP will employ '105%' data collection where a) a second set of data collectors visit 5% of the households, randomly selected, in the distribution area one or several days after the 100% data collectors, and without access to the 100% data collected; b) the overlapping 5% of data is compared to assess data accuracy; c) the data collectors are informed in advance of this mechanism and data comparison and d) effective management is in place to ensure the 5% data collectors are not aware of the initial data set for the 5% households." AMF Uganda 2016 distribution agreement, pg. 4.
- 65
"In 2010, Senegal and Cross River State in Nigeria worked on mop-up campaigns following earlier integrated campaigns targeting households with children under five years of age. In both countries, trained health workers or volunteers undertook a household registration to determine:
- the total net need for each household (this was based on one LLIN for two people, rounding up in the case of odd numbers of people in the household, but Senegal also looked at the number of habitual sleeping spaces in each household)
- how many nets each household already had
- how many nets in each household were still viable (in Senegal this number was based on net condition, while in Cross River State, it was based on how long the net had been hanging)
- how many new nets each household would need for full coverage
Both countries had previously undertaken post-distribution surveys which showed high household coverage with LLINs, but during the mop-up exercise they experienced challenges with finding nets in households. Significantly lower numbers of nets were found (50-60 per cent) than would have been expected based on the surveys...
In both countries, it seemed that families often hid nets once word spread that ownership of nets meant no new nets would be received. Despite efforts to encourage families to hang pre-existing nets prior to the household registration in Senegal, people hid nets in order to receive more." Alliance for Malaria Prevention Toolkit (version 2.0) - Chapter 3, pgs. 5-6 - 66
- 67
- 68
- AMF has told us that it is still waiting on photos for the Balaka 2013 and Dedza 2014 distributions, but that it believes that Concern Universal did take photos. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015. This differs from what AMF told us in 2014, which was that Concern Universal had failed to take photos of the Balaka 2013 distribution:
- "[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
- [AMF:] No, CU did not take photos of this one. They are aware of this slip and there will be photos and video footage from Dedza. Photos and videos are also included in the agreement for the DRC-Kasaï distribution."
Rob Mather, AMF Founder, email to GiveWell, May 22, 2014.
- AMF has told us that it is still waiting on photos for the Balaka 2013 and Dedza 2014 distributions, but that it believes that Concern Universal did take photos. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015. This differs from what AMF told us in 2014, which was that Concern Universal had failed to take photos of the Balaka 2013 distribution:
- 69
- 70
See, for example:
- 71
See our March 2012 update on AMF.
- 72
All weekly reports at AMF page on Balaka 2013 distribution. Examples of problems reported:
- "The exercise despite the verification process and data cleaning faced some duplicates that were discovered during the distributions. The duplicate situation had not been dealt with during data cleaning as it was deemed only when both villages were asked to collect nets from the same distribution point that it became clear that there had been duplicate entries. However, different distribution points at the same time though at the same cluster makes it practically impossible for villagers to collect two nets from different sites by double registering." Concern Universal Balaka 2013 week 1 report
- "There was a high number of absenteeism during the urban distribution, which resulted in some members receiving the nets on behalf of others. This absenteeism was explained by the unavailability of the beneficiaries due to professional reasons. There were some complaints from beneficiaries whose nets were received by representatives claiming that the nets weren’t handed over. Handing over of nets to representatives was cancelled since it was clear that, unlike in rural areas where all community members know each other and certify the representative’s identification and the nets were handed over to the legitimate beneficiaries, nets were being misappropriated... For the urban distributions we anticipate to conduct them during the weekend to assure that most of the household owners are free from their daily work related activities." Concern Universal Balaka 2013 week 5 report
- "One health worker assigned to facilitate the distribution process, in the community under his supervision, was caught by the beneficiaries trying to steal about 25 nets. He was reported to the authorities and discharged of his duties. This episode disrupted the distribution process and CU staff had to intervene to keep the population from beating the health worker." Concern Universal Balaka 2013 week 5 report
- 73
All weekly reports at AMF page on Dedza 2014 distribution. Examples of problems reported:
- "The major challenge encountered during this week’s distributions was the misplacing of villages in clusters, which required us to the transfer the nets and distribution registers to the clusters where the villages have presented themselves. This delayed our distribution process but we still managed to reach and carry out the distributions to the affected villages." Concern Universal Dedza 2014 week 1 report
- "There were 20 villages under Kaphuka that did not receive the nets because of poor communication as their HSAs were attending performance appraisals and failed to communicate the distribution dates to their respective villagers." Concern Universal Dedza 2014 week 3 report
- "The major challenge during the week was duplication of registration in a way that some beneficiaries seem to have been deliberately registered in more than one village. The registration data was corrected and the affected beneficiaries only received the nets they were entitled to according their respective village data." Concern Universal Dedza 2014 week 3 report
AMF notes these issue and an additional issue on its blog, "An isolated incident of 300 nets missing from one storage location. This is being investigated and pursued with the police as any nets missing is taken very seriously. 300 nets represents 0.12% of the total nets being distributed. AMF: "Mid-distribution weekly reports for Dedza distribution, Malawi"
- 74
"Mid-distribution Weekly Reports" at AMF page on Dowa 2015 distribution cover the first 184,554 nets distributed out of a total of 396,900 nets. Examples of problems reported:
- "The major challenge we encountered during the distribution was the breaking down of our distribution vehicles, which forced us to interrupt the distribution for almost one month to have them fixed, as some spare parts could not be sourced locally." Concern Universal Dowa 2015 weeks 1-3 report, pg. 2
- "Some beneficiaries didn’t show up as they were attending a clothing items distribution on the same day. The fact that most inhabitants of the said villages are refugees from Rwanda, Burundi or Somalia caused identification challenges, preventing individuals claiming nets on behalf of the beneficiaries from receiving the nets. These beneficiaries will be considered during the mop up exercise." Concern Universal Dowa 2015 weeks 1-3 report, pg. 3
- "Distributions were not conducted in ten of the planned villages under the three health facilities due to funerals hence we deferred distribution and managed to reach ... 493 of the 503 planned villages. However, arrangements will be made at a later date when we will reach them and conclude the distributions in the deferred villages hence their nets have been currently taken back to the warehouse for safe keeping. The above mentioned villages are: Msaka, Kancheri, Chimbalanga Mononga and Nkhota villages from Mtengowanthenga health facility with, respectively, 20, 100, 201, 91 and 49 nets returned; Masiya, Sintala 2, Mulode 2 and Mgoli from Dzoole health facility with a nets requirement of 42, 274, 26 and 84 nets respectively and Mphinda village under Kayembe health facility with a requirement of 140 nets." Concern Universal Dowa 2015 weeks 1-3 report, pg. 5
- 75
“AMF has provided regular, public updates on the large, ongoing net distribution in the Ntcheu district of Malawi. Expected data collection has occurred and the distribution has proceeded close to schedule. AMF's distribution partner, Concern Universal, has been transparent about problems it has encountered, and seems to have a robust process to catch problems (such as attempts to steal nets) when they arise.” See our March 2012 update on AMF.
- 76
- Rob Mather and Peter Sherratt, conversation with GiveWell, November 6, 2014
- See this footnote for a description of Concern Universal's distribution procedures.
- 77
- 78
- For example, See Dedza 2014 8-month post-distribution check-up data, "Miscellaneous data" tab, where there are 7,718 + 7,111 + 511 = 15,340 nets total, and of those 7,557 + 7,087 + 6 = 14,650 are AMF nets, which is 95.5% of all nets.
- A Concern Universal representative told us that PDCU enumerators assess whether the net is an AMF net by looking at the net's label. Nets sourced from AMF distributions have AMF's logo and the month and year of the intended distribution date on the label. Nelson Coelho, conversation with GiveWell, April 15, 2016
- 79
"Our distribution of 676,000 nets in Kasaï Occidental in partnership with IMA World Health (IMA) is our first one using smartphone technology for data collection. We see this as an exciting development with significant potential benefits including:
- Acts against potential theft
- Improved accountability
- Greater transparency
- Greater data accuracy
- Improved cost effectiveness
- Additional data can be collected
- Reduced operational risk
The use of this technology may become a significant determinant of future net distributions that we fund. We will report publicly on our experience with the Kasaï Occidental 2014 distribution and the data gathered…
GPS information can also be gathered helping to locate households and tie the number of nets delivered to each." AMF: "Introduction of smartphone technology to collect distribution data" - 80
AMF: "Initial net distribution verification data for West Kasaï, DRC"
- 81
IMA World Health, Kasaï Occidental 2014 distribution data
- 82
AMF lists the countries it has provided nets to at @AMF Countries Involved@. The Malaria Atlas Project has compiled data on malaria risk by location at Malaria Atlas Project Endemic countries.
- 83
- 84
- WHO 2014 Malaria World Report, pg. 37, Figures 8.6 and 8.7.
- The Institute for Health Metrics and Evaluation's (IHME) Global Burden of Disease tool estimates 63.04 malaria deaths per 100,000 in DRC in 2013 (95% confidence interval: 31.34 to 117.43). IHME Global Burden of Disease tool
- IHME's Global Burden of Disease tool estimates 28.06 malaria deaths per 100,000 in Malawi in 2013 (95% confidence interval: 15.52 to 49.95). We do not know why the IHME and WHO estimates disagree about Malawi's malaria death rate, though our impression is that cause-specific mortality data collection in the developing world is often of low quality. IHME Global Burden of Disease tool
- 85
- WHO 2014 Malaria World Report, pg. 37, Figures 8.6 and 8.7.
- IHME's Global Burden of Disease tool estimates 63.34 malaria deaths per 100,000 in Ghana in 2013 (95% confidence interval: 43.95 to 88.29). IHME Global Burden of Disease tool
- IHME's Global Burden of Disease tool estimates 78.48 malaria deaths per 100,000 in Uganda in 2013 (95% confidence interval: 50.33 to 120.33). IHME Global Burden of Disease tool
- IHME Global Burden of Disease tool
- 86
- "Figures from Ntcheu District Hospital show that, in the twelve months to July 2011, 109,972 malaria cases were reported in under five children with a further 133,768 cases reported in the rest of the population." Concern Universal Ntcheu 2012 distribution proposal pg. 1
- "All of Malawi (except some isolated highland areas) is designated a high risk malaria area. There were 96,453 reported cases of malaria in Balaka District over the past twelve months." Monthly data broken down into cases among under-5s and others also provided. Concern Universal Balaka 2013 distribution proposal, pg. 2
- 87
- "Continued use of nets is very important. Every six months, a post-distribution survey is carried out to assess net usage and net condition. Approximately 5% of the nets distributed are assessed through visits to randomly selected households. The data collected are used to determine if additional community-level malaria education activities are required. All data are published." AMF information we publish
- Example from a distribution proposal: "Please confirm you will carry out Post‐Distribution Surveys (PDSs)** every 6 months post‐distribution for a period of up to four years to assess the level of net usage (hang‐up %), correct usage and condition of the nets and you will provide us with the findings. Each survey would cover approximately 5% of households." Concern Universal Dedza 2014 distribution proposal, pg. 2
- AMF has to be flexible about the timing of PDCUs because often there are delays that cannot be prevented. If a PDCU is delayed, the next PDCU will occur on schedule (e.g. even though the Dedza 2014 distribution's first PDCU was done at 8 months, the next distribution will occur at 12 months), unless the delay was severe enough that sticking to the schedule would not make sense (as happened with the Ntcheu 2012 PDCUs). Although AMF originally planned to request PDCUs for four years after a distribution, this doesn't make sense in some contexts where it is required that a mass distribution of LLINs occur at least every 3 years (such as Malawi). Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 88
No report, only data, is available for the 19-month Balaka 2013 PDCU, and we do not know if this data is complete.
- 89
No report, only data, is available for the 14-month Dedza 2014 PDCU, and we do not know if this data is complete.
- 90
No report, only data, is available for the 6-month Dowa 2015 PDCU, and we do not know if this data is complete.
- 91
The methodology used in the Balaka 2013 6-month follow up survey seems similar to that used in the Ntcheu surveys discussed in our August 2014 update on AMF. Specifics from Concern Universal Balaka 2013 6-month post-distribution check-up report:
- Sample selection: "Balaka district has 14 health facilities. It was decided to collect data from 5% of households in each HCCA [health center catchment area], which meant a different number of households in each HCCA as per individual health facility populations. Between 25 and 90 households were randomly selected from each of the selected seven to ten villages, depending on the HCCA, with the villages also selected at random. Villages were randomly selected using the village lists generated from the pre-distribution and distribution work for the October 2013 AMF-funded universal coverage LLIN distribution. A random number table was used to select the villages. Households were randomly selected using the household lists produced during the same campaign. A random number table was used to select the households. Five more households were put on reserve in case no one was at home in the selected households." pg. 5
- Who carried out the survey: "10 data collectors and 2 supervisors from the District Health Office were involved in the PDCU. The supervisors were responsible for checking the data collection exercise at the same time monitoring how the data was being collected as per requirement." pg. 5. Previous reports were more specific about who the data collectors were (for example, in the Ntcheu 24-month survey they were Health Surveillance Assistants, Malawi's version of community health workers.
- Data quality audits: "Supervisors were required to visit 5% of the households in their area to check the accuracy of the data collectors’ work and had to check all the completed forms submitted to them before submitting them to the Project Manager. The sampled visited households were also chosen at random so the work of all data collectors was checked." pg. 6
- Cost: "The PDCU cost was US$8,060 equal to US$1.78 per household visited (or $0.052 per net originally distributed which numbered 154,230)." pg. 3
- Data entry/cleaning: "Improvements in the data entry interface since the last PDCU carried out in Ntcheu (Ntcheu PDCU-24) by AMF meant the data entry proceeded with almost no errors. This reduced the error-checking phase to almost nothing." pg. 6
- 92
From a list of all villages that received nets in a distribution, Concern Universal's project manager arbitrarily picks 5% of the villages to carry out PDCUs in. For each PDCU, the project manager tries to not select villages that are geographically close to each other, and does not pick the same villages for each successive PDCU. For each selected village, the project manager arbitrarily selects 25 households to be surveyed during the PDCU. Nelson Coelho, conversation with GiveWell, April 15, 2016.
- 93
The type of technical information we would ideally like to see includes:
- Results of audits on data quality.
- Analysis of the extent to which respondent selection was carried out according to the protocol.
- Analysis of the extent to which selected households were replaced with pre-selected alternate households (i.e. attrition from the sample).
- Survey text used in each survey round and detailed protocols for responses that are directly observed by surveyors.
- 94
- Ntcheu 6-month post-distribution check-up data
- Ntcheu 15-month post-distribution check-up data
- Ntcheu 24-month post-distribution check-up data
- Ntcheu 33-month post-distribution check-up data
- Balaka 2013 6-month post-distribution check-up data
- Balaka 2013 14-month post-distribution check-up data
- Balaka 2013 19-month post-distribution check-up data
- Dedza 2014 8-month post-distribution check-up data
- Dedza 2014 14-month post-distribution check-up data
- Dowa 2015 6-month post-distribution check-up data
- 95
- Rob Mather, AMF Founder, email to GiveWell, June 30, 2014
- A representative of Concern Universal told us that, due to limited indoor space, some recipients take down their nets during the day and hang them up each night. When assessing whether a net is being used, enumerators observe the net and the sleeping space, ask the recipient questions, then make a judgment call. A net that is being used regularly will likely show signs of wear that an unused net would not; and a sleeping space over which a net is hung nightly may show evidence of the hanging even when the net is not present. Nelson Coelho, conversation with GiveWell, April 15, 2016
- 96
In the data for AMF's post-distribution check ups, it looks to us like "Nets Received" = "Hung" + "Present Not Hung" + "Missing" +" Worn out/not usable", implying that "nets hung" and "nets that are worn out" are exclusive categories. See, for example, Concern Universal Ntcheu 2012 33-month post-distribution check-up data
- 97
"'very good condition' (fewer than 2 holes of up to 2cms in size)." Concern Universal Ntcheu 2012 33-month post-distribution check-up report, pg. 4
- 98
- "‘good condition’ (fewer than 10 small holes on them)" Concern Universal Ntcheu 2012 33-month post-distribution check-up report. pg. 3
- Note that nets with fewer than 10 small holes used to be considered in "OK" condition.
"'OK' (fewer than 10 small holes on them)" Concern Universal Ntcheu 2012 24-month post-distribution check-up report, pg. 4
- 99
- "‘viable condition’, with more than 10 holes or 1 hole larger than 10 cm," Concern Universal Ntcheu 2012 33-month post-distribution check-up report, pg. 3
- Note that nets with more than 10 small holes used to considered in "poor" condition. "'Poor' condition with more than 10 holes or 1 hole larger than 10 cm." Concern Universal Ntcheu 2012 24-month post-distribution check-up report, pg. 4
- 100
"Worn out means not usable. Typically, holes will be 10 cms of cms in size and numerous and the net considered effectively unrepairable and/or structural integrity will have gone. If hung the net would not provide protection against mosquitoes given the extent of holes or tears or other loss of structural integrity." Rob Mather, AMF Founder, email to GiveWell, June 30, 2014.
- 101
- @AMF how we work with distribution partners@, pg. 4
- According to a representative of Concern Universal, when assessing whether a net is being used, enumerators make a judgment call. Nelson Coelho, conversation with GiveWell, April 15, 2016
- 102
Rob Mather, AMF Founder, email to GiveWell, June 30, 2014
- 103
For example, consider data from the Ntcheu 2012 33-month PDCU:
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results , the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
[AMF]: "This is because the latter figure includes non-AMF nets i.e. there are 462 non-AMF nets included in the ‘Usage of the nets’ tab as we wished to have an overall figure for sleeping space coverage (52%). We may add a note to this tab to indicate there are some non-AMF nets included in the data on this tab." Rob Mather, AMF Founder, email to GiveWell, October 13, 2015. Note that we did not ask AMF where the 462 figure comes from. - 10,664 AMF nets were found to be "very good", "good", or "viable" condition. 10,664 + 462 non-AMF nets = 11,126 nets, which is less than the 11,497 nets "used correctly", implying that some AMF nets that are "worn out" are included in the "used correctly" category. Concern Universal Ntcheu 2012 33-month post-distribution check-up data
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results , the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
- 104
For example, in the Concern Universal Ntcheu 2012 33-month post-distribution check-up data, we see that:
- 10,664 AMF nets are hung ("Presence of AMF Nets" tab)
- 10,664 AMF nets are also in "viable", "good", or "very good" condition ("Condition of AMF Nets tab"). It seems odd to us that these numbers match.
- 11,497 nets are "used correctly." ("Usage of the nets" tab) AMF told us 462 non-AMF nets were included in this tab, meaning that 11,035 AMF nets are included.
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results , the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
[AMF]: "This is because the latter figure includes non-AMF nets i.e. there are 462 non-AMF nets included in the ‘Usage of the nets’ tab as we wished to have an overall figure for sleeping space coverage (52%). We may add a note to this tab to indicate there are some non-AMF nets included in the data on this tab." Rob Mather, AMF Founder, email to GiveWell, October 13, 2015 Note that we did not ask AMF where the 462 figure comes from.
- [GiveWell]: "Second, I'm trying to better understand the outcomes measured for post-distribution check-ups. In particular, I'm curious what the "People - Covered", "People sleeping under nets", and "Nets Used - Correctly/Not Correctly" metrics mean (under the "Usage of nets" tab). In the Ntcheu 33-month PDCU results , the "Presence of AMF Nets" tab lets me see how many nets are "hung", "present but not hung, "missing", or "worn out": the numbers of "hung" and "present but not hung" nets are 10,667 and 630, respectively, for a total of 11,297 nets that aren't missing or worn out. However, under the "Usage of nets" tab, I see that 11,498 nets are used "Correctly." Since 11,498 > 11,297, I believe that some of the "worn out" nets must be included in the "used correctly" metric. If anyone sleeping under a net that is "used correctly" is considered to be "covered", then some of the people counted in the "covered" category are actually sleeping under worn out nets. Is my understanding correct?"
- 11,035 nets "used correctly" is more than the 10,664 nets "hung", implying that nets that were "present but not hung" or "worn out/not usable" could be included. ("Presence of AMF Nets" tab).
- 11,035 nets "used correctly" is also more than the 10,664 nets found in "viable", "good", or "very good" condition, implying that some "worn out" nets are "used correctly."
- 24,980 people were found to be "sleeping under nets" ("Usage of the nets" tab). These are the people who are also labelled as "Covered" by nets.
- 24,980 people / 11,035 nets "used correctly" = 2.26 people per net, which seems reasonable, though higher than we would have expected. 24,980 people / 10,664 nets "hung" = 2.34 people per net, which also seems reasonable but higher than we would have expected. We are uncertain about whether "sleeping under a net" means sleeping under a "used correctly" net, a "hung" net, or a non-"worn out" net.
Concern Universal Ntcheu 2012 33-month post-distribution check-up data
- 105
- "Usage does not appear to have been near-universal. Most studies report usage rates in the range of 60-80%, though some report 90%+ usage." From our page on Long Lasting Insecticide Treated Nets.
- Note that the World Malaria Reports that we looked at showed similar usage rates.
- 106
- "Alliance for Malaria Prevention 2011 lays out a model for estimating the number of LLINs still in use after distributions: 'the number of LLINs already distributed over the last three years and considered to be available in households should be calculated and subtracted from the total need, working with a decay rate of 8 per cent at one year (0-12 months), 20 per cent at two years (13-24 months) and 50 per cent at three years (25-36 months).'…
Bottom line: We believe that the "8%-20%-50%" model is the most widely used and most reasonable approximation available at the moment for capturing the extent to which LLINs remain in use in the years following distribution, accounting for any factors that might cause LLINs to be discarded or additional LLINs to be purchased. It implies an average of 2.22 years of use for each LLIN distributed. Data and analysis on this topic appears extremely thin; we have little sense for how long LLINs last in practice." - "If we assume that an LLIN has a 92% chance of being in use at a given point in the first year after distribution, this implies that for each LLIN delivered, an average of 0.92 LLIN-years of use are obtained in the first year. Assuming 0.92 LLIN-years of use in the first year, 0.8 in the second year and 0.5 in the third year would yield an overall average of 2.22 years of use per LLIN. This is substantially less than the "official life" of an LLIN. As discussed below, we believe this makes sense because the decay function is intended to account for wastage of all kinds, including loss/failed delivery of LLINs, improper use resulting in disrepair, etc."
- "Alliance for Malaria Prevention 2011 lays out a model for estimating the number of LLINs still in use after distributions: 'the number of LLINs already distributed over the last three years and considered to be available in households should be calculated and subtracted from the total need, working with a decay rate of 8 per cent at one year (0-12 months), 20 per cent at two years (13-24 months) and 50 per cent at three years (25-36 months).'…
- 107
See this spreadsheet for the data referenced in the following discussion.
- 108
See this spreadsheet for the data referenced in the following discussion.
- 109
AMF Malawi universal coverage calculations (September 26, 2011)
- 110
See our March 2012 update on AMF.
Exact net figure from Rob Mather, AMF Founder, email to GiveWell, November 20, 2012.
- 111
This is an understanding formed over many conversations.
- 112
AMF Malawi universal coverage calculations (September 26, 2011)
- 113
- ALMA LLIN gap analysis (June 2016)
- Melanie Renshaw, conversation with GiveWell, March 16, 2016
- Marcy Erskine, conversation with GiveWell, March 29, 2016
- Rob Mather and Peter Sherratt, conversation with GiveWell, April 25, 2016
- 114
Rob Mather and Peter Sherratt, conversation with GiveWell, April 25, 2016
- 115
"Health Surveillance Assistants (HSAs) are Government extension workers- they are the lowest tier of government presence in the decentralized health system." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.
- 116
"As such they are the first line of response to any public health issues in communities. Their job involves disseminating health related information (such as encouraging people to make use of sanitary facilities, go for immunizations, sleep under mosquito nets etc.), carrying out sanitation and hygiene campaigns and sending data on take-up of facilities to the District Council, conducting basic nutrition support, weighing children and reporting levels of stunting and wasting, detecting common communicable diseases and reporting these to clinicians and other health providers, implementing immunization campaigns etc. As you can see being involved in universal net distribution fits very well with their core public health responsibilities. HSAs need to have a primary school completion certificate as a minimum but the majority of them will have O-Levels (exams sat by pupils aged 16 if they have completed the school system at the recommended pace). Once they have been selected as HSAs they are sent on an initial 9 months intensive training course where they will be trained in many aspects of public health including how to recognize common diseases, how to administer immunizations etc." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.
- 117
- Pre-distribution registration surveys appear to be completed relatively quickly: "Approximately 480 personnel will be involved in the PDRS, with the majority, some 460, involved for 5-7 days over the data collection period." Concern Universal, Dowa 2015 planning document, pg. 4
- In the Ntcheu 2012 distribution, the verification of PDRS data took several weeks: "The verification process took the verification team of 10 members 18 days to complete and in a day 20 clusters were verified with 10 verification sites in the morning and 10 verification sites in the afternoon." Concern Universal Ntcheu 2012 distribution report, pg. 7
- In the Ntcheu 2012 distribution, distributions were scheduled for several weeks, but covered approximately 10 "clusters" per day: "The distributions were scheduled to have been concluded within 28 days with the team distributing at 10 clusters per day covering five weeks." Concern Universal Ntcheu 2012 distribution report, pg. 7-8
- Concern Universal has recently started to use fewer staff for post-distribution check ups, which causes them to take somewhat longer: "However, in collaboration with the District Environmental Health Office (DEHO) and Malaria Coordinator (MC) and lessons lea[r]nt from 24 month Ntcheu PDCU, it was recommended to have a focused team of 10 data collectors rather than have the HSAs as data collectors from each HCA. This was based on the following reasons. First, this would reduce the number of data collectors that would need to be monitored and trained. Second, we would be able to select reliable individuals whom we could trust to do a diligent and accurate job of collecting the data. Third, it would leave the majority of HSAs to carry on with the normal health tasks and duties. Fourth, by having the same people covering the whole exercise they will get acquainted to the task and reduce errors on data collection. This meant the data collectors would spend thirty seven days collecting data rather than the one or several days if many more data collectors were to be used. This was judged the preferable way of organising and managing the data collection phase." Concern Universal Ntcheu 2012 33-month post-distribution check-up report, pg. 5. Other recent post-distribution check-up reports have similar language.
- 118
"i) Field Supervisors (FSs)
22 FSs were selected from permanent and senior health staff in Tshikapa Health District.
ii) Community Health Workers (CHWs) – data collectors
Each of the 22 FSs had the responsibility of recruiting, in each HA they were designated, enough CHWs to gather household data and hang nets. Two primary recruitment criteria were literacy and familiarity with using a mobile phone. The number of CHWs recruited depended on the size of the HA and the number of households to be visited. The aim was to recruit enough CHWs to carry out the entire registration and hang‐up, once it commenced, in a five day period. Between 20 and 40 CHWs were recruited by each FS for a total of 4,000 CHWs across the 8 HZs (8 HZs x 20 HAs x 25 CHWs per HA = 4,000 CHWs)." IMA World Health, Kasaï Occidental 2014 distribution report pg. 13 - 119
- 120
See GiveWell estimate of AMF cost per net (May 2016). In estimating the overall cost per LLIN in Malawi, we have used an average that weights distribution costs by distribution size (i.e. costs associated with larger distributions are given proportionally more weight). Another option would be to put more weight on the actual costs (e.g., one could weight the Balaka 2013 distribution at 50%, the Dedza 2014 at 25% and the Dowa 2015 at 25%) to account for the fact that actual costs are often more reliable than budgeted costs.
- 121
Rob Mather, AMF Founder, conversation with GiveWell, May 23, 2014. Managerial staff time is listed in the budget.
- 122
See the “Ntcheu 2012 other costs” sheet in GiveWell estimate of AMF cost per net (October 2015) for details.
- 123
- See the “AMF org” sheet of GiveWell estimate of AMF cost per net (May 2016) for details.
- AMF's fiscal year runs from Jul 1 - Jun 30 (e.g. FY 2012 is Jul 1 2011 - Jun 30 2012).
- 124
For example:
- The spreadsheet IMA shared of actual costs for the Kasaï Occidental distribution is hard to make sense of and appears to contain several errors. Also, about 20% of the costs are "unconfirmed charges," which are not listed individually.
- For Malawi, Concern Universal shared, in 2012, information on non-monetary costs (such as office space) and the value of time provided by government employees. IMA's actual cost data includes an estimate of indirect costs (as a percentage of total direct costs), but it's unclear if it includes costs paid for by the government or other actors. As a rough approximation, we have added $0.13 per net, which was the estimate of non-monetary partner and local government costs from the 2012 Ntcheu, Malawi distribution.
- 125
- The budget AMF shared with us estimated a cost of $9,000 per PDCU, or $0.01 per net, which is significantly lower than the costs of PDCUs in Malawi. IMA World Health, Kasaï Occidental 2014 non-net costs final budget. AMF told us that this is because not all cost data has been received from IMA yet. Rob Mather, AMF Founder, email to GiveWell, September 9, 2015.
- For the costs of PDCUs in Malawi, see GiveWell estimate of AMF cost per net (October 2015) Sheets: "Summary" and "Schedule for Update of NNCs". The Ntcheu 2012 24-month PDCU is the cheapest of all conducted so far, at $0.03 per net (we aren't sure what caused this PDCU to be so inexpensive). Most PDCU costs are estimated to be between $0.05-$0.10 per net.
- 126
- AMF told us that in DRC net costs and non-net costs are very roughly the same. Rob Mather, AMF Founder, conversation with GiveWell, May 23, 2014
- "In other cases, non-net costs are assumed to $2.50 per LLIN, which is the historical average. Non-net costs can vary significantly across countries. For example, in Angola and DRC, non–net costs may be more than $5." Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, May 23, 2014
- 127
- 128
GiveWell estimate of AMF cost per net (May 2016):
- Average of per-net pre-distribution and distribution costs in Malawi and DRC: ($0.71+$2.28)/2 = $1.49
- AMF's estimate of per-net pre-distribution and distribution costs for future distributions: $1.50
- Average of per-net post-distribution and follow up costs in Malawi and DRC: ($0.34+$0.43)/2 = $0.38
- AMF's estimate of per-net post-distribution and follow up costs for future distributions: $0.42
- 129
Note that we have excluded the cost of insecticide resistance research funded by AMF. This is a debatable exclusion but we feel it is appropriate because we believe this research is likely to have impacts which are not captured here.
- 130
"AMF Medium Term Strategy… Increase AMF central staffing by four people before the end of 2016: two program directors, one operations assistant and a technology assistant. Keep under review to assess hiring ahead of need. Hire one to two permanent staff in each country in which large scale distributions are funded." AMF medium term strategy discussion document (May 2016).
- 131
Note that we have estimated the number of nets AMF will distribute in FY 2017 by signed distribution agreements for Ghana and Uganda and assumed that it will sign agreements for half the nets in its distribution pipeline for 2017 (it had 3 distributions in the pipeline as of May 2016). GiveWell estimate of AMF cost per net (May 2016).
- 132
See for example Roll Back Malaria Partnership gap analysis (September 2013).
- 133
- 134
"The gap analysis includes both the cost to purchase LLINs and non-net costs, such as
shipping and distribution. In cases where national governments provided data on non-net costs, this data was used in the analysis. In other cases, non-net costs are assumed to $2.50
per LLIN, which is the historical average. Non-net costs can vary significantly across countries. For example, in Angola and DRC, non–net costs may be more than $5." Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, May 23, 2014 - 135
See our model comparing the cost-effectiveness of long-lasting insecticide-treated nets, deworming, cash transfers, and iodine fortification, Sheet "Bednets."
- 136
Funding commitments as of June 2016 (AMF funds status (June 2016)):
- Malawi 2018 distribution (1.48 million nets and all non-net costs): $5.5 million
- Insecticide resistance study in Nord Ubangi: $0.7 million
- Malawi Unit, Malawi: $0.54 million
- Ghana 2016 distribution (non-standard non-net costs): $1.7 million
- Uganda 2016 distribution (10.7m nets and non-standard non-net costs): $29.7 million
- 137
- 138
- In GiveWell's 2011 metrics year (February 1, 2011 to January 31, 2012), GiveWell-influenced donors gave $2,720,750 to AMF. In AMF's 2011 fiscal year (July 1, 2010 to June 30, 2011), AMF took in $2,092,594 in revenue. Note that our metrics year does not align with AMF's fiscal year, so these figures cannot be directly compared. We have not tried to match our AMF money moved to AMF's revenue at a more granular level.
- In our 2012 metrics year, GiveWell-influenced donors gave $5,895,544 to AMF. In AMF's FY 2012, it took in $3,921,079 in revenue.
- In our 2013 metrics year, GiveWell-influenced donors gave $2,502,125 to AMF. In AMF's FY 2013, it took in $7,956,078 in revenue.
- In our 2014 metrics year, GiveWell-influenced donors gave $9,797,274
to AMF. In AMF's FY 2014, it took in $3,972,611 in revenue. - In total, over GiveWell's 2011 to 2014 metrics years, GiveWell-influenced donors gave $20,915,693 to AMF. Over AMF's FY 2011 to 2014, AMF took in $17,942,362. Note that our metrics year does not align with AMF's fiscal year, so these figures cannot be directly compared.
- For GiveWell money moved figures, see our impact page.
- AMF Financial information
- 139
We use "2015" here to refer to our 2015 "metrics year," the period over which we track donations: February 1, 2015 to January 31, 2016). More here.
- 140
- $43.5 million total - $38.3 million attributable to GiveWell = $5.2 million not attributable to GiveWell-influenced donors
- GiveWell 2015 metrics report
- Andrew Garner, AMF employee, email to GiveWell, February 22, 2016
- $5.2 million attributable to non-GiveWell-influenced donors strikes us as high, given that in December 2015, we estimated that AMF would take in $1.6 million not attributable to GiveWell-influenced donors during the 2015 giving season, and in September 2015, AMF estimated that it would take in $1.5 million not attributable to GiveWell-influenced donors during the 2015 giving season. Generally, a large majority of a year's donations occur during the giving season (roughly October to January of the next year), so these estimates imply only slightly larger estimates for the entire year's money moved.
We do not know the reason for these underestimates. It is possible that a substantial portion of the $5.2 million not attributed to GiveWell was in fact influenced by our research.
- 141
Rob Mather and Peter Sherratt, conversation with GiveWell, February 28, 2016
- 142
- Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, May 23, 2014
- Melanie Renshaw, African Leaders Malaria Alliance Chief Technical Advisor, phone conversation with GiveWell, October 20, 2015
- Melanie Renshaw, conversation with GiveWell, March 16, 2016
- Summary of year-end gaps in Rob Mather, email to GiveWell, May 9, 2016. 2013-2015 year-end gaps were 22m, 38m, and 39m, respectively, and "does not include distributions delayed and pushed into the next year with the hope of finding funding for the subsequent year."
- ALMA LLIN gap analysis (June 2016)
- 143
ALMA LLIN gap analysis (April 2016) and ALMA LLIN gap analysis (June 2016)
- 144
- 145
"Republic of the Congo also did not receive this funding partly due to its poor performance during a previous malaria grant." Marcy Erskine and Melanie Renshaw, conversation with GiveWell, April 18, 2016
- 146
"Gaps that emerge when a routine distribution (for example, targeting pregnant women or infants) does not occur do not roll over, as the opportunity to protect the missed individuals cannot be regained." Marcy Erskine and Melanie Renshaw, conversation with GiveWell, April 18, 2016
- 147
This includes two countries where AMF where AMF's pipeline includes fewer nets than the total gap estimated by ALMA, and one country where AMF is considering funding PBO nets and ALMA estimates no gap. AMF funds status (June 2016) and ALMA LLIN gap analysis (June 2016)
- 148
Rob Mather and Peter Sherratt, conversation with GiveWell, April 25, 2016
- 149
ALMA LLIN gap analysis (April 2016)
- 150
"Accurate net gap projections require knowledge of countries' net needs and funding amounts from the Global Fund (its next replenishment will cover the three-year period from 2018-2020) and other major funders. These have not been confirmed, as funders' 2018 budgets are not finalized. As a result, the 217 million net gap projected for 2018 does not account for funds that major funders are expected to commit and thus is expected to reduce." Marcy Erskine and Melanie Renshaw, conversation with GiveWell, April 18, 2016
- 151AMF funds status (June 2016)
- 152
- Rob Mather, AMF Founder, conversation with GiveWell, April 13, 2016
- Rob Mather and Peter Sherratt, conversation with GiveWell, April 25, 2016
- 153
"AMF Medium Term Strategy… Increase AMF central staffing by four people before the end of 2016: two program directors, one operations assistant and a technology assistant." AMF medium term strategy discussion document (May 2016)
- 154
- Rob Mather and Peter Sherratt, conversation with GiveWell, April 25, 2016
- "The operational plan for a distribution typically covers 12 months. The planning horizon for a specific distribution is, however, typically 18 to 24 months as discussions to assess and approve a distribution start six months earlier and the funds required (we need to have funds in hand in order to have serious discussions) are aggregated in the preceding months." AMF: "Operational planning (12 months) and planning horizon (18 to 24 months)"
- 155
- 156
We believe this is more likely because AMF is paying for only "non-standard" non-net costs for the distributions it has signed most recently (Ghana 2016 and Uganda 2016-17) and has identified potential funders for "standard" non-net costs for at least two of the three distributions that it was in the 'final pre-agreement stage' with as of June 2016. AMF funds status (June 2016) and Rob Mather, email to GiveWell, June 15, 2016.
- 157
- 158
AMF funds status (March 2016) and AMF funds status (June 2016)
- 159
- AMF confirmed this timeline in a comment on a draft of this page.
- In February 2016, AMF was unaware Uganda's 2016 net gap. AMF was able to prioritize Uganda discussions and sign a distribution agreement in April 2016. Rob Mather, AMF Founder, email to GiveWell, April 30, 2016. We don't know if this was an unusual case, or if there were negative consequences of the short timeline.
- AMF's planning horizon was longer for the Ghana 2016 distribution (which was signed in January 2016). We have seen emails indicating that AMF was aware of the need as early as October 2014. We have seen emails from January 2015 that include detailed discussions of the distribution requirements. Rob Mather, AMF Founder, Ghana distribution emails, October 2014 to January 2015
- 160
In 2014, we believed that AMF had additional room for more funding because:
- In 2013, AMF had completed two large-scale distributions, both of which were with a single distribution partner, Concern Universal in Malawi. It had not signed agreements with other partners. As of November 2014, AMF had signed agreements for two large distributions with IMA World Health in the DRC, helping to demonstrate its ability to finalize distributions with other partners in other locations.
- By the end of 2014, AMF had spent or committed a large portion of the funds it raised prior to 2014. As of November 2014, AMF held $4.9 million in uncommitted funds, of which it raised $2.65 million in 2014. Of committed funds, $1.9 million were for expenses that AMF would incur more than a year later, and AMF was considering reallocating these funds to the nearer-term with the expectation of raising enough to cover its later commitments by the time they were due. This reallocation brought AMF's total available funds to $6.8 million.
- "Funds in hand as at Nov 2013: $12.5 m
Committed since Nov13: $10.30m (10.3/12.5 = 82%)- - DRC, West Kasaï 2014 $2.0m (NETS only)
- DRC, Idjwi 2015 $0.37m (NETS + all non-net costs)
- Malawi, Dowa, 2015 $1.49m (NETS + all non-net costs)
- Malawi, Ntcheu 2015 $1.34m (NETS + all non-net costs)
- Malawi, Balaka 2015 $0.94m (NETS + all non-net costs)
- Malawi, Dedza 2017 $1.72m (NETS + all non-net costs)
- DRC, North Ubangi 2015 $2.34m (NETS + shipping)
- Malawi, Dedza, 2014 $0.10m (additional nets)Funds uncommitted $2.2m
New funds raised $2.65m
Total funds uncommitted $4.85m"
Rob Mather, AMF Founder, email to GiveWell, November 26, 2014 - "If ‘make available’ all Dedza17: $1.720m [i.e. if funds committed to the Dedza 2017 distribution were made available for an earlier distribution]
Ditto, Yr2+ PDCU Do/Nt/Ba: $0.2 m [refers to future post-distribution check ups in Dowa, Ntcheu, and Balaka]"
Rob Mather, AMF Founder, email to GiveWell, November 8, 2014
- "Funds in hand as at Nov 2013: $12.5 m
- AMF's pipeline for potential distributions included several distributions with established partners. Given AMF's history with these partners, we guessed that there were fewer barriers to signing agreements for additional distributions. Funding all such distributions would have required AMF to raise significantly more funds. AMF may also have had opportunities to fund distributions with new partners.
- 161
- Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- "AMF and Concern Universal, Malawi (CU) have accepted the Malawi National Malaria Control Programme’s (NMCP) invitation to assume responsibility for the district-wide net distribution re-coverage campaigns in Ntcheu, Balaka, Dedza and Dowa districts in 2018...This timing is consistent with the three year re-coverage cycle needed to keep affected populations protected from malaria-carrying mosquitoes when they sleep at night." AMF: "US$6m commitment to malaria control support in Malawi in 2018"
- 162
- "To keep you up to date, we have concluded a distribution agreement in relation to Ghana (attached) for 2.7 million nets. These nets are for distribution in three of Ghana’s 10 regions, Upper West, Northern and Greater Accra in the first half of 2016 (mainly June). It includes our normal data and PDCU requirements. The Global Fund is covering non net costs, other than the additional AMF requirements (primarily the PDCUs)." Rob Mather, AMF Founder, email to GiveWell, January 28, 2016
- AMF Ghana 2016 distribution agreement
- AMF Distributions
- 163
- "The Uganda agreement is now signed with nets to be distributed between August 2016 and March 2017. We’ll be posting information publicly early next week … AMF is funding 10.7m LLINs (US$26,438,253.60)" Rob Mather, AMF Founder, email to GiveWell, April 17, 2016
- AMF Uganda 2016 distribution agreement
- AMF Distributions
- 164
"With or without a co-funding partner, our sense, is NMCPs will first try and achieve funding from:
a) Organisations from whom they have received funding before and with whom they have established relationships (know how the relationship operates, reporting requirements etc)
b) Organisations with high levels of funding to minimise the number of agreements reached (often 2)
c) Organisations who have the least accountability requirements. We do not have hard information to support this but is a view shared by others within the malaria and wider aid community." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015 - 165
- "The main barrier is there are monitoring staff at [redacted] who are currently expressing some reluctance to see [redacted] money being used for post-distribution check-ups (PDCUs). We have indicated that we are prepared to be flexible on the frequency and sample size, and this is being considered within [redacted]." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- "Our experience over the last several years indicates discussions to reach agreement can be as short as 3 months (Malawi/CU, DRC/IMA/DFID) but can extend to 9-12+ months.
Recent discussions have been continuing for close to 9 months. This may be due to:
- The particular co-funding partner [redacted] involved in all the distributions currently far advanced
- AMF is a new partner (at the scale of millions of nets) with the NMCPs.
- AMF is seeking to introduce elements of accountability that are new to some NMCPs and the co-funding partner.Our experience is when an agreement with a co-funding partner is reached quickly, agreements with the relevant NMCP and distribution partner can follow quickly (within several months)."
Rob Mather, AMF Founder, email to GiveWell, September 9, 2015 - AMF notes that because it is a new partner for the countries it is in discussions with, but it not offering to fill the entire net gap these countries face, it has had to deal with a large amount of slow bureaucracy (e.g. from the NMCPs). Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- 166
- AMF told us that the discussions about the PDCU costs have only delayed an agreement for 1-2 months so far.
- AMF is reluctant to fund PDCU costs because:
- AMF believes that PDCUs provide an opportunity for its co-funding partner to experiment with funding net accountability. If the PDCUs go well, then the partner may be more willing to fund additional monitoring in the future.
- AMF does not have in-country staff to monitor and manage PDCUs, while its co-funding partners often do. AMF thinks that managing the funding of the PDCUs would place a large administrative burden on AMF, whereas the management would be relatively easy for the co-funding partner.
Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015. We do not have a clear sense of why it is more difficult for AMF to manage the funding of PDCUs, but it seems plausible to us that not having staff in-country would make this more difficult.
- 167
- 168
- Rob Mather and Peter Sherratt, conversation with GiveWell, November 6, 2014
- The distributions AMF signed in early 2016 (for 2.7 million nets in Ghana, to be distributed in June-July 2016, and for 10.7 million nets in Uganda, to be distributed August 2016 - March 2017) are the largest distributions AMF has agreed to fund to date. We view the signing of these distributions as evidence supporting AMF's claim that with more funds, it would fund larger distributions, rather than funding a larger number of smaller distributions.
- 169
Rob Mather and Peter Sherratt, conversation with GiveWell, November 6, 2014
- 170
Rob Mather and Peter Sherratt, conversation with GiveWell, November 6, 2014
- 171
Alliance for Malaria Prevention 2015 Q4 Net Mapping Project
- 172
- Alliance for Malaria Prevention 2015 Q4 Net Mapping Project
- See this spreadsheet for our analysis calculating AMF's proportion of all nets distributed.
- 173
- Alliance for Malaria Prevention 2015 Q4 Net Mapping Project
- See this spreadsheet for our analysis calculating AMF's proportion of all nets distributed.
- Note that this means that AMF would provide less than 11% of the funding going to nets, because it does not always pay all non-net costs of distributions it funds.
- 174
AMF provided this information as a comment on a draft of this page.
- 175
- Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- Rob Mather, AMF Founder, conversation with GiveWell, April 13, 2016