The Against Malaria Foundation (AMF) is one of our top-rated charities and we believe that it offers donors an outstanding opportunity to accomplish good with their donations.
More information: What is our evaluation process?
Published: November 2020
What do they do? AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net (LLIN) distributions (for protection against malaria) in developing countries. (More)
Does it work? There is strong evidence that distributing LLINs reduces child mortality and malaria cases. AMF conducts post-distribution surveys of completed distributions to determine whether LLINs have reached their intended destinations and how long they remain in good condition. AMF's post-distribution surveys have generally found positive results but have some methodological limitations. (More)
What do you get for your dollar? We estimate that the cost to purchase and distribute an AMF-funded net is $4.95, or $4.65 excluding in-kind contributions from governments. The numbers of deaths averted and other benefits of distributing LLINs are a function of a number of difficult-to-estimate factors, which we discuss in detail below. (More)
Is there room for more funding? We believe that AMF is likely to be constrained by funding. We estimate that AMF could use up to an additional $37.8 million to support LLIN distributions that will occur between now and late 2023. (More)
- Focus on a program with excellent evidence of effectiveness and cost-effectiveness.
- Processes for ensuring that nets reach their intended recipients and monitoring whether they remain in homes and in good condition over the long-term.
- Room for more funding – we believe AMF will be able to use additional funds to deliver additional nets.
- Transparency – AMF shares significant information about its work with us and we are able to closely follow and understand its work.
Major open questions:
- We expect distributions in the Democratic Republic of the Congo (DRC) to make up a large portion of AMF's funding portfolio in the next few years. As of this writing, we have seen results from followup surveys of distributions in all countries where AMF has supported large-scale distributions, with the exception of DRC. We base our expectation of the impact of future distributions in DRC on results we have seen from other countries, but we are uncertain about the extent to which these results are generalizable to that context. We expect to review results from followup surveys of 2019 distributions in DRC when they become available.
- The best evidence for nets was collected before they were widely used, and there is some evidence that mosquitoes have since adapted to the insecticide used in nets, possibly making them less effective. It seems that insecticide resistance is a growing concern, but it remains difficult to quantify the impact of resistance. We will continue to follow several ongoing studies that may help to quantify the impact of resistance. We discuss this issue in more detail in our page on this topic.
- Our review process
- What do they do?
- Does it work?
- Are LLINs an effective intervention?
- Are AMF's distributions targeted such that they are likely to be effective?
- How do AMF's distributions impact LLIN ownership and usage over time? Do those LLINs remain effective over time?
- Do AMF's distributions increase the total number of LLINs distributed, or would mass distributions of LLINs have taken place in AMF's absence?
- Are there any negative or offsetting impacts?
- What do you get for your dollar?
- Is there room for more funding?
- AMF as an organization
Our review process
We began reviewing AMF in 2009. Our review process has consisted of:
- Reviewing documents AMF made available on its website or shared with us directly.
- Extensive communication, including several meetings at AMF's London headquarters, with AMF Founder Rob Mather and board member Peter Sherratt.
- A visit to AMF's distribution partner organization, United Purpose (formerly Concern Universal), in Malawi in October 2011 (notes and photos from this visit). We also spoke with United Purpose by phone in April 2016.
- A visit to Greater Accra, Ghana in August 2016 to meet with representatives of AMF, AMF's distribution partners Episcopal Relief & Development and Anglican Diocesan Development and Relief Organization (ADDRO), Ghana's National Malaria Control Program, and other non-profit and government organizations involved in the AMF-funded LLIN distributions in Ghana in 2016. Notes and photos from our site visit are available here.
- Conversations with Peter Sherratt, AMF's Executive Chairman; Don de Savigny, a member of AMF's Malaria Advisory Group; and other individuals (who requested to remain anonymous) familiar with AMF's work.
- Conversations with Melanie Renshaw of the African Leaders Malaria Alliance, Marcy Erskine of the International Federation of the Red Cross, and Scott Filler of the Global Fund to Fight AIDS, Tuberculosis, and Malaria about funding needs for nets.
- Conversations with AMF's partner organizations that implement followup surveys of AMF distributions.
All content on AMF, including past reviews, updates, blog posts and conversation notes, is available here. We have also published a page with additional, detailed information on AMF to supplement some of the sections below.
What do they do?
AMF provides long-lasting insecticide-treated nets (for protection against malaria) in bulk to other non-profit organizations or government agencies, which then distribute the nets in developing countries. Through 2019, AMF had supported large-scale distributions in eight countries (DRC, Ghana, Guinea, Malawi, Papua New Guinea, Togo, Uganda, and Zambia), for a total of 41 million LLINs distributed.1 A summary of AMF's distributions can be found in this spreadsheet.
The role of AMF and its partners in LLIN distributions
AMF's role in LLIN distributions is to:2
- Identify countries with funding gaps for LLINs.
- Identify distribution partners (in-country non-profit organizations or government agencies) to carry out LLIN distributions. AMF and its partners agree on expectations for the distribution, including who pays for costs other than the purchase price of LLINs (which are always covered by AMF), the process that will be used to carry out the distribution, and what information will be collected and shared with AMF.
- Identify which net types to use based on malaria prevalence, insecticide resistance, and price data.3
- Purchase LLINs and have them shipped to the distribution partners.
- Work with distribution and independent monitoring partners to ensure that the agreed-upon process for the distribution is implemented, collect reports on the distribution, and conduct follow-up surveys.
Distribution partners implement on-the-ground activities, including registering residents in targeted areas and distributing LLINs. Independent monitors funded by AMF monitor the registration and distribution processes and conduct follow-up surveys.4
Selecting locations for distributions and finding distribution partners
Historically, when selecting locations for distributions, AMF consulted various sources, including the Alliance for Malaria Prevention's (AMP's) list of countries with significant net gaps, other malaria control funders, in-country technical advisors, the relevant national malaria control program (NMCP), implementing organizations, and the African Leaders Malaria Alliance.5 AMF told us that since it started funding larger distributions, it has increasingly been receiving inbound funding requests.6
As AMF investigates countries with existing net gaps, it also looks into organizations working within those countries that could serve as distribution partners.7 AMF looks for distribution partners that have the capacity and willingness to implement registration, distribution, and monitoring processes that meet agreed-upon requirements.8 Through 2019, AMF had worked on large-scale LLIN distributions with United Purpose (formerly Concern Universal) in Malawi, IMA World Health in DRC, Episcopal Relief & Development in Ghana, the Rotary Club of Port Moresby in Papua New Guinea, and NMCPs and their implementing partners in DRC, Ghana, Guinea, Malawi, Togo, Uganda, and Zambia.9
For an example of the process AMF went through to establish the funding gap in Guinea for its 2019 distribution, see Rob Mather, AMF CEO, email to GiveWell explaining Guinea process, October 14, 2018.
Registration and distribution
- Registration: During the registration process, national health system staff or volunteers10 travel door-to-door in targeted areas to collect the information used to determine the number of LLINs to allocate to each household (e.g., the number of sleeping spaces and/or the number of household members), as well as the information used to identify the household for the distribution and post-distribution surveys (e.g., the name of the head of the household and/or household location). The specifics of the registration process and LLIN allocation strategy have differed by country (process details and registration data sources on a separate page with additional details about AMF).
- Distribution: Distribution partners manage the logistics of in-country shipping and storage of LLINs prior to the distribution. To distribute LLINs to recipients, AMF and its distribution partners have primarily used a "point" distribution procedure (LLIN recipients pick up their nets from a specified point in or near their community); they have also occasionally used a "hang-up" distribution procedure (staff or volunteers travel door-to-door to deliver and hang up LLINs). The specifics of distribution processes have varied in the different countries AMF has worked in (details on a separate page with additional details about AMF).
AMF's distribution partners also implement a set of monitoring activities to produce evidence on whether the registration and distribution processes operated as intended and on the long-term impact of the LLIN distribution. Monitoring activities have varied somewhat for different distributions, and AMF has added additional components over time.11 In general, AMF conducts two types of monitoring: 1) process monitoring, whereby an independent partner funded by AMF monitors campaign planning and field activities to ensure they comply with processes required by AMF, and 2) post-distribution monitoring, whereby an independent partner funded by AMF surveys households after distribution.12We have focused on reviewing the monitoring AMF has completed after distributions to understand what proportion of distributed LLINs reached their intended recipients, are used by those recipients, and remain effective while they are in use. See our discussion of AMF's post-distribution monitoring below. In addition, in 2017, GiveWell commissioned a project with IDinsight to better understand the survey methods used in several countries and to provide suggestions for AMF for the future. IDinsight's report with recommendations for the implementation of AMF's post-distribution monitoring is available here; AMF notes that its current monitoring procedures implement the majority of these recommendations.13
AMF occasionally supports malaria control activities beyond the direct distribution of LLINs. For example:
- Funding research on insecticide resistance AMF funded research on the effectiveness of PBO LLINs in conjunction with its Uganda 2017 distribution. PBO LLINs are a newer type of net incorporating piperonyl butoxide (PBO) alongside pyrethroid insecticide used in other LLINs. These nets may be more effective than other LLINs in areas where mosquitoes have developed insecticide resistance. See more details about this research in this blog post. We have seen results from this research but have not reviewed them in depth.14
- Co-funding a "Malaria Unit" in Malawi. From 2016 to 2018, AMF co-funded a "Malaria Unit" in Malawi with United Purpose.15 The Malaria Unit consisted of up to 14 permanent staff members who worked on a variety of malaria control projects: conducting post-distribution surveys, improving malaria case rate data collection practices, monitoring the levels of malaria prevention and treatment supplies at local health centers, developing efficient methods to keep net coverage rates high in between mass distribution campaigns, and more.16 The Malaria Unit assisted with AMF and United Purpose's distributions in Malawi.17 AMF spent about $700,000 over three years on this project.18
The following table shows AMF's total expenditure, categorized into LLIN purchases, other non-net costs (e.g. costs related to distributing nets or conducting post-distribution monitoring), insecticide resistance research costs, and operational costs. We include spending since FY 2012 (July 2011 to June 2012) because this is when AMF shifted to its current model of larger-scale distributions.
|Category||July 2011 to June 2020 spending||Percentage of total spending, FY 2012 to 2020|
|LLIN purchases||$173 million||92.9%|
|Costs of distributing nets and monitoring ("non-net costs")||$7.6 million||4.1%|
|Insecticide resistance research||$3 million||1.6%|
|Salaries and other organizational costs||$2.4 million||1.3%|
|Total spending||$186.2 million||-|
For most distributions, AMF pays for nets and monitoring activities; it partners with organizations—often the Global Fund to Fight AIDS, Tuberculosis, and Malaria—to pay for the other costs of the distribution.20 In the past, AMF also paid for electronic data entry for some paper-based distributions.21
Does it work?
We base our expectation of the impact of AMF's LLIN distributions on:
- the evidence of effectiveness of LLINs,
- characteristics of the populations targeted by AMF's distributions (including malaria incidence and mortality rates and pre-existing LLIN ownership), and
- evidence that AMF's LLINs reach their intended recipients, those recipients use the LLINs, and the LLINs remain effective while they are in use.
This expectation relies on the assumption that AMF's distributions increase the total number of LLINs distributed. To test whether this assumption holds up, we consider evidence about whether LLIN distributions would have occurred in AMF's absence.
Finally, we consider whether there are factors that are not accounted for in the above evidence that would offset or increase the impact of AMF's LLIN distributions, either through reducing their effectiveness or contributing to negative outcomes.
Are LLINs an effective intervention?
In a separate intervention report, we discuss the evidence for the impact of mass distributions of LLINs. We conclude that there is strong evidence from randomized controlled trials (RCTs) that these distributions can be expected to reduce malaria incidence and child mortality. We incorporate the effects measured by these RCTs into our cost-effectiveness model.22
Are AMF's distributions targeted such that they are likely to be effective?
Are LLINs targeted at areas with high rates of malaria?
The impact of LLINs is moderated by rates of malaria incidence and mortality where they are distributed; distributions that target populations affected by higher malaria rates are expected to have higher impact.
At the highest level, AMF appears to exclusively target countries with known malaria risk.23 Going forward, AMF told us that it plans to direct over two thirds of its funding to DRC, the country with the second highest number of malaria cases in the world,24 and where malaria is the leading cause of death.25
We use data on malaria incidence and mortality in the countries where AMF works in our cost-effectiveness model.26 We compare these data to malaria rates in the populations targeted in RCTs, then account for differences in these populations by adjusting the effects measured in RCTs to estimate the impact of LLINs on malaria rates in the countries where AMF works.
Are LLINs targeted at people who do not already have them?
Our expectation of AMF's impact relies on the assumption that AMF's distributions increase the number of people who own usable LLINs. We believe it is likely that AMF's distributions primarily result in people who would not otherwise have usable LLINs receiving them, though it may lead in some cases to households that already own nets receiving more nets than are needed. In our cost-effectiveness model, we make a downward adjustment in our estimate of AMF's impact to account for the possibility of pre-existing nets in recipients' homes.27 Key factors in this assessment include:
- The amount of time elapsed between distributions, as compared to our expectation of how quickly nets become worn out. Distributions are generally scheduled for every three years. Our best guess is that LLINs last, on average, between 2 and 2.5 years.28
- Data collected by AMF during post-distribution monitoring that includes a tally of AMF nets and non-AMF nets found during monitoring. The proportion of AMF nets found in households is often above 95% and is above 85% in all the data we have reviewed.29
- Whether people targeted by AMF's distributions would obtain nets from other sources in the absence of AMF's work. We do not have data on whether people targeted by AMF's distributions obtain nets from other sources; our impression is that mass distributions like those supported by AMF are the primary means by which people obtain nets.30
- AMF's process for allocating nets to households, which involves AMF's distribution partners visiting households and recording how many nets are needed based on the number of people or number of sleeping spaces in each house.31
AMF notes that it interprets the data from PDMs as indicating that LLINs should be replaced at least every three years, and that subtracting usable nets that a household already owns from the calculation of total net need (unless the nets have never been used) increases the period in which a household has to rely on old, ineffective nets until the next LLIN campaign.32 In past distributions in Malawi and DRC, AMF has allocated nets based on the number required per household minus the number of nets already owned by that household. As of June 2018, AMF no longer takes previously-owned nets into account.33
How do AMF's distributions impact LLIN ownership and usage over time? Do those LLINs remain effective over time?
Through its PDMs, AMF collects data at several points in time after a distribution to understand what proportion of distributed LLINs reach their intended recipients, are used by those recipients, and remain effective while they are in use. We rely on results from past distributions to understand the impact we should expect future distributions to have. Our interpretation of these results is informed by their comprehensiveness and the methodology used to collect them. Overall, we believe that these results provide meaningful evidence that a reasonably high proportion of LLINs reach and are used by their intended recipients.
See this spreadsheet for an overview of which PDMs have been completed. In short, we have seen results from all large-scale distributions that occurred from 2012-2018 and from one distribution that occurred in 2019,34 though some PDM rounds have been delayed or skipped. We are thus reasonably confident that we have seen a thorough picture of the impact of AMF's work; we incorporate this assessment into our cost-effectiveness model.35
We focus our review on results from PDMs conducted for distributions from 2017 onward, as we believe they are more likely to be indicative of what we can expect from future distributions. These distributions include Guinea (distribution in 2019), Ghana (2018), Malawi (2018), Togo (2017), Uganda (2017), and Zambia (2017).36 PDMs have also been conducted for the 2017 and 2018 distributions in Papua New Guinea, but we have not reviewed them in depth.37 We have not seen PDMs for the 2019 distributions in DRC and Papua New Guinea, where, as of this writing, distributions had occurred too recently for results to be available.
AMF's PDMs survey households in the areas targeted by a distribution to assess the presence, usage, and condition of LLINs in those households. Typically, PDMs survey a 1.5% sample of households and occur every 9 months for a period of 27 months. As noted above, we have focused on understanding the methodology used in the PDMs of AMF's most recent distributions in Guinea (distribution in 2019), Ghana (2018), Malawi (2018), Togo (2017), Uganda (2017), and Zambia (2017). Full details of those PDMs are in this spreadsheet.
Below, we summarize AMF's general PDM methodology and discuss methodological strengths and weaknesses. Overall, we believe that AMF's PDMs are designed to measure the key indicators of the success of a distribution, and the fact that respondents are selected randomly and LLINs can be directly observed increases the credibility of the results. However, our confidence in the accuracy of these results is reduced by moderate to high sample attrition in several cases, the potential for respondent bias on self-reported measures, and only moderate correspondence in results found by AMF's auditing procedure.
- Respondent selection: AMF selects households to be surveyed during PDMs from the household-level data collected during registration. AMF randomly selects villages and households from all regions or districts that were covered by the distribution of AMF-funded nets. AMF also selects "spare" households that can be visited if households on the original list are unavailable.
AMF's inclusion of all regions or districts and its random sampling of villages and households increase the likelihood that the sample of households it selects is representative of the population that received LLINs. However, AMF's sampling procedure results in households from smaller villages being overrepresented in the sample because in almost all cases, AMF typically selects the same number of households per village.38 Because AMF has data from registration on the total number of households per village, it is possible to correct for this overrepresentation. AMF has shared results from some PDMs that are weighted by village size; we have not reviewed these results in depth. Weighted results show slightly higher (1-5% across PDMs) proportions of LLINs found present and hanging in households compared with unweighted results.39 This suggests that AMF's sampling procedure results in a slight downward bias in results, at least for this outcome.
Another concern is that sample attrition (households randomly selected to be interviewed not being interviewed) may bias results upward, as households that cannot be interviewed during PDMs may also have been less likely to be reached by the distribution; see footnote for AMF's response to this concern.40 In PDMs in Malawi and Zambia, ~60-65% of households selected to be interviewed were interviewed; we believe these figures suggest a high risk of bias in results from those countries. In Guinea, Togo, and Uganda, these figures are ~75-85% of households selected to be interviewed, which we believe suggest a moderate risk of bias.41
- Survey design: AMF contracts with other organizations to implement PDMs ("PDM implementers"). AMF provides PDM implementers with a standard questionnaire,42 which AMF may adapt for each context. Data collectors are instructed to directly assess LLIN presence, hang-up, and condition by observing LLINs in respondents' homes. They also ask respondents to report the number of LLINs received by the household during the distribution, to report nets that are taken down during the day but hung at night, and to answer additional questions about LLIN usage in the previous night.
We believe that some of the responses collected through this survey are at relatively low risk of bias. The presence and condition of LLINs found hanging and reported as taken down during the day but hung at night are objective measures that data collectors can directly assess.43 However, self-reported responses are at greater risk of bias. Responses about LLIN hang-up and usage in the previous night may be at risk of social desirability bias that leads respondents to overreport usage, though validation of these responses is possible through comparison with objective measures.44 Responses about the number of LLINs received by the household report on an event that occurred at least 9 months prior and thus may be at risk of recall bias. AMF has shared results from some PDMs that replace self-reported responses with data on the number of nets that households were registered to receive; we have not reviewed these results in depth. Results using registered LLINs show slightly lower (2-7% across PDMs) proportions of LLINs found present and hanging in households compared with results using self-reported LLINs.45 The direction of these differences is consistent with our expectation that more LLINs are registered than are actually received by households, as net need at times outstrips the number of nets procured, leading to rationing in the distribution.46 The magnitude of this difference suggests that recall bias is not a major concern.
- Survey implementation: The PDM implementer recruits data collectors and supervisors to conduct and oversee data collection. With one exception,47 PDM implementer organizations were not involved in the distribution, which suggests that they are unlikely to have a personal interest in survey outcomes. In multiple cases,48 however, data collectors and/or supervisors hired by the PDM implementer to conduct the survey were involved in the distribution, though in the PDMs they are typically assigned to survey different areas from where they supported the distribution. These individuals may have a personal interest in survey outcomes reflecting favorably on the distribution, such that results are biased upward; we have not seen evidence that this incentive influenced data collection and note it only as a possibility.
AMF selects 5% of surveyed households to be re-surveyed within one to seven days by a different data collector or supervisor who is blinded to the original results. Because data collectors are aware that their work will be audited, this procedure may encourage accurate data collection. It also provides a check on the accuracy of results. The first step in analyzing this data is to pair re-survey records with records from the original data collection using household identification numbers. Overall, AMF has paired around 85% re-survey records with original records in all PDMs.49 AMF told us that records may not be paired for several reasons, most notably because households that were unavailable for original data collection may still be re-surveyed if their availability changes;50 we find this explanation reasonable.
The next step in analyzing this data is to check for correspondence between re-survey records and original records along several data points. We are particularly interested in correspondence levels for the number of LLINs that are present and hanging, which includes both LLINs found hanging by data collectors and LLINs that households report as being taken down during the day but hung at night. Across PDMs, exact correspondence on this number has ranged from 44%-85%. Correspondence within +/- 1 (meaning re-survey records find the exact same number of nets hanging, one higher, or one fewer) has ranged from 78%-92%.51 These correspondence rates are lower than we would have expected, given the seeming objectivity of this measure; they suggest that different data collectors may be assessing this measure differently, that households may be reporting differently on different days, or that these figures reflect a true change in net usage in a matter of days. Regardless of the explanation, these correspondence rates decrease our confidence in the accuracy of original results.
- Data capture: Data collection has been conducted on paper in Uganda and Zambia and conducted electronically in Guinea, Ghana, Malawi, and Togo. We do not know what proportion of data is eventually entered or uploaded into AMF's database, and it seems possible that some data may be lost or discarded;52 because we have data on the proportion of selected households that were actually interviewed, we do not see this as a significant additional concern.53 6% of paper-based data is re-entered by a different data entry clerk to check accuracy: the comparison of second entries with original entries has found rates of matching data above 95% in both Uganda and Zambia.
In the following two sections, we summarize results from the PDMs we have reviewed. Overall, we believe that these results provide meaningful evidence that a reasonably high proportion of LLINs reach and are used by their intended recipients. We incorporate our assessment of the quality of AMF's monitoring methodology into our cost-effectiveness model54 and into our qualitative assessment of AMF's organizational strength.
What proportion of targeted recipients use LLINs over time?
We define LLIN coverage as the proportion of people living in households that were targeted by an AMF distribution who subsequently sleep under LLINs, thereby benefiting from the protective effect against malaria. To understand the coverage achieved by AMF's distributions, we use two types of PDM data: self-reported data from respondents about their usage of all LLINs found hanging in households, and data about the presence of AMF-distributed LLINs in households, from which we can infer usage.
During household interviews, PDM data collectors ask respondents to report their LLIN usage during the previous night by specifying how many household members slept under each LLIN found hanging in the household, including AMF-distributed LLINs and LLINs from other sources.55 This data provides a "reported" coverage rate, defined as the proportion of household members who slept under any hanging net during the previous night. See reported coverage rates for different countries and time points in this spreadsheet, sheet "Results: Summary." In short:
- With one exception, the PDMs we focus on have found reported coverage rates of between 75% and 90%.56
- Reported coverage across time is largely consistent. Across PDMs, average reported coverage is 79% at time points between 6-11 months post-distribution and 81% at time points between 24-30 months post-distribution. In Uganda specifically, results from four time points show notable consistency in reported coverage, with all PDMs finding rates between 77% and 84%.
During household interviews, PDM data collectors also ask respondents how many LLINs their household received from the relevant AMF distribution, then classify each reported AMF LLIN as either present and hanging (which includes both LLINs found hanging by data collectors and LLINs that households report as being taken down during the day but hung at night), present and not hanging, missing, or worn out.57 We use this data to calculate an "inferred" coverage rate representing the coverage provided by AMF-distributed LLINs found hanging. This measure captures a different outcome (coverage from all LLINs versus coverage from AMF-distributed LLINs). See inferred coverage rates for different countries and time points in this spreadsheet, sheet "Results: Summary." In short:
- Inferred coverage rates vary more widely than reported coverage rates, from 55% to 87%.
- Inferred coverage is also slightly less consistent across time than reported coverage. In Uganda specifically, results from four time points show a decrease in inferred coverage over time, from 83% to 72%. This matches our expectation that ownership of usable LLINs decreases over time from the point of distribution, as LLINs wear out, and suggests that households may be replacing AMF-distributed nets with nets from other sources.58
- In almost all cases, reported coverage from all LLINs is higher than inferred coverage from AMF-distributed LLINs, suggesting that households use LLINs from other sources in addition to AMF-distributed LLINs. In Uganda specifically, the difference between reported coverage and inferred coverage grows over time, suggesting that households may be replacing AMF-distributed LLINs with LLINs from other sources as the former wear out.
These two measures are subject to different potential sources of bias. On the one hand, reported coverage is a direct measure that explicitly asks households how they use LLINs, but it may be subject to social desirability bias that leads respondents to overreport usage. On the other hand, inferred coverage is an indirect measure that relies on our assumptions about how households use LLINs, but it uses data that is at somewhat lower risk of bias. We see these data as providing cumulative evidence that, in general, a reasonably high proportion of targeted recipients sleep under LLINs over time.
How long after the distribution do AMF's LLINs remain effective?
Our expectation that LLIN coverage confers a protective effect against malaria requires that LLINs remain effective over time. To understand the effectiveness of AMF's distributed LLINs over time, we review LLIN condition data from AMF's PDMs. To gather this data, PDM data collectors classify each net found hanging in a household as either in "very good," "good," "viable," or "worn out" condition.59 See the proportion of hanging nets in at least "viable" condition for different countries and time points in this spreadsheet, sheet "Results: Summary." In short:
- Across countries, all PDMs conducted between 6 months and 23 months post-distribution have found 90% or more of hanging nets in at least "viable" condition.
- For the distributions that we focus on in this review, we have seen results from two PDMs conducted at 24 months post-distribution, which found slightly lower rates of hanging nets in at least "viable" condition (88% in Togo and 85% in Uganda). We have not yet seen results from PDMs conducted past 24 months post-distribution, when we expect rates of hanging nets in at least "viable" condition to decrease more substantially.
- Across time, the average proportion of nets in at least "viable" condition decreases from 96% to 87%. In Uganda specifically, results from four different time points show a decrease in the proportion of nets in at least "viable" condition from 99% to 85%. This matches our expectation that LLIN condition decreases over time from the point of distribution.
Do AMF's distributions increase the total number of LLINs distributed, or would mass distributions of LLINs have taken place in AMF's absence?
Our expectation of AMF's impact relies on the assumption that AMF's distributions increase the total number of LLINs distributed, compared to what would have happened in AMF's absence. The evidence we have seen suggests that donations to AMF increase the total number of LLINs distributed, but that a portion of the impact is offset by displacing funding from other sources. In our cost-effectiveness model, we make adjustments in our estimate of AMF's impact to account for the likelihood that AMF's funding is displacing funding from other sources.60 Key factors in this assessment include:
- On a separate page, we discuss some cases where AMF was in discussions to fund a distribution, but ultimately did not. In most of these cases, the LLIN funding gap AMF was in discussions to fill persisted for six or more months after AMF's discussions closed, and, in two out of the five cases we looked at, gaps persisted for long periods (18 months and ~3 years, respectively). In most cases, the gap was eventually filled by another funder. As far as we can tell, during the time between AMF withdrawing from discussions and another funder stepping in, the populations targeted for distributions did not receive nets and likely were inadequately protected from malaria.
- We also discuss, on a separate page, what we have been told about what would have happened in the absence of AMF funding in two distributions that AMF did fund. In summary, in both cases our best guess is that there were no other funders who could have closed the gaps, and nets would have been at least partially targeted at higher-risk populations while others would have been left uncovered.
- Across Africa, there are substantial funding gaps for LLINs (more below), and because our impression from following AMF's progress over time is that, due to AMF's more limited funding and, perhaps, greater data requirements, governments often seek funding first from larger funders (particularly the Global Fund to fight AIDS, TB, and Malaria) and then may ask AMF to fill gaps. However, we note that this dynamic may change if AMF has significant resources in the future (more below) and that countries are able to choose how they allocate Global Fund grants among malaria interventions (including LLINs, treatment, and diagnosis), so the availability of funding for LLINs from AMF could cause countries to allocate fewer Global Fund resources to LLINs.61
Are there any negative or offsetting impacts?
In this section, we consider factors that are not accounted for in the above evidence that could offset the impact of AMF's LLIN distributions, either through reducing their effectiveness or contributing to negative outcomes.
- Will insecticide-treated nets continue to be effective? As discussed in our report on insecticide-treated nets, there is strong evidence for the effectiveness of this intervention; however, the best evidence for the intervention was collected before LLINs were widely used and there is some evidence that mosquitoes have since adapted to the insecticide used in LLINs, possibly making them less effective. We have reviewed the evidence on the state of insecticide resistance. We concluded, "Broadly, it seems that insecticide resistance is a larger concern now than it was when we last thoroughly evaluated the evidence in 2012, but it remains difficult to quantify the impact of resistance. Our very rough best guess (methodology described in more detail below) is that insecticide-treated nets (ITNs) are roughly one-third less effective on average across sub-Saharan Africa than they would be in the absence of insecticide resistance. ITNs remain a highly cost-effective intervention after incorporating this discount." We wrote here about recent evidence suggesting that piperonyl butoxide (PBO) nets give additional protection over standard LLINs in areas with insecticide resistance; we also use this recent evidence to inform our estimate of the effects of insecticide resistance in distributions of standard LLINs. AMF expects that a portion of the nets it funds in the future will be PBO nets.62 In our cost-effectiveness model, we make a downward adjustment in our estimate of AMF's impact to account for reduced LLIN efficacy due to insecticide resistance.63
- Do free LLIN distributions distort incentives for recipients or distort local markets for nets? As discussed in our report on insecticide-treated nets, we feel that there is a reasonably strong case for distributing LLINs freely rather than selling them at market (or even below-market) prices. We also think that the benefits of distributing LLINs freely to a population likely outweigh the negative consequences of distortion in local net markets, though we have not factored these potentially negative consequences into our cost-effectiveness analysis of AMF.
- Could distribution of LLINs be inequitable and unfair, causing problems in the targeted communities? We feel that AMF's processes for determining needs for LLINs at the household level are fairly well-suited to ensuring that LLINs are distributed equitably. We have some concerns about whether AMF's process succeeds at identifying all villages or households located outside of villages.
- Does AMF divert skilled labor from other areas? In Malawi, net distributions have been conducted by low-level government health staff in partnership with the staff of AMF's partner NGO.64 AMF's partner in Malawi told us in 2012 that government health staff are normally involved in activities such as disseminating health-related information, reporting on levels of stunting and disease, carrying out immunization campaigns, and providing nutrition support.65 We do not know the extent to which net distribution reduces their ability to complete other duties, though we note that net distributions are generally completed within a few days in each local area, and we would guess that LLIN distributions are likely among the most cost-effective work they engage in.66 Diversion of skilled labor may be more of a concern in DRC, where 22 senior district health staff were employed as Field Supervisors for the Kasaï-Occidental distribution.67 AMF's distributions in Ghana in 2016 were planned by high-level staff from a government health agency and implemented by local government staff and volunteers; monitoring of the distribution was largely implemented by a Ghanaian non-profit organization.68 We have not investigated what other activities the government and non-profit staff and volunteers engage in, or whether the LLIN distribution interferes with their ability to perform other duties.
What do you get for your dollar?
Cost per LLIN distributed
We estimate that on average the total cost to purchase, distribute, and follow up on the distribution of an AMF-funded LLIN is $4.95. Excluding in-kind government contributions, we estimate the cost is $4.65. Country-level cost per LLIN estimates are available in our cost-effectiveness analysis. These estimates rely on a number of uncertain assumptions.
In 2019 and 2020, we updated our analysis of AMF's cost per LLIN but did not publish updated versions of the spreadsheet with the data we used and our calculations because we have not received permission to publish country-specific cost estimates we received from the Global Fund to Fight AIDS, Tuberculosis, and Malaria, which our calculations rely on. We hope to receive permission to publish information on the Global Fund's costs in 2021. Our 2018 cost per LLIN spreadsheet is available here. In 2019 and 2020, we updated the data but used the same assumptions and analysis methods as in 2018.
Below, we also discuss how we estimate the cost per death averted in AMF distributions.
To get the total costs of the program, we attempt to include all partners such that our cost per LLIN represents everything required to deliver the nets. In particular, in our cost per treatment analysis for AMF, we have included these categories in our overall estimate:
- The costs paid by AMF to purchase LLINs. This accounts for 40% of the total cost per LLIN.
- Costs of shipping and delivering nets, monitoring the distribution, and conducting PDMs. In most cases, the Global Fund or another partner pays for most of these costs; in almost every case, AMF has paid for the PDMs. These costs account for 52% of the total.
- Resources contributed by governments, such as staff time, office space, etc. We roughly estimate these costs as a proportion of the total cost (excluding LLIN purchase costs) based on an analysis of a distribution in Malawi in 2012. These costs account for 6% of the total.
- Other AMF costs: staff salaries and other organizational costs, a rough estimate of the value of the CEO's donated time and estimated value of other pro bono support, research AMF has funded on insecticide resistance, and costs of the Malaria Unit in Malawi. These costs account for 2% of the total.
We start with this total cost figure and apply adjustments in our cost-effectiveness analysis to account for cases where we believe the charity's funds have caused other actors to shift funds from a less cost-effective use to a more cost-effective use ("leverage") or from a more cost-effective use to a less cost-effective use ("funging").
We used data and estimates from completed distributions, ongoing distributions, and distributions that AMF has committed to funding in the future.
On a separate page, we compare our AMF estimates of cost per LLIN with a rough global average cost per LLIN.
Shortcomings of our analysis
There are several ways in which our analysis of AMF's cost per LLIN is uncertain:
- The price of an LLIN has been falling and in our estimate we have used a projected price per LLIN, rather than an average of the prices AMF has paid in the past. The average price in distributions AMF funded in 2016-2018 was $2.12. Based on what AMF told us it expects in the future, we have used an estimate of $2.00 per LLIN.
- For distribution costs not paid by AMF, we have generally used rough estimates.69
Cost per death averted
See our most recent cost-effectiveness model for estimates of the cost per death averted through AMF-funded LLIN distributions.
Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. For example, our model does not include the short-term impact of non-fatal cases of malaria prevented on health or productivity, prevention of other mosquito-borne diseases, or reductions in health care costs due to LLINs reducing the number of cases of malaria. It also does not include possible offsetting impacts or other harms. We do include possible developmental impacts on children who sleep under an LLIN.70
There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.
The full details of our cost-effectiveness analysis are in our report on mass distribution of LLINs.
Is there room for more funding?
We believe that AMF could use more funding than it expects to receive to support LLIN distributions in 2023. In short:
- Available funding: As of June 2020, AMF held $112.2 million in funding. It had committed all but $4.2 million of this funding to support future distributions.
- Spending opportunities: AMF has identified opportunities to spend up to $131.8 million, beyond what it has already committed, on LLIN distributions that will occur between now and late 2023. It will need to commit this funding between now and early 2022 to avoid delaying these distributions.
- Expected funding: We project that by early 2022, AMF will have received an additional $90 million to support its work.
In sum, we estimate that AMF could use up to an additional $37.8 million by early 2022 to support LLIN distributions that will occur in 2023. More details and calculations in this spreadsheet. Below, we discuss our approach to assessing AMF's room for more funding.
In general, we assess top charities' funding needs over a three-year period.71 We ask top charities to report their ideal budgets over the next three years, along with information about their current available funding and funding pipeline. The difference between a charity's three-year budget and the funding we project that it will have available to support that budget is the charity's room for more funding.
As of June 2020, AMF held $4.2 million in funding available to support additional work. This figure represents the $112.2 million AMF had in the bank,72 less $108 million it had already committed to future activities, primarily LLIN distributions in DRC and Uganda in 2020 and 2021.
More details and calculations in this spreadsheet, sheet "Available and expected funding (overview)."
AMF has identified opportunities to spend up to $131.8 million on LLIN distributions that will occur between now and late 2023 in four countries: DRC, Guinea, and two countries that AMF has asked us not to identify. See footnote for details on how AMF prioritizes between different spending opportunities.73 AMF has told us that the ideal planning period for supporting an LLIN distribution is 23 months.74 Because AMF has supported distributions in DRC, Guinea, and the two unidentified countries previously, we do not expect that it would require the full planning period to support future distributions. Our best guess is that a planning period of 18 months would be sufficient, suggesting that in order to avoid delaying these distributions, AMF will need to commit funding between now and early 2022.
More details and calculations in this spreadsheet, sheet "Spending opportunities."
We project that AMF will receive an additional $150.8 million to support its work over the next three years, $90 million of which it will receive by early 2022 and will thus have available to commit to the distributions described above. This projection represents our best guess based on past revenue and our understanding of AMF's funding pipeline. It does not attempt to account for any potential impact of the COVID-19 pandemic on future revenue. It excludes any funding we may specifically recommend to AMF, beyond the Maximum Impact Fund grant and our November 2020 recommendation to Open Philanthropy described below.
We include the following sources of funding in our projection:
- Funding currently held by GiveWell to be granted to AMF. We include this amount in our projection of funding available for the next year.
- Funding granted from GiveWell's Maximum Impact Fund. In late 2020, we granted $15.3 million from our Maximum Impact Fund to AMF. We include this amount in our projection of funding available for the next year.
- Funding recommended by GiveWell to be granted by Open Philanthropy.75 In November 2020, we recommended that Open Philanthropy grant $6.7 million to AMF to support distributions that require commitments before August 2021. We include this amount in our projection of funding available for the next year.
- Projected funding due to being a GiveWell top charity. GiveWell maintains a list of all charities that meet our criteria, along with a recommendation for which charity or charities to give to in order to maximize the impact of additional donations. Some donors give based on our top charity list, but do not follow our recommendation for marginal funding. We estimate the amount that AMF will receive from these donors in the next year and roughly project that AMF will receive this amount in each of the next three years.76 This projection assumes that funding of this type is maintained at its high current level across three years; our headline estimate of AMF's expected funding is sensitive to this best guess. The $90 million figure includes only the portion of this funding that we project AMF will receive by early 2022.
- Projected funding independent of GiveWell. We use AMF's annual revenue from the previous year, less the amount we estimate was GiveWell-directed (this includes the funding described above plus funding we specifically recommended to AMF) to estimate the funding that AMF received independent of GiveWell.77 We project that AMF will receive this amount in each of the next three years. The $90 million figure includes only the portion of this funding that we project AMF will receive by early 2022.
More details and calculations in this spreadsheet, sheet "Available and expected funding" sheets.
After applying AMF's available and expected funding to the spending opportunities described above, we estimate that AMF could use up to an additional $37.8 million in funding between now and early 2022 to support the LLIN distributions that will occur in 2023.
Rate of funds moved
When we assessed AMF's room for more funding in late 2019, it held $70 million in funding78 that was earmarked for but not yet formally committed to LLIN distributions. We believed that there was a significant risk that AMF or its partners would decide not to sign the contracts that would commit the funding, as had happened in some past cases. We were thus uncertain about which distributions additional funding would support.
Since then, AMF has formally committed nearly all of the funding it holds. As discussed above, as of June 2020, AMF held $4 million in uncommitted funding.79 We thus believe funding is currently AMF’s primary bottleneck and that additional funding would support the distributions described above.
In addition, we had previously expressed concerns about AMF's ability to scale up.80 We now believe that AMF has a track record of productively spending large amounts of funding annually. It spent between $20 million and $35 million in each of its 2016-2019 fiscal years and $69 million in 2020, up from $4 million or less in each of the prior years.81
Global funding gaps for LLINs
The Global Fund to Fight AIDS, Tuberculosis, and Malaria is the largest funder of LLINs globally. 2020 is the final year of the current Global Fund three-year funding cycle; in 2020, the Global Fund will finalize its country-level allocations for LLINs over the period 2021-23. AMF's expectations for the funding gaps that will exist in this period beyond the Global Fund allocations are informed by its discussions with country governments.
Our own expectations for funding gaps in this period are somewhat informed by data about historical funding gaps, which have been relatively small compared to the total need for LLINs, but larger than the total opportunities that AMF has identified.82 However, there are several reasons to believe that the funding landscape will be different in the upcoming three-year period. First, preliminary data we have seen shows an 30% increase in funding available from the Global Fund for malaria in the 2021-23 period relative to the 2018-20 period,83 suggesting that gaps will be smaller in this round than they were previously. On the other hand, malaria spending per capita has decreased84 and AMF projects that the costs of LLIN distributions will increase by 23% in the 2021-23 period relative to the 2018-20 period, due to population growth, inflation, and the purchasing of a higher proportion of PBO nets, which are more expensive than standard nets.85 In addition, based on evidence from its PDMs, AMF believes that LLINs should be distributed every 30 months,86 rather than every 36 months as has been standard practice, and it has been in discussions with its partners in DRC about decreasing the interval between distributions. If this change is implemented in DRC and potentially other countries longer-term, this would increase the number of distributions that occur in each three-year period. Such changes would increase the total cost of LLIN distributions over the upcoming three-year period, thereby increasing the global room for more funding for LLINs.
What portion of all net distributions are funded by AMF?
This section is out of date.
In 2016, 2017, and the first three quarters of 2018, approximately 473 million nets were distributed in sub-Saharan Africa. 54% of these nets were funded by the Global Fund, 25% were funded by the President's Malaria Initiative, and 8% were funded by UNICEF.87
AMF's contributions accounted for 15.5 million nets in 2016-2018 (this is less than the number of nets that AMF purchased, which was 30.9 million; we have made an adjustment to account for the fact that AMF contributes the nets while other funders pay many of the other costs).88 Therefore, AMF's contribution during this period was about 3% of all nets distributed.89
AMF as an organization
We use qualitative assessments of our top charities to inform our funding recommendations. See this page for more information about this process and for our qualitative assessment of AMF as an organization.
See our Summary of AMF Distributions spreadsheet, sheet "Summary", section “AMF distributions by country."
This understanding is based on many conversations with AMF, and from following AMF's progress over time.
Rob Mather, AMF CEO, comment on a draft of this review, October 31, 2019.
This understanding is based on many conversations with AMF and its distribution partners, and from following AMF's progress over time.
- "This is a list of the countries with known gaps and where there are significant contiguous areas without nets, or a significant percentage required, and for which the estimate of need is believed to be reasonably accurate. It does not include countries where there are gaps, typically up to 40% of what the nation needs, but they are spread more uniformly across the country and would therefore require an ‘in-fill campaign’. An in-fill campaign is different from a so-called ‘universal coverage campaign’ because the percentage installed base of nets is higher in the former case and so a pre-distribution registration survey (PDRS) is an absolute requirement to ensure an efficient allocation of nets. Our methodology would lend itself to these campaigns if the relevant National Malaria Control Programme (NMCP) were to embrace a detailed PDRS. The list does not include, in our view, other countries where the need has not yet been quantified. Given there are many countries with needs estimated, we have not chosen to seek out other countries in need of nets. Our assumption is groups like AMP will be a source of reporting on additional countries as quantified needs emerge." Rob Mather, AMF Founder, email to GiveWell, August 8, 2012.
- In September 2015, we checked in with AMF about its process for determining in which countries it works. Rob Mather noted that AMF continues to review malaria prevalence data (where that data exists), although maintains a level of skepticism about that data given that it can be unreliable. Even so, AMF feels comfortable drawing conclusions about which countries have high malaria mortality burdens based on the data it sees and the conversations that it engages in. AMF checks the following sources to keep up-to-date on which countries have a significant malaria burden:
- The Alliance for Malaria Prevention, which sends out a weekly email with malaria-related data and information.
- The African Leaders Malaria Alliance (ALMA), which AMF has conversations with on a quarterly basis.
- In-country partners, who frequently attend malaria task force meetings and have recent news
- Members of malaria advisory groups
- Other connections
Before AMF decides to approach a country to offer funding for nets, it has many conversations to confirm the level of need that country has with the other actors that are working on malaria in the country. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
- In early 2016, AMF described a location-selection process similar to that described previously: "AMF learns about net gaps and receives funding requests through its network in the malaria control community, particularly through the Alliance for Malaria Prevention and the African Leaders Malaria Alliance." Rob Mather and Peter Sherratt, conversation with GiveWell, February 28, 2016
"As it becomes involved in larger distributions, AMF is receiving a growing number of funding requests. As its funding increases, AMF aims to make more strategic investments by engaging in the planning cycles of countries where it has strong connections and experience." Rob Mather and Peter Sherratt, conversation with GiveWell, February 28, 2016
- Ghana's National Malaria Control Program, one of AMF's distribution partners for the June-July 2016 distribution campaign in Ghana, described negotiating and compromising with AMF on requirements for the distribution:
- "The NMCP has been working with AMF for a relatively short period of time. Their working relationship has proceeded relatively smoothly thus far, especially since AMF has shown willingness to negotiate and compromise on some areas to conform with the country's specific scenario." GiveWell's non-verbatim summary of a conversation with Ghana's National Malaria Control Program, August 16-18, 2016
See our Summary of AMF Distributions spreadsheet, sheet "Summary," section "AMF distribution partners."
AMF told us that registration was previously carried out by AMF's distribution partners, but is now increasingly carried out by national health system staff. AMF's distribution partners continue to carry out monitoring of the registration process.
Rob Mather, AMF CEO, comment on a draft of this review, November 12, 2018.
We describe these processes (as of 2016) in more detail on our page with additional information on AMF.
AMF, comments on a draft of this page, October 2020.
AMF, comments on a draft of this page, October 2020.
Published results can be found here.
- "We are close to concluding discussion with Concern Universal for AMF and CU to co-fund a Malaria Unit (a larger team of people than currently in place dedicated to malaria control related work) in Malawi led by a senior CU Manager and employing 12 Malawians (10 additional hires)." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
- AMF funds status (October 2017) Redacted
"The aim of this unit will be to provide more resource to allow 1. improved efficiency in managing the four contracted three-yearly distributions and associated work and 2. allow us to together innovate and develop additional malaria control support for the four districts and the NMCP, specifically: i) intended close liaising with Health Centre re malaria case data and elements related to the monitoring/recording of malaria data i.e. stock levels of RDTKs, rubber gloves and other diagnosis equipment; qualified staff able to test for malaria, presence at the clinic so diagnosis can happen; systems, capacities etc) ii) research (data), discussion and involvement in ways of ensuring 80% sleeping space coverage throughout the three year net-distribution cycle, including investigation of ‘injection strategy’ net distributions involving mini-mass distributions at two years post-distribution and in the subsequent distribution cycle at one-year post distribution." Rob Mather, AMF Founder, email to GiveWell, September 9, 2015
The distributions and post-distribution monitoring occur frequently enough that having a trained, consistent staff to manage them is worthwhile. Rob Mather and Peter Sherratt, conversation with GiveWell, September 9, 2015
AMF spent $689,104 over three years on the project. Rob Mather, AMF CEO, comment on a draft of this review, October 31, 2019.
For calculations, see this spreadsheet.
- Rob Mather, AMF CEO, comment on a draft of this review, October 31, 2019.
- "There was an exception to this between 2013 and 2015 in Malawi where AMF funded the $1.1 million costs of the distribution of 1.4 million nets in order not to miss a window of opportunity to distribute nets given no other funder could be found for the no-net costs and the rainy season was arriving that would make a net distribution difficult." Rob Mather, AMF CEO, comment on a draft of this review, October 31, 2019.
- See this spreadsheet, “(Added by GW) Detailed Overview” sheet, for details on what costs AMF has paid for each distribution.
- "The Ghana distribution is now going ahead: AMF is funding the costs of the nets and AMF’s additional monitoring costs, and the Global Fund will fund other non-net costs." Rob Mather and Peter Sherratt, conversation with GiveWell, February 11, 2016, pg. 2.
- "The non-net costs will be funded by the Ghana Malaria Global Fund Grant. These costs include those for shipping to Ghana, clearance, in-country transport, pre-distribution, distribution." AMF Ghana 2016 distribution agreement, pg. 1.
- "The non-net costs will be funded by the Uganda Ministry of Health which may use funding from its Roll Back Malaria Partners. These costs include those for shipping to Uganda, clearance, in-country transport, pre-distribution, and distribution." AMF Uganda 2016 distribution agreement, pg 2.
- "Post-Distribution Check-Ups… AMF will fund an NGO to run the process." AMF Uganda 2016 distribution agreement, pg 5.
- AMF Togo 2017 distribution agreement Redacted:
- "1) AMF will fund 2,413,250 LLINs.
2) The non-net costs will be funded by the MSPS which may use funding from The Global Fund or other sources." Pg 7.
- "Post-Distribution Check-Ups (PDCUs) will take place across all of the districts to monitor net use and condition. The results will be owned by Togo and shared with AMF. AMF will fund an NGO to run the process in full consultation with the MSPS. The MSPS will facilitate the check-ups." Pg 10.
- "1) AMF will fund 2,413,250 LLINs.
- AMF Papua New Guinea 2017 distribution agreement Redacted:
- "1) AMF will fund 1,159,400 extra-large LLINs for distribution in 2017.
2) The costs, with the exception of the purchase of the LLINs, will be borne by RCPM which may use dedicated funding from The Global Fund and other sources." Pg 1.
- "Post-Distribution Check-Ups (PDCUs) to assess correct net distribution, net use and condition of LLINs will take place across all of the districts to monitor net use and condition. The results will be owned by PNG but will be shared with AMF. AMF will fund a yet-to-be-decided NGO to run the process in full consultation with the NDoH and RCPM. RCPM will facilitate the check-ups." Pg 4.
- "1) AMF will fund 1,159,400 extra-large LLINs for distribution in 2017.
AMF, comments on a draft of this page, October 2020.
See the most recent version of the model here.
"Over 50% of all cases globally were accounted for by Nigeria (25%), followed by the Democratic Republic of the Congo (12%), Uganda (5%), and Côte d'Ivoire, Mozambique and Niger (4% each)." Pg. 6 of this report.
See section "What causes the most deaths?" of this page.
See the most recent version of the model here.
See the most recent version of the model here, line "Pre-existing net adjustment."
See the "AMF vs. non-AMF nets" tab of this spreadsheet.
- We discuss the evidence for whether households purchase LLINs in the private market on this page. Note that this evidence is largely from before donors scaled up mass LLIN distributions, and it is possible that there is now more willingness to pay for LLINs given greater experience with the benefits of LLINs and/or, perhaps, lower cost of LLINs or higher incomes from economic growth. Our guess is that mass campaigns generally create the expectation of free LLINs and decrease households' willingness to purchase them.
- In the notes from our site visit to Ghana in 2016, we noted, "Everyone we talked to had previously owned bed nets before this distribution. Most had been procured through the area's last mass distribution four years ago, but some had been purchased or received from a clinic more recently." Pg 17.
Discussion of the registration process used in each country is on our page with additional, detailed information on AMF.
Comment provided in response to a draft of this page in November 2017.
Rob Mather, CEO, AMF, conversation with GiveWell, June 18, 2018.
As of this writing in June 2020, we do not yet expect to have received results from the other distributions that occured in 2019.
See the most recent version of the model here, line "Misappropriation without monitoring results."
- As of June 2020, we have seen the following results from each country:
- Guinea (2019): Scheduled PDMs have been completed. We have seen results from the 9-month PDM.
- Ghana (2018): Scheduled PDMs have been completed. We have seen results from the 9-month PDM. Results from the 18-month PDM were expected but were not yet available.
- Malawi (2018): Scheduled PDMs have been completed. We have seen results from the 9-month PDM. Results from the 18-month PDM were expected but were not yet available.
- Togo (2017): PDMs were delayed and were not conducted until 18 months post-distribution. We have seen results from the 18-month and 24-month PDMs.
- Uganda (2017): Scheduled PDMs have been completed. We have seen results from the 6-month, 9-month, 12-month, 18-month, and 24-month PDMs. Results from the 30-month PDM were expected but were not yet available.
- Zambia (2017): AMF rescheduled the 9-month PDM to be conducted at 12 months post-distribution. We have seen results from the 12-month PDM. Results from the 18-month PDM were expected but were not yet available.
- As of June 2020, we have seen the following results from each country:
- For the following reasons, we expect PDMs from Papua New Guinea to be less indicative of what we can expect from future distributions and have chosen not to review them in depth:
- The distributions AMF has supported in Papua New Guinea are relatively small compared to its other distributions. See this spreadsheet, sheet "Detailed overview," column C.
- AMF does not plan to support distributions in Papua New Guinea in 2021 or 2022. See this spreadsheet, sheet "Spending opportunities."
- We include results from these PDMs in this spreadsheet but exclude their methodology.
- For the following reasons, we expect PDMs from Papua New Guinea to be less indicative of what we can expect from future distributions and have chosen not to review them in depth:
See cells C4 on the "Methods" sheets in this spreadsheet. While smaller villages are overrepresented, the smallest villages are excluded from the sample: AMF excludes from selection any villages with fewer than a threshold number of households. In practice, this has led ~5% or less of households to be excluded from the sample, with the exception of Togo, where 9.8% of households were excluded from the sample.
See this spreadsheet, sheet "Source: Allocated vs Distributed + Weightings," cells K10-19. We have not yet incorporated this analysis into this review, which reports unweighted results.
"Below we have shared a summary of our thinking about why households may not be interviewed, and how that relates to the likelihood that the household received nets at a distribution point.
Households not available for PDMs for the following reasons would not be systematically more or less likely to have been reached by the distribution:
- Those that have moved or cannot be located in the village
- Those that are away visiting family
- Those at the market/ making social visits
Households not available for PDMs for the following reasons may be less likely to have been reached, with caveats:
- Those where HH members were at work during the day. If all HH members work every day, it may be that nobody was able to go to the distribution point to collect nets. However, NMCDs usually ensure that distribution points are open for multiple days, and early/late, to mitigate against this risk.
- Those where the only HH member has passed away. If the sole household member was elderly, it is possible that s/he was less likely to have travelled to the distribution point to collect a net.
If for any reason a particular village was less likely to be reached by the distribution (e.g. too far from the distribution point, or did not receive word about the date), then we would expect to see evidence of this in the households that we were able to survey." AMF, comments on a draft of this page, October 2020.
See cells D6 on the "Methods" sheets in this spreadsheet.
See example here: AMF, PDM data collection form, Uganda
AMF told us that data collectors are instructed to directly observe and assess the condition of all LLINs that are present and hanging, which includes both LLINs found hanging by data collectors and LLINs that households report as being taken down during the day but hung at night. Against Malaria Foundation, conversation with GiveWell, August 6, 2020.
"For our recent PDMs where data is collected electronically we have put in place data validation to mitigate against this. We ask partners to validate the information in Question 4 (self-reported responses about hang-up) with the objective measures from Question 3 (LLINs found hanging and or in use at night)." AMF, comments on a draft of this page, October 2020.
For example: "This difference occurs as households report fewer nets received than registered (e.g. 20 nets hung divided by 25 self-reported nets = 80%, versus 20 nets hung divided by 26 registered nets = 77%." AMF, comments on a draft of this page, October 2020.
AMF uses self-reported nets received as the denominator in its net presence results (see this spreadsheet, sheet "Results: Net presence"). Because AMF has data on the number of nets that households were registered to receive, it is possible to replace the self-reported figures with registration data. See this spreadsheet, sheet "Source: Allocated vs Distributed + Weightings," cells D10-19. We have not yet incorporated this analysis into this review, which reports results that use self-reported nets as their denominator.
Separately, AMF has compared distribution data on LLINs received and self-reported PDM data on LLINs received. It found that across countries, the proportion of household records that agreed on this figure was 48% at 6 months and decreased to 37% at 30 months. Other findings included: "1. The % of households that report a number of nets within +/-1 of the number recorded during distribution is >70% at all time points. 2. The % of households that report the same number of nets as recorded in the distribution data reduces over time. 3. The % of households that report fewer nets received than the distribution data indicates increases over time." Against Malaria Foundation, responses to GiveWell questions, May 15, 2020.
AMF, comments on a draft of this page, October 2020.
For the 2017 distribution in Togo, the PDM implementer is a branch of Togo's Ministry of Health and was involved in the distribution. See this spreadsheet, sheet "Methods: Togo (2017)," cell C10.
The cases are Malawi 2018, Togo 2017, Uganda 2017, and Zambia 2017. See cells C10 on the "Methods" sheets in this spreadsheet.
See cells C9 on the "Methods" sheets in this spreadsheet. About 80% of records were paired for Zambia, Uganda, Togo, and Malawi, and 90-95% were paired for Ghana and Guinea. We have summarized this as roughly 85% overall.
"The most common reason, accounting for ~2/3 of instances where we cannot match a re-visit record, is that the household was on the list for both main and re-visit surveys but was only surveyed for the re-visit.
This can occur if a household is unavailable on the day of the main data collection, prompting the use of a spare, but available during the re-visit. In this scenario, there is a re-visit record but no main record to match.
To maximise the number of matching pairs, we could select the re-visit households from the main households surveyed, rather than the main households selected for the household lists. However, to do this we would have to receive, import and select re-visit households in a narrow window of time between main and re-visit data collection. Given the AMF time that would be required and the importance of prompt re-visit data collection, we decided that it would not be practical to make this part of our methodology." Against Malaria Foundation, responses to GiveWell questions, June 23, 2020.
AMF has also provided other reasons that records may not be paired: "Other reasons that account for the remaining ~1/3 of re-visit records that cannot be matched are:
- Household ID recorded incorrectly: Where IDs are entered by data collectors/ clerks, an element of human error will mean there are instances where the main or re-visit ID is incorrect and therefore cannot be matched.
- Duplicate household IDs: Errors can have knock-on effects if they lead to multiple main households with the same ID within a village. Any re-visit record with this ID will remain unmatched, as there is no way to automatically determine which main household should be used for a matching pair.
- Data entered and imported to the wrong part of the DES:As mentioned in our discussions about registration data and matching, our standard code only matches households within a village, so any main or re-visit data entered/imported into the wrong part of the DES will inhibit matching. However, this situation is less likely during PDMs as the locations are exported initially from the DES.
- Data erroneously entered and imported as re-visit records:For Malawi PDM-09, main records were erroneously entered and imported as re-visit records, and as these households were only visited once, the code could not find a matching pair in the main data. These data have are being reimported and the error corrected. See next question for more details.
For electronic data collection, household lists are uploaded directly to the data collection tool, minimising the chance of reasons 3 and 4 occurring." Against Malaria Foundation, responses to GiveWell questions, June 23, 2020.
See cells C9 on the "Methods" sheets in this spreadsheet
In 2017, GiveWell supported an assessment of AMF's monitoring by IDinsight. In the course of this assessment, IDinsight observed "Incomplete or inconsistent forms were being discarded in Ghana during data entry with no clear record of why" and recommended "AMF should require country partners to mandate that supervisors frequently collect all PDM forms, regardless of completion. All complete forms should be entered into the DES." IDinsight, Recommendations for Post Distribution Monitoring Implementation, pg. 7.
AMF told us that since this assessment, it has started to require that supervisors collect PDM forms or electronic data collection devices at least twice a week: "Requirement to collect all PDM forms daily or twice a week added to SO...For electronic data collection, supervisors collect the devices in order to submit the data." AMF, Implementation of IDinsight recommendations, 2020, pg. 1. Our understanding is that in general, supervisors review data for errors and may correct smaller errors or instruct data collectors to re-do household interviews if major errors are found. We do not know the details of these procedures or how often errors are found. Our understanding is that household interviews are re-done very rarely. See cells C13 on the "Methods" sheets in this spreadsheet.
In other words, we expect both 1) households randomly selected to be interviewed but not interviewed and 2) households that were interviewed but whose data was lost or discarded to be captured in the sample attrition data we report above.
See the most recent version of the model here, line "False monitoring results."
"Please complete the following table for all nets found hanging in the household...How many slept under this net last night?" AMF, PDM data collection form, Uganda.
"Households will also have other nets, for example received through routine distribution, which may be used to cover people at night but will not be counted in the presence tables...Information about these nets is captured in Question 3 on our PDM form and is used to calculate our reported sleeping space coverage and people coverage." Against Malaria Foundation, responses to GiveWell questions, June 23, 2020.
The one exception is the PDM in Zambia conducted 12-15 months post-distribution, which found a relatively low coverage rate of 68%. AMF told us that the Zambia distribution had insufficient LLINs, and thus the number of LLINs that could be distributed to households was capped (Against Malaria Foundation, conversation with GiveWell, November 20, 2019). If households did not receive enough LLINs to cover all sleeping spaces, this may have contributed to the relatively low coverage rate measured in Zambia.
- See question 4 in AMF, PDM data collection form, Uganda.
- See full results in this spreadsheet, sheet "Results: Net presence."
- Other potential sources of evidence for LLIN presence are AMF's household-level registration and distribution data. As described above, registration data shows the number of LLINs that were allocated to each household during a pre-distribution household census, while distribution data shows the number of LLINs actually distributed to each household during the distribution. AMF has collected both types of data for all large-scale distributions it has supported. To date, we have preferred to use PDM evidence to understand AMF's impact, as we believe that the informativeness of AMF's registration and distribution data is limited. This is because:
- Registration data captures LLINs allocated, rather than LLINs received. It thus provides only indirect evidence for LLIN ownership.
- Distribution data provides direct evidence for LLIN ownership, but unlike PDMs, it captures only one point in time. Additionally, AMF told us that it has fewer procedures in place to ensure high quality in distribution data than it does for registration data and PDM data. (Against Malaria Foundation, conversation with GiveWell, June 4, 2020.) We are thus less confident in the reliability of this data.
"This suggests that households do replace AMF-distributed LLINs with LLINs from other sources, but this is not as common." AMF, comments on a draft of this page, October 2020.
fewer than 2 holes of less than 2 cm each
fewer than 10 small holes
fewer than 20 small holes and no holes bigger than 20 cm." AMF, PDM data collection form, Uganda.
See the most recent version of the model here, section "Leverage/Funging adjustment."
This is an understanding formed over many conversations.
Rob Mather and Peter Sherratt, AMF Founder and Executive Chairman, conversation with GiveWell, June 18, 2018
See the most recent version of the model here, line "Efficacy reduction due to insecticide resistance."
"Health Surveillance Assistants (HSAs) are Government extension workers- they are the lowest tier of government presence in the decentralized health system." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.
"As such they are the first line of response to any public health issues in communities. Their job involves disseminating health related information (such as encouraging people to make use of sanitary facilities, go for immunizations, sleep under mosquito nets etc.), carrying out sanitation and hygiene campaigns and sending data on take-up of facilities to the District Council, conducting basic nutrition support, weighing children and reporting levels of stunting and wasting, detecting common communicable diseases and reporting these to clinicians and other health providers, implementing immunization campaigns etc. As you can see being involved in universal net distribution fits very well with their core public health responsibilities. HSAs need to have a primary school completion certificate as a minimum but the majority of them will have O-Levels (exams sat by pupils aged 16 if they have completed the school system at the recommended pace). Once they have been selected as HSAs they are sent on an initial 9 months intensive training course where they will be trained in many aspects of public health including how to recognize common diseases, how to administer immunizations etc." Robin Todd, Concern Universal Malawi Director, email to GiveWell, April 27, 2012.
- Pre-distribution registration surveys appear to be completed relatively quickly: "Approximately 480 personnel will be involved in the PDRS, with the majority, some 460, involved for 5-7 days over the data collection period." Concern Universal, Dowa 2015 planning document, pg. 4.
- In the Ntcheu 2012 distribution, the verification of PDRS data took several weeks: "The verification process took the verification team of 10 members 18 days to complete and in a day 20 clusters were verified with 10 verification sites in the morning and 10 verification sites in the afternoon." Concern Universal Ntcheu 2012 distribution report, pg. 7.
- In the Ntcheu 2012 distribution, distributions were scheduled for several weeks, but covered approximately 10 "clusters" per day: "The distributions were scheduled to have been concluded within 28 days with the team distributing at 10 clusters per day covering five weeks." Concern Universal Ntcheu 2012 distribution report, pgs. 7-8.
- United Purpose has recently started to use fewer staff for post-distribution check ups, which causes them to take somewhat longer: "However, in collaboration with the District Environmental Health Office (DEHO) and Malaria Coordinator (MC) and lessons lea[r]nt from 24 month Ntcheu PDCU, it was recommended to have a focused team of 10 data collectors rather than have the HSAs as data collectors from each HCA. This was based on the following reasons. First, this would reduce the number of data collectors that would need to be monitored and trained. Second, we would be able to select reliable individuals whom we could trust to do a diligent and accurate job of collecting the data. Third, it would leave the majority of HSAs to carry on with the normal health tasks and duties. Fourth, by having the same people covering the whole exercise they will get acquainted to the task and reduce errors on data collection. This meant the data collectors would spend thirty seven days collecting data rather than the one or several days if many more data collectors were to be used. This was judged the preferable way of organising and managing the data collection phase." Concern Universal Ntcheu 2012 33-month post-distribution check-up report, Pg 5. Other recent post-distribution monitoring reports have similar language.
"i) Field Supervisors (FSs)
22 FSs were selected from permanent and senior health staff in Tshikapa Health District.
ii) Community Health Workers (CHWs) – data collectors
Each of the 22 FSs had the responsibility of recruiting, in each HA they were designated, enough CHWs to gather household data and hang nets. Two primary recruitment criteria were literacy and familiarity with using a mobile phone. The number of CHWs recruited depended on the size of the HA and the number of households to be visited. The aim was to recruit enough CHWs to carry out the entire registration and hang‐up, once it commenced, in a five day period. Between 20 and 40 CHWs were recruited by each FS for a total of 4,000 CHWs across the 8 HZs (8 HZs x 20 HAs x 25 CHWs per HA = 4,000 CHWs)." IMA World Health, Kasaï-Occidental 2014 distribution report, pg. 13.
- "In the present distribution, core responsibilities of Episcopal Relief & Development and ADDRO include:
- Monitoring the NMCP-led pre-distribution and distribution activities, and providing feedback to the NMCP. NMCP also does its own monitoring.
- Post-distribution monitoring." Pg. 4.
- "Phase 1 – Planning and registration
The registration phase was implemented by the government and is now complete in all three regions.
Step 1 – Informative Meetings with Ghana Health Service Regional Health Directorates
At these meetings, stakeholders discuss the LLIN distribution implementation model, the schedule of activities, and budgets. Stakeholders also agree on a date for the regional planning workshop.
Step 2 – Regional planning workshops
At these workshops, stakeholders discuss the registration and distribution processes, budgets, rules, and responsibilities for different groups. In the Greater Accra regional planning workshop, the stakeholders also discussed which households to target (in other regions, all households were targeted)." Pg. 4.
- "GHS volunteers carry out the registration process, which takes place more than one month before the distribution. Volunteers are organized by the Ministry of Health (MoH) and participate in a number of government health programs, such as vaccination campaigns (immunization days)." Pg. 5.
- " Steps in the distribution process
1. Beneficiaries walk a short distance to their distribution point. One staff member employed by GHS and at least one GHS volunteer are stationed at each distribution point; there are never just two GHS volunteers." Pg. 8.
- "In the present distribution, core responsibilities of Episcopal Relief & Development and ADDRO include:
- Our estimate of the value of in-kind government contributions for all AMF distributions is based on an analysis of a single distribution in Malawi in 2012. See cell B34 on the "Summary" sheet of our 2018 cost per net analysis here.
- Our estimate of the Global Fund and other philanthropic funders’ non-net costs per net relies on a variety of different sources which we have not received permission to publish:
- In our 2018 cost per net analysis, we published a weighted average of the Global Fund and other philanthropic funders’ non-net costs per net (without publishing information on country-level costs). See cell B33 on the "Summary" sheet of our 2018 cost per net analysis here.
- In 2019 and 2020, we decided to prioritize publishing country-level cost per net estimates in our cost-effectiveness analysis (CEA) over publishing an updated version of our cost per net spreadsheet. We made this decision in order to be able to report country-level cost-effectiveness estimates in our public CEA without revealing information from the Global Fund which we do not have permission to publish.
See our most recent model, "Bednets" and "Results" sheets.
For a discussion of why we consider funding a charity's work up to three years in the future, see this blog post.
Some of our top charities have a policy of holding funding reserves. In our room for more funding analyses, we include reserved funding as funding available to support program activities. We do this both to ensure consistency across top charities (as not all top charities hold reserves) and to understand the true effect of granting additional funding (i.e. whether additional funding would support undertaking additional program activities versus building or maintaining reserves).
"AMF prioritises among spending opportunities by considering the following factors, among others:
1. Disease burden, including malaria prevalence, expected cost effectiveness and prospective disease burden (difference in disease with/without AMF)
2. Country finance, including the urgency of the request, the funding gap, and the country’s GDP
3. AMF specifics, including ease to work with NMCP and partners, availability of good partners for monitoring and potential benefit for country of AMF process." AMF, comments on a draft of this page, October 2020.
More details in this blog post this blog post.
Open Philanthropy, a philanthropic organization with which we work closely, is the largest single funder of our top charities. The vast majority of Open Philanthropy's current giving comes from Good Ventures. We make recommendations to Open Philanthropy each year for how much funding to provide to our top charities and how to allocate that funding among them.
In our projections of future funding, we typically count only one year of funding that an organization receives as a result of being on our list of top charities in order to retain the flexibility to change our recommendations in future years. In this case, however, our best guess is instead that AMF should expect to receive roughly this amount in each of the next three years, based on our impression that many GiveWell-influenced donors choose to donate to AMF regardless of our current recommendation for marginal funding.
We make adjustments to this amount based on what AMF has told us about how it expects its revenue to change in the future, for example, large grants that are not being renewed and prospects for new major sources of funding.
See this spreadsheet, sheet "Funding commitments," cell B5.
See this spreadsheet, sheet "Available and expected funding (overview)," cell B11.
- In November 2013, we wrote in this blog post that we believed AMF did not have room for more funding because it had raised over $10.6 million due to GiveWell's recommendation since 2011 and had not spent the bulk of these funds.
- In 2014, AMF completed and signed agreements for several large distributions, increasing our confidence that it could productively use the donated funds it held. By the end of 2014, AMF had spent or committed a large portion of the funds it raised prior to 2014. As of November 2014, AMF held $4.9 million in uncommitted funds, of which it raised $2.65 million in 2014. Of committed funds, $1.9 million were for expenses that AMF would incur more than a year later, and AMF was considering reallocating these funds to the nearer-term with the expectation of raising enough to cover its later commitments by the time they were due. This reallocation brought AMF's total available funds to $6.8 million. Rob Mather, AMF Founder, email to GiveWell, November 26, 2014
- In 2015, AMF did not sign any new distribution agreements for the near future, although it signed agreements to replace nets it had previously funded in Malawi in 2018. AMF: "US$6m commitment to malaria control support in Malawi in 2018"
See this spreadsheet for more detail.
According to data provided by Africa Leaders Malaria Alliance, funding gaps for LLINs in Sub-Saharan Africa represented about 16% of the total need for LLINs in 2019, 13% in 2018, 2% in 2017, and 29% in 2016.
Figures are from this spreadsheet, sheet "Nov 2019 LLIN gap estimates," cell J62, sheet "Forecast analysis," cell R50, and sheet "Historic analysis," cells C48 and E48 (note that some of these figures show the proportion of LLINs funded; here we report the proportion of LLINs unfunded).
See this spreadsheet, cell C71.
"We feel it’s important for this section to include the fact that per capita spending on malaria has fallen in recent years." AMF, comments on a draft of this page, October 2020.
"Taking a distribution of 10m nets and factoring in the lower World Bank population increase (+10%), an example need for new nets of 40% and inflation for the repeat distribution gives the following.
2018-2020 mass distribution with all standard nets
$1.80 per net and $2 per non-net cost
Total cost of distribution: $38 million.
2021-2023 repeat distribution:
- 40% PBOs at $2.6 per net (standard net price plus $0.8 delta)
- 10% population increase
- Inflation of 1% per year in USD
Total cost of distribution: $47 million.
The impact of taking these considerations into account is a 23% increase in the cost of the campaign." AMF, Response to GiveWell on risk of crowdout, October 2020.
This is our understanding from multiple conversations with AMF.
See this spreadsheet, sheet "(Added by GW) Summary," section “AMF distribution by year.” Though AMF purchased more than 15.5 million nets, because it does not pay for other distribution costs, in effect, it funded 15.5 million nets for distributions occurring in 2016-2018.
In 2016-2018 (assuming 75% of the nets that will be delivered in 2018 were delivered in the first 3 quarters), 517 million nets were delivered. 15.5 million nets is 3.0% of this. Alliance for Malaria Prevention 2018 Q3 Net Mapping Project