- Top charities
Published: September 2012
In early April 2012, AMF’s distribution partner in Malawi, Concern Universal, completed its distribution of about 268,000 LLINs.1 AMF is now in the process of searching for future distribution partners.
AMF has located potential distribution partners in two countries: Plan Togo and Plan Mali. AMF’s discussions with these potential partners are at advanced and intermediate stages, respectively. As of August 2012, AMF had not made any agreements with new distribution partners for large-scale distributions.2
In September 2011, AMF told us that if it were to receive $3 million or more from GiveWell, it would use those funds for a distribution with an expected completion date of June-September 2012.15 As of August 2012, AMF had received that level of funding from GiveWell, but had not made sufficient progress to meet its expected completion date for future distributions.
We discussed this slower-than-expected progress with Rob Mather, AMF's founder. He told us that during March and April 2012, AMF had discussions with the National Malaria Control Program (NMCP) in Malawi regarding the country’s upcoming universal coverage distribution. AMF was considering funding the 600,000 nets this campaign required to close the gap in Malawi. However, AMF withdrew from this potential distribution because it could neither obtain the data to fully evaluate the potential distribution nor receive assurances on how the distribution would be carried out. AMF is considering publishing detailed information on its experiences with this distribution.16
AMF currently believes that the earliest date its funds could be spent is October 2012.17 In this scenario, nets would be disbursed in early 2013.
AMF is conscious of not converting funds to nets as quickly as it expected to and has learned it is necessary to consider more potential distributions in parallel to avoid delays it considers suboptimal. Going forward, AMF plans to build a “pipeline” of potential future distributions, so that money can be moved more quickly into buying nets. AMF feels that it was a mistake to place too much stock in a few potential distributions (including the one in Malawi, discussed above), rather than aggressively pursue more potential distribution opportunities back in April.18 AMF has recently provided an explanation and update to donors whose donations are currently unallocated to a distribution. (http://www.againstmalaria.com/futuredistributions.aspx)
Concern Universal, AMF's distribution partner in Malawi, published a final report on the Ntcheu distribution that includes a detailed description of each stage of the distribution process (orientation, registration, data entry, verifications, distribution) with photographs.20 This report also lists challenges met throughout the process and recommendations for future distributions.21
AMF requires its distribution partners to collect monthly malaria case rate data from all health centers in the distribution zone for 12 months preceding and 4 years following the distribution.23 AMF has followed through on its plans to collect and share this data and has published data on malaria cases from each of the 37 health facilities in Ntcheu for the months of July 2010 through June 2012.24 This includes data for nearly four months of full net coverage in Ntcheu.25
Based on independent evidence, we believe that net distributions reduce deaths from malaria. (For more, see our page on mass distributions of insecticide-treated nets.) The data AMF collected are consistent with the notion that AMF's net distribution had a significant impact on malaria rates in Ntcheu. However, we do not currently feel that we are able to draw strong conclusions from this data, so it does not significantly increase our confidence in net distributions in general, or AMF's in particular.26
AMF requires its distribution partners to administer 4 post-distribution surveys (PDSs) to gather data on net usage and condition.27 In Ntcheu, the first of these surveys was administered six months following the distribution in 200-250 households in each of the 37 health centre areas.28
6-month PDS data is now available for some areas of Ntcheu,29 covering 7,646 households and 15,735 nets (5.9% of the total distributed).30 The results report a 90% usage rate and 99% of nets still in very good condition (fewer than two holes of 2 cms or smaller).31 District by district data is available on AMF's website on its survey summary page at http://www.againstmalaria.com/SurveyForms_JobsSummary.aspx (archived).
AMF told us that Concern Universal used several measures to increase the reliability of PDS data.32
We are glad that AMF is collecting this data because it demonstrates that AMF is following through on its plans post-distribution and because the survey data might identify a serious problem, were one to exist. Concern's methodology for conducting these surveys appears reasonable, but we have not yet seen the full technical details of these surveys or results from Concern Universal's data quality auditing procedures. Thus, after only a limited examination of the data, these survey results are consistent with the notion that those who receive AMF's nets use them.
“…Concern Universal Malawi and Ntcheu District Council conducted a Universal Long Lasting Insecticide Net (LLIN) Distribution Programme across Ntcheu District as part of the National Malaria Control Programme Universal distribution efforts. Against Malaria Foundation provided the 268,000 LLINs for distribution in Ntcheu District whilst funds for distribution operations have been provided by Irish Aid. This stock of nets will be sufficient to ensure coverage of every sleeping space in Ntcheu District.” Concern Universal, “Final Distribution Report,” Pg 2.
See Concern Universal, "Final Distribution Report," Pgs 3-14.
Against Malaria Foundation, “Malaria Case Rate Data.”
The final distribution phase that occurred in late March/early April 2012 included 16,574 nets out of about 268,000 nets, so the months of March/April are ones in which there was nearly full coverage: "[The final] distribution will be conducted in late March and early April when the project receives the additional 16,574 LLINs from AMF." Concern Universal, "Final Distribution Report," Pg 14.
AMF performed its own analysis of the data and reached a stronger conclusion about the meaning of the results. Below, we summarize AMF's findings, explain our perspective, and present an email Immo Kleinschmidt (bio, archived) sent to AMF about this data. For the full discussion of AMF's analysis, see AMF's "Ntcheu Update" (http://www.againstmalaria.com/update_ntcheu.aspx) and updated malaria case rate data.
AMF performed a preliminary analysis of the malaria case rate data in which it concludes that the data shows a 40-50% decline in the number of malaria cases in March-June 2012, the months immediately after the distribution, compared to the same months in the prior year. (“We now have two sets of post-distribution data following the December 2011 to February 2012 distribution of 270,000 nets in Ntcheu District, Malawi and the results are strong. Summary 1. Malaria rates in March to June 2012 are already 50%, 45%, 40% and 40% lower than in the corresponding months in 2011.” Against Malaria Foundation, “Ntcheu Update.”)
Our perspective There are two issues that lower our confidence in AMF's conclusions:
The malaria case rate data was collected using different diagnostic mechanisms during the time periods being compared. From July 2010 to June 2011 malaria cases were identified on the basis of clinical observation. From July 2011 onwards, blood slide testing using Rapid Diagnostic Testing (RDT) kits were used.
AMF comments on its website that clinical observation is widely accepted to be prone to over estimate the true level of malaria. In order to estimate the actual drop in malaria cases after the distribution, AMF attempts to determine the over estimation of malaria cases from July 2010 to June 2011. AMF lays out its reasoning in the updated malaria case rate data. We do not find AMF's method robust.
AMF estimates that during the rainy season months of March-June, clinical observation would yield a 25% higher rate of malaria diagnosis than RDTs, so it decreased the reported cases of malaria in the March-June 2011 period by 25%. AMF then compared recent months’ malaria data with its adjusted estimate of the true levels of malaria in the same months of the prior year. (Rob Mather, email to GiveWell, September 6, 2012.) This lowered AMF’s estimated drop in malaria for March – June 2012 to 40-50% (from an unadjusted 50-60%). (“As the March-June 2012 data falls within the peak malaria season, we estimate the actual malaria fall is some 10 percentage points less or 50%, 45%, 40 and 40% respectively." Against Malaria Foundation, "Ntcheu Update.")
However, we believe that this adjustment is based on limited evidence, (“Data in the non-rainy season strongly indicates clinical observation overestimates malaria cases by 40-50%. This is concluded from comparing data from Jul to Nov 2011 (RDTK basis) with Jul to Nov 2010 data (clinical observation). In the rainy season there is limited data to draw conclusions but data for Dec 2011-Jan 2012 compared to Dec 2010-Jan 2011 suggests the over-estimate due to clinical observation in these months may be significantly lower and in the range of 5-15%. This has support from anecdotal evidence (we are trying to collect data from other studies) which suggests 'false positives' decline in the rainy season as a high percentage of those with 'malaria and fever type symptoms' do indeed have malaria." Against Malaria Foundation, "Ntcheu Update.") and therefore this data doesn't lead us to significantly adjust our prior view of this distribution's impact.
Some of the change in malaria cases could be caused by external factors that influence malaria rates, such as rainfall, as opposed to the distribution. ("Rainfall is important for the availability of vector breeding sites. The effect of rainfall depends on the breeding habits of mosquitoes. It may boost the proliferation of species that breed in running water and suppress those that prefer stagnant water and small pools. Most spectacular is the effect of heavy rainfall in semi- arid and arid areas. It may be followed by epidemics that start 3–4 weeks after the rain, an event that has been observed in Khartoum (Sudan) and Djibouti. Not only is the overall amount of precipitation essential, but also the rate. Heavy rainfall produced within a few hours would flush out breeding places, whereas the same amount spread over a few days would facilitate breeding. Thus rainfall should be monitored on a daily basis, and good indicators are the total rainfall and the number of rainy days." World Health Organization, "Guidelines on the Elimination of Residual Foci of Malaria Transmission," Pgs 18-19.)
Dr. Immo Kleinschmidt, an epidemiological statistician from the London School of Hygiene and Tropical Medicine, assessed AMF's malaria case rate data and came to the following conclusions:
It is therefore reasonable to conclude that the component of the decline in cases which cannot be accounted for by the change in diagnosis method, is very likely to be a result of the bednet distribution. This decline could be between 30% and 50%, but it is hard to put an exact figure on it.
It is very plausible that the observed decline in malaria cases, after allowing for the change in diagnosis method, is associated with the mass distribution of nets." Dr. Immo Kleinschmidt, email to Rob Mather, September 11, 2012.
“We now have two sets of post-distribution data following the December 2011 to February 2012 distribution of 270,000 nets in Ntcheu District, Malawi.” Against Malaria Foundation, “Ntcheu Update.”
"This would achieve a survey of more than 5% of the covered [households] and nets and we considered this fine (actual numbers: 7,646 [households], 15,735 LLINs assessed = 5.9% of distributed nets)." Rob Mather, e-mail from Rob Mather, August 8, 2012.
“Net condition six months post-distribution is exceptionally strong. 99% are in 'very good' condition (fewer than two holes of 2 cms or smaller) and 1% are in 'OK' condition (fewer than 10 holes). We would expect nets to be in very good condition after six months but this is an exceptionally high level. The condition of the net is a subjective assessment. It is possible there is bias introduced by those carrying out the survey.” Against Malaria Foundation, “Ntcheu Update.”
”The following operating elements were put in place to mitigate against inaccurate data collection:
We believe the data is reliable. We will however be conducting some spot checks to test the accuracy of the data.” Against Malaria Foundation, “Ntcheu Update.”
”Are all the nets being used correctly? Please ask the householder to demonstrate how the nets are used at night.” Excerpted from post-distribution survey form, Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 4.