# Against Malaria Foundation - August 2014 Update

Published: August 2014

## Summary

Future distributions and room for more funding (More)

AMF currently has $6.8 million available for future distributions, having committed$8 million to distributions planned for 2014-2017. $2.7 million of its allocated funds are allocated to distributions scheduled to occur more than 2 years from now. Since 2012 (the period in which AMF has negotiated larger-scale distributions), about$8 million of the $10.3 million spent or committed has been in partnership with Concern Universal in Malawi. We continue to believe that AMF has limited room for more funding (though we believe there are ways in which AMF's actions are being affected by funding constraints) and are not planning to re-recommend it at this time. Updates on past distributions • AMF, with Concern Universal, completed a distribution in Balaka district, Malawi in late 2013. The distribution in Dedza, Malawi is now scheduled for late 2014. The distributions have been delayed and expanded. Total expected cost is about$1.46 million. (More)
• Two years of follow up data from Ntcheu, Malawi is now available – the first of its kind for an AMF distribution – as well as one round of follow up data from Balaka. AMF is the only organization we know of that is regularly and extensively (sampling ~5% of households that received LLINs) collecting post-distribution data on LLIN condition and use. The Ntcheu two-year survey found that 79% of individuals in households that had received LLINs were sleeping under a useable LLIN, and 77% of LLINs were in "very good" or "ok" condition. The Balaka survey suggests that LLINs may be decaying more quickly in that district. Note that we have not seen full technical details for this survey and have therefore not carefully vetted its methodology. (More)

## Follow up surveys from Ntcheu and Balaka, Malawi

In December 2011 to February 2012, AMF, with Concern Universal, distributed about 270,000 nets in the district of Ntcheu, Malawi. Concern Universal has now completed three follow up surveys in the district: at 6, 15, and 24 months post-distribution.

6-month post-distribution data is also available from the Balaka distribution discussed above.

We have not seen technical details at the level of a high quality academic study on how the surveys were carried out,22 and we therefore have not been able to fully vet the results.

### About the Ntcheu 6-month survey

While data from the 6-month report is available, there is no accompanying narrative report (AMF did not ask for narrative reports until before the Ntcheu 15-month survey23), so we do not have details on how the survey was carried out. AMF told us that similar methods were used in this survey as in subsequent surveys.24

### About the Ntcheu 15-month survey

Concern Universal reported using the following methodology for the 15-month post-distribution survey:

• Ten villages in each of 37 health facility catchment areas were randomly selected for inclusion in the survey. Concern Universal noted, "Random sampling gave a chance to villages that were in remote areas to be visited [that] would not have been visited if random sampling was not done."25
• Within each village, 25 households were randomly selected from the pre-distribution registration list. Only households that received nets during the distribution were included in the sample.26
• Surveys were carried out by "Environmental Health Officers" (EHOs) – officials responsible for supervising Health Surveillance Assistants (HSAs), Malawi's community health workers. They were overseen by the district Malaria Coordinator and his deputy.27

On March 25, 2014, AMF announced that it had agreed to fund 676,000 nets in the province of Kasaï Occidental in DRC, at a cost of about $1.94 million53 – its largest distribution to date. The UK's Department of International Development (DFID) will fund the non-net costs of the distribution and the distribution will be carried out by IMA World Health.54 AMF notes that DRC is a very difficult country to work in, but that the agreement has met its high standards for monitoring and evaluation.55 ### Four districts in Malawi AMF, with Concern Universal, has distributed LLINs, or will soon do so, in three districts in Malawi: Ntcheu (early 2012), Balaka (late 2013), and Dedza (planned for late 2014). AMF has recently announced that is will add a fourth district, Dowa, where it plans to conduct a distribution in November 2014. AMF has also committed funds to replenishing LLINs in the first three districts over the next three years. In total, AMF has committed$5.6 million to distribute 1.4 million LLINs in Malawi in 2014-2017 (not including the previously planned distribution in Dedza in late 2014). Number of LLINs and expected timing by district:56

• Dowa District: 400,000 LLINs in November 2014
• Ntcheu District: 350,000 LLINs in October 2015
• Balaka District: 235,000 LLINs in October 2016
• Dedza District: 430,000 LLINs in October 2017

### North Idjwi, Democratic Republic of the Congo

AMF plans to fund 60,000 LLINs for a universal coverage campaign in the northern part of Idjwi island. It will work in partnership with Amani Global Works. Once this distribution is complete, AMF expects to fund another 60,000 LLINs for the southern part of the island.57 AMF expects each portion of the distribution to cost $186,000, for a total of$372,000.58

### Summary of recent and confirmed distributions

Year Distributions planned (# of LLINs) Cost
2012 Ntcheu (268,420)59 $1,110,890 2013 Balaka (154,230)60 and Dedza (delayed to 2014, 245,000)61$1,210,744
2014 Kasaï Occidental (676,000), Dowa (400,000) and North Idjwi (60,000) $3,726,000 2015 Ntcheu (350,000) and South Idjwi (60,000)$1,586,000
2016 Balaka (235,000) $940,000 2017 Dedza (430,000)$1,720,000

Total funding for distributions in partnership with Concern Universal: $7.98 million. Total funding for distributions with other partners:$2.31 million.

### Distribution discussions that did not succeed

AMF was previously involved in negotiations for two further distributions that would have cost a total of $12.85 million. It is no longer proceeding with these negotiations. The majority of the$12.85 million would have funded a distribution in DRC. There were two main partners involved in the DRC negotiations: a co-funding partner who would have funded the non-net costs of the distribution and a distribution partner who would have carried out the distribution. According to AMF, the distribution did not proceed because "after extensive discussions we did not reach agreement [with the distribution partner] on the detailed aspects of verification of the distribution or monitoring and evaluation within the timeframe necessary to secure a firm commitment from our co-funding partner. During the negotiation, significant budget cuts were also placed on our co-funding partner."

### Distribution pipeline

It is our understanding that AMF currently has a limited pipeline for future distributions. It previously was in discussions for several large distributions and if all discussions had succeeded, it would have had a small funding gap.62 AMF told us that it is, as of June 2014, "looking at building a distribution pipeline to replace this."63

### Funding constraints

We believe that funding constraints are affecting the actions AMF has taken recently and will take in the short-term. Examples include:

• AMF would like to be a major partner in a large distribution, which would cost perhaps $10 million or more. This would allow AMF more leverage to influence how the distribution and monitoring was carried out. • AMF feels that it can only negotiate distributions for which it has high confidence that it will have enough funds available. This means that AMF is limited in the number of distributions it will explore at one time. • AMF would like to have a planning horizon of at least three years. Limited funds now could affect AMF's distribution pipeline for several years into the future. Our determination that AMF has limited room for more is based largely on our level of confidence in AMF's ability to negotiate further distributions. This determination may change as AMF's negotiations proceed. ## Re-estimation of cost per net In our most recent review of AMF, we relied on an estimate from the Roll Back Malaria Partnership's Harmonization Working Group for the cost to purchase and deliver an LLIN ($5.80). In this update we have attempted to improve on that estimate by: (a) talking to the Harmonization Working Group about its estimate; (b) comparing this estimate to AMF's financial history and plans.

We spoke to Melanie Renshaw, Co-Chair of the Harmonization Working Group. Dr. Renshaw told us that the group uses $2.50 per net for non-net costs because it has found this to be the historical average. Actual costs can vary significantly across countries.64 Using budgets and estimates provided by AMF, we estimate that the total cost to purchase, distribute and follow up on an AMF LLIN is$5.20-$5.40 in Malawi and very roughly$6.70 in DRC. We focus on these two countries because they are where AMF has completed recent distributions or is planning distributions.

The estimate for Malawi is based on:

• The purchase price per LLIN, calculated from AMF's total LLIN budget and the number of LLINs for the distributions in Dedza ($3.02 per LLIN) and Balaka ($3.06 per LLIN).
• Concern Universal's budget for non-net costs for the Dedza and Balaka distributions, which include the costs of six follow up surveys. AMF told us that that non-net budget includes all monetary costs incurred by Concern Universal and use of vehicles;65 non-monetary costs such as use of office space and managerial staff time do not appear to be included.
• An estimate of non-monetary costs incurred by Concern Universal and local governments, based on estimates provided in 2012 by Concern Universal for its distribution in Ntcheu province.
• An estimate of AMF's organizational costs per net, including an estimate of the value of donated services and volunteer time. Because these costs generally do not vary with the number of LLINs distributed and remain roughly constant on a per year basis, the per LLIN estimate is highly dependent on the time period used and the number of LLINs distributed in that period. We estimate AMF's organizational costs as $0.58 for 2013-2015. This estimate assumes that AMF will complete the distributions it currently plans for that period and does not account for any additional distributions. We shared our analysis with AMF before publishing and AMF wrote the following in response. Response from AMF: Using actual costs to assess cost effectiveness The AMF view is it more relevant to its donors to look at AMF’s actual costs when calculating its cost-effectiveness, rather than adding in costs where either no cash cost exits or support is provided pro bono. This matters as not to do so distorts cost-effectiveness figures. AMF’s overhead costs per annum are$120,000. Using this figure the cost per delivered and monitored net would be $4.50 -$4.80 (midpoint $4.65). GiveWell uses an overhead figure of$300,000 due to adding in a CEO salary of $100,000 (AMF’s CEO does not draw a salary) and$80,000 of other overhead costs for services provided pro bono to AMF and therefore where no cash cost exists (the organisations providing pro bono support are very happy to work for no immediate financial return since they benefit in other ways from their involvement, e.g. in improved staff morale. This means they incur no net cost from providing the pro bono service).

As a result, GiveWell calculates the cost per net for AMF at between $5.20 -$5.40 (midpoint $5.30). GiveWell has calculated the cost per net for other organisations as$5.80.

At $5.30 per net, AMF is 10% more cost-effective than other organisations. At$4.65 per net, 25% more effective. That is a material difference.

We believe our donors are more interested in a comparison derived from AMF’s actual costs rather than penalising it (reducing its assessed effectiveness) by not considering the no-cost and pro bono cost savings they may achieve. AMF as an organisation will continue to seek pro bono support believing this is more cost effective than paying for services.

For DRC, the estimate is much rougher. It is based on the assumption of $3 to purchase an LLIN,$3 in non-net costs (based on AMF's rough estimate that purchase and non-net costs for DRC are roughly equal), and the same estimates of non-monetary and AMF organizational costs used for Malawi.

We have not yet updated our analysis of the cost per life saved with LLIN distributions. Our previous analysis used $6.13 as the total cost to distribute an LLIN and estimated that this translated to roughly$3,400 per life saved (with a number of caveats). Our revised cost per LLIN would translate into a lower cost per life saved in Malawi and perhaps a slightly higher cost per life saved in DRC. These adjustments do not take into account any differences in the effectiveness across countries and it is not straightforward how to account for such differences.

## Source

Document Source
AMF future distributions Unpublished
AMF How we work with distribution partners Source
AMF non-net cost budget for Balaka and Dedza (October 2013) Source (archive)
AMF page on Balaka 2013 distribution Source (archive)
AMF page on Dedza 2014 distribution Source (archive)
AMF page on Kasaï Occidental 2014 distribution Source (archive)
AMF page on North Idjwi distribution Source (archive)
AMF page on Ntcheu 2012 distribution Source (archive)
AMF page on South Idjwi distribution Source (archive)
AMF: "676,000 LLIN distribution agreed in the Democratic Republic of Congo (DRC)" Source (archive)
AMF: "Distributions update, June 2014" Source (archive)
Balaka 6-month post-distribution check up data (April 2014) Source (archive)
Balaka 6-month post-distribution check up plan (April 2014) Source (archive)
Balaka distribution proposal (2013) Source (archive)
Balaka distribution report (2013) Source (archive)
Balaka distribution weekly report (October 7-12, 2013) Source (archive)
Balaka household survey Source (archive)
Cost per LLIN analysis Source
GiveWell's non-verbatim summary of a conversation with Rob Mather, August 15, 2013 Source
Melanie Renshaw, conversation with GiveWell, May 23, 2014 Source
Ntcheu 12-month post-distribution check-up report Source (archive)
Ntcheu 15-month post-distribution check-up data Source (archive)
Ntcheu 24-month post-distribution check-up report Source (archive)
Ntcheu 24-month post-distribution check-up data Source (archive)
Ntcheu 6-month post-distribution check-up data Source (archive)
Rob Mather, conversation with GiveWell, March 6, 2014 Source
Rob Mather, conversation with GiveWell, May 23, 2014 Unpublished
Rob Mather, email to GiveWell, May 22, 2014 Unpublished
Rob Mather, email to GiveWell, June 30, 2014 Unpublished
Rob Mather, email to GiveWell, August 22, 2014 Unpublished
• 1.
• 2.

"Nets distributed: 154,230." Balaka distribution report (2013), Pg 13.

• 3.

Balaka distribution proposal (2013), Pg 3. Note that AMF later told us that the official estimate of LLINs distributed in May 2012 is 129,800, but that at the time AMF and Concern Universal were planning the distribution, they believed it was 123,627. Rob Mather, email to GiveWell, June 30, 2014.

• 4.
• 5.
• 6.

Number of people: 388,782
Number of sleeping spaces: 229,945
Ratio: 1.69

• 7.
• Population: 388,782 (see previous footnote)
• Population divided by 1.8: 215,990
• Population divided by 1.69: 230,049
• Difference: 14,058
• 8.
• 9.

Nets required: 158,805
Nets distributed: 154,230
Balaka distribution report (2013), Pg 13.

• 10.

Balaka distribution proposal (2013)

• 11.
• 12.

“Another reason for the mismatch is the long delay between the registration process and the beginning of the distribution, which allowed for some households to migrate to other villages.” Balaka distribution weekly report (October 7-12, 2013).

• 13.

Balaka household survey

• 14.

Linked in "Distribution" tab at AMF page on Balaka 2013 distribution.

• 15.

Balaka distribution report (2013)

• 16.

"[GiveWell:] Does AMF expect to post photos/videos from the Balaka distribution?
[AMF:] No, CU did not take photos of this one. They are aware of this slip and there will be photos and video footage from Dedza. Photos and videos are also included in the agreement for the DRC-Kasai distribution." Rob Mather, email to GiveWell, May 22, 2014.

• 17.
• 18.

See our November 2013 update on AMF.

• 19.
• 20.
• 21.

Total funding for nets in Balaka: $457,470. AMF page on Balaka 2013 distribution. Total funding for nets in Dedza:$630,000. AMF page on Dedza 2014 distribution.
Non-net cost budget: $369,648. (Note: this does not take into account the increase in nets in Dedza.) AMF non-net cost budget for Balaka and Dedza (October 2013). Total:$1.46 million

• 22.

The type of technical information we would ideally like to see includes:

• Results of audits on data quality.
• Analysis of the extent to which respondent selection was carried out according to the protocol.
• Analysis the extent to which selected households were replaced with pre-selected alternate households (i.e. attrition from the sample).
• Survey text used in each survey round and detailed protocols for responses that are directly observed by surveyors.
• 23.
• 24.
• 25.
• 26.

"Ntcheu district has 37 Health facilities. The standard sample size for each health facility was 250 respondents in each health facility (catchment Area). The health facility catchment area was taken as one area. Therefore Ntcheu district had a total of 9250 samples.

Ten villages in each catchment area were selected using Probability Proportional to size sampling (PPS) method. The process involved reviewing data from both Ministry of Health list of villages under each catchment area, calculating the cumulative population for each catchment area, calculating the sampling interval by dividing total cumulative population by 10 and choosing the random number using the Random Number Table to determine the first village to be interviewed. This generated a list of 10 villages to be interviewed. The number ten was chosen due to the budget and time. With this process villages with larger population had higher chances of being selected…

In the village households for interview were randomly selected using the household list from Net distribution register. Using a random number table the reference household was selected from this reference household the nearest household was taken using the nearest household 25 households were reached in each area. The household interviewed was supposed to have received the LLINs during the 2011/2012 AMF nets distribution as eligibility for being interviewed. If the household did not receive the AMF nets the next nearest household was taken until the number of twenty-five was reached.

If it happens that villages close by were selected the household interviewed were marked PDS showing that interview has already taken place." Ntcheu 12-month post-distribution check-up report, Pg 3.

• 27.

"The selection of Data Collectors was based on those that were coming within the district health office mainly Environmental Health Officers (EHOs) who are the supervisors for he Health Surveillance Assistants… There were 16 EHOs that were divided into two teams each team was visiting one health facility per day guided by one team leader who was responsible to check that data collection was done as planned (random selection) and the selected villages was visited. The two team leaders were the district Malaria Coordinator and his deputy." Ntcheu 12-month post-distribution check-up report, Pgs 3-4.

• 28.
• 29.

"There was no budget line for the activity hence we lost time during the process of identifying where to take the resources." Ntcheu 12-month post-distribution check-up report, Pg 6.

• 30.
• 31.

"Ntcheu district has 37 health facilities. It was decided to collect data from 250 households in each health centre catchment area with 25 households selected at random from ten villages, also selected at random. Villages were randomly selected using the village lists generated from the pre-distribution and distribution work for the 2011/2012 AMF-funded universal coverage LLIN distribution. A random number table was used to select the villages. Households were randomly selected using the household lists produced during the same campaign. A random number table was used to select 25 villages [Note that based on the second sentence of this quote, we believe this is a typo and should say "25 households"]. Five more households were put on reserve in case no one was at home in the selected households." Ntcheu 24-month post-distribution check-up report, Pgs 5-6.

• 32.

"There were 368 HSAs involved in collecting data." Ntcheu 24-month post-distribution check-up report, Pg 5.

• 33.
• 34.

"Data was entered into a database via a web interface created by AMF. An internet connection was required for this work. This created some problems when the internet connection was down at the data entry location." Ntcheu 24-month post-distribution check-up report, Pg 6.

• 35.

"Once all data had been entered these was a period of several days in which duplicate forms were removed and data errors corrected. The online system highlighted duplicates and errors for checking which made this task relatively easy. The data entry system has now been improved to make it impossible to enter duplicates." Ntcheu 24-month post-distribution check-up report, Pgs 6-7.

• 36.

"The PDCU cost US$7,650 equal to US$0.85 per household." Ntcheu 24-month post-distribution check-up report, Pg 3.

• 37.
• 38.
• 39.
• 40.
• 41.

"'very good condition' (fewer than 2 holes of up to 2cms in size)." Ntcheu 24-month post-distribution check-up report, Pg 4.

• 42.

"'OK' (fewer than 10 small holes on them)" Ntcheu 24-month post-distribution check-up report, Pg 4.

• 43.

"'Poor' condition with more than 10 holes or 1 hole larger than 10 cm." Ntcheu 24-month post-distribution check-up report, Pg 4.

• 44.

"Worn out means not usable. Typically, holes will be 10 cms of cms in size and numerous and the net considered effectively unrepairable and/or structural integrity will have gone. If hung the net would not provide protection against mosquitoes given the extent of holes or tears or other loss of structural integrity." Rob Mather, email to GiveWell, June 30, 2014.

• 45.
• 46.
• 47.

"Usage does not appear to have been near-universal. Most studies report usage rates in the range of 60-80%, though some report 90%+ usage." From our page on Long Lasting Insecticide Treated Nets.

• 48.

"Alliance for Malaria Prevention 2011 lays out a model for estimating the number of LLINs still in use after distributions: 'the number of LLINs already distributed over the last three years and considered to be available in households should be calculated and subtracted from the total need, working with a decay rate of 8 per cent at one year (0-12 months), 20 per cent at two years (13-24 months) and 50 per cent at three years (25-36 months).'…
Bottom line: We believe that the "8%-20%-50%" model is the most widely used and most reasonable approximation available at the moment for capturing the extent to which LLINs remain in use in the years following distribution, accounting for any factors that might cause LLINs to be discarded or additional LLINs to be purchased. It implies an average of 2.22 years of use for each LLIN distributed. Data and analysis on this topic appears extremely thin; we have little sense for how long LLINs last in practice." GiveWell report on the "decay model" for LLINs.

• 49.
• 50.

"We have received a significant quantity of case rate data and expect to post information at the end of June." Rob Mather, email to GiveWell, May 22, 2014.

• 51.

"Concern Universal conducted an audit of the malaria case rate data measuring malaria rates in the regions served by AMF's distribution there (more on this data at

The audit found that the numbers from some of the health centers are less reliable than previously thought due to staff absences, lack of qualified testing personnel, or equipment stock outs, each of which could have led to under-reporting of malaria cases. Also, at times, local health staff reported 'people receiving anti-malaria treatment' as opposed to 'people diagnosed with malaria by a rapid diagnostic test.' Health staff may have given anti-malaria treatment to patients presenting with a fever without testing them, which would have led to over-reporting of malaria cases.

AMF has requested additional data from local health centers and is in the process of reviewing it. It is planning to publish the results of this audit by the end of September." GiveWell's non-verbatim summary of a conversation with Rob Mather, August 15, 2013, Pg 2.

• 52.

AMF: "Distributions update, June 2014"

• 53.
• 54.
• 55.

"At the same time, operating in DRC is challenging given the recent conflict, poor infrastructure, minimal access to health care, difficult geography and long distances between towns and villages.

The distribution of nets and follow-up will be strongly data-driven. The distribution will be accompanied by detailed monitoring and reporting. Post-distribution check-ups of net use and condition will take place on a six to nine month rolling schedule for three years. The detailed planning of the distribution and follow-up will respect the practicalities of working in such a challenging environment as that presented by DRC." AMF: "676,000 LLIN distribution agreed in the Democratic Republic of Congo (DRC)".

• 56.

All information in the "Four districts in Malawi" section is from AMF: "Distributions update, June 2014".

• 57.

AMF: "Distributions update, June 2014"

• 58.
• 59. AMF page on Ntcheu 2012 distribution
• 60. AMF page on Balaka 2013 distribution
• 61. AMF page on Dedza 2014 distribution
• 62.
• 63.
• 64.
• 65.

Rob Mather, conversation with GiveWell, May 23, 2014