Against Malaria Foundation

Print
Email
   

The Against Malaria Foundation (AMF) is one of our top-rated charities, and an organization that we feel offers donors an outstanding opportunity to accomplish good with their donations.

More information:

Published: November 2012


Summary

AMF (www.againstmalaria.com) provides long-lasting insecticide-treated nets (for protection against malaria) in bulk to other organizations, which then distribute them in developing countries.

AMF is a recommended organization because of its:

  • focus on a program with a strong track record and excellent cost-effectiveness (more).
  • standout transparency and accountability - it publishes photographs and reports from each of its distributions and requires that organizations that distribute its nets monitor the usage and condition of nets in the years following the distribution and track and provide monthly malaria case rate data (more).
  • room for more funding - AMF has told us that it can use additional funding to expand its core program and has committed to reporting on how additional funds are used and what results are achieved.

Our review, below, discusses our full assessment of AMF, including what we see as its strengths and weaknesses as well as issues we have yet to resolve.

Our review process

We began reviewing AMF in 2009. Our review has consisted of:

  • Reviewing AMF's public records for each of its net distributions and other documents AMF has shared with us.
  • Extensive communication, including several meetings at AMF's London headquarters with AMF Founder Rob Mather, to discuss AMF's methods and funding needs.
  • A visit to AMF's distribution partner organization, Concern Universal, in Malawi in October 2011 (notes and photos from this visit).
  • Reviewing materials from AMF’s distribution in Malawi in early 2012 (conducted by Concern Universal), including pre-distribution registration data, distribution reports, and data from a 6-month post-distribution follow up survey. See our March 2012 and September 2012 AMF Updates for full details. We have also communicated with Concern Universal’s Malawi Director, Robin Todd.

Previous reports on Against Malaria Foundation:

Table of Contents

A few key terms for this review

  • A net refers to any bednet intended to block mosquitoes, whether treated with insecticide or not.
  • An ITN is an insecticide-treated net. As discussed at our write-up on ITNs, distribution of ITNs has been rigorously studied and associated with declines in under-5 mortality and overall malaria cases.
  • A LLIN is a long-lasting insecticide-treated net, intended to serve as an effective ITN for 4-5 years (details at our write-up on ITNs). The nets distributed by AMF are LLINs,1 and (as noted in our write-up on large-scale ITN distributions) most large-scale donor-financed malaria control today utilizes LLINs.

What do they do?

AMF provides long-lasting insecticide-treated nets (for protection against malaria) in bulk to other organizations, which then distribute them in developing countries.

Prior to November 2011, AMF contributed LLINs to small-scale distributions. Since then, it has de-emphasized small-scale distributions and it expects larger-scale distributions to be the primary way in which it distributes LLINs in the future (more below).

AMF has conducted one large-scale distribution in conjunction with Concern Universal in Malawi, which took place in December 2011-March 2012. For details on the Concern Universal distribution, see our March 2012 and September 2012 updates.

Process for distributions

Selecting locations for distributions

  • When selecting locations for future distributions, AMF told us it consults a series of sources as it believes there is no one reliable resource with up-to-date information to determine where nets are needed. Sources it consults include the Alliance for Malaria Prevention's (AMP's) list of countries with significant net gaps, other malaria control funders, in-country technical advisors, the relevant National Malaria Control Program (NMCP), implementing organizations and the African Leaders Malaria Alliance.2

    In the case of its distribution in Malawi, AMF received estimates of country-level gaps from the Alliance for Malaria Prevention (AMP) and from the Malawi National Malaria Control Program (NMCP). Rob Mather, AMF's founder told us that, although AMF did not have a high level of confidence in the net gap numbers presented by the NMCP, nor those it was sent by the Global Fund, it did have a high level of confidence that the net gap was significantly higher than the 250,000 LLINs AMF was considering providing.3 Note that the materials AMF has sent us from the period prior to its decision to enter Malawi do not include quantified estimates of Malawi's LLIN gap,4 though they do make it clear that there is a gap,5 and later reports (from after AMF had decided on Malawi) show a quantified gap for the country.6

    As of November 2012, AMF was considering future distributions in seven countries (more below). A November 7 update from AMP reports LLIN gaps for all of these countries.7

  • AMF told us that it also consults other malaria control funders, implementing organizations, and the relevant National Malaria Control Program (NMCP) to verify estimates of how many additional LLINs are required to reach full coverage in a country.8 We have seen some correspondence between AMF and the NMCP in Malawi which took place prior to AMF’s decision to fund a distribution in Malawi.9 AMF also shared with us correspondence that it had with representatives of the NMCP in Togo, with regard to potential future distributions.10
  • As AMF investigates countries with existing net gaps, it also looks into organizations working within those countries that could serve as distribution partners.11 AMF's distribution partners must have the capacity and willingness to carry out a distribution that meets AMF's requirements (more below).12

    Before deciding to work with Concern Universal in Malawi, AMF requested a distribution proposal, including information on malaria risk in the distribution area (including data on reported malaria cases), other net distributions in the area, coordination with government, and how the distribution would be carried out.13 In addition to working with Concern Universal in Malawi, AMF has had discussions with Plan in Mali and Togo about potential future distributions (see our September 2012 update for further details.)

Requirements for distribution partners

In order for AMF to partner with an organization, the organization must agree to the following requirements:

  • Pre-distribution registration survey: AMF requires that its distribution partners carry out a pre-distribution registration survey (PDRS) across the entire distribution zone to establish the number of sleeping spaces and LLINs in a condition such that they have at least one year of remaining use. (In the case that such data already exist, AMF and the partner discuss whether the data is reliable and up-to-date, or whether a new PDRS should be conducted.14) In its large-scale distribution in Malawi, AMF worked with Concern Universal, which conducted a PDRS and shared the full results from the survey with us.15
  • Case rate data: AMF asks distribution partners to provide monthly malaria case rate data from all health centers in the distribution zone for 12 months preceding and 4 years following the distribution.16 Concern Universal has provided case rate data for each of the 37 health centers in its distribution area in Malawi.
  • Distribution monitoring: An additional requirement is to provide AMF with photos and videos from the distribution.17 As of November 2012, Concern Universal has provided photos and/or videos for all but 1 of the 14 “sublocations” in Malawi in which it had distributed LLINs earlier that year.18
  • Post-distribution follow-up: In the past, AMF required partners to conduct follow-up surveys at 6, 18, 30 and 42 months after the distribution, assessing net condition, what percent of nets are hung, and whether nets are being used correctly. It now requires partners to conduct follow-up surveys at 6-month intervals, and for a period of 3-4 years.19 Following the distribution in Malawi, AMF reported that Concern Universal visited 6% of households in its distribution area 6 months after the distribution.20 See our September 2012 update for further details.

AMF looks for partners who, in addition to agreeing to the above requirements, are clear and responsive in their communication with AMF.

Partners must be able to procure funds to cover all non-net costs of the distribution.21 We have expressed to AMF that we are concerned that, by requiring partners to pay for the non-net costs of the distribution, it may limit its capacity to distribute nets. AMF was receptive to the possibility of funding non-net costs in the future, if it believes that doing so would increase its ability to distribute LLINs quickly.22 AMF also told us: "a) in over six years of conducting distributions of some 1.5 million nets there has never been a problem with distribution partners securing funding for non-net costs; b) this may or may not change with larger scale distributions."23

Does it work?

On a separate page, we discuss the general evidence behind distribution of LLINs. We conclude that there is strong evidence that these distributions can be expected to reduce child mortality and malaria cases.

When evaluating the effectiveness of an LLIN distribution organization, we seek to answer the following questions:

  • Are LLINs targeted at people who do not already have them? AMF requires partners to complete pre-distribution surveys to determine the number of nets required, if reliable and recent survey data are not available. Concern Universal, AMF’s partner in Malawi, conducted and shared the full results of its pre-distribution survey. We are not fully satisfied with the pre-distribution surveys' ability to ensure that people aren't under-reporting their existing LLIN ownership in order to get more LLINs, but believe that reasonable measures are being taken to address this issue.
  • Do the LLINs reach the intended destinations? AMF requires that distribution partners post pictures and videos of LLINs being handed out. Concern Universal also posted weekly distribution reports on problems with the distribution and how it was addressing them.
  • Are LLINs targeted at areas with high rates of malaria? AMF seeks out distribution partners in countries that are known to have high rates of malaria.
  • Do those who receive the LLINs install them in their homes properly? Do they utilize them consistently over the long term? In the past, AMF required partners to conduct follow-up surveys at 6, 18, 30 and 42 months after the distribution assessing net condition, whether nets are hung, and whether nets are being used correctly. It now requires partners to conduct follow-up surveys at 6-month intervals, and for a period of up to 3-4 years. We have seen 6-month follow-up results from the large-scale distribution in Malawi which are consistent with high, proper usage of nets. We do not have longer-term data from large-scale distributions.
  • Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF? The limited evidence we have suggests the former.

Details follow.

Are LLINs targeted at people who do not already have them?

AMF requires that distribution partners conduct pre-distribution surveys (if recent, reliable survey data are not available) to determine the number of existing LLINs with at least one year of remaining use and the number of nets required in each household.24 This process includes door-to-door surveys of every household in the targeted district by government health workers, followed by village meetings in which staff of AMF's distribution partner read off household names and the number of LLINs that have been allocated to each household; households then indicate if any errors have been made. We observed one such village verification meeting during our visit to Malawi in October 2011.25

Concern Universal conducted a pre-distribution survey, and shared the full results from the survey.26 Based on GiveWell's suggestion, AMF's distribution partner also decided to send its staff to random households at this stage to spot-check the accuracy of its data.27 After the first two weeks of spot checks, the distribution partner reported that it had not found any discrepancies between the number of nets community members said they needed during the verification meeting, and the number of nets found to be needed during the subsequent household visit.28

We believe it is possible that some households may over-report the number of LLINs they need. Cases of households hiding nets in order to receive extra LLINs have been observed in universal campaigns in Senegal and Nigeria.29 That said, we believe that the steps AMF and its partners are taking to minimize the ability of community members to request more nets than they need are reasonable. We will look for the same approach in future AMF distributions, and will request details of how nets were allocated at the household level.

In Malawi, household surveys conducted by the National Malaria Control Programme (in districts not receiving AMF LLINs) have found "much larger LLIN gaps than first thought,"30 and it is not clear to us what the cause of this discrepancy is. We will seek more information when available to assess what this indicates about ensuring nets are accurately distributed and used in large-scale distributions.

Do the LLINs reach the intended destination?

AMF requires that partners send photos and videos of distributions.31 As of November 2012, Concern Universal has provided photos and/or videos for all but 1 of the 14 “sublocations” in Malawi in which it had distributed LLINs earlier that year.32 As we report in our March 2012 update, there were a number of issues that arose during Concern Universal’s distribution in Malawi, which were reported in its weekly updates during the distribution, including attempted thefts, double registrations and logistical problems.33 Overall, Concern Universal seemed to be aware of potential problems and had a system in place to address them.34

Are LLINs targeted at areas with high rates of malaria?

At the highest level, AMF appears to exclusively target countries with known malaria risk.35 In its distribution proposal for AMF, Concern Universal reported the number of malaria cases in the proposed distribution are over a twelve month period (June 2010 to July 2011).36

Do those who receive the LLINs install them in their homes properly?

In the past, AMF required partners to conduct follow-up surveys at 6, 18, 30 and 42 months after the distribution to assess net condition and whether nets are hung and used correctly. It now requires partners to conduct follow-up surveys at 6-month intervals, and for a period of 3-4 years.37 Following the distribution in Malawi, AMF reported that Concern Universal visited 6% of households in its distribution area 6 months after the distribution.38 The survey found a 90% usage rate.39 In general, surveys from mass LLIN distributions in other countries have found usage rates of 60-80%. (For more, see our page on mass LLIN distributions.)
District by district data is available on AMF's website at http://www.againstmalaria.com/SurveyForms_JobsSummary.aspx (archived). See our September 2012 update for further details.

Do those who receive the LLINs utilize them consistently over the long term?

We do not have information on long-term usage rates for LLINs distributed in large-scale AMF distributions, the first of which was conducted in December 2011-March 2012.

Do AMF's LLINs increase the total number of LLINs distributed, or would the recipients have received LLINs from other sources if not for AMF?

For the distribution in Malawi, AMF received data from the National Malaria Control Program on population figures and LLINs already distributed by district, as well as LLINs that had been committed to the country by other funders. AMF estimated, in September 2011, that an additional 909,586 nets were needed to achieve universal coverage,40 of which AMF contributed 268,240.41

Possible negative or offsetting impact

  • Do donated nets displace government health funding? This could be a concern if government funding that otherwise would have been spent on LLINs is spent on other, less worthwhile budget items. We have little sense of how important a concern this is in AMF's case. In the case of the large-scale distribution in Malawi, all anticipated nets were expected from international donors, and the total supply appeared to be still insufficient for country-wide universal coverage.42
  • Will insecticide-treated nets continue to be effective? As discussed at our report on insecticide-treated nets, there is strong evidence for the effectiveness of this intervention, but there is a possibility that environmental changes, including changes in the size and insecticide-susceptibility of mosquito populations, could affect how well insecticide-treated nets work.
  • Do free LLIN distributions distort incentives for recipients or distort local markets for nets? As discussed at our report on insecticide-treated nets, we feel that there is a reasonably strong case for distributing LLINs freely rather than selling them at market (or even below-market) prices.
  • Could distribution of LLINs be inequitable and unfair, causing problems in the targeted communities? We feel that the process that was followed for AMF's large-scale distribution in Malawi is well-suited to ensuring that all who want LLINs receive them.
  • Does AMF divert skilled labor from other areas? Net distribution is conducted by low-level government health staff in partnership with the staff of AMF's partner NGO.43 According to AMF's partner in Malawi, government health staff are normally involved in activities such as disseminating health-related information, reporting on levels of stunting and disease, carrying out immunization campaigns, and providing nutrition support.44 We don't know the extent to which net distribution reduces their ability to complete other duties, though we note that net distributions are generally completed within a few days in each local area.

What do you get for your dollar?

Cost per LLIN distributed

In this section, we estimate the projected cost per LLIN for 2012-2013. We focus on the projected cost per LLIN in 2012-2013 rather than on the past cost per LLIN because, as discussed above, AMF has shifted from a focus on smaller-scale to larger-scale distributions, and in the future AMF expects to distribute far more LLINs than it has in past years (it is aiming to distribute 1 million LLINs per year).45

In the table below, we estimate the costs per LLIN for 2012-2013 distributions, using a combination of past costs and future projections, along with details about how we made each estimate.

We prefer to include all costs incurred to carry out a project, not just those that the charity in question pays for itself. We believe that this gives the best view of what it costs to achieve a particular impact (such as saving a life), and also avoids the lack of clarity and complications of leverage in charity. The estimates below include (1) costs paid by AMF to purchase LLINs, (2) the costs AMF incurs to run its organization, and (3) the costs paid by the distribution partners that AMF works with to ship and distribute LLINs and to monitor the distributions.

Cost per LLIN, assuming 1 million LLINs distributed annually

For full details and calculations, see our total costs and Concern Universal and local government costs spreadsheets.46

Item Cost Components
LLIN purchase cost47 $4.23 N/A
Value of donated services per LLIN48 $0.21 Audit fee, information technology, marketing, banking fees, legal fees, etc.49, and our estimate of the equivalent cost of the CEO salary (the CEO does not currently take a salary and does not intend to in the future).50
AMF organizational costs per LLIN51 $0.17 General office expenses, information technology, and marketing52
Non-AMF cost per LLIN $0.92 Cost of shipping LLINs, pre-distribution surveys, distribution costs (including distribution staff salaries paid by the government53 and transportation), monitoring/follow-up at 6-42 months, Concern Universal staff salaries and office costs54
Total cost per LLIN $5.54
Marginal cost per LLIN $5.15 Excludes pro bono and charity central costs; see next section for further discussion.

Total cost versus marginal cost per LLIN

We have calculated both the total cost per LLIN and the marginal cost per LLIN. The total cost includes every cost (or cost equivalent)55 that we have identified that was incurred by all involved groups. The marginal cost excludes fixed costs, i.e., costs that remain the same regardless of the number of LLINs distributed, and only includes costs that are incurred for each additional distributed LLIN.

The marginal costs exclude AMF’s organizational costs and the cost equivalent of services donated to AMF.56 We exclude these costs from our estimate of the marginal cost per LLIN because, based on our understanding of AMF's model, we believe they will not increase if AMF distributes additional nets. In the past, these costs have remained relatively stable (or decreased, in the case of donated services) as AMF has increased the number of LLINs it distributes.57 AMF agrees with this, telling us that it plans to expand the size of its distributions (i.e., doing fewer but larger distributions) and maintains that it will be able to do so while keeping its current organizational costs stable.58

Note that we believe that the marginal cost per LLIN is the most relevant for donors, since this is what we expect will be the cost per additional LLIN of future AMF distributions.

Cost per life saved

Using the 2012-2013 cost per LLIN, we estimate the cost per child life saved through an AMF LLIN distribution at just under $2,300 using the marginal cost ($5.15 per LLIN) and just under $2,500 using the total cost ($5.54 per LLIN).59

This does not include other potential benefits of LLINs (non-fatal cases of malaria prevented, prevention of deaths in age groups other than under-5 year olds, prevention of other mosquito-borne diseases, etc.). Full details at our report on mass distribution of LLINs.

Our process

In order to arrive at an estimate of the total cost per LLIN provided by AMF, we included the following information:

  • Costs incurred by AMF as reported in its audited financial statements and in a document AMF sent to us.60
  • AMF's projected costs per year, from a conversation with AMF director Rob Mather.61
  • Data from Concern Universal on (a) costs allocated to the project by Concern Universal, (b) Concern Universal's estimate of costs paid by the government to support the project, and (c) other organizational costs, such as the value of office space and management staff time, incurred by Concern Universal.62

Room for more funds?

In late 2011, AMF projected that it would be able to distribute up to $5 million worth of nets by September 2012.63 The funding AMF raised did not exceed $5 million and AMF was not able to distribute nets by September 2012 as it had expected. We are therefore somewhat concerned about AMF's ability to spend funds quickly. However, we believe that AMF has recognized what mistakes it made in 2012 and is working to correct those mistakes for 2013. In particular, it has significantly expanded the number of opportunities it is pursuing and appears to be in a good position to fund a large number of nets in early 2013.

Details:

  • In 2012, AMF pursued opportunities to contribute LLINs to three countries: Malawi, Togo, and Mali. If negotiations with these countries had been successful, AMF would have been able to spend all of the funding it had received in these countries (see our September 2012 update). In November 2012, AMF told us that it would not be proceeding with distributions in Mali (due to unrest in the country) and Togo (due to stalled discussions with the government regarding conducting a pre-distribution survey). It is planning to distribute 250,000 LLINs in the districts of Dedza and Balaka in Malawi with Concern Univeral, fewer LLINs than it previously thought it might be able to contribute to the country (also due to stalled discussions with the government regarding conducting a pre-distribution survey).64
  • AMF has been open about the fact that it has not met the expectations it had set and believes the lessons it has learned will help it to plan better going forward. AMF told us in November 2012:
    Is the failure to progress with these distributions a concern? Yes, in that it has meant we have taken longer to convert funds into nets being distributed.

    What would we do differently if faced with the same situation?
    We would progress more potential distributions at the same time. We are doing that now.

    Could this be indicative of what we might find when trying to do more large-scale distributions with the degree of accuracy, transparency and accountability we require?
    I do not think so. What supports this is there is a significant need for nets and the results we have achieved in Ntcheu have generated interest. More distributions with similar results will help further.

  • As of November 2012, AMF held funding to purchase about one million LLINs,65 excluding the 250,000 nets it had committed to distribute in Dedza and Balaka districts of Malawi. It had begun discussions with representatives in Angola, Burkina Faso, Cameroon, Malawi, Mali, Nigeria, and Senegal about potential future distributions.66 The sum of the LLIN gaps that AMF would consider funding in these countries is between 9.3 and 16.9 million LLINs. AMF would need an additional $35 to $67 million to fill these gaps. We give these figures as an upper bound on AMF's room for more funding in the next year. Its actual room for more funding in the short-term is likely to be considerably less than this, as some of AMF's discussions will likely not be successful.

Potential "separate funds" for non-net costs and insecticide resistance research

AMF is now considering the possibility of raising funds to cover non-net costs of distributions, if it believes that doing so would increase its ability to distribute LLINs quickly.67

AMF is also considering raising funds specifically for research on resistance among mosquitos to the insecticide used on LLINs. If it did so, it would fund a research organization to collect data on insecticide susceptibility before and after its LLIN distributions.68 We have done an initial survey of the research on insecticide resistance and would like to see more research on the topic.

AMF as an organization

We believe the Against Malaria Foundation to be an exceptionally strong and effective organization:

  • Track record: We feel AMF has built up a reasonably strong track record of raising money, finding distribution partners, getting partners to report information publicly at an unusual level, and verifiably getting bednets delivered.
  • Communication: AMF has always communicated extremely clearly and directly with us and given thoughtful answers to our critical questions.
  • Self-evaluation: AMF has invested heavily in self-evaluation, going above and beyond what’s usual for data collection on bednet distributions: it has demonstrated a commitment to collecting long-term net usage data and malaria case rate data.
  • Transparency: AMF appears to value transparency as much as any organization we’ve encountered. It hasn’t just shared information with us; it publishes significant amounts of useful information publicly on its own, far more than the norm. We have never seen AMF hesitate to share information publicly (unless it had what we consider a good reason).

However, we see some potential room for improvement:

  • AMF does not seem to be as aggressive and ambitious about growth and capacity as it could be. It has operated for nine years and still has only two full-time staff members; it does not appear to us to have built as robust a fundraising operation as it could.
  • AMF has not conducted any field visits to distribution partners. We see this as a negative since we find such visits valuable.

More on how we think about evaluating organizations at our 2012 blog post.

Financials/other

All data is from AMF's Audited Financial Statements for 2005-2012.69 Since AMF is a British charity, all figures are presented in British pounds.

Revenue and expense growth (about this metric): AMF is relatively young and relatively small. Its revenues and expenses have been growing since 2007.


Assets-to-expenses ratio (about this metric): AMF has been holding assets for a longer period of time as it seeks new distribution partners for its large-scale distributions, and its assets-to-expenses ratio has risen significantly.


Expenses by program area (about this metric): AMF only runs one program, so all expenses support ITN distributions.

Expenses by IRS-reported category (about this metric): These are within the range we believe is reasonable.

Note: AMF is a British charity, and we therefore use the terms that appear to equate to IRS-reported categories. We equate "charitable activities" with "program expenses" and "governance costs" with "administration". We did not see a line item for fundraising costs in AMF's financial statements.

Unresolved issues

Some of AMF's measures for collecting information on key questions are relatively new. We believe that AMF has a strong track record on ensuring that nets are delivered, and on general transparency, and we find its commitment to improving its documentation credible; but it does not have a robust track record on the answers to some key questions, particularly (a) whether individuals' needs for nets are accurately determined and (b) whether people use their nets properly over the long run.

In 2012, AMF was unable to spend funds as quickly as it expected. We believe it is on track to distribute LLINs more quickly in 2013, however AMF continues to have little experience with moving funds quickly into large-scale distributions.

Sources

  • 1.

    "How will the money raised be spent? On bednets (mosquito nets). Specifically, long-lasting insecticidal nets (LLINs)." Against Malaria Foundation, "Frequently Asked Questions."

  • 2.

    "This is a list of the countries with known gaps and where there are significant contiguous areas without nets, or a significant percentage required, and for which the estimate of need is believed to be reasonably accurate. It does not include countries where there are gaps, typically up to 40% of what the nation needs, but they are spread more uniformly across the country and would therefore require an ‘in-fill campaign’. An in-fill campaign is different from a so-called ‘universal coverage campaign’ because the percentage installed base of nets is higher in the former case and so a pre-distribution registration survey (PDRS) is an absolute requirement to ensure an efficient allocation of nets. Our methodology would lend itself to these campaigns if the relevant National Malaria Control Programme (NMCP) were to embrace a detailed PDRS. The list does not include, in our view, other countries where the need has not yet been quantified. Given there are many countries with needs estimated, we have not chosen to seek out other countries in need of nets. Our assumption is groups like AMP will be a source of reporting on additional countries as quantified needs emerge." Rob Mather, email to GiveWell, August 8, 2012.

    Rob Mather, email to GiveWell, November 20, 2012.

  • 3.

    Rob Mather, email to GiveWell, November 20, 2012.

    "Initial (directional) information

    a) Sources of collated numbers (eg AMP, ALMA)
    We liaise with groups that collate statistics on net numbers. AMP’s (Alliance for Malaria Prevention) weekly email is one such source as you correctly identify and ALMA’s (African Leaders Malaria Alliance) monthly update is another. The numbers are not always up to date and do not always agree. However they are useful as a first alert that there is a gap in a national campaign. I have attached two emails that show you the sort of thing we receive. b) Distribution Partners Groups involved in distributions in particular countries may be the first to be aware there is a net gap and they may approach us by email or telephone. c) Individuals/Advisors (to national campaigns). Those involved in national campaigns have also approached us in the past to ask if we are in a position to help close a net gap. This may be a technical malaria advisor who has just come out of a meeting where it has become apparent there is a gap in funding for nets for a near-term program. We refer to this information as ‘directional’ as in all cases detailed information is then required to confirm there is a gap." Rob Mather, email to GiveWell, June 29, 2011.

    Rob Mather, email to GiveWell, November 20, 2012.

  • 4.

    Examples:

    • Alliance for Malaria Prevention, "Conference Call Minutes (June 22, 2011)," Pg 13.
    • Melanie Renshaw, email to Rob Mather, June 26, 2011.

    "GiveWell: Neither of the AMP emails you sent (May 28, 2011 and June 22, 2011) nor the ALMA email (from June 26, 2011) provide a figure for net gap in Malawi. Do you know why this is? Did you use these groups for directional information in the case of Malawi? Has the net situation changed in Malawi recently?
    AMF: I thought it did – about 500,000 nets. However, a number we needed further proof on as our assessment was these were indicative numbers and not hard and fast numbers. We needed hard and fast numbers to be able to commit funds for nets as we would not do so if there wasn’t a gap. Liaising with the Global Fund, PMI (the two major funders) and the NMCP was, and is, key to this process. We have yet to receive confirmation of net gap which is why we have not as yet committed funds for nets. In part, this is due to the NMCP being required by the Global Fund (rightly in our view) to re-tender a particular element of the net distribution campaign and that has taken priority over work to finalise numbers. As with us, this has halted Global Fund money for nets until all is resolved satisfactorily. I also believe in part it is due to the NMCP having estimates and now they are being quizzed by us and others, they are having to think what to do – admit they are estimates (which we don’t necessarily have a problem with but we have asked to see the assumptions and logic behind those estimates) and/or because they are now coming round to the viewpoint (which we hold) that given the number of perfectly usable nets in place being above 15% of nets needed country-wide, it is economic and has other benefits, to conduct household by household surveys to determine exactly where (on a household basis) nets are needed. I can run through the $9m (cost of nets saved) vs $2-3m (cost of surveys country-wide) economic argument later."
    Rob Mather, email to GiveWell, August 6, 2011.

  • 5.
    • "Issues on quantification: it has been raised that LLINs durability in the country is of 2 years rather than 3 years + the country needs to account for existing nets
    • GF [Global Fund] has confirmed that an AMP TA [Alliance for Malaria Prevention Technical Assistance] mission should take place at the latest in June 2011
    • Strong routine delivery system in place - country will be looking at accounting for existing nets
    • Malawi would be a good case study for how continuous distribution works - one of the countries flagged by VCWG [Vector Control Working Group] for case study.
    • Malawi achieved UC [universal coverage] by the end of 2010 in 3 of 27 districts"

    Alliance for Malaria Prevention, "Conference Call Minutes (June 22, 2011)," Pg 13.

  • 6.

    "Malawi...LLIN gap: 2.7 million w/o taking into account any existing LLINs, or 1.5 million if the nets they assume are there are accounted for during HH registration." Alliance for Malaria Prevention, "Conference Call Minutes (October 26, 2011),” Pg 19.
    As of November 2011, the Alliance for Malaria Prevention had posted minutes from its conference calls since August 3, 2011 at Alliance for Malaria Prevention, "Conference Call Minutes."

  • 7.

    Alliance for Malaria Prevention, "Conference Call Minutes (November 7, 2012)."

    1. Nigeria: "LLIN gap for mass distribution: 4,442,852 LLINs." There is no mention of the particular state that AMF is considering contributing LLINs to.
    2. Senegal: It is listed among a 28 countries with a LLIN gap. The section on Senegal says, ""Gap to be determined."
    3. Cameroon: "LLIN gap for mass distribution: 2.1 million"
    4. Burkina Faso: It is listed among a 28 countries with a LLIN gap.
    5. Mali: "LLIN gap for mass distribution: 4,473,801."
    6. Malawi: "LLIN gap for mass distribution: 600,000."
    7. Angola: "LLIN gap for mass distribution: 3,669,125 LLINs."
  • 8.

    "We liaise with a variety of sources including the Global Fund, in-country NGOs such as the Presidents Malaria Initiative, World Vision, UNICEF, Plan International, Concern Universal, and the relevant NMCP to establish a) who has been the source of the AMP estimate; b) how has it been arrived at; and c) what is the level of confidence of these groups it is accurate. We view this information as interesting and necessary but not sufficient for us to allocate nets." Rob Mather, email to GiveWell, August 8, 2012.

  • 9.

    "We will typically approach the NMCP (National Malaria Control Program) directly and a) ask if there is a gap; b) ask for data on i) the population, number of nets required, number of nets previously distributed, number of nets considered to still be ‘usable nets’, number of nets required to achieve universal coverage (and all of the above on a regional/district basis) and ii) data on existing funding commitments so we can see who else is funding nets and at what level." Rob Mather, email to GiveWell, June 29, 2011.

    Data in Against Malaria Foundation, "Malawi Universal Coverage Calculations (September 26, 2011)." In addition, AMF provided the email correspondence it had with the Malawi National Malaria Control Program, which involved extensive back-and-forth about the sources of the data and the assumptions used in the estimates of net needs.

  • 10.

    Tossa Kokou, email to Rob Mather, May 15, 2012.

  • 11.

    Rob Mather, phone conversation with GiveWell, July 19, 2012.

  • 12.

    Rob Mather, email to GiveWell, August 8, 2012.

  • 13.

    Against Malaria Foundation, "LLIN Distribution Proposal Form." For the Malawi distribution proposal, see Concern Universal, "Distribution Proposal for Ntcheu District (October 2011-February 2012)."

  • 14.

    Rob Mather, phone conversation with GiveWell, July 19, 2012.

  • 15.

    Concern Universal, "Pre-Distribution Survey Data"

  • 16.

    "Malaria case rate data, pre and post‐distribution: This information is crucial to being able to monitor the impact of the nets over time. We ask for the following information and confirmations: a) Please provide 12 months historic monthly malaria case rate information for each health clinic in the intended distribution area b) Please indicate for each health centre if this malaria data is via Rapid Diagnostic Testing Kit (RDTK) analysis or via clinical observation c) Please indicate if there is a plan to ensure each health centre will always have an adequate stock of RDTKs to ensure malaria case rate information can be continually gathered? d) Please confirm you will be able to gather and provide that information for the next four years?” Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 3.

  • 17.

    Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 2.

  • 18.

    Against Malaria Foundation, “Concern Universal, Ntcheu, Malawi Distribution”

  • 19.

    "Post‐Distribution Surveys (PDSs): These occur 6, 18, 30 and 42 months after a distribution. They assess three things: 1. Hang‐up % ‐ are the nets still being used? 2. Correct usage – are the nets being used properly? 3. Net condition – in what state are the nets? A 6‐months Post‐Distribution Survey (PDS‐6) involves sampling 50 households per sub‐distribution location for the above information. The PDS‐6 must be carried out between months 5 and 7 post the initial distribution. Similarly for subsequent annual PDSs. The PDS form is shown below.” Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 3.

    Rob Mather, email to GiveWell, November 20, 2012.

  • 20.

    "7,646 randomly selected households were visited and 15,735 nets surveyed (6% of the nets distributed) across the 37 health centre catchment areas.” Against Malaria Foundation, “Concern Universal, Ntcheu, Malawi Distribution.”

  • 21.

    Rob Mather, phone conversation with GiveWell, July 19, 2012.

  • 22.

    "Creating a source of funds for non-net costs might have benefit in allowing distributions to happen more quickly if there are no obvious or quick sources of funds for a potential distribution partner.

    I have three reasons for hesitating. First, to what extent would it compromise or make unclear our focus and message that ‘100% buys nets’? Second, would non-net costs rise as we are less well placed to evaluate what costs are reasonable? Third, would distribution partner performance be affected negatively as they would no longer have their own funds invested in a net distribution?

    It is worth noting we have always had non-net costs found, and quickly, by the distribution partner. However, we are now talking about larger distributions with greater associated non-net costs and this may change the speed at which non-net costs can be found." Rob Mather, email to GiveWell, November 16, 2012.

  • 23.

    Rob Mather, email to GiveWell, November 20, 2012.

  • 24.

    Rob Mather, phone conversation with GiveWell, July 19, 2012.

  • 25.

    GiveWell, "Notes from Site Visit with Concern Universal in Malawi (October 2011)."

  • 26.

    Concern Universal, "Pre-Distribution Survey Data"

  • 27.

    "For the subsequent verification visits (which Givewell witnessed) HSAs don't go into every household- instead they call a village register at a central location to check with the village leaders and individuals as to whether the data from the house to house survey is accurate. Givewell suggested that we should be doing some randomised checks on individual households at this verification stage- we have agreed and have now incorporated these into our verification model." Robin Todd, email to Rob Mather, November 18, 2011.

  • 28.

    Robin Todd, email to Rob Mather, November 18, 2011.

  • 29.

    "In 2010, Senegal and Cross River State in Nigeria worked on mop-up campaigns following earlier integrated campaigns targeting households with children under five years of age. In both countries, trained health workers or volunteers undertook a household registration to determine:

    • the total net need for each household (this was based on one LLIN for two people, rounding up in the case of odd numbers of people in the household, but Senegal also looked at the number of habitual sleeping spaces in each household)
    • how many nets each household already had
    • how many nets in each household were still viable (in Senegal this number was based on net condition, while in Cross River State, it was based on how long the net had been hanging)
    • how many new nets each household would need for full coverage

    Both countries had previously undertaken post-distribution surveys which showed high household coverage with LLINs, but during the mop-up exercise they experienced challenges with finding nets in households. Significantly lower numbers of nets were found (50—60 per cent) than would have been expected based on the surveys...
    In both countries, it seemed that families often hid nets once word spread that ownership of nets meant no new nets would be received. Despite efforts to encourage families to hang pre-existing nets prior to the household registration in Senegal, people hid nets in order to receive more." Alliance for Malaria Prevention, "Toolkit (Version 2.0)," Chapter 3, Pgs 5-6.

  • 30.

    Alliance for Malaria Prevention, "Conference Call Minutes (October 26, 2011)," Pg 20.

  • 31.

    Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 2.

  • 32.

    Against Malaria Foundation, “Concern Universal, Ntcheu, Malawi Distribution”

  • 33.

    GiveWell, “AMF March 2012 Update”

  • 34.

    “AMF has provided regular, public updates on the large, ongoing net distribution in the Ntcheu district of Malawi. Expected data collection has occurred and the distribution has proceeded close to schedule. AMF's distribution partner, Concern Universal, has been transparent about problems it has encountered, and seems to have a robust process to catch problems (such as attempts to steal nets) when they arise.” GiveWell, “AMF March 2012 Update”

  • 35.

    AMF lists the countries it has provided nets to at Against Malaria Foundation, "Countries Involved." The Malaria Atlas Project has compiled data on malaria risk by location at Malaria Atlas Project, "Endemic Countries."

  • 36.

    “Figures from Ntcheu District Hospital show that, in the twelve months to July 2011, 109,972 malaria cases were reported in under five children with a further 133,768 cases reported in the rest of the population.” Concern Universal, “Distribution Proposal,” Pg 1.

  • 37.

    "Post‐Distribution Surveys (PDSs): These occur 6, 18, 30 and 42 months after a distribution. They assess three things: 1. Hang‐up % ‐ are the nets still being used? 2. Correct usage – are the nets being used properly? 3. Net condition – in what state are the nets? A 6‐months Post‐Distribution Survey (PDS‐6) involves sampling 50 households per sub‐distribution location for the above information. The PDS‐6 must be carried out between months 5 and 7 post the initial distribution. Similarly for subsequent annual PDSs. The PDS form is shown below.” Against Malaria Foundation, “How We Work With Distribution Partners,” Pg 3.

    Rob Mather, email to GiveWell, November 20, 2012.

  • 38.

    "7,646 randomly selected households were visited and 15,735 nets surveyed (6% of the nets distributed) across the 37 health centre catchment areas.” Against Malaria Foundation, “Concern Universal, Ntcheu, Malawi Distribution.”

  • 39.

    Against Malaria Foundation, “Ntcheu Update.”

  • 40.

    Against Malaria Foundation, "Malawi Universal Coverage Calculations (September 26, 2011)."

  • 41.

    GiveWell, “AMF Update (March 2012).”

    Exact net figure from Rob Mather, email to GiveWell, November 20, 2012.

  • 42.

    Against Malaria Foundation, "Malawi Universal Coverage Calculations (September 26, 2011)."

  • 43.

    "Health Surveillance Assistants (HSAs) are Government extension workers- they are the lowest tier of government presence in the decentralized health system." Robin Todd, email to GiveWell, April 27, 2012.

  • 44.

    "As such they are the first line of response to any public health issues in communities. Their job involves disseminating health related information (such as encouraging people to make use of sanitary facilities, go for immunizations, sleep under mosquito nets etc.), carrying out sanitation and hygiene campaigns and sending data on take-up of facilities to the District Council, conducting basic nutrition support, weighing children and reporting levels of stunting and wasting, detecting common communicable diseases and reporting these to clinicians and other health providers, implementing immunization campaigns etc. As you can see being involved in universal net distribution fits very well with their core public health responsibilities. HSAs need to have a primary school completion certificate as a minimum but the majority of them will have O-Levels (exams sat by pupils aged 16 if they have completed the school system at the recommended pace). Once they have been selected as HSAs they are sent on an initial 9 months intensive training course where they will be trained in many aspects of public health including how to recognize common diseases, how to administer immunizations etc." Robin Todd, email to GiveWell, April 27, 2012.

  • 45.

    For numbers of LLINs distributed in the past, see Against Malaria Foundation, "Net Cost for GiveWell." Rob Mather told us AMF aims to distribute one million nets per year. Rob Mather, conversation with GiveWell, May 11, 2012.

  • 46.

    Note that Concern Universal spreadsheet has a tab ("What has changed") that compares the pre-campaign cost estimate for the Malawi campaign to the actual costs reported after the campaign.

    AMF's FY2012 financial statements were available at the time we updated this review. Because AMF's organizational costs (including donated services) were similar in FY2012 to what they were in previous years (about 20% lower in FY2012), and because these costs account for a fairly smaller percentage of the overall cost of providing a LLIN, we have not taken the time to update our spreadsheets and calculations.

  • 47.

    AMF told us (Rob Mather, phone conversation with GiveWell, May 11, 2012) that it expects the purchase cost for an LLIN to fall from its historic average of USD 4.23. We do not know what the future price will be. We have used the historic average cost per LLIN until we have more exact information. This is a conservative estimate.

    See Against Malaria Foundation, "Net Cost for GiveWell" for details of how the $4.23 average was calculated.

  • 48.

    We have used the value of donated services in FY 2011 as an estimate of the value of donated services in 2012-2013. The value of donated services was higher in 2007-2010. AMF told us that the value of donated services in the past is unlikely to be representative of this value in the future for two reasons: (1) AMF has become more efficient, including automating financial information for reports, consolidating bank accounts, etc. (2) Estimates of the value of donated services in 2007-2010 were likely overestimates of the actual value of services received; in 2011 AMF began to pay closer attention to estimating the value of services donated correctly. Rob Mather, phone conversation with GiveWell, May 11, 2012.

  • 49. AMF’s organizational costs were paid for in part by donated services and in part by private funding. The part covered by donated services totaled £71,837 in equivalent costs in fiscal year 2011. They were constituted by:
    • General office (audit fee): £10,500
    • Financial office: £2,500
    • Information technology: £34,232
    • Marketing: £3,307
    • Banking support: £10,200
    • Legal/professional support: £11,098

    Against Malaria Foundation, “Audited Financial Statements (2011),” Pg 15.

  • 50. GiveWell has factored in the cost of an annual CEO salary, though the founder of AMF, Rob Mather, opts to not receive a salary for his work. We have done this because we want to include all costs in our total estimate, including those for volunteer time. We have used the figure of $100K, which is our own estimate roughly in line with non-profit salary surveys. We used surveys for NYC salaries because we did not find survey data for London; we assume that the two cities have roughly comparable salary ranges. Nonprofit Staffing, "NY Salary Report 2011."
  • 51. AMF told us that its organizational costs are expected to be in line with what it has spent on average over the course of its history. Rob Mather, phone conversation with GiveWell, May 11, 2012.
  • 52. AMF’s organizational costs were paid for in part by donated services and in part by private funding. The part covered by private funding totaled £95,625 in fiscal year 2011. In 2011, these costs were constituted by:
    • General office: £1,797
    • Information technology: £72,736
    • Marketing: £21,092

    Against Malaria Foundation, “Audited Financial Statements (2011),” Pg 15.

  • 53.

    The distribution staff, called health surveillance assistants (HSAs), would be paid by the Malawi government regardless of whether the distribution took place. We are including the cost of their salaries for the duration of the distribution because we seek to include all costs of the distribution, regardless of who is paying the costs. We do this to avoid the problem of seeking leverage in charity.

  • 54.

    For the Malawi distribution, the total Concern Universal and government costs were $246,188.80. All costs were paid by Concern Universal (and Irish Aid, which supported Concern Universal) except for the health surveillance assistant (HSA) salaries, which were paid by the Malawi government. The break-down of costs was:

    • $132,382.70 for all distribution costs, including monitoring. Monitoring includes pre-distribution surveys, photos/videos of distributions and follow-up surveys four times (6-42 months) after distributions take place.
    • $75,522 for net shipping costs.
    • $31,144.55 for costs incurred by the government for health surveillance assistant (HSA) salaries. (We are including these costs, even though they were not paid by Concern Universal, because we want to capture the full cost of the distribution.)
    • $7,139.54 for Concern Universal organizational costs, including 10% of the Finance and Admin Manager and Country Director's time for 4 months, Ntcheu office overhead (25% of total overhead for 4 months), and the salaries of borehole drilling crew who worked on the distribution for 4 months.

    The "Concern Universal organizational costs" are staff salaries and office costs that Concern Universal would have paid for regardless of whether distribution took place. We include the fraction of the office overhead costs and Concern Universal staff costs that were dedicated specifically to work on the distribution rather than Concern Universal's other projects. We are including these costs because we seek to include all expenses involved in the distribution, whether or not they are costs that an organization would have paid otherwise. We seek to include all costs to avoid the complications of relying on leverage in charity.

    Full details in the Concern Universal Costs spreadsheet . We assume that these costs will be similar for future distributions, and will update this figure as more data becomes available in the future.

  • 55.

    In some cases, AMF had services donated. We include the cost of these donated services, had AMF paid for them rather than receiving them for free.

  • 56. We do not exclude any of Concern Universal's costs because we believe that Concern Universal's costs (including staff, office, transportation, etc) increase with further nets. It's possible that further nets could be distributed at slightly less cost; if this were true, our estimate of costs would be conservative i.e., would tend to overestimate rather than underestimate Concern Universal's costs.
  • 57.

    Against Malaria Foundation, “Net Cost for GiveWell Updated.”

  • 58.

    “If funds for nets significantly increase, no, we will not… have to increase the size or cost of our operations activity. In fact, we see the opposite trend. This is because we are now funding fewer, larger distributions and this reduces administration and other operational activities and costs.” Rob Mather, email to Emily Brotman, November 8, 2011.

  • 59.

    GiveWell, “Cost-effectiveness Analysis for LLIN Distribution Updated”

  • 60.

    Against Malaria Foundation, “Audited Financial Statement (2011)” and “Net Cost for GiveWell Updated.”

  • 61.

    Rob Mather, phone conversation with GiveWell, May 11, 2012.

  • 62.

    Robin Todd, emails to GiveWell, March 25, 2012, March 30, 2012 and April 3, 2012. Robin Todd, phone conversation with GiveWell, March 20, 2012. Concern Universal, “Concern Universal Costs Updated.”

  • 63.

    Against Malaria Foundation, "Room for More Funding Analysis (September 2011)."

  • 64.

    "Distributions not progressed
    a)Malawi: Through Jan to Jun 2102 we offered to contribute 600,000 to 1 million nets to close the gap in nets required to achieve universal coverage. The NMCP would not progress discussions about sharing the extensive pre-distribution registration data they said they had.
    b)Togo: Through Jan to Aug 2012, we offered to contribute 500,000 nets when there were no other sources of funding to fill the net gap to complete a universal coverage campaign. The offer was not accepted. The NMCP would not agree to a Pre-Distribution Registration Survey (PDRS) being carried out even though our Distribution Partner had agreed to do so and had agreed to fund it. The NMCP said they had conducted a universal coverage campaign on the basis of one net per two people and they did not want to change the basis of the distribution now. We said if the data was good, we could potentially agree to that. We asked to see the data. No data was forthcoming. A PDRS was required in our view as the existing data the NMCP had was:

    1. for the number of people per household rather than the number of sleeping spaces (although not the best basis, we could have agreed to this if the data had been recently collected and accurate, with additional accuracy being sacrificed in order to achieve an immediate distribution and people protected); and
    2. gathered at least six months previously when discussions first began and at least 12 months old when we withdrew from discussions.

    c)Mali: Unrest and fighting, including killings, have led to security fears and safety concerns. Our distribution partner has on several occasions withdrawn staff from the country. This has put our potential contribution of nets on hold. New timing is unknown. The need is for 4 million nets."
    Rob Mather, email to GiveWell, November 16, 2012.

  • 65.

    On November 16, 2012, AMF's website stated that it had received enough funding, over its history, to fund 2,876,940 nets and that it had purchased 1,611,033 LLINs as of June 2012 (the end of its fiscal year). Against Malaria Foundation, "Financial Information (as of November 16, 2012)."

    It is our understanding from conversations with AMF that it did not purchase any LLINs between June and November 2012.

  • 66.

    Rob Mather, email to GiveWell, November 16, 2012. Mr. Mather has shared further details with us but has asked us to keep this information confidential for the time being.

  • 67.

    "Creating a source of funds for non-net costs might have benefit in allowing distributions to happen more quickly if there are no obvious or quick sources of funds for a potential distribution partner.

    I have three reasons for hesitating. First, to what extent would it compromise or make unclear our focus and message that ‘100% buys nets’? Second, would non-net costs rise as we are less well placed to evaluate what costs are reasonable? Third, would distribution partner performance be affected negatively as they would no longer have their own funds invested in a net distribution?

    It is worth noting we have always had non-net costs found, and quickly, by the distribution partner. However, we are now talking about larger distributions with greater associated non-net costs and this may change the speed at which non-net costs can be found." Rob Mather, email to GiveWell, November 16, 2012.

  • 68.

    "There is a need for research into, and monitoring of, mosquitos becoming resistant to the insecticide used in LLINs. Not enough is being done. With our data-driven approach to net distributions, including significant pre-distribution work and post-distribution follow-up, we have the distribution profile alongside which a research project could fit well. By comparison, at least many other net distributions do not have a) the same recording of how many nets go where and b) the same degree of post-distribution monitoring – which involves visiting households to assess nets. Both these elements are cited as important to an insecticide resistance research project.

    Note, we would not carry out such research. That would be done by the Liverpool School of Tropical Medicine (or similar organisations) with whom we have had discussions and who are keen to work alongside AMF. They face lengthy application procedures when applying for funds for such projects. The approval timeframes do not match the time we have from agreeing to fund nets for a distribution and the distribution taking place. Initial baseline field research needs to be done pre-distribution. If AMF was to have a fund that could pay for this research, it might make the difference between the research happening, or not, around our distributions and over the next few years could make a meaningful contribution to this area of research.

    I am not convinced this is a path we should go down. I need to find out more."
    Rob Mather, email to GiveWell, November 16, 2012.

  • 69.

    Against Malaria Foundation, "Audited Financial Statements (2005-2012)."

Clicky Web Analytics