What is the evidence regarding the general effectiveness of LLIN distributions?

Evidence from small-scale, high-quality studies

The best evidence for the effectiveness of LLIN distributions comes from randomized controlled trials of insecticide-treated net campaigns, which are reviewed in two Cochrane reviews (Lengeler 2004; Gamble et al. 2006, Pgs 11-12). We have separately found, examined, and summarized the papers reviewed in Lengeler 2004.9The studies are mostly short-term, examining insecticide-treated nets (not necessarily LLINs) over a period of 6 months to 2 years.

Lengeler's (2004) meta-analysis, examining 22 studies,10 found:

• Mortality (5 studies): a statistically significant impact on all-cause mortality in children under 5, summarized as "5.53 deaths averted per 1000 children protected per year" with no clear dependence on one measure of the regional malaria transmission dynamics.11 2 studies attempted to examine malaria-specific mortality and found smaller or similar-sized effects, which the review author attributes to the difficulty of attributing mortality to malaria.12

  Note that the above figure ("5.53 deaths averted per 1000 children protected per year") is based on people who could be covered by the ITNs distributed, not on people who are confirmed to be using ITNs (i.e., the implication is that .00553 lives are saved for every child under five who could be covered by a distributed ITN, not that .00553 lives are saved for every child under five who is confirmed to be using an ITN).13

• Anemia (9 studies): statistically significant impacts on haemoglobin levels, about +1.3 g/L when insecticide-treated nets were compared to untreated nets and +5.7 g/L when insecticide-treated nets were compared to no nets.14 (More on how to interpret these g/L figures at our writeup on deworming.)
• Splenomegaly (enlarged spleen) (5 studies): about 23% protective efficacy (which we presume means that the nets reduced splenomegaly by 23%) when insecticide-treated nets were compared to untreated nets and 30% protective efficacy when insecticide-treated nets were compared to no nets.15
• Effects that were usually statistically significant in individual trials, but were not combined into summary analysis, for severe malarial disease,16 uncomplicated clinical episodes of malaria,17 prevalence of malaria parasites,18 high parasitemia (prevalence of malaria parasites in the blood),19 and nutrition-related measures (weight for age, weight for height, mean mid-upper arm circumference but not height-for-age or unspecified other measures).20

Gamble et al. (2006) focused on the effects on pregnant women; it examined fewer studies than Lengeler (2004) (6 vs. 24). It connected ITNs with statistically significant reductions in the risk of low birthweight and fetal loss (only in women with four or fewer previous pregnancies) and in placental malaria (overall), but not in anemia/haemoglobin measures.

We focus here on Lengeler (2004) because (a) it reviewed more studies; (b) it had a general-population focus and was thus more in line with the programs we seek to evaluate and the effects we seek to assess (particularly the effects on mortality).

What sorts of programs were carried out in small-scale studies?

We have examined and summarized the papers reviewed in Lengeler (2004),21 seeking to better understand the basic approaches of the programs that led to the results discussed above. We found:

• The details of the programs are often unclear.
• The programs' approaches varied; most consisted of free ITN distribution, or treatment of existing untreated nets with insecticide (effectively turning untreated nets into ITNs), but some involved promotion/marketing of nets and treatment.
• All of the studies examining child mortality involved distribution of ITNs to entire communities, not just to children under five. Only two studies distributed ITNs specifically to children under five.
• In most studies, coverage of ITNs was very low prior to the program (though there was often high coverage of untreated nets), and very high afterward.
• Many studies report intensive measures to ensure that people used their ITNs consistently and properly - measures well beyond what we would expect to be feasible in a larger-scale distribution (and well beyond the measures taken by the Against Malaria Foundation). Sample quotes:22

  • "Each village was visited daily by a supervisor who checked the dilution of the permethrin and the progress of the installation."
  • "Care was taken to place the nets over all beds in each selected house"
  • "Mothers of the children in the study cohort were reminded weekly how to use the net. Nets which became torn or damaged were repaired or replaced. A survey was conducted every 4 weeks during the rainy season to determine whether the bed nets had been tucked in and the entry flaps placed correctly."
  • "After distribution, study staff went door-to-door to ensure that nets were hung properly."
  • "At mass meetings of the intevention group males, insecticide dipping procedures and net erection methods were demonstrated...Two months after bed net distribution, the teams revisited the trial families to give further encouragement."
• Usage does not appear to have been near-universal. Most studies report usage rates in the range of 60-80%, though some report 90%+ usage.
• Only two studies specifically reported both reported usage and actual usage as determined by surprise visits to homes. In one, actual usage was 70-73% while reported usage was 85%; in the other, reported usage was 97% and actual usage was near-universal among those whose homes the surveyors were able to observe.

The author of Lengeler (2004) has stated to us that few if any randomized controlled trials have been done since this review, and that few are likely to be done, since the efficacy of ITNs is well enough established that such studies could face challenges with ethics boards.23

How have larger-scale distributions compared to the programs addressed in these studies?

Funding for malaria control has increased substantially since 2004, making a large number of national scale-ups possible.24

As noted above, the studies discussed above may have differed substantially from what can be expected of an "average" ITN distribution. The review author notes this, stating, "the bulk of data in this review describe impact under ideal trial conditions (efficacy) rather than impact under large-scale programme conditions (effectiveness). While the difference between efficacy and effectiveness is likely to be small for certain medical interventions (such as vaccination or surgery), it can potentially be large for preventive interventions such as ITNs."25

In order to get a sense for how the large-scale performance of LLINs has compared with the promise of the smaller-scale studies discussed above, we have asked the following key questions:

1. How have officially delivered LLINs matched up with LLINs confirmed to be in use by actual households?
2. What has been the general pattern of LLIN usage, i.e., have people used their LLINs?
3. What has been the connection between LLIN usage and drops in malaria deaths and morbidity?

We feel that malaria scholars have used credible data to provide helpful answers to #1 and #2, and that #3 is hard to answer due to issues with malaria data and the difficulty of isolating the impact of LLINs. Details follow.

From LLIN distribution to LLIN ownership

The 2010 World Malaria Report states:

The World Health Organization's Global Malaria Programme has compared the number of nets delivered to data from national-level household surveys in order to ascertain the discrepancy between "how many LLINs are distributed by manufacturers" and "how many LLINs are actually owned by households."27 While the results are reported in a presentation and not a scholarly paper, it was fully transparent about its data sources and methods and we were able to perform a confirming spot-check on how it was generating its estimates.

The key figures compare:28

• The number of ITNs owned by households, estimated based on (a) the number of ITNs owned per household (from national Demographic and Health Surveys29); (b) the number of people per household (from the same surveys); (c) the total number of people (using United Nations population estimates).30
• The number of LLINs distributed, according to manufacturers.
• The number of LLINs distributed, according to National Malaria Control Programs.
• The expected discrepancies based on the authors' preferred model, which assumes that 8% of distributed LLINs "decay" (fall out of use) within the first year after distribution, that 20% decay within the first two years, that 50% decay within the first three years, and that 100% decay within the first four years.31

They find:

• The decay model's prediction is relatively accurate for Kenya and Senegal.
• For Uganda, the number of ITNs in use exceeds the number delivered over the last three years. It's possible (our speculation) that this is a function of ITNs lasting longer than anticipated, of delivery data failing to capture all ITNs delivered, or of survey respondents' exaggerating their ownership of ITNs.
• For Liberia and Sao Tome and Principe, decay appears substantially worse than what the decay model and manufacturer data would predict (there are fewer than half as many ITNs in ownership as the model would predict), and National Malaria Control Program data is not available. We speculate that this may be a function of problematic national control programs.
• For Sierra Leone and Rwanda, substantially fewer ITNs were found than the decay model and delivery data would predict (less than half as many for Sierra Leone; about 75% in Rwanda).

Overall, the decay model overestimates coverage for four countries (two of which may have problematic National Malaria Control Programs), underestimates coverage for one, and is roughly accurate for two. Given the many uncertainties in extrapolating from delivered to owned ITNs, we find these figures surprisingly close, and so we are inclined to accept the standard decay model as a default best guess for cases where the national control program is believed to be functioning properly.

Bottom line - we believe the standard decay model is a reasonable (though far from fully reliable) best guess at the discrepancy between LLIN delivery and LLIN ownership, and we use it in our cost-effectiveness calculations below. Note that this model implies just over 2 years of effective use per LLIN;32 LLINs are designed to last substantially longer (and in some cases have been claimed to last more than seven years).33 So the decay model is likely capturing some of the impact of wastage (i.e., ITNs that aren't successfully delivered to people in the target population), as well as normal decay.

From LLIN ownership to LLIN usage

The 2010 World Malaria Report tabulates results from national population surveys and compares (a) the percentage of the population that could theoretically be protected by owned ITNs, based on an assumption that each ITN protects two people;34 (b) the percentage of the population that reports using an ITN.35 It finds 2% ownership/0% use for Swaziland, apparent 100% use for Rwanda, and just over 100% use for Democratic Republic of the Congo and Madagascar (this could be a function of more than 2 people covered by each ITN in some cases, combined with possible over-reporting of usage in surveys). Putting these countries aside, the 8 remaining countries have 40%-85% apparent usage using this model (all but one have 69%+ apparent usage). Using an assumption of 1.8 people covered per ITN, rather than 2 (as we do in our cost-effectiveness analysis below), would result in 44-94% estimated usage for these countries (77%+ for 7 of 8). The charts on page 22 show strong relationships between the two figures (apparent ITN ownership and apparent ITN use).

It is possible that survey data overstates usage; in our discussion of small-scale studies above, we cite one study where actual usage (as assessed by spot visits to homes) was 70-73% while reported usage was 85%. Applying this ratio to the reported figures assuming 1.8 people covered per net generates a range (for the seven "mid-range" countries discussed above) of 65%-80% usage,36 similar to the range reported in small-scaled studies.37 These figures imply usage of ITNs that is, very broadly speaking, as good as what was found in the small-scale studies discussed above, though this data and our analysis of it could easily be unreliable.38

From LLIN usage to reduced malaria burden

Funding for malaria control has increased substantially since 2004, making a large number of national scale-ups possible.39 In most cases it appears that these scale-ups have included significant funding spent on LLIN distribution but also (concurrently) significantly increased spending on other malaria control measures,40 making it difficult to isolate the impact of LLIN distribution.

A discussion of these issues is available at our page on the macro evidence for a connection between ITN distribution and malaria burden; we exclude it here for the sake of brevity. In brief: we find the available data to be easily consistent with the idea that scaling up ITN coverage often - or even always - results in a reduction in malaria burden in line with what small-scale studies would predict. However, we do not find this data to provide strong affirmative evidence of this idea. We believe we will improve our understanding of this topic when we speak with the authors of the World Malaria Report.

Possible developmental effects

As noted above, ITN distribution appears to have benefits other than reduced mortality, such as reduced anemia (an effect size larger than what we have seen for deworming). In addition, Bleakley 2010 makes a case that reducing the burden of malaria may have a lasting impact on children's development, and thus on their ability to be productive and successful throughout life. This is a similar paper, with similar results, to one of the two major pieces of evidence for the developmental impact of deworming.

Bleakley 2010 analyzes a number of attempts to eliminate malaria in the Americas during the early-to-mid 20th century, focusing on the U.S., Brazil, Columbia, and Mexico. He concludes:

There are good reasons to be cautious in using this study as evidence relevant to ITN distribution:

• The programs studied were very different from LLIN distribution campaigns: they focused on mosquito control, and did not involve bednets.42
• The context was very different from that of modern-day ITN distributions: the campaigns took place in the U.S.A. of the 1920s and Latin America of the 1950s, where the impact of malaria could have been very different from the consequences in the modern-day developing world (and the infections themselves may have been quite different as well).
• Bleakley 2010 is not an experimental study, but a retrospective one. That is, rather than setting out to answer a question by collecting new data, the author collected and analyzed a large amount of pre-existing data, raising strong possibilities for publication bias. We believe it is unlikely that this paper would have been published in a major economics journal if it had simply concluded that there was no strong evidence for major benefits of malaria eradication. We are generally very hesitant to use papers of this nature in our work. This worry is enhanced by the use of multiple different outcomes and measures of malaria prevalence across the different countries studied, raising the possibility that the author could have selected those specifications that make the outcomes most interesting (in this case, positive for malaria eradication).

That said, we believe the paper merits some weight on the question of developmental benefits, because:

• It has a plausible strategy for separating the effects of malaria infection from effects of other things (such as poverty) that may correlate with malaria infection. Specifically, it exploits the fact that the eradication campaigns caused a relatively rapid drop in malaria infection rates (which was plausibly exogenous because of technological advancement in the creation of DDT) and that areas with higher initial malaria prevalence saw greater falls in malaria prevalence.43 Thus, it seems possible that a connection between the fall in malaria prevalence and positive life impacts - coinciding with the timing of the campaign - could be attributed specifically to malaria eradication, and not to other factors.
• It uses graphs to illustrate a relationship between malaria prevalence and later-in-life income that, while varying across countries, shows that the correlation between baseline malaria rates and adult income was negative and fairly constant, then turned into "zero effect" (when adjusted using a set of controls), coinciding well with the timing of the eradication campaign. It seems difficult to explain this pattern except by attributing the change to the drop in malaria prevalence, though it is worth noting that the relationship in Colombia becomes positive, which is unexpected.44
• It addresses multiple alternate possible explanations for its observations, including a fairly robust set of controls, though they vary by country (because there is no consistent set of cross-country and cross-time data).45
• It covers a number of very large-scale campaigns. While randomized controlled trials allow for a cleaner connection between a program and its effects, a large-scale study like this seems likely to be less dependent on idiosyncratic aspects of a particular mini-program designed to be studied.

Bleakley also finds effects on school enrollment that vary by country.46

Possible negative/offsetting impact

• Allergic reactions. We have heard anecdotes of minor allergic reactions (e.g., itching skin) to LLINs. We have located no mention of this issue in the Cochrane reviews discussed above and have located very little information on it in general. We did discuss it with program staff during our visit to an ongoing LLIN distribution (we were told that this issue affects few people and can be avoided by letting an LLIN hang for a few days before use). We believe this to be a very minor concern.
• Equity concerns. It is possible that LLIN distribution may cause conflicts if it is not perceived as fair and equitable. We have seen very little evidence on whether this is an issue. We feel that the Against Malaria Foundation takes appropriate steps to mitigate this concern.
• Could LLINs become less effective? It is possible that environmental changes, particularly changes in the size and insecticide-susceptibility of mosquito populations, could affect how well LLINs work. We have found no information on this issue.

Two key issues around LLIN distributions

Targeted vs. universal coverage

It appears that there has been a shift in emphasis over the last several years from targeted coverage (aiming primarily to cover children under five and pregnant women with ITNs) to universal coverage (aiming to protect everyone in a community with ITNs).

The 2004 Cochrane review discussing the impact of ITNs on under-5 mortality (discussed above) appears to advocate primarily for coverage of children under five,47 and as of 2006 this seemed to be the focus of the Roll Back Malaria Partnership, the United Nations Millennium Development Goals, and the US President's Malaria Initiative as well.48 However, in 2007 the World Health Organization issued a recommendation for universal coverage that appears to have been an explicit change in position.49

We did a brief investigation of the arguments for targeted vs. universal coverage, because we wondered whether the Against Malaria Foundation's focus on universal coverage was justified. (It seemed to us that if the main benefit of ITNs is in reducing mortality for children under five, then targeting children under five could achieve most of the benefits of universal coverage, for a fraction of the cost.) Our investigation consisted of:

• Speaking with Christian Lengeler, the author of the Cochrane review of the evidence regarding ITN distribution discussed above. (This Cochrane review appears to us to be the most widely cited evidence of the efficacy of ITNs.50) We have published our notes from this conversation.51
• Speaking with two scholars that Professor Lengeler referred us to on this question, Dave Smith and Thomas Smith. We have published notes from each of these conversations.52
• Reviewing literature that these scholars referred us to.

Our conclusions are:

• The evidence for the efficacy of ITNs is based on studies of universal coverage programs, not targeted programs. In particular, all five studies relevant to the impact of ITNs on mortality involved distribution of ITNs to the community at large, not targeted coverage (see above). (We confirmed this in our examination of the individual studies cited in the Cochrane review.53) Thus, there is little basis available for determining how the impact of ITNs divides between individual-level effects (protection of the person sleeping under the net, due to blockage of mosquitoes) and community-level effects (protection of everyone in communities where ITN coverage is high, due to reduction in the number of infected mosquitoes, caused either by mosquitoes' being killed by insecticide or by mosquitoes' becoming exhausted when they have trouble finding a host).
• The people we spoke to all believe that the community-level effect of ITNs is likely to be a significant component of their effect, though none believe that this effect has been conclusively demonstrated or well quantified.54
• There is some empirical evidence suggesting that the community-level impact of ITNs is significant.
  • We reviewed one study, Hawley et al. 2003, which examined data from a randomized study of ITNs and compared (a) households not selected to receive ITNs but living near other households that had been selected to receive ITNs to (b) households not selected to receive ITNs living far from other households that had been selected to receive ITNs.55 Non-ITN households were divided into quartiles based on proximity to ITN households, and statistically significant trends were observed for child mortality, hemoglobin level and percentage anemic (e.g., non-ITN households showed more favorable performance on these measures as they got closer to ITN households), though the trends were not significant for two other measures of malaria burden.56 For the set of non-ITN households closest to ITN households, malaria outcomes were similar to those for ITN households themselves.57 We find this study credible, although we note that this was a separate analysis of data from a trial of ITN efficacy, so it may be susceptible to publication bias.
  • This study cites eight other studies arguing for significant community-level effects on malaria transmission and one study arguing against this idea. Of these nine, only two (both arguing for community-level effects) examine measures of malaria morbidity/mortality as opposed to simply transmission.58 We have not examined these studies.
  • Counter-evidence comes from the Cochrane review focused on pregnant women (which is not the main one we have discussed):
    

    The most recent trial from western Kenya by Njagi et al. is informative in this respect, as it is the only trial that compared the effects of ITNs versus no nets using simple randomisation by individual in an area with low ITN coverage (little or no mass effect) [24,29]. This trial and the community-randomised trial by ter Kuile et al. [27] were conducted simultaneously in contiguous areas with similar malaria transmission at baseline, and similar socioeconomic and educational status and ethnicity of the trial population. The effect estimates were similar between the two trials (in women not randomised to IPTp-SP), suggesting that ITNs may work equally well when provided to individuals as part of antenatal care in the second trimester or when provided to entire communities.59

The scholars we spoke with also pointed us to papers attempting to model malaria transmission mathematically.

• One such paper, Killeen et al. (2007), that we were pointed to by two of the scholars we spoke to and has been described as a particularly influential paper,60 uses a relatively detailed model of mosquito behavior, which we have not independently vetted and assessed, in order to conclude that "high (80% use) but exclusively targeted coverage of young children and pregnant women (representing <20% of the population) will deliver limited protection and equity for these vulnerable groups. In contrast, relatively modest coverage (35%–65% use, with this threshold depending on ecological scenario and net quality) of all adults and children, rather than just vulnerable groups, can achieve equitable community-wide benefits equivalent to or greater than personal protection."61
• Smith et al. (2009) proposes a model for the relationship between ITN coverage and effective protection for the community. This model implies that each incremental increase in ITN coverage is more valuable to the community than the previous increase (e.g., going from 60% to 70% coverage would be more valuable than going from 50% to 60% coverage, which in turn would be more valuable than going from 40% to 50% coverage).62

We asked one scholar whether there were any prominent papers in this category making the opposite argument (that community-level effects are likely to be insignificant) and he replied that he did not know of any.63

We do not find the evidence for community-level impact to be conclusive, but we believe that the best interpretation of the available evidence suggests at least some community-level impacts, consistent with the consensus of the scholars we spoke to and the World Health Organization.

Free vs. cost-recovering distributions

There has been some debate about whether ITNs should be sold or given freely, with some arguing that selling them (even for highly subsidized prices) may improve the likelihood that they get to people who will use them. We believe that the weight of the (limited) available evidence supports giving out ITNs rather than selling them. Evidence implies that charging a fee has significantly reduced demand for the product, without leading to corresponding increases in utilization rates (and has not significantly impacted the costs of the program).

• One high-quality study evaluated the impact of user fees on net purchase and on utilization of the net. A program distributing nets at prenatal clinics in Kenya found that increased prices (from $0 to $0.75, the price at which they were sold) reduced demand by approximately 75%,64 but was not associated with higher rates of utilization.65
• A review of high-quality studies on this general issue cites two other studies. One found that charging for deworming drugs significantly reduced demand while raising little revenue and failing to improve the targeting of recipients66; another found that charging for water disinfectant "led to a rapid drop-off in take-up, with no evidence of increased targeting to the most vulnerable" while slightly increasing utilization rates.67
• The study on The Gambia (discussed above) reports that individuals given free nets in year one, that were then asked to pay for insecticide to retreat them in year two, resulted in a significant reduction in coverage and a rise in child mortality.68
• The study on Kenya's program (discussed above) reports that Kenya began selling nets at substantially high prices starting in 2002.69 Net coverage rates remained extremely low, at 25%,70 so in 2004, Kenya sold heavily subsidized nets. 71 In 2006, the program began distributing nets for free,72 raising net coverage to 79% between 2006 and 2007.73
• As discussed above, there is a case that ITNs provide community-level protective benefits, so increasing the level of coverage in a community could be beneficial to everyone, not just ITN users. This gives some additional reason for giving out ITNs for free.

How cost-effective is LLIN distribution?

We provide a spreadsheet74 that estimates the (a) cost per life saved, (b) cost per person protected per year, and (c) cost per child under 14 protected per year. Cells highlighted in green represent particularly debatable and/or variable parameters; readers can change these (or any other figures) to see how the outcomes are affected.

Our analysis is relatively simplified and unlikely to capture all of the key issues. Some key assumptions and choices made in our model:

• We assume that each effective year of protection for children under five results in 0.00553 lives saved, consistent with the literature review discussed above. Note that this figure is based on people who could be covered by the ITNs distributed, not on people who are confirmed to be using ITNs (i.e., the review discussed above implies that .00553 lives are saved for every child under five who could be covered by a distributed ITN, not that .00553 lives are saved for every child under five who is confirmed to be using an ITN).75 Our use of this figure entails a couple of implicit simplifications:
  • We do not model variations in the transmission setting (prevalence and behavior of mosquitoes, presence of cattle, etc.). Note that the review we are using states that its ".00553 lives saved per child protected" figure does not appear sensitive to one measure of transmission dynamics, because ITNs are less effective in high-transmission settings but there are also more deaths to avert.76
    
  • Our use of this figure also implies that each person-year of protection provides an equivalent protective effect - e.g., going from 80% to 81% coverage has the same effect as going from 0% to 1% coverage. The information we have (discussion above) suggests that ITNs are actually more effective at higher levels of coverage (because part of their effect is on community-level transmission of malaria); this in turn implies that distributing ITNs in an area where coverage is already reasonably high should be more cost-effective than distributing ITNs in an area of low prior coverage, as in the small-scale studies. However, we are not confident in this dynamic and do not model it.
• LLINs are assumed to last 2.22 years on average, consistent with the decay model discussed above that we believe is reasonably well-fitted to available data on LLIN distributions and LLIN ownership. We assume that this decay model accounts for a substantial amount of the extent to which LLINs are lost, not delivered, or discarded. We provide a less optimistic estimate where LLINs are assumed to last 1 year on average (note that the evidence on ITN effectiveness is generally limited to a time frame of one year); we provide a more optimistic estimate where LLINs are assumed to last 3 years on average (this could be warranted if Against Malaria Foundation distributions result in fewer lost/undelivered/discarded LLINs).
• We assume that LLIN usage is in line with what was observed in the small-scale studies discussed above. We believe this is an appropriate assumption based on the available data, as discussed above. Those who believe that effective usage is likely to be lower can model this assumption using the "Wastage" cell in our sheet (which allows one to specify a percentage of ITNs that are lost, misused, or otherwise paid for but unimpactful, beyond the already-implicit assumptions in the model). Our model may not sufficiently adjust for this sort of additional "wastage," if (as discussed above) ITNs are more effective at higher levels of prior coverage.
• Rather than explicitly model usage, we base our calculation only on coverage, using a "lives saved per person-year of protection" figure that is itself based on coverage, not usage. This is why, when calculating pre-existing coverage of the population, we take reported usage figures and divide them by 70% (this assumes that coverage is greater than usage such that usage = 70% * coverage).
• We model community-level effects crudely. We use a single figure (referred to in this paragraph as X) for the "percentage of the effects of ITNs that comes from community-level effects." If coverage of children under 5 is C1, and coverage of the population as a whole is C2, then we model the "effective coverage" of children under 5 as (X*C2+(1-X)*C1). In other words, when 10% of the ITN effect is presumed to be from community-level effects, this means that the "effective coverage" of children under 5 comes from taking a weighted average of the under-5 ownership and the community ownership, with 90% weight on the former and 10% weight on the latter. Using this model, and 50% for X, would have an implication similar to that of the more sophisticated model we discuss above.

Our best-guess estimate, contained in the first column, comes out to just under $2,000 per life saved. This does not include other benefits of ITNs. We also show the number of children under 14 who are protected for each life saved, since protection may yield non-mortality-related benefits comparable to those of deworming, and we provide some rows that account for this additional benefit in order to allow more intuitive comparisons between the two interventions.

We also provide reference points for conversions to the cost per disability-adjusted life-year (DALY), a common metric in public health. We have found this metric to largely add more confusion than clarification,77 but we provide this information for those who find it helpful. Note that our cost per DALY figures are based only on lives saved and do not account for other benefits.

Is there room for more funding in LLIN distribution?

As discussed above, we have what we believe to be fairly comprehensive data on the number of LLINs distributed to each country over the last several years. By combining this with different versions of the decay function discussed above and different estimates of the population at risk for malaria, we have estimated how many LLINs would still be needed to achieve 100% (theoretical) coverage, for each country as of the end of 2010 and 2011.78

Regardless of which version of the calculation we use - even when assuming that LLINs remain 100% in use for 3 years after distribution and when using the lowest "population at risk" figures we have - we see substantial gaps in LLIN coverage. Each of Malawi, Mali and Ghana - three of the countries emphasized by AMF in its discussion of room for more funding - has a gap of ~1.5 million LLINs or more, thus each of these gaps would take $7+ million (using Against Malaria Foundation costs per LLIN distributed) to close. When examining the full list of countries listed by AMF, 8 of 11 have LLIN gaps under every set of assumptions we have modeled, and adding up the smallest modeled LLIN gap for each country with a gap results in a total LLIN gap of over 10 million (thus over $50 million would be needed to close it).

Our coverage analysis79 extends back to 2006 (using delivery data since 2004), showing a picture of fairly persistent LLIN gaps.

Sources

• African Leaders Malaria Alliance. Manufacturers nets delivered (2008-2011) (XLS).
• Alliance for Malaria Prevention. LLIN distributions since 2004 (XLS).
• Bleakley, Hoyt. 2010. Malaria eradication in the Americas: A retrospective analysis of childhood exposure. American Economic Journal: Applied Economics 2(2): 1–45. Abstract available at http://www.aeaweb.org/articles.php?doi=10.1257/app.2.2.1 (accessed December 21, 2011). Archived by WebCite® at http://www.webcitation.org/646LMomRP.
• Buj, Valentina. Distribution Strategies for LLINs:
  Scaling Up and Sustaining Coverage (PDF).
• Cohen, Jessica and Pascaline Dupas. 2007. Free distribution or cost-sharing: Evidence from a randomized malaria prevention experiment (PDF). Global Economy and Development Working Paper 11. Washington, DC: Brookings Institution.
• D'Alessandro, U., et al. 1995. Mortality and morbidity from malaria in Gambian children after introduction of an impregnated bednet programme. Lancet 345: 479-483.
• Gamble, C. L., J. P. Ekwaru, and F. O. ter Kuile. 2006. Insecticide-treated nets for preventing malaria in pregnancy. Cochrane Database of Systematic Reviews 2006, Issue 2. Summary available at http://www.cochrane.org/reviews/en/ab003755.html (accessed April 19, 2010). Archived by WebCite® at http://www.webcitation.org/5p6RmENZp.
• GiveWell. Cost-effectiveness analysis for LLIN distributions (XLS).
• GiveWell. Country-level charts of ITN coverage vs. malaria mortality (DOC).
• GiveWell. LLIN gap analysis (XLS).
• GiveWell. Summary of ITN RCTs (XLS).
• GiveWell. Tabulation of ITN coverage vs. malaria mortality (XLS).
• Hawley, William A. et al. 2003. Community-wide effects of permethrin-treated bed nets on child mortality and malaria morbidity in western Kenya (PDF). American Journal of Tropical Medicine and Hygiene 68(Suppl 4): 121–127.
• Jamison, Dean T. et al., eds. 2006. Disease control priorities in developing countries (PDF). 2nd ed. New York: Oxford University Press.
• Killeen, Gerry F. et al. 2007. Preventing childhood malaria in Africa by protecting adults from mosquitoes with insecticide-treated nets (PDF). PLoS Medicine 4(7): e229.
• Kremer, Michael, and Alaka Holla. 2009. Pricing and access: Lessons from randomized evaluations in education and health. In What works in development? Thinking big and thinking small, ed. Jessica Cohen and William Easterly, 91-119. Washington, DC: Brookings Institution Press.
• Lengeler, Christian. 2004. Insecticide-treated bed nets and curtains for preventing malaria. Cochrane Database of Systematic Reviews 2004, Issue 2. Summary available at http://www.cochrane.org/reviews/en/ab000363.html (accessed April 19, 2010). Archived by WebCite® at http://www.webcitation.org/5p6S58mQu.
• Lengeler, Christian. Author of Cochrane Review of insecticide-treated bed nets. Phone conversation with GiveWell (DOC), November 2, 2011.
• Lopez, Alan D., et al., eds. 2006. Global burden of disease and risk factors (PDF). New York: Oxford University Press.
• Measure DHS. Homepage. http://www.measuredhs.com/ (accessed November 26, 2011). Archived by WebCite® at http://www.webcitation.org/63SyrJFUf.
• Noor, Abdisalan M., et al. 2007. Increasing access and decreasing inequity to insecticide-treated net use among rural Kenyan children (PDF). PLoS Medicine 4(8).
• O'Meara, Wendy Prudhomme et al. 2010. Changes in the burden of malaria in sub-Saharan Africa. Lancet 10(8): 345-355. Full text available with free registration at http://www.thelancet.com/journals/laninf/article/PIIS1473-3099(10)70096-7/abstract (accessed November 25, 2011). Archived by WebCite® at http://www.webcitation.org/63SxZ9YKa.
• Otten, Mac, and Jo Lines. Where did the LLINs go? An analysis of data from 7 countries with the most recent surveys (2008-2009) (PDF).
• Quiñones, M.L. 1998. Permethrin-treated bed nets do not have a 'mass-killing effect' on village populations of Anopheles gambiae s.l. in The Gambia. Transactions of the Royal Society of Tropical Medicine and Hygiene 92(4): 373-378. Abstract available at http://www.ncbi.nlm.nih.gov/pubmed/9850383?dopt=Abstract (accessed November 26, 2011). Archived by WebCite® at http://www.webcitation.org/63SyGviBA.
• Smith, Dave. 2009. Predicting changing malaria risk after expanded insecticide-treated net coverage in Africa (PDF). Trends in Parasitology 25(11): 511-516.
• Smith, Dave. University of Florida Associate Professor of Zoology. Phone conversation with GiveWell (DOC), November 4, 2011.
• Smith, Thomas. Swiss Tropical Institute Head of the Biostatistics and Computational Sciences Unit. Email to GiveWell, November 8 2011.
• Smith, Thomas. Swiss Tropical Institute Head of the Biostatistics and Computational Sciences Unit. Phone conversation with GiveWell (DOC), November 8, 2011.
• Sumitomo Chemical. Olyset net. http://www.olyset.net/olysetnet/ (accessed November 25, 2011). Archived by WebCite® at http://www.webcitation.org/63SydfWcV.
• Tami, Adriana et al. 2004. Evaluation of Olyset™ insecticide-treated nets distributed seven years previously in Tanzania. Malaria Journal 3(19).
• WebMD. Malaria: Topic overview. http://www.webmd.com/a-to-z-guides/malaria-topic-overview (accessed November 26, 2011). Archived by WebCite® at http://www.webcitation.org/63SyXKdZk.
• World Health Organization. Insecticide-treated mosquito nets: A WHO position statement (2007) (PDF).
• World Health Organization. Profiles: 31 high-burden countries (2009) (PDF).
• World Health Organization. Publications on insecticide-treated materials: Guidelines. http://www.who.int/malaria/publications/insecticide_treated_materials-gu... (accessed November 26, 2011). Archived by WebCite® at http://www.webcitation.org/63SynNTsK.
• World Health Organization. World malaria report (2010) (PDF).

• 1.

  "In Africa, malaria accounts for an estimated 25% of all childhood mortality below age five, excluding neonatal mortality (WHO 2003)." Lengeler 2004, Pg 2.

• 2.

  "Human infection begins when the malaria vector, a female anopheline mosquito, inoculates plasmodial sporozoites from its salivary gland into humans during a blood meal. The sporozoites mature in the liver and are released into the bloodstream as merozoites. These invade red blood cells, causing malaria fevers. Some forms of the parasites (gametocytes) are ingested by anopheline mosquitoes during feeding and develop into sporozoites, restarting the cycle." Jamison et al. 2006, Pg 413.

• 3.

  "Most malaria infections cause symptoms like the flu, such as a high fever, chills, and muscle pain. Symptoms tend to come and go in cycles. One type of malaria may cause more serious problems, such as damage to the heart, lungs, kidneys, or brain. It can even be deadly." WebMD, "Malaria: Topic Overview."

• 4.

  See Jamison et al. 2006, Pg 416, Table 21.3 for estimates of cases of hearing impairment, visual impairment, epilepsy, etc. caused by malaria.

• 5.

  "Using mosquito nets as a protection against nuisance insects was practiced in historical times (Lindsay 1988). During World War II, Russian, German, and US armies treated bed nets and com- bat fatigues with residual insecticide to protect soldiers against vector-borne diseases (mainly malaria and leishmaniasis) (Curtis 1991). In the late 1970s, entomologists started using synthetic pyrethroids: their high insecticidal activity and low mammalian toxicity made them ideal for this purpose.

  In the 1980s, studies of ITNs showed that pyrethroids were safe and that ITNs had an impact on various measures of mosquito biting (such as the proportion of mosquitoes successfully feed- ing on humans and the number of times a mosquito bit humans in one night). These studies showed that pyrethroids worked by both repelling and killing mosquitoes. In addition, researchers determined optimal doses of various insecticides with different materials (Curtis 1991; Curtis 1992a; Curtis 1996; Lines 1996; Rozendaal 1989a). The cost-effectiveness of ITNs has also been demonstrated (Goodman 1999; Hanson 2003)." Lengeler 2004, Pg 2.

• 6.

  See "Abbreviations" - World Health Organization, "World Malaria Report (2010)," Pg ix.

• 7.
  • "Re-treating ITNs semiannually
    or just before the annual peak in transmission is essential for
    effective vector control and is proving a major logistical and
    financial challenge. Fortunately, new types of nets with a long-
    lasting insecticidal property are now available, and re-
    treatment will soon cease to be an issue." Jamison et al. 2006, Pg 421.
  • "Sumitomo Chemical's Olyset Net uses the latest research and technology to achieve a breakthrough in the global fight against malaria. Permethrin is incorporated inside the Olyset fibres to create a bed net guaranteed to last at least five years.* It is tear-proof, wash-proof, and never requires treatment.

    The pyrethroid insecticide permethrin is a synthetic molecule similar to natural pyrethrin, which comes from a species of chrysanthemum. Because it poses minimal toxic risk to humans, Olyset is particularly valuable where babies and small children are concerned." Sumitomo Chemical, "Olyset Net."

• 8.

  "WHO recommendations for vector control are the following:

  1. Because high coverage rates are needed to realize the full potential of insecticide-treated nets (ITNs) and IRS, WHO recom- mends that all people at risk in areas targeted for malaria preven- tion should be covered with LLINs, i.e “universal coverage” (9,10). It is currently proposed that one LLIN should be distributed for every two persons. This approach may require reinement for implementation at household level: for example, one option is to distribute to each household one LLIN for every two members of the household, rounding up in households with an odd number of members.

  2. LLINs should be either provided free of charge or highly subsi-
  dized. Cost should not be a barrier to making them available to all people at risk of malaria, especially those at greatest risk such as
  young children and pregnant women (9).

  3. Universal coverage with LLINs is best achieved and maintained by
  a combination of delivery systems: mass distribution campaigns
  can achieve rapid initial coverage, but need to be supplement-
  ed by routine delivery to pregnant women through antenatal
  services and to infants at immunization clinics (9).

  4. In order to be protected, households must not only own LLINs but also use them. Behaviour change interventions including informa- tion, education, communication (IEC) campaigns and post-distri- bution “hang-up campaigns” are strongly recommended (9).

  5. Only LLINs recommended by the WHO Pesticide Evaluation Scheme (WHOPES) should be procured by national malaria control programmes and partners for malaria control. These nets are designed to maintain their biological eicacy against vector mosquitoes for at least three years in the ield under recommend- ed conditions of use, obviating the need for regular insecticide treatment (11,12)." World Health Organization, "World Malaria Report (2010)," Pg 4.

• 9.

  GiveWell, "Summary of ITN RCTs."

• 10.

  "The remaining 22 trials, including 1 trial that is currently unpublished, met the inclusion criteria for this review. These trials are described below." Lengeler 2004, Pg 6.

• 11.

  "The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to 7.67; Analysis 1.2). I performed a regression analysis of the natural logarithm of the rate difference on the entomological inoculation rate and could not find a trend (r 2 = 0.52, F = 3.2 on 1,3 degrees of freedom, P = 0.2). In contrast to protective efficacies, the risk differences seemed to have a tendency towards a higher effect with a higher entomological inoculation rate. This apparent paradox is because the baseline mortality rates are higher in areas with high entomological inoculation rates." Lengeler 2004, Pg 9.

• 12.

  "The impact of ITNs on malaria-specific death rates was looked at only briefly because of the problems using verbal autopsies in determining malaria deaths. In the two trials for which the data were available, the percentage reduction in malaria-specific mortality was similar or smaller than the percentage reduction in all-cause mortality: 14% (versus 23%) for Gambia (D’Alessandro), and 22% (versus 18%) for Ghana (Binka). One interpretation is thatmalaria-specific death rates were not reflecting the true impact of ITNs on mortality (since a much higher specific impact would have been expected)." Lengeler 2004, Pg 9.

• 13.

  "The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to7.67; Analysis 1.2)." Lengeler 2004, Pg 8. We have confirmed with the author that this figure is based on an "intention to treat" analysis, e.g., "protected" refers to children in the treatment group, not to children who were confirmed to own or use ITNs.

• 14.
  • "Anaemia: expressed in mean packed cell volume (PCV); it is equivalent to the percentage haematocrit. Results given in g/decilitre were converted with a standard factor of 3:1, that is, 1 g/decilitre equals 3%PCV." Lengeler 2004, Pg 4.
  • " The nine trials that measured anaemia were conducted in areas of stable malaria; six trials compared treated to untreated nets (Appendix 13), and three trials compared treated nets to untreated nets (Appendix 14). Overall, the packed cell volume of children in the ITN group was higher by 1.7 absolute packed cell volume per cent compared to children not using nets. When the control group used untreated nets, the difference was 0.4 absolute packed cell volume per cent." Lengeler 2004, Pg 9.
  • 0.4 packed cell volume converts to (0.4/3 * 10) = 1.3 g/L using the conversion factor provided (and multiplying by 10 to convert from g/dL to g/L). Similarly, 1.7 packed cell volume converts to (1.7 / 3 * 10) = 5.7 g/L.
• 15.

  "Four out of the five trials that measured splenomegaly were carried out in areas with stable malaria (Appendix 15 and Appendix 16). Because the exception was one trial carried out in Thailand whose weight is very small (only 2.6% in the relevant comparison) (Thailand (Luxemberger)), I did not carry out a subgroup analysis. Splenomegaly was significantly reduced for both types of controls: there is a 30% protective efficacy when controls were not using nets, and a 23% protective efficacy when the control group used untreated nets." Lengeler 2004, Pg 9.

• 16.

  "Only one trial examined severe malarial disease as an outcome Kenya (Nevill). The trial used passive and hospital-based case as- certainment, and observed a 45% (cluster-adjusted 95% CI 20 to 63) reduction in the frequency of severe malaria episodes follow- ing the introduction of ITNs (Appendix 4)." Lengeler 2004, Pg 8.

• 17.

  "Uncomplicated clinical episodes:

  The trial results are available in Appendix 5 for no nets controls and in Appendix 6 for untreated nets controls. A summary of the main findings for protective efficacies is available in Appendix 7; confidence intervals were not calculated as this analysis includes both cluster and individually randomized controlled trials.No risk or rate differences were calculated because the denominators were not uniform and the sensitivity of the reporting systems of the different trials is likely to have varied considerably. Three findings can be highlighted.

  • The effect of ITNs on uncomplicated clinical episodes of malaria is shown by large effect estimates in all trials. Overall, the reduction in clinical episodes was around 50% for all subgroups (stable and unstable malaria; no nets and untreated nets) and for both P. falciparum and P. vivax.
  • The protective efficacy is higher (at least 11% for P. falciparum) when the control group had no nets. This was expected and it was the reason to create two sepa- rate comparisons. In areas with stable malaria (entomo- logical inoculation rate > 1) the differences in protec- tive efficacies against uncomplicated malaria was 11% (50% no nets versus 39% untreated nets). In areas with unstable malaria (entomological inoculation rate < 1), the differences were bigger: 23% (62% no nets versus 39% untreated nets) for P. falciparum, and 41% (52% no nets versus 11% untreated nets) for P. vivax.
  • In areas of unstable malaria (entomological inocula- tion rate < 1), the impact against P. falciparum episodes seemed to be higher than the impact against P. vivax episodes."

  Lengeler 2004, Pg 9.

• 18.

  "Parasite prevalence:

  The results are available in Appendix 8 for no nets and in Ap- pendix 9 for untreated nets controls. The results for both groups are summarized in Appendix 10; confidence intervals were not calculated as this analysis includes both cluster and individually randomized controlled trials. Two points can be highlighted from these results.

  • In areas of stable malaria, impact on prevalence of in- fection (measured through cross-sectional surveys) was small: 13% reduction when the control group did not have any nets and 10% reduction when the control group had untreated nets.
  • In areas with unstable malaria, the results are of lim- ited value because there was only a single trial in each subgroup (treated versus no nets; and treated versus un- treated nets).

  Lengeler 2004, Pg 9.

• 19.

  "High parasitemia:

  The results are shown in Appendix 11 for no nets and Appendix 12 for untreated nets controls. This outcome was only assessed for trials in areas of stable malaria, where parasitaemia does not necessarily lead to a clinical episode, and where parasitaemia cut- offs are useful to define disease episodes. Five trials measured this outcome: four used 5000 trophozoites/ml as the cut-off, while the fifth trial used an age-specific cut-off (Kenya (Phillips-Howard)). The protective efficacy was 29% for the two trials in which the control group did not have nets, and was 20% for the three trials in which controls had untreated nets." Lengeler 2004, Pg 9.

• 20.

  "Three trials carried out with ITNs have demonstrated a positive impact on anthropological measurements in children sleeping un- der treated nets.

  In The Gambia (Gambia (D’Alessandro)), mean z-scores of weight-for-age and weight-for-height were higher in children from treated villages (-1.36 and -0.98, respectively) than in those from untreated villages (-1.46 and -1.13, respectively). The differences were statistically after adjustment for area, age, differential bed net use, and gender (P = 0.008 and P = 0.001, respectively). There was no statistically significant difference in me significant an z-scores for height-for-age.

  In the trial carried out in Kenya (Kenya (Nevill)), infants sleeping under ITNs in the intervention areas had statistically significantly higher z-scores for weight-for-age than control infants not under treated nets (analysis of variance allowing for season, gender, and age: F = 21.63, P = 0.03). Mean mid-upper arm circumference z-scores were also statistically significantly higher among infants in the intervention communities (analysis of variance allowing for survey, gender, and age: F = 19.0, P = 0.005) (Snow 1997).

  In Kenya (Kenya (Phillips-Howard)), protected children under two years of age had a statistically significantly better weight-for- age z-score than unprotected children (P < 0.04). No other statistically significant differences were measured for other parameters or other age groups, although all z-score differences between intervention and control groups were in favour of the protected group." Lengeler 2004, Pgs 9-10.

• 21.

  GiveWell, "Summary of ITN RCTs."

• 22.

  GiveWell, "Summary of ITN RCTs."

• 23.

  "To the best of my knowledge there have been no more RCTs with treated nets. There is a very strong consensus that it would not be ethical to do any more. I don't think any committee in the world would grant permission to do such a trial." Christian Lengeler, phone conversation with GiveWell, November 2, 2011.

• 24.

  See World Health Organization, "World Malaria Report (2010)," charts on Pg 13.

• 25.

  "The results presented in this review are from randomized controlled trials where the intervention was deployed under highly controlled conditions, leading to high coverage and use rates. The one exception is Gambia (D’Alessandro), which was a randomized evaluation of a national ITN programme in which the intervention deployment was not as good as in the other trials. Therefore, the bulk of data in this review describe impact under ideal trial conditions (efficacy) rather than impact under large-scale programme conditions (effectiveness). While the difference between efficacy and effectiveness is likely to be small for certain medical interventions (such as vaccination or surgery), it can potentially be large for preventive interventions such as ITNs.

  Some of the consequences of moving from a scientific trial towards a large-scale programme is illustrated by the results of the two mortality trials carried out in The Gambia. The first trial was carried out under well-controlled implementation conditions, with a high coverage rate in the target population (Gambia (Alonso)). Unfortunately it was not randomized and hence not included in the present analysis. The second one was the evaluation of a national impregnation programme carried out by primary health care personnel and which faced some operational problems (leading, for example, to a lower than expected insecticide dosage) and a lower coverage rate (around 60%) of the target population (Gambia (D’Alessandro)). The difference of impact between the two studies is important: the first trial achieved a total reduction in mortality of 42%, while the protective efficacy in the second trial was 23%. It is not clear whether the difference in the baseline mortality rate (42.1 versus 24.3 deaths per 1000 in the control group) played a role in this difference of impact." Lengeler 2004, Pgs 10-11.

• 26.

  World Health Organization, "World Malaria Report (2010)," Pgs 17-18.

• 27.

  Otten and Lines, "Where Did the LLINs Go? An Analysis of Data from 7 Countries with the Most Recent Surveys (2008-2009)."

• 28.

  Otten and Lines, "Where Did the LLINs Go? An Analysis of Data from 7 Countries with the Most Recent Surveys (2008-2009)," Pg 11.

• 29.

  Measure DHS, "Homepage."

• 30.

  Otten and Lines, "Where Did the LLINs Go? An Analysis of Data from 7 Countries with the Most Recent Surveys (2008-2009)," Pgs 4-5.

• 31.
  

  Otten and Lines, "Where Did the LLINs Go? An Analysis of Data from 7 Countries with the Most Recent Surveys (2008-2009)," Pg 18. The text was not terribly clear in describing this model, so we reconstructed the calculation from the data given. The authors assume that 92% of nets are in use the year of distribution, 80% are in use the second year after distribution, 50% are in use the third year after distribution, and 0% are in use after three years. This works out to an average lifespan of 2.22 years per net.

  This implied decay model is the same one spelled out more explicitly in Buj, "Distribution Strategies for LLINs: Scaling Up and Sustaining Coverage."

• 32.

  Taking the formula used by Otten and Lines, "Where Did the LLINs Go? An Analysis of Data from 7 Countries with the Most Recent Surveys (2008-2009)," to calculate the expected decay of nets, we estimate that the total number of protection per net is 2.22 ((0.92 + 0.80 + 0.5) = 2.22).

• 33.
  • "Sumitomo Chemical's Olyset Net uses the latest research and technology to achieve a breakthrough in the global fight against malaria. Permethrin is incorporated inside the Olyset fibres to create a bed net guaranteed to last at least five years.* It is tear-proof, wash-proof, and never requires treatment." Sumitomo Chemical, "Olyset Net."
  • " Background: Insecticide-treated nets represent currently a key malaria control strategy, but low insecticide re-treatment rates remain problematic. Olyset™ nets are currently one of two longlasting insecticidal nets recommended by WHO. An assessment was carried out of the effect of Olyset™ nets after seven years of use in rural Tanzania.

    Methods: A survey of Olyset™ nets was conducted in two Tanzanian villages to examine their insecticide dosage, bioassay efficacy and desirability compared with ordinary polyester nets.

    Results: Of 103 randomly selected nets distributed in 1994 to 1995, 100 could be traced. Most nets were in a condition likely to offer protection against mosquito biting. Villagers appreciated mainly the durability of Olyset™ nets and insecticide persistence. People disliked the small size of these nets and the light blue colour and preferred a smaller mesh size, features that can easily be modified. At equal price, 51% said they would prefer to buy an Olyset™ net and 49% opted for an ordinary polyester net. The average permethrin content was 33%-41% of the initial insecticide dose of 20,000 mg/Kg. Bioassay results indicated high knock-down rates at 60 minutes, but the mosquito mortality after 24 hours was rather low (mean: 34%). No significant correlation was found between bioassay results and insecticide concentration in and on the net.

    Conclusions: Olyset™ nets are popular, durable and with a much longer insecticide persistence than ordinary polyester nets. Hence, Olyset™ nets are one of the best choices for ITN programmes in rural malaria-endemic areas." Tami et al. 2004.

• 34.

  "In reviewing household surveys that provide the most recent results available on ITN coverage for 27 malaria-endemic countries between 2003 and 2009, it was evident that relatively low propor- tions of households own an ITN (median 16%, lower quartile 5%, upper quartile 45%); only 7 surveys were conducted during the massive expansion of ITN programmes from 2008 to 2010. However, within all surveys, a high proportion of available nets appear to be used (approximately 80%) assuming that one net can cover two people (Fig. 4.6a). Some countries such as Madagascar (2008) and Rwanda (2008) have higher rates of use than others. These results are consistent with previous analyses which suggest that the main constraint to enabling persons at risk of malaria to sleep under an ITN is lack of availability of nets (3). " World Health Organization, "World Malaria Report (2010)," Pg 23.

• 35.

  World Health Organization, "World Malaria Report (2010)," Table 4.2, Pg 19.

• 36.

  Excluding Rwanda, Democratic Republic of the Congo and Madagascar (>100% implied usage) as well as Swaziland (2% coverage, 0% usage) and Namibia (15% coverage, 6% usage).

• 37.

  GiveWell, "Summary of ITN RCTs."

• 38.

  It does seem possible that the surveys used for the above analysis are less prone to over-reporting than the surveys done in the small-scale studies, because the latter were likely done in the context of ITN promotion programs whereas the analysis above relies on large-scale surveys covering a large number of topics (full surveys are available via Measure DHS). On the other hand, the intensiveness of the interventions in the small-scale studies may have meant more accurate data collection and/or a higher ratio of actual (correct) usage to reported usage.

• 39.

  See World Health Organization, "World Malaria Report (2010)," charts on Pg 13.

• 40.

  Ibid.; for some country-by-country breakdowns, see the Country Profiles on Pgs 63-137 of the same document (World Health Organization 2010).

• 41.

  Bleakley 2010, abstract.

• 42.
  • "The federal government’s large-scale efforts against malaria in the south began with World War I (WWI). In previous wars, a significant portion of the troops were made unfit for service because of disease contracted in or around encampments. The PHS, working now with a strong knowledge base on malaria control and greatly increased funding, undertook drainage and larviciding operations in southern military camps, as well as in surrounding areas. After WWI, the IHB and PHS expanded the demonstration work further. By the mid-1920s, the boards of health of each state, following the IHB/PHS model, had taken up the mantle of the malaria control in all but the most peripheral areas of the region (Williams 1951). The south experienced a drop in malaria mortality of more than 60 percent in the decade of the 1920s." Bleakley 2010, Pgs 5-6.
  • "In the early 1950s, the World Health Organization (WHO) proposed a worldwide campaign to eradicate malaria. While the WHO mostly provided technical assis- tance and moral suasion, substantial funding came from USAID and UNICEF. The nations of Latin America took up this task in the 1950s. While individual nations had formal control of the design and implementation of the programs, their activities were comparatively homogeneous as per the dictates of their international funders. The central component of these programs was the spraying of DDT, principally in the walls of houses. Its purpose was not to kill every mosquito in the land, but rather to interrupt the transmission of malaria for long enough that the existing stock of parasites would die out. After that, the campaigns would go into a maintenance phase in which imported cases of malaria were to be managed medically.
    The Latin American countries analyzed in the present study (Mexico, Colombia, and Brazil)5 all mounted malaria eradication campaigns, and all saw large declines in malaria prevalence. Panel A of Figure 1 shows malaria cases per capita in Colombia. (Comparable time-series data were not available for the US, Brazil, or Mexico. Nevertheless, there is little doubt that malaria declined in all four countries immediately following these campaigns.) A decline of approximately 80 percent is seen in the graph. Throughout Latin America, the campaign ultimately proved inad- equate to the task, and, in many areas, malaria partially resurged two decades later. But in almost all parts of the hemisphere, malaria never returned to its levels from before the application of DDT." Bleakley 2010, Pg 6
• 43.

  "How realistic is the assumption that areas with high infection rates benefited more from the eradication campaign? Mortality and morbidity data indicate drops of 50 to 80 percent in the decade after the advent of the eradication efforts. Such a dramatic drop in the region’s average infection rate, barring a drastic reversal in the pattern of malaria incidence across the region, would have had the hypothesized effect of reducing infection rates more in highly infected areas than in areas with moderate infection rates. Data on malaria cases by Colombian departments allow us to examine this directly. The decline in malaria incidence as a function of intensity prior to the eradication campaign is found in panel B of Figure 1. The basic assumption of the present study, that areas where malaria was highly endemic saw a greater drop in infection than areas with low infection rates, is borne out. (Similar results are seen for US and Mexican states. Data for Brazilian states were not available.)
  Finally, the timing of the eradication campaign should induce variation in childhood malaria infection that has a marked pattern across year-of-birth cohorts. The present study considers the effects of childhood malaria infection on later- life outcomes, so it is useful to characterize childhood exposure to an eradication campaign. This is shown in Figure 2. Consider a campaign that starts in year zero and takes effect instantaneously. Cohorts born after this date will be exposed to the campaign for their entire childhood. On the other hand, those cohorts who were already adults in year zero will have no childhood exposure to the campaign, while the “in-between” cohorts will be partially exposed during childhood, as shown in Figure 2. I exploit this timing in two ways. First, in Section III, I compare the “born after” cohorts to the “already adult” cohorts by taking differ- ences across these cohort groups. Second, in Section IV, I use the functional form of childhood exposure in estimation using data for all cohorts. (I discuss some alternative functional forms in Section IV.B.)
  These four factors (the external origin of the campaigns, the quick reduction of malaria that followed, the use of nonmalarious areas for comparison, and the differential incidence of eradication benefits across cohorts) combine to form the research design of the present study." Bleakley 2010, Pgs 8-9.

• 44.

  See Bleakley 2010, Figure 4, Pg 26.

• 45.

  See Bleakley 2010, Appendix III, Pg 40, for the full list of controls by country.

• 46.

  "Mixed results are found for years of education, in contrast with consistently positive effects of malaria eradication on income and literacy. Furthermore, in no country can the change in income be accounted for by the change in years of schooling. These facts are in no way discordant with the economic theory of schooling, which compares returns with opportunity costs. Childhood health plausibly raises both, leaving an ambiguous effect on the optimal time to spend in school. This combina- tion of results, interpreted with simple price-theoretic reasoning regarding the education decision, show that we should be cautious in using changes in time in school as a sufficient statistic by which development and health policies are evaluated." Bleakley 2010, Pg 35.

• 47.

  "An approximate extrapolation to the current population of children under five years of age at risk for malaria in sub-Saharan Africa (14% of approximately 480 million population at risk, or 67 million children) indicates that approximately 370,000 child deaths could be avoided if every child could be protected by an ITN." Lengeler 2004, Pg 10.

• 48.

  "The Roll Back Malaria Partnership, the United Nations Millennium Development Goals, and the US President's Malaria Initiative have set a target of at least 80% use of ITNs by young children and pregnant women (the people most vulnerable to malaria) by 2010." Killeen et al. 2007, Pg 1258. Note that this paper was "received December 22, 2006."

• 49.
  • "This Position Statement from the WHO Global Malaria Programme (WHO/GMP) describes a shift in guidance on malaria prevention through the use of insecticide- treated nets (ITNs).

    The WHO/GMP calls upon national malaria control programmes and their partners involved in insecticide-treated net interventions to purchase only long-lasting insecticidal nets (LLINs). LLINs are designed to maintain their biological efficacy against vector mosquitoes for at least three years in the field under recommended conditions of use, obviating the need for regular insecticide treatment.

    In order for their full potential to be realized, LLINs should be deployed as a vector control intervention. WHO/GMP, therefore, recommends full coverage of all people at risk of malaria in areas targeted for malaria prevention with LLINs." World Health Organization, "Insecticide-Treated Mosquito Nets: A WHO Position Statement (2007)," Pg 1.

  • "GMP now recommends universal coverage with Long Lasting Insecticidal Mosquito Nets. Wherever any of the earlier documents refers to targeted coverage of risk groups only, please refer to superseding guidance on this matter now provided in the 2007 GMP position statement Insecticide-treated mosquito nets: a position statement [the source quoted directly above]." World Health Organization, "Publications on Insecticide-Treated Materials: Guidelines."
• 50.
  
  Examples:

  • The World Health Organization position statement discussed immediately previously cites this review as its main source: "On the basis of five community-randomized trials, a Cochrane review concluded that, when full cover- age is achieved, ITNs reduce all-cause child mortality by an average 18% (range 14–29%) in sub- Saharan Africa (5). The general implication of this is that 5.5 lives could be saved per year for every 1000 children under 5 years of age protected. It was also concluded that ITNs reduce clinical episodes of malaria caused by Plasmodium falciparum and P. vivax infections by 50% on average (range 39– 62%), as well as reducing the prevalence of high-density parasitaemia." World Health Organization, "Insecticide-Treated Mosquito Nets: A WHO Position Statement (2007)," Pg 3.
  • The Disease Control Priorities Report also cites this review as its main source for ITN efficacy: "We drew estimates of the effectiveness of ITNS from a recent meta- analysis of WHO/TDR-sponsored trials conducted in Sub-Saharan Africa (Lengeler 2004)." Jamison et al. 2006, Pg 423.
• 51.

  Christian Lengeler, phone conversation with GiveWell, Nov. 2, 2011.

• 52.
  • Dave Smith, phone conversation with GiveWell, Nov. 4, 2011.
  • Thomas Smith, phone conversation with GiveWell, Nov. 8, 2011.
• 53.

  GiveWell, "Summary of ITN RCTs."

• 54.
  • "When you're moving from protecting only children to protecting everybody, you multiply your costs by a factor of five, but you also get at least twice as much effect and we know that from both modeling and empirical impact assessments … I don't think we will ever really have the reply to that question [about the relative magnitude of community-level effects]. Though one thing you can do is mathematical modeling. There are different groups in the world who are doing modeling for the effect of increasing coverage of ITNs, and they will be able to give you some sort of answer … The two people who occur to me are Thomas Smith at the Swiss Tropical Institute and Dave Smith at the University of Florida's Emerging Pathogens Institute." Christian Lengeler, phone conversation with GiveWell, November 2, 2011.
  • "There are empirical studies that show that there seems to be a community-level impact of ITNs. There are theoretical studies that show that there ought to be a community-level effect. What I don't think there is is any attempt to estimate the magnitude of the community-level effect from empirical data. We've done this with mathematical models, but haven't completed the linkage with empirical data that could help with this estimate." Thomas Smith, phone conversation with GiveWell, November 8, 2011.
  • "The argument for ITNs has always been that ITN coverage is a public good. Some of the good accomplished is not internalized to the person using the net. Some people argue that there's a negative externality as well - unprotected people are getting bitten by the mosquitoes that would have bitten the protected people - but I don't think you can make that argument with ITNs … What I do is analysis of data using mathematical models. In general, the leading models say that the more people use the nets, the bigger the positive externalities are … If you're talking about the marginal dollar for more nets, there's a lot more uncertainty on the cost-effectiveness of that than we'd like. The only data that we have is bad data. Improved surveillance would be helpful." Dave Smith, phone conversation with GiveWell, November 4, 2011.
• 55.

  "In late 1996, the villages in Asembo were randomized into ITN (intervention) and control villages … for two years, 1997 and 1998, people in half of the villages (intervention villages) in Asembo had access to properly treated bed nets and half (control villages) did not. Periodic spot checks during this period showed that few (<3%) nets were sold or moved outside the study area, although adherence to net use, measured as the proportion of study participants directly seen to be sleeping under nets during early morning observational surveys, was usually approximately 70% … Although the study area is divided into villages with approximate boundaries as shown in Figure 1A, these are social constructs that do not reflect variation in population density … The 300-meter interval for the main exposure variables was based upon the observation that this distance range divides the number of compounds of each type into approximately equal quartiles (Table 1). All distance calculations were done on the basis of intent to treat; all compounds in ITN villages were assumed to have ITNs and all compounds in control villages were assumed to lack ITNs." Hawley et al. 2003, Pg 121.

• 56.

  "… Compounds in control villages that are ≥900 meters from the nearest ITN village served as the comparison group for these analyses. A total of seven dichotomous variables were created that contained all other compounds. For control villages, we created three variables for compounds that were 0–299, 300–599, or 600–899 meters from the nearest ITN compound. For ITN villages, we created four variables for compounds 0–299, 300–599, 600–899, and ≥900 meters from the nearest control compound. The intent of this analysis was to allow us to illustrate spatial patterns of effect in a straightforward way; however, the cost of this approach was some loss in statistical power. These distance categories approximate compound distribution by quartiles as shown in Table 1 … The effect of the primary geographic exposure variable, distance to nearest compound of different type, is illustrated in Figure 2, which shows the effect of each of seven different exposure categories on five malaria-related outcomes, as well as geohelminth infection. For geohelminth infection, no effect of ITN use or distance is apparent. In contrast, a protective effect of living in ITN villages (stippled area of Figure 2) is apparent at all distance categories for all-cause mortality of children 1–59 months of age and for high-density parasitemia, anemia, and hemoglobin level … The overall pattern of effects is similar for all five malaria-related outcomes: protective effects are observed within the ITN compounds, with a similar level of protection seen in control compounds located within 300 meters of intervention compounds; these effects are statistically significant for the morbidity measures most sensitive to changes in transmission levels: anemia and hemoglobin level … Results of tests of trend reflect the patterns visible in Figure 2 for clinical malaria (odds ratio [OR] = 0.92, 95% confidence interval [CI] = 0.75, 1.12, P = 0.38), high-density parasitemia (OR = 0.89, 95% CI = 0.78, 1.01, P = 0.08), moderate anemia (OR = 0.78, 95% CI = 0.69, 0.89, P = 0.0001), hemoglobin level regression coefficient (OR = 0.18, 95% CI = 0.06, 0.31, P = 0.0048), and child mortality (hazard ratio = 0.94, 95% CI = 0.90, 0.98, P = 0.002). For all of these malaria-related outcomes, trends are in the direction indicating greater protection in compounds closer to intervention villages, with strongly significant trends seen for mortality, anemia, and hemoglobin level. For geohelminth infection, no trend was observed (OR = 1.09, P = 0.47)."
  
  Hawley et al. 2003, Pgs 123-125.

• 57.

  "This community effect on nearby compounds without nets is approximately as strong as the effect observed within villages with ITNs." Hawley et al. 2003, Pg 121.

• 58.

  "studies from various parts of Africa and Papua New Guinea indicate the presence of a beneficial community effect (we propose use of this term rather than the less descriptive mass effect) on malaria transmission [references 1-6], severe malaria [reference 7] and mortality [reference 8] in children. One study, in contrast, yielded no evidence for either beneficial or harmful community-wide effects. [reference 9]" Hawley 2003, Pg 121.

  We looked up reference 9 and confirmed that it discusses only transmission: "In The Gambia, the use of permethrin-treated bed nets has led to a reduction in morbidity and mortality from malaria in children. However, no clear evidence has been found for a 'mass-killing effect' on the mosquito vectors as a result of this intervention. Two further entomological studies to investigate this phenomenon have been carried out. In one study, 20 villages were paired so that bed nets in one member of each pair were treated with permethrin. In the other, a cross-over design was used in which treated and untreated bed nets were exchanged between 2 villages. Longevity, biting rate and resting density of the malaria vector population and sporozoite rates were assessed in both studies. Malaria vectors were equally abundant and long-lived, and as likely to be infective, in villages with treated bed nets as in those with untreated nets. However, a clear reduction in the density of the indoor-resting population of mosquitoes in rooms with treated bed nets was found, probably reflecting the excito-repellency of the insecticide. This study confirmed that, in The Gambia, the protection against death and morbidity from malaria seen in children using treated bed nets must be due primarily to personal protection rather than to a 'mass-killing effect' on the mosquito vector population at a village level." Quinones et al. 1998, abstract.

• 59.

  Gamble et al. 2006, Pgs 11-12.

• 60.

  "An important paper that had some influence in the political debate about mass distribution (as opposed to distribution to infants and pregnant women only) was:
  Preventing childhood malaria in Africa by protecting adults from mosquitoes with insecticide-treated nets. Killeen GF, Smith TA, Ferguson HM, Mshinda H, Abdulla S, Lengeler C, Kachur SP. PLoS Med. 2007 Jul;4(7):e229." Thomas Smith, email to GiveWell, November 8, 2011.

• 61.

  Killeen et al. 2007. The paper includes a spreadsheet with the full specification of its model.

• 62.

  See Smith et al. 2009, Table 1, Pg 514. Note that the higher the existing level of ITN coverage (column headings), the lower the additional amount of ITN coverage needed (figure in the cell minus figure in the column heading) to reach the specified target. For example, at a PfPR of 50% and a goal of reducing it to 1% (10th row, 4th-6th columns), one would need to go from 0% to 71% effective coverage (71% absolute change), or from 10% to 77% coverage (67% effective change), or from 20% to 81% effective coverage (61% effective change).

• 63.

  "Holden Karnofsky: Do you know of any papers or scholars arguing the opposite, that community-level effects are negligible and/or that universal coverage has little value-added above and beyond covering pregnant women and children under five?

  Thomas Smith: I don't." Thomas Smith, phone conversation with GiveWell, Nov. 8, 2011.

• 64.

  "This is very consistent with the results based on net sales; they suggest that demand for ITNs is 75 percent lower at the current cost-sharing price in Kenya (50Ksh or $0.75) than it would be under a free distribution scheme." Cohen and Dupas 2007, Pg 11. See also Figure 1, Pg 11.

• 65.

  Cohen and Dupas 2007, Pg 15, Figure 2.

• 66.

  "When a school-based NGO deworming program in western Kenya introduced cost sharing, the take-up of deworming medication fell sharply and revenue failed to outpace administrative costs, despite the health and education benefits of the program. User fees did not help target treatment to the sickest children. As deworming pills are delivered directly into children's mouths, there was no gap between take-up and usage, so pricing had no potential psychological impact on use through sunk costs." Kremer and Holla 2009, Pg 95.

• 67.

  "Two-stage pricing randomization has also been used to test the two potential routes by which pricing could affect use in a door-to-door marketing campaign for water disinfectant on the outskirts of Lusaka, Zambia. As with deworming and nets, pricing led to a rapid drop-off in take-up, with no evidence of increased targeting to the most vulnerable. Pricing had no statistically significant psychological effect on use. The authors of the study argue that there was some screening effect since those who were more willing to pay for the disinfectant were more likely to have chlorine in their water at later random checks than those who received it for free." Kremer and Holla 2009, Pg 99.

• 68.

  "In 1993 a cost-recovery programme was introduced and villages given free insecticide during the first year of the study were asked to pay 5.00 Dalasi (US$0.50) per bednet treated. Unfortunately this led to a dramatic drop in coverage and a return of child mortality rates in these villages to their pre-intervention values." D'Alessandro et al. 1995, Pg 483.

• 69.

  "In January 2002 the UK Department for International Development (DFID) awarded PSI-Kenya US$33 million over 5 y to socially market partially subsidised ITN within the existing retail sector... A two-tier pricing system of 350 Kenya Shillings (KES) (equivalent to US$4.7) in urban settings versus KES100 (US$1.3) in rural settings was implemented." Noor et al. 2007, Pg 1342.

• 70.

  Noor et al. 2007, Pg 1344, Table 1.

• 71.

  "The programme began in October 2004, and during the first 6 mo Supanet ITNs were bundled with separate Powertab net treatment tablets (for every 6 mo) and distributed to MCH attendees. In May 2005 an additional US$37 million was committed by DFID to PSI to procure and distribute Supanet-branded long-lasting insecticidal nets (LLINs), Olyset and Permanet. These public sector nets were heavily subsidized pretreated nets (KES50; US$0.7) and branded with the MoH logo [20]." Noor et al. 2007, Pg 1342.

• 72.

  "During round four of the GFATM awards in April 2004, Kenya's application was successful and US$17 million was approved to procure and distribute 3.4 million LLINs (Olyset and Permanet brands) free of charge to children under the age of 5 y. This represented, at the time, the largest successful award for free distribution of LLINs in Africa." Noor et al. 2007, Pg 1342.

• 73.

  Noor et al. 2007, Pg 1344, Table 1.

• 74.

  GiveWell, "Cost-Effectiveness Analysis for LLIN Distribution."

• 75.

  "The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to7.67; Analysis 1.2)." Lengeler 2004, Pg 8. We have confirmed with the author that this figure is based on an "intention to treat" analysis, e.g., "protected" refers to children in the treatment group, not to children who were confirmed to own or use ITNs.

• 76.

  The summary rate difference, which expresses how many lives can be saved for every 1000 children protected, was 5.53 deaths averted per 1000 children protected per year (95% CI 3.39 to 7.67; Analysis 1.2). I performed a regression analysis of the natural logarithm of the rate difference on the entomological inoculation rate and could not find a trend (r 2 = 0.52, F = 3.2 on 1,3 degrees of freedom, P = 0.2). In contrast to protective efficacies, the risk differences seemed to have a tendency towards a higher effect with a higher entomological inoculation rate. This apparent paradox is because the baseline mortality rates are higher in areas with high entomological inoculation rates." Lengeler 2004, Pg 8.

• 77.

  See our blog posts on the topic:

  • General issues with current DALY-based analyses
  • Specific problems with published DALY figures on deworming
• 78.

  GiveWell, "LLIN gap analysis."

• 79.

  GiveWell, "LLIN gap analysis," "Basic gap analysis" sheet