- Top charities
Published: December 2012
Funding for malaria control, including LLIN distributions, has increased substantially since 2004.1
This page addresses the question of how strong the evidence is that this scaleup has led to a decline in the burden of malaria. It focuses on the question of the general scaleup of malaria control, though when it is possible to discuss the impact of LLINs alone we do so.
Note that this is part of our broader review of mass distribution of long-lasting insecticide-treated nets to fight malaria.
Chapter 7 of the 2011 World Malaria Report (World Health Organization 2011) focuses on assessing the impact of malaria control, with a focus on Africa.2 It discusses four different sub-regions of Africa, citing a mix of smaller-scale studies and trends in country-level data;3 this section focuses on the latter, and smaller-scale studies are discussed in a later section.
Looking at the years 2000-2010,4 the report lists 12 countries that are believed to have seen a substantial decrease in malaria cases (first two columns of the table below); 2 countries in which malaria cases are believed not to have risen or fallen more than 25% (third column); and 28 countries for which there is insufficient data to determine what has happened to the malaria burden.
Unfortunately, we have not been able to determine precisely how the report makes these determinations. It refers to the previous year's report for detail on its methodology,5 and the previous year's report sketches guidelines for interpreting the data provided by Ministries of Health.6 However, it is not entirely clear how to apply the guidelines to the data provided on each country in order to arrive at the same conclusions as the report. For example, when looking at the country profile for Guinea-Bissau,7 it appears to us that the criteria for considering the data meaningful are met: annual blood examination rate data is available and fairly stable from 2005 on, and confirmed cases are available from 2005 on and don't seem to have fallen, despite a scale-up in LLIN coverage. Yet (as shown above) the WHO lists this as a country where the data is insufficient to say what has happened to malaria cases. On the flipside, Zambia8 appears to have no data on confirmed cases of malaria, yet it is listed (in the table above) as a country for which malaria cases are confirmed to have fallen 25-50%. (Our full notes, from scanning the country profiles of all African countries in which substantial (30%+) LLIN coverage appeared to have been achieved between 2000-2010, are available in the document "GiveWell Analysis of World Malaria Report Country Profiles.")
In addition to the question of where malaria burdens have fallen vs. risen, there is the question of whether these changes coincided with LLIN coverage, with other forms of malaria control, or with neither of these. Unfortunately, it is difficult to answer this question on a country-by-country basis. The report's Country Profiles9 provide sets of charts with figures related both to malaria burdens and malaria control measures, but as discussed above, we aren't able to determine how to interpret the malaria burden charts in a way that reaches the conclusions shown in the chart above.
We have spoken with the primary author of the report in an attempt to understand how to interpret the data it provides, but it appears that many of the decisions are made informally and the reasoning is not made fully explicit in the report.10
Note that the report provides data for Africa as a whole that implies that the burden of malaria was rising until 2004 and fell after that point,11 but it appears that this data comes from a model that assumes the impact of malaria control measures.12
While there are a couple of countries for which the data seems to imply that malaria control was impactful (e.g., a couple of countries showing a decline in cases coinciding with the scale-up in control efforts),13 overall we do not find this data to provide strong evidence regarding trends in malaria burden or the impact of malaria control. At the same time, we acknowledge the possibility that scholars with more context on data-quality-related issues and individual countries' histories may rightly consider the data convincing.
As discussed above, the World Malaria Report states that country-level data varies in quality, creating obstacles to interpreting it. In addition to discussing country-level data, the report also cites ten smaller-scale studies that focus on smaller regions, but may collect more reliable data.14 In addition, when we spoke to the primary author of the report, he referred us to three other papers that discuss trends in the burden of malaria and the connection to malaria control measures.15 We found the most useful of these to be O'Meara et al. 2010, which discusses 46 studies that "have reported recent changes in the incidence or prevalence of malaria in sub-Saharan Africa"16. We first discuss O'Meara et al. 2010, then give briefer notes on the other papers found in the World Malaria Report and referred to in our conversation with its lead author.
Note that these studies are all studies of malaria trends, not the impact of insecticide-treated nets. For the latter, see our full writeup on LLINs. As discussed at that page, studies on the impact of insecticide-treated nets are older (mostly from before 2000), more rigorous in terms of isolating impact (they randomly assign some people to receive LLINs and others not to and assess differences in the two groups), and possibly less representative of everyday conditions (since they look at programs that are implemented and randomized for the purpose of the study). By contast, these smaller-scale studies are trying to pick up malaria trends without specifically looking for the impact of (or running special study-centric versions of) malaria control.
O'Meara et al. 2010 attempts a comprehensive review of studies that report at least 2 years of malaria burden data from the time period 2000-2010 (with a few other stipulations).17 As the summary table on pages 547-54818 shows, the majority of these studies show a substantial decline in malaria burden (as assessed by a variety of different measures). By our tally, out of 44 studies shown in the table, 27 show a decrease in malaria burden while 4 show an increase (the other 13 show no change/inconclusive); of the 27 showing a decrease, 25 show at least a 25% decrease, 22 show at least a 50%+ decrease, 14 show at least a 75% decrease and 6 show at least a 90% decrease. (Note that all of these refer to the more pessimistic end of the range, i.e., the lower-magnitude decline in malaria burden, given in the table; in addition, of the 4 increases, 3 are 50%+, so it's worth keeping in mind that malaria cases may be volatile and large changes can't necessarily be confidently attributed to control efforts.)
It is less clear what role malaria control (and LLINs in particular) played in these improvements. The review provides a few approaches to this question:
1. Charts for seven of the cases discussed in the review.
It appears to us that there are reasonable cases for the impact of insecticide-treated nets in Ethiopia (B), the Gambia (C), and possibly Kenya (D). We are not clear on how these cases were chosen to be featured.19
2. A visual summary of the timing improvements that suggests a loose clustering around the time of global malaria control scaleup. As the chart at the top of this page shows, malaria control began scaling up worldwide around 2004, and the following chart from O'Meara et al. 2010 provides some loose evidence that progress improved around that time:
3. Discussions of a few cases. The text of the review asserts timing-based arguments for the impact of insecticide-treated nets in Rwanda, Gambia, Zambia and Sao Tome, though it also lists multiple cases in which the observed decline in malaria appeared specifically not to be attributable to this factor.20
The paper also lists significant caveats to its analysis:21
We must bear in mind the limitations in using published scientific literature to assess the progress of ongoing malaria control programmes. Some of the reports in this Review might be biased towards presenting data that reflect well on the outcome of a control programme (to justify investments) or even presenting data that suggest the malaria burden is worsening (to encourage further investment into malaria control); maintenance of the status quo is unlikely to result in publication in the scientific literature. For example, three studies from Kenya show exciting reductions in the burden of malaria, but data from 17 hospitals reveal that there are many areas where malaria is not declining. Many of the reports we have reviewed are limited in time and geographic scope, and therefore might not accurately reflect nation-wide trends. Most of these reports rely on clinical diagnosis of malaria at a health facility. We cannot account for the effect of changes in access to care or use of health services on incidence measured at the facility.
The authors additionally note:22
On the basis of current models of malaria transmission, we would not expect partial coverage with ITNs and the introduction of ACTs to result in the substantial changes in malaria incidence seen in areas with moderate transmission. Alternative explanations for these changes should be considered.
After considering several other potential hypotheses, the authors to do not reach a firm conclusion about what might explain the large observed declines.
However, it provides little discussion of the question of whether such changes can be attributed to malaria control (or to particular aspects of malaria control). With the exception of a discussion of the Island of Bioko (also discussed in other sources that we examine above), it does not discuss patterns of malaria burden prior to scaleups in control efforts, and with the exception of a brief and informal discussion28 it does not address the possibility of alternative explanations for declines in malaria burden.
Available data and studies appear to show some cases of apparent malaria control success, and also seem to indicate that the overall burden of malaria in Africa is more likely to be falling than rising. However, in most cases it is difficult to link changes in the burden of malaria to particular malaria control measures, or to malaria control in general; and the data remains quite limited and incomplete, such that we cannot confidently say that the burden of malaria has been falling on average.
We can imagine that a malaria scholar, with more context than we have on the strengths and weaknesses of different data sets and the histories of malaria control in different areas, could have a higher degree of confidence in the idea that malaria control (and ITNs in particular) has contributed to major declines in the burden of malaria.
World Health Organization, "World Malaria Report (2011)," Pgs 15 and 27.
World Health Organization, "World Malaria Report (2011)," Chapter 7 - Pgs 51-78. Pgs 52-59 deal specifically with the African region (as the World Health Organization defines it).
"The percentage of the population potentially covered by ITNs delivered was high (>70%) in 2010 in Burundi, Central African Republic, the Democratic Republic of Congo and Equatorial Guinea (Fig.G). Of these countries, all except the Democratic Republic of Congo have at least moderately good access to ACTs (Fig.H). Although progress appears to have been made in delivering interventions within the subregion it has not been possible to evaluate the impact of these efforts because the quality of routinely collected data is generally poor, the parasitological confirmation rate is low, and there are few alternative sources of information such as population-based surveys or specific studies of the impact of interventions. Following substantial investments in malaria control in this subregion, greater emphasis needs to be placed on monitoring and evaluation." World Health Organization 2011, Pg 52.
"Apart from Senegal ( box 7.2), the strongest associations between interventions and impact are seen in data from two small island countries, Cape Verde and Sao Tome and Principe (Fig. e). The diagnostic testing effort in Sao Tome and Principe is high: the ABER exceeds 30% on average, far greater than in other countries in this subregion ( Fig.C). Cape Verde and Sao Tome and Principe both use IRS at high coverage, and in Sao Tome and Principe IRS is used together with ITNs. In addition, a more detailed evaluation in Sao Tome of malaria cases, admissions and deaths, and of malaria infection rates, has linked malaria decline to the intense use of IRS, ITNs and ACTs ( 5, 6).
"Two other special studies in Burkina Faso and Gambia have pointed to some additional successes in malaria control. In Gambia, a retrospective study carried out at four sites found reductions in the slide positivity rate, and in the proportions of hospital admissions and deaths due to malaria over the period 2003–2007 ( 7). And a malaria survey in a rural area of north-western Burkina Faso reported a 27% decline in rates of parasitaemia in 2009 compared to 1999 following an increase in ITN coverage from 22% to 73% (8). Many more special studies of this kind are needed to gain a full understanding of the effects of malaria control in this and other African subregions. Continued strengthening of routine health information systems is also necessary." World Health Organization 2011, Pg 54.
"The declines in malaria admissions and deaths seen in nationally aggregated hospital data are consistent with published studies of data from health facilities in Eritrea, Ethiopia, Rwanda, and United Republic of Tanzania (Zanzibar) (9,10,11). In coastal areas of Kenya (Kilifi, Msambweni), district hospitals have reported that malaria cases declined among all paediatric admissions by 8%–63% between 1999 and 2007 (12). The observed increase in malaria admissions in Uganda agrees with an independent study, which found that hospitalizations increased by 47%–350% between 1999 and 2009 in four of five health facilities studied (13). An evaluation of malaria programmes in United Republic of Tanzania (mainland) from 1999 to 2010 found a 45% decline in the under-five mortality rate, and a 50% decline in severe anaemia prevalence in children 6–59 months of age following a 36-fold increase in ITN use among children
"ITNs are the principal method of vector control in this subregion. A relatively high coverage of ITNs in Madagascar, Rwanda, and United Republic of Tanzania (mainland and Zanzibar, Fig.G) might explain why cases declined substantially between 2000 and 2010 (box 7.3 ). But this association has not been observed in the Comoros (Figs. F, G). Mozambique had the lowest reported coverage of ITNs and IRS, and yet malaria admissions were falling between 2007 and 2010. Deeper investigations are needed to understand these inconsistencies." World Health Organization 2011, Pg 56.
Pg 77. The table doesn't explicitly state that it refers to these years, but the graphic on the following page implies that it does.
"The reported numbers of malaria cases and deaths are used as core indicators for tracking the progress of malaria control programmes (the working definition of a case of malaria is considered to be “fever with parasites” (1)). The main sources of information on these indicators are the disease surveillance systems operated by ministries of health … Changes in the numbers of cases and deaths reported by countries do not, however, necessarily reflect changes in the incidence of disease in the general population, because: (i) not all health facilities report each month … When reviewing data supplied by ministries of health in malaria-endemic countries, the following strategy was used to minimize the influence of these sources of error and bias … Further description of the procedures used is provided in the World Malaria Report 2010. The aim is to exclude data-related factors, such as incomplete reporting or changes in diagnostic practice, as explanations for a change in the reported incidence of disease. Even so, trends in health facility data may not reflect changes in the entire community. The conclusion that trends inferred from health facility data reflect changes in the community has more weight if (i) the changes in disease incidence are large (ii) coverage with public health services is high and (iii) interventions that promote a reduction in cases, such as use of ITNs, are delivered throughout the community and not restricted to health facilities." World Health Organization 2011, Pgs 51-52.
"When reviewing data supplied by ministries of health in malaria-endemic countries, the following strategy was used to minimize the inluence of these sources of error and
The aim of these procedures is to rule out data-related factors, such as incomplete reporting or changes in diagnostic practice, as expla-nations for a change in the incidence of disease and to ensure that trends in health facility data relect changes in the wider community. The conclusion that trends inferred from health facility data relect changes in the community has more weight if: (i) the changes in disease incidence are large, (ii) coverage with public health services is high, and (iii) interventions promoting change, such as use of ITNs, are delivered throughout the community and not restricted to health facilities." World Health Organization 2010, Pgs 39-40.
World Health Organization 2011, Pg 125.
World Health Organization, Pg 183.
World Health Organization, Pgs 81-184
" When we try to judge the reliability of the data, overall, in terms of assessing trends, we try to focus on confirmed malaria cases - we don't consider the "presumed fevers" as malaria. We try to make sure the diagnostic effort is consistent - the number of patients tested is reasonably consistent over time. We try to look at subnational data. We also do a cross check between admissions for malaria and reported deaths and we look for inconsistencies; if there are inconsistencies we make queries; if we can't resolve them we may decide the data is inconclusive. We also get informal evidence about the quality of the surveillance systems over time. I know certain countries very well, while other colleagues know others and we also have a sense of whether the data can be trusted or not. So there are some judgment calls … In most cases the assessment of trends is quite straightforward but there are a few borderline cases in which a decision needs to be made one way or another. We try to be on the conservative side; if there's any room for doubt we say there is insufficient evidence. We do take notes, but it would probably be difficult to recover the precise chain of thought that went into every decision." Cibulskis 2012.
World Health Organization 2011, Table 7.3 (Pg 74).
" Child malaria deaths were estimated using a verbal autopsy multi-cause model (VAMCM) developed by the WHO Child Health Epidemiology Reference Group (CHERG) to estimate causes of death for children aged 1–59 months in countries with less than 80% of vital registration coverage. The VAMCM is a revised model based on work described elsewhere (20, 21). The VAMCM derives mortality estimates for malaria, as well as 7 other causes (pneumonia, diarrhea, congenital malformation, other neonatal causes, injury, meningitis, and other causes) using multinomial logistic regression methods to ensure that all 9 causes are estimated simultaneously with the total cause fraction summing to 1. The regression model is first constructed using the study-level data and then populated with year 2000–2010 country-level input data to provide time-series estimates of causes of death in children aged 1–59 months. Deaths were retrospectively adjusted for coverage of ITNs and use of Haemophilus influenzae type b vaccine. The bootstrap method was employed to estimate uncertainty intervals by re-sampling from the study-level data to estimate the distribution of the predicted percent of deaths due to each cause." World Health Organization 2011, Pg 73.
See charts for Namibia (World Health Organization 2011 Pg 145) and Rwanda (World Health Organization 2011 Pg 157).
From World Malaria Report 2011, Pgs 52-56.
"There have been some broad evaluations done as well. One was done by Steve Lim and colleagues published in PLos Med in 2011. Others by Rick Steketee, published in the Malaria Journal in 2010 and O'Meara and others in the Lancet in 2010. The Lim paper came up with effects of ITNs comparable to those seen in randomized controlled trials." Cibulskis 2012.
O'Meara et al. 2010, abstract.
"We searched the National Library of Medicine’s Medline database with the Medical Subject Headings 'Africa South of the Sahara', 'Malaria/epidemiology', 'Malaria/mortality', 'Malaria/prevention and control', and 'Malaria/transmission', omitting papers that contained the keywords 'Malaria/immunology'. The search was limited by restricting retrieval to articles published in the past 10 years. Our last search was done on March 29, 2010. The search identified 1528 publications. Articles were included if they reported at least 2 years of data on malaria-specific indicators (clinical or slide diagnosed case numbers, incidence, prevalence, or malaria-specific mortality) in a population of more than 1000 people. Cross-sectional studies reporting a single timepoint, those involving only pregnant women, intervention studies covering 1 year or less of follow-up, and entomological studies without any clinical or parasitological components were excluded. Also excluded were studies published in the selected time frame but which did not report data from the period 1999–2009. The titles for each citation were screened and 297 were selected for review of their abstracts. Screening of the abstracts yielded 82 publications for full review. After reviewing the full texts, 46 studies that fully met the inclusion criteria were identified." O'Meara et al., Pg 553.
The authors note, “Many reports attribute decreases in malaria morbidity to speciﬁc interventions, although the causal link between the decline and the intervention is more convincing in some cases than in others (ﬁgure 3). In several reports, the decline began before the speciﬁc intervention was deployed, or the decline was already underway at the beginning of the study period, suggesting that factors not investigated contributed to the decline. This is highlighted in the two reports from the coast of Kenya, one of which shows an association between ITN distribution and a decline in admissions of children with malaria to hospital whereas the other shows substantial changes in prevalence of malaria infection before the decline in admissions and the distribution of ITNs. Similarly, in The Gambia and Zanzibar, the decline in malaria began before ITNs were rolled out. In Eritrea and Ethiopia, a substantial outbreak occurred from late 2002 to 2005, 60 thus declines in malaria incidence back to historical levels were already underway as interventions were introduced in 2005 and 2006. In other reports, the temporal association with the introduction of speciﬁc interventions is compelling, particularly in the data from the northern provinces of South Africa and Bioko Island.” Pgs 551-552.
Instances in which there was a decline in malaria burden, but not one whose timing suggests a strong role for LLINs:
Instances in which there was a decline in malaria burden whose timing does suggest a strong role for LLINs:
Elsewhere, however, the authors note elsewhere that, “in The Gambia and Zanzibar, the decline in malaria began before ITNs were rolled out.”
O'Meara et al. 2010, Pg 553.
O'Meara et al. 2010, Pg 553.
From World Malaria Report 2011:
"In contrast to the largely encouraging reports from Kenya and Rwanda, data from a highland and a lowland area in western Uganda showed steadily increasing numbers of malaria cases and deaths in district hospitals from 1991 to 2000, with a two-fold to four-fold overall increase in the number of children admitted to hospital with the disease. A slight decline in the proportion of positive blood films was seen in a single facility in an area of moderate transmission in Uganda after one round of indoor residual spraying in 2007. 14 months after indoor residual spraying, the proportion of blood films that tested positive began to increase, suggesting that trends are easily reversed if control measures are not sustained." O'Meara et al. 2010, Pgs 547-8
World Health Organization 2011, Pg 56.
"National survey data, published literature, and organization or country reports produced during 2000-2009 were reviewed to assess available malaria financing, intervention delivery, household or target population coverage, and reported health benefits including infection, illness, severe anaemia, and death." Steketee and Campbell 2010, Abstract.
Steketee and Campbell 2010, Figures 5 and 6.
"Factors that could offer alternative explanations for the suggested link between malaria control scale-up and malaria morbidity and mortality reductions might include: 1) variations in rainfall and temperature; 2) broad socio-economic change; 3) changing HIV conditions; 4) other child health interventions discussed previously that might account for the differences; and/or 5) biologic changes in the malaria-vector-human cycle that is making malaria infection and illness less virulent. Many of the studies address and account for rainfall and temperature patterns and demonstrate that these are not plausible explanations of marked reductions in malaria during this time interval in most African countries. In Ethiopia, weather patterns are thought to have contributed to a substantial malaria epidemic from 2003 through 2005, so some of the findings there may be accounted for by this earlier period with high malaria as a comparison time for more recent scale-up and impact; however, this is not the case for other country settings. Some socio-economic change certainly occurred in Bioko Island with the growth of the oil industry, and improved copper prices in the 2005 through 2008 interval may have contributed indirectly in Zambia--but again, such socio-economic improvements occur in many countries but are not likely to explain the dramatic reduction in childhood mortality documented in the setting of SUFI. HIV rates have not dropped consistently across these countries, but improved treatment with anti-retroviral drugs may have contributed partially to the improved child survival and reductions in fever and malaria incidence and prevalence." Steketee and Campbell 2010. Pgs 11-12
"We considered all demographic and health surveys (DHS) and malaria indicator surveys (MIS) from sub-Saharan Africa countries conducted since 2000 for which the unit-record data were available. Prior to 2000, ITN ownership and use in sub-Saharan Africa was universally low . We included only surveys that collected data on the health outcomes of interest (child mortality or parasitemia prevalence) as well as information on ITN ownership and use (including when the ITN was received or purchased, and when it was retreated) and all covariates specified in the analyses. We excluded the Ghana DHS conducted in 2003 as no child deaths were observed in the small number of households that owned ITNs. The results on the association between ITNs and child mortality are based on 29 DHS in 22 sub-Saharan African countries, while the results on the association between ITNs and parasitemia prevalence are based on 6 MIS and one DHS from seven sub-Saharan African countries." Lim et al. 2011. Pg 2.
"We used matched logistic regression to assess the individual-level association between household ITN ownership or use in children under 5 years of age and the prevalence of parasitemia among children using six malaria indicator surveys (MIS) and one demographic and health survey. We used Cox proportional hazards models to assess the relationship between ITN household ownership and child mortality using 29 demographic and health surveys." Lim et al. 2011. Pg 2.
"The pooled relative reduction in parasitemia prevalence from random effects meta-analysis associated with household ownership of at least one ITN was 20% (95% confidence interval [CI] 3%–35%; I2 = 73.5%, p0.05 for I2 value)." Lim et al. 2011. Pg 6.