- Top charities
Water quality interventions (e.g., filtration, chlorination) aim to prevent waterborne disease, particularly diarrheal disease, a major cause of death among children in developing countries, by removing or disabling pathogens in water.
There is little debate that contaminated water causes diarrhea and that the methods used in interventions to improve water quality, chlorination for example, reduce contamination. However, in developing world settings where diarrhea is endemic, because of a lack of sanitation, hygiene, and access to safe water, the evidence for the effectiveness of these interventions is less clear.
Many randomized controlled trials in the developing world find that water quality interventions reduce diarrheal disease. But these trials are unblinded and rely on self-reported diarrhea to evaluate the interventions. None of the small number of blinded trials of water quality interventions has shown an effect. These blinded trials, like some of the unblinded studies, have various methodological weaknesses, and these weaknesses may undermine their conclusion.
Overall, we are ambivalent about the effect of water quality interventions on diarrhea. We find plausible theories grounded in the available evidence for both believing that water quality interventions reduce diarrhea and for the more pessimistic conclusion that these interventions do not have an effect. We also do not see any analysis that may lead to a more definitive answer without a significant, additional investment of time. We therefore summarize our work so far, provide feedback we've received from scholars and leave questions for further investigation.
Published: November 2013
Published: November 2013
Waterborne diseases are diseases commonly transmitted through contaminants in water. Bathing, washing or drinking contaminated water as well as eating or preparing food with contaminated water can cause a range of diseases, including diarrheal disease, hepatitis A, hepatitis E, poliomyelitis, gastroenteritis and typhoid fever.1 The Global Burden of Disease estimated that in 2010, diarrheal disease alone caused about 10% of child deaths in developing countries.2
Many different interventions address water and sanitation in developing countries. Their emphasis ranges from the proper disposal of excreta to promoting hygiene to increasing supply (see our page on water supply interventions). Water quality interventions aim to remove or disable the pathogens found in water, usually through one of the following methods3:
The central treatment and direct supply of safe drinking water in high-income countries requires significant startup costs and ongoing maintenance.4 Lower cost interventions in developing countries provide wells to protect groundwater, ceramic filters to remove pathogens, chlorine tablets to disinfect water, or sealable containers for safe storage among other approaches that disinfect water in the home or improve it at its source.5
There is little debate that contaminated water causes diarrhea and that the methods used in interventions to improve water quality, chlorination for example, reduce contamination.6 However, the lack of sanitation, hygiene or access to water presents practical challenges. For example, previously clean water can get recontaminated on the walk home from an improved source, participants may fail to regularly use chlorine tablets, or other pathways for pathogens, such as hands and food, may counteract gains from cleaner water. 7 While we have not reviewed the literature on the biological mechanisms through which pathogens cause diarrhea, some scientists believe it is plausible that water quality interventions will have no effect at all on diarrhea unless they are used consistently or unless all sources of pathogens are eliminated.8 For these reasons, the evidence that real-world programs in developing countries have improved health is less clear.9
A Cochrane review examining 20 randomized control trials (RCTs) enrolling about 30,000 participants in developing countries, found that interventions to improve water quality were effective at preventing diarrhea.10 On average, ten studies reporting odds ratios for all ages showed a 35% reduction in the odds of getting diarrhea with individual study results ranging from showing no effect to showing a 70% reduction.11
The studies included in this analysis share at least one potential methodological weakness. In most water quality studies, researchers measure diarrhea rates through participant reports and study participants know whether or not they have received an intervention. This knowledge may influence reporting of diarrhea cases. For example, participants who know that their water is being treated may be less likely to report diarrhea, inflating estimates of the treatment effect. This knowledge may also influence the investigators surveying participants. Because of this potential bias, some studies conceal, or "blind", who has received the intervention by distributing a placebo (e.g. a sham filter) to the control group. None of the studies included in the above meta-analysis of effects on odds ratios are blinded, so this result may be biased.
The 5 blinded studies of water quality interventions in developing countries did not find a statistically significant effect.12 All 5 blinded studies examine household interventions (4 evaluate chlorination and 1 evaluates a water filter), which the Cochrane review finds to be more effective than interventions at the water source.13 Four of the blinded studies report longitudinal prevalence ratios (LPRs) as their primary outcome and one of the studies reports rate ratios. Pooling the 4 blinded studies that report LPRs, we calculate a 5% reduction with a 95% confidence interval ranging from a 6% increase to a 14% reduction in the percent of days with diarrhea for children under 5 and no reduction with a 95% confidence interval ranging from a 12% increase to a 11% reduction for all ages. These 5 studies include 2 studies (Kirchhoff et al 1985 and Austin 1993) with less than 200 participants, 1 study with 1,144 participants (Boisson et al 2010), and 1 study with 2,986 participants (Boisson et al 2013). Jain et al 2010 had a sample size of 3,240.14
The lack of effect found in these studies may not be entirely due to blinding. These studies also may have other weaknesses:
Though these methodological weaknesses may explain why the blinded studies have not found an effect, we have examined these studies in much greater depth than we have examined the large number of unblinded studies. Some of the unblinded studies also have similar weaknesses.
Few studies have evaluated the success of interventions based on objectively measured outcomes.
1 RCT reported on mortality, 14 of the 2,124 in the intervention group receiving flocculant-disinfectant died, compared to 28 out of the 2,277 in the control group, but this study was not designed primarily to look at death as an outcome, so we worry about selective reporting of positive results and the association between the deaths and diarrheal disease.24
1 RCT analyzed samples of stool for contaminants that cause diarrheal disease.25This study reported an 80% reduction (95% CI: 20%-97%) in the odds ratio for the presence of Campylobacter and no significant reduction in enteric parasitic infections.26
Water quality interventions may prevent malnutrition and stunting.27 3 studies examined changes in anthropometrics related to malnutrition, such as weight-for-age.28 We have not been able to obtain the papers for 2 of these studies.29. At least 1 study found a positive effect and at least 1 study found no statistically significant effect.30
2 observational studies have compared historical changes in mortality in communities that had improved water quality to mortality at the same time in otherwise similar communities:
We have not examined these studies closely, but we note that Watson 2005 considered more intensive interventions than those described on this page. The intervention on Indian reservations included “digging wells, providing latrines or septic tanks, building or improving water or sewer treatment plants, extending water and sewer lines and connecting individual homes to those lines.”33. We have not looked closely enough at the historical context for Cutler and Miller 2005 to evaluate whether sanitation and hygiene conditions were similar at the advent of chlorination in the United States to the conditions found in developing countries now.
Several RCTs have shown large effects for water quality interventions, but they have been unblinded and relied on self-reported diarrhea to measure a program’s success. The few blinded studies that have been conducted find no effect but, like some of the unblinded studies, also have methodological weaknesses. It is unclear how large a bias is introduced by self-reporting of diarrhea when respondents know whether or not they have received an intervention. Given the results of the blinded studies, it is possible that this bias accounts for all the effect reported in unblinded studies.
The strength of the evidence relies on how one chooses to weigh a few, blinded studies with some methodological weaknesses that report no effect against a large number of unblinded studies, some of which may also have some methodological weaknesses, that, on average, report large effects. It should be noted that if the intervention were effective, we would guess that a carefully designed and well-executed program would likely be in the same range of cost-effectiveness as our priority interventions.34
Overall, we are ambivalent about the effect of water quality interventions on diarrhea. We find plausible theories grounded in the available evidence for both believing that water quality interventions reduce diarrhea and for the more pessimistic conclusion that these interventions do not have an effect. We also do not see any analysis that may lead to a more definitive answer without a significant, additional investment of time. We therefore summarize our work so far and leave questions for further investigation.
We asked Dr. Alix Zwane, the Executive Director of Evidence Action, and Dr. Thomas Clasen, one of the authors of the Cochrane review and a professor with appointments at Emory University and the London School for Tropical Hygiene and Medicine, to read and comment on this page. See their comments here:
2 additional randomized control trials, funded by the Gates Foundation, are currently being conducted that will measure more objective outcomes in addition to diarrhea morbidity and also seek to answer some of the questions in this section, such as whether an approach combining multiple different types of interventions works better than an approach aimed at one pathway for pathogens.37
|Boisson et al 2010||Source (archive)|
|Boisson et al 2013||Source (archive)|
|Checkley et al 2008||Source (archive)|
|Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013||Source|
|Clasen 2009||Source (archive)|
|Clasen et al 2005||Source (archive)|
|Clasen et al 2006||Source (archive)|
|Clasen et al 2007||Source (archive)|
|Clasen, Brown and Collin 2006||Source (archive)|
|Colford et al 2002||Source (archive)|
|Colford et al 2005a||Source (archive)|
|Colford et al 2005b||Source (archive)|
|Colford et al 2009||Source (archive)|
|Condran and Crimmins-Gardner 1978||Source (archive)|
|Crump et al 2005||Source (archive)|
|Cutler and Miller 2005||Source (archive)|
|GBD Compare||Source (archive)|
|Gorchev and Ozolins 1984||Source (archive)|
|Jain et al 2010||Source (archive)|
|Kirchhoff et al 1985||Source (archive)|
|Kremer et al 2011||Source (archive)|
|Mahfouz et al 1995||Source (archive)|
|Meeker 1972||Source (archive)|
|Moe et al 1991||Source (archive)|
|Quick et al 1999||Source (archive)|
|Rose et al 2005||Source (archive)|
|Universidad Rafael Landivar 1995||Source (archive)|
|Watson 2005||Source (archive)|
|WASH Benefits||Source (archive)|
|Wood et al 2008||Source (archive)|
|Alix Zwane, Executive Director of Evidence Action, email to GiveWell, November 22, 2013||Source|
|Zwane et al 2011||Source (archive)|
Diarrhea accounted for roughly 10% of deaths of children under 5 in 2010. GBD Compare
“A number of interventions have been developed to improve the microbiological quality of water and can be grouped into four main categories.
Water quality can also be enhanced by protecting it from recontamination, for example, by residual disinfection, piped distribution, and safe storage. A combination approach is also common in conventional systems since individual approaches are not effective against the full range of microbial pathogens under all water conditions. Mechanical removal of viruses, for example, presents a challenge to most filters due to their submicron size. Similarly, certain encysted protozoa are resistant to chemical disinfection. The microbiological performance of these approaches may also be impacted by the temperature, pH, turbidity, chemical content, and other characteristics of the water.” Clasen et al 2006, pg. 4.
“In higher income countries, and in many urban settings worldwide, drinking water is treated centrally at the source of supply and is distributed to consumers through a network of pipes and household taps. However, such conventional systems involve significant upfront investment and continued maintenance.” Clasen et al 2006, pg. 4.
“Health authorities generally accept that microbiologically safe water plays an important role in preventing outbreaks of waterborne disease (Hunter 1997). Accordingly, the most widely accepted guidelines for water quality allow no detectable level of harmful pathogens at the point of distribution (WHO 2004).” Clasen et al 2006, pg. 3.
“Based on epidemiological modeling, even occasional consumption of untreated water can vitiate the impact of HWTS. This may be an almost insurmountable challenge of household-based solutions when children and adults are regularly consuming water in places outside the home” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, pg. 1
“In settings that are not served by reliable water treatment and distribution systems, diarrhoeal disease is often endemic, that is, present or usually prevalent in the population at all times. In such settings much of the epidemiological evidence for increased health benefits following improvements in the quality of drinking water has been equivocal (Esrey 1986; Lindskog 1987; Cairncross 1989)” Clasen et al 2006, pg. 3.
“Subgroup analyses suggest that household interventions are more effective in preventing diarrhoea than interventions at the water source” Clasen et al 2006, pg. 1.
We calculated pooled estimates using the random effects model employed in the Cochrane review. We extracted data on outcomes and their standard errors from the Cochrane review for studies included in that review. For Boisson et al 2010 and Boisson et al 2013, we extracted the longitudinal prevalence ratio and a 95% confidence interval from the paper and then approximated the standard error of the log longitudinal prevalence ratio based on those inputs and assuming a log-normally distributed outcome. See spreadsheet.
Column 2, Table 1, Zwane et al 2011, pg. 1822.
"Chlorine use fell in both groups, but column 4 shows that usage was 5 percentage points higher (P = .06) in the biweekly survey group. This smaller treatment effect (compared with round 9) is consistent with the hypothesis that a survey serves as a reminder to chlorinate, with an effect that falls over time in the absence of reminders” Zwane et al 2011, pg. 1823.
“And since diarrhoeal diseases inhibit normal ingestion of foods and adsorption of nutrients, continued high morbidity is an important cause of malnutrition, leading to impaired physical growth and cognitive function (Guerrant 1999), reduced resistance to infection (Baqui 1993), and potentially long-term gastrointestinal disorders (Schneider 1978).” Clasen et al 2006, pg. 3
"On average, filtration reduced typhoid fever mortality by 46%, total mortality by 16%, infant mortality (ages 0-1) by 43%, and child mortality (ages 1-4) by 46%. These are large effects. The second row shows estimates of the chlorination effects, suggesting that chlorination alone had no detectable effect on mortality. The third row shows the coefficient estimates for the interaction between filtration and chlorination. These coefficients are positive for typhoid fever mortality and total mortality, suggesting that filtration and chlorination were substitute technologies” Cutler and Miller 2005, pg. 11.
"The empirical analysis shows that sanitation interventions have large effects. A 10 percentage point increase in the fraction of homes receiving sanitation improvement reduced Indian infant mortality by 0.51 per 1000 births, or by 2.5 percent.” Watson 2005, pg. 5.
Watson 2005, pg. 8.
Using approximate costs from Dispensers for Safe Water, we undertake a very rough estimate of cost-effectiveness and find that the intervention is in the range of our priority interventions. Users can modify inputs in the spreadsheet to see how cost-effectiveness changes.
"...water quality interventions appear to be effective in preventing diarrhoea regardless of whether they are deployed in settings where sanitation is ‘improved (ie connection to a public sewer or septic system, pour-flush latrine, simple pit latrine, or ventilated improved pit latrine) or ‘unimproved’” Analysis 17.1, Analysis 18.1, Analysis 19.1 Clasen et al 2006, pg. 15.
“…bias associated with lack of blinding seemed restricted to trials with subjectively assessed outcomes (ratio of odds ratios 0.75 (0.61 to 0.93))…” Wood et al 2008, pg. 4.
“Little difference was observed between the illness rates of children drinking water good quality water ([Less than] 1 E. coli per 100 ml) and those drinking moderately contaminated water (2 – 100 E. coli per 100 ml)” Moe et al 1991
“Chlorine has been widely promoted by HWT advocates, partly because of its accessibility (local availability and up front affordability), despite challenges in it acceptability and use. Given the recent evidence from the GEMS study on the role of chlorine resistant crypotsporidium as the aetiological agent for severe diarrohea (it was ranked 2 or 3 in most settings) (Kotloff 2013), I think we should refrain from promoting chlorine-only solutions (sodium hypochlorite, NaDCC, etc.), just as we should ceramic filters that are ineffective against viruses.” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, pg. 1.
“Paul Hunter pointed to this in a paper in 2009; others have done more work on this (Brown & Clasen 2012; Enger 2013). Based on epidemiological modeling, even occasional consumption of untreated water can vitiate the impact of HWTS. This may be an almost insurmountable challenge of household-based solutions when children and adults are regularly consuming water in places outside the home.” Thomas Clasen, Professor at the London School for Hygiene and Tropical Medicine, email to GiveWell, November 15, 2013, pg. 1.