Helen Keller International (HKI) wrote the following in response to GiveWell's interim review of HKI. We have since updated our review of HKI so the below may not fully be up-to-date.
Published: November 2017
Helen Keller International (HKI) appreciates GiveWell’s invitation to be considered for a top charity recommendation for its vitamin A supplementation program. We have appreciated the transparency and thoroughness of GiveWell’s investigative process thus far. We also appreciate being named a standout charity based on the interim review while GiveWell undertakes additional investigation to determine if HKI qualifies as a top-rated charity.
We would like to offer the following statements in response to points made in GiveWell’s interim review:
- Does vitamin A supplementation (VAS) work? This is an important question and one that has received recent attention considering the shifting epidemiologic and programmatic landscape. The epidemiologic landscape has changed since the first VAS trials were published in the early 1990s. Overall, child mortality rates have declined by 49% since 1990 (Unicef 2014), but the rate of decline has been slowest in Oceania, sub-Saharan Africa and Asia. Likewise, the proportionate cause-specific mortality has also changed. In 1990, the three main killers were pneumonia (21% of under-5 mortality; U5MR), diarrhea (20%), and measles (7%) (van den Ent et al 2011), while in 2010 the main killers were pneumonia (18%), diarrhea (11%) and malaria (7%) (Liu L et al 2012).
There is no question that in contexts exhibiting public health levels of vitamin A deficiency (VAD) and UFMR, VAS (and other interventions that improve the underlying vitamin A status of risk groups) is both sight- and life-saving. This conclusion stems from the results of large, rigorously conducted community trials in South Asia and Africa, which collectively provide incontrovertible evidence that vitamin A interventions, including 6-monthly VAS, reduce early childhood mortality and blindness in undernourished populations (Mayo-Wilson et al 2011). The impact is particularly striking on fatality not only from measles but also from more common diseases such as diarrhea, dysentery and other infectious illnesses. In contexts where uncertainly exists about deficiency and mortality levels (due to the lack of recent data or other reasons) stopping or modifying VAS targets potentially puts children’s lives at risk.
But even in countries with marked mortality declines and changes in causes of death, one cannot rule out a child survival benefit in many contexts. In all, 54 countries globally had a high U5MR (defined as ≥50 per 1000 live births) in 2012 (Unicef 2014). A large proportion of these deaths are caused by infections. Furthermore, in these high-mortality countries VAD is also likely to be high (Schultink 2002), thus reinforcing the need to maintain VAS and other vitamin A interventions. Where U5MR, VAD and infectious disease rates are low, the mortality effect of VAS will likely be reduced. Nevertheless, we must bear in mind two important facts (1) the original VAS studies observed mortality impacts in settings with a wide range of mortality and morbidity rates (Beaton et al 1993), and (2) one cannot rule out the role of VAS in helping to bring down U5MR (Bishai et al 2005; Masanja et al 2008).
- How to think about the Deworming and Enhanced Vitamin A (DEVTA) program evaluation? GiveWell’s report mentions several times the disputed and controversial DEVTA program evaluation study which suffered from important methodological limitations related to supplementation adherence and vital event monitoring systems, as acknowledged by other scientists (Mannar et al 2013; Mayo-Wilson et al 2013; Habicht et al 2013; Sommer et al 2013; Sloan et al 2013). In addition to weighing the methodological flaws of DEVTA, we feel the results of the DEVTA study should be viewed within the context of the larger body of evidence on VAS and child survival. Recently, the WHO examined evidence from all 17 trials (11 in Asia, 5 in Africa and 1 in Latin America) conducted to date for all-cause mortality. Findings revealed that VAS reduces the overall risk of death by 24% (risk ratio (RR) 0.76; 95% confidence interval (CI) 0.69–0.83). When adding the DEVTA findings to the analysis, the all-cause mortality benefit of VAS remained statistically and clinically significant at 12% (RR 0.88; 95% CI 0.84–0.94) (Mayo-Wilson et al 2011)
- The current best evidence indicates that VAD remains prevalent in south Asia and sub-Saharan Africa, but there is a need for more current, reliable and valid estimates of VAD prevalence. GiveWell raises the question, “How prevalent is vitamin A deficiency in areas where HKI works?”. While HKI recognizes the urgent need for updated and valid estimates of vitamin A status in the countries and sub-regions where we work, HKI relies on the best available evidence from scientific sources to ensure that its VAS programs are targeting at risk populations. The most recent global and region-specific estimates of VA deficiency prevalence come from a pooled analysis of population-based surveys from 138 low- and middle-income countries between 1991 and 2013 and published in the Lancet Global Health Journal in 2015. In this publication, the authors estimated the prevalence of deficiency in 2013 to be highest in sub-Saharan Africa (48%) and south Asia (44%) (Stevens et al 2015). Region and country-specific VAD prevalence estimates should be updated as new data become available. Currently, many countries implementing VAS programs have no VAD data or the data do exist are >10 years old (Wirth et al 2017). Clearly, there is an urgent need to fill this data gap and for funders and host-country governments to invest in high-quality surveys to assess VA (and other micronutrient) status and program coverage in children.
- Achieving and sustaining high VAS coverage through HKI’s technical assistance. We provided GiveWell with evidence from two countries (Cameroon and Kenya) which demonstrated that HKI’s technical assistance contributed to significantly higher coverage rates. We appreciate GiveWell’s desire to understand HKI’s added value by assessing VAS program performance using a counterfactual paradigm. Unfortunately, due to the lack of funding in Mali and Cote d’Ivoire in 2017, HKI has been unable to provide VAS technical support to either country providing counterfactual examples. Sadly, both countries missed a VAS distribution round in the first semester of 2017 suggesting that in the absence of HKI’s support the VAS programs in both countries were negatively affected. During their planned country visit, we encourage GiveWell to look further into the added value HKI provides to VAS coverage.
- Cost per supplement delivered and cost-effectiveness of VAS. We feel it is important to note that VAS often serves as the driver behind Child Health Days (CHDs) and Child Health Weeks onto which other vital health and nutrition services (such as deworming, measles immunization, distribution of insecticide-treated bednets, screening for acute malnutrition, and others) are piggy-backed. For example, CHDs delivered nearly half of all global deworming treatments to preschool children in 2013, thus illustrating the strategic importance of this delivery mechanism for attaining high coverage of vital services targeting preschool-age children (Kumapley et al 2015). The design of CHDs and the package of interventions offered can be tailored to the local contexts; and in fragile health systems, CHDs serve as a major delivery platform for high-impact interventions targeted to preschool age children. Because the semi-annual delivery of VAS to preschool children is often the main driver behind CHDs, we feel it is important for cost per supplement delivered and cost-effectiveness models to consider these added benefits. The CHD delivery platform was largely propelled by the need to reach preschool-age children twice each year with a large dose of vitamin A.
- Questions that need more information. HKI appreciates the rigor that GiveWell applies to organizations that are being considered for “top charity” selection. GiveWell’s interim report identifies many remaining questions related to VAS and HKI that it hopes to answer or about which it wants to develop a deeper understanding. Some of these questions will require investments in new data collection. For example, the only way to assess levels of VA deficiency or U5MR in countries or sub-regions where HKI works is to measure these using reliable and valid methods. In low-resource and low-capacity settings, this will require significant investment by the global community and should be done. It is even more difficult to answer the question about the expected child survival impact of VAS given the changing epidemiologic landscape, especially since conducting placebo-controlled trials to address this question would be unethical given the weight of evidence of the benefit of VAS. HKI’s view is to trust the scientific community’s best estimates of benefit based on thoughtful and systematic meta-analyses. HKI keeps abreast of new scientific evidence as it emerges. If and when estimates of benefit are revised, HKI will revise impact expectations and program approaches.
The question of HKI’s added value with respect to VAS programs is, in our view clear. HKI remains a global leader, innovator, advocate and technical support to VAS programs in countries and contexts where VAS should remain a priority intervention. We look forward to GiveWell’s site visits so they can learn more about the important role HKI has provided to VAS programs especially in Sub-Saharan Africa and the support it wishes to continue to provide until the scourge caused by VAD no longer plagues vulnerable populations. VAD will not disappear until vulnerable populations have achieved normal vitamin A status by sustained changes in dietary vitamin A intake. HKI strives to improve the diets through its fortification, nutrition education and food production programs. Until the time when the diets of vulnerable populations are replete with adequate intake of vitamin A, HKI believes periodic high-dose vitamin A has a vital public health role in protecting child health and survival, and thus remains committed to this sight- and life-saving intervention.
References
Beaton GH, Martorell R, Aronson KJ, Edmonston B, McCabe G, Ross AC, Harvey B. Effectiveness of Vitamin A Supplementation in the Control of Young Child Morbidity and Mortality in Developing Countries. Geneva, Switzerland: Administrative Committee on Coordination–Subcommittee on Nutrition (ACC/SCN); 1993.
Bishai D, Kumar K C S, Waters H, Koenig M, Katz J, Khatry SK, West KP Jr. The impact of vitamin A supplementation on mortality inequalities among children in Nepal. Health Policy Plan. 2005 Jan;20(1):60-6.
Habicht JP, Victora C. Vitamin A supplementation in Indian children. Lancet. 2013 Aug 17;382(9892):592.
Kumapley RS, Kupka R, Dalmiya N. The Role of Child Health Days in the Attainment of Global Deworming Coverage Targets among Preschool-Age Children. PLoS Negl Trop Dis. 2015 Nov 6;9(11).
Liu L, Johnson HL, Cousens S, et al. Global, regional, and national causes of child mortality: an updated systematic analysis for 2010 with time trends since 2000. Lancet 2012;379:2151-61.
Mannar V, Schultink W, Spahn K. Vitamin A supplementation in Indian children. Lancet. 2013 Aug 17;382(9892):591-2.
Masanja H, de Savigny D, Smithson P, Schellenberg J, John T, Mbuya C, Upunda G, Boerma T, Victora C, Smith T, Mshinda H. Child survival gains in Tanzania: analysis of data from demographic and health surveys. Lancet. 2008 Apr 12;371(9620):1276-83.
Mayo-Wilson E, Imdad A, Herzer K, Yakoob MY, Bhutta ZA. Vitamin A supplements for preventing mortality, illness, and blindness in children aged under 5: systematic review and meta-analysis. BMJ 2011; 343.
Mayo-Wilson E, Imdad A, Herzer K, Bhutta ZA. Vitamin A supplementation in Indian children. Lancet. 2013 Aug 17;382(9892):594
Schultink W. Use of under-five mortality rate as an indicator for vitamin A deficiency in a population. J Nutr 2002;132:2881S-3S.
Sloan NL, Mitra SN. Vitamin A supplementation in Indian children. Lancet. 2013 Aug 17;382(9892):593. .
Sommer A, West KP Jr, Martorell R. Vitamin A supplementation in Indian children. Lancet. 2013 Aug 17;382(9892):591.
Stevens GA, Bennett JE, Hennocq Q, et al. Trends and mortality effects of vitamin A deficiency in children in 138 low-income and middle-income countries between 1991 and 2013: a pooled analysis of population-based surveys. Lancet Glob Health. 2015;3:e528-e536.
UNICEF. The State of the World’s Children 2014 In Numbers: Every Child Counts. New York 2014.
UNICEF, WHO, Bank W, UN. Levels & Trends in Child Mortality. Report 2014. New York: UNICEF;
2014.
van den Ent MM, Brown DW, Hoekstra EJ, Christie A, Cochi SL. Measles mortality reduction contributes substantially to reduction of all cause mortality among children less than five years of age, 1990-2008. The Journal of infectious diseases 2011;204 Suppl 1:S18-23.
Wirth JP, Petry N, Tanumihardjo SA, Rogers LM, McLean E, Greig A, Garrett GS, Klemm RD, Rohner F. Vitamin A Supplementation Programs and Country-Level Evidence of Vitamin A Deficiency. Nutrients. 2017 Feb 24;9(3).