Helen Keller International's Vitamin A Supplementation Program

Donate

Published: April 2024

Summary

What do they do? This review focuses on Helen Keller International (“Helen Keller”)’s vitamin A supplementation (VAS) program in sub-Saharan Africa. Helen Keller provides funding to governments to deliver VAS campaigns for preschool age children. It also provides technical support on various aspects of these campaigns including planning, training of distributors, and promoting awareness of the campaign, and conducts monitoring to understand how many children are reached (more).

We recommend Helen Keller International’s VAS program as a top charity because:

• We believe that VAS is a highly cost-effective program. (More)
• Helen Keller has a long track record of supporting VAS, which positions it well to carry out and expand VAS programs. (More)
• Transparency—Helen Keller shares significant information about its work with us. (More)

Our main reservations about Helen Keller International’s VAS program are:

• While we think it is valuable overall, we have some reservations about Helen Keller’s monitoring. This reduces our confidence that its programs are reaching a high proportion of targeted children. (More)
• We have sometimes found information from Helen Keller hard to interpret. This means that we have not been able to understand differences between the delivery models used in different locations in as much detail as we would like. (More)

This assessment of Helen Keller is based on the following components:

What do you get for your dollar? GiveWell believes that Helen Keller’s VAS program is one of the most cost-effective programs that donors can support. As of February 2024, we estimate that it costs between ~$1,000 and ~$8,500 (varying by country) to avert a death through Helen Keller-supported VAS campaigns. We think VAS is cost-effective because there is evidence that it reduces child mortality and it is very cheap to deliver.

We have a number of uncertainties about the evidence for VAS in general, including whether it is still effective in the context of improved child health since the studies we use in our analysis were conducted and some studies unexpectedly finding small or no effects. See our separate VAS report for more details.

What information has Helen Keller shared on its program? We ask organizations that we fund to share monitoring data and other detailed information on their programs. We use the data as inputs in our cost-effectiveness analysis, and its quality and reliability also inform our overall assessment of the program. For Helen Keller, this includes post-campaign surveys to estimate the proportion of children reached, cost data, and information on other actors’ likely spending on future campaigns in the countries where it supports VAS. (More)

We think this information is valuable overall, and provides some evidence that Helen Keller’s programs are reaching a relatively high proportion of targeted children at low cost. However, we have some reservations that reduce our confidence in it:

• Helen Keller’s surveys have only covered some campaign rounds, and in some surveys only some regions or districts are included. We do not have a strong understanding of why these rounds and locations were selected. Implementers may also know which locations will be surveyed in advance, and have a stronger incentive to ensure high coverage in the surveyed locations. This raises concerns that the monitoring data we’ve seen may not be representative, and that non-surveyed locations might have lower coverage rates. (More)
• Our analysis of Helen Keller’s cost and coverage data suggests a wide variation in costs between different locations where its program operates. We think it’s possible that this is because the cost information we received from Helen Keller was not comprehensive of all costs from other actors contributing to its campaigns, although Helen Keller has also suggested other explanations that we think are plausible. (More)

What is GiveWell’s qualitative assessment of Helen Keller? We make qualitative assessments of our top charities alongside our cost-effectiveness analyses to inform our grantmaking. Overall, our assessment of Helen Keller on most criteria is average, and we have not assessed it as standing out in any dimension (although we see all GiveWell top charities as exceptional relative to the majority of organizations). Factors informing our assessment include (more):

• Our impression has been that Helen Keller is not as concerned as GiveWell about the monitoring issues we identify above.
• Helen Keller has a long track record of delivering VAS, with in-country presence and relationships with ministries of health in many high-priority countries.
• The information Helen Keller has shared with GiveWell is detailed, but we have sometimes found it challenging to fully understand and interpret.

• 1

  “Today, Helen Keller Intl helps children and families in 20 countries across Asia, Africa, Europe and the United States grow and eat nutritious food, stave off malnutrition, build strong immune systems, access life-saving medical treatments, and prevent and treat blindness and vision loss.
  By providing the right support at the right time, we help millions of families and communities overcome longstanding cycles of poverty, helping them create lasting change in their own lives.” Helen Keller, ‘What we do’ page. Accessed June 2nd 2023.

  “42 states require regular vision screenings for students, but many school districts are unable to secure the budget to screen the youngest members of society. For just $35 a student, Helen Keller bridges this gap by partnering with schools to reach every child with a vision screening, and when needed, a pair of prescription eyeglasses.” Helen Keller, ‘Protecting Vision in the United States’ page. Accessed June 2nd 2023.

• 2

  Between 2018 and 2021, Helen Keller supported VAS campaigns in nine countries with GiveWell funding. See this spreadsheet for details by country.

• 3

  "In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation). The quality of the available evidence for all-cause mortality was high, whereas for all other critical outcomes it was moderate to very low. The quality of the available evidence for outcomes in human immunodeficiency virus (HIV)- positive children was moderate for all-cause mortality." WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011, p. 1.

• 4

  “Vitamin A deficiency (VAD) impairs body functions and may cause death. Adverse health consequences may also include xerophthalmia (dry eyes), susceptibility to infection, stunting and anaemia (Sommer 1996; Rice 2004).” Imdad et al. 2010, p. 9.

  Xerophthalmia (dry eyes), is “the leading cause of preventable childhood blindness” WHO, Global prevalence of vitamin A deficiency in populations at risk, 2009, Pg 1.

• 5

  “Chronic VAD may develop when animal sources and fortified foods are limited, as in diets that rely heavily on vegetables and fruits (Ramakrishnan 2002).” Imdad et al. 2010, Pg 9.

• 6

  “In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation).”
  The 4-6 month recommended schedule is in the WHO guidelines on pg 5.
  WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011, pp. 1, 5.

• 7

  “Mass distribution campaigns are the main delivery mechanism for VAS. These campaigns are organized at least every 6 months…”
  "Because mass campaigns take place only every 4 to 6 months, children who reach the age of 6 months between two campaigns, may have to wait several months before they get their first dose of Vitamin A despite being the most vulnerable age group.

  "To remedy this, HKI is working closely with country-level health sector experts to add a contact point in national immunization calendars – at 6 months, when no other vaccination is scheduled.

  "Additionally, HKI supports routine facility-based and outreach delivery of vitamin A for all children under 5 in countries where stronger health systems offer sufficient access to quality services. Few countries are ready for this approach and these still need to develop social mobilization actions to create demand to match the capacity to offer services." Helen Keller, VAS overview brochure, p. 2.

• 8

  When paired with polio campaigns, these campaigns are known as “national immunization days” (NIDs). These campaigns are typically conducted door-to-door. When paired with deworming and other child health interventions, these campaigns are known as “Child Health Days” or “Child Health Week”. These may be either door-to-door or fixed site.

  “In polio vaccination campaigns – also known as national immunization days (NIDs) – health workers go door-to-door providing oral polio vaccines. These campaigns generally occur one or more times per year. Because teams are already going door-to-door, it is relatively simple and inexpensive to add an additional person to each distribution team to deliver VAS to children aged 6 to 59 months.

  Child health days (CHDs)
  The term ‘child health day’ can refer to many different types of programs, but is generally some type of biannual event that provides a package of health interventions for preschool-aged children. There are two broad types of CHDs:

  • Mobile aka door-to-door distribution
    • Mobile CHDs involve health workers going door-to-door to provide communities with vitamin A supplements and other health services. This strategy is operationally very similar to polio NIDs.
  • Fixed-site distribution
    • In fixed-site CHDs, caregivers must bring their children to health centers or other fixed distribution sites to receive the package of health services.
    • Many CHD programs utilize a ‘fixed + outreach’ strategy, in which they implement outreach activities to encourage caregivers to bring their children to the fixed site to receive health services.”

  GiveWell's notes from a site visit with Helen Keller to Conakry, Guinea, October 9-11, 2017, p. 3.

• 9

  Note: the World Health Organization recommends VAS every four to six months, but we have not seen Helen Keller supporting campaigns at four-month intervals.

  “In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation).”
  The 4-6 month recommended schedule is in the WHO guidelines on pg 5.
  WHO Guideline: Vitamin A supplementation in infants and children 6-59 months of age 2011, pp. 1, 5.

• 10

  Note: Helen Keller reports that six countries use a door-to-door campaign approach (Guinea, Mali, Niger, Burkina Faso, Cameroon, and Madagascar). See Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, figure 2 (pg. 7).

  Helen Keller also reports that its program in DRC and Côte d’Ivoire uses a hybrid “routine + catch-up campaigns” approach, consisting of routine delivery through primary healthcare clinics for most of the year, with biannual “catch-up campaigns” to reach children who are missed. However, our understanding is that these campaigns are virtually identical to the conventional door-to-door campaigns that Helen Keller supports in other locations, with the only operational difference being more checks of vaccine cards to avoid double dosing. We therefore include these countries in our summary of countries where Helen Keller supports door-to-door campaigns. See Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, figure 2 (pg. 7).

  Our understanding of the operational similarities between these campaigns and the campaigns in other countries is based on an unpublished conversation with Helen Keller, December 7th, 2023.

• 11

  “Fixed-site distribution

  • In fixed-site CHDs [Child Health Days], caregivers must bring their children to health centers or other fixed distribution sites to receive the package of health services.
  • Many CHD programs utilize a 'fixed + outreach' strategy, in which they implement outreach activities to encourage caregivers to bring their children to the fixed site to receive health services."

  GiveWell's notes from a site visit with Helen Keller to Conakry, Guinea, October 9-11, 2017, Pg 3.

• 12

  See Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, pp. 21-22 (for Kenya) and pp. 26-7 (for Nigeria). Note that Helen Keller also supports campaigns using this approach in Tanzania (see pp. 29-30), but this is covered by other funding sources and has not used GiveWell funding.

• 13

  Examples:

  • Kenya: “Since 2020, Helen Keller has been conducting a study in Siaya county to compare the effectiveness of using community volunteers to distribute VAS routinely in their catchment areas versus distribution through facilities, and multiple outreach platforms including campaigns. The intervention is completed, and data analysis is currently underway. The report is expected to be completed before the end of quarter one in 2022.” Helen Keller, Annual Report to GiveWell on Helen Keller International’s Vitamin A Supplementation Activities, 2022, p. 7. GiveWell has since reviewed this report.
  • Cameroon: “Two studies were conducted in Cameroon from January 2022 to July 2023. A bottleneck study of the twice-yearly VAS delivery through campaign took place from September to October 2022. Results from this study were crafted into actions to improve the SASNIM that took place in December 2022. From February 2022 to February 2023, Helen Keller supported the Ministry of Health in a project that implemented a self-monitoring approach in the routine health system. This approach aimed to strengthen the routine delivery of VAS in health facilities and increase caregiver uptake of VAS at health facilities in the health districts of Kaele and Guidiguis. In this project, administrative service utilization data were analyzed and compared to the census population data of target children who are due for VAS. If children were identified as nearing their due date for supplementation, the community health volunteers visited, reminded, and referred caregivers to the nearest health facility so that their child could receive VAS. If the child did not receive VAS within the month following the reminder/referral, a second visit was made to the household during the following month. During the review of administrative coverage data the end of every month, health workers organized and the community health workers would go to a targeted fixed point within the catchment areas with highest default rate (or, in other words, lowest VAS coverage of targeted children for that month). Community mobilization activities such as radio and community leadership announcements would promote/advertise that parents take their children to these fixed points if they were due or overdue for VAS. Using this approach for routine VAS distribution, Guidiguis and Kaele were successful in achieving high VAS coverage on a monthly basis throughout the pilot study (overall coverage throughout the study period: 87.6% in Guidiguis and 89.11% in Kaele). The cost per child supplemented with vitamin A in Kaele and Guidiguis was an estimated 497.11 FCFA (approximately 0.75 USD). The results from this study are available in a report and will be further developed into a paper for submission to an international journal for consideration for publication. These results are also currently being considered by the Ministry of Health and partners to inform the scale up of routine VAS delivery in Cameroon.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, pg. 36.

• 14

  “The Ministry of Health aims to transition all 113 districts to this hybrid approach in the coming years. To ensure that the coverage of vitamin A supplementation (VAS) remains above 80%, Helen Keller Int’l will accompany this transition. In the initial years, some districts will continue distributing VAS through the traditional campaign approach, while others will be supported in adopting the hybrid approach combining routine delivery and campaigns. In certain districts, a combination of facility delivery, outreach distribution sessions, and delivery by community volunteers will be tested and later scaled up.” Helen Keller reports that it plans to work with the Ministry of Health to transition 75 of the 100 districts it plans to support to this latter model by 2027. See table 11, Helen Keller International, 2023 room for more funding report, pg. 19.
  We do not have a strong understanding of what this transition will involve. Our initial understanding is that Helen Keller will be conducting an operational pilot in 8 health districts before expanding to the remaining planned districts. The pilot will test a model of routine VAS delivery that we understand is similar to the Cameroon pilot described in the footnote above. Health workers will conduct a census of children in their areas, and conduct outreach visits to children nearing their scheduled VAS date to encourage them to go to health clinics. Fixed post outreach events to deliver VAS will be conducted for areas with low coverage. However, we have not seen a detailed operational plan for this approach. This understanding is based on a conversation between GiveWell and Helen Keller, November 7th, 2023.

• 15

  “Initially, VAS coverage rapidly reached more than 80% of children aged 6 to 59 months in sub-Saharan Africa through its integration in door-to-door polio vaccination campaigns. However, as these campaigns began to phase out following progress in the eradication of polio in Africa in the 2010’s, VAS coverage started to decline.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, p. 6.

• 16

  “When polio campaigns take place, the World Health Organization and the Global Polio Eradication Initiative cover the cost of the core teams, and VAS stakeholders “only” have to support the addition of one distributor. When there is no polio campaign, VAS partners usually support two distributors per campaign, resulting in significantly higher costs. The frequency of polio campaigns organized in Helen Keller countries continues to decline (see Table 2). The darkest cells in Table 2 show occurrences of polio campaigns where VAS could not be integrated, either for reasons of timeline or refusal of WHO. In situations where VAS could not be “piggy-backed” on polio campaigns, we had to increase funding support for VAS in these countries. Thus, in these situations, large variations occurred between the amounts originally budgeted by the country and amounts spent.” Helen Keller, Room for More Funding Report, July 2021, Pg. 7.

  Note: although we would expect campaign costs to have risen over time because of fewer opportunities to piggyback on polio campaigns, GiveWell’s analysis suggests that Helen Keller’s cost per supplement actually fell between 2020 and 2022 (from $1.10 on average in our 2020 analysis to $1.02 in our 2022 analysis). We do not have a good understanding of the reason for the discrepancy.

• 17

  "For distribution sites visited by an independent HKI supervisor, 86% met the criteria for minimum quality threshold for service delivery defined as 1) health worker used scissors to cut the capsule 2) health worker asked the age of the child 3) health worker squeezed the drops into the child’s mouth 4) health worker used the correct dose of VAS and 5) were there no stock-outs of VAC." Helen Keller VAS project year 1 report 2014 (unpublished).

  See the guidelines for health workers in Helen Keller International, VAS supervision checklist: universal and Helen Keller International, Tanzania social mobilization toolkit: VAS administration guide.

• 18

  Helen Keller International, Tanzania social mobilization toolkit: VAS administration guide:

  • "Ask the age of the child to determine the appropriate dose of vitamin A (6-59 months) and whether the child is old enough for a deworming tablet (12-59 months)."
  • "If the distribution point runs out of red (200,000 IU) capsules, two blue (100,000 IU) capsules can be given in place of one red capsule. If the distribution points runs out of blue capsules squeeze half the number of drops from a red capsule into the mouth of a child aged 6-11 months."

  See additional guidelines for dosage selection in Helen Keller International, VAS supervision checklist: universal.

• 19

  Other NGOs (e.g., the World Bank and World Health Organization) have also provided some financial support for VAS campaigns in some locations. Our understanding is that this is usually funding other interventions co-delivered with VAS. The level of support has varied by location and over time. See the explanatory notes on our cost per supplement analysis for further detail by country.

• 20

  This understanding is based on many conversations with Helen Keller and other VAS stakeholders over time.

• 21

  “Health workers distribute vitamin A capsules at health facility level, and community health workers are responsible for vitamin A distribution in communities, using either door-to-door visits or outreach distribution stations. Community distributors are usually organized in pairs and are equipped with tally sheets to record each capsule they distribute, indicating the age group and the sex of the child receiving it.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, pg. 10.

• 22

  For a detailed description of each partner organization’s role by country in 2021 campaigns, and Helen Keller’s expectations for their roles in 2022 campaigns, see Helen Keller, Room for More Funding Report, July 2022. Examples:

  • Cameroon: “Helen Keller resumed its support to the government of Cameroon for VAS during the second semester of 2021 in six regions: Centre, Littoral, West, South, Adamawa and East. These six regions represent close 3 million children 6-59 months of age (see Table 9), accounting for around 50 percent of all children in Cameroon. It also includes internally displaced persons (IDP) who are victims of insecurity in Northwest and Southwest regions. The other 4 regions were supported by Nutrition International (2) and UNICEF (2).” P. 9.
  • Guinea: “No significant change occurred in the support provided by Helen Keller in Guinea. For the two rounds of 2022, Helen Keller supported 5 regions and UNICEF 3 regions.” P. 13.
  • Mali: “In 2021, as UNICEF experienced funding gaps, Helen Keller was requested to support the additional region of Koulikoro for both semesters. Nutrition International also supported several regions. In 2022, it is planned that Helen Keller continues to support the 3 regions of Kayes, Ségou and Koulikoro for both semi-annual VAS distribution 2 rounds. UNICEF will support the remaining 8 regions and the Bamako District with various sources of funds, including some received from Nutrition International.” P. 15.

• 23

  GAVA, ‘About’ page. “Core Partner Agencies: UNICEF, Nutrition International, Helen Keller International”. Accessed June 1st, 2023.
  “These coordination forum[s] are always embedded in national coordination bodies chaired by the Ministry of Health. Decision on which regions each actor will support is taken based on availability of funds for each actor and historical presence of the actors in specific regions, for VAS or for other programs.” Helen Keller International, comments on a draft of this page, December 14th, 2023.

  Note that GiveWell only has a limited understanding of exactly how this coordination process works.

• 24

  “Micronutrient Initiative (MI) (the name of the organization changed ~1 month ago to Nutrition International or NI) is only active in 4 of the 13 countries were HKI is operational, however MI provides the needed number of vitamin A capsules to all countries where HKI works. MI’s role essentially takes place at the national level, providing technical and policy guidance to governments. In most cases, MI delivers the vitamin A capsules to UNICEF, who organizes their management with the national government and ensures that they reach the field."
  HKI country-level technical support related to vitamin A supplementation (unpublished), p. 2.

  “We have been a global leader in vitamin A supplementation since our inception 30 years ago, supporting governments to integrate VAS into existing health platforms and ensuring children are not missed with lifesaving VAS. We have led in setting the manufacturing standards and providing vitamin A capsules to eligible countries through the in-kind donation program – implemented with UNICEF, and with support from the Government of Canada – procuring and donating more than 10 billion capsules since 1997.”
  Nutrition International, ‘Strengthening health systems to deliver lifesaving vitamin A’. Accessed May 2023.

• 25

  Burkina Faso, Cameroon, Côte d'Ivoire, Democratic Republic of the Congo (DRC), Guinea, Kenya, Mali, Niger, and Nigeria. In 2021, Helen Keller also used a small amount of funding to support non-campaign VAS activities in Cameroon, Senegal, and Sierra Leone. See Helen Keller International, 2022 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, pp. 5-6.

• 26

  Created using https://www.mapchart.net/.

• 27

  See this spreadsheet for a breakdown of Helen Keller’s budgets on campaigns from 2018 to 2021, by country. Budgets for "sub-agreements" are funds that Helen Keller plans to grant to governments for program implementation.
  Helen Keller International told us that grants are typically provided to both national and subnational governments, with different activities covered by each.

  “The model used in most countries involves contracts with administrative units and at central level.
  a) Sub-agreements at the national level facilitate coordination meetings, supervision of Ministry of Health staff at the central level in the field, and occasionally logistics for sending supplies from the national level to the field.
  b) Sub-agreements at the administrative unit level cover training of actors, communication, social mobilisation activities, logistics, supervision, local coordination meetings and data validation meetings.” Helen Keller International, comments on a draft of this report, January 11th, 2024.

• 28

  “The resulting budget, along with the detailed information from the microplans, form the basis for sub-agreements signed with government entities.

  These sub-agreements define in detail the roles and responsibilities of Helen Keller and the governments in the implementation of VAS activities and are mainly used for campaigns. Sub-agreement funds for government activities and staff represent approximately 70% of the total implementation costs and are established with local administrative entities. They describe the legal obligations of the government and all aspects of campaign implementation such as staff training, social mobilization, distribution, and data management. Sub-agreements also describe payment terms, milestones and deliverables associated with each payment.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, pg. 9.

  Helen Keller notes that some funding for specific activities (e.g., coordination, planning meetings and data review meetings) is provided as “direct financial support”, paid for by Helen Keller, and not covered by the sub-agreement. Helen Keller, comments on a draft of this report, January 11th, 2024.

• 29

  "To date, HKI’s VAS project has undertaken three main types of activities, which can roughly be categorized as disbursing sub-grants to the government, providing technical assistance, and engaging in advocacy efforts." HKI External Evaluation and HKI Response - Canada DFATD VAS Project 2015, p. 35 (unpublished).

• 30

  “Helen Keller supports governments in developing microplans for both routine and campaigns. For routine, the process takes place at the beginning of every year with the government and partners. For campaigns, micro-planning begins approximately six weeks prior to the campaign (or routine mop up) activities and is characterized by several key inputs:

  • Standardized micro-planning: Helen Keller has developed standardized guidelines to support governments to conduct this process, with daily objectives for distribution teams in both urban and rural areas of each supported region. These guidelines serve as a basic framework for accurately calculating the number of distributors, supervisors and social mobilisers needed. Implementing these standards ensures that the workload for each category of actor remains manageable, significantly increasing the campaign's chances of success.
  • Comprehensive micro-plans: based on these standards, comprehensive micro-plans are developed with budgeted activities at all levels, from national level to health facilities. In addition defining the target populations and staffing requirements, these micro-plans specify supplies needs, address logistics processes, and create structured monitoring circuits.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, p. 9.

• 31

  “Helen Keller is involved in coordinating and supporting training for all actors engaged in vitamin A supplementation at all levels of the health system. Our involvement includes the comprehensive review and improvement of training tools to capitalize on lessons learned from previous campaigns and the rollout of training tools. We work closely with Ministries of Health to facilitate training at all levels, usually training trainers ourselves at national level and supporting these trainers cascade the training down to the field level. This cascading approach ensures that health workers at facility level are trained by the district management teams, while community health workers are equipped with the necessary skills through training by health workers.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, pg. 9-10.

• 32

  “Social mobilization: Promoting participation and buy-in from different social groups and local actors is critical to the success of the campaign. To inform communities about VAS campaigns and other related activities, we build the capacity of communication focal points and community mobilisers. Social mobilization activities, carried out by well-trained mobilisers from neighborhoods, villages, or communities, play a key role in behavior change. For campaigns, these mobilisers carry out door- to-door communication working closely with distribution teams, identify children, address possible refusals, and use their community connections to increase the effectiveness of distribution. These communication activities last for five days, starting three days before the start of the campaign and continuing for the first two days of campaign implementation. Helen Keller promotes the use of pre-identified mobilization channels tailored to specific environments. Local radio stations, administrators, health workers, educators, traditional and religious leaders, and town criers are used to disseminate information effectively. Banners and posters are used for visibility. Local radios are used to announce campaign dates and provide information on the benefits of vitamin A, the target age groups, administration methods, and other related activities.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, p. 10.

  Examples of social mobilization and marketing materials:

  • Helen Keller International, Tanzania social mobilization toolkit: VASD posters
  • Helen Keller International, Tanzania social mobilization toolkit: mobilization script
  • Helen Keller International, Tanzania social mobilization toolkit: mobilization script (Swahili)
  • Helen Keller International, VAS television commercial: DRC (French)

  Helen Keller told us that it uses more time and funding on social mobilization in countries where caregivers bring children to receive vitamin A supplements at fixed sites than in countries using door-to-door distributions. David Doledec, Helen Keller Regional VAS Program Manager, and Rolf Klemm, Helen Keller Vice President of Nutrition, conversation with GiveWell (unpublished), July 18, 2018.

• 33

  “Independent monitoring: Throughout the distribution process, Helen Keller establishes a robust independent monitoring system that follows a WHO-approved method to ensure equitable access to vitamin A capsules for all children. 1 Carefully trained independent monitors with no affiliations to the health system are contracted to assess the coverage of localities previously visited by distributors, and monitoring visits are conducted to ensure the effective execution of campaign activities. Independent monitoring is conducted in areas with low coverage based on the administrative and coverage data from previous campaigns. Additionally, it is conducted in hard-to-reach areas suspected to not have been reached (or at risk of not being reached) by distributors.

  • Supervision: Supervision of distribution activities is a joint endeavor by supervisors from Ministries of Health (central, regional, district, and health facility levels) and Helen Keller teams. During campaign implementation, supervisors follow predefined circuits with districts and health facilities to monitor the campaign implementation. Daily synthesis meetings are conducted at each level during the campaign to assess its strengths and weaknesses, identify areas for improvement, and address any urgent challenge.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, p. 10.

• 34

  "In most countries, HKI teams spent around 10% of their time working with the national government to advocate for VAS. HKI advocated for domestic budgets to take a greater proportion of the costs of VAS, to integrate VAS in national health and nutrition policy documents and in pluriannual strategies or action plans, supporting coordination between actors and sectors and promoting monitoring of VAS at national level to provide the government with a comprehensive vision of the services for the whole country." HKI country-level technical support related to vitamin A supplementation (unpublished), p. 1.

• 35

  "Because mass campaigns take place only every 4 to 6 months, children who reach the age of 6 months between two campaigns, may have to wait several months before they get their first dose of Vitamin A despite being the most vulnerable age group.
  To remedy this, HKI is working closely with country-level health sector experts to add a contact point in national immunization calendars – at 6 months, when no other vaccination is scheduled." Helen Keller, VAS overview brochure, p. 2.

• 36

  GiveWell-directed funding, including donations from Open Philanthropy and other funders, supported 81% of Helen Keller's VAS campaign work in 2018, 78% in 2019, 95% in 2020, and 99% in 2021. See this spreadsheet for more details about Helen Keller's spending on VAS campaigns from 2018 to 2021.

• 37

  We have also received information from Helen Keller on its spending from January 2022 to June 2023, but have not yet incorporated this into our summary.

• 38

  See the ‘Summary [2021]’, ‘Summary [2020]’, and ‘Summary [2019]’ sheets in this spreadsheet.

• 39

  See this section of the spreadsheet for a breakdown by country.

• 40

  See this section for a discussion of what these agreements cover.

• 41

  See this section of the spreadsheet for a breakdown by category.

  Direct program costs include all budget items in this sum (25% of total spending) and the activities section of the budget, as combined in this sum (25% of total spending). 25% + 25% = 50%.

• 42

  See this section of the spreadsheet.

• 43

  We raised this issue with Helen Keller in December 2022. Helen Keller has informed us that it plans to share the results from this procedure in all future surveys. We have not yet reviewed the subsequent round of surveys to analyze the findings from this procedure. Helen Keller International, Email to GiveWell, Jan 9th 2023 (unpublished).

• 44

  Our 2022 analysis includes the first round 2022 campaigns in Cameroon, DRC, and Kenya. See this section of our analysis for more details.
  Since we last updated our analysis, Helen Keller has shared the remaining surveys it conducted in 2022 and additional surveys from the first half of 2023. We have not yet analyzed these surveys.

• 45

  See this section of our analysis. Note that this calculation includes only surveys from 2018 to 2021. We exclude 2022 from our analysis because, at the time the analysis was conducted, some 2022 campaign coverage surveys were due to be conducted but data was not yet available.

• 46

  See columns B and D in this sheet in our analysis for details of which round and which locations surveys were conducted in, by year.

• 47

  Source: GiveWell conversations with Helen Keller International, September 6 and October 4, 2022 (unpublished).

• 48

  Our analysis suggests that Helen Keller’s surveys may increasingly be sampling fewer regions or districts over time.

  • In 2019, all the surveys we reviewed sampled all Helen Keller-supported regions included in the campaign (with the exception of the Guinea round 2 survey, where some locations could not be reached because of security risks). See this row in our analysis.
  • In 2020, all the surveys we reviewed sampled all Helen Keller-supported regions included in the campaign. The exception was the Kenya round 2 survey, where 2 of 6 regions were surveyed. See this row in our analysis.
  • In 2021, two of six surveys that we reviewed (Burkina Faso round 2 and Nigeria round 2) did not comprehensively sample all regions. See this row in our analysis.
  • Our 2022 analysis only contains 3 2022 surveys which were available at the point of analysis. Of these, the DRC survey did not comprehensively sample all provinces. The Kenya survey surveyed 1 county and the Cameroon survey surveyed 2 regions. We expect that these surveys were not comprehensive of all regions, although at the time we conducted the analysis we did not know how many regions were included in these 2 surveys. See this row in our analysis.

• 49

  For further details by survey, see row 4, “What was the study population?”, in the following sheets in our analysis of Helen Keller’s monitoring:

  • 2022 surveys
  • 2021 surveys
  • 2020 surveys
  • 2019 surveys

  As an example, the Cameroon 2022 coverage survey (round 1) was conducted in Centre and Littoral regions only, out of the six regions supported by Helen Keller in 2022. We do not know why these two regions were selected.

  In one coverage survey report we have analyzed (Burkina Faso, round 2, 2021), Helen Keller states that some non-random factors were used to select survey districts (two districts in two regions were surveyed, but the campaign took place in 13 regions). “This study will be conducted in two health districts. These are the health district of Kombissiri which is in the health region of the Center South and the health district of Yako which is in the health region of the North...These two sites were highlighted in collaboration with the Directorate of Nutrition, the Regional Health Directorates of the Center South and North, for several reasons including: (i) accessibility. Districts are easily accessible for data collection teams; (ii) security. These are sites that are not prey to recurrent terrorist attacks in certain localities of Burkina Faso; (iii) the existing collaboration between HKI, UNICEF and the health authorities of the two study sites." (Helen Keller, post-event coverage survey report, Burkina Faso, 2021 (translation)), p. 15. We do not know whether these factors were also used to select districts or regions in other coverage surveys.

• 50

  GiveWell, Conversation with Helen Keller, October 4th, 2022 (unpublished).

• 51

  GiveWell, Conversation with Helen Keller, October 4th, 2022 (unpublished).

• 52

  GiveWell, Conversation with Helen Keller, September 6th, 2022 (unpublished).

• 53

  Helen Keller, email to GiveWell, November 7th, 2023 (unpublished).

• 54

  See Helen Keller, Post-event coverage survey guide, 2019.

• 55

  Rolf Klemm, Helen Keller Vice President of Nutrition, and David Doledec, Helen Keller Regional VAS Program Manager, conversation with GiveWell, March 30, 2020 (unpublished).

• 56

  According to Helen Keller, a different sampling method was used for 2020 coverage surveys in Kenya and Nigeria. For more detail, see the "Nigeria (round 1) PECS" and "Kenya (round 2) PECS" columns of this spreadsheet. We don't know why Helen Keller used a different methodology for these surveys.

• 57

  "In the case of PECS, a stratified two-stage cluster survey will be conducted. The stratification criteria are: the intervention area and the nature of the areas (urban, rural) according to the
  Context." Helen Keller, Post-event coverage survey guide, 2019, p. 17.
  For details of standard sampling procedure, see Helen Keller, Post-event coverage survey guide, 2019, pp. 18-24.

• 58

  See this row in the “Methods (2021)” sheet of our analysis.

• 59

  See this row in the “Methods (2021)” sheet of our analysis.

• 60

  “This first step will consist in listing all the eligible households in each cluster. The maps of sampled clusters will be handed to the enumerators who, once in the cluster, will enumerate all the households in the cluster and enter the identification number of each identified household on their door and on the household census form. . . . Each evening, all the team leaders will report the households surveyed by cluster to the senior supervisor, indicating the number of eligible households identified per cluster. A random sampling using the function RANDBETWEEN (min, max) of Excel will be used for the household sampling. It's a simple matter of entering as minimum value 1 and maximum value the number of eligible households in the cluster and then draw the function on the number of households to be surveyed. Thus, we have a list of number of households and only these households will be surveyed (since each household in the cluster is numbered from 1 to n). An inventory form of households selected in this way will be established for monitoring.” Helen Keller, Post-event coverage survey guide, 2019, pp. 30-31.

• 61

  "This number of households to be surveyed per day recommended by WHO is generally between 5 and 15, and is influenced by the number of people who work in each field data collection team. We usually choose 10 but this number can increase with the proportion of non-response measured in a previous survey. To further increase the representativeness of the survey, a weighing system is applied to data before analysis to account for number of households in each enumeration area and number of children." David Doledec, Program Director, Vitamin A Supplementation, Helen Keller International, Suggested edits to this page, January 2023 (unpublished).

• 62

  See this row in the “Methods (2021)” sheet of our analysis.

  Our understanding that all eligible children in selected households are surveyed comes from Rolf Klemm, Helen Keller Vice President of Nutrition, and David Doledec, Helen Keller Regional VAS Program Manager, conversation with GiveWell, March 30, 2020 (unpublished).

• 63

  “To determine the eligibility of a child, enumerators should follow the procedure below:

  1. Ask the mother or caregiver if the child's health card or birth certificate is available.
  2. If the health card is not available, ask the caregiver if he/she remembers the exact date of birth of the child.
  3. If the caregiver does not remember the child's exact date of birth, the interviewer should ask if he/she remembers the specific month and year in which the child was born.
  4. If the caregiver cannot remember the month of birth, a calendar of local events can be used to estimate when the child was born. A young child who walks is at least 9 months old and would be eligible for the interview.”

  Helen Keller, Post-event coverage survey guide, 2019, p. 24.

• 64

  “If the respondent is absent at the first visit, return at least twice more.” Helen Keller, Post-event coverage survey guide, 2019, pg. 31.
  "In the new sampling approach, replacement households are integrated in the calculation of the sample, based on assumptions (and past surveys experience) of the proportion of households that would need to be replaced, so there is no more need to provide extra replacement households to field teams even if some households they visit happen to refuse the survey or not be eligible." David Doledec, Program Director, Vitamin A Supplementation, Helen Keller International, suggested edits to this page, January 2023 (unpublished).

• 65

  See:

  • This row in the “Methods (2022)” sheet of our analysis,
  • This row in the “Methods (2021)” sheet of our analysis, and
  • This row in the “Methods (2020)” sheet of our analysis.

• 66

  See example from 2019 here: Helen Keller, Post-event coverage survey questionnaires, 2019.

  The same questionnaires were used in 2020, with minor updates to account for changes in the package of interventions delivered alongside VAS in each campaign and to collect information on COVID-19. Helen Keller, answers to GiveWell's questions, November 2021 (unpublished).

• 67

  See:

  • This row in the “Methods (2022)” sheet of our analysis,
  • This row in the “Methods (2021)” sheet of our analysis, and
  • This row in the “Methods (2020)” sheet of our analysis.

• 68

  See Helen Keller, Post-event coverage survey questionnaires, 2019, pp. 1 - 6.

• 69

  See:

  • This row in the “Methods (2022)” sheet of our analysis,
  • This row in the “Methods (2021)” sheet of our analysis, and
  • This row in the “Methods (2020)” sheet of our analysis.

• 70

  “(Show capsules or a picture of vitamin A)” Helen Keller, Post-event coverage survey questionnaires, 2019, pg. 3.
  “(Show a tablet or deworming photo).” Helen Keller, Post-event coverage survey questionnaires, 2019, pg. 4.

• 71

  See:

  • This row in the “Methods (2022)” sheet of our analysis,
  • This row in the “Methods (2021)” sheet of our analysis, and
  • This row in the “Methods (2020)” sheet of our analysis.

• 72

  “Is it correct that the 10% re-survey procedure was used in all surveys, including those for which it is not explicitly mentioned in reports? Yes the 10% re-survey procedure was used in all surveys, indeed, the procedure for revisiting the 10% of households is part of the methodology used for the PECS surveys.

  What are the details of this procedure (How are re-survey households selected? A quality control of 10% of the data collected by the interviewers is carried out by the supervisor, thus, in each locality the supervisor randomly selects one (01) household already surveyed and asks some essential questions (i.e. linked with coverage) from the household questionnaire and adapted for a double interview in order to evaluate the coverage of services provided to these surveyed households. The household responses from the first interview will be compared to those from the second interview. The household identification number writen on their door by the interviewers helps the supervisor to easily locate the drawn households.

  Are supervisors blinded to initial results? Yes, The supervisor does not have access to the initial results before going to the field to draw the 10% of households to be surveyed. The random sampling of the 10% households is done once in the cluster by the supervisor.

  What is the process for correcting issues? If both the interviewer and the supervisor are in the cluster, the supervisor conducts the verification and provides feedback to the interviewer to return to the household if necessary or to improve future surveys. If the interviewer and the supervisor are not in the cluster together, the verification data are sent to the platform and the ONA platform manager provides daily feedback on the concordance of the data to return to improve future surveys. In all cases, the household responses from the first interview will be compared to those from the second interview.” Helen Keller, answers to GiveWell’s questions, September 30, 2020 (unpublished).

• 73

  We have seen results from this monitoring procedure in five of nine coverage surveys we have reviewed in 2021 and 2022:

  • DRC, 2022: reports data on both (a) the correspondence between the main survey and the audit and (b) coverage rates from the audit. It found that supervisors and survey teams disagree on whether 7.6% of the children surveyed received VAS. There doesn’t seem to have been any evidence of bias in the results. The coverage rates found by supervisors were very similar to those by the original survey team (91.72% vs 89.66%). Correspondence on other audit measures was also high (94% correspondence on the number of children in a household, 98% correspondence on children’s sex).
  • Kenya, 2022: reports data on coverage rates from the audit compared to the original data, but not on correspondence. Reported coverage was similar in both the original and audited data (80.5% vs 81.9%).
  • Mali, 2021: reports both correspondence and a comparison of coverage rates. Supervisors and survey teams disagreed on whether 5.7% of the children surveyed received VAS. Coverage rates found by supervisors and the original survey team were very similar (78.3% vs 76.6%). Correspondence on other measures varied between 81% (child age), 83% (number of children in the household) and 94% (child’s sex).
  • Guinea, 2021: reports on overall coverage rates in the audit and original data,. These were also very similar (87% vs 86.2%). It does not report on correspondence.
  • Niger, 2021: reports on correspondence rates (93%), but does not compare coverage rates between the initial and audited data. Correspondence on other measures varied between 78% (child age), 86% (number of children in the household) and 93% (child’s sex).

  Sources:

  • Helen Keller, DRC Post-event coverage survey report, Kasai Oriental, 2022, pp. 33-34.
  • Helen Keller, Kenya Post-event coverage survey report, 2022, Table 26, p. 45.
  • Helen Keller, Mali Post-event coverage survey report, 2021 (translation), pp.30-31.
  • Nutrition International, Guinea Post-event coverage survey report, 2021 (translation), Table E8, p. 37
  • Helen Keller, Niger Post-event coverage survey report, 2021 (translation), Tables 20 - 22, pp. 40-41.

  See this row in the “Methods (2022)” sheet of our analysis and this row in the “Methods (2021)” sheet of our analysis for further details.

• 74

  Open questions about the data we have not fully investigated:

  • Sample sizes for the supervisor audits being lower than 10% of the total sample in some cases.
  • Sample sizes for the supervisor audits differing between different questions without accompanying explanation.

  In more detail:

  • DRC, 2022: 157 supervisor surveys were completed, compared to 925 in the main sample (157 / 925 = ~17%, or higher than the 10% target). However, the reported sample size for each audited question was slightly lower than 157, at 145. We’re not sure why there was a discrepancy. Helen Keller, DRC Post-event coverage survey report, Kasai Oriental, 2022, p. 32.
  • Kenya, 2022: The supervisor sample size is not reported. The maximum sample for any single question is 72, compared to 1,088 for the main sample (72 / 1,088 = ~7%). Helen Keller, Kenya Post-event coverage survey report, 2022, Table 26, p. 45
  • Mali, 2021: 418 supervisor surveys were completed, compared to 1,695 in the main sample (418 / 1,695 = ~25%). However, the reported sample size for each audited question varied widely, from 69 to 415. We’re not sure why there was a discrepancy. Helen Keller, Mali Post-event coverage survey report, 2021 (translation), pp. 29-30.
  • Guinea, 2021: Supervisor sample sizes are not reported.
  • Niger, 2021: 82 supervisor surveys were completed, compared to 1,688 for the main sample. The reported sample size for each audited question varied from 79 to 82. We’re not sure why there was a discrepancy, although this discrepancy is small enough that we don’t see it as a significant concern. Helen Keller, Niger Post-event coverage survey report, 2021 (translation), Tables 18 - 22, pp. 40-43.

  Note that we also saw some previous audit results from Kenya and Guinea in 2020. We did not have high confidence in the quality of this data, although this was based only on a cursory review:

  • The auditing data we've seen reports sample sizes much smaller than we would expect if 10% of households had been re-surveyed, and we aren't sure whether this data is representative of the original sample. It is also unclear whether the rates of agreement in survey responses reported in the auditing data represent rates of agreement at the household level—which is the information we would be most interested in reviewing—or in the aggregate.
  • See this spreadsheet, sheet "Methods (2020)," row "Are there methods in place for assessing the accuracy of a random sample of each enumerator's entries? Does the enumerator know that auditing will occur but does not know which entries will be monitored? Were high levels of discrepancies found? What is the process for correcting issues found?"
  • The 2020 Kenya and Guinea PECS results and the audits of these results are unpublished.

• 75

  The surveys reporting results from this audit are the five mentioned in the previous footnote. See this row in the “Methods (2022)” sheet of our analysis and this row in the “Methods (2021)” sheet of our analysis for details of coverage survey reports which are missing descriptions of the audit procedure.

• 76

  Helen Keller, email to GiveWell, January 9th 2023, unpublished.
  We have not yet reviewed the subsequent round of surveys to analyze the findings from this procedure.

• 77

  See:

  • This row in the “Methods (2022)” sheet of our analysis,
  • This row in the “Methods (2021)” sheet of our analysis, and
  • This row in the “Methods (2020)” sheet of our analysis.

• 78

  Some of the survey results we have seen report coverage rates at the subnational level (such as for each individual region or district within the survey area), while others report a national coverage rate that encompasses the entire survey area. In order to avoid surveys that provide subnational coverage breakdowns having disproportionate influence on the metrics listed below, we have chosen to discuss national-level survey results in this section. In some cases, where a national coverage rate was not provided in the survey results for a particular country, we used the average of the subnational coverage rates, weighting these averages by target population where population estimates were available. See this spreadsheet, sheet "Results Analysis (2013-22)," column "Program-level VAS coverage of children aged 6-59 months, from coverage surveys."

• 79

  See this row in our analysis.

• 80

  See this row in our analysis.

• 81

  See this row in our analysis.

• 82

  See this row in our analysis.

• 83

  This is our understanding from multiple conversations with Helen Keller.

• 84

  As of February 2024, we estimate that it costs approximately $1 to deliver a vitamin A capsule to a child in Helen Keller-supported programs (varying by country, from $0.80 in Mali to $1.39 in Guinea—see this range in our cost per supplement calculations). See this section of our report on vitamin A supplementation for further details.

• 85

  See these rows of our cost-effectiveness analysis for this adjustment (-15% adjustment for misappropriation without monitoring results and -2% adjustment for false monitoring results: -15% - 2% = -17%). Specific factors feeding into the figure that we use include:

  • In most countries Helen Keller only conducts a survey after one of two campaign rounds. We have seen monitoring results for campaign rounds representing 64% of Helen Keller’s VAS spending in 2019, 52% in 2020 and 29% in 2021. (In part, our understanding is that this fall has been because of temporary disruption resulting from the coronavirus pandemic.) The lack of comprehensive monitoring makes us concerned that the coverage data we have seen overestimate the overall coverage level. Our understanding is that VAS implementers may know in advance which campaign will be surveyed. If that is the case, we would guess that workers would be more incentivized to ensure high coverage in surveyed campaigns than non-surveyed campaigns (which have less oversight). GiveWell conversations with Helen Keller International, September 6 and October 4, 2022 (unpublished).
  • We have heard feedback from Helen Keller that, in countries which have rainy seasons, its monitoring surveys tend to take place in the non-rainy season This raises a concern that, if coverage is lower during campaigns conducted in the rainy season, the results we see may be systematically overestimating coverage. GiveWell, Conversation with Helen Keller, September 6th, 2022 (unpublished).
  • Some methodological features of Helen Keller’s surveys may lead to coverage estimates being inflated. In particular, we would expect responses based on caregiver self-reports to inflate coverage because of social desirability bias.

• 86

  See this row of our cost-effectiveness analysis. Note that this estimate refers to the total cost per capsule delivered, including in-kind government costs (e.g., staff time) and donated vitamin A capsules. We use a different estimate (~$0.65 to $0.90, varying by country) without in-kind costs when estimating the number of children reached per $1m of Helen Keller spending in our cost-effectiveness analysis, and account for in-kind costs with a separate adjustment.

  The ~$0.65 to $0.90 estimate comes from this range of our cost-effectiveness analysis, which lists our estimates for the cost per child to deliver VAS each year, excluding in-kind costs (~$1.30 to $1.80, varying by country). Since 2 capsules are delivered to a child each year, we divide this range by 2 to yield the cost per capsule delivered. ~$1.30 / 2 = ~$0.65 and ~$1.80 / 2 = ~$0.90.

  For more details, see this section of our separate report on VAS.

• 87

  Our reasoning differs somewhat country-to-country:

  • Côte d’Ivoire: Our cost per supplement estimate is implausibly low ($0.53) and we have previously heard from Helen Keller that some costs may have been omitted from the information they shared with us.
  • Kenya: Our cost per supplement estimate is low ($0.78), and we have low confidence in this estimate because it relies on two coverage surveys, one in 2019 and one in 2020, that may not be representative of all Helen Keller-supported areas in Kenya.
  • DRC: Our cost per supplement estimate is implausibly low ($0.58), especially considering that Helen Keller’s program in DRC is relatively new, and we would have expected start-up costs to be high.
  • Nigeria: Our cost per supplement estimate is low ($0.72), and Helen Keller didn't report spending by any other actors in Helen Keller-supported regions for these campaigns, other than a small amount of spending by Saving One Million Lives in 2020. We would guess that some other actors’ spending may have been excluded from the totals we received.

  See this section of our explanatory notes onwards for further information.

• 88

  Helen Keller has informed us:

  • In Côte d’Ivoire, the government uses a different model with less training for distributors than in other locations.
  • In DRC, there is a lot of pressure from other actors to keep costs low. Helen Keller also attributed low costs to a highly experienced VAS team in DRC.
  • In Nigeria, VAS campaigns use a fixed point rather than a door-to-door model, which is likely to keep costs low.

  Helen Keller also agreed that costs were likely to be lower in Kenya than other countries, although we are uncertain what specific factors contributed to this. We note that Kenya (like Nigeria) also uses a fixed site + outreach distribution model, which might contribute to lower-than-average costs (more above), although Helen Keller did not cite this as a factor.

  Sources: Helen Keller, conversations with GiveWell (unpublished), October 28th 2022 and January 26th, 2023.

• 89

  See this row in our cost-effectiveness analysis and our analysis of cost per supplement in this spreadsheet. To calculate this weighted average, we weight our estimates of cost per child in each country by the number of supplements we think Helen Keller has delivered in that country.

• 90

  See this column in our cost per supplement analysis.

• 91

  Our -17% adjustment for the quality of Helen Keller’s monitoring and evaluation (discussed above) is intended to account for this concern.

• 92

  Each year, Helen Keller shares a “room for more funding” report with GiveWell. This contains Helen Keller’s estimates of the funding required to deliver the VAS campaigns it plans to support. In each country, this includes its expectations for how many regions it is aiming to support and how many can be supported by other GAVA partner agencies (UNICEF and Nutrition International). The reports also contain summaries of how many regions were supported by Helen Keller and the other GAVA partner agencies in previous years.

  Examples of these reports:

  • 2023 report
  • 2022 report
  • 2021 report

• 93

  Our understanding from the information Helen Keller has shared, and conversations with other VAS stakeholders over time, is that the global funding landscape for VAS is not very crowded, relative to some other programs that GiveWell funds.

  We think that the main funders of VAS campaigns globally are national governments (through in-kind contributions like staff time, and sometimes through financial support), Global Affairs Canada (GAC) (through grants to Nutrition International and UNICEF), UNICEF's flexible funding, and GiveWell-directed donations. Other NGOs (e.g., the World Bank and World Health Organization) have also provided some financial support for VAS campaigns in some locations. These contributions have varied by country and over time. See this sheet in our cost per supplement analysis for more details.

  Typically, in countries with support from multiple external partners, different partners focus on different regions or activities within each country. Our understanding of the way this process works is that Helen Keller, UNICEF, and Nutrition International coordinate on which regions or activities each organization will focus on within each country. Our understanding is that this process is coordinated via the Global Alliance for Vitamin A (GAVA), a forum in which Helen Keller, UNICEF and Nutritional International are the three core partner agencies. GAVA, ‘About’ page. “Core Partner Agencies: UNICEF, Nutrition International, Helen Keller International”. Accessed November 27th, 2023.

• 94

  See these rows of our cost-effectiveness analysis for these estimates and details on our country-specific reasoning.

• 95

  See this row of our cost-effectiveness analysis.

• 96

  For example, we have seen more detailed information from Against Malaria Foundation about other actors’ funding for malaria programs in the locations where it supports insecticide-treated net (ITN) campaigns. See this section of our review of Against Malaria Foundation for more detail.

• 97

  Helen Keller, email to GiveWell, November 7th, 2023 (unpublished).

• 98

  We see this as concerning because it suggests that, if practiced more widely, non-surveyed areas could have systematically lower coverage than surveyed areas. See the discussion above for more details.

• 99

  DRC: “Helen Keller proposed an "Intensive Community-Based Routinization" approach, or a catch-up activity to reach children over a period of 4 to 7 days, twice a year. The approach is implemented by CACs, supported by health facilities, and under the supervision of the health zone, the health directorate, and PRONANUT.

  Awareness-raising campaigns are carried out before distribution, over at least seven days. A VAS register and tally sheets are given to each CAC, which are later compiled by health facilities and sent to the health zone (district), which then compile data from all health facilities in their catchment area, and send these to the province. The province aggregates the health zone data before transmitting these to the national level.” Helen Keller International, 2023 annual report to GiveWell on Helen Keller International’s vitamin A supplementation activities, pg. 19.

  Côte d’Ivoire: “In Côte d'Ivoire, vitamin A supplementation is distributed through mass door-to-door vitamin A distribution campaigns in 40 districts, and an hybrid approach is used in 73 others. The hybrid approach consists of routine delivery in primary health care facilities throughout the semesters, and catch-up campaigns are conducted at the end of the semester to reach children who may have been missed by routine services.” Helen Keller International, 2023 room for more funding report, pg. 18.

• 100

  See our discussion of these audits above.

• 101

  This impression is based on many conversations with Helen Keller over time.

• 102

  GAVA is a forum in which Helen Keller, UNICEF and Nutritional International are the three core partner agencies. GAVA, ‘About’ page. “Core Partner Agencies: UNICEF, Nutrition International, Helen Keller International”. Accessed November 27th, 2023.

• 103

  In 2018, Helen Keller supported VAS campaigns in four countries with GiveWell funding: Burkina Faso, Guinea, Mali, and Côte d'Ivoire. By 2021 its program had grown to nine countries: Burkina Faso, Cameroon, Côte d'Ivoire, DRC, Guinea, Kenya, Mali, Niger, and Nigeria. See this spreadsheet for a breakdown of campaigns by location.

• 104

  Between 2018 and 2022 the median coverage in Helen Keller’s coverage surveys was 85%, and the lowest coverage estimated was 59% in one campaign in Mali (Kayes region) in 2020. See this sheet in our analysis of Helen Keller’s monitoring for further details.

  Note that our best guess is that these figures are somewhat inflated because of the concerns we list above, but we still interpret them as evidence that Helen Keller is able to scale up programs in new locations relatively quickly.

• 105

  Examples:

  • “Kenya. Since 2020, Helen Keller has been conducting a study in Siaya county to compare the effectiveness of using community volunteers to distribute VAS routinely in their catchment areas versus distribution through facilities, and multiple outreach platforms including campaigns. The intervention is completed, and data analysis is currently underway. The report is expected to be completed before the end of quarter one in 2022.” GiveWell has since reviewed this report.
  • “Niger. In 2022, Niger will start a study to implement and test two interventions for VAS delivery and measure their impact on VAS coverage. The two proposed delivery models are hybrid approaches to routine VAS delivery, providing additional supports to health facilities through the use of community health volunteers for sensitization activities and also offering mop up delivery. It is expected that a cost-effectiveness analysis will be conducted after the pilot of the two interventions.”

  Helen Keller, Annual Report to GiveWell on Helen Keller International’s Vitamin A Supplementation Activities, 2022, p. 7.

• 106

  This understanding is based on multiple conversations with Helen Keller and other VAS stakeholders over time. Additional support for this claim:

  • “Experience has shown that relying on routine health facility delivery results in significant decreases in VAS coverage. To reduce this risk, the hybrid model includes the organization of national weeks of intensification of nutrition activities with several other services such as VAS, deworming and screening for malnutrition.” Helen Keller, Room for More Funding Report, July 2022, p. 11.

• 107

  See this row of our cost-effectiveness analysis. Note: This range is based on the following locations, and not the full list of locations in our cost-effectiveness analysis: Burkina Faso, Cameroon, Chad, Côte d'Ivoire, DRC, Guinea, Mali, Madagascar, Niger, and five states in Nigeria: Adamawa, Benue, Ebonyi, Nasarawa, and Taraba.

  We focus on these locations because they are the locations that GiveWell actively supported in our most recent grant cycle (see our grant page for Helen Keller’s program here). Some columns in our cost-effectiveness analysis are for countries that GiveWell has either considered for previous grants but not yet funded (e.g., Uganda), funded in the past but discontinued funding because we estimate that VAS in that country was below our cost-effectiveness bar (e.g., Kenya), or funded through Nutrition International's VAS program (e.g., Chad). We think that the range used here is the best reflection of the value of Helen Keller's VAS program in locations supported by GiveWell donors.

• 108

  See this row of our cost-effectiveness analysis. Note: This range is based on the following locations, and not the full list of locations in our cost-effectiveness analysis: Burkina Faso, Cameroon, Chad, Côte d'Ivoire, DRC, Guinea, Mali, Madagascar, Niger, and five states in Nigeria: Adamawa, Benue, Ebonyi, Nasarawa, and Taraba.

  We focus on these locations because they are the locations that GiveWell actively supported in our most recent grant cycle (see our grant page for Helen Keller’s program here). Some columns in our cost-effectiveness analysis are for countries that GiveWell has either considered for previous grants but not yet funded (e.g., Uganda), funded in the past but discontinued funding because we estimate that VAS in that country was below our cost-effectiveness bar (e.g., Kenya), or funded through Nutrition International's VAS program (e.g., Chad).We think that the range used here is the best reflection of the value of Helen Keller's VAS program in locations supported by GiveWell donors.