Lookback: Expansion Grant for Helen Keller International’s Vitamin A Supplementation Program in Nigeria

Note: This page compares the impact of Helen Keller Intl's grant against our initial expectations. Helen Keller staff reviewed this page prior to publication.

Published: May 2025

Background

In April 2021, we recommended a grant to Helen Keller Intl's vitamin A supplementation (VAS) program to support expansion into new Nigerian states ($7.5m) and Cameroon ($3.5m) in 2021-2024.2 This lookback focuses on the portion of the grant dedicated to Nigerian states, because we think there's a good chance we allocate significant funding to Helen Keller in Nigeria going forward.

We made this grant because we thought Helen Keller's VAS program was a cost-effective way to avert child mortality and that these locations would be unlikely to achieve high VAS coverage without funding Helen Keller. (See this grant page for more information.)

Because we recommended flexible funding, we didn't know which Nigerian states Helen Keller would expand into at the time we recommended the grant.3 Helen Keller ultimately used grant funding to expand into five Nigerian states: Adamawa, Akwa Ibom, Ebonyi, Ekiti, and Taraba.4 These states were selected based on a combination of proxy indicators, such as rates of malnutrition and mortality data, and, in the case of Ebonyi, GiveWell's initial cost-effectiveness estimate.5 Helen Keller then exited Akwa Ibom and Ekiti, based on low estimated cost-effectiveness.6

The expansion and exit timeline for these locations is summarized in the below table:

	2021 S1	2021 S2	2022 S1	2022 S2	2023 S1	2023 S2	2024 S1	2024 S2
Adamawa		Started
Akwa Ibom		Started			Exited
Taraba		Started
Ekiti			Started		Exited
Ebonyi7				Started

The rest of this report provides our assessment of how the grant went and next steps, based on what we’ve learned.

Would we have made this grant again, knowing what we know now?

Knowing what we know now, we think we likely would have made a smaller grant, focusing on fewer states in Nigeria and completing a state-specific cost-effectiveness assessment to guide our work prior to directing funding to individual states.

Using our updated (2024) state-specific cost-effectiveness estimates, we would have only funded two out of the five states, Adamawa and Taraba, which are above our 8x funding bar for top charities. We estimate that the other three, Akwa Ibom, Ebonyi, and Ekiti, are below our 8x funding bar (more).

How did implementation go?

Our best guess is that the campaigns reached the intended number of children (~12 million, compared to our expectation of ~11 million), though we have some concerns about the coverage data underlying these estimates, and early exit from two states created challenges.

Children reached and coverage estimates

We think our estimate of the number of children reached by this grant was fairly accurate at the time we recommended the grant. In the first two years (mid-2021 to mid-2023), we expected the grant to deliver ~11 million supplements to ~2.75 million children.8 We think this grant actually delivered ~11m supplements to ~2.5 million children, based on data from the first two years of the grant.9

However, our underlying assumptions have changed. Previously, we assumed that Helen Keller would cover a lower proportion of total program costs, but with a higher cost per supplement. This was based on an average estimate from countries outside of Nigeria. Now, we're assuming that Helen Keller would cover a higher proportion of total program costs, but with a lower cost per supplement. This is based on information from Helen Keller's program in Nigeria over the past two years.10

We think the campaigns went fairly well, based on high reported coverage rates for the campaigns where we've seen coverage data (see table below). However, we now have some open questions about Helen Keller's monitoring and are working with Helen Keller to enact changes going forward.

Here is the reported change in coverage for the states where Helen Keller conducted PECS (post-event coverage surveys) after starting work. The January to June 2021 baseline data is from administrative data, because PECS weren't yet available, so this is a comparison across data sources that should be interpreted with some caution.

	Adamawa	Akwa Ibom	Ebonyi	Taraba
Coverage estimate, January to June 2021. See footnote for details.11	0%	16.7%-29.6%	No data available	9.9%-11.2%
PECS, 2021 S2		65%
PECS, 2022 S1	96%
PECS, 2022 S2				95%
PECS, 2023 S2			96%

As shown in the table above, Helen Keller reported increases in coverage following the start of its support. While we think it is very likely that coverage increased significantly following Helen Keller's entry, we have some reservations about taking these data at face value because:

• We think distributors may have been incentivized to ensure high coverage in surveyed rounds, because we understand that they were informed in advance whether rounds will be surveyed.12
• We have concerns about the quality of implementation of coverage surveys (particularly sampling bias), based on discrepancies between Helen Keller's coverage surveys and Demographic and Health Surveys (DHS) results in several countries where Helen Keller operates and a review of two coverage surveys from Kenya.13
• The coverage data from before Helen Keller started its support is based on a short timeframe (the first half of 2021 only) and may not be representative of typical pre-Helen Keller coverage. However, we're not aware of any specific reason to think they're unrepresentative.

Our next steps: In 2024, we discussed with Helen Keller improvements to its coverage surveys that would address some of our concerns about incentives and potential bias. Since Helen Keller agreed to make updates in response to these concerns in February 2024, we have seen initial indicators that they have updated their processes, such as having independent monitors re-visit 10% of clusters and conduct a household census,14 though we have not fully vetted these changes. Going forward, continued improvement in monitoring will be important for our ongoing support. We also plan to consider making a grant to a third party evaluator who can triangulate Helen Keller's monitoring and do a costing analysis to improve our understanding of what drives cost variations between countries.

Early exits in Akwa Ibom and Ekiti

Helen Keller exited two Nigerian states (Akwa Ibom and Ekiti) shortly after entering,15 potentially straining relationships with state governments. The exit from these states was based on our updated cost-effectiveness estimates showing these were below our funding bar (see below for more on cost-effectiveness).16

Helen Keller told us that ideally it would not have entered and exited Akwa Ibom and Ekiti in such a short time period, and that a longer exit period would enable it to develop a plan for transitioning the program back to the government.17

Helen Keller described relationships with governments as being a challenge when exiting.18 With Akwa Ibom and Ekiti, it told us that the state governments were not happy about the exit.19

According to Helen Keller, the states immediately reported lower VAS coverage after its exit, based on administrative data.20 We have not independently verified this. Helen Keller attributed the decline in coverage to a lack of a sustainability plan with the government, which it said there was no time to put in place.21 As a result of this experience, Helen Keller told us that going forward they will encourage states to develop sustainability plans immediately, rather than waiting a year or more into the program.22 We plan to follow up on this learning as part of a future grant investigation.

We’re unsure how much weight to put on Helen Keller's view that having a plan would have prevented the drop in coverage, particularly given that both Akwa Ibom and Ekiti cited funding as the major obstacle to VAS coverage in the qualitative assessments conducted prior to Helen Keller's support.23

However, quickly entering and exiting locations seems intuitively likely to create poor funding dynamics and could limit potential long-term impact, if a sustainability plan is feasible.

Our next steps:

• Consider other factors beyond cost-effectiveness, especially when making exit decisions. The large changes in cost-effectiveness from when we made the grant to now highlights the inherent uncertainty in our cost-effectiveness estimates, and they raise questions about how much weight to place on cost-effectiveness estimates. Going forward, we plan to more deliberately consider qualitative, outside-the-model factors. This is especially true in cases when we’re deciding whether to exit geographies, given potential disruptions caused by exit.
• Work with Helen Keller to get its perspective on the most cost-effective locations before starting a program, as well as its feedback on our own location-specific cost-effectiveness estimates. We think this will help us stay aligned on priorities and hopefully avoid a situation in which Helen Keller quickly enters and exits a location based on GiveWell's estimates.
  
• Consider longer exit grants for all supported geographies to enable smoother transitions and maintain relationships, and learn more about the potential for such sustainability plans and how Helen Keller sees the longevity of its support going forward.
• Understand better how Helen Keller decides when and how to responsibly exit a location.
• Investigate potential "optimizer's curse" in state selection. By moving from national- to state-level estimates of cost-effectiveness, we risk being misled by noisy state-specific estimates of burden or costs. We can guard against this by triangulating cost-effectiveness estimates against perspectives from Helen Keller, ministries of health, and other data sources.24

Spending

Spending has been largely in line with expectations. Roughly assuming even spending over the grant period, we would have predicted that Helen Keller would have spent around $5 million from mid-2021 to mid-2023.25 In actuality, we estimate that Helen Keller spent $5.5 million over this period.26

Our next steps: We will receive regular updates about Helen Keller's spending through the duration of the grant period.

Helen Keller's activities and stakeholder input

We haven’t investigated in detail the activities Helen Keller conducted in Nigeria due to this grant in four of the five states. For example, what were the most important challenges to increasing VAS coverage and how did Helen Keller help alleviate them? What did local stakeholders think of Helen Keller's program? We think these questions could help us understand more about the grant's impact beyond cost-effectiveness updates.

In October 2024, we spoke with a government official in Taraba state, who confirmed our understanding of the broad outline of the program.27 We also talked to officials from other Nigerian states (not supported by this grant), which helped us better understand how vitamin A supplements are given out across Nigeria.28

Broadly, we heard a consistent story that most children receive VAS during Maternal, Newborn, and Child Health Weeks (MNCHWs), and that support from partners like Helen Keller alleviates barriers to achieving high coverage. Helen Keller's role includes planning for VAS delivery at MNCHWs, providing training to health workers who deliver VAS, as well as monitoring the program. This aligns with our understanding of the support that Helen Keller typically provides.

Our next steps: In future VAS grants, we plan to prioritize conversations with local stakeholders to understand barriers to coverage and how Helen Keller can help alleviate them.

How cost-effective do we think this grant was?

In April 2021, we estimated Helen Keller’s VAS program as ~24 times as cost-effective as unconditional cash transfers (GiveWell's benchmark for comparing different programs), or "24x cash" in Nigeria.29 We did not make state-specific cost-effectiveness estimates in Nigeria at that time.

In 2024, we estimate the cost-effectiveness to be ~3x to 11x cash30 across the five states supported by this grant. Three states, Akwa Ibom, Ebonyi, and Ekiti, would be below our 8x funding bar.

The two biggest drivers of the decrease in cost-effectiveness between 2021 and 2024 were:

• Our estimate of vitamin A deficiency (VAD) prevalence rates and, less significantly, the burden of measles and diarrhea (responsible for a ~60% decrease in cost-effectiveness). We discuss how we updated our VAD prevalence estimate and our ongoing uncertainty about it below.
• Our estimate of the percentage of children who would receive VAS in the absence of Helen Keller's program (responsible for a ~40% decrease in cost-effectiveness). We discuss how we updated this estimate below.

	National	Adamawa	Akwa Ibom	Ebonyi	Ekiti	Taraba
Cost-effectiveness estimated at the time of the grant (2021)31	~24x	-	-	-	-	-
2024 cost-effectiveness estimate32	~9x	~8x	~3x	~3x	~2x	~11x

Using these cost-effectiveness estimates, this grant had a smaller impact than we expected over the first two years (averting ~455 deaths,33 rather than the ~2,000 deaths we anticipated34 ).

Below, we explain what changes we made to our estimates and what stayed the same.

Note: The analysis in this section was conducted using the most current version of our VAS cost-effectiveness analysis (CEA) at the time of publication. This CEA was completed in December 2024. See a copy of our CEA displaying only relevant columns for this lookback here and the full CEA here.

Vitamin A deficiency prevalence

Vitamin A deficiency (VAD) prevalence informs our “external validity” (EV) adjustment for extrapolating the effects of VAS on mortality from trials to settings we fund today.35 Using our current EV adjustment, which incorporates new data, as well as data from multiple sources, our estimated cost-effectiveness is ~60% lower than it would be if we used our EV adjustment from when we made the grant.36 The biggest driver of this change was our updated VAD estimate for Nigeria. Details on how we updated this estimate below.

In 2021, we estimated VAD prevalence in populations today based on 2017 GBD data. We estimated 25% national VAD prevalence in Nigeria. In 2024, we estimated VAD prevalence in Nigeria based on a newly published 2021 national survey of VAD in Nigeria. We adjusted the survey findings using proxies to come up with state-specific VAD estimates. We now estimate 14% national VAD prevalence and 10 to 18% in the states supported by this grant. Although we have significant uncertainty about the 2021 survey results because two measures of VAD point in very different directions, we think this is a stronger data source than GBD data. We understand GBD data does not incorporate the 2021 survey. The 2021 survey is valuable because it is very recent and nationally representative, and because it includes multiple measures of VAD. We think this inclusion of multiple measures is informative, even if we're uncertain how to interpret their divergence.

Here's how our approach to estimating VAD in Nigeria has changed over time:

• VAD is also reported in GBD 2019 and 2021, though these estimates are surprisingly low, and we have low confidence in them.37 Because of this, we did not incorporate GBD 2019 or GBD 2021 data on VAD prevalence in Nigeria into our models.
• In 2023, we updated our methodology to incorporate VAD prevalence estimates from national and regional surveys (see here in our changelog). Because the most recently available national survey from Nigeria was from 2001,38 we adjusted the estimate using proxies to account for improving circumstances over time and differences between Nigerian states.39

In 2024, we reviewed a newly published 2021 survey of VAD in Nigeria.40 Based on our analysis of the survey's findings, we now use a national VAD estimate of 13.5% for Nigeria in our model (which we adjust based on proxies by state).41 We no longer put weight on GBD estimates of VAD in Nigeria.

Source	VAD prevalence (Nigeria average) estimate, following GiveWell analysis	Cost-effectiveness estimate, (2024 model with different VAD prevalence inputs) (see calculations here)
GBD 2017 [2021 best guess]	25%	16.3x
2001 national survey (adjusted using proxies to account for changes over time)	23%	14.8x
GiveWell's analysis of a 2021 Nigeria survey [current best guess]	14%	9.2x

While we think the 2021 survey provides the most up-to-date estimates on VAD in Nigeria, there is a significant discrepancy between two measures of VAD included in the survey (serum retinol and modified relative dose-response test) that we don't fully understand. See this document for our current thinking on how we've derived an estimate for our model.

Our cost-effectiveness estimates for the Nigerian states funded by this grant are fairly sensitive to this input. Our current cost-effectiveness estimates are ~30% to 60% lower, varying by state, than they would be if we used our 2021 VAD estimates (from GBD 2017) in our current model.42 At the national level, our cost-effectiveness estimates would range from 16x to 9x depending on which of these data sources we used (GBD 2017, our adjusted 2001 survey estimate, or our analysis of the 2021 survey. See table above).

Our next steps: VAD prevalence remains a major uncertainty in our VAS CEA. We may fund new VAD surveys in the future, although we are not expecting to support a survey in Nigeria, since one was conducted relatively recently in 2021. We may also explore additional ways to triangulate our VAD estimates in Nigeria and elsewhere.

VAS coverage without Helen Keller's support

In our model, we use our best guess of the proportion of children who would receive VAS without Helen Keller's support to estimate how many additional children receive VAS as a result of the program.

In 2021, we assumed that 5% of children received VAS outside of campaigns supported by Helen Keller.43

As of December 2024, we think ~45% of children in the Nigerian states funded by this grant would receive VAS, either from routine coverage or through campaigns that would occur in the absence of Helen Keller's support.44 In Nigeria, this includes both facility-based routine delivery and VAS delivered through Maternal, Newborn, and Child Health Weeks (MNCHWs), which are institutionalized campaigns that would likely continue without NGO support.45

This is based on new data on coverage in Nigerian states as well as earlier data that we reviewed. We don't have data specific to the states supported in this grant, so we use an average of those we do have for those. Our 2021 estimate was a rough guess that was not specific to Nigeria (we used a 5% guess in all locations supported by Helen Keller). Details on our 2024 estimates are available here.

Our model is fairly sensitive to this parameter. Our current cost-effectiveness estimates are ~45% lower than they would be if we used our 2021 estimate of baseline coverage, holding other parameters constant.46

This change was partially counterbalanced by assuming a lower chance that the government or other philanthropic actors would support the program in our absence. (more)

Our next steps:

• We plan to look for more opportunities to fund surveys of VAS coverage prior to campaigns we decide to fund in the future. We think we missed an opportunity to do this in at least three states (Adamawa, Akwa Ibom, and Taraba) that did not have partner support prior to Helen Keller.
• We’ve been looking for examples where we’ve underestimated counterfactual coverage in areas beyond VAS, too. We discuss plans for this in our report on red teaming our top charities here.

Funging

Without this grant, we think it’s unlikely Helen Keller’s support would have been provided in these states. That’s based on (a) continued funding gaps in Akwa Ibom and Ekiti, the two states we exited and (b) Helen Keller's best guess that Ebonyi would not have been funded without our support.47

Our next steps: We plan to check whether the funding gaps for the two states in Nigeria we’ve exited get filled.

Additional updates to our model between 2021 and 2024

We made many other updates to our cost-effectiveness model between 2021 and 2024. The following were important, but had a smaller impact on the overall decrease in cost-effectiveness of this grant compared to the parameters discussed above:

• Estimating the share of mortalities caused by measles and diarrhea48
• Publication bias adjustment49
• State-specific cost estimates50
• Updated baseline mortality51
• Lower funging likelihood (our best guess that others would fund the grant if we didn't)52

See footnote for information about additional updates to our CEA.53

Cost-effectiveness in states we exited

When GiveWell developed our state-specific cost-effectiveness estimates for Nigeria in 2022, we estimated Akwa Ibom and Ekiti to be below our funding bar (~5x and ~6x, respectively),54 compared to ~24x (across Nigeria) when we initially made the grants.55 Helen Keller exited Akwa Ibom and Ekiti as a result of our lower estimated cost-effectiveness.56

In our 2022 CEA, the key drivers of lower cost-effectiveness in these states, compared to the Nigeria average, were lower baseline all-cause mortality, lower shares of mortality due to measles and diarrhea, and lower rates of VAD in Akwa Ibom and Ekiti.57 We have limited confidence in these inputs into our model because they rely on state-specific GBD estimates (GBD 2019 for mortality and disease burden data and GBD 2017 for VAD). We're more uncertain about GBD's state-specific estimates because we assume they have more limited data and may be making more extrapolations.

We now rely on multiple sources of mortality and disease burden data in our estimates of Nigerian states' cost-effectiveness. Akwa Ibom and Ekiti continue to fall below our funding bar at ~3x, respectively, which gives us some additional confidence that the decision to exit was not solely driven by noisy burden estimates.

Major uncertainties we haven't learned more about

• Funding by others. We remain uncertain whether we have a complete picture of other actors' spending on VAS in these Nigerian states. If we're not incorporating it, we may be underestimating the cost per supplement delivered (by excluding costs from others) or underestimating the likelihood others would fund these programs without us. We plan to learn more about this when we consider future support for VAS in Nigeria.
• Effect of VAS on mortality. We have not received new information about the effect of VAS on mortality, though we make a new publication bias adjustment mentioned above. This continues to be one of our main uncertainties about the impact of VAS programs.58 We may support additional research on this in the future.

Did we set ourselves up to learn?

We think we missed some opportunities to learn through this grant:

• Opportunity to run a baseline survey and learn about increases in VAS coverage due to NGO support. At least three states (Adamawa, Akwa Ibom, and Taraba) did not have partner support prior to Helen Keller, according to the qualitative assessments Helen Keller conducted.59 This could have been an opportunity to run a baseline survey to understand how NGO support impacts VAS coverage. (See above.)
• Coverage monitoring. We did not set ourselves up particularly well for learning. Since recommending the grant, we've conducted a deeper investigation into Helen Keller's monitoring and identified important concerns; we're now working with Helen Keller to implement improvements. (See above.)
• Forecasts. We did not register forecasts with the grant. (See below.)

How calibrated were our forecasts?

We didn't register forecasts for this grant in 2021.

Our next steps: We now include forecasts on all our grant pages, and plan to continue this for subsequent grants going forward.

Sources

Document	Source
2021 GiveWell cost-effectiveness analysis — version 2	Source
2021 Nigeria National Food Consumption and Micronutrient Survey	Source
2022 GiveWell cost-effectiveness analysis — version 4 [change 3 archive]	Source
2022 GiveWell cost-effectiveness analysis — version 5	Source
Counterfactual VAS coverage estimates for codelivery grant, 2024	Source
Email from Helen Keller Intl, May 27, 2022	Unpublished
GBD 2019 Nigeria VAD estimates	Source
GiveWell, 2022 Cost-Effectiveness Analysis Changelog	Source
GiveWell, 2023 Cost-Effectiveness Analysis Changelog	Source
GiveWell, Allocation of Funds Donated to the Top Charities Fund in Q1 2021	Source
GiveWell, Analysis of all-cause mortality rate, December 2024	Source
GiveWell, CEA of vitamin A supplementation, 2024 v2	Source
GiveWell, Helen Keller International — vitamin A supplementation (December 2024) (page forthcoming as of publication)	Source
GiveWell, Helen Keller International cost per supplement [2020]	Source
GiveWell, Helen Keller International cost per supplement, 2022	Source
GiveWell, Helen Keller International cost per supplement, 2024 v2	Source
GiveWell, Helen Keller Intl lookback CEA comparison, 2024	Source
GiveWell, Helen Keller Intl’s Vitamin A Supplementation Program	Source
GiveWell, Helen Keller Nigeria supplemental calculations	Source
GiveWell, HKI VAS - information for country-level cost-effectiveness analyses [2021 - GBD 2019 mortality edits]	Source
GiveWell, Update to Nigeria VAD rate estimate, October 2024	Source
GiveWell, Vitamin A Supplementation	Source
GiveWell's CEA of vitamin A supplementation, 2024 v0	Source
GiveWell's Conversation with Helen Keller Intl, August 28, 2024	Unpublished
GiveWell's conversation with Justina Bitrus, State Nutrition Officer, Taraba State, Nigeria; Mamman Bello Kakulu, Director, Primary health care department, Primary Health Care Development Agency, Taraba State, Nigeria; and Henry Masari, Assistant State Nutrition Officer, Taraba State, Nigeria, October 11, 2024 (unpublished)	Unpublished
GiveWell's conversation with Mr. Bello A. Aliyu, State Nutrition Officer, Kebbi State Primary Health Care Development Agency, Kebbi State, Nigeria	Unpublished
GiveWell's conversation with Pharm Jiya Daniel Tsado (B.Pharm, MPH), Director Food and Drugs, Ministry of Primary Health Care, Niger State, Nigeria, November 18, 2024 (unpublished)	Unpublished
GiveWell's conversation with Ramatu Musa Haruna, State Nutrition Officer, Kaduna State, Nigeria, October 8, 2024 (unpublished)	Unpublished
Helen Keller Intl, 2022 room for more funding report	Source
Helen Keller Intl, Forecast for VAS campaigns in Nigeria, 2023	Source
Helen Keller Intl, Post-event coverage survey: Guinea, Round 1 2024	Source
Helen Keller Intl, Post-event coverage survey: Mali, Round 1 2024	Source
Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment In Adamawa, Akwa Ibom, Katsina and Taraba States, Nigeria, September 2021	Source
Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Ekiti State, Nigeria, January 2022	Source
Maziya-Dixon et al. 2006	Source

• 1

  See our VAS intervention report and Helen Keller charity page for more information.

• 2

  See GiveWell, Allocation of Funds Donated to the Top Charities Fund in Q1 2021

• 3

  "We intend for Helen Keller Intl to use this grant flexibly to support VAS in whichever states it determines have the greatest need. At the time we made this grant, Helen Keller Intl had identified four Nigerian states that may need support for VAS campaigns4 and was planning to conduct state-level evaluations to confirm whether its support is indeed needed." GiveWell, Allocation of Funds Donated to the Top Charities Fund in Q1 2021

• 4
  • "Heller Keller resumed its support to the Ministry of Health of Nigeria in 2020, first supporting Nasarawa State. It added Benue State in May 2021; the states of Adamawa, Akwa Ibom, Taraba in November 202[1]; and Ekiti State in May 2022. Delivery of VAS is organized through health facilities and outreach distribution points. It requires social mobilization to motivate caregivers to attend the distribution.

  "Helen Keller supported the 6 states mentioned above for the two rounds of 2022, but also added the state of Ebonyi for the second round of 2022." Helen Keller Intl, 2022 room for more funding report, p. 17

• 5

  Conversation with Helen Keller International, August 28, 2024 (unpublished)

• 6

  "Helen Keller will, however, cease to support the two states of Akwa Ibom and Ekiti from 2023, as they do not meet the minimum threshold for cost-effectiveness.
  "Over the next three years, Helen Keller will continue to support 5 states--Nasarawa, Benue, Adamawa, Ebonyi and Taraba." Helen Keller Intl, 2022 room for more funding report, p. 17

• 7In late 2024, we decided to exit our support for VAS in Ebonyi state based on low estimated cost-effectiveness (3x). At the time of writing, our grant page for our December 2024 grant is not yet public.
• 8
  • We directed a $7.5m, 3-year grant to Helen Keller for expansion to additional Nigerian states. We roughly assume even spending over three years, i.e., $5m in spending by the end of year 2. Because in reality the program was implemented in different states at different times, this is a rough assumption.
  • We estimate the expected number of supplements delivered at the time we recommended the grant based on our 2021 estimate of $1.10 per supplement delivered and assumption that Helen Keller covers 41% of program costs: ($5m/41%)/$1.10 = ~11m supplements delivered
  • Children receive VAS twice per year under Helen Keller's programs. 11m supplements/2 years/2 rounds per year = ~2.75m children reached.

• 9

  This is a rough estimate based on Helen Keller's reported target populations and coverage survey data. For campaigns for which we have not seen coverage estimates, we use the median coverage rate from Helen Keller's coverage surveys of GiveWell-supported campaigns in Nigeria as a rough assumption.

  • For supplements delivered in the second half of 2021, see Column J here, excluding Benue and Nasarawa states, which were not part of the expansion supported by this grant = ~1.8m
  • For supplements delivered from 2022 to mid-2023, see Column J here, excluding Benue, Nasarawa, and FCT = ~9.2m
  • 1.8m + 9.2m = 11m supplements delivered from mid-2021 to mid-2023
  • ~11 million supplements/2 years/2 rounds per year = ~2.75 million children reached. We make a rough downward adjustment of -10% to account for the fact that some children likely only received coverage in one round, and thus were not fully covered in each year. 2.75 million x .90 = 2.5 million children reached.

• 10

  In our August 2024 analysis that includes cost data through mid-2023, we estimated that Helen Keller covered around 60% of total program costs in Nigerian states, compared with 41% in our 2021 model. Our August 2024 model, which also incorporates an adjustment for forward-looking budgets, estimated an average cost per supplement of $0.84 across Nigerian states, compared to $1.10 in our 2021 model.

  In December 2024, we further updated our cost per supplement model to include data through mid-2024. We now estimate that Helen Keller covers 70% of costs in Nigerian states, and an average cost per supplement of $0.70 in Nigeria.

• 11

  January-June 2021 coverage estimates are from Helen Keller's qualitative assessment, based on administrative data from governments. We think that administrative data generally over-reports coverage (compare administrative coverage to coverage survey data in this sheet), though we have no specific concerns about this 2021 data for Nigeria. See: Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Adamawa, Akwa Ibom, Katsina and Taraba States, Nigeria, September 2021, Section 3.1, p. 2

• 12

  See this section of our review of Helen Keller for more detail on this concern.

• 13

  This is based on internal, unpublished analyses.

• 14
  • "As part of the coverage survey, a team of independent monitors, led by a different consultant from the one conducting the main survey, was specially trained and deployed to ensure rigorous data quality control. This team carried out independent enumeration in 10% of clusters, i.e. 12 clusters, and surveyed households already interviewed by the main consultant's teams, checking the consistency between audio recordings and captured data to ensure accuracy. Daily analyses and reports were produced, enabling key indicators to be monitored in real time via a dashboard. Comparative results show that a higher household count was achieved in 4 clusters than in the initial coverage survey, and that in 10 clusters the figures were higher than INS projections, demonstrating the rigor and accuracy of the fieldwork. In sum, the coverage survey (PECS) counted 2,586 households, while the independent monitors (IM) counted 2,161, against the 1,812 initially forecast, suggesting an initial underestimate and the need to adjust resources for future interventions." Helen Keller Intl, Post-event coverage survey: Guinea, Round 1 2024, p. 29
  • "As part of the coverage survey, a team of independent monitors from another consultant different from the one conducting the coverage survey was carefully trained and deployed to carry out rigorous data quality control. This team carried out independent enumeration of 10% of the clusters (i.e. 12 clusters) and conducted household surveys, checking the consistency of the audio recordings with the data entered to ensure the accuracy of the information. Daily analyses and reports were produced, enabling real-time monitoring via a dashboard of key indicators.

    "The results presented in the table reveal disparities between expected households, those counted by PECS and those counted by independent monitoring (IM). In fact, the total number of households counted by IM (1,165) exceeds those counted by PECS (1,048), well above the expected number of 1,018, particularly in clusters 52, 53 and 97, where the number of households counted far exceeds forecasts.

    "In contrast, several clusters, such as Cluster 44, show an underestimate in MI enumeration compared to PECS, suggesting inconsistencies in data collection or household identification across the two methodologies. These results underline the importance of cross-validating data to identify potential sources of enumeration error and improve the accuracy of estimates. Analysis of these discrepancies is crucial for refining data collection protocols, optimizing the methods used and guaranteeing the reliability of the information gathered for future analysis." Helen Keller Intl, Post-event coverage survey: Mali, Round 1 2024, p. 31

• 15

  Helen Keller stopped its support after providing three semesters of funding in Akwa Ibom and two semesters of funding in Ekiti.

  • Helen Keller supported VAS in Akwa Ibom in the second semester of 2021 (see here) and in both Akwa Ibom and Ekiti in both semesters of 2022 (see here). Helen Keller confirmed with us in May 2022 that phase out from both states would occur after the second semester of 2022. Source: email from Helen Keller Intl, May 27, 2022 (unpublished)
  • "Helen Keller will, however, cease to support the two states of Akwa Ibom and Ekiti from 2023, as they do not meet the minimum threshold for cost-effectiveness.

    "Over the next three years, Helen Keller will continue to support 5 states--Nasarawa, Benue, Adamawa, Ebonyi and Taraba." Helen Keller Intl, 2022 room for more funding report, p. 17

• 16

  Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 17

  Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 18

  "From Helen Keller's perspective, we are there at the grace of host country governments. In almost all countries where we operate, the approach has been relatively long term, which allows us to work as long as we have. Relationships are critical in all of these. Pulling out is always a challenge. Governments depend on you, count on you. Even if you really advocate for their co-investment - and they are co-investing, in time, personnel, moving of supplies, there is co-investment going on. It is a process. Even if they say it's a 2-3 year process…They have to take deliberate action when this happens. Two year runway is minimum. Even then they can anticipate shortfalls and consequences on their side, in terms of performance and shortfalls of coverage and so forth, and try to do their best to minimize that." Source: GiveWell's non-verbatim summary of a conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 19

  Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 20

  According to Helen Keller, administrative data showed that VAS coverage declined from over 80% with Helen Keller support to around 60% after Helen Keller's exit. Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 21

  Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 22

  Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 23
  • Reasons why MNCHW (maternal, newborn, and child health week) was not conducted in the health facilities:
    • Awka Ibom: "Reasons why MNCHW was not conducted in health facilities, 2020: COVID-19 / Inadequate supply of VAS and failure of govt. to show interest (100% of respondents), funding (53.3% of respondents) Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Adamawa, Akwa Ibom, Katsina and Taraba States, Nigeria, September 2021, Section 3.3, p. 3
    • Ekiti: "As for reasons why MNCHW was not conducted, 93.3% reported that it was due to lack of funding." Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Ekiti State, Nigeria, January 2022, p. 3
  • What are the biggest challenges that hamper high VAS coverage among 6–59-month-old children during the MNCHW? Respondents could select multiple reasons. By state, the biggest factors (>50%) were:
    • Akwa Ibom: funding (96.7%), inadequate supply of VAS / PPEs and failure of govt. to show interest / Insecurity (86.7%), poor logistics (76.7%), poor advocacy (70%), poor social mobilization (70%), lack of capacity building (66.7%), poor planning (50%) Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Adamawa, Akwa Ibom, Katsina and Taraba States, Nigeria, September 2021, section 5.1, pp. 4-5
    • Ekiti: funding (96.7%), poor social mobilization (53.3%) Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Ekiti State, Nigeria, January 2022, p. 2

• 24

  This could include data on poverty levels, since would broadly expect VAD to be higher in more impoverished areas.

• 25

  We recommended a $7.5m, 3-year grant, so on average we expected Helen Keller to spend $5m by the end of year two. This is a rough assumption because in reality the program began in different states at different times. We did not make a specific prediction on Helen Keller's spending timelines over the grant period when we made the grant. This grant supports Helen Keller from mid-2021 through the end of 2024. (See our grant write-up from 2021 here).

• 26

  We estimate that Helen Keller actually spent around $5.5m from mid-2021 to mid-2023 (see calculations here).
  In the states supported by this grant, Helen Keller only reported a small funding gap for Taraba during this grant period ($29k for Taraba in FY25, which spans mid-2024 through mid-2025), suggesting that it expects to largely spend in alignment with the grant budget by the end of the grant period. See list of funding gaps for these states here (See "Funding Gaps, FY25" column, "Total forecast" rows for Adamawa, Ebonyi, and Taraba).

• 27

  GiveWell's conversation with Justina Bitrus, State Nutrition Officer, Taraba State, Nigeria; Mamman Bello Kakulu, Director, Primary health care department, Primary Health Care Development Agency, Taraba State, Nigeria; and Henry Masari, Assistant State Nutrition Officer, Taraba State, Nigeria, October 11, 2024 (unpublished)

• 28
  • GiveWell's conversation with Ramatu Musa Haruna, State Nutrition Officer, Kaduna State, Nigeria, October 8, 2024 (unpublished)
  • GiveWell's conversation with Mr. Bello A. Aliyu, State Nutrition Officer, Kebbi State Primary Health Care Development Agency, Kebbi State, Nigeria, October 16, 2024 (unpublished)
  • GiveWell's conversation with Pharm Jiya Daniel Tsado (B.Pharm, MPH), Director Food and Drugs, Ministry of Primary Health Care, Niger State, November 18, 2024 (unpublished)

• 29

  See our 2021 cost-effectiveness analysis here.

• 30

  Our estimate of the value per dollar donated to cash transfers (our benchmark for comparing different programs) is out of date as of 2024. We are continuing to use this outdated estimate for now to preserve our ability to compare across programs, while we reevaluate the benchmark we want to use to measure and communicate cost-effectiveness.
  For more on our update on the impact of unconditional cash transfers, see this page.

• 31

  Source: See our 2021 CEA of Vitamin A supplementation here.

• 32

  Source: See a copy of our current (December 2024) CEA here. Note that the "National" estimate reflects our estimates for all states in Nigeria, not just the ones funded by this grant. Our current unweighted average of cost-effectiveness for only these states is 6x. See here in our calculations.

• 33

  The number of deaths averted is a rough estimate, based on Helen Keller's reported spending per state in the first two years of the program and our 2024 CEA estimate of the cost per life counterfactually saved.

  • Spending:
    • For 2021, we do not have a state-by-state breakdown of Helen Keller's spending in Nigeria; We allocate the total country budget of ~$2.1m to Nigerian states in proportion to our estimate of the number of supplements delivered.
    • We use state-by-state spending data reported by Helen Keller for 2022-mid-2023. See here in our cost per supplement analysis for state-level spending data.
  • Cost per life counterfactually saved (December 2024)
    • Adamawa: $9,648
    • Akwa Ibom: $23,377
    • Ebonyi: $21,963
    • Ekiti: $31,651
    • Taraba: $6,831
  • We divide spending per state by the cost per life counterfactually saved in each state, and sum them, to get a total number of lives counterfactually saved. See our calculations here.

• 34
  • We directed a $7.5m, 3-year grant to Helen Keller for expansion to additional Nigerian states. We roughly assume even spending over three years, which implies $5m in spending by the end of year 2. Because the program began at different times, this assumption is rough and may not exactly match reality.
  • The number of supplements delivered is based on our estimate of $1.10 per supplement delivered and an assumption that Helen Keller covers 41% of program costs: ($5m/41%)/$1.10 = ~11m supplements delivered
  • The number of deaths averted is based on our estimate of $2,478 per death averted: ($7.5m*(⅔))/$2,478 = ~2,000 deaths averted in first two years of the grant.

• 35

  See this section of our report on Vitamin A supplementation for how we calculate our external validity adjustment. As of April 2025, we are in the process of updating our VAS intervention report to reflect our latest methodology for calculating external validity. At a high level, the process of adjusting the results of VAS trials to account for differences in disease burden and VAD prevalence still applies, but the specific choices we've made about how to do that have changed since our last intervention report was published in early 2024.

• 36

  See here in our calculations.

• 37

  "Reasons for our low confidence in the Global Burden of Disease Project state-level estimates of VAD in Nigeria:

  • "We investigated the state-level VAD estimates for Nigeria and found them to be implausibly low (2% to 4% in many states, see this spreadsheet).
  • "These low rates are in tension with what we would expect, based on other sources of information:
    • "They are considerably lower than VAD rates in other sub-Saharan African countries where Helen Keller and Nutrition Intl works (in the ~25% to 35% range in GBD 2017 - see this row of our cost-effectiveness analysis).
    • "They are considerably lower than the level recorded in the most recent nationally representative survey in Nigeria that we have found (30%). The survey was conducted in 2001, and while we might expect some fall in VAD over time, we think this fall is too dramatic to be plausible.
  • "We’re unsure what modeling changes were driving this update between GBD 2017 and GBD 2019." Source: this footnote in our intervention report on VAS.

• 38

  Maziya-Dixon et al. 2006 reports a nationwide VAD prevalence of 29.5% (Table 4, p. 2259). "Data collection took place from August to October 2001." p. 2257

• 39

  See here in the April 2024 version of our CEA for this method of estimating VAD in individual states, and see here for the national estimate using this methodology. See here in the April 2024 version of our CEA.

• 40

  See the 2021 Nigeria National Food Consumption and Micronutrient Survey. Read our October 2024 updated analysis of the survey's results here.

• 41

  See this row in our CEA for national VAD prevalence according to the latest survey. For our state-level adjustments, we use proxy values for VAD prevalence. See this section of the "External validity" tab of our CEA. See here for our final state estimates of VAD.

• 42

  See our calculations and sensitivity analysis for sources of VAD prevalence here.

• 43

  See the "double treatment" adjustment in our 2021 CEA here. This was a rough guess we made at the time.

• 44

  See this estimate in our current CEA here. We don't have state-specific counterfactual coverage data for these states, so we use an average of estimates from states where we do have data. See those estimates here.

• 45

  See the cell note here in our CEA.

• 46

  See our calculations and sensitivity analysis for baseline coverage estimates here.

• 47
  • (a) Akwa Ibom and Ekiti. Helen Keller reports that Akwa Ibom and Ekiti have not had any new partners enter since Helen Keller's exit. Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)
  • (b) Ebonyi. Due to the flexibility of our approach in allowing Helen Keller to select states, only one out of the five—Ebonyi—was one for which we had expressed a preference (via our high CE estimate). This makes it a good candidate for checking whether our preference influenced other donors who might have otherwise funded the state. According to Helen Keller, no donor has shown interest in Ebonyi and they think it is very likely that Ebonyi would remain unfunded without GiveWell's support (apart from UNICEF's supply of VAS capsules). Source: Conversation with Helen Keller Intl, August 28, 2024 (unpublished)

• 48

  Our external validity adjustment accounts for differences in contexts between the trials we rely on for the effect size of VAS on mortality and the locations where VAS is implemented today. We compare both VAD prevalence (see above) and the share of mortalities caused by measles and diarrheal disease. For more detail, see this section of our intervention report. (Note: As of April 2025, we are in the process of updating our VAS intervention report to reflect our latest methodology for calculating external validity. At a high level, the process of adjusting the results of VAS trials to account for differences in disease burden and VAD prevalence still applies, but the specific choices we've made about how to do that have changed since our last intervention report was published in early 2024.)
  We've updated our estimates for the composition of mortality to include new data and multiple data sources. This had a less significant impact on overall cost-effectiveness than did our updates to VAD prevalence estimates (-9% nationally compared to -44% for VAD prevalence). See this section of our calculations.

• 49

  In 2024, we updated our estimate of the effect VAS has on mortality from a 24% reduction to a 19% reduction. Our previous estimate came directly from the Imdad et al. 2022 meta-analysis of VAS trials. Due to concerns about potential publication bias, we conducted our own meta-analysis of only trials that used all-cause mortality as a primary outcome, which yielded a lower estimate. See our current CEA here, our meta-analysis here, and our 2021 CEA here. Our current cost-effectiveness estimates are ~20% lower than if we used our previous effect size estimate (see this section of our calculations).

• 50

  We estimate a lower cost per supplement in Nigeria today than we did in 2021, which has increased our cost-effectiveness estimates in all of these states. See our cost-effectiveness comparison here.

  • In 2021, we estimated a Nigeria cost per Vitamin A supplement of $1.10, based on a weighted average of cost estimates from other countries (see our 2020 analysis here).
  • In 2024, we now use state-specific data from Helen Keller on budgets, target populations, and coverage to estimate cost per supplement. In states we currently support, we incorporate both past and forward-looking data. See our cost per supplement analysis here and our CEA here.

• 51

  Our current estimates of baseline mortality in the absence of VAS in these Nigerian states range from 0.64% in Ebonyi to 1.41% in Taraba (see here), compared to 1.21% in our 2021 CEA. This corresponds to a -50% decrease in cost-effectiveness in Ebonyi and a 16% increase in cost-effectiveness in Taraba. See comparison here.

  • In 2021, we estimated a baseline mortality rate among children 6-59 months of 1.21% in Nigeria (converted from 12.1 baseline deaths per 1,000 child-years), based on data from GBD 2019. See our 2021 CEA here. This estimate relied on a weighted average of state-specific data from the states we thought Helen Keller would work in at the time we developed the CEA, which ended up being slightly different. See calculations here.
    • In 2024, we now use an estimate of baseline mortality adjusted for different data sources, similar to our adjustment for the share of measles and diarrhea mortalities in different data sources discussed in the footnote above. See this adjustment in our CEA here and our analysis here. We put 40% weight on data from GBD 2021, 20% weight on IGME 2021, and 40% weight on data from recent national surveys, if available, though we may adjust this weighting in the future.

• 52

  We estimate a lower risk of funging today in Nigeria than we did when we recommended the 2021 grant, and our current cost-effectiveness estimates are ~10% to 25% higher than they would be if we used our 2021 funging estimates in our current CEA. See calculations here.

  • In 2021, we estimated a 40% funging risk in Nigeria, split between a 20% chance that domestic governments would replace Helen Keller's costs, and a 20% chance that other philanthropic actors would replace Helen Keller's costs. See here in our 2021 CEA.
  • In 2024, we now estimate a 20% to 25% funging risk in Nigeria, split between a 5% chance that domestic governments would replace costs, and a 15%-20% chance that other philanthropic actors would replace costs. See here in our current CEA.
  • Our current values are rough guesses, based on understanding of the VAS landscape in Nigeria based on conversations with Helen Keller. We learned in 2023 that some Nigerian states received World Bank funding for a program called ANRIN (Accelerating Nutrition Results in Nigeria), we estimate a slightly higher funging risk in those states because they may still be receiving residual funding. See cell notes here.

• 53

  There have been numerous smaller updates to our CEA since 2021. See this tab of our calculations for a comparison of all public CEA versions from 2021 to December 2024 and a list of the most impactful updates in each model. See our intervention report for a full explanation of our CEA. Note: As of April 2025, we are in the process of updating our VAS intervention report to reflect recent updates to our model. The currently public version of the intervention report was published in April 2024.

• 54

  See our state-specific estimates from 2022 here.

• 55

  See our 2021 CEA here.

• 56

  "Helen Keller will, however, cease to support the two states of Akwa Ibom and Ekiti from 2023, as they do not meet the minimum threshold for cost-effectiveness." Helen Keller Intl, 2022 room for more funding report, p. 17

• 57

  We developed state-specific estimates and changed our IV and EV adjustments in 2022 version 5 of our model. Prior to that, our Nigeria-national estimate was fairly stable following the recommendation of this grant. See here for our 2021-2022 national estimates, and here for our first set of state-specific estimates from 2022.
  The state-specific inputs we relied on to develop state-specific estimates included:

  • Baseline mortality rates (lower in Akwa Ibom and Ekiti than in Nigeria as a whole in our previous model. GBD 2019 was the source for both).
  • We made sharper internal validity (IV) and external validity (EV) adjustments in Akwa Ibom and Ekiti (74% IV and ~17% EV) than in Nigeria as a whole in our previous model (85% IV and 44% EV). See the CEA changelog here for details.
    • Using the 2022 state-specific model but reverting to our previous EV and IV adjustments, Akwa Ibom and Ekiti are well above our bar at 14.3x and 16.3x.
    • Using the 2022 state-specific model but reverting to our previous IV adjustment (and keeping the updated EV adjustment), Akwa Ibom and Ekiti were below our bar at 5.3x and 6.7x. This suggests our updated EV adjustment led to a significant decrease in cost-effectiveness.
    • Using the 2022 state-specific model but reverting to our previous EV adjustment (and keeping the updated IV adjustment), Akwa Ibom and Ekiti were 12.6x and 14.4x. This suggests that updating our IV adjustment did not have a major effect on cost-effectiveness on its own.
    • The updates to our IV and EV adjustments in these state-specific estimates were modeling choices rather than "real-world" updates.
      • Our IV adjustment changed based on a plausibility analysis of the meta-analysis we use for the effect size of VAS.
      • Our EV adjustment update consisted of reducing the weight placed on the proportion of mortalities caused by measles or diarrhea (from 85% to 80%), and adding an adjustment for the non-independence of VAD prevalence and infectious disease burden.
      • See here in our 2022 CEA changelog for a detailed description of the updates to our IV and EV adjustments.

  • 58

    Some of our key uncertainties are:

    • Whether VAS is still effective today, since most of the main studies we use in our analysis were conducted three to four decades ago.
    • What explains large differences in outcomes between VAS trials.

    See here in our intervention report for more on our main uncertainties.

  • 59
    • "Adamawa, Akwa Ibom and Taraba do not have partner support for the MNCHW." Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Adamawa, Akwa Ibom, Katsina and Taraba States, Nigeria, September 2021, p. 7
    • Ekiti: "There was no partner support for MNCHW in 2021. Although Saving One Million Lives SOML supported micronutrient deficiency campaign (MNDC), this was not efficiently conducted and had a low VAS coverage (see below for more details). Further probe to SOML activities revealed that the 2021 MNDC support was the last one for the state as the program has ended" Helen Keller Intl, Rapid State and Health Facilities VAS Qualitative Assessment in Ekiti State, Nigeria, January 2022, p. 2
    • We have not reviewed the qualitative assessment for Ebonyi.