Note: This page summarizes the rationale behind a GiveWell-recommended grant to Nutrition International. Nutrition International staff reviewed this page prior to publication.
In January 2022, GiveWell recommended that Open Philanthropy grant $8.95 million to Nutrition International. Nutrition International expects to use this funding to support vitamin A supplementation (VAS) campaigns in Chad. Vitamin A supplementation involves treating all children aged approximately 6 months to 5 years in areas at high risk for vitamin A deficiency with high-dose vitamin A supplements, usually twice per year. This is the first grant that we have recommended to Nutrition International, but we have recommended grants to support VAS previously through Helen Keller International’s VAS program.
We recommended this grant because we believe that the work that the grant will support will be cost-effective. VAS is among the most cost-effective programs we know of when conducted in areas with high child mortality rates, and child mortality rates are high in Chad.
Published: April 2022
Table of Contents
- The program
- The grantee
- Planned activities and budget
- The case for the grant
- Risks and reservations
- Plans for follow up
- Internal forecasts
- Our process
Vitamin A deficiency is a common condition in low- and middle-income countries that can cause stunting, anemia, dry eyes (the leading cause of preventable childhood blindness), susceptibility to infection, and death.1 Vitamin A deficiency is most common in Africa and South-East Asia.2 Infants, children, and pregnant or lactating mothers with low vitamin A intake are at particularly high risk of the negative health impacts of VAD.3
To prevent childhood morbidity and mortality, the World Health Organization recommends vitamin A supplementation (VAS) every four to six months for all children aged 6 to 59 months in areas where VAD is a public health problem.4 The version of the intervention we focus on here is the periodic administration of prophylactic VAS to preschool children aged 6 to 59 months who are at risk of vitamin A deficiency. See more details about the program in our intervention report on vitamin A supplementation.
GiveWell has a history of funding VAS through Helen Keller International since 2017,5 and this is the first time we are funding VAS through Nutrition International.
Nutrition International is an international non-governmental organization headquartered in Canada6 that provides financial and technical assistance to governments in delivering nutrition-related interventions.7 Nutrition International has worked in VAS since 1994.8 Nutrition International has supported VAS programs both by donating vitamin A capsules and by providing support to governments to carry out VAS distribution.9
Planned activities and budget
This grant of $8.95 million will enable Nutrition International to support implementation of biannual VAS campaigns in Chad to children aged 6 to 59 months over three years, from 2022 through 2024.10 We expect the structure of Nutrition International’s support for VAS programming in Chad to be similar to that described here in our review of Helen Keller International’s VAS program, with a difference being that Helen Keller International typically does not work in a country until it has established a country office and built a team of staff in a country to provide this support, while Nutrition International has some experience with scaling up support in a country through working with consultants before establishing a country office or hiring staff.11
The budget includes:12
- $6.57 million for program activities. Program activities include campaign activities, vitamin A capsule costs,13 and post-event coverage (PEC) surveys to be conducted after the campaigns.
- $1.2 million for program management. Program management costs include support for country staff, global staff, and travel.
- $1.2 million for indirect costs.
Funding Nutrition International’s VAS campaigns for three years is consistent with the three-year funding runway we typically provide to grantees, which enables them to use funding budgeted for future years to pay for any cost increases in an earlier year.
The case for the grant
- Cost-effectiveness. We estimate that this grant exceeds our bar for cost-effectiveness. We relied on the cost-effectiveness model that we had built for Helen Keller International’s VAS program and updated it for Nutrition International’s program. More below.
- Funding landscape for VAS campaigns. On a global level, there is a need for more funding for VAS, especially since opportunities to co-deliver VAS with other interventions have been decreasing as there are fewer polio vaccination campaigns. In Chad, we think there is a funding gap because Nutrition International only has funding secured through 2021 and maybe to partially cover the first 2022 campaign in Chad. Nutrition International’s existing grant is ending in 2022, and it seems unlikely that other funders would replace this grant. Making a grant to support VAS in Chad remains cost-effective after accounting for the potential to crowd out other funders. More below.
A note on how we use cost-effectiveness estimates in our grantmaking
After assessing a potential grantee's room for more funding, we may then choose to investigate potential grants to support the spending opportunities that we do not expect to be funded with the grantee's available and expected funding, which we refer to as "funding gaps." The principles we follow in deciding whether or not to fill a funding gap are described on this page.
The first of those principles is to put significant weight on our cost-effectiveness estimates. We use GiveDirectly's unconditional cash transfers as a benchmark for comparing the cost-effectiveness of different funding gaps, which we describe in multiples of "cash." Thus, if we estimate that a funding gap is "10x cash," this means we estimate it to be ten times as cost-effective as unconditional cash transfers. As of this writing, we have typically funded opportunities that meet or exceed a relatively high bar: 8x cash. We also consider funding opportunities that are between 5x and 8x cash.
Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes to other grants we have made or considered making, and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.
Based on our cost-effectiveness analysis of Nutrition International’s VAS program in Chad, we believe this program is in the range of cost-effectiveness of programs we expect to direct funding to, as of 2022. Our estimate is that Nutrition International’s VAS program in Chad is 30 times as cost-effective as GiveDirectly's program.14
Our cost-effectiveness model of this program relied heavily on the work we have done previously creating a cost-effectiveness model of VAS for Helen Keller International.15 We updated the following elements of our model:
- Disease burden data – We used country-specific estimates of child mortality rate, proportion of child deaths due to infectious diseases overall, measles, and diarrhea, and the burden of vitamin A deficiency from the Institute for Health Metrics and Evaluation’s Global Burden of Disease 2019 model.16
- Cost per supplement – For Chad, we estimate that VAS costs $1.65 per child per supplementation round. Using Nutrition International's projected budget and target populations for Chad yields an estimate of $1.09 per child per supplementation round.17 We have used a higher figure as a best guess, based on actual results from Helen Keller International ($1.10 average). We then increased that figure by 50% to roughly account for our understanding that it is more expensive to operate programs in Chad than in most other countries in Africa.18 We haven't increased the grant size to match this higher expected cost per child, so there is a risk that if we're right about higher costs, Nutrition International may need additional funding to fully fund its work over the next three years. We will keep in touch with Nutrition International about this and consider additional grants if justified.
- Likelihood that other funders would replace funding for this grant if we didn’t make it – We adjust our cost-effectiveness estimates to account for the extent to which we believe our funding may be crowding out funding that would otherwise have come from other sources. Specifically, these adjustments represent the proportion of a grantee's funding that we believe may crowd out funding from other sources (for example, if we use an adjustment of 25%, we believe that 25 cents of every 1 dollar spent by the grantee would otherwise have come from other sources). See more details in this blog post. We have roughly estimated that if we did not make this grant, the government would replace 10% of Nutrition International's costs and other philanthropic actors (most likely Global Affairs Canada (GAC) and/or UNICEF) would replace 40% of this grant.19 More below.
Funding landscape for VAS campaigns
Global VAS funding landscape
Based on conversations with funders and implementers working on VAS over five years, our understanding is that the primary sources of funding for VAS are domestic governments (generally in the form of in-kind contributions), Global Affairs Canada (GAC) (through grants to Nutrition International and UNICEF), UNICEF's flexible funding, and GiveWell-directed donations. We're not aware of major recent changes to resources provided by domestic governments or UNICEF through its flexible funding. The available funding has been insufficient—we have continued to identify highly cost-effective campaigns that need funding. As discussed in our recent page on Helen Keller International, Helen Keller told us that UNICEF's funding from GAC has been decreasing over the last few years. Its current GAC grant is scheduled to end in 2022 and may be renewed at a lower, similar, or higher amount. This suggests that funding for VAS may be slightly shrinking.
Another consideration in the funding landscape for VAS globally is that our understanding is that there are fewer polio vaccination campaigns for VAS to piggyback on.20 VAS can be delivered more cheaply when it piggybacks on another campaign and historically polio campaigns were the chosen vehicle.21 With fewer opportunities to piggyback, costs of VAS campaigns have been and may continue increasing.
Given the global picture of limited and possibly slightly shrinking funding for VAS, paired with increasing costs of delivering VAS due to fewer polio immunization campaigns to co-deliver VAS with, we believe it is unlikely that this grant will cause much funding to be crowded out of the space of VAS in highly cost-effective locations.
Chad VAS funding landscape
To better understand the likelihood that funding for VAS in Chad will be available from another source, we looked at what funding was available in the last two years. In Chad, in 2020 only one VAS campaign was held due to the COVID-19 pandemic.22 We don't know what the coverage rates were nationally. In January and March 2021, VAS was delivered across the country by co-delivering it with a vaccination campaign.23 We assume that UNICEF funded the costs of adding VAS to the vaccination campaign, as that has been what's happened in other countries. This type of VAS campaign tends to cost significantly less than a standalone VAS campaign. In the second half of 2021, Nutrition International contributed to the VAS campaign out of an “emergency" grant from GAC; that grant is ending in September 2022. If we did not make this grant, Nutrition International would have some remaining "emergency" funding from GAC for the first campaign of 2022, but it doesn't expect to have enough funding for the full costs of a campaign in the first half of 2022. Another possible source of funding is a vaccination campaign, perhaps funded by UNICEF or GAC.
In making this grant, we may be crowding out other funders from Chad; however, even after modeling this probability in our cost-effectiveness analysis, this grant remains highly cost-effective. As discussed above, we've made a rough guess that each $1 we contribute crowds out $0.40 from other philanthropic sources. We estimate that most of the risk is that we may crowd out funding from UNICEF and vaccination campaigns. We would need to have a value above 85% of the funding being replaced to make the grant below 8x cash in our model.24
After considering the funding landscape for VAS campaigns both globally and in Chad, we think that making a grant to support VAS in Chad prevents the possible delay of a highly cost-effective program and remains cost-effective despite probably crowding out some funding from other sources for Chad’s VAS programming.
Risks and reservations
- Uncertainties in our cost-effectiveness model. We are uncertain about the values we use for some of the parameters included in our cost-effectiveness model, including the relative risk of all-cause mortality, serial correlation of mortality, and external validity. More below.
- Uncertainties about Nutrition International as a grantee. As when we start working with any new grantee, we have more uncertainty about Nutrition International’s work than our long-term partners.' More below.
Uncertainties in our cost-effectiveness model
We are uncertain about the values we use for the following parameters included in our cost-effectiveness model:
- Relative risk of all-cause mortality for children aged 6 to 59 months participating in VAS programs25 – As discussed in our intervention report on VAS, there is conflicting evidence about the effectiveness of VAS in reducing child mortality. For this cost-effectiveness model, we use an estimate of 0.76 for the relative risk of all-cause mortality with vitamin A supplementation. See more details on how we came to this estimate in the footnote.26
- Serial correlation of mortality – We have not adjusted our cost-effectiveness analysis for serial correlation of mortality. Would randomized control trials with longer follow-up periods have found a smaller effect size on all-cause mortality because children who didn't die due to receiving VAS had a greater overall risk of death? We care most about how this affects the cost-effectiveness of VAS relative to that of other interventions we fund. To date, we have found little evidence that sheds light on this question; we may do a deeper investigation in the future.
- External validity27 – Our external validity adjustment takes into account differences in the prevalence of vitamin A deficiency between places where trials of VAS took place and Chad today and changes in the composition of causes of child mortality. Our biggest question about the cost-effectiveness of VAS is the extent to which results from ~30 years ago translate to today's contexts. The five studies that receive 80% of the weight in the meta-analysis of VAS that our cost-effectiveness analysis is based on were published in 1986-1993.28 We use an external validity adjustment to reduce the expected effect size in our cost-effectiveness analysis. This adjustment is based on vitamin A deficiency rates and the proportion of deaths that are due to infectious diseases, measles, and diarrhea.29 We are continuing to work on the structure and inputs for the external validity adjustment and may substantially revise this in the future. In particular, the adjustment is highly sensitive to estimates of vitamin A deficiency that we have sourced from the Global Burden of Disease,30 and recent work to learn more about changes in these estimates has led us to reconsider whether to continue using the estimates. That work is ongoing.
Uncertainties about Nutrition International as a grantee
As when we start working with any new grantee, we have more uncertainty about Nutrition International’s work than our long-term partners' due to the fact that we do not yet have a history of working with the organization.
Our sense is that Nutrition International's role in supporting VAS differs from the role typically played by our prior grantee in VAS, Helen Keller International. Most of Nutrition International's spending on VAS goes to buying and donating vitamin A capsules to all countries that need them.31 In addition, it provides some financial and technical support for the delivery of VAS to a set of "core" countries. In 2019 and 2020, its annual spending on supporting VAS programs (excluding purchases of vitamin A capsules) was $3 million per year in Africa (in five countries) and $4-5 million per year overall (in nine countries).32 The amount it spent in a given country varied a lot between those two years. In 2021-2022, Nutrition International is providing financial and technical support for VAS in 15 countries under an 18-month emergency response grant from GAC. For this grant, it has chosen to hire consultants as technical advisors to assist Ministries of Health in each country.33 Nutrition International has told us that this is different from its typical model of establishing offices and hiring staff in each country it works in. This is a somewhat of a simplification, as the model that each organization employs varies by country. Each model has pros and cons: for example, Nutrition International may be better positioned to start up support in new locations, while being less able to provide certain types of support to a program than Helen Keller International could in a place where it has a staff presence. These tradeoffs are largely speculative, and we expect to learn more about Nutrition International's strengths and weaknesses as we follow its work in Chad and continue conversations about possibly supporting its VAS work in other locations.
Plans for follow up
- We will ask Nutrition International to have check-in calls every two to three months.
- We will ask for annual updates on spending, the size of target populations, and methods and findings of coverage surveys. We will use this information to update our estimate of the cost per child per supplementation round for Chad.
- We will also consider supporting Nutrition International for VAS campaigns in other countries.
We plan to ask Nutrition International the following questions:
- Have campaigns been happening every six months? What portion of the country have campaigns reached?
- Have there been opportunities to co-deliver VAS with immunization campaigns or other types of campaigns? Has Nutrition International been able to take advantage of these opportunities?
- Who, if anyone, has supported the campaigns besides Nutrition International?
- We will read the reports from coverage surveys that Nutrition International conducts and ask any follow-up questions to help us understand the potential sources of bias in the data.
- Was this grant sufficient to fund three years of VAS in Chad? Was there remaining funding at the end of the grant period?
|Conditional on updating our cost-effectiveness analysis to incorporate actual spending, we conclude that the cost-effectiveness of this grant was greater than 8x cash.
|End of 2025
|GiveWell funds a third-party to support Nutrition International on coverage assessments (either coverage rates or direct counting of people served) in Chad.
|End of 2023
|GiveWell recommends at least one additional grant to Nutrition International to support VAS in another country.
|End of 2022
Our process for this grant relied heavily on (a) our prior work on modeling the cost-effectiveness of vitamin A supplementation, and (b) our discussions with Nutritional International. For this particular grant:
- We began talking to Nutrition International about VAS in late 2020.
- We requested that Nutrition International give us a list of a few countries where Nutrition International was potentially interested in supporting VAS and did not have sufficient funding to do so. Nutrition International highlighted Angola, Benin, Chad, four states in Nigeria, Sierra Leone, Zimbabwe, and Mozambique. The latter three were for supporting non-campaign delivery of VAS; since that's more complicated for us to assess, we decided to start with cases that are simpler for us to assess.
- Nutrition International sent us grant proposals and budgets, and we had a series of meetings to clarify questions.
- We discussed the methodology for post-event coverage surveys with Nutrition International.34 We also discussed the possibility of employing a third-party research group to consult on Nutrition International's monitoring or collect data independently.
For internal review, a Senior Researcher, two Program Officers, and a Senior Program Officer who were not otherwise involved in the grant investigation gave feedback on the plan for investigating the grant. GiveWell researchers also gave feedback prior to approval of the grant.
We aim to get feedback on our grantmaking from stakeholders other than our top charities, such as government officials, other implementers involved in delivering the program, or other organizations working in the relevant context. The goals of these conversations are to learn more about the context in which a program will be delivered, to confirm the need for additional support of the program, and to seek feedback on the activities that a potential grant to support the program would enable. The external conversations we had about this grant include:
- A conversation with representatives of Global Affairs Canada's nutrition team
- A conversation with USAID
- A conversation with members of the nutrition team at Bill and Melinda Gates Foundation
- A conversation with representatives of the National Primary Health Care Development Agency in Nigeria who could speak to Nutrition International's work in Nigeria
We value the insights we gained by speaking with these organizations and appreciate the time they spent answering our questions. We note that the views expressed on this page, and any errors, are our own.
"Vitamin A deficiency (VAD) impairs body functions and may cause death. Adverse health consequences may also include xerophthalmia (dry eyes), susceptibility to infection, stunting, and anaemia (Sommer 1996; Rice 2004)....VAD is common in low- and middle-income countries." Imdad et al. 2017, Pg 7.
“WHO regional estimates indicate that the highest proportion of preschool-age children affected by night blindness, 2.0%, is in Africa, a value that is four times higher than estimated in South-East Asia (0.5%). This also means that Africa has the greatest number of preschool-age children affected with night blindness (2.55 million), and corresponds to almost half of the children affected globally (Table 10). A comparable and high proportion of pregnant women affected by night blindness are in Africa (9.8%) and South-East Asia (9.9%), each of which is estimated to have over 3 million pregnant women affected, or one third of the pregnant women affected globally. The estimates show that the Africa and South-East Asia regions also contain the highest proportions of preschool-age children with biochemical VAD, as indicated by a serum retinol concentration <0.70 µmol/l, with South-East Asia having the greatest number of children and pregnant women affected” WHO, Global prevalence of vitamin A deficiency in populations at risk, 2009, Pgs 10-11.
“Low vitamin A intake during nutritionally demanding periods in life, such as infancy, childhood, pregnancy and lactation, greatly raises the risk of health consequences, or vitamin A deficiency disorders (VADD).” WHO, Global prevalence of vitamin A deficiency in populations at risk, 2009, Pg 1.
"In settings where vitamin A deficiency is a public health problem, vitamin A supplementation is recommended in infants and children 6–59 months of age as a public health intervention to reduce child morbidity and mortality (strong recommendation). The quality of the available evidence for all-cause mortality was high, whereas for all other critical outcomes it was moderate to very low. The quality of the available evidence for outcomes in human immunodeficiency virus (HIV)- positive children was moderate for all-cause mortality." WHO, Guideline: Vitamin A supplementation in infants and children 6-59 months of age, 2011, Pg 1.
“We are a dedicated team headquartered in Ottawa, Canada, with offices across Africa and Asia.” Nutrition International, "About Us"
“Working as an expert ally to governments, we combine a deep technical expertise with a flexible approach to deliver proven nutrition interventions at scale.” Nutrition International, "Our Work"
“Since 1994, Nutrition International has been leading the global effort to reduce deaths of children under five by making sure that vitamin A supplements are delivered to developing countries around the world.” Nutrition International, "Vitamin A Supplementation"
For Nutrition International’s spending on support for VAS delivery and capsules in Fiscal Years 2019-2020 and 2020-2021, see here. Nutrition International notes that it has donated 80% of the global supply of vitamin A capsules.
To be exact, the grant is for $8,951,600. See Nutrition International’s budget here.
Nutrition International's support in Chad, for example, has to date been without a country office and through working with the government and consultants. Call with Nutrition International on October 7, 2021. We expect that in the initial stage of this grant, that arrangement may continue, and longer-term Nutrition International may build a team of staff in Chad.
See Nutrition International’s budget here.
This grant includes $570,000 worth of vitamin A capsules. In the past, Nutrition International has used funding from Global Affairs Canada (GAC) to buy all the capsules that it had committed to donate (for a given set of countries). The cost of capsules has increased and it is now seeking to raise more funding for capsule purchases. We might have been able to negotiate with Nutrition International to fund the capsules for Chad from its GAC grant, but we chose not to because the capsules are a critical component of the program and a small portion of the grant.
See our cost-effectiveness analysis for Nutrition International here.
See 2021 GiveWell cost-effectiveness analysis version 3, sheet "Helen Keller International." We discuss our model of cost-effectiveness for Helen Keller on this page.
We used projected figures from Nutrition International to estimate cost per supplement here.
For example, the cost of SMC per child in Chad is higher than any other country we have data from. See here.
See our adjustments here.
- “...since 2018, VAS has typically been co-delivered with immunization campaigns using a door-to-door approach. More recently however, opportunities to integrate vitamin A have diminished as partner priorities have changed.” Nutrition International, Chad proposal, 2021, Pg. 2.
- Helen Keller also reports that there has been a decrease in the frequency of polio vaccination campaigns in the last few years in the countries in which it works on VAS. "The frequency of polio campaigns organized in Helen Keller countries continues to decline (see Table 2)." Helen Keller International, Room for more funding report, 2021, Pg 7.
“When polio campaigns take place, the World Health Organization and the Global Polio Eradication Initiative cover the cost of the core teams, and VAS stakeholders 'only' have to support the addition of one distributor. When there is no polio campaign, VAS partners usually support two distributors per campaign, resulting in significantly higher costs.” Helen Keller International, Room for more funding report, 2021, Pg 7.
“Chad only held one campaign in 2020 due to COVID-19.” Nutrition International, Chad proposal, 2021, Pg 2.
“In January 2021, due to resource constraints, VAS was co-delivered with immunizations in 10 provinces, targeting 9–59-month-old children. In March 2021, the remaining 13 districts were covered through an integrated VAS/immunization campaign, also targeting 9-59-month-old children.” Nutrition International, Chad proposal, 2021, Pg 2.
Adjust the value in row "Scenario 2: Other philanthropic actors would replace HKI's costs" for Chad and check results in row "Cost-effectiveness in multiples of cash transfers, after all adjustments."
See this parameter in our cost-effectiveness analysis here.
Imdad et al. 2017 is a meta-analysis of vitamin A supplementation trials in young children. A fixed-effect model suggests a relative risk (RR) of all-cause mortality of 0.88 with vitamin A supplementation. A random-effects model suggests a RR of 0.76. The studies in Imdad et al. 2017 involve different locations, populations, doses, time periods, etc. GiveWell finds it plausible that variations in these factors may lead to meaningful differences in vitamin A efficacy in different studies. As a result, vitamin A trials may be better suited for analysis with a random-effects model than a fixed-effect model. The large difference between the random-effects and fixed-effect results is primarily driven by how the two models handle the results of the DEVTA trial. DEVTA was a very large trial, and it receives the majority of the weight in the fixed-effect meta-analysis. Our decision to rely on the random-effects model is strongly supported by an unpublished work that we have not discussed publicly. Additional discussion about the research on the efficacy of Vitamin A supplementation programs is available here.
- “[T]he overall effect [on all-cause mortality] was strongly influenced by five studies that accounted for over 80% of the weighted mean.” Imdad et al. 2010, Pg 19.
- The five studies are: Ross et al. 1993, West et al. 1991, Herrera et al. 1992, Daulaire et al. 1992, and Sommer et al. 1986. For more details on these studies, see our page on VAS here.
The calculations for the external validity adjustment are in the following spreadsheet: GiveWell, Nutrition International VAS - information for country-level cost-effectiveness analyses [Feb 2022], sheet "External validity by country."
See this spreadsheet, "Cost of Vitamin A Capsules" row.
See this spreadsheet.
Calls with Nutrition International on January 20, 2022, and October 7, 2021 (unpublished)
Conversation with Nutrition International on November 30, 2021 (unpublished).