2023 Cost-Effectiveness Analysis Changelog

This page provides details about changes that were made to our cost-effectiveness analysis (CEA) in 2023. Each changelog entry represents our understanding at the time the change was made. For past versions of our CEA, see this page.

Table of Contents

Version 2 — Published April 3, 2023

Link to the cost-effectiveness analysis (CEA) file: 2023 CEA — version 2

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Updating cost per child estimates in our Deworm the World CEA

Our Deworm the World cost per child analysis relies on program information on the costs incurred and the number of children treated for past campaigns. The previous version of our analysis was based on program information for the 2014-2020 period. We updated our analysis to incorporate program information from 2021.1 More information about our updated cost per child analysis is available here.2

Change 2: Updating cost per supplement estimates in our Helen Keller International CEA

We estimate the cost-effectiveness of Helen Keller International (Helen Keller)'s vitamin A supplementation (VAS) program using estimates of the average cost per child per supplementation round from previous campaigns. We updated our Helen Keller International cost per supplement analysis to incorporate program information from 2020 and 2021.3

Change 3: Updating the leverage/funging probabilities in our Helen Keller International CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.4 As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur without philanthropic support for a particular program as well as how cost-effectively funding by other actors would be spent if they were not contributing to the program.

Based on new information from Helen Keller International's 2022 room for more funding proposal, as well as conversations we've had with UNICEF and Global Affairs Canada, we updated our estimates of the likelihood that domestic governments or other philanthropic actors would fund the vitamin A supplementation (VAS) programs in our Helen Keller International (Helen Keller) CEA in Helen Keller's absence.5 We also increased our estimate of the counterfactual value of other philanthropic actors' spending for Burkina Faso, Cameroon, Côte d'Ivoire, DRC, and Guinea. Our understanding is that much of the funding UNICEF spends on VAS in these countries comes from Global Affairs Canada, which earmarks this funding for VAS. Therefore, we expect that UNICEF funding displaced by Helen Keller would likely be used to support VAS in other, potentially less cost-effective, locations. We very roughly guess that the counterfactual value of this spending would be equal to the counterfactual value of the Global Fund's spending.6 (More recently we updated this to a slightly less rough estimate, see below.) This is higher than the value we use for countries where we don't expect a large portion of UNICEF's spending to be earmarked for VAS.7

Change 4: Updating the external validity adjustments in our Helen Keller International CEA

We base our estimate of Helen Keller International (Helen Keller)'s vitamin A supplementation (VAS) program's effect on mortality among 6-59-month-olds on the findings of Imdad et al. 2017, a meta-analysis evaluating the impact of VAS on child mortality. We use an external validity parameter to capture differences in causes of mortality in populations targeted by Helen Keller's program relative to populations who participated in the trials included in Imdad et al. 2017.8 Vitamin A deficiency (VAD) rates are a key input into this external validity adjustment. Our previous analysis used VAD estimates from the 2017 results of the Institute for Health Metrics and Evaluation (IHME)'s Global Burden of Disease (GBD) as a key input for most programs.9 Because we're uncertain about the accuracy of IHME's model for producing VAD estimates, we decided to calculate new best-guess VAD rates for these programs that incorporate VAD prevalence rates found in national and regional surveys alongside the GBD 2017 estimates. 10 While most VAD surveys are outdated, we have found the survey-based VAD estimates to be more transparent than IHME's methodology. To create these estimates, we:

  • Searched for existing vitamin A deficiency surveys from relevant countries. When multiple national surveys were available, we used the most recent survey. When no national VAD results were available, we searched for regional surveys representing portions of relevant countries. We found regional surveys only for Burkina Faso and Mali, which we weigh less highly than national surveys.
  • Adjusted available survey results to account for improving circumstances. There are no VAD surveys available from the past few years, with the most recent one we found from 2010.11 We make a rough and uncertain assumption that improving circumstances in relevant countries will tend to lead to better nutritional outcomes and reduced rates of VAD over time and that these outdated VAD surveys likely overstate current rates of deficiency as a result. We assume that VAD rates fall by 1.33% per year based on the annualized rate we found in our more detailed improving circumstances analysis for Nigeria.12
  • Created best-guess VAD rates that combine survey results with IHME estimates from GBD 2017. For countries where no survey results were available, we relied exclusively on GBD 2017 estimates. For countries where there were survey results, we averaged the VAD rates from the survey results (adjusted for improving circumstances since the time of the survey) and the VAD rates from GBD 2017. For countries where national survey results were available, we put equal weight on adjusted national survey estimates and GBD estimates.13 For countries where only regional surveys were available, we put a lower 25% weight on the survey results and 75% weight on GBD 2017 because we assume results from small regional surveys are less reliable than national results.

This change increased our estimates of VAD prevalence in some countries and decreased them in others, which doesn't suggest to us that IHME has systematically over- or under-estimated VAD prevalence in these countries.14 We remain uncertain about the VAD estimates we've arrived at with this new methodology and we may revisit these estimates when IHME publishes a new GBD model. We conducted a sensitivity analysis to investigate how sensitive our final cost-effectiveness estimates would be to changes in our VAD prevalence estimates and concluded that further work on these estimates would be unlikely to affect our short-term funding decisions.15

Change 5: Updating the quality of monitoring adjustment in our Deworm the World CEA

In our cost-effectiveness analyses, we make adjustments to account for how different charity-level factors affect our best guess of cost-effectiveness, including two adjustments that account for the quality of a charity's monitoring and evaluation. We revisited these parameters for our Deworm the World CEA after reviewing their recent monitoring information (see our review here).

The first of these adjustments ("misappropriation without monitoring results") is intended to account for the possibility that coverage results from the subset of programs for which monitoring data was collected is not representative of all programs in that geography which are supported by GiveWell. If this is the case, the monitoring data collected may over or understate program coverage, and we make a downward adjustment for the risk that it is overstated. We increased this adjustment from 1% to 5% for all Nigerian states. Deworm the World has generally provided monitoring from all deworming rounds in India, Kenya and Pakistan (with some exceptions because of disruption to their programs during the COVID-19 pandemic).16 The situation is different in Nigeria, where Deworm the World normally conducts a full coverage survey for only 1 of the 2 rounds of deworming it conducts per year (in states where it conducts 2 rounds of deworming).17 We therefore use a higher value for this adjustment in Nigeria than other countries.

The second adjustment ("false monitoring results") is intended to account for the possibility that the monitoring results we have seen are partially or fully fabricated or biased in a way that leads us to overestimate coverage. We increased this adjustment from 0% to 1% for all countries to account for two factors: (1) disruptions to Deworm the World's standard monitoring procedure caused by the COVID-19 pandemic in 2021, and (2) we have not reviewed Deworm the World's process monitoring updates since 2019.18 Our overall adjustment value remains low because we continue to believe that Deworm the World has a strong monitoring procedure.19

Overall, these two changes resulted in decreases to our cost-effectiveness estimates for Deworm the World's program across all countries.20

Change 6: Updating the quality of monitoring adjustment for our Sightsavers CEA

In our cost-effectiveness analyses, we make adjustments to account for how different charity-level factors affect our best guess of cost-effectiveness, including adjustments that account for the quality of a charity's monitoring and evaluation. We revisited these adjustments for our Sightsavers CEA after reviewing their recent monitoring information (see our recent review here). One of these adjustments, ("misappropriation without monitoring results") is intended to account for the possibility that coverage results from the subset of programs for which monitoring data was collected is not representative of all programs in that geography which are supported by GiveWell. If this is the case, the monitoring data collected may over or understate program coverage, and we make a downward adjustment for the risk that it is overstated. We decreased this adjustment from 10%
to 3% across all countries to reflect our updated view of increased comprehensiveness in Sightsavers' monitoring. Our previous higher adjustment accounted for a lack of monitoring in 2019.21 We have now seen monitoring results representing 80% of Sightsavers' spending on relevant programs since 2018.22 Furthermore, we have seen monitoring results representing nearly all of Sightsavers' spending on relevant programs for its Year 4 (2020-2021) and Year 5 (2021-2022) program years.23 Changing this adjustment led to an increase in our estimated cost-effectiveness for Sightsavers across all countries.24

Change 7: Splitting Nigeria into state-level columns in our Malaria Consortium CEA

Our cost-effectiveness analysis previously calculated a single cost-effectiveness estimate for Malaria Consortium's seasonal malaria chemoprevention (SMC) program for several states in Nigeria, as well as separate estimates for two states, the Federal Capital Territory (FCT) and Oyo.25 We have now updated our CEA to calculate separate cost-effectiveness estimates for each Nigerian state in which Malaria Consortium supports SMC using GiveWell-directed funding.26 To do so, we updated malaria mortality and prevalence parameters using state-level estimates rather than national-level estimates.27 Our new state-level cost-effectiveness estimates for Nigeria are here.

Change 8: Fixing the malaria seasonality parameter for FCT and Oyo (Nigeria) in our Malaria Consortium CEA

We corrected an error we had made in estimating malaria seasonality for the Federal Capital Territory (FCT) and Oyo states in Nigeria in our CEA for Malaria Consortium's seasonal malaria chemoprevention (SMC) program. SMC is delivered in monthly cycles during the season of the year when malaria transmission is high. While a full SMC round has typically comprised four monthly cycles, some locations may choose to deliver more or fewer cycles, in line with the length of their high-transmission season. We include a parameter in our CEA that represents the proportion of annual malaria mortality that occurs during the SMC round and therefore can potentially be averted by SMC. Because rainfall data28 suggests that the high-transmission season for malaria is longer in FCT and Oyo than in other SMC-eligible areas, we previously assumed that this meant that a lower proportion of annual malaria mortality occurs during the SMC round and adjusted our value for this parameter to 60%, down from our default value of 70%.29 We now believe this was a mistake because FCT and Oyo deliver five cycles of SMC instead of the usual four, and so these states have a longer SMC round than other locations in our model.30 We updated our CEA to use the default value of 70% for FCT and Oyo, under the assumption that a similar proportion of annual malaria mortality occurs during the SMC round in FCT and Oyo as in other locations in our model, but stretched over a period of five months instead of four.31 We continue to make this same assumption and use the same 70% value for the other locations in our model that deliver five cycles of SMC, including Burkina Faso and several other Nigerian states, which are also areas with longer seasons of high transmission.32

Change 9: Updating the leverage and funging adjustments for our Malaria Consortium CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.33
As part of an investigation into a grant to renew our support for seasonal malaria chemoprevention (SMC) in Burkina Faso, Chad, Nigeria, and Togo, we revisited our estimates for the likelihood that GiveWell-directed funding would crowd out funding from the other major funders of SMC in those countries. See this section of our grant page for a discussion of the information we took into consideration, which modestly lowered our estimates—and therefore increased our cost-effectiveness estimates—for each country.34

Change 10: Updating the cost per SMC cycle estimates for Malaria Consortium

We estimate the cost-effectiveness of Malaria Consortium's seasonal malaria chemoprevention (SMC) programs using estimates of the average cost per cycle of SMC administered from previous campaigns. We updated our Malaria Consortium cost per SMC cycle analysis to incorporate program information on costs and coverage from 2021 for Burkina Faso, Chad and Nigeria (see here). We also made several methodological changes to the analysis, including:

  • Updating our calculations to use data from only more recent years (2018 onwards). We had previously used data from 2015 onwards,35 but we decided to make this change because we expect the cost per SMC cycle from more recent program years to be more indicative of the cost per SMC cycle that Malaria Consortium will achieve in future program years.36
  • Updating our adherence adjustment (to account for some children not swallowing all three doses of the SMC drugs) with more recently available data.37
  • Not putting any weight on projected future budgets for two Nigerian states (FCT and Oyo) where we had previously relied partly on data from previous SMC campaigns in other states in Nigeria and partly on Malaria Consortium’s future budgets. Following this update, all our Nigeria estimates are now based on previous campaign data rather than projections.38 This means that our FCT and Oyo estimates are based on campaign data from other states, since campaigns in FCT and Oyo started in 2022 and data was not yet available at the time of this update.39

Because Malaria Consortium's SMC program in Togo is co-funded by the Global Fund and UNICEF but we do not have information on the costs paid by these actors, we do not calculate a separate cost per SMC cycle estimate for Togo, but rather use a weighted average of our estimates for Burkina Faso, Chad, and Nigeria.40 This weighted average for Togo also changed due to the calculation adjustments described above.

See the impact of these changes on our cost-effectiveness estimates here.

Change 11: Updating the adjustment for marginal funding going to lower priority areas in our Malaria Consortium CEA

In our cost-effectiveness analyses for our top charities, we adjust for the likelihood that when a program is already receiving substantial funds from other sources, additional funds may be more likely used to cover lower-priority areas within the program's target area.41 As part of splitting Nigeria into state-level columns for Malaria Consortium's SMC program (see change 7 above), we set the values we used for this adjustment to zero because GiveWell is the only major funder of SMC in those states.42

Change 12: Updating the supplemental charity-level adjustments for our Malaria Consortium CEA

In our cost-effectiveness analyses, we include an adjustment to account for the possibility that a charity may use funding we direct to them to support research or other work that they see as being related to the program we intended to fund, but that we don't find valuable ("change of priorities"). We are not aware of Malaria Consortium using GiveWell-directed funds in this way in recent years. In general, we believe that we are highly aligned with Malaria Consortium on how it plans to use GiveWell-directed funding, and it has consistently requested our feedback on opportunities it is seeing to redirect funding to a different purpose or on decisions it is making on how to use its discretionary research budget. We have reduced this adjustment from 2% to 1% in our CEA of Malaria Consortium's seasonal malaria chemoprevention program.43

Change 13: Updating cost per child dewormed estimates for our Sightsavers CEA

We estimate the cost-effectiveness of Sightsavers' deworming program using estimates of the average cost per child dewormed per year from previous campaigns and/or cost per child projections based on forward-looking budgets and treatment plans. To generate these estimates, we take into account program information on the costs of the program and the number of children reached. We updated our Sightsavers' cost per child analysis.44 For the Democratic Republic of Congo (DRC), Cameroon, Guinea, Guinea-Bissau, and some Nigerian states,45 we updated our cost per child estimates based on historical program data from 2019-2021. For Chad, Senegal, and some other Nigerian states,46 we calculated cost per child using projections from forward-looking budgets and, in some cases, averages of past program data. For specifics on each country, see the cell notes here.47

Change 14: Updating the worm burden adjustment for our Sightsavers CEA

Our recommendation of mass deworming programs primarily relies on a series of follow-up studies to the experiment described in Miguel and Kremer 2004.48 We apply a "worm burden" adjustment to our cost-effectiveness analyses of deworming programs to account for differences in the prevalence and intensity of worm infections between the population studied in Miguel and Kremer 2004 and populations reached by the deworming programs we consider funding.

See a more detailed explanation of our worm burden adjustment here.

We updated our worm burden adjustments for Sightsavers' deworming program in Chad based on new information about which subnational areas the program would cover. We also noticed that we had been using two different methods for imputing infection intensity for our worm burden adjustments in different Nigerian states (exact intensity figures are not available for most states in Nigeria). We have now updated our methodology to be consistent across states.49 See how these updated worm burden adjustments impacted our cost-effectiveness estimates here.

Change 15: Updating the counterfactual value of philanthropic funding for our Helen Keller International CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have.50 Our leverage and funging adjustments are informed by our estimates of how cost-effectively funding by governments and other philanthropic actors would be spent if they were not contributing to these programs. In our Helen Keller International CEA, we updated our estimate of the counterfactual value of other actors' philanthropic spending on vitamin A supplementation (VAS) in Burkina Faso, Cameroon, Côte d'Ivoire, DRC, and Guinea.51 We had previously linked our estimate of the counterfactual value of other philanthropic actors' spending in these countries to our estimate of the counterfactual value of Global Fund spending, which was in turn tied to our cost-effectiveness estimate for an Against Malaria Foundation (AMF)-supported long-lasting insecticide-treated net distribution in DRC.52 We decided to come up with a new method for estimating this value so that our cost-effectiveness estimates for Helen Keller would be more accurate and well-reasoned. Our new estimate is based on the assumption that there is a 50% chance that other philanthropic actors' spending on VAS would be counterfactually used to fund VAS in other countries versus non-VAS programs.53

Change 16: Adding a decay adjustment to long-term income benefits for our deworming CEAs

In response to criticism of our deworming cost-effectiveness analyses by the Happier Lives Institute, we revisited our method for modeling the long-term income benefits of deworming programs. We added an adjustment to these benefits to account for the possibility that the benefits of deworming could decline over time. This is a rough adjustment that incorporates both our prior assumption that the effects of the program remain constant over time and evidence from three long-term follow-ups54 to the RCT Miguel and Kremer 2004 that could be interpreted as suggesting decaying benefits over time.55 See our full write-up on this topic here. This change decreased our cost-effectiveness estimates for deworming programs by 10-12% across the board.56

Change 17: Updating development effects references for our Helen Keller International and New Incentives CEAs

In our cost-effectiveness analyses for Helen Keller International's vitamin A supplementation program and New Incentives' conditional cash transfers for childhood vaccinations program, we include an estimate of development effects, which are the long-term effects of the program on income/consumption.57 For both programs, we don't have direct information on development effects, so we estimate the effects based on the magnitude of the development effects we've modeled for SMC.58 We previously estimated the development effects for these programs by adjusting a benchmark of SMC in Nigeria, because it had roughly average cost-effectiveness among locations where Malaria Consortium implements SMC programs.59 However, we have since updated our model to separate out Nigerian states,60 and Nigeria no longer represents roughly average cost-effectiveness, making it less practical to use as a benchmark.61 We now model development benefits for Helen Keller International's and New Incentives' programs based on the average development benefits of Malaria Consortium's SMC programs overall.62

Change 18: Adding province-level estimates for Pakistan to our Deworm the World CEA

We previously calculated a cost-effectiveness estimate for Deworm the World's program in Pakistan using one overall estimate.

See the version of the CEA prior to this change here.

We split our Deworm the World cost-effectiveness analysis for Pakistan into province-level columns, because we have now investigated worm burden data at the province level and concluded that worm burden differs significantly across provinces.63

Change 19: Updating the supplemental charity-level adjustments in the Helen Keller International CEA

We revisited several of the supplemental charity-level adjustments in our Helen Keller International (Helen Keller) CEA as part of our investigation into a potential renewal grant and made several changes.

We adjust our cost-effectiveness estimates for vitamin A supplementation (VAS) to account for the possibility that children may have already received VAS from another source. We increased this adjustment from 15% to 25%, to account for new information suggesting that VAS coverage through routine delivery systems may be relatively high for 6-12 month-old babies, who may receive vitamin A supplementation when they visit clinics for routine vaccinations.64 We also updated our adjustment that accounts for the possibility that Helen Keller may use funding we direct to them to support research or other work that they see as being related to the program we intended to fund, but that we don't find valuable ("change of priorities"). We reduced this adjustment from 5% to 0% because, over the last couple of years, Helen Keller has consistently used GiveWell grants to support VAS campaigns and associated research aimed at improving VAS delivery.

We also revisited our adjustments that account for the quality of Helen Keller's monitoring and evaluation. The first of these adjustments is intended to account for the possibility that coverage results from the subset of programs for which monitoring data was collected may overstate program coverage overall ("misappropriation without monitoring results"). We updated this adjustment from 6% to 15% because the monitoring results we've seen are not fully comprehensive of Helen Keller's campaigns in recent years, and we have some concerns about the representativeness of these results.65 The second adjustment is intended to account for the possibility that the monitoring results we have seen are partially or fully fabricated or biased in a way that leads us to overestimate coverage ("false monitoring results"). We updated this adjustment from 3% to 2% to account for our increased confidence in coverage survey results after seeing the results of auditing procedures for surveys conducted after three recent campaigns.66

Change 20: Fixing an error in our worm burden adjustment for Sightsavers' program in West, Cameroon

Our recommendation of mass deworming programs primarily relies on a series of follow-up studies to the experiment described in Miguel and Kremer 2004.67 We apply a "worm burden" adjustment to our cost-effectiveness analyses of deworming programs to account for differences in the prevalence and intensity of worm infections between the population studied in Miguel and Kremer 2004 and populations reached by the deworming programs we consider funding.68 We found a copy/paste error in our worm burden adjustment for Sightsavers' program in West, Cameroon. Correcting this error led to a 40% decrease in our cost-effectiveness estimate for the program.69

Version 1 — Published January 13, 2023

Link to the cost-effectiveness analysis (CEA) file: 2023 CEA — version 1

See this spreadsheet for the impact of each of the changes below on our cost-effectiveness estimates.

Change 1: Added additional Nigerian states to the New Incentives CEA and separated out cost-effectiveness estimates by state.

Our CEA previously calculated a single cost-effectiveness estimate for New Incentives' conditional cash transfer program in Nigeria by averaging together vaccination and disease burden estimates for the three states New Incentives originally operated in: Jigawa, Katsina, and Zamfara. New Incentives has now expanded its program into areas within a couple other Nigerian states70 and is planning to expand to even more states in the future.71 In order to more accurately estimate the cost-effectiveness of New Incentives' program in different areas, we have now updated our CEA to calculate separate cost-effectiveness estimates for each state and incorporated additional vaccination and disease burden data specific to current and potential expansion states.72

Change 2: Updated the counterfactual vaccination coverage rate in the New Incentives CEA

In our CEA, we model the benefits of New Incentives' program in terms of "counterfactually vaccinated" infants, meaning we only count the benefits to vaccinated infants who wouldn't have been vaccinated in the program's absence. In order to make this calculation, we estimate the proportion of infants enrolled in New Incentives' program who would have been vaccinated whether the program existed or not. How we set this estimate can have a significant effect on cost-effectiveness, since the costs of vaccinating these infants are still included in the CEA, even though the benefits aren't.

In the previous version of the CEA, we based this estimate on the endline vaccination rate for the BCG vaccine among the control group of the randomized controlled trial (RCT) of New Incentives' program. After adjusting for self-reporting bias, the results of the RCT imply that nearly half (48%) of enrolled infants would have received BCG vaccinations in the absence of the program.73

New Incentives has begun conducting rapid assessments of baseline vaccination coverage before expanding to new areas, and we have now seen the results of the earliest of these rapid assessments. After adjusting for self-reporting bias, the rapid assessments imply lower average baseline BCG vaccination coverage (34%) than the results of the RCT.74 We expect vaccination coverage in New Incentives' current areas of operation and future expansion areas to be more similar to the areas assessed in the rapid assessments than the areas where the RCT was conducted. However, we also think data from the RCT is likely to be higher quality than data from the rapid assessments.75 We have accordingly decided to put 35% weight on the results of the RCT and 65% weight on the results of the rapid assessments, resulting in a final counterfactual vaccination coverage estimate of 39%.76

Change 3: Updated the cost per infant estimates in the New Incentives CEA

We updated our cost per infant analysis for New Incentives to incorporate program data from September 2021 to July 2022. This resulted in a slight decrease to our estimate of New Incentives' cost per enrolled infant.77

We also added an adjustment to our estimates of spending incurred by the Nigerian government and Gavi, an international organization that primarily supports vaccination programs in low-income countries, to account for some vaccination costs being fixed costs of the immunization platform. We very roughly guess that fixed costs make up about 30% of total costs and have accordingly reduced our estimates of the government's and Gavi's costs per counterfactually vaccinated infant by 30%.78 We are highly uncertain about the appropriate value for this adjustment and may investigate this question further in the future.

Change 4: Updated the proportion of PBO nets purchased for South Sudan in the cost per net and insecticide resistance analyses for the Against Malaria Foundation (AMF) CEA

For some distributions of long-lasting insecticide-treated nets (LLINs), a portion of the nets AMF purchases are treated with a piperonyl butoxide (PBO) synergist in addition to standard insecticide (pyrethroid). These "PBO nets" are more expensive than standard nets, but there is evidence that PBO improves the effectiveness of nets in areas where a significant proportion of mosquitoes are resistant to standard insecticide.

In a previous changelog entry, we described our work creating a CEA for LLIN distributions in South Sudan, including our work estimating an appropriate insecticide resistance adjustment for this country. Based on the data we'd seen on insecticide resistance rates in South Sudan and our initial understanding of AMF's plans, we initially expected that all of the nets purchased by AMF for a distribution in South Sudan would be PBO nets. This assumption increased our cost per net estimate for the program and decreased the size of our insecticide resistance adjustment, which estimates the reduction in LLIN efficacy caused by local insecticide resistance.79

However, we have since learned that about 80% of the LLINs needed for the upcoming 2023 distributions in South Sudan are standard nets that were ordered prior to AMF's involvement in planning the campaign. AMF has told us that it will backfill the funding for those standard nets and that it plans to purchase PBO nets to make up the remaining 20% of nets needed for the campaign. We have updated our cost per net estimate and insecticide resistance adjustment in our South Sudan CEA to incorporate the lower proportion of PBO nets. We also incorporated some additional cost information we received from AMF into our cost per net estimate. These changes decreased our cost per net estimate for South Sudan slightly and increased our insecticide resistance adjustment for South Sudan substantially, leading to an overall decrease in cost-effectiveness.80

Change 5: Updated the leverage and funging and campaign spacing adjustments in the Against Malaria Foundation (AMF) CEA for South Sudan

We learned additional information about the funding landscape for LLIN campaigns in South Sudan and how AMF's contributions will impact those campaigns. This led us to update two adjustments in our AMF CEA for South Sudan: our leverage and funging adjustment and our adjustment for the impact of the program being to reduce the amount of time between LLIN distributions.

Leverage and funging

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur in the absence of philanthropic support for a particular program (e.g., "government costs would replace philanthropic costs" or "distributions would go unfunded"). In our AMF CEA, one of the scenarios we include is that the Global Fund to Fight AIDS, Tuberculosis and Malaria (the Global Fund) or the President's Malaria Initiative (PMI) would fund the program in the absence of AMF's support. We had previously estimated that there was a 30% chance the Global Fund would replace philanthropic costs for a distribution in South Sudan (PMI does not provide funding to South Sudan). We have since gotten feedback from AMF that, given the time-sensitivity of this funding gap, it was very unlikely that the Global Fund would reallocate funding in time to fill it. Based on this feedback, we have decreased our estimate to 15%.81

Campaign spacing

For most countries, our AMF CEA estimates the cost-effectiveness of philanthropic funding causing people to receive new LLINs who otherwise would not have received LLINs. In situations where we believe the true impact of philanthropic funding is to allow a distribution to occur sooner than it would have otherwise, we apply an adjustment to account for the difference in cost-effectiveness between these two scenarios. Based on information we received from AMF and conversations we had with other stakeholders, we expect that the impact of providing funding for LLIN campaigns in South Sudan would be to allow them to take place an average of 10 months sooner than they would have otherwise, moving them from an average interval of 45 months between campaigns to an average of 35 months between campaigns.82 We estimate that this is 37% less cost-effective than the scenario modeled in the unadjusted AMF CEA,83 and have accordingly added a 37% downward adjustment to the CEA.84

Change 6: Updated the counterfactual value of spending by Gavi in the New Incentives CEA

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

In order to estimate the impact that leverage and funging have on the cost-effectiveness of a program, we need to estimate how cost-effectively other actors would hypothetically spend their money if they didn't spend it on the program, which we refer to as the "counterfactual value" of their spending. One of the key benefits we model in our CEA of New Incentives' conditional cash transfers for vaccinations program is leveraging funding from Gavi, an international organization that primarily supports vaccination programs in low-income countries. We previously assumed that the counterfactual value of Gavi's spending is equal to the counterfactual value of the Global Fund's spending, which we estimate to be about 38% as cost-effective as an Against Malaria Foundation (AMF)-funded LLIN distribution in the DRC.85

Conceptually, this estimate was based on the idea that our estimate of the counterfactual value of Gavi's spending should approximate the cost-effectiveness of other programs Gavi supports, assuming that Gavi funds that would be leveraged by New Incentives' program would otherwise be spent on other programs within Gavi's portfolio. However, after investigating this issue in more depth, we concluded that Gavi has consistently been able to raise sufficient funding to cover its entire portfolio of programs, and we expect it will continue to be able to do so in the future. Therefore, it makes more sense to conceptualize the counterfactual value of Gavi's spending as how cost-effectively Gavi's donors, primarily the Bill and Melinda Gates Foundation and high-income country governments, would otherwise spend the funds that would be leveraged by New Incentives' program.

Upon investigating the cost-effectiveness of spending by Gavi's donors, we revised our estimate of the counterfactual value of Gavi's spending downward by more than half, from 0.0167 units of value per dollar to 0.007 units of value per dollar, which increased the overall cost-effectiveness of New Incentives' program.86

Change 7: Added ten additional Nigerian states to the New Incentives CEA

We learned that New Incentives is considering expanding its conditional cash transfer program to ten additional Nigerian states. We have added those states to our New Incentives CEA.87

  • 1

    See the previous version of our cost per child analysis here and the updated version of this analysis here.

  • 2

    See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 3

    See the previous version of our cost per supplement analysis here and the updated version of this analysis here. See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 4For a full introduction to our approach to leverage and funging adjustments, see this blog post.
  • 5 We decreased our estimate of the likelihood of funging for most countries. We did not change our estimates for Cote d'Ivoire or DRC. For Cameroon, we increased the funging probability from 25% to 33%.
  • 6See here.
  • 7We estimate a lower counterfactual value for countries where the funding isn't earmarked, because we think VAS is a more effective use of funds than other programs the funding would likely be spent on.
  • 8

    See how these external validity adjustments are used in our cost-effectiveness model here.

  • 9

    See our previous analysis here. We created separate best-guess VAD rates for Nigerian states and Kenya that incorporated information from country surveys.

  • 10

    We had been using VAD estimates from GBD 2017 because IHME made a significant methodological change between its GBD 2017 and GBD 2019 models that caused VAD prevalence estimates to significantly decrease. We do not fully understand the reasons for this decrease and have therefore continued to rely on GBD 2017. However, we are also uncertain about the methodology behind the GBD 2017 estimates, and we think it's likely that they are based on outdated survey data.

  • 11

    See the years of the most recent VAD survey in each country here.

  • 12

    See here.

  • 13

    For example, see equal weighting for Cameroon here.

  • 14

    See the impact of this change on our cost-effectiveness estimates here.

  • 15

    See our sensitivity analysis here.

  • 16

    See details of which rounds of monitoring took place in the "Results" sheets of this spreadsheet.

  • 17

    "In Nigerian states, coverage validation is only conducted once a year, either in the first or second round." Deworm the World, responses to GiveWell's questions, November 2022 (unpublished).

  • 18

    See our review of Deworm the World here for more discussion of their process monitoring.

  • 19

    More details in this cell note.

  • 20

    See the version of the CEA before this update here, and after this update here.

  • 21

    See our previous cell note here.

  • 22

    See here, Sightsavers coverage surveys [2022], sheet "Comprehensiveness".

  • 23

    See here, Sightsavers coverage surveys [2022], sheet "Comprehensiveness".

  • 24

    See here, 2023 V1 to 2023 V2 CEA change tracker, sheet "Deworming".

  • 25

    See here for our previous overall Nigeria estimate and here for our FCT and Oyo estimates.

  • 26

    We created estimates for the following states: Kebbi, Sokoto, Bauchi, Nasarawa, Borno, Kogi, and Plateau. See here.

  • 27

    State-level data is in this spreadsheet.

  • 28

    See here.

  • 29

    See here.

  • 30

    See here in our CEA, row "average number of SMC cycles per year".

  • 31

    The previous version before this update is here; see this updated parameter here. This change increased our cost-effectiveness estimate of Malaria Consortium's program in FCT and Oyo, Nigeria by about 15%.

  • 32

    See here for Burkina Faso, for example, where some areas deliver 5 cycles and some areas deliver 4 cycles. See here for average number of cycles in each location.

  • 33

    For a full introduction to our approach to leverage and funging adjustments, see this blog post.

  • 34

    See the impact of this change on our cost-effectiveness estimates here.

  • 35

    See our previous analysis here.

  • 36

    See our note here.

  • 37

    See here for our updated calculations and assumptions.

  • 38

    Our current analysis has one estimate for Nigeria here; our previous analysis had separate estimates for FCT and Oyo here.

  • 39

    "In October 2021, GiveWell recommended that Open Philanthropy grant $15.9 million to Malaria Consortium, which we expect will enable them to support seasonal malaria chemoprevention (SMC) in Nigeria in the Federal Capital Territory (FCT) in 2022-2024 and in Oyo state in 2022." GiveWell, Malaria Consortium — Support for SMC in FCT and Oyo States, Nigeria (October 2021).

  • 40

    See here.

  • 41

    See the "Marginal funding goes to lower priority areas" parameter here.

  • 42

    See the version before this update here and after this update here.

  • 43

    See the impact of this change on our CEA here.

  • 44

    See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 45

    Kebbi, Kogi, Kwara, Sokoto, Benue, Yobe, and Taraba states. See here.

  • 46

    Kaduna, Zamfara, Niger, Kano, Adamawa, Katsina states. See here.

  • 47

    See years that we pull source data from here.

  • 48

    For further discussion, see:

  • 49

    See the worm burden adjustments before this update here and after this update here.

  • 50

    For a full introduction to our approach to leverage and funging adjustments, see this blog post.

  • 51

    See the value before this change here and after this change here.

  • 52

    In our previous calculations for the value of Global Funds spending here, we pulled from the AMF CEA.

  • 53

    See our updated calculations here, which feed into our parameter for the counterfactual value of other philanthropic actors supporting VAS campaigns here.

  • 54

    The main piece of evidence we use for the long-term effects of deworming is an RCT in Kenya with follow-ups at ~10 years (KLPS-2), ~15 years (KLPS-3) and ~20 years (KLPS-4) after children received deworming treatment. The RCT is Hamory et al. 2021.

  • 55

    See the adjustment for benefits decaying over time here. Our updated analysis also shows the present value of lifetime benefits from a year of deworming both before and after the adjustment.

  • 56

    See the impact of this change on our cost-effectiveness estimates here.

  • 57

    See our estimate of the development effects of New Incentives here and a write-up about how we estimate VAS development effects here.

  • 58

    See our note about estimating VAS development effects and our development effects estimation model for New Incentives.

  • 59

    See our cell note here in the version of our CEA before this update.

  • 60See Change 7 above.

  • 61

    See cost-effectiveness estimates for SMC here.

  • 62

    See our new method for estimating development effects for Helen Keller here and for New Incentives here.

  • 63

    See our province-level worm burden adjustments here.

  • 64

    Vitamin A supplementation may be delivered through both routine (facility-based) systems and mass campaigns. We have heard of cases where coverage may be relatively high for 6-12 month-olds who may receive vitamin A supplementation when they visit clinics for routine vaccinations. This rough value is attempting to account both for (a) the possibility that some children receive vitamin A through campaigns supported by Helen Keller International when they recently received it through visits to clinics, and (b) that the children most likely to receive double treatment are the youngest in the age range (because 6-12 month-olds are more likely to visit clinics as part of receiving routine vaccinations) and this may be a group where vitamin A supplementation has a higher than average impact on mortality (since mortality tends to be highest in the youngest children). Examples:

    • Helen Keller International notes, "Vitamin A supplements in Kenya are administered to children aged 6-59 months through various delivery approaches. Throughout the year, children can access VAS in primary health care facilities, but this routine coverage only accounts for around 20 percent of children, essentially children below 12 months, as many caregivers do not bring their children to the health facilities after the end of the immunization contact points at one year of age. Helen Keller International, Room for More Funding Report, 2021, p. 21.
    • The 2021 Demographic and Health Survey in Madagascar reports (translated by Google), "Among last-born children aged 6–23 months living with their mother, 40% have received vitamin A supplements in the last 6 months [...] Between 2008–09 and 2021, the percentage of children aged 6–59 months who received vitamin A supplements decreased significantly, from 72% to 40%. This is mainly due to the change in mode of distribution of vitamin A, which went from 'campaign' mode to routine mode since 2021." National Institute of Statistics, Antananarivo, Madagascar, Demographic and Health Survey in Madagascar, 2021, p. 224.
    • In 2022, Helen Keller conducted coverage surveys in parts of three countries where there was no external support for VAS campaigns and where Helen Keller may support in the future. These were locations that GiveWell had not yet funded because they were thought to have stronger routine delivery of VAS—they are therefore not representative of all countries supported by Helen Keller. Coverage found in these surveys was 42%, 46%, and 53%. Helen Keller International, email to GiveWell, January 9, 2023 (unpublished).

    Our understanding from Helen Keller is that most of the countries it works in to deliver VAS through campaigns have particularly weak health systems, so may have lower rates of health facility-delivered VAS than the examples above.

  • 65

    For example, we are concerned that 1) Helen Keller does a survey for only one of the two rounds of VAS each year, and surveys are more frequently occurring during the non-rainy season, when coverage rates may be substantially lower during the rainy season, and 2) some decision-makers for campaigns may know in advance whether or not a coverage survey will be conducted for a campaign and may know which districts will be surveyed. Both of these factors could bias coverage results upwards. See more details in the cell note here.

  • 66

    We have reviewed the findings of coverage survey audits for Helen Keller's campaigns in Kasai Oriental, DRC in 2022, Guinea in 2021, and Mali in 2021. These audits found high correspondence between the coverage results reported by initial surveyors and supervisors revisiting surveyed households.

  • 67

    For further discussion, see:

  • 68

    See a more detailed explanation of our worm burden adjustment here.

  • 69

    See the version before this change here, and after this change here. See the change in cost-effectiveness here.

  • 70
    • We recommended grants to fund expansions of New Incentives' program into new areas in August 2021, January 2022, and May 2022. New Incentives has told us that some of the expansion areas funded by the May 2022 grant are located in the states of Bauchi and Sokoto.
    • “This means we can now start operations in Sokoto once the baseline rapid assessments are completed (assuming coverage rates are low). We have submitted a draft MoU for expansion and collaboration to Bauchi State and have requested some members of the Bauchi State Health Research Ethics Committee (BASHREC) to review our draft application for rapid assessments. While we will continue to expand within current states and start expansion in Sokoto, going to Bauchi should help us minimize the proportion of LGAs with a likelihood of network shutdown or security issues, and also help us reduce vaccine supply and stakeholder risks since Bauchi is in the North East Zone of Nigeria. Bauchi was selected after reviewing states for low immunization rates, high Under-5 mortality (IHME), security, stakeholder support, measles incidence (2016), and other factors.” New Incentives, Program update to GiveWell, December 2021 [unpublished].

  • 71

    We recommended a grant in November 2022 that New Incentives expects to use to expand its program into six additional states in Nigeria. More information on this grant will be published in a forthcoming grant page.

  • 72

    See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 73

    See the version of the CEA preceding this change here.

  • 74

    See the results of the rapid assessments and our calculations here.

  • 75

    For more detail, see this write-up.

  • 76

    See our calculations here. See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 77

    See the previous version of our cost per infant analysis here and the updated version of this analysis here.

  • 78

    See our calculations here. See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 79

    We had previously set the cost per net estimate in South Sudan equal to the cost per net estimate in DRC, partially on the basis of our expectation that AMF would purchase a similarly large proportion of PBO nets for campaigns in South Sudan as in DRC. See our previous cost per net estimate for South Sudan here. See the previous version of our insecticide resistance adjustment calculations here.

  • 80

    See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 81

    See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 82

    See more detail in this spreadsheet

  • 83

    See our calculations here.

  • 84

    See the version of the CEA preceding this change here and the version of the CEA following this change here.

  • 85

    See the version of the CEA preceding this change here.

  • 86

    See our full write-up on the counterfactual value of Gavi spending here and our calculations here. See the version of the CEA following this change here.

  • 87

    See the version of the CEA preceding this change here and the version of the CEA following this change here.