New Incentives (Conditional Cash Transfers to Increase Infant Vaccination)

Published: April 2024

Summary

Basics

New Incentives runs a conditional cash transfer (CCT) program in northern Nigeria. The program primarily aims to increase vaccination rates through distributing cash incentives. Caregivers who bring their children for routine vaccinations, which are provided through government clinics free of charge, can receive a total of 6,000 naira (about $7) over six visits.

Clinic staff employed by the government conduct the vaccinations themselves, while New Incentives' field officers check children’s eligibility, enroll children into the program, and distribute cash incentives. (More)

New Incentives also conducts awareness-raising activities and aims to ensure sufficient vaccine supply by working with partners to address bottlenecks in the vaccine supply chain (see our separate report on New Incentives as an organization for more detail). Around a third of GiveWell’s funding for New Incentives is used for cash incentives, with the rest going to staff costs, distribution and transport costs, awareness-raising activities, and other costs (see here in our separate report on New Incentives).

How cost-effective is it?

We estimate that it costs approximately $1,000 to $5,000 (depending on the specific state)1 to avert a death through the program in the areas where New Incentives works. We also estimate that the program equates to being 11x - 51x as effective as spending on unconditional cash transfers (GiveWell’s benchmark for comparing different programs).

In simple terms, we think New Incentives’ program is cost-effective because:

• New Incentives' program increases the number of children who receive routine vaccines. Vaccination rates in northern Nigeria are low (~25% to ~65% in locations where New Incentives works), and we think New Incentives’ program addresses some of the barriers to vaccination (e.g., opportunity costs of taking children to clinics and perceived low value of vaccinations). (More) Extrapolating from the results of a randomized controlled trial (RCT) of New Incentives' program and triangulating against other studies of vaccine incentive programs, we estimate the program increases vaccination rates by approximately 11 - 22 percentage points in areas where it now works. (More)

• Child mortality is very high in northern Nigeria, and we’d expect many children to die from vaccine-preventable diseases without vaccination. We estimate that unvaccinated children under the age of five have an approximately 4% to 8% chance of dying from vaccine-preventable diseases before their fifth birthday in the locations where New Incentives works. Our analysis is mainly based on estimates of mortality from vaccine-preventable disease from the Global Burden of Disease (GBD) model. (More) We also estimate that vaccines indirectly avert about 0.75 deaths for every death they directly avert from the disease they target. This is based on evidence that vaccination programs sometimes have larger impacts on mortality than would be expected from their impact on the diseases they target alone (more).

• The vaccines that New Incentives incentivizes are effective at reducing mortality. Based largely on evidence from vaccine RCTs, we estimate that being vaccinated reduces a child’s mortality risk from vaccine-preventable diseases by approximately 50% - 55% from the point of vaccination up to age five (more). Our impression is also that routine childhood vaccination is widely viewed in the global health community as an effective way to reduce child mortality, strengthening our confidence in these estimates. (More)

• While the per-child cost of this program is higher than many other programs we fund, the benefit per child enrolled is also very high. This program provides cash transfers to caregivers, a cost not incurred by many other programs we fund. It also relies on a large network of in-country staff to disburse the cash transfers. We currently think that each child enrolled in the program costs New Incentives approximately $21 (more), compared to a cost per child of approximately $5 to $6 per year for Malaria Consortium's seasonal malaria chemoprevention (SMC) program. Although the per-child cost is high, we estimate that New Incentives' program is above our cost-effectiveness bar in many areas because vaccines are highly effective at averting mortality.

• We think New Incentives’ program leverages a large amount of external funding and is unlikely to be funded by other actors if we did not fund it. We attempt to adjust our cost-effectiveness analysis to account for the impact of the program on other actors’ spending. In this case, we think New Incentives’ program leverages a large amount of funding from Gavi and the Nigerian government (who pay for purchasing the vaccines and the staff to administer them). This increases our cost-effectiveness estimate because we think that the leveraged funding would have otherwise gone to less cost-effective programs (more). We also think that the program, or a similar vaccine incentive program, would be unlikely to be funded by other actors if we did not fund it (because we have seen relatively little interest from other actors in funding comparable programs). We account for the chance that another actor would fund the program or a similar one in GiveWell’s absence with a downward adjustment of approximately 10%, which is small compared to some other programs we fund (more).

• In addition to averting child mortality, we think that the program probably results in various additional benefits. These include:
  • Reduced mortality for children in later life, because we think vaccines provide some long-term protection against disease. This accounts for 15% to 20% of the benefits we model, varying by state (more).
  • Increased income for children in later life, by averting disease in a sensitive developmental period of childhood. This accounts for approximately 20% of the benefits we model (more).
  • Increased consumption for children and their families in the short term because of the cash incentives themselves. This accounts for around 1% to 5% of the benefits we model, varying by state (more).
  • Various other benefits that we don’t model and apply as percentage best guesses. These include reduced morbidity from disease and averted costs that would have been spent on treatment of disease. These increase our cost-effectiveness estimate by 33% (more).

We use a cost-effectiveness analysis to quantify our reasoning. Here is a summary of our analysis, using one state, Bauchi, as an example.

We’ve also considered other perspectives that might not be captured explicitly in these cost-effectiveness estimates (e.g., whether experts see New Incentives as a good program). Overall, none of the work we’ve done on these questions has substantially undermined our view that the case for New Incentives is strong.

However, we have spent considerably less time on these questions than we have on our main cost-effectiveness model. We hope to engage more with them in the future. (More)

How could we be wrong?

Overall, we are reasonably confident in the case for New Incentives’ program, and our level of uncertainty is in a similar range to GiveWell’s other top charities. New Incentives’ program is more complex than these other programs (implying additional uncertainty) and may have a higher risk of unintended negative consequences (e.g., risk of fraud, potential for backlash if the program is taken away). However, we think it is less likely that New Incentives’ program or similar programs would be funded in GiveWell’s absence than for other top charities, giving us confidence that our funding is leading to additional positive impact.

Our key open questions:

• Are we accounting for increases in vaccination rates that would have taken place anyway? The RCT found a substantial increase in control group vaccination rates over the course of the study. Other sources of data also show vaccination rates were increasing in northern Nigeria before New Incentives began scaling up. Because we’d expect the program to be less effective in areas with higher baseline coverage, this raises a concern that the impact in the future will be smaller than the impact observed in the RCT. We currently account for this with a rough -18% adjustment, but this is a guess and we have not explicitly modeled this. We also haven’t deeply investigated what other programs are being delivered to increase vaccination rates in northern Nigeria, which increases our uncertainty on this issue. (More)

• How reliable are the mortality estimates we rely on? Our cost-effectiveness analysis uses estimates of mortality from IHME's Global Burden of Disease (GBD) Project. These estimates are based on a number of modeling assumptions that we have not reviewed in detail, and we have substantial uncertainty about them. While we use GBD estimates across a number of our programs, we are particularly uncertain in this case because our analysis requires us to take a stance on the proportion of each cause of death attributable to different pathogens (e.g., the share of diarrhea deaths attributable to rotavirus), which adds additional uncertainty. We also assume that 0.75 deaths are indirectly attributable to vaccine-preventable disease for every death directly attributed to these diseases. This is a rough best guess and could be improved with further research. (More) Our 25th - 75th percentile confidence interval for a child’s probability of death before age five in Bauchi is 3.7% to 7.9%. This implies a program cost-effectiveness of 15 to 31 times as cost-effective as direct cash transfers (“15x-31x”).

• How effective are vaccines at reducing mortality in Nigeria? Some evidence we have seen suggests that the measles vaccine might be less effective in Nigeria than would be expected based on published literature. We adjust for this risk in our analysis (reducing our estimate of vaccine efficacy by around 20%). But our adjustment is very speculative and further research could lead us to update this upward or downward. (More) Our 25th - 75th percentile confidence interval for vaccine efficacy against vaccine-preventable disease (which incorporates this adjustment) is 38% to 68%, implying a cost-effectiveness of 17x to 31x.

• How will the effect of New Incentives' program in states with higher vaccine coverage compare to the effect in the RCT of the program? Our understanding of the effect of New Incentives' program on vaccination rates is largely based on the results of an RCT of the program conducted in Jigawa, Katsina, and Zamfara in 2017-2020. We extrapolate the RCT results to other states based on their vaccine coverage rates, but we are unsure how well these results will translate to new states. (More) New Incentives has begun to conduct vaccine coverage surveys in areas where it works, and we plan to use these as another source to check that the program is having the expected effect at scale. But we’ve only just started to analyze these surveys (as of February 2024), and don’t incorporate them into our analysis yet. (More)

• How high is the risk of fraud? We’d expect fraud to be a significant risk for a program like New Incentives, which handles large volumes of cash. We currently account for this with a -10% adjustment to the number of children enrolled (reflecting that some children may be enrolled multiple times) (more), and -2% adjustment to account for other ways New Incentives’ monitoring data could be wrong (including fraud) (more). New Incentives has a number of systems to monitor and prevent fraud, but it’s possible that there are types of fraud these are unable to detect. We discuss this issue in detail on a separate page.

• Are we properly accounting for the community-level benefits of vaccines? Our analysis is based on the impact of vaccination for individual recipients reported in RCTs. We use rough supplementary adjustments of approximately 25% to account for the impact of increased vaccination on disease transmission, including during outbreaks, and the potential for development of herd immunity. (More) Incorporating these as percentage guesses allows us to use a relatively simple main cost-effectiveness model. However, our understanding is that community-level benefits are considered one of the major benefits of vaccination, and it's possible that explicitly modeling them could change our estimates up or down. We plan to get more input from epidemiologists and disease modelers on our analysis in the future. (More)

• Could the program lead to falling vaccination rates if it is discontinued? We’ve heard some criticisms that financial incentives could “crowd out” intrinsic motivations for vaccination. This raises a concern that if it was discontinued in a given area, vaccination rates might fall below where they would have been if New Incentives had never entered an area. We also think it’s possible (but uncertain) that the opposite factor could apply, and the program could lead to increased vaccination rates if it was discontinued (e.g., by habit formation or improving vaccine supply). At the moment, we don’t account for either factor quantitatively in our analysis. But we’re unsure about this because we haven’t deeply engaged with critics of cash incentive for immunization programs. We plan to do more work on this in the future. (More)

• Will New Incentives maintain program quality at scale? Since the RCT, New Incentives has expanded quickly and is operating in nine states (as of the end of 2023). We think it’s plausible that this could lead the program to be delivered less effectively (e.g., if managers have less effective oversight, or vaccine supply is not able to keep up with demand). The monitoring data we’ve reviewed has remained broadly stable since the RCT, although we have seen an increase in supply problems for some vaccines. We interpret this to mean that New Incentives has been able to deliver the program to a similar high quality as it has grown, although we plan to keep monitoring this. (More)

• Are there errors in our analysis we’ve missed? Our analysis of New Incentives’ program is more complicated than most of the other programs we support, and the program is comparatively newer (meaning our analysis has been reviewed less rigorously). It’s possible that this increases the likelihood of errors that could affect our bottom line.

• How accurate was our analysis of New Incentives in hindsight? GiveWell’s cost-effectiveness analyses are “forward-looking” and aim to predict the impact a program will have at the time we make a grant decision. We have paid less attention to backward checks to understand how accurate the predictions in our New Incentives grants were. This is a weakness in our approach and something we aim to improve in the future. (More)

• 1

  See this row in our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 2

  “Vaccines train your immune system to create antibodies, just as it does when it’s exposed to a disease. However, because vaccines contain only killed or weakened forms of germs like viruses or bacteria, they do not cause the disease or put you at risk of its complications.” World Health Organization, “Vaccines and immunization.”

• 3

  See World Health Organization, “Immunization dashboard”, “Vaccination coverage globally” section.

• 4

  See the following maps showing each country’s vaccination rates for the main vaccines we model in our analysis of New Incentives’ program:

  • DTP: see Nigeria (56% vaccinated) in the following map, which uses the WHO/UNICEF, “DTP3 (% of one-year-olds immunized)" dataset.. Our World in Data, "Share of one-year-olds vaccinated against diphtheria, pertussis, and tetanus," 2021
  • BCG: see Nigeria (75% vaccinated) in the following map, which uses the WHO/UNICEF, “BCG (% of one-year-olds immunized)" dataset. Our World in Data, "Share of one-year-olds vaccinated against tuberculosis, 2021"
  • Measles: see Nigeria (59% vaccinated) in the following map, which uses the WHO/UNICEF, “MCV1 (% of one-year-olds immunized)" dataset. Our World in Data, "Share of one-year-olds vaccinated against measles, 2021"
  • HiB: see Nigeria (56% vaccinated) in the following map, which uses the WHO/UNICEF, “Hib3 (% of one-year-olds immunized)" dataset. Our World in Data, "Share of one-year-olds vaccinated against Haemophilus influenzae type B, 2021"
  • PCV: see Nigeria (52% vaccinated) in the following map, which uses the WHO/UNICEF, “3.b.1 - Proportion of the target population with access to pneumococcal conjugate 3rd dose (PCV3) (%) - SH_ACS_PCV3" dataset. Our World in Data, "Share of children vaccinated with pneumococcal conjugate, 2021"
  • Rotavirus: Nigeria has no data for rotavirus, based on the WHO/UNICEF, “RotaC (% of one-year-olds immunized)" dataset. See this map: Our World in Data, "Share of one-year-olds vaccinated against rotavirus, 2021"

• 5
  • “Northern Nigerian communities were less likely to be vaccinated compared to Southern Nigerian communities. Studies suggested this was driven by higher levels of education and wealth in the South, higher proportions of Muslim households in the North, and greater insecurity in the North.” Mahachi et. al, 2022
  • See also National Bureau of Statistics and United Nations Children's Fund, Multiple Indicator Cluster Survey 2021, Statistical Snapshot Report, 2022, p. 30, figure labeled “Immunisation Coverage across Nigeria.”

• 6
  • “In partnership with WHO, Gavi, the Vaccine Alliance, UNICEF, and other partners, the Government of Nigeria, through the National Primary Health Care Development Agency (NPHCDA), administers 13 vaccines on its Expanded Programme on Immunization (EPI) schedule.” World Health Organization, “Beyond the numbers—Nigeria steps up measures to reach eligible children with potent vaccines,” May 19, 2023.
  • “The Expanded Programme on Immunization (EPI) was launched in 1974 to ensure that all children, in all countries, benefited from life-saving vaccines… There are now 13 vaccines (antigens) recommended by WHO for the EPI programme.” World Health Organization, “Essential Programme on Immunization”.

• 7
  • "In Nigeria, routine childhood vaccinations are provided at government clinics free of charge, but caregivers often find it difficult to afford transportation and face other challenges to taking their baby to a clinic – often half a day’s trip away or longer. These round-trip journeys to the clinic must be made six times to complete a childhood routine vaccination schedule, all within the first year of a baby’s life." New Incentives, "How it works"
  • See the infographic at New Incentives, "How it works". This conversion uses the naira:USD market exchange rate, which the Nigerian Central Bank reports as ~887 naira per $1 as of December 7, 2023. See this page (archived version available here).

• 8

  New Incentives has made a number of updates to the incentive schedule (originally 4,000 naira for all incentivized vaccinations) over time:

  • New Incentives originally added a 500 naira incentive for measles 2 (which had not yet been introduced into Nigeria's routine vaccination schedule at the time of the RCT). This brought the total incentive across all visits to 4,500 naira. In early 2023, New Incentives increased the incentive for measles 2 to 1,000 naira. This brought the total incentive across all visits to 5,000 naira.
  • In August 2023, New Incentives decided to change its schedule to offer 1,000 naira per visit (6,000 in total).

  New Incentives, January 14th 2022, December 16th 2022, July 7th 2023 Program Updates (unpublished).

• 9

  “CCTs are now present in 64 countries, a dramatic increase from 2 countries in 1997 and 27 in 2008.”
  “Conditional cash transfers (CCTs) are periodic monetary benefits to poor households that require beneficiaries to comply with specific behavioral requirements to encourage investments in human capital (such as school attendance, immunizations, and health checkups).” World Bank Group, The state of social safety nets 2015, p. 1 and 8.

• 10
  • Adapted from New Incentives, "How it works"’.
  • We directly model the benefits of increasing coverage of the directly incentivized vaccines and the rotavirus vaccine in our cost-effectiveness analysis. We do not directly model the remaining indirectly incentivized vaccines. We expect that they will provide only a small proportion of the overall benefits and so the additional time required to directly model their benefits would not be worth the added complexity (more below). Instead, we account for these vaccines with a rough percentage adjustment (+4% as of February 2024) elsewhere in our analysis.

• 11

  Oral polio vaccine.

• 12

  Inactivated polio vaccine.

• 13
  • “We closely collaborate with the state governments of Bauchi, Gombe, Jigawa, Kaduna, Kano, Katsina, Kebbi, Sokoto, and Zamfara in northern Nigeria.” New Incentives, “Our work” page (as of February 2024).
  • Note: Our cost-effectiveness analysis includes columns representing all 36 states in Nigeria and the Federal Capital Territory, but New Incentives does not currently operate in most of those states.

• 14

  We include these states only because they are the states that, as of January 2024, meet or exceed GiveWell’s cost-effectiveness bar for funding (which is set in terms of cost-effectiveness in multiples of direct cash transfers, and is currently for programs which we think are at least 10x as cost-effective as cash transfers—more here). This includes eight of the nine states where New Incentives’ program is operating (excluding Kaduna state, where we currently estimate New Incentives’ program is 8x as cost-effective as cash transfers overall, though we estimate select areas within the state are at least 10x as cost-effective as direct cash transfers). Our reasoning is as follows:

  • Under GiveWell’s May 2023 grant for the program, New Incentives expanded only to specific local government areas (LGAs) if GiveWell estimated that those LGAs met GiveWell’s 10x cost-effectiveness bar (based on New Incentives' assessment of baseline vaccination coverage in those LGAs). See this section of our grant page for more details.
  • This effectively means that, as of January 2024, New Incentives is only working in LGAs exceeding GiveWell’s 10x bar (i.e., the range included in this report), even if GiveWell’s cost-effectiveness estimate for a state as a whole was under 10x.

• 15

  We used the following process:

  • Two GiveWell staff members familiar with our New Incentives research gave confidence intervals for the parameters in the Simple CEA sheet of our cost-effectiveness analysis. The intervals were for New Incentives’ program in Bauchi state (the example program we discuss in this report). These intervals address the following question: “For each parameter, provide an upper and lower bound for the range of values that you think has a 50% probability of containing the true value (i.e., a 25% probability the true value is lower than the range, and a 25% probability the true value is higher).”
    • Where we felt that it was difficult to form an intuitive judgment about a parameter (e.g., because it comprised several separate parameters), we gave intervals for each individual parameter.
    • For example, our estimates of the probability that a child will die from vaccine-preventable disease before age 5 are aggregated from (i) the probability that an unvaccinated child will die directly of vaccine-preventable causes before age 5 (more) and (ii) our estimate of the number of deaths indirectly caused by vaccine-preventable diseases per direct death (more).
  • A third GiveWell staff member reviewed the intervals given by the first two staff members and decided upon a final interval for each parameter, using their subjective judgment.
  • We applied the intervals used for Bauchi to other states as follows:
    • We first calculated both the relative differences and absolute differences between our best guesses and our 25th and 75th percentile values for each parameter. For example, if our best guess value for a parameter was 50% with 25th and 75th percentile values of 25% and 75%, then the relative differences would be -50% and +50%, and the absolute differences would be -25 percentage points and +25 percentage points.
    • We assigned confidence intervals to each country that matched the widths of our confidence intervals for Bauchi, putting some weight on the relative difference between the confidence interval bounds and the best guess and some weight on the absolute difference. The higher our best guess value for a parameter in a country was compared to our best guess for Bauchi, the more weight we put on the absolute difference between the confidence interval bounds and our best guess. The purpose of this method was to avoid assigning extremely wide confidence intervals to parameters in countries where our best guess was much higher than our best guess for Bauchi. In some limited cases, we manually adjusted estimates where the intervals seemed implausible, compared to our best guess for Bauchi.
  • Finally, we used monte carlo simulations to transform the confidence intervals for individual parameters into an aggregated confidence interval for each parameter. These are the intervals that appear in the summary of this page, and the Bauchi columns in the Sensitivity Analysis sheet of our cost-effectiveness analysis.

• 16

  We use the term “counterfactual” to refer to the state of the world that would exist if we did not provide funding to a grantee for a program. In this case, we refer to children who are “counterfactually vaccinated” to mean children who are vaccinated as a result of New Incentives’ program who would not otherwise have been vaccinated. We exclude children whose caregivers receive the cash incentive, but who we think would have been vaccinated anyway, even without the program.

• 17

  $1 million is an arbitrary amount that we use to quantify the benefits of the program in the rest of our analysis.

• 18

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 19

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section. We calculate this by dividing $1 million by our estimate of the number of additional children reached by the program in each state, e.g., ~12,100 in Bauchi.

• 20

  See this row of our cost-effectiveness analysis. Note that this estimate corresponds to New Incentives’ cost per child enrolled in 2023, and we have not yet updated our estimates for 2024.

• 21

  See this sheet in our analysis for a breakdown by category.

• 22

  We estimate that the cost per child fell from $35.40 between June 2019 and May 2020 to $20.73 between June and December 2022. We think this is because a decreasing share of New Incentives’ overall costs were start-up costs during that period. See this row in our analysis.

• 23

  $11,275,557 / 627,845 = ~$17.96.

• 24

  New Incentives reports a cost per infant enrolled of $18.10 in 2022, compared to GiveWell’s calculation of $17.96. We decided to use the New Incentives figure because (i) New Incentives may have had access to more data when producing its estimate and (ii) the two estimates are very similar, so this doesn't make a large difference to our bottom line.

• 25
  • "We are continuing to take actions to improve Measles 2 immunization rates and believe that an increase to 1,000 Naira (from 500 Naira) for the Measles 2 CCT (CCT #6 given at the 15-month RI visit) could lead to positive results. This would increase the total amount of cash transfers from 4,500 Naira to 5,000 Naira. It would represent a 25% increase from the program measured during the RCT, in-line with Nigeria’s inflation rates." New Incentives, Program Update, December 2022 (unpublished). New Incentives confirmed that it is moving forward with this change in a call with GiveWell in January 2023 (unpublished).
  • Note: This change does not include New Incentives’ subsequent incentive increase to 6,000 naira (beginning in July 2023). We haven’t yet updated our analysis to account for this change. More here.

• 26

  See this section of our analysis for our calculations.

• 27

  Our calculation is based on:

  • Expected BCG scar rate: 90%. This is based on a quick literature review. We found three studies reporting scarring rates of 82-96%. These average to 87% (unweighted) which is roughly in line with the 90% cited by WHO as the BCG scar rate in its 2018 position paper ("BCG vaccination usually causes a scar at the site of injection due to local inflammatory processes. However scar formation is not a marker for protection and approximately 10% of vaccine recipients do not develop a scar." p. 84). We therefore use 90% as our rough best guess for the BCG scar rate. (More details above).
  • Scar rate in New Incentives’ monitoring data: New Incentives tracks the percentage of enrolled children with BCG scars. In 2022, it found that 99% of children had one BCG scar and 0.03% had two scars (New Incentives monitoring results December 2022). This is higher than the 90% rate we’d expect. Our best guess is that the explanation for this is that some children received the BCG vaccine more than once, and developed a scar on a subsequent occasion even if they didn’t scar at first.
  • We calculate our ~10% repeat enrollment estimate as the difference between the expected and actual scar rate / the expected scar rate + the percentage of children with two or more scars. This is 9% / (90% + 0.03%) = 9.97%. See this section of our analysis for our calculations.

• 28

  See this section of our analysis for this calculation.

• 29
  • Our understanding is that Nigeria operated two exchange rates, an official rate and a parallel rate, until June 2023. On average, in 2022 New Incentives received a foreign exchange rate of 668 Naira per US dollar. New Incentives, 2022 Progress Report (unpublished).
  • “Nigeria has for years operated multiple exchange rates for the naira—with the official exchange rate dictated by the central bank, while a far higher unofficial rate determined the price of imported commodities like wheat, which are priced in dollars.
    The exchange rate now will be determined by market forces and no longer the central bank, a move that analysts on Thursday said would boost inflows of money and help stabilize an economy battered by surging inflation and a record unemployment rate.” Associated Press, "Nigeria lets market set currency exchange rate to stabilize economy, woo investors, 2023.

• 30

  We estimate that in 2023 New Incentives will receive an exchange rate of 650 naira per US dollar. This is a weighted average of our best guesses of various probabilities for different exchange rate scenarios. See here for our calculations.

• 31

  Our calculation is based on publicly available data on vaccine coverage, costs, and population size between 2014 and 2018:

  • Vaccine coverage: We estimate that overall vaccine coverage was 51% in Nigeria between 2014 and 2018, using WHO and UNICEF estimates of coverage (available here).
  • Vaccine costs: We use publicly-available WHO estimates of expenditure on routine vaccinations between 2014 and 2018, compiled in this sheet. The WHO estimates that total spending in this period in Nigeria was ~$1.6 billion ($314 million per year), and the government covered around 29% of this. See this section of our analysis.
  • Population size: We use estimates from the Global Burden of Disease project (compiled here) that the average population under age 1 in Nigeria was ~6.9 million between 2014 and 2018. Assuming 51% of these children were vaccinated, this implies 3.5 million children per year vaccinated.
  • % of fixed costs: Our estimates are for the cost of each additional child vaccinated to the Nigerian government and Gavi. We would guess that this is lower than the average cost of each child vaccinated, because some proportion of total costs are fixed costs. We estimate that 30% of total costs to these actors for routine vaccinations are fixed costs. This is a very rough guess.

  Using these inputs, we estimate that each additional child vaccinated costs $62.92. ($314m x (100% - 30%) / ~3.5m children = $62.92. See this section of our analysis for our calculations.
  Finally, we roughly allocate costs between Gavi and the Nigerian government (as indicated in this section):

  • Based on the WHO data cited above, we estimate that the Nigerian government covered 29% of vaccine costs between 2014 and 2018, and Gavi covered 71%.
  • Based on Nigeria’s 2022 - 2024 strategy for immunization, which sets out a roadmap for transitioning vaccination spending to the government over time, we estimate that future costs will be covered 80% by the government and 20% by Gavi.
  • A straight average of these gives an estimate of 54% costs covered by the government and 46% covered by Gavi. This is a rough guess.
  • Multiplying these percentages by the total cost per additional child vaccinated ($62.92) gives our final estimates of $34.29 per additional child vaccinated to the government, and $28.64 to Gavi.

• 32

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 33

  We estimate that the cost per child fell from $35.40 between June 2019 and May 2020 to $20.73 between June and December 2022. We think this is because a decreasing share of New Incentives’ overall costs were start-up costs during that period. See this row in our analysis.

• 34

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 35

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 36

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 37

  We use the term “coverage” to refer to the proportion of children vaccinated. We define “baseline vaccination coverage” as the proportion of children who would be vaccinated in the absence of New Incentives’ program.

• 38

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 39
  • "New Incentives will group local government areas (LGAs) it expands to within a given state at a given point in time into ‘expansion groups’. New Incentives will then collect coverage data in these expansion groups once before the start of operations to establish baseline coverage rates." IDinsight, Coverage monitoring analysis plan, 2021, p. 1.
  • These expansion groups are called "cohorts."

• 40

  See the number of LGAs included in each cohort for which we have received coverage survey results (as of March 2024) here.

• 41

  "The Multiple Indicator Cluster Survey (MICS) was carried out in 2021 by the National Bureau of Statistics (NBS) as part of the Global MICS Programme. . . . The Global MICS Programme was developed by UNICEF in the 1990s as an international multi-purpose household survey programme to support countries in collecting internationally comparable data on a wide range of indicators on the situation of children and women." National Bureau of Statistics (NBS) and United Nations Children's Fund (UNICEF), Multiple Indicator Cluster Survey 2021, Statistical Snapshot Report, 2022, p. ii.

• 42

  See the MICS data we use in this sheet. Note that we do not use the MICS 2017 data in this sheet. This feeds into our calculations in a different part of our analysis.

• 43

  See this section of our analysis for the rules we apply to the data.

• 44

  For example in Zamfara, where New Incentives started operations before instituting coverage surveys in 2021. "New Incentives will group local government areas (LGAs) it expands to within a given state at a given point in time into ‘expansion groups’. New Incentives will then collect coverage data in these expansion groups once before the start of operations to establish baseline coverage rates." IDinsight, Coverage monitoring analysis plan, 2021, p. 1.

• 45

  Reasoning for each:

  • Measles vaccine: We think the coverage surveys may systematically underestimate coverage of the measles vaccine because they only survey children through 12 months, whereas MICS surveys 12-23 month olds. The first dose of the measles vaccine is recommended to be administered at 9 months, though some infants may receive it later. Thus, the MICS data captures infants getting the measles vaccine after 12 months (see the cell notes in this section for more detail). ("The unit of analysis is the individual 6-12-month-old infant." IDinsight, Coverage monitoring analysis plan, 2021, p. 4)
  • PCV vaccine: Because the coverage surveys do not include PCV, we use the coverage estimates for the Penta vaccine according to the rules above, and apply an adjustment based on the ratio of PCV coverage to Penta coverage observed in the MICS data (see this section for our calculations). (The coverage surveys include coverage of BCG, Penta, and measles vaccines, as indicated by this research question the assessments are intended to answer: "What is the self-reported coverage of the following routine childhood immunizations among 6-12 month olds at a given point in time in the state-LGA cohort: BCG, any Penta, Measles 1, full vaccination (loose)?" IDinsight, Coverage monitoring analysis plan, 2021, p. 2)
  • Rotavirus vaccine: We do not have data from either source on the rotavirus vaccine because it was not introduced in Nigeria’s immunization schedule until August 2022 (see Gavi, "Dealing with diarrhoea: Nigeria introduces rotavirus vaccine into its immunisation plan," August 30, 2022). We assume rotavirus coverage is the same average of Penta and PCV coverage, as both vaccines are administered at the same time (see this section for more detail).

• 46

  See this section of our analysis. For example, in Bauchi we estimate that the second dose of PCV contributes 19% to the overall under-five mortality benefit of the program, and the first dose of the measles vaccine contributes 6%.

• 47

  Our calculations are in this sheet. Each of the factors we include are discussed below.

  • Mortality associated with diseases targeted by each vaccine: We use state-level estimates of mortality from vaccine-preventable diseases from the Institute for Health Metrics and Evaluation's 2021 Global Burden of Disease model to calculate the proportion of vaccine-preventable deaths attributed to the diseases targeted by each vaccine. These are the same as the estimates that we use to model risk of death from vaccine preventable-disease, and we apply the same adjustments (discussed in this section).
  • Vaccine efficacy: We then adjust those proportions to account for the estimated efficacy of each vaccine in preventing disease. Adjusting for vaccine efficacy increases the overall contribution of vaccines with high estimated efficacy (such as BCG) and decreases the contribution of vaccines with lower estimates of efficacy (such as the rotavirus vaccine). We discuss our vaccine efficacy estimates in this section.
  • Efficacy by dose: Our vaccine efficacy estimates are for a full course of that vaccine. To translate dose-specific coverage to overall coverage for vaccines with multiple doses (PCV, Penta, and rotavirus), we came up with rough estimates of the marginal efficacy of each dose (i.e., how much protection is provided by a first dose compared to a second dose, etc.), based on a quick literature review. Our calculations are in this section of our analysis. The literature broadly matched our understanding that the marginal efficacy of the final dose of each vaccine tends to be lower than previous doses.

• 48

  See this row of our analysis for the coverage survey data and this row for the MICS data.

  Our method

  We estimate the level of overreporting by comparing caregiver-reported data on BCG vaccination coverage to data from the coverage surveys on BCG scarring (BCG vaccination typically leaves a scar, providing a more objective measure of vaccination rates). We roughly assume, based on a quick literature review, that 90% of children who are vaccinated against BCG develop a scar that will be detected and correctly identified when checked for (details discussed below).

  • The proportion of children with BCG scars in coverage surveys to date has been 55%, compared to 61% of children reported to be vaccinated by caregivers.
  • These values imply that the true rate of BCG vaccination was ~1% higher than caregivers reported (55% / 90%) / 55% - 100% = ~1%). See calculations here.
  • We make this same adjustment to coverage of all vaccines (not just BCG), assuming that the level of bias was probably around the same regardless of the specific vaccine.
  • We use a smaller (~0%) adjustment for the MICS data because we estimate that only 22% of the MICS data comes from self-reported questions (the rest comes from reviewing children’s vaccination cards). See this section of our analysis.

  Data on BCG scarring rates

  We found three studies reporting scarring rates of 82-96%. These average to 87% (unweighted) which is roughly in line with the 90% cited by WHO as the BCG scar rate in its 2018 position paper. We therefore use 90% as our rough best guess for the BCG scar rate. Because the estimates from the literature rely on an individual detecting and correctly identifying BCG scars (rather than being a theoretical percentage of all infants who receive BCG who scar, we no longer make additional adjustments for the percentage of scars which are detected or the percentage of detected scars which are in fact BCG scars. We had included such adjustments in earlier versions of our CEA. See here.

  • "BCG vaccination usually causes a scar at the site of injection due to local inflammatory processes. However scar formation is not a marker for protection and approximately 10% of vaccine recipients do not develop a scar." World Health Oragnization, BCG vaccines: WHO position paper – February 2018, p. 84.
  • "Two hundred and six subjects (96.3%) had a postvaccination BCG scar." Atimati and Osarogiagbon 2014.
  • "The prevalence of BCG scar was 79.5% among the male infants and 84.7% among the female infants, while the overall prevalence of BCG scar was 81.5%." Gambo et al. 2014.
  • "Although 84.2% had physical evidence of BCG inoculation only 69.8% had developed detectable sensitization to the tubercle bacilli as shown by the Mantoux test." Odujinrin and Ogunmekan 1992.

• 49

  See our analysis of self-report bias in the RCT here.

• 50

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 51

  We divide the impact of the program by the share of unvaccinated children at baseline to estimate the reduction in unvaccinated children caused by the program: 22pp / (100% - 36%) = ~33%. More information in this section.

• 52

  "This study is an impact evaluation of the NI-ABAE CCTs for RI Program in Katsina, Zamfara, and Jigawa
  States in North West Nigeria funded by Open Philanthropy at the recommendation of GiveWell."
  "The evaluation consisted of a two-arm clustered randomized controlled trial." IDinsight, Impact evaluation of New Incentives, final report, 2020, p.7.

• 53

  The report is available here: IDinsight, Impact evaluation of New Incentives, final report, 2020.

• 54

  “The evaluation consisted of a two-arm clustered randomized controlled trial. We worked with NIABAE to identify clinic catchments in the three evaluation states that met its operational criteria. We then randomly selected our sample of 167 clinics from among these clinics.” IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 7.

• 55

  "We measured most outcomes using caregiver reports from a household survey, during which enumerators also checked for BCG vaccine scars and recorded data from vaccination records kept in the home." IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 7.

• 56

  Adapted from IDinsight, Impact evaluation of New Incentives, final report, 2020, Table 3, pg. 23.

• 57

  This is referred to as “Penta 1” throughout the IDInsight final RCT report. However, our understanding based on viewing the survey questions (not published) is that "Penta 1" refers to receiving any dose of the Penta vaccine.

• 58

  In addition to the primary outcomes, this includes all three doses of PCV, and the three doses of Penta individually.

• 59

  See this section of our analysis for the impact on each vaccine dose.

• 60

  “Fully vaccinated” here refers to children receiving BCG, all 3 doses of the Penta vaccine, and the measles vaccine. “When using the strict definition of full immunization (BCG vaccine, Penta vaccine 1-3, Measles 1 vaccine), the likelihood of being fully immunized was 27 percentage points (95% CI: 23, 31, p-value 0.001) higher among children in the treatment areas.” IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 28.

• 61

  Note that this aggregation method implies that the program increased take-up of each vaccine by a different amount. This is different to our previous method for estimating the program’s effect size, which viewed the effects on the three main primary study outcomes as noisy estimates of the same underlying effect (more in a previous version of our research report here, footnote 7). We think it is plausible that New Incentives’ program would have different effects depending on where a vaccine falls in the schedule (e.g., less effect on early vaccines and more effect on later vaccines), although we’re unsure about this.

• 62

  We use the term “coverage” to refer to the proportion of children vaccinated.

• 63

  See this row in our supplementary analysis.

• 64

  See this section in our analysis.

• 65

  See this section in our analysis.

• 66

  "Across the study area, coverage in the control group was substantially higher at endline than at baseline. This was true for each of the primary study vaccines, with the largest difference for the Measles vaccine: 17.8% of children at baseline versus 57.2% of children at endline had reportedly received Measles 1 vaccine (Table 16). We did not expect a change of this magnitude." See also Table 16, IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 50.
  Note that the baseline survey was conducted in Zamfara and Katsina states only, and so these estimates aren’t directly comparable to the overall coverage estimate (including Jigawa state) that we discuss above.

• 67

  "Changes in the questionnaire or enumerator technique could have led to increased recording of vaccinations via the survey even if true vaccination coverage had not changed. This could result from either improved recall or increased social desirability, or both. This hypothesis could explain why we do not see similar increases in clinic tally sheets, which record relatively stable control-group vaccination volumes between baseline and endline (see Appendix I)." IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 51.

• 68

  “I don’t understand the reason for the huge increases in vaccination rates in the control areas in the RCT. I agree with your conclusion that those effects are unlikely to change the determination that NI’s program significantly increased vaccinations but I have two reservations. First, it seems important to determine what it was that increased vaccination rates in the control areas that much because that change was bigger than the treatment effect from NI. My hunch is that it was survey effects or spillovers that weren’t totally captured. I have seen these big changes in control group outcomes before in some of my own studies.”

  “The RCT report shows a very big increase in clinic outreach days in the control areas (from 2 to 3.5 month, a 75% increase). Is that accounted for in your model?” Dr. Jessica Cohen, Bruce A. Beal, Robert L. Beal and Alexander S. Beal Associate Professor of Global Health, Comments on GiveWell’s draft New Incentives Report, November 2023, unpublished. Note that Dr. Cohen’s response was shared before we compared the control group results to independent vaccine coverage surveys.

• 69

  See this spreadsheet for our calculations.

  Our approach

  • We compared RCT data on control group vaccine coverage increases from baseline to endline (in Katsina and Zamfara states only, since the baseline survey was only conducted in these states) with 3 independent vaccine coverage surveys conducted before and after the RCT: (1) MICS 2017 (here), (2) DHS 2018 (here), and (3) MICS 2021 (here). We compared data for BCG and measles 1 because they are the first and last vaccines covered in the RCT (in terms of time of vaccination).
  • The New Incentives baseline survey took place in 2017, and the endline took place in 2020 (in between these surveys). We extrapolated results from the three surveys using a linear trend to estimate what we’d expect vaccine coverage to have been in the New Incentives RCT at baseline and endline in both states. In both states and for both vaccines there was a trend of increasing BCG and measles vaccine coverage between 2016-2017 and 2021.
  • We compared the implied increase in vaccine coverage with the observed increase in the New Incentives RCT. Overall, the implied increase was 13 percentage points for the BCG vaccine and 12 percentage points for the measles 1 vaccine based on extrapolating from independent surveys, compared to 28 percentage points (BCG) and 38 percentage points (measles) in the New Incentives RCT. This implies 48% of the control group increase (BCG) and 30% (measles) is explained by the regional trend, or roughly 40% overall.
  • There are two reasons why we think these could be underestimates:
    • Different age ranges: The RCT used a different age range (12-16 months) than DHS/MICS vaccine coverage (12-23 months). We’d expect that this means the RCT would show larger increases than DHS/MICS given the same underlying changes. For example, for BCG received at birth, if coverage at birth increases over a short period of time, the BCG coverage measured among children aged 12-16 months will increase more rapidly than coverage measured amongst children aged 12-23 months because the children receiving the increase make up a larger share of the measured population.
    • New Incentives clinics may not be representative: The RCT was conducted in a set of clinics that met NI’s operational criteria, which meant excluding clinics that had high security risk and that were barely functional, amongst other reasons. We think it’s plausible that coverage would have increased faster in these clinics than average, regardless of New Incentives’ program.

  Uncertainties

  • The DHS/MICS state level results are fairly noisy and there are substantial survey-to-survey fluctuations. For example, in Zamfara in the DHS/MICS, BCG coverage was 19% in 2016, 16% in 2018, and 51% in 2021. This means we’re uncertain whether it’s valid to assume a linear trend.
  • Our analysis is based on only two states and two vaccines. Given the noisy estimates mentioned in the previous bullet, this means this comparison is fairly uncertain.

• 70

  Note: For our estimate of the main effect size of the program (21 percentage points), we use IDinsight’s estimates of the impact on all vaccine doses, not just the pre-specified outcomes (BCG, measles, and any dose of Penta vaccine). We do not see this as a significant threat to the validity of our analysis because (i) there were also significant increases in the pre-specified outcomes, (ii) we also incorporate information in the pre-specified outcomes, and (ii) we think it’s plausible that the impact of the program might vary according to when a vaccine is delivered in the vaccine schedule.

• 71

  From the independent review by the International Initiative for Impact Evaluation (3ie): "Overall, I found that IDinsight was receptive to suggestions, made major adjustments between baseline and endline that substantially improved the design, was diligent in documenting the Stata code, and their work and assessment are a valid assessment of the New Incentives Conditional Cash Transfer program. While there are remaining questions about the change in coverage (greatly improved) in control areas, and the poor accuracy of the biomarkers and its implications for the efficacy of the measles vaccine in Nigeria, we do not believe that these questions change the interpretation and conclusions of the study. The New Incentives program substantially and significantly increased immunizations rates." International Initiative for Impact Evaluation, Quality assurance of IDinsight's evaluation of New Incentives, 2020, pg. 3.

• 72

  “Vaccination volumes recorded at clinics have experienced statistically significantly larger increases since baseline in treatment clinics than in control clinics. Using clinic tally sheets (which count vaccination doses given at each clinic by vaccine and by month), we found that the average increase in monthly BCG vaccine doses administered between baseline and endline was 17.6 (95% CI: 9.1, 26.2, p-value < 0.001) doses greater in treatment clinics than control clinics.” IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 31-32.

• 73

  IDinsight, Impact evaluation of New Incentives, final report, 2020, Figures 16-18, pg. 65-66.

• 74

  “Programmatically, New Incentives’ theory of change continues to look promising. First, the sources of vaccinations measured align with the program. Outside of the national measles campaigns, almost all infants receive vaccinations from the sources New Incentives’ program will cover: health facilities and health facility outreach. Second, most caregivers cited lack of knowledge or ambivalence and relatively few caregivers cited socio-cultural reasons or mistrust and fear, as reasons for not vaccinating. It seems likely that an incentive, coupled with awareness raising activities, can overcome these stated reasons for not vaccinating. Finally, New Incentives’ program appears to be unique. While small incentives are relatively common, incentives worth more than 500 Naira are rare and only two caregivers received cash in our sample.” IDinsight, New Incentives evaluation baseline report, 2019, p. 71.

• 75

  We use an internal validity adjustment of -12%. Applying this to the impact of New Incentives’ program on full vaccination coverage of 27 percentage points (the measure we use here for comparison since it’s available across almost all of the other studies we looked at) implies an updated impact of 24 percentage points. See this row of our supplementary analysis for this calculation. This estimate would still be higher than any of the other studies we included in our analysis. Applying our internal validity adjustment to the estimate of program impact we actually use in our main analysis (21 percentage points, discussed above) implies an impact of ~18% (21pp x (100% - 12%)). This is higher than all but one of the other studies we looked at (Banerjee et. al. 2010).

• 76

  This is based on the estimates that we calculate above from the RCT: that it increased vaccine coverage by 22 percentage points from a baseline of 36%. 22pp / (100% - 36%) = ~33%. See this row in our analysis.

• 77

  See this section in our cost-effectiveness analysis.

• 78

  We reviewed nine studies in a 2021 World Bank meta-analysis of conditional cash transfers for immunization. We also conducted a literature search for additional studies. We found two studies published since the World Bank analysis and one study (Bannerjee et. al. 2010) that tested the impact of non-cash incentives (raw lentils and plates) that was not included in the World Bank analysis. Although this is not strictly speaking a conditional cash transfer for immunization, we included this study because the monetary value of these was similar to the cash transfer of the New Incentives program. The final study was the New Incentives RCT. See this spreadsheet for a summary of all 13 studies. This column summarizes which studies were randomized controlled trials.

• 79

  The exception to this is Morris et al. 2004, which reports outcomes on DTP3 instead. Since DTP3 comes last in the vaccination schedule, we think this is a reasonable proxy for full vaccination. See this row in our accompanying spreadsheet.

• 80

  See this column in our analysis spreadsheet.

• 81

  See the graphs in our accompanying spreadsheet. There seems to be a negative relationship between baseline coverage and effect size, such that effect size reduces as baseline coverage increases. We think this makes intuitive sense, as higher baseline coverage implies a smaller pool of caregivers that can be influenced by incentives.

• 82

  We use the following calculation:

  • (RCT effect size x RCT weight) + (skeptical prior x skeptical prior weight) = (27pp x 70%) + (16pp x 30%) = 24pp.
  • This implies a downward adjustment of ~12% ((24 / 77) - 100%). See this section of our accompanying spreadsheet.

• 83

  See this chart in our accompanying spreadsheet.

• 84

  See this row in our cost-effectiveness analysis.

• 85

  Our 95% estimate is based on estimates from the RCT indicating that 11% of children received the measles vaccine through campaigns, out of 59% overall who received the measles vaccine (~20%). Our understanding is that measles is the only vaccine we model in our main analysis which is regularly distributed through periodic campaigns, and the other vaccines are less likely to be distributed this way. We therefore assume ~5% of children overall were vaccinated through campaigns (20% x 25% because measles is one of the four main vaccines we model).

  “The increase in coverage in control areas since baseline is not fully explained by methodological changes or by campaigns. Coverage for all three primary outcomes appears to have increased via routine immunization activities. For BCG and Penta vaccines, campaigns do not account for any meaningful proportion of vaccinations in control areas. Accordingly, we would assume that nearly all of the vaccinations recorded in the control group for BCG vaccine and Penta vaccine represent always-vaccinators who receive vaccinations through RI activities. If we assume that the NI-ABAE CCTs for RI Program pays incentives for all RI-delivered vaccinations, then control coverage is a reasonable proxy for the number of incentives paid to always-vaccinators for BCG and Penta vaccines.

  “For the Measles vaccine, we expect that control coverage included a non-trivial proportion of campaign-vaccinators to whom the program may not always pay incentives. There remains uncertainty about the precise proportion, and our data only allows us to generate approximations. However, our best-guess is that RI Measles vaccine coverage in control is about 48% (or 11 percentage points lower than unadjusted self-reports coverage). This value falls in the middle of our various RI-only coverage estimates, which range from 44% to 53% depending on the adjustments applied. If GiveWell assumes that a large proportion of campaign-delivered vaccinations do not lead to program enrollment, then GiveWell should make a downward adjustment to control Measles vaccine coverage before using it as an estimate of enrolled always-vaccinators.” IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 62.

• 86

  See this section of our cost-effectiveness analysis.

• 87

  See this sheet for our calculations. These calculations include deaths averted across all age groups, and they account for variation between vaccines in disease burden, vaccine efficacy, baseline coverage, and treatment effect of the program. They also account for GiveWell’s moral weights.

• 88

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 89

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 90

  We refer to the diseases targeted by routine childhood vaccinations that are included in our analysis as “vaccine-preventable disease.” This does not include diseases targeted by other vaccines not included in our model.

• 91

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 92

  See this row of our cost-effectiveness analysis.

• 93

  These estimates (available here) are from the 2021 Global Burden of Disease model, which has not yet been fully published. We publish them here with permission from IHME.

• 94

  See this sheet for our calculations.

• 95

  See this row in our cost-effectiveness analysis. Because the GBD age bands (0 to 6 days, 7 to 27 days, under 1 year, and 1 to 4 years) don’t align exactly with the scheduled vaccination dates, this adjustment relies on some rough guesses.

  For example, in Bauchi State, the GBD model estimates that 9% of under-five meningitis deaths occur in the first 27 days of life (see this row). We roughly guess that the proportion of under-five meningitis deaths occurring in the first six weeks (before the first doses of PCV and HiB are scheduled) is 10% higher than this, or 10% in total (9% x 110% = 10%).

• 96

  See this row in our analysis.

• 97

  GiveWell conducted a literature review in 2020 to learn more about what proportion of pneumonia deaths are caused by different pathogens. As part of this review, we spoke to Dr. Maria Knoll, a researcher at the Bloomberg School of Public Health, Johns Hopkins University. Dr. Knoll told us:

  • The best available estimates of the contribution of SP and Hib to mortality are available at View-Hub, a data visualization platform for vaccine-treatable diseases (created at Johns Hopkins and supported by Gavi and BMGF).
  • The WHO and member states use View-Hub data rather than IHME data to quantify their SP and Hib burdens.
  • She expressed skepticism about the IHME data, in part because it allegedly does not reflect recent vaccination trends, in contrast to View-Hub data.
  • The model used to generate the SP and Hib data in View-Hub was created specifically for SP and Hib, whereas IHME methods are more general and therefore potentially less accurate for specific conditions.

  The GiveWell literature review concludes: “I recommend using View-Hub estimates. I found Dr. Knoll’s arguments persuasive and they conform to my priors about IHME, although I would feel more confident if I had independent confirmation that the WHO and member states use View-Hub data and that they are typically viewed as superior by other researchers.” (Pg. 3 here).

  We also decided to use View-Hub rather than other studies of pneumonia disease etiology (PERCH and GABRIEL), in part because these studies provide estimates of the etiology of severe pneumonia hospitalization (not mortality).

• 98

  See this write-up for further discussion of how we ought to weight the findings of the PERCH study in our estimates of the proportion of pneumonia deaths that are caused by S. pneumoniae and HiB.

• 99

  We exclude rotavirus from this adjustment because rotavirus was only introduced in Nigeria’s childhood vaccination schedule in 2022 (see Gavi, "Dealing with diarrhoea: Nigeria introduces rotavirus vaccine into its immunisation plan," August 30, 2022), and so we would not expect it to have any impact on the GBD mortality estimates. See this row in our supplementary analysis.

• 100

  This calculation uses the following formula:

  • Mortality among unvaccinated children = Mortality overall / (% of unvaccinated children) + (% vaccinated x % relative risk as a result of vaccination).

  As an example, in Bauchi, we estimate that the vaccine-preventable mortality rate among children who have not been vaccinated is 3.0% / ((100% -28%) + (28% x (100% - 53%)) = ~3.5% (see this row).

  This calculation relies on estimates of vaccine coverage for each state in our analysis. For this, we use vaccine coverage data from the Multiple Indicator Cluster Survey (MICS), a household survey which provides data on vaccine coverage at the state level. This is the same survey that we use in our estimates of New Incentives’ impact on vaccination rates (discussed above). Our estimates are calculated as follows in this section:

  • We average vaccination data on the proportion of children who received each vaccine dose from MICS 2021 and MICS 2017 for each state. We use data from both surveys because we want the average vaccination rate across the 5 years preceding the GBD estimates of under-5 mortality (i.e., children who are four years old in 2021 would have been vaccinated in 2017).
    • For PCV, which was not included in MICS 2017, we estimate coverage rates in 2017 based on the average ratio of MICS 2017 to MICS 2021 coverage across all other vaccines.
  • We adjust both sources of data by -1% for self-report bias, using the same method discussed above.
  • MICS provides data on how many children have received each dose of a given vaccine, whereas our vaccine efficacy estimates are for a full course of that vaccine. To translate dose-specific coverage to overall coverage for vaccines with multiple doses (PCV, Penta, and rotavirus vaccines), we came up with rough estimates of the marginal efficacy of each dose (i.e., how much protection is provided by a first dose compared to a second dose, etc.), based on a quick literature review. Our calculations are in this section.

  As an example, this produces the following estimates for vaccine coverage in Bauchi State:

  • PCV: 33%
  • DTP 33%
  • HiB: 33%
  • Measles: 27%
  • BCG vaccine: 45%
  • Rotavirus: 0%

  A weighted average of these vaccines, using each vaccine’s contribution to the primary benefit of New Incentives’ program (reducing under-five mortality), is 28% in Bauchi. See this row in our cost-effectiveness analysis.

• 101

  See this row in our cost-effectiveness analysis.

• 102

  Our estimate of 0.75 for malaria is a rough best guess, based primarily on evidence that malaria control interventions often have bigger impacts on mortality than their impacts on malaria-related mortality alone would imply. See this section of our report on seasonal malaria chemoprevention for more details.

  • We would guess that non-specific effects would be similar for vaccines because the age profile of the beneficiaries is the same (children under 5) and the mechanism seems similar (i.e., beneficial health effects reduce the likelihood of death from other causes).
  • We'd guess these effects might be somewhat lower for vaccines, since all of these vaccines together are preventing a slightly larger share of overall deaths than SMC (and so there are fewer deaths "available" to prevent indirectly). However, vaccines also prevent a wider range of diseases. As a result, an adjustment of 0.75 indirect deaths seemed appropriate for both programs.

• 103

  See our original cost-effectiveness analysis for New Incentives here.

• 104

  Higgins et al. 2016 finds:

  • The BCG vaccine decreases all-cause mortality by 30% in clinical trials: "BCG vaccine was associated with a reduction in all cause mortality: the average relative risks were 0.70 (95% confidence interval 0.49 to 1.01) from five clinical trials." pg. 1.
  • The measles vaccine decreases all-cause mortality 26% in clinical trials: "Receipt of standard titre MCV was associated with a reduction in all cause mortality (relative risks 0.74, [CI] 0.51 to 1.07) from four clinical trials." pg. 1.
  • The DTP vaccine increases all-cause mortality by 38%, though this is from observational studies that the authors rate as having a high risk of bias: "Receipt of DTP (almost always with oral polio vaccine) was associated with a possible increase in all cause mortality on average (relative risk 1.38, [CI] 0.92 to 2.08) from 10 studies at high risk of bias." pg. 1.

• 105

  Lucero et al. 2009 finds that PCV decreases all-cause mortality by 11%: "Pooled vaccine efficacy (VE) for VT‐IPD was 80% (95% confidence interval (CI) 58% to 90%, P < 0.0001); all serotypes‐IPD, 58% (95% CI 29% to 75%, P = 0.001); World Health Organization X‐ray defined pneumonia was 27% (95% CI 15% to 36%, P < 0.0001); clinical pneumonia, 6% (95% CI 2% to 9%, P = 0.0006); and all‐cause mortality, 11% (95% CI ‐1% to 21%, P = 0.08). Analysis involving HIV‐1 positive children had similar findings." pg. 2.

• 106

  0.74 (for measles) x 0.70 (for BCG) x 0.89 (for PCV) = 0.46.

• 107

  This rough calculation is based on our earlier cost-effectiveness analysis implying a 12% reduction in all-cause mortality, and our rough estimate based on Higgins and Lucero that these vaccines might lead to a 60% reduction in all-cause mortality. (60% - 12%) / 12% = ~4.

• 108

  Further reservations include:

  • Wide confidence intervals. The 95% confidence intervals in Higgins et al. 2016 and Lucero et al. 2009 include a risk reduction on all-cause mortality of 0%.
  • Characteristics of infants in trials may not be representative. Two of the BCG trials were limited to a sub-sample of low-birthweight infants, for whom we would guess this effect is larger than for the average infant in North West Nigeria today.
  • There is high uncertainty about the mechanisms for non-specific effects. Our impression is that the mechanisms driving non-specific effects are poorly understood, which leads us to discount them slightly.
  • There is evidence of a negative effect of DTP on mortality. The negative effect of DTP on mortality suggests that we ought to downweight non-specific effects, though this is based on observational studies with high risk of bias.

  Sources:

  • Low birthweight infants: “We considered four results from cohort studies to be at very high risk of bias and excluded them from meta-analyses. The clinical trial results, including two at low risk of bias in low birthweight infants and two in Native American children in the 1930s and 40s, suggested a beneficial effect of BCG on mortality (average relative risk 0.70, 95% confidence interval 0.49 to 1.01).” Higgins et al. 2016, pg. 4.
  • Evidence of a negative effect of DTP on mortality: “Receipt of DTP (almost always with oral polio vaccine) was associated with a possible increase in all cause mortality on average (relative risk 1.38, 0.92 to 2.08) from 10 studies at high risk of bias; this effect seemed stronger in girls than in boys.” Higgins et al. 2016, abstract.

• 109

  These are the specific diseases targeted by the vaccines that Higgins et al. 2016 and Lucero et al. 2009 find lead to large impacts on all-cause mortality (PCV, the measles vaccine, and the BCG vaccine).

• 110

  We estimated in an earlier version of our analysis that the vaccines incentivized by New Incentives’ program would reduce all-cause mortality by approximately 12%, if they only reduced mortality through the diseases they directly avert (see here). 12% x (1 + 0.75) = ~21%.

• 111

  For example, we estimate the probability of death before age 5 from vaccine-preventable diseases ranges from ~2% in Kano to ~3.9% in Zamfara. The range for other states in Nigeria is considerably wider. See this row in our cost-effectiveness analysis.

• 112

  For estimates from IGME, see this page.

• 113

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 114

  Note: We exclude hepatitis B and yellow fever (indirectly incentivized) because IHME estimates show a very low probability of death from these diseases in Nigeria and so we do not think they would make a significant difference to our analysis. See IHME data here.
  We exclude the oral polio vaccine because we expect that there is a very low probability of death from polio in Nigeria today.

  • The last reported case of wild polio virus in Nigeria was in 2016. Global Polio Eradication Initiative, "Polio-free countries".
  • Cases of vaccine-derived polio have been reported in recent years in Nigeria, but we expect that these case numbers are small enough that the average probability of child mortality from polio remains very low.
  • "No cVDPV2 case was reported this week. There have been 16 cVDPV2 cases reported this year and 48 cases in 2022." Global Polio Eradication Initiative, "Where we work: Nigeria", accessed July 26, 2023.

  Instead of explicitly modeling them, we account for the benefits of these vaccines with a rough +5% adjustment elsewhere in our analysis (discussed here).

• 115

  These efficacy numbers correspond to 1 minus the relative risk.

• 116

  IDinsight, New Incentives evaluation, pre-analysis plan, 2019, pg. 2.

• 117

  "Pooled vaccine efficacy (VE) for VT‐IPD was 80% (95% confidence interval (CI) 58% to 90%, P < 0.0001); all serotypes‐IPD, 58% (95% CI 29% to 75%, P = 0.001); World Health Organization X‐ray defined pneumonia was 27% (95% CI 15% to 36%, P < 0.0001); clinical pneumonia, 6% (95% CI 2% to 9%, P = 0.0006); and all‐cause mortality, 11% (95% CI ‐1% to 21%, P = 0.08). Analysis involving HIV‐1 positive children had similar findings." Lucero et al. 2009, pg. 2.

• 118

  World Health Organization, Diphtheria, pertussis, tetanus vaccines information sheet, 2014.

• 119

  Fulton et al. 2016:

  • "Study outcomes were required to be based on the current WHO definition of (1) "typical" pertussis (>14 days of cough with at least one of the following: paroxysmal cough, inspiratory whoop, or posttussive vomiting, in addition to laboratory confirmation); or (2) "severe" pertussis (>21 days of paroxysmal cough with laboratory confirmation of Bordetella pertussis infection, or epidemiological linkage). Studies using less stringent clinical criteria were also included if their laboratory criteria provided a high level of confidence for pertussis infection (eg, positive culture or polymerase chain reaction assay for B. pertussis)." Pg. 1101.
  • "Regional Databases, with no date restrictions, for English-language studies using the following search terms: pertussis, whooping cough, DTwP, DTaP, vaccine, efficacy, morbidity, and mortality." Pg. 1101.
  • "Meta-analysis of the 2 aP vaccine efficacy studies generated a random-effects pooled vaccine efficacy of 84% (95% confidence interval [CI], 81%–87%; P<.00001; Figure 2)." Pg. 1107.
  • “Meta-analysis of 3 wP vaccine effectiveness studies (assessing the Behringwerke, Pasteur/Mérieux, and SmithKline Beecham formulations) yielded an overall wP vaccine effectiveness of 94% (95% CI, 88%–97%) (both I2 = 0%).”
  • We use the effect of the acellular pertussis vaccine in the cost-effectiveness analysis, since it is based on two RCTs and is the effect used in the Lives Saved Tool (LiST); see Table 3, Pg. 1103. We do not use the estimate for whole-cell vaccines (94% efficacy).

  The WHO does not seem to take a stance on using acellular or whole-cell vaccines, though we have not reviewed in depth; see Word Health Organization, Table 2: Summary of WHO Position Papers, Recommended Routine Immunizations for Children, 2023.
  Note: Fulton et al. also reports an estimate for whole cell vaccines of 94% efficacy. We opted to use the more conservative estimate in our analysis, and did not check whether the acellular or whole cell vaccine is used in the Nigerian immunization schedule. We have since learned that many countries use the whole cell vaccine. Because this only makes a small difference to our bottom line, we have not yet checked which type of vaccine is used in Nigeria.
  “Two forms of vaccine are in use, the whole-cell vaccine (wP), and the acellular vaccine (aP). Whole-cell pertussis vaccines were developed first and are suspensions of the entire B. pertussis organism that has been inactivated, usually with formalin…Given that the relative protective efficacy of the best wP and aP vaccines are comparable and the adverse events of both vaccines are relatively minor, wP vaccines remain the vaccine of choice in many developing countries.” World Health Organization, "Pertussis" page.

• 120

  See calculations here.

• 121

  See World Health Organization, Haemophilus influenzae type b (Hib) Vaccination Position Paper – July 2013, pg. 414. “Hib infection and disease start with colonization of the nasopharynx. Following colonization the organism can cause disease either (i) through invasion of the bloodstream with secondary spread to other sites leading to meningitis, pneumonia, and other serious diseases including septic arthritis, osteomyelitis, pericarditis, cellulitis and epiglottitis (referred to collectively as invasive Hib disease) or (ii) through contiguous spread to the paranasal sinuses or the middle ear leading to sinusitis and otitis media.”

• 122

  "Nine randomized studies were included in the analysis. Pooled vaccine efficacy using a fixed effects model against confirmed invasive Hib disease following the 3, 2 and 1 primary dose schedule were 82% [95% confidence interval (CI) 73-87], 79% (95% CI 54–90) and 65% (95% CI 23–84), respectively, and the overall efficacy was 80% (95% CI 72–85)." Thumburu et al. 2015, pg. 31.

• 123

  Note: we only incorporate the impact of the first dose of the measles vaccine in our analysis. This is because the second dose was only added to the Nigerian routine vaccination schedule in 2022. We do not expect the additional dose to have a significant impact on the overall mortality reduction from the program, and so we have not prioritized adding it to our estimates.
  See Figure 1, Sudfeld, Navar, and Halsey 2010, p. I50.

  The same meta-analysis estimates efficacy for two doses of ~98%. This is based on serological and observational studies rather than randomized controlled trials.

  “The effect of a second dose of measles vaccine on measles disease or mortality compared with no vaccination has not been evaluated on individual children in prospective randomized studies as this type of trial would be unethical. Therefore, the best estimate of the effect of a two-dose measles vaccine schedule on measles mortality must be extrapolated from serology data, studies looking at effectiveness of two dose vs one-dose measles vaccination, and observational studies. Caution should be taken when using serology data to estimate the impact on mortality. A recent WHO review of serology studies determined that a median 97% [inter-quartile range (IQR) 87–100%] of children that failed to seroconvert to first dose measles vaccine developed immunity after a second dose.64 If 85% efficacy is assumed for single dose measles vaccine, these serology results would correlate to an efficacy of 99.6% for two dose measles vaccine with a range of 98.1–100% based on the IQR of the review. In addition, the effectiveness of two doses of measles vaccine will vary by setting based on the age of vaccination. Epidemiologic studies comparing the effectiveness of early two dose vaccination vs single dose have found varying results in developing country settings; a study in Niger found two doses (first dose at 6–8 months and second at 9 months) was 23% less effective than single dose whereas studies in India (first dose at 9–12 months and second at 15–18 months) and Guinea Bissau (first dose at 6–8 months and second at 9–12 months) determined two doses of vaccine were respectively 83 and 90% more effective than one dose of measles vaccine.66–69 In order to produce a conservative estimate of the efficacy of two dose measles vaccine per LiST rules, we felt an input of 98% based on the lower quartile of the WHO two dose measles vaccine serology review was reasonable.” Sudfeld, Navar, and Halsey 2010, p. I52.

• 124

  "Diseases prevented: Disseminated disease and meningitis caused by M. tuberculosis." Feikin et al. 2016, Annex 10A, p. 4

• 125

  Mangtani et al. 2014, Figure 5, pg. 478.

• 126

  "Two forms of TB are life threatening: disseminated or miliar disease, and meningitis." Plotkin et al. 2018, pg. 1098.

  Note: Our understanding is that BCG vaccine efficacy wanes significantly over time and offers lower protection to older children and adults. Our understanding is that it still provides significant protection against the most severe forms of TB when given in infancy, but we have not dug into this literature in detail.

  “BCG is usually administered only in infants, immediately after birth, in countries that have a high incidence of TB. The vaccine then produces an early immune response that has been demonstrated to protect children against severe forms of TB. In particular, BCG protects very well against the development of disseminated forms of TB. Usually TB occurs in the lungs, but the bacteria can also be found in other parts of the body – this is called dissemination. In children, the bacteria can be found in the brain – this is called TB meningitis. The BCG vaccine is very effective at protecting against TB meningitis and is a great example of how vaccines can be of huge benefit.
  However, this immunity usually wanes in adolescence and thereafter. Protection by BCG in adults is highly variable – ranging from 0% to 80% depending on the country and environment. The reasons for this remain a mystery and much effort has been placed recently in developing biomarkers that will identify which new vaccines will eventually yield long-lasting immunity. Biomarkers are signals that one can pick up in blood or other clinical specimens that give a predictive sense of whether a vaccine is going to work. If a certain set of signals in blood predicts good protection, we can check if a new vaccine also induces the same set of signals.” Gavi, “TB prevention has relied on the same vaccine for 100 years. It’s time for innovation," 2021.

• 127

  "Diarrhea remains the second leading cause of death around the world for children under 5 years of age. Because the majority of diarrhea deaths occur in the low and middle-income countries, the etiologic agents responsible for diarrhea deaths among young children are unknown. Using hospitalization data as a best estimate of severe diarrheal disease and a proxy for diarrhea mortality, it has been estimated that rotavirus may be responsible for up to 39% of child deaths, the majority of which occur in low and middle income countries." Walker and Black 2011, pg. 1.

• 128

  See Lamberti et al. 2016, Table 1.

• 129

  “The Lives Saved Tool is a mathematical modeling tool which allows users to estimate the impact of coverage change on mortality in low and middle income countries.” LiST, Home page, accessed July 26, 2023. The tool provides meta-analyses as sources for its estimates.
  To obtain the vaccine efficacy estimates above, we accessed LiST’s ‘Explore Data’ tool (available here), and looked at its sources for estimates of vaccine efficacy for children under-five. We used the efficacy estimates cited in its sources for each disease available (see the footnotes for the table above where we cite the meta-analyses available via the LiST tool).

• 130

  An "intention-to-treat" (ITT) analysis includes all RCT subjects originally allocated to receive the intervention, whereas a "per-protocol" analysis is a comparison of treatment groups that includes only subjects who in fact completed the intervention.
  The meta-analyses referenced in LiST include a combination of ITT and per-protocol effects. Where possible, we've used ITT effects.

• 131

  See this section of our analysis.

• 132

  The effect size for measles is based on a combination of RCTs and observational studies, but there is other evidence consistent with the effect size reported from serological studies of measles (see sources below). Our impression (which we have not investigated in detail) is also that measles has been reduced in many countries that have had good vaccination coverage.

  A Cochrane review (Di Pietrantonj et al. 2020) finds an effect of 95% for a single dose at age 9 months to 15 years, though this is also based on cohort studies and includes a much broader age range. We therefore do not put any weight on it in our analysis.

  "The proportion of children who develop protective antibody levels following measles vaccination depends on the presence of inhibitory maternal antibodies and the immunological maturity of the vaccine recipient, as well as on the dose and strain of the vaccine virus as described in detail below. In general, some 85−90% of children develop protective antibody levels when given one dose of MCV at 9 months of age, and 90−95% respond when first vaccinated at 12 months of age. Median MCV effectiveness (i.e. protection from disease) following a single dose of MCV administered at 9−11 months of age was 84% (interquartile range [IQR] 72–95%) across several studies, and increased to 92.5% (IQR 84.8–97%) among children first vaccinated at 12 months or older. Thus most, but not all, children are protected following a single dose of MCV." World Health Organization, Immunological Basis for Immunization Series, Measles, 2020, pg. 12.

• 133

  The study we use for HiB efficacy (Thumburu et al. 2015) uses ITT effects. However, other studies use a combination of ITT and per-protocol effects. The appropriate adjustment would also take into account efficacy achieved through partial vaccination.

• 134

  "Vaccination with the protein-polysaccharide conjugate vaccine, PCV7, has significantly reduced the burden of pneumococcal disease in populations where it is in widespread use and has had an important public health benefit. This vaccine targets only 7 of the more than 92 pneumococcal serotypes, and there have been concerns that the non-vaccine serotypes (NVTs) could increase in prevalence and reduce the benefits of vaccination." Weinberger, Malley, and Lipsitch 2011, pg. 1.

• 135

  For example, we estimate that the diseases targeted by PCV are responsible for 36% of vaccine-preventable deaths among children under age five in Bauchi, compared to 5% for the measles vaccine. See this section of our analysis.

• 136

  Note that this figure does not appear directly in our cost-effectiveness analysis. In our cost-effectiveness analysis, we adjust each vaccine’s efficacy to reflect lower vaccine efficacy in Nigeria (more) and imperfect coverage during the underlying trials (more) in this sheet. We then aggregate them here in our analysis.

  We present the aggregated estimate of unadjusted vaccine efficacy here for simplicity. This change to the order of operations does not make a quantitative difference.

  Note: The variation between states is because different diseases are responsible for varying shares of mortality in each state.

• 137

  See this sheet in our cost-effectiveness analysis for these estimates.

• 138

  The adjustments we apply are:

  • Removing deaths that we think are likely to occur before each vaccine is administered (more).
  • An adjustment for disease etiology (cause of disease) (more).
  • An adjustment for unvaccinated children having higher mortality (more).

  We do not apply the adjustment for all-cause mortality to these estimates because we are unsure how vaccine nonspecific effects vary across vaccines. This is equivalent to assuming nonspecific deaths are the same across vaccines.

• 139

  See this section of our cost-effectiveness analysis.

• 140

  For example, Thumburu et al. 2015 includes studies from Finland, Chile, Gambia, the US, and Indonesia, among others.

• 141

  One piece of evidence for this is that the meta-analysis we rely on to estimate rotavirus efficacy finds efficacy varies widely by region. Efficacy in sub-Saharan Africa (the estimate we use in our analysis) is significantly lower than efficacy in wealthy countries. “Efficacy against severe rotavirus diarrhea ranged from 90.6% [95% confidence interval (CI): 82.3–95.0] in the developed region to 88.4% (95% CI: 67.1–95.9) in Eastern/Southeastern Asia, 79.6% (95% CI: 71.3–85.5) in Latin America and the Caribbean, 50.0% (95% CI: 34.4–61.9) in Southern Asia and 46.1% (95% CI: 29.1–59.1) in sub-Saharan Africa”, Lamberti et al 2016, pg 1.

• 142

  See this row in our cost-effectiveness analysis.

• 143

  This interval was selected by GiveWell staff members familiar with our New Incentives analysis based on their subjective judgment. It does not appear in our cost-effectiveness analysis.

• 144

  We assign the following weights based on our subjective impression of how informative each source is about the likely efficacy of the measles vaccine in Nigeria today:

  • 10% weight to the main LiST measles vaccine efficacy estimate.
  • 20% weight to the results from the IDinsight biomarker pilot.
  • 35% weight to each of Fowotade et al. 2015 and Uzicanin and Zimmerman 2011.

  The weights are in this section of our cost-effectiveness analysis.

• 145

  This figure is a weighted average of the -26% (measles) and -19% (non-measles) adjustments, weighted by each vaccine’s contribution to reducing deaths among children under five. The adjustment varies slightly between states because we think that each vaccine contributes a different amount to the program benefit in each state. See this row for our calculations.

• 146

  See this row in our cost-effectiveness analysis.

• 147

  We have considered two reasons why the biomarker results could be measles-specific, although we haven’t deeply investigated either:

  1. Children in Nigeria could receive the measles vaccine too young. The first dose of measles is given at 9 months in the Nigeria vaccination schedule, compared to 12 - 15 months (for the MMR vaccine) in many high-income countries. This could be too early for them to develop antibodies in response to vaccination.
  2. No measles 2 vaccine: At the time the biomarker pilots were conducted, Nigeria only offered one dose of the measles vaccine (compared to two doses in many high-income countries). We would guess that this is insufficient to induce full immunity for some children.

• 148

  See here.

• 149

  We estimate vaccine efficacy as a function of (i) the test sensitivity (the proportion of children with antibodies correctly identified by the test), (ii) the proportion of children falsely reported to be vaccinated, (iii) the proportion of children who were immune before the test, because of a previous measles infection, and (iv) the proportion of children testing positive.

  Our calculations are in this supplementary spreadsheet. We use the following assumptions:

  • Test sensitivity: we use a reported figure that the test sensitivity is 86%. We do not have permission to share the source for this estimate.
  • False report: we roughly guess that 15% of children are falsely reported to be vaccinated when they were not.
  • Prior immunity: we roughly estimate that 5% of children had a prior measles infection. This is a rough estimate, based on a number of sources, discussed here.
  • Rate testing positive: we use the pilot’s finding that 22% of children tested positive.

  These assumptions imply a vaccine efficacy of 26% (see calculation in this row).
  Note: Our analysis assumes that vaccine efficacy is linearly correlated with the proportion of children developing antibodies. We have not deeply investigated this assumption.

• 150

  "This study was designed to assess the seroconversion rate of measles vaccine among infants receiving measles immunization in Ilorin, Nigeria. . . . Only 286 (71.5%) of the vaccines returned to give post-vaccination samples. All the infants screened had low pre-vaccination measles antibody titers. Thirty one (8.0%) of the infants had measles prior to vaccination. The seroconversion pattern showed that 196 (68.6%) of the infants developed protective antibody titers." Fowotade et al. 2015, Abstract

• 151

  "This study was designed to assess the seroconversion rate of measles vaccine among infants receiving measles immunization in Ilorin, Nigeria. . . . Only 286 (71.5%) of the vaccines returned to give post-vaccination samples. All the infants screened had low pre-vaccination measles antibody titers. Thirty one (8.0%) of the infants had measles prior to vaccination. The seroconversion pattern showed that 196 (68.6%) of the infants developed protective antibody titers." Fowotade et al. 2015, Abstract

• 152

  "The loss in vaccine virus titers in our centre may be attributed partly to repeated thawing and freezing due to erratic power supply and the absence of standby power generating set serving the immunization clinic where the vaccines are stored. Another contributory factor is the absence of a refrigerator thermometer to ensure that the correct storage temperature is maintained. The low potency found has been the usual trend in potency studies carried out by various researchers in Nigeria. This finding is however different from vaccine studies carried out in other parts of the world. Techatharyat et al. in Thailand and Saha et al. in India reported measles vaccine potency test results of 100.0% and 95.0% respectively. Other adverse factors such as poor handling by vaccinators, existence of chains of salesmen, lack of good storage system for vaccines and difficulty in maintaining a cold chain system have also been suggested to be responsible for the loss in potency of vaccines used in Nigeria and other African countries." Fowotade et al. 2015, Discussion.

• 153

  "These vaccines were obtained from United Nations International Children’s Emergency Fund (UNICEF), through the State EPI Unit and were stored in the freezer until required for immunization. These lyophilized Ruvax vaccines were maintained in cold chains and reconstituted each time per vial according to manufacturer’s instruction." Fowotade et al. 2015, Materials and Methods.

• 154

  See this row in our cost-effectiveness analysis.

• 155
  • “One hundred and twenty-two out of 130 children (93.9%) who had received DPT had protective levels of anti-tetanus IgG compared to 278 out of 288 children (96.5%) who had received the pentavalent vaccine… DPT and pentavalent vaccines are equally effective in inducing protective levels of anti-tetanus IgG in children”, Uket et al. 2018, abstract.
  • “Four hundred and eighteen children participated in the study. The mean IgG antibody level was 1.021 ± 0.9 IU/ml. Four hundred children (95.7%) had protective levels of antibodies”, Ekanem, Uket, and Okpara 2018, abstract.

• 156
  • “The prevalence of BCG scar was 79.5% among the male infants and 84.7% among the female infants, while the overall prevalence of BCG scar was 81.5%”, Gambo et al. 2014, results.
  • “Two hundred and six subjects (96.3%) had a post-vaccination BCG scar”, Atimati and Osarogiagbon 2014, abstract.
  • “84.2% had physical evidence of BCG inoculation”, Odujinrin and Ogunmekan 1992, abstract.

• 157

  “Only one‑third (49/138) of those vaccinated had identifiable BCG scars”, Orogade et al. 2013, abstract.

• 158
  • “Of the vaccinated children, 157 (57.5%) developed protective measles virus HI antibody, which is not enough to maintain protective herd immunity”, Onoja and Adeniji 2013, abstract
  • "A total of 150 (52.8%) of the 284 children who received measles vaccine returned for post-vaccination screening, and of these, 82 (54.7%) seroconverted (titres, 10-320) following vaccination; 68 (45.3%) did not seroconvert (Table 1)." Adu et al. 1992, pg. 458.
  • “The seroconversion pattern showed that 51(60%) had potent antibody titres ranging from 1:40 to 1:1280, while the remaining 34 (40%) had a low antibody titres between < 1:20 and 1:20”, Omilabu et al. 1999, abstract.

• 159

  Many have small sample sizes and many are published in less well-recognized journals.

• 160
  • Uzicanin and Zimmerman 2011 find a median vaccine efficacy for individuals who were 9-11 months old at age of first dose of 77%, with vaccine efficacy ranging from 73% in the WHO Africa region to 96% in the WHO European region (Table 2, pg. 145).
  • "Results of this literature review suggest that the VE of MCV1 administered at 9–11 months of age is generally lower than 85%, which is the usual expected rate of immune response after vaccination at that age." Uzicanin and Zimmerman 2011, pg. 135.
  • "Generally, the reasons related to low VE estimates can be grouped into 3 broad categories, including (1) issues related to study methods; (2) program-related factors, such as appropriate vaccine storage, handling, and administration; and (3) host-related factors, most notably, age at vaccination." Uzicanin and Zimmerman 2011, pg. 144.

• 161

  BCG vaccine: we reviewed Pimpin et al. 2013, which notes that the BCG vaccine is less effective closer to the equator. It is not clear to us whether this result applies to BCG vaccine administered at birth and to the BCG vaccine's effects on fatal forms of tuberculosis.

  “In general, the protective effect of BCG vaccination was either absent or low in studies conducted close to the equator, whereas there was reasonably consistent evidence of good protection observed in studies conducted at latitudes exceeding 40°. Relatively high protection was observed in studies (Saskatchewan Infants and MRC) conducted above 50° latitude: rate ratio 0.22 (95% CI 0.16 to 0.30), corresponding to a VE of 78% (95% CI 70% to 84%). Latitude explained a substantial amount of the between-study variation in the protective effect of BCG vaccination." Pimpin et al. 2013, p. 24.

  "Because of the evidence that BCG protects against miliary and meningeal tuberculosis, in developing countries BCG vaccination is recommended at birth (or first contact with health services), taking into account HIV status. Our systematic review suggests that BCG also confers protection against pulmonary disease, the greatest burden from tuberculosis, when administered both in infancy and at school age, providing that children are not already infected with M tuberculosis or sensitised to other mycobacterial infections. Protection against pulmonary disease was seen in the Bombay Infants trial suggesting that, even close to the equator, if BCG is administered prior to exposure to tuberculosis and environmental mycobacteria it can provide significant protection. Further evidence of protection in populations close to the equator from BCG given before infection would strengthen these findings." Mangtani et al. 2014, p. 479.

  PCV: The two African trials in a meta-analysis of the effects of PCV we reviewed, Lucero et. al. 2009, have smaller effects than trials in other settings, though the difference does not appear to be statistically significant, and it is not clear what is causing this difference.

  • Cutts et al. 2005 in Gambia: risk ratio of 0.57 [0.38, 0.85].
  • Klugman et al. 2003 in South Africa: risk ratio of 0.58 [0.26, 1.31].

  The overall risk ratio is 0.42 [95% CI 0.25, 0.71]. Effects are lower in the American Indian trial (O'Brien 2003) (0.48), Gambia (Cutts 2005), South Africa (Klugman 2003), and the Philippines (Lucero 2009) (1.00), and highest in Finland (Eskola 2001 and Klipi 2003) and the US (Black 2000) (0.11-0.33). (See Figure 4, p. 15). It is not clear to us why these effects vary. The commentary does not explicitly mention changes varying by distance from equator or lower impact in low- and middle-income countries.

• 162

  The 95% guess appears here in our cost-effectiveness analysis.

• 163

  This appears as a 100% ratio between vaccine-preventable disease and vaccine-preventable mortality in our cost-effectiveness analysis. See this row.

• 164

  See this section in our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 165

  This excludes our supplemental adjustments discussed in sections 4.5 and 4.6. See this row in our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 166

  For ease of modeling, we quantify our uncertainty ranges for benefits other than mortalities averted among people under age 5 as percentage adjustments applied to other modeled benefits of the program. For Bauchi:

  • Our 25th to 75th percentile range for income effects is an adjustment ranging from +10% to +44% (with the percentage adjustment applied to the modeled benefits of averted mortality for children under five). See this row in the “Sensitivity analysis” section of our cost-effectiveness analysis.
  • This roughly translates to between 7% and 24% of the total modeled benefits of the program, as shown in the table above.
  • This conversion is imperfect because varying the proportion of the program's total impact that is contributed by one type of benefit will also affect the proportion of impact contributed by other types of benefits, but we think the values presented in the table are a reasonably good approximation of our uncertainty level for this parameter.

• 167

  Note: We use a similar method to the one described above to estimate vaccine-preventable mortality among unvaccinated children (as opposed to all children). This method incorporates estimates of vaccine coverage among this age group during the time period in which this age group would have been scheduled to receive child vaccinations. To roughly estimate this, we assume that people aged 5 - 14 were 50% less likely to have been vaccinated than children under age 5 today, 15 - 49 year olds were 80% less likely to have been vaccinated, and 50 - 74 year olds were 90% less likely to have been vaccinated. See the following rows in our cost-effectiveness analysis:

  • 5 - 14 year olds
  • 15 - 49 year olds
  • 50 - 74 year olds

• 168

  A discount rate of 0.5% implies that we discount the value of averting a death by 0.5% if it happens one year from now, compared to if it happens now. The discount rate compounds, so deaths averted far in the future are valued less than deaths averted in the near future. See this document for more details on discount rates.

• 169

  Our method for 15 - 49 year olds and 50 - 74 year olds is the same, but we apply larger discounts for reduced mortality and vaccine efficacy in the future. See these sections of our cost-effectiveness analysis for our specific assumptions for each age group.

• 170

  To reach this figure, we also apply our discount rate. This slightly reduces the effective number of deaths we think will be averted in the future, because each death counts for less than if it was averted now. See this row of our cost-effectiveness analysis for our calculation.

• 171

  This calculation takes into account both the numbers of deaths averted in each age group and GiveWell’s moral weights for averting a death in each age group (we assign more weight to averting the deaths of children than adults). See this row in our cost-effectiveness analysis for our calculations.

• 172

  See this row in our cost-effectiveness analysis. Note that this estimate doesn’t account for the benefits we incorporate as rough supplemental adjustments, and so can’t be interpreted as an estimate of the total benefits of the program.

• 173

  For ease of modeling, we quantify our uncertainty ranges for benefits other than mortalities averted among people under age 5 as percentage adjustments applied to other modeled benefits of the program. For Bauchi:

  • Our 25th to 75th percentile range for income effects is an adjustment ranging from +5% to +60% (with the percentage adjustment applied to the total modeled benefits of averted mortality). See this row in the “Sensitivity analysis” section of our cost-effectiveness analysis.
  • This roughly translates to between 5% and 37% of the total modeled benefits of the program, as shown in the table above.
  • This conversion is imperfect because varying the proportion of the program's total impact that is contributed by one type of benefit will also affect the proportion of impact contributed by other types of benefits, but we think the values presented in the table are a reasonably good approximation of our uncertainty level for this parameter.

• 174

  Our estimate of the development effects of deworming programs is based on long-run follow-ups to the experiment described in Miguel and Kremer 2004. See our report on mass deworming programs for our full assessment of the study. For malaria programs, we rely on the findings of two natural experiments: Bleakley 2010 and Cutler 2010. See this section of our report on mass distribution of insecticide-treated nets (ITNs) for a detailed discussion of these studies.

• 175

  “This study uses the malaria-eradication campaigns in the United States (circa 1920), and in Brazil, Colombia and Mexico (circa 1955) to measure how much childhood exposure to malaria depresses labor productivity. The campaigns began because of advances in health technology, which mitigates concerns about reverse causality. Malarious areas saw large drops in the disease thereafter. Relative to non-malarious areas, cohorts born after eradication had higher income as adults than the preceding generation. These cross-cohort changes coincided with childhood exposure to the campaigns rather than to pre-existing trends. Estimates suggest a substantial, though not predominant, role for malaria in explaining cross-region differences in income.” Bleakley 2010, abstract.

  “We examine the effects of exposure to malaria in early childhood on educational attainment and economic status in adulthood by exploiting geographic variation in malaria prevalence in India prior to a nationwide eradication program in the 1950s. We find that the program led to modest increases in household per capita consumption for prime age men, and the effects for men are larger than those for women in most specifications. We find no evidence of increased educational attainment for men and mixed evidence for women.” Cutler 2010, abstract.

• 176

  See this section of our report on insecticide-treated nets for more detail on these estimates.

• 177

  See this row in our cost-effectiveness analysis.

• 178

  See this row in our cost-effectiveness analysis.

• 179

  For the income increase per household member, see here in our cost-effectiveness analysis. For the proportion of benefits this makes up, see this row. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 180

  For ease of modeling, we quantify our uncertainty ranges for benefits other than mortalities averted among people under age 5 as percentage adjustments applied to other modeled benefits of the program. For Bauchi:

  • Our 25th to 75th percentile range for income effects is an adjustment ranging from +2% to +5% (with the percentage adjustment applied to all the other modeled benefits of the program, including averted mortality and long-term income effects). See this row in the “Sensitivity analysis” section of our cost-effectiveness analysis.
  • This roughly translates to between 2% and 5% of the total modeled benefits of the program, as shown in the table above.
  • This conversion is imperfect because varying the proportion of the program's total impact that is contributed by one type of benefit will also affect the proportion of impact contributed by other types of benefits, but we think the values presented in the table are a reasonably good approximation of our uncertainty level for this parameter.

• 181

  To compare cost-effectiveness across different programs, we use "moral weights" to quantify the benefits of different program impacts (e.g., increased income versus reduced deaths). We benchmark the value of each benefit to a unit value of 1, which we define as the value of doubling someone’s consumption for one year. For more on our approach to moral weights, see this document.

  We model units of value for income increases in terms of the natural log of consumption. The logarithmic model captures the idea that money has diminishing value as you get more and more of it. For example, our model considers a 50% increase in income as a little better than 50% as good as an 100% increase in income. Using this model, a $1.76 increase in income from a $286 baseline equates to 0.01 units of value. See this row in our cost-effectiveness model.

• 182

  See this row in our cost-effectiveness analysis.

• 183

  Since we initially conducted our analysis, New Incentives has made a number of changes to its incentive schedule as follows:

  • New Incentives originally added a 500 naira incentive for measles 2 (which had not yet been introduced into Nigeria's routine vaccination schedule at the time of the RCT). This brought the total incentive across all visits to 4,500 naira. In early 2023, New Incentives increased the incentive for measles 2 to 1,000 naira. This brought the total incentive across all visits to 5,000 naira.
  • In July 2023, New Incentives decided to change its schedule to offer 1,000 naira per visit.

  New Incentives, January 14, 2022; December 16, 2022; July 7, 2023 Program Updates (unpublished).

• 184

  See this row in our cost-effectiveness analysis.

• 185

  See IDinsight, Impact evaluation of New Incentives in North West States of Zamfara and Katsina: Report on June field activities, 2017.

• 186

  “The 97% of clinic records are from settlements less than 250 naira by motorbike from the clinic. Visits from the outlier settlements do not seem to be associated with New Incentive’s program. All settlements with reported travel cost over 250 naira were from PHC Damri. IDinsight, Impact evaluation of New Incentives in North West States of Zamfara and Katsina: Report on June field activities, 2017, pg. 7.

• 187

  See this spreadsheet for our calculations (underlying data from StatCompiler available here). The states we include in this average are: Zamfara, Katsina, Jigawa, Sokoto, Bauchi, Kaduna, Gombe, Kano, and Kebbi. New Incentives will also consider expanding to Adamawa and Taraba, but as of October 2023 New Incentives had not yet identified cost-effective areas to operate in in either of those states.

  For simplicity and because these estimates make such a small difference to our bottom line, we use a cross-state average rather than a state-specific estimate.

  See our May 2023 grant page for further details on New Incentives’ most recent expansion.

• 188

  The calculation behind this value can be found in the "Cash" section of the "Parameters" tab at this link. Our analysis is based on Haushofer and Shapiro 2013, an RCT of GiveDirectly’s program. In Haushofer and Shapiro 2013, non-durable expenditure in the control group is measured as USD 157.40 PPP per month (p. 49). We multiply this value by 12 to put it in annual terms, converted to nominal USD, and divided by the average household size.

• 189

  New Incentives’ monitoring data is available here. In 2022, New Incentives estimated that retention rates varied between 70% and 95%, depending on the specific vaccine. These rates were similar to those found in the RCT. See this column.

• 190

  The World Bank reports estimates in 2022 that Nigeria’s GDP per capita was $2,184 and Kenya’s was $2,099 (estimates available here).

• 191

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 192

  This is because we think they would be challenging to model effectively or their likely effect is small enough that we do not expect explicit modeling to be worth the time investment required.

• 193

  See this row of our cost-effectiveness analysis.

• 194

  In our cost-effectiveness analysis, this appears as (mortality and morbidity effects). We have grouped them here for simplicity. Because of rounding, these sum to 4% (3.6% + 0.7%).

• 195

  We add each percentage adjustment together to obtain our total adjustment.

• 196

  Step one: Estimate the likely size of each effect (a rough subjective best guess).

  • Step two: Evaluate each effect using three criteria, each assigned a score of up to 3:
    • Can it be objectively justified (i.e., is there direct evidence for the effect)? (Where 0 is little evidence and 3 is strong direct evidence).
    • How easy would it be to model? (Where 0 is impossible and 3 is simple).
    • Consistency - is this effect included in our cost-effectiveness analysis for other programs? (Where 1 is no, 2 is partially, and 3 is yes).
  • Step three: Convert the total three criteria score out of 9 into a percentage (e.g., a total score of 5 converts to a percentage of 50%).
  • Step four: Weight our estimate of each effect by this percentage to produce our overall adjustment (e.g., an effect size guess of 15% multiplied by a three criteria score of 60% = an adjustment of 9% (15% * 60% = 9%)).

  We use the method described for all supplemental intervention-level adjustments in our cost-effectiveness analysis for this program, with the exception of our estimate of costs saved from averted malaria. We account for these savings using an explicit model, discussed in more detail here.

• 197

  See this spreadsheet for more details on our reasoning for each adjustment.

• 198

  "Vaccination confers both direct and indirect effects. The direct effect implies protection against disease in vaccinated individuals. Indirect protection is when susceptible individuals avoid infection because the people who surround them are immunized. The magnitude of indirect effects is a function of transmissibility of the infectious agent, population mixing patterns, distribution of vaccine, and distribution of immunity in the population. 'Herd immunity' refers to population-scale immunity." Nymark et al. 2017.

• 199

  “Herd immunity is usually achieved by interrupting the transmission of the organism by preventing infections in immunized individuals. Thus, there is less opportunity for the unimmunized individual to be exposed to the organism.” Pollard et al. 2015.

• 200

  We believe the excluded morbidity effects are likely to be small (no more than 5% of the total DALY burden). This is based on the finding that only a small percent of DALYs is due to years lost to disability (YLDs) for children under 5 (less than 2% of DALYs across all diseases; see here). Excluding these morbidity effects seems consistent with our decision to exclude morbidity from our cost-effectiveness analyses of other interventions primarily aimed at reducing mortality. See here for our DALY analysis.
  Definitions:

  • DALY stands for “disability-adjusted life year.” The Global Burden of Disease Project defines it as: “a universal metric that allows researchers and policymakers to compare very different populations and health conditions across time. DALYs equal the sum of years of life lost (YLLs) and years lived with disability (YLDs). One DALY equals one lost year of healthy life.”
  • YLDs (years lived with disability) are years lived in less than ideal health. YLLs are measured by multiplying the prevalence of a health condition by the “disability weight” (how bad it is) of that condition. We use YLDs as a measure of morbidity.
  • YLLs (years of life lost) are years lost due to premature mortality. For example, if the longest life expectancy for men in a given country is 75, but a man dies of cancer at 65, this would be 10 years of life lost due to cancer. We use YLLs as a measure of mortality.

  Definitions are from the Institute for Health Metrics and Evaluation, "Global Burden of Disease (GBD)".

• 201

  Note: in our cost-effectiveness analysis this appears as two rows: (1) mortality (+4%), (2) morbidity (+1%). Because of rounding, together these sum to 4% (3.6% + 0.7%). See here. We have grouped them here for simplicity.

• 202

  Yellow fever and acute hepatitis B have a much lower DALY burden for children under five (53 and 69 per 100,000, respectively), relative to other diseases prevented by vaccines (see this column of our supplementary spreadsheet).
  For polio, meningitis A, mumps, rubella, and varicella, our impression is that the number of cases are low, but we haven’t attempted to quantify the burden. In our cost-effectiveness analysis, we include a rough guess for additional benefits from effects on these diseases.

• 203

  See this row in our cost-effectiveness analysis.

• 204

  As of January 2024, these are the Against Malaria Foundation (AMF), Malaria Consortium, Helen Keller International, and New Incentives. See our "Top charities" page here.

• 205

  Our model estimates a 14% upward adjustment for long-lasting insecticidal nets (here), a 20% upward adjustment for seasonal malaria chemoprevention (here) and a 21% upward adjustment for vitamin A supplementation (here). We did not model these benefits for New Incentives, and so the New Incentives figure is benchmarked to the other three programs.

• 206

  The RCT found that New Incentives’ program increased the timeliness of the measles vaccine, also increased the timeliness of the Penta 1 vaccine (on some but not all measures), and led to a decrease in timeliness for the BCG vaccine, although this was small and not statistically significant. See IDinsight, Impact evaluation of New Incentives, final report, 2020, Table 8, pg. 30.

  “The incentive had the largest impact on the timeliness of the Measles 1 vaccine. Children in the treatment group who received the Measles 1 vaccine were 33 percentage points (95% CI: 28, 38; p-value < 0.001) more likely to have received the vaccination within one month of the recommended age (9 months) compared to those who received the Measles1 vaccine in the control group. Measles 1 vaccinations were also more likely to be timely in the treatment group than in the control group when using a cutoff of within 2 weeks of the recommended age." IDinsight, Impact evaluation of New Incentives, final report, 2020, pg. 30.

• 207

  The measles vaccine is recommended at 9 months in high-transmission settings to balance higher efficacy with age (which suggests vaccinating later) with high risk of measles for those unvaccinated (which suggests vaccinating sooner).

  “The age at vaccination is one of the most important determinants of the immune response to MCV, with older infants usually showing better responses than younger infants (Figure 4). The optimal age for measles vaccination is determined by consideration of the age-dependent increase in seroconversion rates following measles vaccination and the average age of infection. In regions of intense MV transmission, the average age of infection is low and the optimal strategy is to vaccinate against measles at as young an age as possible (usually 9 months of age, Figure 4). By contrast, in settings where MV transmission has been reduced, the age of administration of the first dose of MCV can be increased to 12 months or older. Antibody responses to MCV increase with age up to around 15 months because of the declining levels of inhibitory maternal antibodies and decreasing immaturity of the immune system. This immaturity of the immune system in neonates and very young infants includes a limited B-cell repertoire and inefficient mechanisms of antigen presentation and T-lymphocyte help. The recommended age at vaccination must balance the risk of primary vaccine failure, which decreases with age, against the risk of MV infection prior to vaccination, which increases with age.” World Health Organization, Immunological basis for immunization series, measles, 2020, pg.13.

  As a result, vaccinating too soon may risk lowering vaccine efficacy while vaccinating too late may risk measles infection. We have not sought to vet this claim or quantify this effect, and we include a small adjustment for improved timeliness in our cost-effectiveness analysis.

• 208

  See this spreadsheet, "Investment of income increases" rows for each intervention, for details on the specific values used for each program. Note that we have not recently updated these adjustments, and so the variation in values between programs may no longer be accurate. We have not prioritized updating this because it makes a very small difference to our cost-effectiveness estimates.

• 209

  This is in line with our method for modeling the impact of the program across different states with different levels of baseline vaccination rates. We assume that states with higher baseline vaccination rates will see a smaller benefit from the program. More details above.

• 210

  This calculation is based on a rough internal analysis, and is uncertain because the impact of higher vaccine coverage on cost-effectiveness varies by state, depending on that state’s level of baseline vaccine coverage. Nonetheless, we think 5 percentage points is a reasonable approximation of the increase in baseline coverage implied by our adjustment.

  We use a three-year window because this is roughly the period between New Incentives’ baseline coverage surveys being conducted (2021 - 2023) and the midpoint of grants we’d expect to make at the time we conducted our internal analysis.

• 211
  • “Most studies indicate that the degree of mucosal immunity in the intestine is significantly less than that provided by OPV, although this difference may be less pronounced in the pharyngeal mucosal lining.” World Health Organization, Standards and specifications, "Poliomyelitis".
  • “While IPV elicits a much weaker mucosal immune response than OPV,5 and is thus less effective at averting transmission, it is very protective against disease.” Alfaro-Murillo et al. 2020. Note that we have not reviewed the evidence for this in detail.

• 212
  • “In very rare cases, the administration of OPV results in vaccine-associated paralysis associated with a reversion of the vaccine strains to the more neurovirulent profile of wild poliovirus.” World Health Organization, Standards and specifications, "Poliomyelitis".
  • A write up on this issue is available here.
  • New Incentives may offset this risk through its effect on inactivated polio vaccine (IPV), which may lower risk of outbreak from oral polio vaccines: "Circulating VDPVs occur when routine or supplementary immunization activities (SIAs) are poorly conducted and a population is left susceptible to poliovirus, whether from vaccine-derived or wild poliovirus. Hence, the problem is not with the vaccine itself, but low vaccination coverage. If a population is fully immunized, they will be protected against both vaccine-derived and wild polioviruses." WHO, Poliomyelitis: Vaccine derived polio, 2017.

• 213

  “Vaccine-associated paralytic poliomyelitis (VAPP) is a rare adverse event associated with oral poliovirus vaccine (OPV). This review summarizes the epidemiology and provides a global burden estimate… Using all risk estimates, VAPP risk was 4.7 cases per million births (range, 2.4–9.7), leading to a global annual burden estimate of 498 cases (range, 255–1018). If the analysis is limited to estimates from countries that currently use OPV, the VAPP risk is 3.8 cases per million births (range, 2.9–4.7) and a burden of 399 cases (range, 306–490).” Platt et al. 2014. Note that we have not reviewed this evidence in detail.

• 214

  "Vaccination with the protein-polysaccharide conjugate vaccine, PCV7, has significantly reduced the burden of pneumococcal disease in populations where it is in widespread use and has had an important public health benefit. This vaccine targets only 7 of the more than 92 pneumococcal serotypes, and there have been concerns that the non-vaccine serotypes (NVTs) could increase in prevalence and reduce the benefits of vaccination." Weinberger, Malley, and Lipsitch 2011, pg. 1.

• 215

  See IDinsight, Impact evaluation of New Incentives, final report, 2020, Table 13, pg. 36: "Outcome: Ever visited clinic… Adjusted OLS results: 0.05 ([95% CI] 0.03, 0.08)"

• 216

  We also received feedback from Dr. Jessica Cohen in her review of our report that this could cause us to underestimate impact: “Not including potential benefits of clinic visits for young children (where health concerns can be addressed in addition to the receipt of vaccinations) could cause you to underestimate impact.” Dr. Jessica Cohen, Bruce A. Beal, Robert L. Beal and Alexander S. Beal Associate Professor of Global Health, Comments on GiveWell’s draft New Incentives Report, November 2023 (unpublished).

• 217

  "Children who are HIV-infected when vaccinated with BCG at birth are at increased risk of developing disseminated BCG disease. However, if HIV-infected individuals, including children, are receiving ART, are clinically well and immunologically stable (CD4% >25% for children aged <5 years or CD4 count ≥200 if aged >5 years) they should be vaccinated with BCG. In general, populations with high prevalence of HIV infection also have the greatest burden of TB; in such populations the benefits of potentially preventing severe TB through vaccination at birth are outweighed by the risks associated with the use of BCG vaccine. Therefore, it is recommended that in such populations:

  • Neonates born to women of unknown HIV status should be vaccinated as the benefits of BCG vaccination outweigh the risks.
  • Neonates of unknown HIV status born to HIV infected women should be vaccinated if they have no clinical evidence suggestive of HIV infection, regardless of whether the mother is receiving ART.
  • Although evidence is limited, for neonates with HIV infection confirmed by early virological testing, BCG vaccination should be delayed until ART has been started and the infant confirmed to be immunologically stable (CD4 >25%)."

  World Health Organization, BCG vaccines: WHO position paper – February 2018, p. 95.
  Adverse events linked to BCG vaccination range from mild, localized complications to more serious, systemic or disseminated BCG disease in which M. bovis BCG is confirmed in one or more anatomical sites far from both the site of injection and regional lymph nodes. Disseminated BCG disease is associated with a case-fatality rate of >70% in infants. By comparison, the background mortality rate among South African HIV-infected infants is 12.2 per 100 person-years (95% CI: 8.2–17.4).
  Systemic or disseminated BCG disease may be clinically indistinguishable from tuberculosis and can only be confirmed through positive mycobacterial culture species identification, preferably by polymerase chain reaction (PCR) for the RD1 genetic region that is lost during attenuation of BCG" Hesseling et al. 2009.
  "Disseminated BCG disease is associated with a case-fatality rate of >70% in infants. . . . [The risk of disseminated BCG disease] has been shown to be 1100 to 4170 per 1 million in HIV-infected infants routinely vaccinated at birth." Hesseling et al. 2009.

• 218

  "Children who are HIV-infected when vaccinated with BCG at birth are at increased risk of developing disseminated BCG disease. However, if HIV-infected individuals, including children, are receiving ART, are clinically well and immunologically stable (CD4% >25% for children aged <5 years or CD4 count ≥200 if aged >5 years) they should be vaccinated with BCG. In general, populations with high prevalence of HIV infection also have the greatest burden of TB; in such populations the benefits of potentially preventing severe TB through vaccination at birth are outweighed by the risks associated with the use of BCG vaccine. Therefore, it is recommended that in such populations:

  • Neonates born to women of unknown HIV status should be vaccinated as the benefits of BCG vaccination outweigh the risks.
  • Neonates of unknown HIV status born to HIV infected women should be vaccinated if they have no clinical evidence suggestive of HIV infection, regardless of whether the mother is receiving ART.
  • Although evidence is limited, for neonates with HIV infection confirmed by early virological testing, BCG vaccination should be delayed until ART has been started and the infant confirmed to be immunologically stable (CD4 >25%)."

  World Health Organization, BCG vaccines: WHO position paper – February 2018, p. 95.

• 219

  "HIV PREVALENCE AMONG PERSONS AGE 15-64 YEARS: Zamfara 0.4%, Jigawa 0.3%, Katsina 0.3%" National Agency for the Control of AIDS, "Nigeria Prevalence Rate," 2019.

• 220

  "The risk of a reaction at the injection site following certain injected vaccines, such as DTaP (diphtheria, tetanus, and pertussis) or pneumococcal vaccine, increases if the doses are not separated by the recommended amounts of time. In these cases, it is the spacing of the doses, not the number of doses, that creates the risk. These reactions can be unpleasant, but they are not life-threatening." Centers for Disease Control, "Ask CDC - Vaccines & Immunizations".

  "We searched VAERS for US reports where an excess dose of vaccine was administered to a person received from 1/1/2007 through 1/26/2018. . . .
  More than three-fourths of reports of an excess dose of vaccine did not describe an AHE [adverse health effect]. Among reports where an AHE event was reported, we did not observe any unexpected conditions or clustering of AEs [adverse events]." Moro et al. 2019, abstract.

• 221

  As of October 2020, New Incentives considered 20% of their partner clinics’ catchment areas to be "high risk." This means one or more cases in which armed violence resulting in death has been recorded near the clinic, but New Incentives judged it to be safe to travel in the area during the day. The assessment was made by New Incentives Security Manager on the basis of information collected from a variety of sources, including field officer (FO) reports. New Incentives, Clinic and settlement security assessments (unpublished)

• 222

  "As of 17-June-2020, there have been 23 incidents in total. The cases include those related to minor theft (e.g. phone and sometimes cash), minor injuries (e.g. bruises and scorpion bites), and encounters with bandits." GiveWell, Questions for New Incentives about potential negative and offsetting effects, 2020, p. 1.

• 223

  New Incentives, Security Procedures and Status (unpublished), "Incidents Involving Staff" sheet:
  Incidents resulting in death:

  • "Year: 2017; Week: 47; A field staff in [redacted] . . . was involved in a road traffic accident (RTA) as he was traveling on a non-work day from [redacted] after visiting his relative in a hospital there. On his way back from the hospital, his vehicle got into an accident with a lorry, which put him in a critical condition. He was rushed to the hospital but unfortunately he passed away on 22-Nov."
  • "Year: 2018; Week: 36. On Tuesday 4/9/18 at about 1:00am armed bandit attacked [redacted], killed two people and kidnapped three people including two children of the district head. Reportedly, the bandit was specifically asking for [routine immunization (RI)] incharge of the clinic, saying that they were informed that the RI incharge and our staff (FV) [Field Volunteer] goes around some of their settlement and sometime in the clinic disbursing cash to some women."

  Incidents resulting in injuries:

  • "Year: 2018; Week: 29. One of our FVs . . . was involved in a motor accident on his way back from the outreach session. The tyre of the vehicle he boarded bursted and the driver lost control consequently hitting a roadside tree. [The FV] sustained an injury to the shoulder. He is stable presently and will seek further medical attention."
  • "Year: 2019; Week: 35. At [redacted], about 4 men armed with dangerous weapons attempted to burgle and rob an NI/ABAE staff residence in [redacted]. The armed youths took about 30 minutes attempting to break the barriers (doors and windows) and attacked neighbours who had responded to the distress calls of the residents of the apartment inflicting various degrees of injuries on the first responders. Upon arrival of security forces, the attackers withdrew from the residence and proceeded to raid at least two nearby houses where they robbed residents of cash and personal effects. Subsequently, members of the community mobilized in their numbers, confronted the robbers causing them to withdraw from the location."
  • "Year: 2019; Week: 46. Two of our FOs including the clinic staff assigned (VCHEW) had an accident today on their way back from the outreach session and sustained slight bruises, and in the process [name redacted] broke his phone and right now he is unable to send his data and do the end process for today."
  • "Year: 2020; Week: 4. Staff reported; 'On their way back from [redacted], they were involved in an accident. He said on their way back they were involved in an accident together with the RI staff, falling on a stone and sustaining injuries and a crack to his phone screen; though phone still remains functional."
  • "Year: 2020; Week: 25. FO was involved in a RTA. Someone with a car hit FO on a commercial bike and as a result FO's office phone got lost. FO sustained a minor injury on his right leg and was treated in a hospital. The person responsible for the accident has gone ahead to replace FOs phone."

• 224

  We last asked New Incentives for detailed information on its procedures in 2020. New Incentives has also shared a log of security monitoring activities, communications of security findings to staff, and security training for staff. GiveWelly lightly reviewed weeks 17-21 of 2021 and found that security processes were reported to have taken place as expected. We have not conducted a more detailed review since that time. See this section of our New Incentives monitoring analysis.

• 225
  • Collecting information about potential security threats and communicating with staff about threats. Information about security incidents is sourced on an ongoing basis by field officers (FOs), as well as by New Incentives' Security Manager and Security Focal Point. New Incentives has shared a list of these incidents with us. The list included an average of 40 reports per month from June 2019 (when this logging system was introduced) to October 2020 (the date of our review); it was up to date at the time of our review. We spot-checked records for 50 incidents: for all of them, it was indicated whether further steps should be taken to mitigate the risk and, if so, what action had been taken. When an ongoing and urgent threat is identified, the Security Manager directly contacts relevant FOs. To communicate less-urgent security information, the Security Manager compiles weekly summaries, which are circulated to managers, who share them with FOs. At times, New Incentives designates high-risk areas as "no go" and prevents FOs from creating work plans or submitting expenses for these areas in the app used for this purpose. We have seen the Security Manager's weekly reports for five weeks in 2020; these reports describe the process used to identify threats and designate areas as "no go."
  • Training staff to avoid security threats where possible and address them where necessary. The onboarding process for New Incentives staff members includes a UN safety training course and a training course on New Incentives' internal security procedures. Ongoing monthly training is also provided, covering topics including road safety and abduction and kidnapping. We have seen the Security Manager's weekly reports on onboarding training for five weeks in 2020.

  Sources:

  • FOs collect information on a rolling basis through drivers (while traveling to the clinic), town criers, and community members (during outreach sessions). When a clinic is marked as "high risk," FOs reach out to clinic staff and community members (for outreach sessions) the day before an immunization day to collect information about security threats. If the FO determines there is reason for concern, they have the option of not joining the outreach session. Svetha Janumpalli, Founder and CEO, and Pratyush Agarwal, Co-founder and COO, New Incentives, conversations with GiveWell, March 25-27, 2020 (unpublished).
  • The Security Manager collects information from security industry networks and media on a rolling basis. The Security Focal Point makes calls to local government authorities every two weeks to collect information about security threats. Svetha Janumpalli, Founder and CEO, and Pratyush Agarwal, Co-founder and COO, New Incentives, conversations with GiveWell, March 25-27, 2020 (unpublished).
  • "SM and SFP: Verifies all security reports received through myDay to ensure that these are true incident.Verifications entails - Reach out to our key sources of information (community stakeholders, security forces, security networks, staff, sister NGO security units) - Triangulation of information from more than one source - Consideration of the authenticity of each source." New Incentives, Security Procedures and Status (unpublished), "Security Procedures and Status" sheet, cell C26
  • "The security incidents are compiled into reports sent internally every week" New Incentives, Overview of NI-ABAE Anti-Bribery and Security Policies, p. 2.
  • "SFP: To collate and send out all security notifications to NC, SRD, SFMs, SM, COO, and external members" New Incentives, Security Procedures and Status (unpublished), "Security Procedures and Status" sheet, cell C14.
  • "SM: Reviews the Security Incident Review sheet to see if himself, FMs, SFMs or NC has taken action for cases where an action is required. This is a follow up of his responsibility to call the attention of the Manager concerned to a recommended action." New Incentives, Security Procedures and Status (unpublished), "Security Procedures and Status" sheet, cell C11.
  • "FM: Will discuss each FOs travel to the clinics or settlements based on highlighted Risk Rating for Disbursement Day or Field Activity, and submit this as part of the FM Check-in. FOs should be able to report if they are uncomfortable to travel based on the Security Assessment and Recommendations and recommend for activities to be cancelled." New Incentives, Security Procedures and Status (unpublished), "Security Procedures and Status" sheet, cell C13.
  • System: Displays Travel Safety Measures from Clinic and Settlement Security Assessments. Will not allow expenses or in-person activities to get logged against No Go clinics and settlements." New Incentives, Security Procedures and Status (unpublished), "Security Procedures and Status" sheet, cell C18.
  • "All Staff:1. Completion of UN B-SAFE and certificates in Zoho 2. Signed CSP + Quiz + Statement 3. Companion Card Received." New Incentives, Security Procedures and Status (unpublished), "Security Procedures and Status" sheet, cell C19.
  • New Incentives training includes (among other topics):
    • UN BSafe training: how to prepare for travel, emergency communications, and how to respond to violence
    • Training on the Country Security Plan, including standard operating procedures on road safety/travel, cash management, abduction, political, ethnic and religious instability, bad governance, and health risks.
    • Road safety training, including control measures to prevent life-threatening accidents, such as using seatbelts and enforcing speed limits.
    • Abduction and kidnapping, including procedures to follow in cases of suspected abduction.

  Svetha Janumpalli, Founder and CEO, and Pratyush Agarwal, Co-founder and COO, New Incentives, conversations with GiveWell, March 25-27, 2020 (unpublished).

• 226

  The number of security incidents reported involving New Incentives staff was 0.4 per 100 disbursement days in March - December 2020, 0.3 in 2021, and 0.3 in 2022. See this row of our monitoring analysis.

• 227

  Days on which clinics provide routine immunizations are called “immunization days”. See this section of our report on New Incentives’ program for more details.

• 228

  See this row in our summary of New Incentives’ monitoring.

• 229

  New Incentives, Incidents involving staff (unpublished):

  • "Year: 2017, Week: 49. On 10-Oct, staff reported the following: According to him . . . he carried motorcyclist to convey him to [redacted], about 3 km from colony then the motorcyclist said he want to ease himself, [he] agreed and the man stopped, immediately he stopped 2 men came out from the farm one with cutlass and the other with pistol, they asked him to take a small foot path which he . . . complied, as they went in they met two other guys each with cutlass and axe, he . . . saw the motorcyclist parked his own bike and joined them, they asked for money and handset which he . . . gave them, they searched all compartments of his back bag they could not find any thing because all the money is in his waist bag except transport money."
  • "Year: 2018, Week: 28. On 10-July, staff reported that 'Thieves broke into his home and Stole cash amounting to N71000 and his big Npower tab phone with 2 chargers and power bank. Staff also reported incident to the police."
  • "Year: 2019; Week: 6. This evening at about 8 pm I received a call from my FV . . . reporting that his room was buggled at [redacted] and all his valuable was stolen including the organization's phone."
  • "Year: 2019; Week: 20. On Sunday 12/05/19, at about 9:30 pm, FM got a call from [FO] who called and narrated how his phone was stolen as he went for evening prayers after breaking his fast. He mentioned that he had dropped his work phone inside his room to charge it, as the battery of the phone was very low. But when he returned from the mosque, he noticed that the phone has been stolen."
  • "Year: 2019; Week: 23. It was reported that today 06/06/19 at around 1:00 pm, armed bandit attacked 2 of our staff on their way back home from the . . . clinic. According to the staffs the attack took place at [redacted] immediately after crossing the river they met 2 bandits with 2 AK 47 rifles waiting for their arrival. After several questions from the bandit they said they are suspecting the bike men are informants, so they are going to hurt them, but finally they collected 1 office smartphone, 1 personal phone, N62,000 for Disbursement, and N13,000 for Transport from [redacted], and they collected N69,000 for Disbursement N13,000 only for the Transport, from [redacted], the incident took place about 4km to . . . clinic."
  • "Year: 2020; Week: 10. On 10-Mar along [redacted] road, it was reported that someone stole FOs phone on his way from [redacted] to [redacted] in public car. FO reported that he had the phone conveniently placed in his pocket during the trip only to discover when he alighted from the vehicle that the phone was no longer there. Despite making several efforts in retrieving the phone by going to the park to lodge same complain, it proved abortive."
  • "Year: 2020; Week: 16. Staff reported that: At [redacted], at night before sleeping, he plugged his phone to charge around 12:30 am as he always do, only to wake up in the morning around 6:43 am and could not find the phone. It is evident the phone must have been stolen because the door was not properly locked. Amongst the items stolen are; 1 NI/ABAE Phone and 2 other phones , belonging to his mother and brother."
  • "Year: 2020; Week: 20. At [redacted], staff reported that her home was burgled around 0300hrs on 14-May and valuables including her personal phone, generator set and twenty five thousand naira (25,000) belonging to NI/ABAE was stolen. She further reported that the thieves attempted to make away with her car when they were awoken by the car alarm. At the instance of the alarm, the thieves allegedly withdrew from the scene."
  • "Year: 2020; Week: 20. At [redacted], Staff went to the Bank to request for a Bank statement as requested by HR during one of his non-clinic operational days. Upon coming out from the bank, he realized his bike has been stolen. In a secret compartment of the bike is a pass issued to the staff to allow access during COVID-19 lockdown."

• 230

  New Incentives told us: "We believe that this is infrequent since the bandits rely on the community for support. Also, it is unlikely that bandits would target female caregivers traveling to the clinic who are receiving small amounts like N500 and N2,000. While we ask questions regarding caregivers losing part of their cash transfers due to bribes or 'dashes' during each disbursement, we do not specifically ask whether caregivers have experienced theft by third parties." GiveWell, Questions for New Incentives about potential negative and offsetting effects, 2020, pg. 1.

• 231

  As of 2020, New Incentives reports that there are cases in which this might have occurred: "While we don't have data, below are the cases where we anticipate that supply-side efforts could have led to shortages at other clinics in the state:
  Sometimes if there is a stockout at the LGA store, our clinic might request vaccines from a nearby clinic. These requests can be refused if the neighboring clinic does not think they have adequate additional stock to spare, we have seen many cases of refusals from nearby clinics. There are at least 87 cases recorded where we attempted to borrow vaccines from a nearby clinic (these cases can be found by searching for 'borrow' in the Clinic level Case Log).
  The consumption increase due to our clinics could cause a short-term shortage in the LGA (we have records of this happening in one of the LGAs in Zamfara with 5 of our partner clinics)." GiveWell, Questions for New Incentives about potential negative and offsetting effects, 2020, pg. 1.

• 232

  “Vaccines are allocated through a calculation made based on factors such as estimated population and consumption. Each LGA is allocated a consignment per quarter in which maximum and minimum stock levels are indicated. SCCOs can authorize LGAs to conduct vaccine ‘mopping’ in which LGAs can request stock from neighboring LGAs with an excess stock, but we do not expect the SCCO to authorize such a request if it would compromise those minimum levels. In general, LGAs are hesitant to participate in vaccine mopping to preserve adequate stock for their clinics and because they have no incentive to share their stock. NI promotes vaccine supply in all LGAs through transportation support to pull supplies (typically from the State’s Cold Chain storage) in both NI- and non-NI LGAs when needed. We are cognizant of the possibility that LGAs where we operate might get prioritized and try to take measures to avoid this.” New Incentives, email to GiveWell, October 24, 2022 (unpublished).

  “NI works with LCCOs and SCCOs to resolve stockout concerns in both operating and non-operating locations. In 2023, our team launched the Monthly RI Vaccine Round Table, a virtual meeting attended by ZCCOs and SCCOs from all states in NW and NE Zones. The goal of this meeting is to foster relationships amongst key partners, gain insight into the vaccine supply space, and proffer targeted solutions to supply challenges. NI also doesn’t limit its financial support for vaccine distribution to NI-operating states and LGAs. For example, in Q1 2023 we provided support to all the LGAs in Katsina for Measles antigen to be distributed to all the LGAs in the state, not only those where we operate.” New Incentives, Vaccine Supply-Side Dynamics: Follow-up, March 2023 (unpublished).

• 233

  See this row in our cost-effectiveness analysis. Note, we also consider other grantee-level factors in our analysis that we do not discuss in this report:

  • Risk of wastage:
    • Double treatment (children not benefitting from the program because they have already received it elsewhere). We account for children receiving more than one of the same vaccination separately in our analysis of program costs (here).
    • Ineffective goods (whether the program might use goods that are out of date, poorly made, or otherwise ineffective). We account for this separately in our adjustment for lower vaccine efficacy in Nigeria (here).
    • Goods purchased and left in storage until they expire. We exclude this factor because New Incentives does not itself distribute vaccines. More detail in this cell note.
  • Factors affecting our confidence in funds being used for their intended purposes:
    • Change of priorities (i.e., New Incentives uses funds designated for the core program for something else). We do not believe that this is a meaningful risk with New Incentives, which runs a single program.
    • Within-org fungibility (i.e., our funding frees up funding for New Incentives to use on another program). We do not believe that this is a meaningful risk with New Incentives, which runs a single program.

  We do not discuss these factors in this report because we assign a 0% downward adjustment in each case. See this section of our cost-effectiveness analysis for more details on our reasoning.

• 234

  See this section in our cost-effectiveness analysis.

• 235

  For information about the various audits for compliance and fraud that New Incentives conducts, see this section of our New Incentives report.

• 236

  "The Central Bank of Nigeria (CBN) recently announced that they will be replacing ₦200, ₦500, and ₦1,000 bills. They plan on starting this on December 15, 2022 while allowing time until January 31, 2023 to replace all the old bills. We expect that this could result in a temporary shortage of bills that can be used to disburse CCTs to caregivers and could result in a temporary increase in the number of infants not served." New Incentives, Program Updates, November 2022 (unpublished)

• 237

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 238

  See this row in our cost-effectiveness analysis.

• 239

  See this row in our cost-effectiveness analysis.

• 240

  See this section of our cost-effectiveness analysis.

• 241

  For details of how we estimate this, see this section.

• 242
  • We estimate that each dollar spent by the Nigerian government generates 0.005 units of value if used for other activities, and each dollar spent by Gavi generates 0.007 units of value.
  • In total, we think these actors spend ~$760,000 on vaccines per $1 million spent by New Incentives (~$414,000 for the Nigerian government, ~$346,000 for Gavi).
  • This implies that shifting these resources away from other activities results in 4,519 units of value being lost. (~414,000 x 0.005) + (~346,000 x 0.007) = 4,519.

  See this row in our cost-effectiveness analysis.

• 243

  See these rows in our cost-effectiveness analysis.

• 244

  4,067/92,661 = ~4%. See this row in our cost-effectiveness analysis.

• 245

  See this row in our cost-effectiveness analysis.

• 246

  We estimate that the Nigerian government’s spending on other activities generates 0.005 of value per $, compared to 0.093 for New Incentives’ program in Bauchi. 0.005 / 0.093 = ~5%. See this section for how we generate these estimates.

• 247

  See this row in our cost-effectiveness analysis.

• 248

  8,760 / 92,661 = ~9%. See this row in our cost-effectiveness analysis.

• 249

  See this row of our cost-effectiveness analysis. This range includes the following states in Nigeria: Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara for the reasons discussed in this section.

• 250

  See these rows of our cost-effectiveness analysis.

• 251

  See this section of our cost-effectiveness analysis.

• 252

  See this section of our cost-effectiveness analysis.

• 253

  See this section of our separate report on New Incentives’ program for more details.

• 254

  Nigeria Strategy for Immunisation and PHC System Strengthening (NSIPSS), 2018.

• 255

  See Nigeria Strategy for Immunisation and PHC System Strengthening (NSIPSS), 2018, section 6.3, pgs 34 - 40.

• 256

  We learned about this program in 2019, but have not deeply investigated what has happened since then. As of 2020, our understanding was that in the three states where New Incentives worked at the time:

  • Katsina chose an education-focused condition for this program over a health-focused condition in part because it judged the health sector to already be better served than the education sector; knowledge of New Incentives' program may have affected that assessment.
  • Jigawa has chosen health-focused conditions for the transfers.
  • Zamfara had not selected a condition as of June 2020.

  Dr. Obinna Ebirim, National Coordinator, New Incentives, email to GiveWell, June 12, 2019 (unpublished):
  "Comment 1: In 2019-2023, the World Bank CCT program is being operated as an experiment and is only expected to reach a small portion of the population (<1% of the population for the conditional transfer; plus Katsina State is not including immunization in its conditions and we don't yet know whether Zamfara State will)
  Response to Comment 1: The World Bank CCT program is being operated as a program. The reason why the proportion of the entire population of the State that is benefiting is small (<1%) is because the target of the program is not the entire population but the poorest of the poor that are in the National Social Register."
  “Jigawa State chose health as their conditionality for the top up conditional cash transfer and got above average in the supply-side evaluation and re-evaluation.”
  “We are unsure to what extent our program has influenced the choice of Katsina State. Our last discussion with the Social Safety Net Office in Katsina State on this matter revealed that Katsina State prioritizes various sector including health and education but feels that health, unlike education, has been receiving lots of interventions by both Government and NGOs. The government official we spoke to gave examples of the government's Saving One Million Lives Program for Result (SOML-PforR) which cuts across Maternal, Newborn and Child Health (MNCH), the New Incentives CCTs for Routine Immunizations program, and Save the Children's nutrition program. An example of Katsina State government's investments in health is the State Emergency Routine Immunization Coordination Centre (SERICC); Katsina State is one of the 18 States in Nigeria that have the equivalent state setup of the National Emergency Routine Immunization Coordination Centre (NERICC). The purpose of these coordination centres is to utilize a war-room like approach of allocating more resources to closely monitor trends, identify issues and solutions, and implement them to improve routine immunization coverage."
  "Katsina State has been implementing the base unconditional cash transfer and has recently selected girl-child education as their conditionality. . . . Zamfara State is yet to choose a conditionality."
  "Based on our latest understanding, Zamfara is still at the base phase and yet to get to the top-up phase where the conditionality is chosen by the state." GiveWell, Questions for New Incentives about potential negative and offsetting effects, p. 2.

• 257

  See this row of our analysis.

• 258

  See this row of our cost-effectiveness analysis.

• 259

  0.005 / 0.093 = ~5%.

• 260

  See this cell in our supplementary analysis.

• 261

  Health

  • We use data from the Uganda National Health Expenditure Accounts in 2013-2014 (available here) as a proxy for how domestic governments in low-income countries allocate their health spending. In that year, approximately 30% of the health spending was allocated to HIV/AIDS, 20% to malaria, and the rest to other programs (see this column in our accompanying spreadsheet).
  • We roughly guess the dollar value required to save a life for eight of the largest categories of spending (e.g., we guess $30,000 of spending on HIV/AIDS programs is required to save one life (here), compared to $10,000 for malaria (here). As a simplifying assumption, we divide each category of spending into spending intended to save adult lives (e.g., HIV/AIDS), or spending intended to save under-five lives (e.g., malaria) (see this column in our accompanying spreadsheet). This allows us to calculate the weighted average cost required to save one child’s life ($12,254) and one adult life ($30,000).
  • Finally, we translate these figures into standardized units of value using GiveWell’s moral weights for the value of averting a child death (116 units) and adult death (73 units). We also use our estimates of the proportion of spending allocated to programs intended to save adult lives (55%) and under-five lives (45%). This results in an overall weighted average of 0.0056 units of value per US dollar spent on health programs by domestic governments. See this section of our cost-effectiveness analysis for our calculations.

  Education

  • As with health, we use data from Uganda as a proxy for how domestic governments in low-income countries allocate their education spending. Specifically, we rely on World Bank 2014 data that 7% of Uganda’s GDP in 2014 went towards primary education, 3% went towards secondary education and 2% went towards tertiary education (here). We then rescale these figures, resulting in estimates that 59% of education spending in Uganda goes towards primary education, 25% towards secondary education and 16% towards tertiary education (here).
  • As a simplifying assumption, we assume that the primary benefit of education spending is that it increases long-run income and consumption. We benchmark our estimates to our estimates of the impact of GiveDirectly’s unconditional cash transfer program (which we think also increases income and consumption). Specifically, we roughly guess that primary education spending is 100% as valuable as GiveDirectly’s program, secondary education is 70% as valuable and tertiary education is 50% as valuable (here).
  • We take a weighted average of these values, using the % of government spending on each category of education as weights. This results in an overall estimate of 0.0028 units of value per $ spent on education programs by domestic governments. See this section of our cost-effectiveness analysis for our calculations.

  Social security

  We use a similar approach to estimating the value of social security spending as education spending. Specifically:

  • We use data from Uganda as a proxy for how domestic governments in low-income countries allocate their social security spending. We use World Bank 2014 estimates that social security spending accounted for 0.62% of Uganda’s GDP (here). Within this, 0.29% of GDP is in-kind spending, 0.1% is social pension spending, 0.1% is conditional cash transfers, and 0.05% is other cash transfers (here).
  • We roughly guess the value generated by each of these four largest categories of social security spending, excluding the smaller categories. As with education spending, we use the simplifying assumption that the primary benefit of social security spending is that it increases income and consumption. Benchmarking to GiveDirectly’s cash transfer program, we guess that cash transfers generate 100% of the value of GiveDirectly’s program and in-kind and social pension spending each generate 70% of the value (here).
  • We take a weighted average of these values, using the % of government spending on each category of social security spending as the weights (re-scaled to ignore smaller categories). This results in an overall estimate of 0.0026 units of value per $ spent on social security programs by domestic governments. See this section of our cost-effectiveness analysis for our calculations.

• 262

  See here in our cost-effectiveness analysis.

• 263

  See this row of our cost-effectiveness analysis.

• 264

  0.007 / 0.093 = ~7%.

• 265

  For example, pledges to Gavi in 2020 (for the 2021-2025 period) exceeded its replenishment ask: “US$8.8 billion worth of new commitments were pledged by public and private sector donors towards Gavi’s replenishment, well exceeding the target set in support of Gavi’s investment opportunity for the next 5 years. These pledges will add to the US$1.7 billion previously secured by Gavi, bringing its resources for 2021-2025 to more than US$10.5 billion. Gavi is now well positioned to accelerate the roll out of current vaccines, reach 300 million more children in developing countries by 2025, and in so doing contribute to saving 7 to 8 million lives.” Global Vaccine Summit London 2020.

• 266

  Our understanding is that there is some opposition to New Incentives’ program among some individuals in the Nigerian federal government and that concerns about sustainability and potential backlash effects are parts of this concern. See this section of our separate page on New Incentives’ program for more details. New Incentives also reports hearing this concern from other partners.

  “A concern . . . stakeholders (including UNICEF representatives) have shared with us is about incentives replacing intrinsic motivations to immunize infants in the absence of incentives. This was experienced by WHO when some women in Northern Nigeria started refusing vaccinations if they didn't get the in-kind donations that were previously being offered during Polio campaigns.” New Incentives, Responses to questions from GiveWell, September 23, 2020 (unpublished)

• 267

  More on these aspects of its work here and here on our separate page.

• 268

  “A concern . . . stakeholders (including UNICEF representatives) have shared with us is about incentives replacing intrinsic motivations to immunize infants in the absence of incentives. This was experienced by WHO when some women in Northern Nigeria started refusing vaccinations if they didn't get the in-kind donations that were previously being offered during Polio campaigns.” New Incentives, Responses to questions from GiveWell, September 23, 2020 (unpublished).
  New Incentives also told us it believes incentives play a different role in its program than they did in the polio vaccination campaign, since campaigns provide vaccinations door-to-door (and thus create limited costs for caregivers), while New Incentives mostly incentivizes vaccinations in clinics (which have higher time and transport costs for caregivers).
  “Note: we have received conflicting reports and opinions regarding in-kind incentives during Polio campaigns. CCTs for Routine immunization are different from offering in-kind incentives or ‘pluses’ during Polio campaigns where caretakers do not bear any cost to vaccinate their children with an oral vaccine at their home. We take concerns around intrinsic motivation and perverse incentives very seriously. We do not think these are unique to the application of CCTs to immunizations as similar concerns exist for application of CCTs to other health and non-health behaviors. We have incorporated messaging in various components of the program to help caretakers understand that vaccinations are important to protect their children against deadly diseases.” New Incentives, Responses to questions from GiveWell, September 23, 2020 (unpublished).

• 269

  “Withdrawal of SMS reminders and incentives was associated with statistically insignificant decreases in MCV1-seeking for subsequent children (SC) and statistically significant decreases in MCV2-seeking for some former M-SIMU children. Decreased MCV1-seeking translated to lower-than-expected coverage among SC of former MSIMU intervention caregivers compared to Control SC.
  SMS reminders can increase vaccination uptake. Unconditional incentives may have no added effect on MCV1 uptake over SMS reminders alone. Withdrawal of SMS reminders and incentives could reduce measles vaccine-seeking.” Kagucia 2018, pp. iii-iv.

• 270

  "Post-trial follow-up visits were conducted between August 4, 2018 and November 30, 2018. CIs visited 1,467 of 1,599 M-SIMU households (91.7%); 132 M-SIMU households were not visited as funding for the study ended before the households could be visited. Of the households visited, 218 (14.9%) follow-up visits were completed. Follow-up visits were not conducted for the remaining households either because they did not meet MSBC eligibility criteria (n= 1,000; 68.2%) or were lost to follow-up (n= 249; 17.0%).” Kagucia 2018, pg. 318. 218 / 1,599 = ~14%.
  Note that most of these households were not followed up because they didn’t meet study eligibility criteria, rather than because they could not be reached. Of the 86% not followed up:

  • ~72% were not eligible for the study (e.g., because they didn’t have subsequent children), 1,000 / (1,599 - 218).
  • ~18% were lost to follow up (249 / (1,599 - 218).
  • ~10% were not visited because the study ended before they could be visited (132 / (1,599 - 218)).

• 271

  “A second issue concerns the extent to which the large up-front cost can be considered not only as a way of encouraging current use of skilled delivery services but as an investment that generates a longer term impact on health seeking behavior going well-beyond the end of the programme. The timing of the study allows us to examine what happened after the ending of funding for the intervention. Information on use of services 2 years beyond the end of the programme suggest that the effect of the intervention may have continued well-beyond the period of funding. The investment in the supply side into equipment and the facility was not immediately lost when the programme closed so potentially explaining the continued effect. It is notable, however, that the persistent effect is largely evident in the CCT areas and not the SP areas which also received the supply side investment.” Onwujekwe et al 2020.

• 272

  “Results: The CCTs contributed to increasing facility attendance and utilization of MCH services by reducing the financial barrier to accessing healthcare among pregnant women. However, there were unintended consequences of CCT which included a reduction in birth spacing intervals, and a reduction of trust in the health system when the CCT was suddenly withdrawn by the government.” Ezenwaka et al. 2021, abstract.

• 273

  Studies we’ve reviewed:

  • McCoy et al. 2017, an RCT of ~800 participants estimating the effect of CCTs and food transfers incentivizing antiretroviral therapy adherence in Tanzania. The primary outcome was the percentage of participants who possessed antiretrovirals for over 95% of the time they were tested (medication possession ratio (MPR) ≥95%). It found that the program had a ~21pp effect at the end of the intervention and roughly 20pp six months after that.
    • "At three clinics, 805 participants were randomized to three groups in a 3:3:1 ratio, stratified by site: nutrition assessment and counseling (NAC) plus cash transfers (~$11/month, n=347), NAC plus food baskets (n=345), and NAC-only (comparison group, n=113, clinicaltrials.gov NCT01957917)" (abstract)
    • "The primary intent-to-treat analysis included 800 participants. Achievement of MPR≥95% at 6 months was higher in the NAC+cash group compared to NAC-only (85.0% vs. 63.4%), a 21.6 percentage point difference (95% confidence interval (CI): 9.8, 33.4, p<0.01). MPR≥95% was also significantly higher in the NAC+food group versus NAC-only (difference=15.8, 95% CI: 3.8, 27.9, p<0.01). When directly compared, MPR≥5% was similar in the NAC+cash and NAC+food groups (difference=5.7, 95% CI: −1.2, 12.7, p=0.15). Compared to NAC-only, appointment attendance and LTFU were significantly higher in both the NAC+cash and NAC+food groups at 6 months. At 12 months, the effect of NAC+cash, but not NAC+food, on MPR≥95% and retention was sustained.” (abstract)
    • "At 12 months, MPR≥95% was significantly greater among those in the NAC+cash group than the NAC-only group (74.9% vs. 55.4%, adjusted difference=20.3, 95% CI: 8.4, 32.2, p<0.01) in both ITT and adjusted analyses." (p. 7)
  • de Walque, Dow, and Nathan 2014, an RCT including ~2400 participants, estimating the effect of CCTs incentivizing safe sexual practices in Tanzania. It found a 25% decrease in sexually transmitted infections for a $20 transfer after 12 months. A follow-up at 24 months found a decrease of approximately 20%, although this was not statistically significant..
    • “The Rewarding Sexually Transmitted Infection Prevention and Control in Tanzania (RESPECT) study is a randomized controlled trial testing the hypothesis that a system of rapid feedback and positive reinforcement that uses cash as the primary incentive can be used to reduce risky sexual activity among young people, male and female, who are at high risk of HIV infection. The study enrolled 2,399 participants in 10 villages in rural southwest Tanzania.” (abstract)
    • "at month 12 (column 3) the high value CCT arm corresponds to a 25% risk reduction compared to the control group (relative risk of 0.749 statistically lower than 1)" (p. 13)
    • See Table 3, p.28: “Month 12 combined prevalence of 4 STIs tested at every round: 0.749** and Table 3, p.28: “Month 24 combined prevalence of 4 STIs tested at every round:0.798” (pg. 28).
  • Fahey et al. 2021, a follow-up to a trial of cash or food transfers for monthly healthcare appointments for adults starting HIV treatment in Tanzania. The study found no statistically significant differences in retention in care between groups 24 and 36 months after the program ended, and a non-significant trend in favor of the intervention group after 36 months, indicating no evidence of crowding out.
    • “We traced former participants in a 2013–2016 trial, which randomised 800 food-insecure adults starting HIV treatment at three clinics to receive either usual care (control) or up to 6 months of cash or food transfers (~US$11/month) contingent on timely attendance at monthly clinic appointments. The primary intention-to-treat analysis estimated 24-month and 36-month marginal risk differences (RD) between incentive and control groups for retention in care and all-cause mortality, using multiple imputation for a minority of missing outcomes. We also estimated mortality HRs from time-stratified Cox regression." (abstract)
    • "From 3 March 2018 to 19 September 2019, we determined 36-month retention and mortality statuses for 737 (92%) and 700 (88%) participants, respectively. Overall, approximately 660 (83%) participants were in care at 36 months while 43 (5%) had died. There were no differences between groups in retention at 24 months (86.5% intervention vs 84.4% control, RD 2.1, 95% CI −5.2 to 9.3) or 36 months (83.3% vs 77.8%, RD 5.6, –2.7 to 13.8), nor in mortality at either time point. The intervention group had a lower rate of death during the first 18 months (HR 0.27, 95% CI 0.10 to 0.74); mortality was similar thereafter (HR 1.13, 95% CI 0.33 to 3.79).” (abstract).
  • Czaicki et al. 2018, a trial comparing food and cash incentives for healthcare appointment visits for HIV-positive adults in Tanzania against standard of care. The study found no difference in intrinsic motivation between groups (as measured by the Treatment Self-Regulation Questionnaire (TSRQ)) at baseline, endline, and approximately six months after the program ended.
    • “We analyzed data from 469 individuals randomized to one of three study arms: standard of care, short-term cash transfers, or short-term food assistance. Eligible participants were: 1) ≥18 years old; 2) HIV-infected; 3) food insecure; and 4) initiated antiretroviral therapy (ART) ≤90 days before the study. Food or cash transfers, valued at ~$11 per month and conditional on attending clinic visits, were provided for ≤6 months. Intrinsic motivation was measured at baseline, 6, and 12 months using the autonomous motivation section of the Treatment Self-Regulation Questionnaire (TSRQ). We compared the change in TSRQ score from baseline to 6 and 12 months and the change within study arms." (abstract)
    • The mean intrinsic motivation score was 2.79 at baseline (range: 1–3), 2.91 at 6 months (range: 1–3), and 2.95 at 12 months (range: 2–3), which was 6 months after the incentives had ended. Among all patients, the intrinsic motivation score increased by 0.13 points at 6 months (95% CI (0.09, 0.17), Cohen’s d = 0.29) and 0.19 points at 12 months (95% CI (0.14, 0.24), Cohen’s d = 0.49). Intrinsic motivation also increased within each study group at 6 months: 0.15 points in the food arm (95% CI (0.09, 0.21), Cohen’s d = 0.37), 0.11 points in the cash arm (95% CI (0.05, 0.18), Cohen’s d = 0.25), and 0.08 points in the comparison arm (95% CI (-0.03, 0.19), Cohen’s d = 0.21); findings were similar at 12 months. Increases in motivation were statistically similar between arms at 6 and 12 months.” (abstract)

• 274

  The rate of stockouts of any vaccine rose from ~8 per 100 disbursement days during the RCT to ~47 per 100 disbursement days in 2022. While we see this as concerning, we do not currently see it as a significant threat to the program because the increase in stockouts is concentrated heavily in indirectly incentivized vaccines, particularly the rotavirus vaccine (which was only introduced in the Nigerian immunization schedule in August 2022, see Gavi, "Dealing with diarrhoea: Nigeria introduces rotavirus vaccine into its immunisation plan," August 30, 2022). These form a relatively small share of the overall benefits of the program. More here and in this row of our summary of New Incentives' monitoring data.

• 275

  "New Incentives will group local government areas (LGAs) it expands to within a given state at a given point in time into ‘expansion groups’. New Incentives will then collect coverage data in these expansion groups once before the start of operations to establish baseline coverage rates." IDinsight, Coverage monitoring analysis plan, 2021, pg. 1.
  For the dates of the surveys we have received from New Incentives, see here.

• 276

  Some of the estimates we use in our main analysis of vaccine effects rely in part on meta-analyses that include some observational studies. See the footnotes in this section for more details.

• 277

  King et al. 2020: Each percentage point increase in PCV coverage was associated with a 1-2% decrease in non-traumatic infant mortality:

  • Results from Study 1: "Adjusted negative binomial regression of monthly mortality against three-dose coverage found every 1 percentage point increase in coverage was associated with a 2.0% (95% CI: 1.5 to 2.4; p value<0.001) decrease in mortality"
  • Results from Study 2: “Across 354 geographical clusters, three-dose PCV13 coverage ranged from 58% to 100% and mortality ranged from 0 to 87/1000 live births. Cluster-level impact analysis, using negative binomial regression adjusted for RV1 coverage, found that every 1% absolute increase in three-dose coverage by geographical cluster was associated with a 1.3% reduction in non-traumatic infant mortality (95% CI: 0.3% to 2.4%; p value=0.02).”

  Bar-Zeev et al. 2018: Each percentage point increase in vaccine coverage was associated with a 1.1% decrease in diarrhea-related mortality. “Two-dose vaccine coverage ranged from 63·6% to 100% across clusters; each percentage point increase in vaccine coverage was associated with a 1·6% (95% CI 0·8–2·5) lower diarrhoea-associated mortality rate (appendix). Adjusting for sociodemographic covariates, the reduction was 1·1% (95% CI 0·9–1·3).”
  Sifuna et al. 2023: A coverage increase from 0-75% was associated with a 44% reduction in all-cause child mortality. “Following its introduction in late 2014, the span of rotavirus vaccine coverage for children increased to 75% by 2017. Receiving the rotavirus vaccine was associated with a 44% reduction in all-cause child mortality (95% confidence interval = 28-68%, p < 0.0001), but not diarrhea-specific mortality (p = 0.401).”

• 278

  See GiveWell’s grant page to IRD Global for more details.

• 279

  “US$8.8 billion worth of new commitments were pledged by public and private sector donors towards Gavi’s replenishment, well exceeding the target set in support of Gavi’s investment opportunity for the next 5 years. These pledges will add to the US$1.7 billion previously secured by Gavi, bringing its resources for 2021-2025 to more than US$10.5 billion. Gavi is now well positioned to accelerate the roll out of current vaccines, reach 300 million more children in developing countries by 2025, and in so doing contribute to saving 7 to 8 million lives.” Global Vaccine Summit London 2020

  “Over US$7.5 billion was pledged by 31 public and private sector donors, which means that Gavi, the Vaccine Alliance is well positioned to achieve these milestones, expand vaccine coverage, strengthen health systems and accelerate the introduction of new vaccines. The pledges announced today will be combined with US$2 billion in resources already committed for the 2016-2020 period to enable Gavi to meet the US$9.5 billion cost of funding vaccine programmes in developing countries over the next five year period.” Gavi, Reach Every Child: Gavi Pledging Conference, 2015

• 280

  “Respondents who had either never vaccinated their children or had missed one or more vaccinations (89.8% of the sample), were asked the question 'Why have they not received these vaccinations?'. Responses were categorized by enumerators into one of 18 possible categories”. IDinsight, New Incentives evaluation baseline report, 2019, p. 48.

• 281

  IDinsight, New Incentives evaluation baseline report, 2019, figure 14, p. 49.

• 282

  We received feedback in 2022, in a submission to GiveWell’s “Change our Mind” contest, that we should use an epidemiological model rather than a static model: “Our first critique is the core modelling approach taken for this cost-effectiveness model. We understand the model developed by GiveWell to be a static model; the gold standard of models for vaccination cost effectiveness are dynamic models (1). Models such as Susceptible-Infected-Recovered (SIR) models may be especially suited. However, we understand from the literature that it is not straightforward to estimate the extent to which these types of models provide differing estimates to static models.” Naik and Field 2022, p. 1.

• 283

  "The World Health Organization (WHO) has recommended a new vaccine, R21/Matrix-M, for the prevention of malaria in children. . . . The R21 vaccine is the second malaria vaccine recommended by WHO, following the RTS,S/AS01 vaccine, which received a WHO recommendation in 2021." World Health Organizatgion, "WHO recommends R21/Matrix-M vaccine for malaria prevention in updated advice on immunization," 2023.

• 284

  We estimate that each dollar donated to GiveDirectly’s direct cash transfer program generates 0.003 units of value. 0.005 / 0.00335 = ~1.5.

• 285

  We estimate that each dollar donated to GiveDirectly’s direct cash transfer program generates 0.003 units of value. 0.015 / 0.00335 = ~4.5.