Lookback: Grants to New Incentives' Conditional Cash Transfer Program for Childhood Vaccination

Note: This page compares the impact of New Incentives' grants against our initial expectations. This page reflects our analysis and understanding as of December 2024. New Incentives staff reviewed this page prior to publication.

Published: July 2025

Background

New Incentives has been a GiveWell Top Charity since 2020.3 Since then, we have directed over $120 million4 to its program incentivizing vaccinations in northern Nigeria.

We made these grants because we think New Incentives' program was a cost-effective way to avert child mortality and because these grants would allow New Incentives to scale across northern Nigeria.5

The rest of this report provides our assessment of how these grants went and next steps, based on what we’ve learned.

Would we have made these grants again, knowing what we know now?

Yes. Our current assessment suggests the program is twice as cost-effective as we initially thought (more), and implementation has gone well despite rapid scaling (from ~70,000 children in 3 states in 2020 to ~1.5 million children in 9 states in 2023) (more).

Even if we had known cost-effectiveness would be higher than expected, we don’t think it would have been possible to scale the program any faster.6

How did implementation go?

Even as the program scaled from ~70,000 children enrolled in 2020 to ~1.5 million enrolled in 2023, the data we reviewed suggest program quality has remained stable. However, we’re still waiting on data that show effects on vaccination pre and post New Incentives’ entry, which we think will provide more reliable information on New Incentives’ impact over time.

New Incentives told us it made adjustments to respond to some threats to program impact. For example, it implemented stronger anti-fraud measures, including GPS checks and biometric matching, and adjusted its incentive structure in response to inflation and the removal of a fuel subsidy in Nigeria.

We’re concerned about risk of fraud and New Incentives has shared some other implementation challenges, but we don’t think these significantly compromised the effectiveness of the program.

Children enrolled

We believe New Incentives has been effective at scaling its program (both in terms of geographic scope and number of children enrolled) at a rapid pace with GiveWell funding.7 In 2020, New Incentives enrolled 70,705 children across 3 states.8 In 2023, New Incentives enrolled 1,518,900 children across 9 states.9

Monitoring data

We conduct an annual review of New Incentives' monitoring data, and continue to believe this data is both comprehensive and high quality.10 Our review of New Incentives' monitoring data suggests New Incentives was able to maintain high program quality during scale-up. This is based on comparing several indicators between current implementation and during the 2017-2020 RCT, which we use to estimate the effect of New Incentives on vaccination rates.11

In the following table we present a subset of the indicators we use to assess the performance of New Incentives' program over time. We elaborate on these indicators below.

Category	Indicator	Comparison: result during RCT	Since end of RCT (March 2020 - December 2023)
Retention (enrolled infants complete the vaccination schedule)	Retention rate: PENTA3	89%	89%
Supply-side issues	% of disbursement sessions with stockout of any directly incentivized vaccine	6.0	10.9
	% of disbursement sessions with stockout of any indirectly incentivized vaccine	0.4	17.0
Real value of transfer	Value of transfers relative to RCT	-	77%
Fraud	% of disbursement days audited	5%	8%

Effect on vaccination rates over time

We base our estimate of the program's impact on vaccination rates on the RCT results, with a fair amount of uncertainty about whether we'd see the same effects at scale.12 We’re planning to use New Incentives' follow-up vaccination coverage surveys to measure the effect on vaccination rates after New Incentives enters a new state.

Our next steps:

• We’ve recently begun to receive the results of the follow up surveys and expect to more fully review those data in 2025.
• We also plan to compare predicted vaccination rates to vaccination rates reported in the 2023-2024 Nigeria Demographic and Health Survey (DHS) in 2025.

Risk of fraud

Fraud was not called out in our 2020 grantee review as a major reservation/open question, but we did list it as a potential negative or offsetting effect. We now list the risk of fraud as one of our main reservations about the program.

In 2021, we increased our downside grantee-level adjustment from -5% to -7%, with -2% quantifying the risk of biased monitoring results.13 This -2% adjustment was not based on evidence of biased monitoring results, but on gaps in our understanding of how monitoring results are collected and verified, meaning they could be biased in ways we are unaware of.

We estimate the rate of repeat enrollments (one form of caregiver fraud) has increased from 5% in 2020 to roughly 11% in 2023 (see here).14

New Incentives has added new measures to try to detect and deter fraud since 2020. These measures include adding new fraud-related indicators to its monitoring framework, additional checks to detect fraud (from both caregivers and staff) in the disbursement process, and incentives to encourage employees' timely compliance with internal reporting and performance requirements.15

Our next steps: Although we don’t have substantial reservations about the policies New Incentives has put in place and think it seems to be taking the risk of fraud seriously, we think there's a risk that some types of fraud would not get detected and we plan to contract a third party to do a deep dive into New Incentives' monitoring and evaluation in 2025.

Implementation challenges and adjustments to the program

New Incentives has reported implementation challenges over time and adaptations to the program, in response to these challenges and feedback from stakeholders. Some examples are below:

• Vaccine stockouts. New Incentives reported an increase in the percentage of disbursement days experiencing vaccine stockouts, compared to the RCT (~7% vs. ~28%). However, this increase was largely among indirectly incentivized vaccines, such as the yellow fever and rotavirus vaccines, which account for a much smaller share of the program benefits.16 In response to stockouts, New Incentives expanded from local-level management to broader coordination with state and zonal officials, including monthly cold chain management meetings.
• More local staff coverage. New Incentives initially planned a "hub and spoke" structure where Field Officers would cover multiple clinics, but this proved impractical for caregivers and clinics. The organization instead simplified training and processes to hire more Field Officers, expanding from less than 100 clinics during the RCT to 5,862 clinics across 167 local government areas by December 2023.
• Staff fraud prevention. In 2020, New Incentives identified issues with false expense reports and disbursement claims from staff members. The organization implemented new measures including the myDay app for documentation, GPS location checks, enhanced audit protocols, and performance-based assessments for auditors.
• Clinic documentation compliance. New Incentives found inconsistent recording of vaccinations by clinic staff, which could enable multiple claims for the same vaccine. The organization responded by implementing clinic operation pauses for non-compliance with documentation requirements, requiring photo documentation of clinic registers (Tally Sheets), improving vaccine administration observation protocols, and piloting monthly stipends for compliant clinic staff.
• Fraudulent repeat enrollments. New Incentives has implemented multiple checks to try to catch and deter repeat enrollments, including BCG scar checks, photo comparison using Amazon Rekognition, and in 2024 implemented new measures to differentiate between legitimate multiple enrollments by one caregiver and fraudulent ones. We estimate roughly 11% of 2023 enrollments were repeats, and in early 2025 we plan to review 2024 program data to see if this rate has come down.
• Coverage survey implementation. As New Incentives expanded following the RCT, it began conducting its own routine vaccination coverage surveys.17 Field Officers were initially responsible for these surveys, but a dedicated survey team was later created to manage the workload. Over time, the survey team has reduced average survey time from over an hour to around 30 minutes through improved training and streamlined questioning.
• Measles retention rates. New Incentives has faced lower uptake for measles vaccines (87-91% for first dose, 75% for second dose) compared to earlier vaccines in the schedule. The organization made multiple adjustments to its incentive structure to try to encourage greater measles uptake, including differential payment amounts and an additional 5,000 naira incentive for completing the full schedule.

How cost-effective do we think these grants were?

In 2020, we estimated the program to be ~11x as cost-effective as cash transfers. As of December 2024, we think New Incentives is ~22x18 in the original states we funded in 2020 and ~9-38x across all 9 states where New Incentives is currently operating. The main driver of this increase in cost-effectiveness is the reduction in New Incentives' cost per child enrolled from ~$39 to ~$18, which roughly doubled cost-effectiveness.

We estimate that our ~$120 million to New Incentives since 2020 has or will avert ~27,000 deaths.19 If we assume that New Incentives' cost per child enrolled remained at its 2020 level, this estimate would be ~17,000.20

In addition, we've updated the data we use to estimate the burden of disease from vaccine-preventable diseases. These updates did not have a major effect on cost-effectiveness on aggregate, but we think exploring these data further is a high priority. We’ve also made some smaller updates to our model.21

Although we've made many updates to our cost-effectiveness analysis since 2020, there are some parts of our analysis we've not changed, either because they still represent our best guess or because we've chosen not to prioritize updating them (often because we expect changing certain parameters to have very little impact on cost-effectiveness).

Costs per child enrolled

In 2020, we estimated New Incentives' program cost ~$39 per child enrolled. In 2024, we estimated the cost per child enrolled would be ~$18.22

We believe this reduction is a result of economies of scale (as New Incentives has grown its fixed costs have been spread across a larger number of children),23 explicit efforts on New Incentives' part to achieve greater cost efficiencies,24 and devaluation of the naira increasing New Incentives' purchasing power.25 However, we have not investigated the drivers of this change in depth.

Our next steps:

• Understand whether these are the drivers of change for New Incentives. What share is driven by naira devaluation vs. other factors? How likely is it that costs will “rebound” to be higher in the future?
• Consider whether there are other programs, beyond New Incentives, where we’re underestimating cost-effectiveness by not anticipating potential for large reductions in costs at scale or where currency devaluations have had a similar effect.

Burden of disease

We made two updates to the data we use to estimate the burden of vaccine-preventable disease in states where New Incentives works.

• State-level estimates. We now use state-level mortality estimates from the Institute for Health Metrics and Evaluation's Global Burden of Disease (GBD) project rather than adjusting national estimates for an assumed higher rate of child mortality in northern Nigeria. While we think getting state-level information is important, since we provide funding at the state level, we haven't yet validated whether the variation we see between states reflects genuine differences in mortality rates or statistical noise in the underlying data.
• Updated version of GBD. We’ve begun using the most recent data from GBD (GBD 2021; we used to use GBD 2019), though we plan to explore whether we should put weight on different GBD releases and whether to weight other sources as well.26 We’ve done initial checks of GBD 2021 against other sources and have not found discrepancies that would change which grants we make, but we plan to do further work on this.

Our next steps:

• In 2025, update our approach for which burden of disease estimates we use and how much weight we put on other sources.
• In 2025, conduct a sense check of our state-level estimates against other sources.

Major uncertainties we haven’t learned more about

In our recent intervention report, we identify several key open questions about New Incentives' program model. We think the following open questions are important but we haven't made significant progress looking into them to date:

• Are we adequately accounting for increases in vaccination rates that would have taken place anyway? The RCT found a substantial increase in control group vaccination rates over the course of the study.27 Other sources of data also show vaccination rates were increasing in northern Nigeria before New Incentives began scaling up. Because we’d expect the program to be less effective in areas with higher baseline coverage, this raises a concern that the impact in the future will be smaller than the impact observed in the RCT. We currently account for this by assuming vaccination coverage will continue to improve by the same rate seen in the years before New Incentives' scale up (2013-2021). However, our estimates could be meaningfully off in either direction. We also haven’t deeply investigated what other programs are being delivered to increase vaccination rates in northern Nigeria, which increases our uncertainty on this issue.
• How effective are vaccines at reducing mortality in Nigeria? We’ve reviewed evidence which suggests that the measles and BCG vaccines might be less effective in Nigeria, and we think it is plausible that vaccine effectiveness may generally be lower in settings where proper storage and handling of vaccines is more difficult. We adjust for this risk in our analysis (reducing our estimate of vaccine efficacy for all vaccines by around 14%). But our adjustment is very speculative and further research could lead us to update this upward or downward. Our 25th - 75th percentile confidence interval for vaccine efficacy against vaccine-preventable disease (which incorporates this adjustment) is 42% to 63%, implying a cost-effectiveness of 16x to 24x in Bauchi.28
• How will the effect of New Incentives' program in states with higher vaccine coverage compare to the effect in the RCT of the program? Our understanding of the effect of New Incentives' program on vaccination rates is largely based on the results of an RCT of the program conducted in Jigawa, Katsina, and Zamfara in 2017-2020. We extrapolate the RCT results to other states based on their vaccine coverage rates, but we are unsure how well these results will translate to new states. New Incentives has begun to conduct vaccine coverage surveys in areas where it works, and we plan to use these as another source to check that the program is having the expected effect at scale. We expect to review follow up coverage survey data in early 2025, once we have results from 24 months of program operations in several areas.

Did we set ourselves up to learn?

We’ve been able to learn a lot about New Incentives’ program because:

• New Incentives conducts regular baseline and follow-up surveys of vaccination coverage, providing valuable data on the program's impact over time. These surveys offer information on vaccination rates and program effectiveness in different areas. We haven’t reviewed these in full yet but believe we’ve set ourselves up to learn from these in the future. (We discuss these above.)
• New Incentives maintains extensive program data collection, allowing for detailed analysis of various aspects of the program's implementation and outcomes. (We discuss these data above.)

There are several areas where we could have set ourselves up for better learning, though we decided not to prioritize these:

• We lack comparison data from non-program areas, which limits our ability to estimate what would’ve happened without New Incentives (e.g., how much would vaccination rates rise counterfactually, and are stockout rates similarly high in areas where New Incentives does not work?)
• Our engagement with other stakeholders in Nigeria, particularly government officials, has been limited. As a result, we may be missing important considerations about how New Incentives' program impacts government systems and priorities, or other organizations working in Nigeria.
• Our engagement with external experts, including epidemiologists and disease modelers, has also been limited. This lack of external input may have limited our ability to gain diverse perspectives and expertise on the program.

Our next steps:

• Speak more with government officials in Nigeria and others with experience working in the states where New Incentives works by early 2025.
• Speak more with experts on some of our outstanding questions about vaccine efficacy and other issues in 2025.

What about additional factors influencing our assessment of the impact of the program?

Funging

We have not seen the government of Nigeria or other funders express interest in funding a CCT program on the scale of New Incentives, so we still think this is an area where additional funding is needed.29

External input

We’ve spoken to additional funders and find mixed interest in supporting CCTs for vaccination. Our understanding is that there is some skepticism of the program among some members of the Nigerian federal authorities as well.

• New Incentives faces skepticism of its program from some members of the National Primary Health Care Development Agency (NPHCDA). Critics within the federal government have raised concerns about the potential for backlash if New Incentives discontinues its program in a given area, which could lead to falling vaccination rates.30
  • Despite these concerns, New Incentives has been able to continue operating and expanding its program (e.g. New Incentives has been able to sign Memorandums of Understanding (MoUs) with new states, and renew MoUs with the three original states).
  • GiveWell has not spoken with Nigerian government officials about the skepticism of New Incentives' program.
• We've asked other funders for their take on CCT programs and have received mixed responses.
• One of the objections to the program (and the one we hear most frequently) is that there might be negative effects when the program is discontinued, by crowding out caregivers' intrinsic motivation to vaccinate their children. In 2024 we updated our CEA to include a -11% to -30% adjustment for potential negative post-program effects in prospective new states. This is a fairly speculative adjustment based on limited evidence, and we don’t think the evidence completely substantiates concerns raised by critics nor suggests that the potential consequences outweigh the benefits of New Incentives' program.

We find the mixed interest in CCTs for vaccination surprising, since we view this program as one of the most cost-effective funding gaps we’ve come across.

How calibrated were our forecasts?

Our forecasting results were mixed. Of the 21 forecasts we made, 8 were accurate. We were too pessimistic about program quality maintenance during scaling. We were too optimistic about stockout rates and grant sizes. We failed to make explicit predictions on key metrics such as enrollment numbers or expansion speed.

Our forecasts are in the table below:

Prediction	%	Date made	Did this happen?	Notes. If wrong, too optimistic or pessimistic?
New Incentives will be unable to operate for >20% of the disbursement days in the clinics it has begun operations in during the first six months of 2022. This may be due to security concerns, telecommunication shutdowns, or other unanticipated factors.	50%	1/1/22	No	Too pessimistic.31
Number of stockouts per 100 disbursement days in clinics that New Incentives works in will be >15 in the first six months of 2022, according to New Incentives' data.	35%	1/1/22	Yes	Too optimistic.32
Rotavirus vaccines will be available and incentivized by New Incentives in >75% of the clinics it operates in by the end of 2022.	65%	1/1/22	Yes	Accurate but slightly pessimistic.33
New Incentives will be unable to operate for >20% of the disbursement days in the clinics it has begun operations in during the second six months of 2022. This may be due to security concerns, telecommunication shutdowns, or other unanticipated factors.	35%	5/27/22	No	Accurate34
New Incentives will have an average measles retention rate >85% in the second six months of 2022. (Measles retention during the RCT was 91%. It has been 81% since the end of the RCT.)	30%	5/27/22	Yes	Too pessimistic. Measles 1 retention was 88% in the second half of 2022.35
Number of stockouts per 100 disbursement days in clinics that New Incentives works in will be >15 in the second six months of 2022, according to New Incentives' data. (This figure was 5 out of 100 days during the RCT and 13 since the RCT ended.)	25%	5/27/22	Yes	Too optimistic. There were ~60 stockouts of any vaccine per 100 disbursement days in the last 6 months of 2022.36
Rotavirus vaccines will be available and incentivized by New Incentives in >75% of the clinics it operates in by the end of 2022.37	40%	5/27/22	Yes	Too pessimistic. (see footnote 33).
New Incentives will have an average measles retention rate >85% in 2023. (Measles retention during the RCT was 91%. It has been 87% since the end of the RCT and was 86% in the first 7 months of 2022.)	55%	11/4/22	Yes	Accurate but with low confidence. Measles 1 retention was 87% on average in 2023.38
Number of stockouts of incentivized vaccines per 100 disbursement days in clinics that New Incentives works in will be >10 in 2023, according to New Incentives' data. (This figure was 5 out of 100 days during the RCT for all vaccines. Since the RCT ended, it's been 9 per 100 days. In the first 7 months of 2022, it was 8.)	40%	11/4/22	Yes	Too optimistic. Stockouts of directly incentivized vaccines per 100 disbursement sessions was 14 in 2023.39
Rotavirus vaccines will be in stock at >80% of the disbursement days that New Incentives attends in 2023.	50%	11/4/22	Yes	Too pessimistic. Rotavirus was out of stock at 13% of disbursement sessions in 2023.40
Rotavirus vaccines will be in stock at >60% of the disbursement days that New Incentives attends in 2023.	80%	11/4/22	Yes	Accurate. Rotavirus was out of stock at 13% of disbursement sessions in 2023. (see footnote 40)
By December 2023, New Incentives, with input from GiveWell, will implement the program in all five of the new states (Adamawa, Kaduna, Kano, Kebbi, Taraba) after assessing baseline vaccination coverage.	55%	11/4/22	No	Too optimistic. New Incentives did not expand to Taraba or Adamawa by December 2023.41
By December 2023, New Incentives, with input from GiveWell, will implement the program in at least four of the new states (Adamawa, Kaduna, Kano, Kebbi, Taraba) after assessing baseline vaccination coverage.	70%	11/4/22	No	Too optimistic. New Incentives did not expand to Taraba or Adamawa by December 2023.
We will identify a substantial concern (i.e., that causes us to delay this grant past April 2023, or leads to a new CEA adjustment with a >10% change to the average result across all states) as a result of routine review of monitoring metrics or discussion with New Incentives	15%	2/15/23	Yes	Too optimistic. Delayed to May 2023 due to RFMF and exchange rate assumptions.
Likelihood of a grant to New Incentives by end of May 2023 of $35.5 million or more	25%	2/15/23	No	Too optimistic. Grant was $12.6 million.
Likelihood of a grant to New Incentives by end of May 2023 of $33.2 million or more	75%	2/15/23	No	Too optimistic. Grant was $12.6 million.
New Incentives will expand to >50% of the 47 potential LGAs in this grant	75%	5/3/23	Yes	Accurate. Expanded to 66%.42
New Incentives will expand to >75% of the 47 potential LGAs in this grant	25%	5/3/23	No	Accurate. Expanded to 66%. (see footnote 42)
Cost-effectiveness for the 5 states New Incentives has already fully expanded to (Bauchi, Jigawa, Katsina, Sokoto, Zamfara) will remain >10x after we finalize the fourth version of the 2023 CEA for New Incentives	95%	5/3/23	Yes	Accurate.43
Cost-effectiveness will be <10x for 2-4 of the 6 expansion states (Kebbi, Kaduna, Gombe, Taraba, Adamawa, Kano) after we finalize this version of the CEA	70%	5/3/23	Yes	Accurate. Adamawa, Taraba, and Kaduna were <10x.44
We will recommend funding additional caregiver incentives in the form of a livelihood grant by March 2024	85%	1/30/24	No	Too optimistic. Decision delayed to April 2024.

Sources

Document	Source
GiveWell Blog, Our recommendations for giving, 2020	Source
GiveWell, Allocation of Funds Donated to the Top Charities Fund in Q2 2021	Source
GiveWell, CEA of New Incentives' conditional cash transfers to increase infant vaccination, 2024	Source
GiveWell, Cost-Effectiveness Analysis Changelog, 2021	Source
GiveWell, Cost-Effectiveness Analysis Changelog, 2022	Source
GiveWell, Grant page - New Incentives — Nigeria Expansion and Extension, 2023	Source
GiveWell, Grant page - New Incentives — Nigeria, 2022	Source
GiveWell, New Incentives cost per infant immunized, 2024	Source
GiveWell, New Incentives Grantee Review, 2020	Source
GiveWell, New Incentives Intervention Report, 2024	Source
GiveWell, Recommendation to Open Philanthropy for Grants in November 2020	Source
GiveWell, Review of New Incentives monitoring results, 2024	Source
GiveWell, Top Charities	Source
GiveWell, Top Charities - New Incentives, 2024	Source
New Incentives, Annual Report 2020	Source
New Incentives, Annual Report 2023	Source
New Incentives, Program report by state and LGA, Q4 2023	Source
GiveWell, New Incentives' Coverage Assessments: Plans as of October 2021	Source
GiveWell, New Incentives — Expansion to Niger and Yobe States (October 2024)	Source

• 1

  Note that this does not include a small recent grant we made to New Incentives in October 2024, which will fund New Incentives' expansion to 20 new local government areas in Nigeria through 2027.
  As of the time of this writing, the rest of the program was funded through December 2026 by previous GiveWell grants.

• 2

  See our New Incentives intervention report and charity page for more information.

• 3

  "We named New Incentives a top charity this year after considering many factors, including the results of a high-quality study of its program." GiveWell Blog, "Our recommendations for giving in 2020," November 19, 2020

• 4

  See all grants we have recommended to New Incentives in the table on this page.
  $7,820,076 + $12,606,851 + $30,186,650 + $35,000,000 + $9,400,000 + $9,400,000 + $16,800,000 + $1,897,905 = $123,111,482 in total funding from 2020 up to September 2024.

• 5
  • On cost effectiveness: "We estimate that it costs approximately $1,500 to $6,000 (depending on the specific state) to avert a death through the program in the areas where New Incentives works. We also estimate that the program equates to being 9x - 38x as effective as spending on unconditional cash transfers (GiveWell’s benchmark for comparing different programs)." GiveWell, New Incentives (Conditional Cash Transfers to Increase Infant Vaccination)
  • An example of scaling across northern Nigeria: "This grant will fund expansion of New Incentives' current program to new geographies within Nigeria. Prior to this grant, New Incentives had raised funding to work in 92 local government areas (LGAs) through the end of 2024, primarily through GiveWell and our funding partners. The current grant would fund expansion to 67 LGAs by mid-2023, with funding for 12 LGAs through December 2024 and funding for 55 LGAs through December 2025." GiveWell, New Incentives — Nigeria Expansion (November 2022)

• 6

  In November 2020, we recommended a $16.8 million grant to New Incentives to allow New Incentives to scale as fast as it believed it could in 2021 and maintain that scale through 2023. In mid 2021, we made another $9.4 million grant to fund New Incentives' scale up to an additional 18 local government areas, as New Incentives was able to scale more quickly than anticipated with the November 2020 grant.
  We made several additional expansion grants between mid 2021 and 2024, and the pace of expansion has slowed recently due to fewer areas meeting our funding bar rather than due to any logistical constraints on New Incentives' end. For example, in May 2023 we provided funding for New Incentives to expand to an additional 47 LGAs across the states of Adamawa, Kaduna, Kano, Kebbi, and Taraba, conditional on the results of baseline vaccination coverage surveys indicating those LGAs would meet our funding bar. However, Adamawa and Taraba ended up below our funding bar, so as of October 2024 New Incentives has not expanded to either of those states. See our latest cost-effectiveness estimates for Adamawa and Taraba here.

• 7

  In November 2020, we recommended a $16.8 million grant to New Incentives to allow New Incentives to "scale as fast as it believes it can" in 2021 and maintain that scale through 2023. New Incentives expected to use that funding to scale to an additional 17 local government areas. By mid 2021, we made another grant to New Incentives to expand to an additional 18 local government areas, as New Incentives was able to complete the first expansion faster than anticipated. Around 6 months later, we made another expansion grant to New Incentives to support operations in an additional 32 LGAs.

• 8
  • "In 2020, New Incentives enrolled 70,705 infants and issued 323,837 cash transfers after verifying vaccinations." New Incentives, 2020 Annual Report, p. 10
  • "To create the best opportunity for change, New Incentives’ CCTs for RI program is implemented in 3 states in rural North West Nigeria: Katsina, Zamfara, and Jigawa." New Incentives, 2020 Annual Report, p. 5

• 9

  See New Incentives, 2023 Annual Report, p.8-9

• 10

  Our annual review of New Incentives' monitoring data focuses on the following indicators:

  • Retention through the vaccination schedule
  • Repeat enrollments, and other indicators of potential fraud
  • Supply-side issues
  • Reductions in the value of the incentive because of inflation

  We think that these indicators help us understand how the impact of the program may have changed since the RCT. We generally think that New Incentives' monitoring data is high quality as it is comprehensively gathered by New Incentives Field Officers during all disbursements and reviewed as part of New Incentives' expense approvals process.
  See our New Incentives charity page for more on how we use monitoring data.

• 11

  See our New Incentives charity page for more on how we assess the impact and effectiveness of New Incentives' program over time.

• 12

  "The primary evidence for the program's effect size comes from a GiveWell-funded randomized controlled trial (RCT). The RCT found that the program increased overall vaccination rates by 22 percentage points from a baseline level of 36%." GiveWell, New Incentives (Conditional Cash Transfers to Increase Infant Vaccination)

• 13

  See 2021 Cost-Effectiveness Analysis Changelog. Note that in the 2021 CEA we used the term 'false monitoring results' instead of 'biased monitoring results', which we now use in our CEA.

• 14

  We don’t think repeat enrollments are likely to have significant negative effects on the treated population, since (a) we estimate that a relatively small percentage of children enrolled in the program receive repeated immunizations and (b) a brief review of the evidence did not indicate that repeated immunizations are likely to cause significant negative health effects:

  • "The risk of a reaction at the injection site following certain injected vaccines, such as DTaP (diphtheria, tetanus, and pertussis) or pneumococcal vaccine, increases if the doses are not separated by the recommended amounts of time. In these cases, it is the spacing of the doses, not the number of doses, that creates the risk. These reactions can be unpleasant, but they are not life-threatening." Centers for Disease Control, "Ask CDC - Vaccines & Immunizations".
  • "We searched VAERS for US reports where an excess dose of vaccine was administered to a person received from 1/1/2007 through 1/26/2018. . . .More than three-fourths of reports of an excess dose of vaccine did not describe an AHE [adverse health effect]. Among reports where an AHE event was reported, we did not observe any unexpected conditions or clustering of AEs [adverse events]." Moro et al. 2019, abstract.

• 15

  New Incentives, Program Updates, May 2022 - June 2024 (unpublished)

• 16

  See this page for more information on how we think about vaccine stockouts and supply-side issues in our assessment of New Incentives.

• 17

  These surveys measure vaccination coverage prior to New Incentives' expansion to a new area, and then on a regular cadence post-expansion to estimate changes in vaccination rates that have taken place since. More information on how we use these surveys can be found here.

• 18

  We estimate the cost-effectiveness of New Incentives' program is 18.0x in Jigawa, 16.7x in Katsina, and 31.1x in Zamfara, the three states where the randomized controlled trial of the program was conducted. A simple average of these figures comes out to 22x.

• 19

  For each year between 2020 and 2024, we calculated New Incentives' average cost per death averted (based on our final cost-effectiveness analysis of New Incentives for the year) and the total value of grants we made to New Incentives. We divided each year's grant funding by the average cost per death averted to estimate the number of deaths averted based on the amount of funding and our assumptions regarding cost-effectiveness in each year. We then took the sum of the number of deaths averted across years and arrived at approximately 27,000.
  Note that this is a rough estimate of deaths averted, as it assumes the $120 million was evenly divided among all operating states at the time each grant was made. In practice, New Incentives operates at a larger scale in some states than others, so some states will have received a larger share of the total funding without receiving a proportionally larger weight in the cost per death averted calculations. In addition, this assumes that all funding granted to New Incentives in a given year was used under the cost-effectiveness conditions of that year. In reality it is likely that some funding disbursed in a given year was put to use by New Incentives later, under different operating conditions (ex. lower costs) which influence the cost per death averted.

• 20

  To estimate the number of deaths averted assuming 2020 costs per child enrolled, we used our current Simple CEA for New Incentives. We subbed $39.21 (the 2020 cost per child enrolled) in as the cost per child reached and recalculated the cost per death averted in each of the nine states New Incentives is active in. We then calculated the average cost per death averted across these nine states ($7,249). We divided $120 million by $7,249 to arrive at an estimate of approximately 17,000 deaths averted.
  Note that this is a very rough estimate, as it assumes the $120 million was divided equally among the nine currently operating states. In practice, New Incentives expanded to some states a few years earlier than others so they would have received a larger share of the total funding. This calculation also does not account for some states having much larger annual budgets than others (even 2x-3x larger) due to New Incentives operating in more local government areas in those states.

• 21

  We have not listed these in this report.

• 22

  See GiveWell, New Incentives cost per infant immunized.
  Note that our 2024 cost per child enrolled estimate was a forecast, not a calculation based on 2024 cost data (which we expect to analyze in early 2025). We decided to use New Incentives' forecasted cost per child enrolled, adjusted for our best guess of the rate of repeat enrollments, rather than our cost per child estimate based on 2023 costs, because New Incentives added a new 5,000 naira incentive to its program in 2024. You can see the inputs into our 2024 cost per child estimate here.

• 23

  We have inferred that economies of scale have helped drive down the cost per child enrolled as New Incentives has scaled. This seems like a reasonable assumption to make, but we're unsure how much of the reduction in cost per child this might account for as we're unsure what proportion of New Incentives' costs are true fixed costs. We expect many of New Incentives costs (such as staff salaries and transportation costs) do scale alongside growth in the number of children reached.

• 24

  New Incentives shared the following examples of efforts it has implemented to achieve cost efficiencies:

  • Prioritizing hiring Field Officers based closer to clinics to reduce transportation costs
  • Implementing detailed expense reviews (including travel logs) with the aim to reduce leakage and the costs of goods and services
  • Developing a cost model which it uses to conduct regular budget variance analyses, to help manage costs in a high-inflation environment

  We have not vetted the extent to which these measures help New Incentives achieve cost efficiencies.

• 25
  • The Nigerian Naira has depreciated in value substantially against the US dollar since June 2023. In June 2023, the exchange rate was ~460 Naira per US dollar. As of December 19, 2024 the exchange rate is over 1,500 Naira per US dollar.
  • The devaluation of the naira means that New Incentives has been able to get more naira for every dollar donated over time. While the value of the incentive it provides to caregivers has also increased over time, changes to the incentive structure are not automatic and often lag behind reduction in the value of the naira. Additionally, we estimate the cost of incentives to be roughly ⅓ of total program costs (see conditional cash transfers + cash transfer commitments in this spending breakdown), so we do not expect that increasing incentives will fully cancel out the additional purchasing power from naira devaluation.

• 26

  Updating our CEA to GBD 2021 data had mixed impacts on cost-effectiveness; our cost-effectiveness estimates for some states increased by up to 22% while others decreased by up to 29%. Note that we updated our methodology for calculating vaccination coverage rates at the same time that we incorporated GBD 2021 data, so the changes on cost-effectiveness cannot be solely attributed to updated burden data. See more in our 2023 changelog entry.

• 27

  The RCT we use as the basis for our estimate for the impact of New Incentives on vaccination rates can be found here: IDinsight, Impact evaluation of New Incentives, final report, 2020.
  "According to both primary and secondary outcomes, NI-ABAE’s CCTs for RI Program had a substantial positive impact on vaccination coverage. Among children in treatment clinics compared to those in control clinics, self-reported vaccination coverage was 16 percentage points higher for the BCG vaccine [95% CI: 12, 21], 21 percentage points higher for the Penta 1 vaccine [95% CI: 16, 26], and 14 percentage points higher for the Measles vaccine [95% CI: 10, 18]. All of these increases were statistically significant." IDinsight, Impact evaluation of New Incentives, final report, 2020, p. 8.

• 28

  For context, our best guess for vaccine efficacy in Bauchi is 52% which implies a cost-effectiveness of 20x.

• 29

  Gavi has recently expressed more interest in funding demand-side work, and is supporting a small pilot in two Nigerian states that includes immunization incentives (unpublished source), but as far as we can tell this seems to be a pretty small program and we're unsure how much appetite Gavi or others would have for funding something more intensive like New Incentives.

• 30

  For more information, see the Relationships with government section of our New Incentives charity page.

• 31

  In the first half of 2022, New Incentives was offering cash transfers at 87% of scheduled immunization days. GiveWell, New Incentives monitoring results, December 2022, (unpublished).

• 32

  There were 21 stockouts (of any vaccine) per 100 disbursement days in the first half of 2022. GiveWell, New Incentives monitoring results, December 2022, (unpublished).

• 33

  From New Incentives, November 21, 2022 program update (unpublished): "We expect that the rotavirus vaccine has started getting administered at approximately 89% of operational clinics – 2,542 clinics out of 2,845 operational clinics."

  • Note that we are considering "available" in this case to be separate from "in stock" (available refers to whether clinics are now offering the rotavirus vaccine, which was introduced to the Nigerian vaccination schedule in August 2022. There may be clinics that offer the vaccine that are counted in this 89% figure even if there are periods where those clinics do not have any stock on hand). New Incentives noted in its program update that many clinics were experiencing stockouts of the rotavirus vaccine.

• 34

  New Incentives was offering cash transfers at 88% of scheduled immunization days in the second half of 2022. GiveWell, New Incentives monitoring results, December 2022, (unpublished).

• 35

  See GiveWell, New Incentives monitoring results, December 2022

• 36

  See GiveWell, New Incentives monitoring results, December 2022

• 37

  This is a repeat of the forecast we present in the third row of this table, but this is intentional. We made the same forecast during two different grant investigations in 2022, but had different levels of confidence in its likelihood of coming true.

• 38

  See GiveWell, New Incentives monitoring results, January 2024

• 39

  See GiveWell, New Incentives monitoring results, January 2024

• 40

  See GiveWell, New Incentives monitoring results, January 2024

• 41

  See New Incentives, Program report by state and LGA, Q4 2023

• 42

  GiveWell, 6-month check-in on New Incentives grants (unpublished)

• 43

  Cost-effectiveness estimates were 19.5x (Bauchi), 18.4x (Jigawa), 17.1x (Katsina), 42.1x (Sokoto), and 26.2x (Zamfara). GiveWell, Cost-effectiveness analysis version 4, 2023.

• 44

  See GiveWell, Cost-effectiveness analysis version 4, 2023