Note: This page compares the impact of New Incentives' grants against our initial expectations. New Incentives staff reviewed this page prior to publication.
Note on corrections to this page
We first published this page on July 9, 2025. Later that month, we found several mistakes in our analysis, related to how we framed and calculated our impact estimates. This updated version corrects those mistakes, which we describe in detail below.
Summary
New Incentives offers small cash incentives to increase coverage of routine childhood vaccination in northern Nigeria. Since 2020, GiveWell has directed over $120 million to New Incentives for the years 2020-2026.1 (more) New Incentives is a Top Charity, and we think it is one of the most impactful programs donors can support.2
This page provides a lookback on our grants to New Incentives since 2020. We assess these grants’ performance against initial expectations and identify key lessons learned. We focus on performance during the years 2021-2023, since this allows us to compare actual results from completed programming against our initial expectations.
Since GiveWell began funding New Incentives, it has scaled substantially (from ~70,000 children per year in 3 states in 2020 to ~1.5 million children per year in 9 states in 2023). (More)
We focus our cost-effectiveness comparison on our initial $17 million grant (2021-2023) to provide the clearest retrospective assessment.3 We think this grant was much more cost-effective than we thought when we initially made it. When we made the grant, we estimated New Incentives was ~11x as cost-effective as our benchmark in the original three states we funded. We now think these grants were ~25x. We estimate that our funding has or will avert ~7,900 deaths.4 If we had used our initial assumptions, this estimate would be ~4,300 (more). We would make this grant again, knowing what we know now (more).
The increase in our estimate of cost-effectiveness is driven primarily by a large drop in cost per child enrolled. We expected the cost per child enrolled to be ~$38 in that initial grant and for ~440,000 children to be enrolled in New Incentives’ program. We think New Incentives enrolled ~910,000 children at ~$18 per child enrolled. We think this decline in costs is due to economies of scale, efficiency efforts by New Incentives, and the devaluation of the naira. However, we haven’t deeply investigated the reasons for the decline in costs, and given the impact of this change in our cost-effectiveness estimates, we think this is worth probing further. We also plan to explore whether we’re underestimating the potential for this type of decline in costs at scale for other programs we’ve considered. We plan to do this in 2025. (More)
Additional takeaways and next steps:
- Data collected by New Incentives suggest the program has maintained a similar level of quality even during scale-up (e.g., ~90% retention through the vaccination schedule from 2020 through 2023). (More) However, we’re waiting on surveys that measure vaccination coverage pre and post New Incentives’ entry to see if the effect matches our predictions. We think this will provide a more reliable measure of quality of implementation at scale. We expect to review these data in 2025 (more).
- While we’ve been able to learn a lot about program implementation from conversations and data from New Incentives (more), we think we’ve historically underinvested in speaking to local stakeholders in Nigeria, including government officials, and vaccine experts. We spoke to some local officials as part of our most recent grant to New Incentives and plan to do more of this going forward (more).
- Out of 21 forecasts we made for this grant, we were right about 8 of them (~40%). While we accurately predicted some outcomes like measles retention rates, we were too optimistic about grant sizes and too pessimistic about program quality at scale. (More)
- There are still major uncertainties we haven’t learned more about since making these grants, including (a) the extent to which vaccination rates would have increased anyway without the program and (b) how effective vaccines are at reducing disease and mortality in northern Nigeria. Further work on these could substantially update our assessment of New Incentives’ impact. (More)
Originally published: July 2025, revised October 2025
Background
New Incentives has been a GiveWell Top Charity since 2020.5 Since then, we have directed over $120 million6 to its program incentivizing vaccinations in northern Nigeria.
We made these grants because we think New Incentives' program was a cost-effective way to avert child mortality and because these grants would allow New Incentives to scale across northern Nigeria.7
The rest of this report provides our assessment of our grantmaking to New Incentives from 2021-2023, based on what we’ve learned.
Would we have made these grants again, knowing what we know now?
Yes. Focusing on our initial $17 million grant (2021-2023), our current assessment suggests it was twice as cost-effective as we initially thought. (More).
Looking at the broader program, implementation has gone well despite rapid scaling (from ~70,000 children per year in 3 states in 2020 to ~1.5 million children per year in 9 states in 2023). (More)
Even if we had known cost-effectiveness would be higher than expected, we don’t think it would have been possible to scale the program any faster.8
How did implementation go?
Even as the program scaled from ~70,000 children enrolled in 2020 to ~1.5 million enrolled in 2023, the data we reviewed suggest program quality has remained stable. However, we’re still waiting on data that show effects on vaccination pre and post New Incentives’ entry, which we think will provide more reliable information on New Incentives’ impact over time.
New Incentives told us it made adjustments to respond to some threats to program impact. For example, it implemented stronger anti-fraud measures, including GPS checks and biometric matching, and adjusted its incentive structure in response to inflation and the removal of a fuel subsidy in Nigeria.
We’re concerned about risk of fraud and New Incentives has shared some other implementation challenges, but we don’t think these significantly compromised the effectiveness of the program.
Children enrolled
We believe New Incentives has been effective at scaling its program (both in terms of geographic scope and number of children enrolled) at a rapid pace with GiveWell funding.9 In 2020, New Incentives enrolled 70,705 children across 3 states.10 In 2023, New Incentives enrolled 1,518,900 children across 9 states.11
Monitoring data
We conduct an annual review of New Incentives' monitoring data, and continue to believe this data is both comprehensive and high quality.12 Our review of New Incentives' monitoring data suggests New Incentives was able to maintain high program quality during scale-up. This is based on comparing several indicators between current implementation and during the 2017-2020 RCT, which we use to estimate the effect of New Incentives on vaccination rates.13
In the following table we present a subset of the indicators we use to assess the performance of New Incentives' program over time. We elaborate on these indicators below.
| Category | Indicator | Comparison: result during RCT | Since end of RCT (March 2020 - December 2023) |
|---|---|---|---|
| Retention (enrolled infants complete the vaccination schedule) | Retention rate: PENTA3 | 89% | 89% |
| Supply-side issues | % of disbursement sessions with stockout of any directly incentivized vaccine | 6.0 | 10.9 |
| % of disbursement sessions with stockout of any indirectly incentivized vaccine | 0.4 | 17.0 | |
| Real value of transfer | Value of transfers relative to RCT | - | 77% |
| Fraud | % of disbursement days audited | 5% | 8% |
Effect on vaccination rates over time
We base our estimate of the program's impact on vaccination rates on the RCT results, with a fair amount of uncertainty about whether we'd see the same effects at scale.14 We’re planning to use New Incentives' follow-up vaccination coverage surveys to measure the effect on vaccination rates after New Incentives enters a new state.
Our next steps:
- We’ve recently begun to receive the results of the follow up surveys and expect to more fully review those data in 2025.
- We also plan to compare predicted vaccination rates to vaccination rates reported in the 2023-2024 Nigeria Demographic and Health Survey (DHS) in 2025.
Risk of fraud
Fraud was not called out in our 2020 grantee review as a major reservation/open question, but we did list it as a potential negative or offsetting effect. We now list the risk of fraud as one of our main reservations about the program.
In 2021, we increased our downside grantee-level adjustment from -5% to -7%, with -2% quantifying the risk of biased monitoring results.15 This -2% adjustment was not based on evidence of biased monitoring results, but on gaps in our understanding of how monitoring results are collected and verified, meaning they could be biased in ways we are unaware of.
We estimate the rate of repeat enrollments (one form of caregiver fraud) has increased from 5% in 2020 to roughly 11% in 2023 (see here).16
New Incentives has added new measures to try to detect and deter fraud since 2020. These measures include adding new fraud-related indicators to its monitoring framework, additional checks to detect fraud (from both caregivers and staff) in the disbursement process, and incentives to encourage employees' timely compliance with internal reporting and performance requirements.17
Our next steps: Although we don’t have substantial reservations about the policies New Incentives has put in place and think it seems to be taking the risk of fraud seriously, we think there's a risk that some types of fraud would not get detected and we plan to contract a third party to do a deep dive into New Incentives' monitoring and evaluation in 2025.
Implementation challenges and adjustments to the program
New Incentives has reported implementation challenges over time and adaptations to the program, in response to these challenges and feedback from stakeholders. Some examples are below:
- Vaccine stockouts. New Incentives reported an increase in the percentage of disbursement days experiencing vaccine stockouts, compared to the RCT (~7% vs. ~28%). However, this increase was largely among indirectly incentivized vaccines, such as the yellow fever and rotavirus vaccines, which account for a much smaller share of the program benefits.18 In response to stockouts, New Incentives expanded from local-level management to broader coordination with state and zonal officials, including monthly cold chain management meetings.
- More local staff coverage. New Incentives initially planned a "hub and spoke" structure where Field Officers would cover multiple clinics, but this proved impractical for caregivers and clinics. The organization instead simplified training and processes to hire more Field Officers, expanding from less than 100 clinics during the RCT to 5,862 clinics across 167 local government areas by December 2023.
- Staff fraud prevention. In 2020, New Incentives identified issues with false expense reports and disbursement claims from staff members. The organization implemented new measures including the myDay app for documentation, GPS location checks, enhanced audit protocols, and performance-based assessments for auditors.
- Clinic documentation compliance. New Incentives found inconsistent recording of vaccinations by clinic staff, which could enable multiple claims for the same vaccine. The organization responded by implementing clinic operation pauses for non-compliance with documentation requirements, requiring photo documentation of clinic registers (Tally Sheets), improving vaccine administration observation protocols, and piloting monthly stipends for compliant clinic staff.
- Fraudulent repeat enrollments. New Incentives has implemented multiple checks to try to catch and deter repeat enrollments, including BCG scar checks, photo comparison using Amazon Rekognition, and in 2024 implemented new measures to differentiate between legitimate multiple enrollments by one caregiver and fraudulent ones. We estimate roughly 11% of 2023 enrollments were repeats, and in early 2025 we plan to review 2024 program data to see if this rate has come down.
- Coverage survey implementation. As New Incentives expanded following the RCT, it began conducting its own routine vaccination coverage surveys.19 Field Officers were initially responsible for these surveys, but a dedicated survey team was later created to manage the workload. Over time, the survey team has reduced average survey time from over an hour to around 30 minutes through improved training and streamlined questioning.
- Measles retention rates. New Incentives has faced lower uptake for measles vaccines (87-91% for first dose, 75% for second dose) compared to earlier vaccines in the schedule. The organization made multiple adjustments to its incentive structure to try to encourage greater measles uptake, including differential payment amounts and an additional 5,000 naira incentive for completing the full schedule.
How cost-effective do we think these grants were?
Estimating retrospective cost-effectiveness and deaths averted
We focus our cost-effectiveness comparison on our initial $17 million grant, which funded New Incentives in 3 states (Jigawa, Katsina, and Zamfara) from 2021-2023.
We’re focusing on this grant because we think it allows us to compare our expectations against more complete implementation data. Unlike other grants which are still in progress, the activities funded by this grant are complete, allowing us to compare our initial projections against New Incentives' actual performance with a full set of data.
When we made this grant in 2020, we estimated the program was ~11x as cost-effective as our benchmark, based on projections for cost per child enrolled and other parameters on program impact (e.g., impact of New Incentives on vaccination rates, burden of vaccine-preventable diseases, vaccine efficacy).20 We estimated a cost per death averted of ~$3,900, which implies that ~4,300 deaths would be averted by the grant.21
For this lookback, we re-estimated what we think the cost-effectiveness and cost per death averted of that $17 million would be, given what we know now. To estimate this, we used the following parameters:22
- Actual costs per child enrolled from New Incentives for 2021-2023
- Actual state-by-state enrollment figures from New Incentives for 2021-2023
- Our estimates of how many children would’ve been vaccinated without the program during this period
- Our current estimates of probability a child dies from vaccine-preventable diseases, and the impact of vaccination on mortality
- Our current estimates of how many future deaths are averted by vaccines after age 5
- Additional adjustments based on our current estimates (e.g., we have made adjustments to our models to account for medical costs averted and other adjustments)
Based on these estimates, we now estimate the grant was ~25x our benchmark and ~$2,100 per death averted. This implies ~7,900 deaths have been or will be averted by the grant.23
| Current estimates | Initial estimates | |
|---|---|---|
| Total grant amount | $16,800,000 | $16,800,000 |
| Cost per child enrolled across the program 2021 to 2023 | $18 | $38 |
| Proportion of reached children who would be vaccinated in the absence of the program | 76% | 67% |
| Risk that an unvaccinated child dies from vaccine-preventable diseases before the age of 5, adjusted for indirect deaths | 6.1% | 4.4% |
| Vaccine effectiveness at preventing vaccine-preventable diseases before the age of 5 | 55% | 57% |
| Total deaths averted for children under-5, preliminary | 6,452 | 3,676 |
| Adjustment for deaths averted later in life | 24% | 40% |
| Total supplemental grantee-level adjustment | -7% | -5% |
| Total supplemental intervention-level adjustment affecting lives saved | 24% | 1% |
| Leverage and funging adjustment for counterfactual impact estimates | -14% | -13% |
| Total deaths averted, with final adjustments | 7,937 | 4,297 |
| Cost per death averted | $2,117 | $3,910 |
| Cost-effectiveness estimate in multiples of our benchmark24 | 25 | 11 |
These estimates are intended to be rough and have several caveats:
- This estimate of ~7,900 includes some children who have been enrolled in New Incentives’ program and received vaccines but whose deaths will be averted after 2023.25
- We’re assuming the allocation of funding across states (Jigawa, Katsina, and Zamfara) is proportional to the number of enrollments between 2021-2023 .26 We also assume the cost per child enrolled is the same across these three states. New Incentives expanded to operate in nine states total between 2021 and 2023, and the total costs and enrollments used to calculate actual cost per child enrolled includes the impact of grants we made after 2020. We use these simplifying assumptions because New Incentives does not report spending by state. This could be wrong if New Incentives spent disproportionately more per child in certain states.
- We’re using our current mortality estimates for 2024, rather than retrospectively estimating rates in 2021-2023. If mortality is falling over time, then mortality rates in 2021-2023 were likely higher than our 2024 estimate. This means we may be underestimating the number of deaths averted by the $17 million grant.
- The retrospective analysis compares old and new CEAs, but the underlying methodologies have changed over time. We’ve made adjustments to account for these differences to make the comparisons more direct.27 However, it’s possible we’re overlooked some differences.
Drivers of changes in cost-effectiveness and deaths averted
The main drivers of increases in deaths averted were lower costs (which roughly doubles deaths averted) and, to a lesser extent, higher estimates of the burden of vaccine-preventable disease in unvaccinated children (which increases our estimate of deaths averted among children under 5 by ~25%).28
Costs per child enrolled
In 2020, we estimated New Incentives' program cost ~$38 per child enrolled. Based on actual spending and enrollments from New Incentives from 2021-2023, we now estimate the cost per child enrolled for this $17 million grant was ~$18.
We believe this reduction is a result of economies of scale (as New Incentives has grown its fixed costs have been spread across a larger number of children),29 explicit efforts on New Incentives' part to achieve greater cost efficiencies,30 and devaluation of the Naira increasing New Incentives' purchasing power.31 However, we have not investigated the drivers of this change in depth.
Our next steps:
- Understand whether these are the drivers of change for New Incentives. What share is driven by Naira devaluation vs. other factors? How likely is it that costs will “rebound” to be higher in the future?
- Consider whether there are other programs, beyond New Incentives, where we’re underestimating cost-effectiveness by not anticipating potential for large reductions in costs at scale or where currency devaluations have had a similar effect.
- Investigate whether we can obtain state-specific cost data from New Incentives to better understand cost variations across states and years. This could help us assess whether certain states are driving the cost reductions more than others and improve our estimates of state-level cost-effectiveness.
Burden of disease
In 2020, we estimated that ~4% of unvaccinated children in the three states where New Incentives operated would die from vaccine-preventable diseases before the age of five.32 We now believe this number is closer to ~6%.
This change is due to two updates in our modeling:
- Updated burden of disease estimates from IHME. We previously used national mortality estimates from the Institute for Health Metrics and Evaluation's (IHME) Global Burden of Disease (GBD) study and applied a rough 30% adjustment to account for our assumption that mortality rates are higher in northern Nigeria. Our current cost-effectiveness analysis (CEA) uses more recent, state-level GBD estimates, which are higher than our previous figures. We have not yet investigated how reliable the state-level data are.
- Updates to our estimates of indirect deaths. We assume that vaccines may prevent deaths due to causes other than the vaccine-preventable diseases they directly target. These “non-specific” effects may occur through generally improving children’s health and reducing their risk of death from other sources.33 In our 2020 CEA, we assumed that vaccines averted 0.5 indirect deaths for each death directly averted. We now use a parameter of 0.75 for this assumption. We are highly uncertain about these effects. Our current approach is based on limited data, and we expect to prioritize this for further investigation.
Collectively, these issues increased deaths averted and cost-effectiveness by ~25% total.34
Next steps:
- In 2025, update our approach for which burden of disease estimates we use and how much weight we put on other sources.
- In 2025, conduct a sense check of our state-level estimates against other sources.
- In 2026, explore our indirect deaths assumptions. These continue to be a major assumption in our work, and we have not thoroughly assessed whether these magnitudes, or their relative sizes across top charities, are plausible.
Major uncertainties we haven’t learned more about
In our recent intervention report, we identify several key open questions about New Incentives' program model. We think the following open questions are important but we haven't made significant progress looking into them to date:
- Are we adequately accounting for increases in vaccination rates that would have taken place anyway? The RCT found a substantial increase in control group vaccination rates over the course of the study.35 Other sources of data also show vaccination rates were increasing in northern Nigeria before New Incentives began scaling up. Because we’d expect the program to be less effective in areas with higher baseline coverage, this raises a concern that the impact in the future will be smaller than the impact observed in the RCT. We currently account for this by assuming vaccination coverage will continue to improve by the same rate seen in the years before New Incentives' scale up (2013-2021). However, our estimates could be meaningfully off in either direction. We also haven’t deeply investigated what other programs are being delivered to increase vaccination rates in northern Nigeria, which increases our uncertainty on this issue.
- How effective are vaccines at reducing mortality in Nigeria? We’ve reviewed evidence which suggests that the measles and BCG vaccines might be less effective in Nigeria, and we think it is plausible that vaccine effectiveness may generally be lower in settings where proper storage and handling of vaccines is more difficult. We adjust for this risk in our analysis (reducing our estimate of vaccine efficacy for all vaccines by around 14%). But our adjustment is very speculative and further research could lead us to update this upward or downward. Our 25th - 75th percentile confidence interval for vaccine efficacy against vaccine-preventable disease (which incorporates this adjustment) is 42% to 63%, implying a cost-effectiveness of 16x to 24x in Bauchi.36
- How will the effect of New Incentives' program in states with higher vaccine coverage compare to the effect in the RCT of the program? Our understanding of the effect of New Incentives' program on vaccination rates is largely based on the results of an RCT of the program conducted in Jigawa, Katsina, and Zamfara in 2017-2020. We extrapolate the RCT results to other states based on their vaccine coverage rates, but we are unsure how well these results will translate to new states. New Incentives has begun to conduct vaccine coverage surveys in areas where it works, and we plan to use these as another source to check that the program is having the expected effect at scale. We expect to review follow up coverage survey data in early 2025, once we have results from 24 months of program operations in several areas.
Did we set ourselves up to learn?
We’ve been able to learn a lot about New Incentives’ program because:
- New Incentives conducts regular baseline and follow-up surveys of vaccination coverage, providing valuable data on the program's impact over time. These surveys offer information on vaccination rates and program effectiveness in different areas. We haven’t reviewed these in full yet but believe we’ve set ourselves up to learn from these in the future. (We discuss these above.)
- New Incentives maintains extensive program data collection, allowing for detailed analysis of various aspects of the program's implementation and outcomes. (We discuss these data above.)
There are several areas where we could have set ourselves up for better learning, though we decided not to prioritize these:
- We lack comparison data from non-program areas, which limits our ability to estimate what would’ve happened without New Incentives (e.g., how much would vaccination rates rise counterfactually, and are stockout rates similarly high in areas where New Incentives does not work?)
- Our engagement with other stakeholders in Nigeria, particularly government officials, has been limited. As a result, we may be missing important considerations about how New Incentives' program impacts government systems and priorities, or other organizations working in Nigeria.
- Our engagement with external experts, including epidemiologists and disease modelers, has also been limited. This lack of external input may have limited our ability to gain diverse perspectives and expertise on the program.
Our next steps:
- Speak more with government officials in Nigeria and others with experience working in the states where New Incentives works by early 2025.
- Continue having conversations with global vaccination stakeholders to get their perspectives on New Incentives’.
- Speak more with experts on some of our outstanding questions about vaccine efficacy and other issues in 2025.
What about additional factors influencing our assessment of the impact of the program?
Funging
We have not seen the government of Nigeria or other funders express interest in funding a conditional cash transfer (CCT) program37 on the scale of New Incentives, so we still think this is an area where additional funding is needed.38
External input
We’ve spoken to additional funders and find mixed interest in supporting CCTs for vaccination. Our understanding is that there is some skepticism of the program among some members of the Nigerian federal authorities as well.
- New Incentives faces skepticism of its program from some members of the National Primary Health Care Development Agency (NPHCDA). Critics within the federal government have raised concerns about the potential for backlash if New Incentives discontinues its program in a given area, which could lead to falling vaccination rates.39
- Despite these concerns, New Incentives has been able to continue operating and expanding its program (e.g. New Incentives has been able to sign Memorandums of Understanding (MoUs) with new states, and renew MoUs with the three original states).
- GiveWell has not spoken with Nigerian government officials about the skepticism of New Incentives' program.
- We've asked other funders for their take on CCT programs and have received mixed responses.
- One of the objections to the program (and the one we hear most frequently) is that there might be negative effects when the program is discontinued, by crowding out caregivers' intrinsic motivation to vaccinate their children. In 2024 we updated our CEA to include a -11% to -30% adjustment for potential negative post-program effects in prospective new states. This is a fairly speculative adjustment based on limited evidence, and we don’t think the evidence completely substantiates concerns raised by critics nor suggests that the potential consequences outweigh the benefits of New Incentives' program.
We find the mixed interest in CCTs for vaccination surprising, since we view this program as one of the most cost-effective funding gaps we’ve come across.
How calibrated were our forecasts?
Our forecasting results were mixed. Of the 21 forecasts we made, 8 were accurate. We were too pessimistic about program quality maintenance during scaling. We were too optimistic about stockout rates and grant sizes. We failed to make explicit predictions on key metrics such as enrollment numbers or expansion speed.
Our forecasts are in the table below:
| Prediction | % | Date made | Did this happen? | Notes. If wrong, too optimistic or pessimistic? |
|---|---|---|---|---|
| New Incentives will be unable to operate for >20% of the disbursement days in the clinics it has begun operations in during the first six months of 2022. This may be due to security concerns, telecommunication shutdowns, or other unanticipated factors. | 50% | 1/1/22 | No | Too pessimistic.40 |
| Number of stockouts per 100 disbursement days in clinics that New Incentives works in will be >15 in the first six months of 2022, according to New Incentives' data. | 35% | 1/1/22 | Yes | Too optimistic.41 |
| Rotavirus vaccines will be available and incentivized by New Incentives in >75% of the clinics it operates in by the end of 2022. | 65% | 1/1/22 | Yes | Accurate but slightly pessimistic.42 |
| New Incentives will be unable to operate for >20% of the disbursement days in the clinics it has begun operations in during the second six months of 2022. This may be due to security concerns, telecommunication shutdowns, or other unanticipated factors. | 35% | 5/27/22 | No | Accurate43 |
| New Incentives will have an average measles retention rate >85% in the second six months of 2022. (Measles retention during the RCT was 91%. It has been 81% since the end of the RCT.) | 30% | 5/27/22 | Yes | Too pessimistic. Measles 1 retention was 88% in the second half of 2022.44 |
| Number of stockouts per 100 disbursement days in clinics that New Incentives works in will be >15 in the second six months of 2022, according to New Incentives' data. (This figure was 5 out of 100 days during the RCT and 13 since the RCT ended.) | 25% | 5/27/22 | Yes | Too optimistic. There were ~60 stockouts of any vaccine per 100 disbursement days in the last 6 months of 2022.45 |
| Rotavirus vaccines will be available and incentivized by New Incentives in >75% of the clinics it operates in by the end of 2022.46 | 40% | 5/27/22 | Yes | Too pessimistic. (see footnote 33). |
| New Incentives will have an average measles retention rate >85% in 2023. (Measles retention during the RCT was 91%. It has been 87% since the end of the RCT and was 86% in the first 7 months of 2022.) | 55% | 11/4/22 | Yes | Accurate but with low confidence. Measles 1 retention was 87% on average in 2023.47 |
| Number of stockouts of incentivized vaccines per 100 disbursement days in clinics that New Incentives works in will be >10 in 2023, according to New Incentives' data. (This figure was 5 out of 100 days during the RCT for all vaccines. Since the RCT ended, it's been 9 per 100 days. In the first 7 months of 2022, it was 8.) | 40% | 11/4/22 | Yes | Too optimistic. Stockouts of directly incentivized vaccines per 100 disbursement sessions was 14 in 2023.48 |
| Rotavirus vaccines will be in stock at >80% of the disbursement days that New Incentives attends in 2023. | 50% | 11/4/22 | Yes | Too pessimistic. Rotavirus was out of stock at 13% of disbursement sessions in 2023.49 |
| Rotavirus vaccines will be in stock at >60% of the disbursement days that New Incentives attends in 2023. | 80% | 11/4/22 | Yes | Accurate. Rotavirus was out of stock at 13% of disbursement sessions in 2023. (see footnote 40) |
| By December 2023, New Incentives, with input from GiveWell, will implement the program in all five of the new states (Adamawa, Kaduna, Kano, Kebbi, Taraba) after assessing baseline vaccination coverage. | 55% | 11/4/22 | No | Too optimistic. New Incentives did not expand to Taraba or Adamawa by December 2023.50 |
| By December 2023, New Incentives, with input from GiveWell, will implement the program in at least four of the new states (Adamawa, Kaduna, Kano, Kebbi, Taraba) after assessing baseline vaccination coverage. | 70% | 11/4/22 | No | Too optimistic. New Incentives did not expand to Taraba or Adamawa by December 2023. |
| We will identify a substantial concern (i.e., that causes us to delay this grant past April 2023, or leads to a new CEA adjustment with a >10% change to the average result across all states) as a result of routine review of monitoring metrics or discussion with New Incentives | 15% | 2/15/23 | Yes | Too optimistic. Delayed to May 2023 due to RFMF and exchange rate assumptions. |
| Likelihood of a grant to New Incentives by end of May 2023 of $35.5 million or more | 25% | 2/15/23 | No | Too optimistic. Grant was $12.6 million. |
| Likelihood of a grant to New Incentives by end of May 2023 of $33.2 million or more | 75% | 2/15/23 | No | Too optimistic. Grant was $12.6 million. |
| New Incentives will expand to >50% of the 47 potential LGAs in this grant | 75% | 5/3/23 | Yes | Accurate. Expanded to 66%.51 |
| New Incentives will expand to >75% of the 47 potential LGAs in this grant | 25% | 5/3/23 | No | Accurate. Expanded to 66%. (see footnote 42) |
| Cost-effectiveness for the 5 states New Incentives has already fully expanded to (Bauchi, Jigawa, Katsina, Sokoto, Zamfara) will remain >10x after we finalize the fourth version of the 2023 CEA for New Incentives | 95% | 5/3/23 | Yes | Accurate.52 |
| Cost-effectiveness will be <10x for 2-4 of the 6 expansion states (Kebbi, Kaduna, Gombe, Taraba, Adamawa, Kano) after we finalize this version of the CEA | 70% | 5/3/23 | Yes | Accurate. Adamawa, Taraba, and Kaduna were <10x.53 |
| We will recommend funding additional caregiver incentives in the form of a livelihood grant by March 2024 | 85% | 1/30/24 | No | Too optimistic. Decision delayed to April 2024. |
Notes on corrections to this page
We first published this page on July 9, 2025.54 On July 21, we found mistakes in our analysis. This updated version corrects the following mistakes:
- Our initial framing was misleading. In the previous version of this page, we highlighted initial and current estimates of the full $120 million we’ve spent on New Incentives. We think this was misleading because this includes funds that have not been spent yet and infants that have not been enrolled yet. It’s also challenging to know what initial beliefs to use for these grants.55 As a result, this updated version focuses on the initial $17 million grant from 2021-2023. We think this provides a more firmly retrospective analysis and cleaner comparison of what we thought initially vs. what we think now. We estimate that in those states, our funding has or will avert ~7,900 deaths. If we had used our initial assumptions, this estimate would be ~4,300. (We describe these estimates above.) However, we should note that even this approach still includes some deaths that have not yet been averted, so the analysis is still not completely retrospective. While we believe the bulk of these deaths have been averted already, we think some will be averted in the future due to longer-term protection from vaccines.
- We made a mistake in how we estimated cost per death averted in our analysis of New Incentives’ impact. We think this caused us to significantly underestimate New Incentives’ impact on deaths that have or will be averted. We initially said that our $120 million in funding to New Incentives has or will avert ~27,000 deaths. We now think that estimate is ~45,000. The driver of this change was in how we estimated costs. In our original write-up, we mistakenly used our forward-looking estimates to calculate cost per death averted by the grant (i.e., costs that we estimated at the time of making the grant). We should have been using backward-looking estimates (i.e., costs that we estimated for the grant period after learning more about how much New Incentives actually ended up spending). Because we learned costs fell over time, this means cost per death averted were lower.56
We were first prompted to re-analyze our New Incentives lookback by an external critique of our estimates. While this criticism is not related to the errors we found, it prompted us to look further. We also noticed that New Incentives had stated that it had enrolled its 5 millionth infant (in mid 2025). This was lower than our estimates of 6 million infants (through 2026), which prompted us to continue digging in and surface these other issues.57
Sources
- 1
Note that this does not include a small recent grant we made to New Incentives in October 2024, which will fund New Incentives' expansion to 20 new local government areas in Nigeria through 2027.
As of the time of this writing, the rest of the program was funded through December 2026 by previous GiveWell grants. - 2
See our New Incentives intervention report and charity page for more information.
- 3
Since the activities from this period are now complete, we can make a direct comparison between our initial projections and the program’s actual results using a full set of data, which isn't possible for more recent, ongoing grants.
- 4
While most deaths have already been averted, some future deaths will be prevented because of the long-term protection vaccines provide.
- 5
"We named New Incentives a top charity this year after considering many factors, including the results of a high-quality study of its program." GiveWell Blog, "Our recommendations for giving in 2020," November 19, 2020
- 6
See all grants we have recommended to New Incentives in the table on this page.
$7,820,076 + $12,606,851 + $30,186,650 + $35,000,000 + $9,400,000 + $9,400,000 + $16,800,000 + $1,897,905 = $123,111,482 in total funding from 2020 up to September 2024. - 7
- On cost effectiveness: "We estimate that it costs approximately $1,500 to $6,000 (depending on the specific state) to avert a death through the program in the areas where New Incentives works. We also estimate that the program equates to being 9x - 38x as effective as spending on unconditional cash transfers (GiveWell’s benchmark for comparing different programs)." GiveWell, New Incentives (Conditional Cash Transfers to Increase Infant Vaccination)
- An example of scaling across northern Nigeria: "This grant will fund expansion of New Incentives' current program to new geographies within Nigeria. Prior to this grant, New Incentives had raised funding to work in 92 local government areas (LGAs) through the end of 2024, primarily through GiveWell and our funding partners. The current grant would fund expansion to 67 LGAs by mid-2023, with funding for 12 LGAs through December 2024 and funding for 55 LGAs through December 2025." GiveWell, New Incentives — Nigeria Expansion (November 2022)
- 8
In November 2020, we recommended a $16.8 million grant to New Incentives to allow New Incentives to scale as fast as it believed it could in 2021 and maintain that scale through 2023. In mid 2021, we made another $9.4 million grant to fund New Incentives' scale up to an additional 18 local government areas, as New Incentives was able to scale more quickly than anticipated with the November 2020 grant.
We made several additional expansion grants between mid 2021 and 2024, and the pace of expansion has slowed recently due to fewer areas meeting our funding bar rather than due to any logistical constraints on New Incentives' end. For example, in May 2023 we provided funding for New Incentives to expand to an additional 47 LGAs across the states of Adamawa, Kaduna, Kano, Kebbi, and Taraba, conditional on the results of baseline vaccination coverage surveys indicating those LGAs would meet our funding bar. However, Adamawa and Taraba ended up below our funding bar, so as of October 2024 New Incentives has not expanded to either of those states. See our latest cost-effectiveness estimates for Adamawa and Taraba here. - 9
In November 2020, we recommended a $16.8 million grant to New Incentives to allow New Incentives to "scale as fast as it believes it can" in 2021 and maintain that scale through 2023. New Incentives expected to use that funding to scale to an additional 17 local government areas. By mid 2021, we made another grant to New Incentives to expand to an additional 18 local government areas, as New Incentives was able to complete the first expansion faster than anticipated. Around 6 months later, we made another expansion grant to New Incentives to support operations in an additional 32 LGAs.
- 10
- "In 2020, New Incentives enrolled 70,705 infants and issued 323,837 cash transfers after verifying vaccinations." New Incentives, 2020 Annual Report, p. 10
- "To create the best opportunity for change, New Incentives’ CCTs for RI program is implemented in 3 states in rural North West Nigeria: Katsina, Zamfara, and Jigawa." New Incentives, 2020 Annual Report, p. 5
- 11
See New Incentives, 2023 Annual Report, p.8-9
- 12
Our annual review of New Incentives' monitoring data focuses on the following indicators:
- Retention through the vaccination schedule
- Repeat enrollments, and other indicators of potential fraud
- Supply-side issues
- Reductions in the value of the incentive because of inflation
We think that these indicators help us understand how the impact of the program may have changed since the RCT. We generally think that New Incentives' monitoring data is high quality as it is comprehensively gathered by New Incentives Field Officers during all disbursements and reviewed as part of New Incentives' expense approvals process.
See our New Incentives charity page for more on how we use monitoring data. - 13
See our New Incentives charity page for more on how we assess the impact and effectiveness of New Incentives' program over time.
- 14
"The primary evidence for the program's effect size comes from a GiveWell-funded randomized controlled trial (RCT). The RCT found that the program increased overall vaccination rates by 22 percentage points from a baseline level of 36%." GiveWell, New Incentives (Conditional Cash Transfers to Increase Infant Vaccination)
- 15
See 2021 Cost-Effectiveness Analysis Changelog. Note that in the 2021 CEA we used the term 'false monitoring results' instead of 'biased monitoring results', which we now use in our CEA.
- 16
We don’t think repeat enrollments are likely to have significant negative effects on the treated population, since (a) we estimate that a relatively small percentage of children enrolled in the program receive repeated immunizations and (b) a brief review of the evidence did not indicate that repeated immunizations are likely to cause significant negative health effects:
- "The risk of a reaction at the injection site following certain injected vaccines, such as DTaP (diphtheria, tetanus, and pertussis) or pneumococcal vaccine, increases if the doses are not separated by the recommended amounts of time. In these cases, it is the spacing of the doses, not the number of doses, that creates the risk. These reactions can be unpleasant, but they are not life-threatening." Centers for Disease Control, "Ask CDC - Vaccines & Immunizations".
- "We searched VAERS for US reports where an excess dose of vaccine was administered to a person received from 1/1/2007 through 1/26/2018. . . .More than three-fourths of reports of an excess dose of vaccine did not describe an AHE [adverse health effect]. Among reports where an AHE event was reported, we did not observe any unexpected conditions or clustering of AEs [adverse events]." Moro et al. 2019, abstract.
- 17
New Incentives, Program Updates, May 2022 - June 2024 (unpublished)
- 18
See this page for more information on how we think about vaccine stockouts and supply-side issues in our assessment of New Incentives.
- 19
These surveys measure vaccination coverage prior to New Incentives' expansion to a new area, and then on a regular cadence post-expansion to estimate changes in vaccination rates that have taken place since. More information on how we use these surveys can be found here.
- 20
This estimate is based on our initial CEA accompanying our initial grant in 2020 to New Incentives. See this spreadsheet.
- 21
Our calculations are in this spreadsheet.
- 22
Our calculations are in this spreadsheet.
- 23
Our calculations are in this spreadsheet.
- 24
Note: the calculations we use to estimate the cost per death averted are slightly different from those we use to calculate our cost-effectiveness estimate. This is primarily because of adjustments that we make for deaths averted later in life.
- When calculating the impact in multiples of our benchmark, we use different “moral weights” to weight the benefits of averting a death under age 5, and deaths older than age 5.
- By contrast, when estimating the cost per death averted, we do not apply a different moral weight for older deaths averted.
- In both sets of calculations, we discount future deaths averted later in life.
In this table, we’ve focused on our cost per death averted estimate. You can see how we derive our cost-effectiveness estimate here.
- 25
We haven’t explicitly estimated this value. However, ~20% of the deaths averted we estimate occur after age 5. In addition, there will be deaths averted for children who, for example, received vaccines in 2023 and are protected when they’re 2, 3, 4, and 5 years old.
- 26
To estimate how funds from our 2020 grant were allocated, we use New Incentives’ annual reports, which provide state-by-state enrollment numbers. While our original grant did not fund all of New Incentives’ programming from 2021-2023, and the allocation of funds may have shifted from year to year, we think it is a reasonable approximation of how funds were used.
- 27
We describe some of these adjustments in our side-by-side comparison here.
- 28
See this column which illustrates how changes in each parameter between our 2020 and 2024 CEAs have influenced changes in our cost-per-death-averted calculations.
- 29
We have inferred that economies of scale have helped drive down the cost per child enrolled as New Incentives has scaled. This seems like a reasonable assumption to make, but we're unsure how much of the reduction in cost per child this might account for as we're unsure what proportion of New Incentives' costs are true fixed costs. We expect many of New Incentives costs (such as staff salaries and transportation costs) do scale alongside growth in the number of children reached.
- 30
New Incentives shared the following examples of efforts it has implemented to achieve cost efficiencies:
- Prioritizing hiring Field Officers based closer to clinics to reduce transportation costs
- Implementing detailed expense reviews (including travel logs) with the aim to reduce leakage and the costs of goods and services
- Developing a cost model which it uses to conduct regular budget variance analyses, to help manage costs in a high-inflation environment
We have not vetted the extent to which these measures help New Incentives achieve cost efficiencies.
- 31
- The Nigerian Naira has depreciated in value substantially against the US dollar since June 2023. In June 2023, the exchange rate was ~460 Naira per US dollar. As of December 19, 2024 the exchange rate is over 1,500 Naira per US dollar.
- The devaluation of the naira means that New Incentives has been able to get more naira for every dollar donated over time. While the value of the incentive it provides to caregivers has also increased over time, changes to the incentive structure are not automatic and often lag behind reduction in the value of the naira. Additionally, we estimate the cost of incentives to be roughly ⅓ of total program costs (see conditional cash transfers + cash transfer commitments in this spending breakdown), so we do not expect that increasing incentives will fully cancel out the additional purchasing power from naira devaluation.
- 32
This includes “indirect deaths.” Some studies have found the vaccines incentivized by New Incentives' program have larger impacts on all-cause mortality than would be expected from preventing the diseases they directly target alone. This might be because vaccines have “nonspecific” effects that generally improve children’s health and reduce their risk of death from other causes. We account for this effect with a rough guess that the vaccines targeted by New Incentives indirectly avert 0.75 deaths for every one death directly averted through the diseases they target. See this section of our intervention report.
- 33
See this section of our intervention report for more on our thoughts on indirect deaths.
- 34
See calculations here.
- 35
The RCT we use as the basis for our estimate for the impact of New Incentives on vaccination rates can be found here: IDinsight, Impact evaluation of New Incentives, final report, 2020.
"According to both primary and secondary outcomes, NI-ABAE’s CCTs for RI Program had a substantial positive impact on vaccination coverage. Among children in treatment clinics compared to those in control clinics, self-reported vaccination coverage was 16 percentage points higher for the BCG vaccine [95% CI: 12, 21], 21 percentage points higher for the Penta 1 vaccine [95% CI: 16, 26], and 14 percentage points higher for the Measles vaccine [95% CI: 10, 18]. All of these increases were statistically significant." IDinsight, Impact evaluation of New Incentives, final report, 2020, p. 8. - 36
- For context, our best guess for vaccine efficacy in Bauchi is 52% which implies a cost-effectiveness of 20x.
- See our confidence intervals for each of the parameters in our simplified New Incentives CEA and their implied cost-effectiveness ranges here.
- 37
Conditional cash transfers programs provide cash payments to recipients conditional on specific behaviors or actions. In New Incentives' case, caregivers receive small cash transfers when they bring their infants for routine vaccinations.
- 38
Gavi has recently expressed more interest in funding demand-side work, and is supporting a small pilot in two Nigerian states that includes immunization incentives (unpublished source), but as far as we can tell this seems to be a pretty small program and we're unsure how much appetite Gavi or others would have for funding something more intensive like New Incentives.
- 39
For more information, see the Relationships with government section of our New Incentives charity page.
- 40
In the first half of 2022, New Incentives was offering cash transfers at 87% of scheduled immunization days. GiveWell, New Incentives monitoring results, December 2022, (unpublished).
- 41
There were 21 stockouts (of any vaccine) per 100 disbursement days in the first half of 2022. GiveWell, New Incentives monitoring results, December 2022, (unpublished).
- 42
From New Incentives, November 21, 2022 program update (unpublished): "We expect that the rotavirus vaccine has started getting administered at approximately 89% of operational clinics – 2,542 clinics out of 2,845 operational clinics."
- Note that we are considering "available" in this case to be separate from "in stock" (available refers to whether clinics are now offering the rotavirus vaccine, which was introduced to the Nigerian vaccination schedule in August 2022. There may be clinics that offer the vaccine that are counted in this 89% figure even if there are periods where those clinics do not have any stock on hand). New Incentives noted in its program update that many clinics were experiencing stockouts of the rotavirus vaccine.
- 43
New Incentives was offering cash transfers at 88% of scheduled immunization days in the second half of 2022. GiveWell, New Incentives monitoring results, December 2022, (unpublished).
- 44
See GiveWell, New Incentives monitoring results, December 2022
- 45
See GiveWell, New Incentives monitoring results, December 2022
- 46
This is a repeat of the forecast we present in the third row of this table, but this is intentional. We made the same forecast during two different grant investigations in 2022, but had different levels of confidence in its likelihood of coming true.
- 47
See GiveWell, New Incentives monitoring results, January 2024
- 48
See GiveWell, New Incentives monitoring results, January 2024
- 49
See GiveWell, New Incentives monitoring results, January 2024
- 50
See New Incentives, Program report by state and LGA, Q4 2023
- 51
GiveWell, 6-month check-in on New Incentives grants (unpublished)
- 52
Cost-effectiveness estimates were 19.5x (Bauchi), 18.4x (Jigawa), 17.1x (Katsina), 42.1x (Sokoto), and 26.2x (Zamfara). GiveWell, Cost-effectiveness analysis version 4, 2023.
- 53
- 54
A copy of that page is here.
- 55
For example, when estimating the impact of grants we made in 2023, should we use our beliefs on, say, cost per child reached from 2020 or in 2023?
- 56
These estimates are in this spreadsheet.
To estimate the impact of the full $120m in grants, we took a similar approach to what we described above to estimate the impact of the $17m grant we made in 2020. The differences are:- For the period covering 2020 to 2023, we use actual costs and enrollments reported by New Incentives in each year to estimate program impact (roughly $40m out of the $120m was spent during this period). We include all 9 states that New Incentives operated in across programming years, rather than just focusing on the initial 3 states supported by the $17m grant in 2020.
- For the period covering 2024 onwards, we assume cost per child enrolled would be similar to the average cost per child enrolled between 2020 and 2023 (roughly $19 dollars per child enrolled). We took the remaining $80m in funding that had not been spent by 2023, and allocated it proportionally based on the New Incentives’ projected enrollments for 2027-2028 because we think this is a reasonable approximation for 2024 onwards. (We did not have 2024 enrollment numbers at the time of the original lookback.) By contrast, their historical enrollments between 2021-2023 do not reflect steady state activities in many locations.
- In 2024, we made a $1.3m grant to support New Incentives’ operations in two additional states. This funding and the expected program impact are not included in this lookback.
These estimates are also subject to a similar set of caveats, in particular:
- This estimate of ~45,000 includes some children who will be enrolled in New Incentives’ program between 2020 and 2026, but whose deaths will not be averted until a later year because we expect many vaccines to provide some durable protection against disease.
- We’re assuming an proportional allocation of funding across states, and an equal cost per child enrolled across states. We use this simplifying assumption because New Incentives does not report spending by state.
- We use the same consistent mortality estimates across all years. If mortality is falling over time, this would cause us to underestimate impact in early years (i.e. 2020 and 2023) and overstate impact in later years (i.e. 2024 onwards).
- 57
In our re-analysis prompted by the external critique, we estimated that New Incentives would enroll roughly 6.3m infants as a result of these grants by taking the total amount of funding (roughly $120 million) divided by the cost per child enrolled (roughly $19 per child enrolled) between 2020 and 2024. This assumes that the cost per child enrolled between 2025 and 2026 will remain similar to the historical weighted average.
This differs slightly from our current estimates (~6.5m infants enrolled in the program as a result of these grants) because we used the actual value of the grants ($123,111,482) rather than a rounded figure ($120 million) for our calculations.