You are here

New Incentives

Donate

New Incentives is one of our top-rated charities and we believe that it offers donors an outstanding opportunity to accomplish good with their donations.

More information: What is our evaluation process?

Published: November 2020

Summary

What do they do? New Incentives (newincentives.org) runs a conditional cash transfer (CCT) program in North West Nigeria which seeks to increase uptake of routine immunizations through cash transfers, raising public awareness of the benefits of vaccination and reducing the frequency of vaccine stockouts. (More)

Does it work? A randomized controlled trial (RCT) of New Incentives' program found strong evidence that the program increases vaccination rates. (This RCT was funded by Open Philanthropy on GiveWell's recommendation.) There is also strong evidence that vaccines effectively prevent the diseases they target. New Incentives collects data on an ongoing basis to monitor whether CCTs reach intended recipients. (More)

What do you get for your dollar? We estimate that on average the total cost to immunize one child through New Incentives' program is $47. Excluding vaccination-related costs paid for by the Nigerian government and Gavi, the total cost is $38. The numbers of deaths averted and other benefits of the program are a function of a number of difficult-to-estimate factors, which we discuss in detail below. (More)

Is there room for more funding? We conduct "room for more funding" analysis to understand what portion of New Incentives' ideal future budget it will be unable to support with the funding it has or should expect to have available. We may then choose to either make or recommend grants to support those unfunded activities. (More)

New Incentives is recommended because of its:

  • Focus on a program with very strong evidence of effectiveness and cost-effectiveness.
  • Processes for monitoring that CCTs reach the intended recipients and that other aspects of the program operate effectively.
  • Room for more funding – we believe New Incentives will be able to use additional funds to enroll additional infants.
  • Transparency – New Incentives shares significant information about its work with us, and we are able to closely follow and understand its work.

Major open questions:

  • Compared to our other top charities, New Incentives is at an earlier stage of its organizational development. In the next year, we expect it to roughly double the number of infants it reaches. This could lead to changes in cost per enrolled infant, effectiveness of its incentives, or unexpected problems. It may scale up less quickly than we expect, reducing its room for more funding.
  • New Incentives' impact depends on what proportion of caregivers would not have vaccinated their infants in New Incentives' absence and what portion of those caregivers change their behavior in response to cash incentives and information. The RCT of the program was designed to measure these proportions by employing a control group and randomly selecting which clinics received the program. As the program scales up, it will become more difficult to measure New Incentives' impact. This situation differs to some degree from that of many of our other top charities, which deliver health commodities that have reasonably well-understood biological mechanisms.
  • Government opposition is a potential risk to New Incentives' ability to operate its program. New Incentives has the support of several government agencies in Nigeria, including the state agencies whose permission it requires to work in clinics in each state and several national health and social protection agencies. Support for CCTs for immunization within the government of Nigeria is not universal, however, and its critics have raised concerns about whether CCTs are the most effective strategy for increasing immunization rates and whether CCTs, once ended, could lead to backlash against clinic staff or to caregivers declining to vaccinate infants without incentives. We believe these are important questions and have decided to recommend the program after considering them.

Our review process

We first recommended a grant to New Incentives in 2014. Our review process has consisted of:

  • Making seven Incubation Grants to support New Incentives' development (details here).
  • Funding IDinsight to lead, in partnership with Hanovia Limited, a randomized-controlled trial of the impact of New Incentives' program on infant vaccination rates in 2018-2020.
  • Extensive communication with New Incentives' US-based leadership, founders Svetha Janumpalli and Pratyush Agarwal.
  • Reviewing documents New Incentives shared with us about its operations, monitoring, costs, plans for the future, and risks it faces.
  • Conversations with academics who have worked on evaluating programs to increase demand for vaccinations and a funder that has worked on immunizations in Nigeria.

All content on New Incentives, including details of past grants, blog posts and conversation notes, is available here.

What do they do?

New Incentives runs a conditional cash transfer (CCT) program in North West Nigeria. Caregivers who bring their infants to clinics for routine vaccines, which are provided through government clinics free of charge, can receive a total of $11 over the course of five clinic visits. New Incentives also does outreach to caregivers about the importance of vaccinating children, and works with its government partners to improve vaccine supply by identifying and addressing bottlenecks in the vaccine supply chain.

New Incentives started this program in May 2017. The program was evaluated by a randomized control trial (RCT) from July 2018 until February 2020.1 In 2019, New Incentives enrolled approximately 90,000 children, made 380,000 disbursements, and disbursed about $820,000.2 It currently works in three states: Katsina, Zamfara and Jigawa.

In Nigeria, New Incentives is known as All Babies Are Equal (ABAE) Initiative.3

Vaccines incentivized

New Incentives directly incentivizes (i.e., makes cash transfers conditional on infants receiving) the following vaccines:

  • BCG (against tuberculosis)4
  • PENTA (against diphtheria, tetanus, whooping cough, hepatitis B, and Haemophilus influenzae type b)5
  • PCV (against pneumococcal disease)6
  • MCV (against measles)7

These vaccines are part of Nigeria's routine immunization schedule and are provided by the government in public clinics (more on this below). Some of these vaccines are provided in more than one dose. New Incentives also indirectly incentivizes the Hepatitis B vaccine, OPV and IPV (the vaccines against polio), and the yellow fever vaccine (these vaccines are on the same schedule and are administered at the same visits as the directly incentivized vaccines; see schedule below).8 The meningitis A vaccine (MenA) was introduced into the Nigerian routine immunization schedule in 2019 as a single dose administered at 9 months (the same time as the measles and yellow fever vaccines) and is also one of the indirectly incentivized vaccines.9

Immunization schedule

Age of visit Directly incentivized Indirectly incentivized
At birth BCG Hep B, OPV0
6 weeks PENTA1, PCV1 OPV1
10 weeks PENTA2, PCV2 OPV2
14 weeks PENTA3, PCV3 OPV3, IPV
9 months Measles Yellow Fever, MenA

Between 2020 and 2021, Nigeria is expected to add two doses of rotavirus and a second dose of measles (MCV2) to the schedule.10 New Incentives plans to indirectly incentivize the rotavirus vaccine;11 it has not yet decided how it will incentivize MCV2 and plans to pilot different options.12

Size of conditional cash transfers

New Incentives provides CCTs worth 500 naira (approximately $1.40) for each of the first four vaccine visits (from birth to 14 weeks) and one CCT worth 2000 naira (approximately $5.50) for a fifth visit for the measles vaccine (at 9 months).13 New Incentives decided on the transfer amount for the first four visits with the goals of covering the cost of transportation and providing a small additional incentive. The larger incentive amount for the measles vaccine was decided upon taking into account lower baseline vaccination rates for that vaccine, a relatively long time gap between the fourth and fifth vaccination visits, and the measles vaccine's high impact on mortality.14

Activities

The main goals of New Incentives' program are:

  1. increasing vaccine demand by (a) distributing CCTs and (b) raising awareness about the importance of vaccinating children and the availability of CCTs for vaccination
  2. improving vaccine supply by identifying and addressing bottlenecks in the vaccine supply chain

We cover these components in turn below.

Distributing CCTs

Vaccines incentivized by New Incentives are provided by the staff of government clinics that partner with New Incentives. Vaccinations and disbursements take place on days during which clinics provide routine immunizations ("immunization days").

When a child is vaccinated, New Incentives checks whether the child meets the eligibility criteria to receive a CCT and, if so, disburses a CCT to the child's caregiver. New Incentives also conducts audits and remote vetting of cash disbursements to ensure disbursements reach eligible caregivers.

Partnership with clinics

New Incentives pursues partnerships with clinics that:

  • offer routine immunization services15
  • are located in an area where the clinic is safe to visit during the day16
  • are no more than three hours away from an ATM, bank, or point-of-sale terminal17

During the RCT, each of New Incentives' partner clinics aimed to serve infants in its own catchment area (that is, the geographic area whose population it is intended to serve), and New Incentives aimed to disburse incentives during all immunization days at partner clinics. At scale, New Incentives plans to determine the location and frequency of disbursements on the basis of the number and location of clinics and the estimated population that can access these clinics rather than aiming to disburse incentives during all immunization days at all clinics. New Incentives believes that a large proportion of infants who received disbursements during the RCT came from outside the catchment areas of the partner clinics. Since, at scale, partner clinics in a local government area (LGA) will be located closer to each other than during the RCT, New Incentives expects they will at times decrease the frequency of disbursements to sustain the same level of cost per child at scale.18

Eligibility for enrollment

Children are enrolled in New Incentives' program during their first vaccination visit (for BCG, for example, see the schedule above). The eligibility criteria for an infant to enroll in the program are:

  1. The infant's caregiver is a resident of the clinic's catchment area (going forward the criterion will be residency in the LGA in which the clinic is located). This is checked by looking at the residence reported on the Child Health Card (see below) by the clinic staff and by asking questions of caregivers.19 New Incentives employs this criterion to avoid excessive vaccination volumes at its partner clinics.20
  2. The child has received a BCG vaccine during that immunization day (and has not previously received a BCG vaccine). Because infants and caregivers do not consistently have documents that would allow New Incentives to identify them, New Incentives uses a fresh injection mark as evidence that a child has been injected with BCG during the day, and it uses the scar left by the BCG vaccine as a way of checking that children are not enrolled more than once.21
Eligibility for CCT disbursement

To be eligible to receive a CCT, an infant must:

  1. Be enrolled in New Incentives' program. New Incentives checks that infants are enrolled by asking caregivers to provide the infant's ABAE ID (see below).22 This ensures New Incentives can identify each child and avoids repeatedly disbursing incentives to the same child for the same vaccination. (See the eligibility criteria for enrollment above.)
  2. Have received a directly incentivized vaccination during the same immunization day. New Incentives verifies this by checking that there is a vaccination marked with the current date and a golden dot on the child's Child Health Card (see below).23
  3. Be older than nine months, if receiving the measles vaccine. New Incentives verifies this by comparing the current date with the date of birth reported on the Child Health Card.24 The Nigerian government recommends that children receive measles vaccinations after 9 months of age.25 (We have not closely reviewed the evidence for this claim.)
Immunization days

Disbursements are provided during immunization days.26 The process is as follows:27

  1. New Incentives typically assigns one field officer (FO) to a clinic.28 On immunization day (or the day before), the FO collects cash from an ATM, bank, or point-of-sale terminal before traveling to the clinic.29
  2. At the start of the immunization day, clinic staff hold a health talk for caregivers, during which the FO introduces the New Incentives program.30
  3. Clinic staff provide new children with a Child Health Card (CHC). A CHC is a document provided to caregivers that reports the infant's residence, date of birth, and vaccinations the infant has received on which dates.31
  4. Infants are vaccinated according to the routine immunization schedule. When administering the vaccine, the clinic staff also write down the vaccination date on the CHC and mark it with a golden dot near the record of the vaccination.32
  5. After vaccinations, the FO enrolls new infants and disburses cash to the caregivers of eligible infants. (See the eligibility criteria for enrollment and disbursement above.) Newly enrolled children are provided with an ABAE Card (a card that includes basic information about the New Incentives program and immunization schedule) and an ABAE ID (an identification number printed on stickers that are applied to the CHC and the ABAE Card).33
  6. If the infant is being enrolled, the FO takes a picture of the infant's BCG injection mark. For all disbursements, the FO takes a picture showing the caregiver holding the cash, the child, and the ABAE Card.34 Later, the pictures are checked by remote staff (see below).35

During the RCT, New incentives disbursed approximately 80% of incentives on immunization days at clinics.36 At scale, New Incentives does not expect this to change drastically.37

Process compliance and anti-fraud checks

New Incentives monitors immunization days through:

  • Console checks: Remote staff (called "console agents") check (a) that cash was disbursed to caregivers by cross-checking different sources that track disbursements and reviewing evidence submitted by FOs,38 and (b) that caregivers who received cash were eligible, by reviewing the pictures taken by FOs during immunization days documenting caregivers' eligibility.39
  • Audits: New Incentives employs staff in the role of auditors who visit each catchment roughly every two months to assess compliance with protocols and check for evidence of fraud by (a) observing activities; (b) interviewing caregivers, clinic staff, and New Incentives staff; and (c) conducting a money spot check (i.e., verifying that the FO is in possession of the amount of cash expected based on New Incentives' records).40
  • Fraud investigations: Cases of suspected fraud are collected through the console checks, audits, and staff complaints.41 Those are then assessed by auditors, who review the evidence to establish whether there is reason to believe fraud occurred.42

Awareness-raising activities

New Incentives' awareness-raising activities aim to increase the demand for routine immunization by increasing awareness of the program and sharing information with caregivers about immunization.43

During immunization days, New Incentives conducts the following awareness-raising activities:

  • During the health talk, New Incentives' FO introduces the program and explains eligibility requirements.44
  • After enrollment, FOs distribute promotional plastic bags to increase program visibility.45
  • After CCTs are disbursed, FOs communicate and/or write down on caregivers' ABAE Cards the date of the next vaccination.46
  • New Incentives advertises the program by putting up posters outside the entrance of partner clinics.47

New Incentives runs "awareness meetings" in targeted communities aimed at increasing knowledge of the program and addressing concerns about vaccinations. The meetings are run by New Incentives staff and clinic staff.48 Awareness cards (including basic information about the program) are distributed during these meetings.49 New Incentives also organizes targeted outreach sessions, which are vaccination sessions that occur at settlements, aimed at addressing low vaccination rates in certain areas.50 Vaccines for these outreach sessions are delivered by clinic staff, who are reimbursed by New Incentives for transportation costs.51

New Incentives also recruits community members to increase program awareness and to track infants who are behind schedule for receiving vaccinations and encourage their caregivers to complete the immunization schedule. Community members receive a stipend of $1.40 to $2.80.52 New Incentives also hires "town criers" to make announcements informing caregivers about immunization days and outreach sessions. New Incentives provides stipends for town criers of $1.40 per week of engagement.53

New Incentives shared the following estimates of frequency and reach of awareness activities:54

Activity Engagements per clinic, future expected frequency Estimated number of caregivers reached per engagement
Immunization day activities 2 per week 30
Awareness meetings 1 every 3 months 20-30
Community member engagements 1-2 per month 9
Targeted outreach sessions 1 per month 11
Town criers 1 every 1-2 months 20

New Incentives also shared data on how new program participants say they became aware of the program:55

Program Awareness Source %
Neighbor 48%
Friends 39%
Family 31%
Traditional birth attendant 11%
Village leader 11%
Town crier 7%

Supporting vaccine supply

New Incentives' supply-side work targets vaccine supply at the local, zonal, and state levels for BCG, PENTA, PCV, OPV, measles, and yellow fever vaccines.56 This work consists of collecting data on issues with the supply of vaccines, and investigating and addressing the problems tracked. Supply chain issues are identified via:57

  • Clinic-level data: The day before a scheduled immunization day, New Incentives' FO calls partner clinic staff to check whether the clinic has vaccinations in stock for the following day.58 During each immunization day, FOs fill out a form about the vaccine stock at the clinic.59
  • Data at the apex clinic, zonal, and state levels: Every two weeks, New Incentives staff call cold chain60 officers working at "apex clinics" (larger clinics that store vaccines for clinics with more limited storage capacity), and at the LGA, zonal and state levels to collect information about vaccine stocks.61

Based on the data collected, New Incentives staff identify problems with the vaccine supply, such as stockouts, low vaccine stocks, or local authorities not having received the required documentation from clinics (in order to receive vaccine supply, clinics need to submit reports detailing their utilization and supply needs).62 New Incentives then informs relevant decision-makers about problems identified63 and, occasionally, provides financial support to relieve bottlenecks (for instance, by paying for transport costs to deliver vaccines).64

Staff structure

New Incentives has roughly 170 staff.65

New Incentives' Operations Unit is in charge of distributing CCTs and running awareness-raising activities, as well as providing support for supply-side activities. Employees in this unit include:

  • 120 Field Officers, who are responsible for disbursing CCTs, running awareness activities, and collecting information about vaccine supply during immunization days.66
  • Ten Field Managers, who are responsible for supervising field officers, reviewing expenses, and identifying and addressing problems with enrollment, retainment, and vaccine supply at the clinic level.67
  • Three State Field Managers, who are responsible for supervising field managers, tracking targets, and managing the state budget.68

Other New Incentives staff members include:69

  • Two Supply-Side Officers, who are in charge of identifying issues based on data collected by Field Officers and maintaining regular contact with cold chain officers at the area, local, and zonal levels.
  • Two Stakeholder Relations Officers, one Stakeholder Manager, and one Stakeholder Relationship Director who are responsible for managing stakeholder relations and resolving supply-side issues at the local and state levels.
  • One Security Manager, who is responsible for management of staff security and safety procedures.
  • One Security Focal Person who liaises with Field Officers and stakeholders to obtain and relay security information and advisories.
  • One National Coordinator, who manages stakeholder relations and resolves supply-side issues at the national level, in addition to the general coordination role between operations, security, stakeholder relations, and supply-side activities.
  • One software engineer.

New Incentives staff who review the data collected by FOs include:70

  • One Console Manager and four Console Supervisors.
  • 21 Console Agents.
  • Three auditors.

New Incentives' also has two human resources staff members.71

Spending breakdown

Below we break down New Incentives' spending between November 2017 (when the program began) and February 2020 (the most recent data available as of this report).

Expenses, November 2017 to February 202072

Expense category Total (m) %
Conditional Cash Transfers (CCTs) $1.7 29%
Staff compensation $1.5 26%
Transport and supply costs for in-clinic and awareness activities $1.2 21%
Contractors and consultants, including console agents73 $0.5 9%
Stakeholder relations (meetings and vaccine transport) $0.2 3%
Other $0.7 12%
Total expenses $5.7 100%

Does it work?

On a separate page, we discuss the evidence on the effectiveness of New Incentives' program. We conclude that there is strong evidence that the intervention increases vaccination rates. Below, we discuss factors we plan to evaluate to assess the program's ongoing impact over time and possible negative and offsetting effects of the program.

Is the intervention effective?

A randomized controlled trial (RCT) of New Incentives' program found strong evidence that the program increases vaccinations. (This RCT was funded by Open Philanthropy on GiveWell's recommendation.) Combined with evidence that (a) vaccine-preventable diseases are a significant cause of child mortality in North West Nigeria, and (b) vaccines effectively prevent the diseases they target, we believe New Incentives is likely to reduce child mortality in North West Nigeria. More details are available in our intervention report.

Assessing the program's impact over time

New Incentives collects data on various aspects of its program's performance. In this section, we highlight certain types of data that New Incentives collects that we plan to use to help us assess how the program's impact over time compares to that measured during the RCT.

Retention throughout the vaccination schedule

Why this matters: If retention rates decrease over time, this may indicate that New Incentives' program is becoming less effective at incentivizing immunizations.

Available evidence: New Incentives estimates retention rates by calculating the percentage of enrolled infants who receive later vaccines.74 At enrollment, New Incentives provides infants with an ABAE ID (see above). When providing disbursements, New Incentives collects the infant's ABAE ID data, allowing it to tag the disbursements to a specific infant.75 We plan to compare New Incentives' retention rates over time to its retention rates during the period of the RCT.76

Repeat enrollments

Why this matters: It is possible that some infants are enrolled in the program more than once, either at the same or different clinic locations.77 We would guess that infants are unlikely to benefit from receiving the same vaccination more times than it is scheduled to be received, so if we count infants who are enrolled more than once as unique infants, then we will overestimate the cost-effectiveness of the program.

Available evidence: Caregivers have an incentive to enroll their infants in the program multiple times in order to receive additional CCTs. New Incentives aims to prevent the same infant being enrolled multiple times by (a) only allowing enrollment during a visit in which the infant receives the BCG vaccine, and (b) checking new enrollees for BCG scars, which indicate that the infant received the BCG vaccine previously. This system is imperfect because most but not all infants develop BCG scars and because scars take a couple of weeks to form, leaving a window for re-enrollment. To check the frequency of the same infant being enrolled multiple times, Field Officers look at infants' arms when they return for subsequent vaccinations and record the number of BCG scars on each arm.78

The World Health Organization (WHO) reports that about 90% of infants vaccinated with BCG develop a scar.79 Data that New Incentives has collected indicates that 97% of infants had a BCG scar when they came for subsequent vaccinations, while nearly no infants had two or more scars,80 which suggests that a small proportion of infants received the BCG vaccination more than once. Based on this data, we estimate that 8% of disbursements were for repeated vaccines.81 We increase this estimate slightly, to 10%, to account for clinics being closer together during the program implementation than they were during the RCT, making it easier for caregivers to travel to more than one clinic to enroll.82

New Incentives has told us that it plans to conduct periodic household surveys to estimate vaccination coverage in the areas where it works, which could potentially detect large differences in the number of children enrolled in the program and the number of children found to be vaccinated in the community. This may shed further light on the frequency of infants receiving the same vaccination more than once. We do not yet know the details of how these surveys will be conducted.83 New Incentives is also considering employing biometric identification of caregivers as an additional method of identifying infants and preventing infants from being enrolled more than once.84

Supply-side issues

Why this matters: An increase in the number of vaccine stockouts and/or infants not served per immunization day may indicate that the program is becoming less effective at causing additional infants to be vaccinated.85

Available evidence: New Incentives' FOs collect data on stockouts (i.e., vaccines missing at the beginning of an immunization day), runouts (i.e., vaccines running out during an immunization day), and infants not served at the end of each immunization day. New Incentives estimates that there were five stockouts, one runout, and 20 infants not served per 100 immunization days during the period of the RCT.86 This data:

  • covers 97% of immunization days87
  • includes stockout and runout data for all vaccines directly incentivized
  • primarily relies on FOs' direct observations, supplemented with clinic records and conversations with nurses as needed88

Reduction in the value of the transfer due to inflation

Why this matters: Inflation might cause the real value of New Incentives' CCTs to decrease, which may weaken their effectiveness as an incentive.

Available evidence: In addition to reviewing independent information on inflation rates in Nigeria, we plan to review data that New Incentives collects on caregivers’ transport costs as evidence of changes over time in overall costs in the areas where New Incentives works. Caregivers’ reported transport costs are collected during immunization days by FOs who ask each caregiver how expensive it is for them to reach the clinic. There may be an incentive for caregivers who traveled to the clinic from non-eligible areas to underreport their true transportation costs (since those costs could imply that they traveled from further away than they claim).89

Fraud at the expense of program participants

Why this matters: If fraud reduces the value of the CCTs that caregivers receive, New Incentives' ability to incentivize caregivers may decrease. Fraud of this type could include FOs not giving caregivers the full transfer amount or clinic staff, FOs, or others getting kickbacks after transfers.

Available evidence: Sources of evidence on this type of fraud include:

  • Data on CCT disbursements: After disbursing a CCT, FOs take pictures of the caregiver holding the cash, the infant, and the ABAE ID. The photos are then reviewed by console agents.90 New Incentives reports that console agents have comprehensively reviewed the pictures and have observed caregivers holding the expected cash amount in 99% of cases.91
  • Data on "dashes": New Incentives FOs ask caregivers, after each disbursement, whether they gave out "dashes," or tips, to clinic or New Incentives staff. Of the respondents, 0.1% reported giving dashes.92
  • Data from audits: During audits, auditors collect data on fraud through (a) direct observation and (b) interviews of clinic staff, New Incentives staff, and program participants. This includes asking program participants whether they have noticed fraud by New Incentives or clinic staff. Auditors are instructed to conduct caregiver interviews out of earshot of FOs and clinic staff. We have seen reports for audits covering 5% of immunization days; 4.8% of those audits reported cases of suspected New Incentives or clinic staff fraud.93
  • Data on New Incentives' fraud mitigation measures: See below for details.

Potential negative or offsetting effects

In our report of the program’s effectiveness, we discuss potential negative and offsetting effects included in our cost-effectiveness estimate. Below, we discuss potential negative and offsetting effects of the program that we believe to be too small or unlikely to have been included in our quantitative model.

Risk of HIV-infected children developing disseminated BCG disease

Children who are HIV-infected when vaccinated with BCG at birth are at increased risk of developing disseminated BCG disease (a disease with symptoms resembling tuberculosis, the disease against which the BCG vaccine is used). According to the World Health Organization, up to 4 in 1000 HIV-positive infants vaccinated with BCG develop disseminated BCG disease, and the disease has a case-fatality rate greater than 70%.94 Our understanding is that HIV prevalence is relatively low in the areas where New Incentives works.95

Side effects from repeated immunizations

As discussed above, we believe there is some risk of caregivers bringing infants to receive the same vaccine multiple times within a short interval. This might negatively affect infants' health beyond the ordinary side effects of vaccinations. However, we expect that this is unlikely to have significant negative effects on the treated population, since (a) we estimate that a relatively small percentage (roughly 10%) of children enrolled in the program receive repeated immunizations (more below) and (b) a brief review of the evidence did not indicate that repeated immunizations are likely to cause significant negative health effects.96

Security threats to staff

New Incentives works in areas at moderate to high risk of security threats.97 New Incentives reports that, over roughly two and a half years, it has recorded 23 incidents that were connected to the program in some way and that resulted in theft, injury, or death.98 The list includes incidents that involved program staff outside their work capacity as well as those that did not involve program staff but may relate to the program. Sixteen of the 23 incidents involved theft, including minor theft such as phones and cash. Five of the 23 incidents involved injury, and two involved deaths, including an incident in which two people died and three people were kidnapped. This incident did not involve New Incentives staff directly, but the assailants mentioned a vaccination cash transfer program as a reason for the attack. It is unclear from the report whether they said this because they intended to steal the CCT money or for some other reason.99

New Incentives procedures to decrease risks to its staff include:

  • Collecting information about potential security threats and communicating with staff about threats. Information about security incidents is sourced on an ongoing basis by FOs,100 as well as by New Incentives' Security Manager and Security Focal Point.101 New Incentives has shared the list of these incidents with us.102 The list includes an average of 40 reports per month from June 2019 (when this logging system was introduced) to October 2020 (the date of our review); it was up to date at the time of our review. We spot-checked records for 50 incidents: for all of them, it was indicated whether further steps should be taken to mitigate the risk and, if so, what action had been taken. When an ongoing and urgent threat is identified, the Security Manager directly contacts relevant FOs.103 To communicate less-urgent security information, the Security Manager compiles weekly summaries, which are circulated to managers, who share them with FOs.104 At times, New Incentives designates high-risk areas as "no go" and prevents FOs from creating work plans or submitting expenses for these areas in the app used for this purpose.105 We have seen the Security Manager's weekly reports for five weeks in 2020; these reports describe the process used to identify threats and designate areas as "no go."106
  • Training staff to avoid security threats where possible and address them where necessary. The onboarding process for New Incentives staff members includes a UN safety training course and a training course on New Incentives' internal security procedures.107 Ongoing monthly training is also provided, covering topics including road safety and abduction and kidnapping.108 We have seen the Security Manager's weekly reports on onboarding training for five weeks in 2020.109

Discontent of people who are not served on a particular day

It is possible that New Incentives' program causes discontent among caregivers who have to wait a long time at clinics or who are not served during a particular immunization day. We would guess the negative effects from discontent of this kind are likely small, because:

  • There appear to be only a limited number of children who leave without being served. FOs collect data on infants not served at the end of each immunization day. During the RCT, New Incentives estimates that, on average, 20 infants were not served for every 100 immunizations days during which New Incentives disbursed cash.110
  • When interviewed by New Incentives auditors, only 4% of caregivers complained about long wait times or not being served during a certain immunization day.111

"Crowding out" other motivations for vaccinating children

It is possible that, by creating a financial motivation to vaccinate infants, New Incentives' program "crowds out" intrinsic motivations to vaccinate infants. This might potentially lead to lower vaccination rates after the program is discontinued in an area than there would have been if the program had not been implemented. New Incentives told us of one case indicating this might be a concern in the area where it works: some caregivers reportedly refused to vaccinate their infants after in-kind incentives for a polio vaccination campaign were suspended.112 We would guess this potential effect would be partly offset by New Incentives' work to educate caregivers about immunization.113 Overall, we judge it unlikely that this consideration would significantly offset the program's benefits, though we have not investigated this question in detail.

Increased fraud and theft

New Incentives' program might lead to increased fraud or theft, including:

  1. Fraud by caregivers, clinic staff, and/or New Incentives staff at the expense of New Incentives (e.g., non-eligible caregivers tampering with Child Health Cards to receive transfers)
  2. Fraud by clinic staff and New Incentives staff at the expense of caregivers (e.g., clinic staff and New Incentives staff taking a portion of the disbursement)
  3. Theft by third parties (e.g., stealing the cash transfer from caregivers or New Incentives’ staff)

New Incentives' procedures to prevent the first type of fraud are described here, and procedures to prevent the second type are described here. New Incentives reports nine cases of theft by third parties affecting New Incentives staff to date.114 New Incentives does not collect information about theft by third parties affecting program participants; it believes the risk is low, in part because caregivers only receive relatively small sums.115

Increased vaccine supply shortages in areas where New Incentives does not work

It is possible that New Incentives' program increases the likelihood of vaccine supply shortages in areas that New Incentives does not work in, via a combination of (a) CCTs and awareness-raising activities increasing demand for vaccines in areas New Incentives works in, and (b) vaccine supply support provided by New Incentives increasing vaccine supply flow to the areas it works in by diverting supply from other areas.116

As part of their supply support activities, New Incentives works to improve the communication between states, local government areas, and clinics in the states where it works. We would guess this benefits a wider set of areas than the ones where New Incentives works.117 As a result, we would guess it is unlikely that New Incentives has a significant negative effect overall in areas in which it doesn't work. However, we have not seen data addressing this directly.

Crowding out World Bank CCT funding

The World Bank has provided funding for a CCT program targeting very low-income households in Nigeria, under which each state can choose the condition for the transfers. Our understanding is that Katsina chose an education-focused condition for this program over a health-focused condition in part because it judged the health sector to already be better served than the education sector; knowledge of New Incentives' program may have affected that assessment. (Jigawa has chosen health-focused conditions for the transfers, while Zamfara had not selected a condition as of June 2020.)118 It is possible that New Incentives could have a similar effect on other government decisions in the future, in ways that are hard to predict. However, we believe this consideration is unlikely to significantly offset the benefits of New Incentives' program, since (a) our understanding is that New Incentives was only one of several organizations whose activities affected this decision, and (b) the World Bank program is expected to reach less than 1% of the population.119

What do you get for your dollar?

What is the cost per child immunized?

We estimate that, on average, the total cost to immunize one child through New Incentives' program is $47.120 Excluding costs paid for by the Nigerian government, Gavi, and other donors who fund vaccinations in Nigeria, the total cost is $38.121 This estimate relies on a number of uncertain assumptions.

Our approach

Our estimate of the total costs of the program includes:

  • All costs paid by New Incentives.
  • The proportion of the costs paid by other donors that would not have been incurred in New Incentives' absence. We would guess those are mostly shouldered by the government and Gavi. These actors incur costs for each child vaccinated, but since some of the caregivers who receive incentives would have had their children vaccinated in any case, the Nigerian government and Gavi only incur additional costs related to vaccination for children who would not have been vaccinated in New Incentives' absence.122

We start with this total cost figure and apply adjustments in our cost-effectiveness analysis to account for cases where we believe the charity's funds have caused these other actors to shift funds from a less cost-effective use to a more cost-effective use ("leverage") or from a more cost-effective use to a less cost-effective use ("funging"). Since the costs borne by the Nigerian government and Gavi account for a relatively small percentage of total program costs, our cost-effectiveness estimate is not highly sensitive to these questions, so we have not investigated them very deeply.

Some of the key assumptions we've made in estimating the cost-effectiveness of New Incentives' program over the next few years include the following:

  • We expect New Incentives' future costs to be similar to its costs in the recent past. We divide New Incentives' total costs from June 2019 through May 2020 by the number of infants enrolled in the program during that time, as measured by the number of cash disbursements for the BCG vaccination.123 This is a key uncertainty in our model, since New Incentives is planning to scale up its program over the next few years, and operating at a different scale may lead to significant increases or decreases in charity costs.
  • We incorporate a 10% adjustment to account for our estimate of the proportion of children who have been enrolled in the program multiple times (see above for more detail).124
  • We apply two adjustments to account for anticipated changes to the program over the next few years: (a) decreased staff transportation costs, since New Incentives' partner clinics will likely be closer together than they were during the RCT, and (b) a smaller number of infants served per immunization day, since the smaller distance between clinics will likely lead to fewer infants coming from outside of a clinic's catchment area. These adjustments roughly cancel each other out.125
  • We use data published by the government of Nigeria to estimate its costs and Gavi's costs.126

What is the cost per death averted?

See our cost-effectiveness model for estimates of the cost per death averted through New Incentives' incentives for immunization program.

Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. For a list of factors excluded from our model, see this section of our report on New Incentives' incentives for immunization program.

There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

Is there room for more funding?

We conduct "room for more funding" analysis to understand what portion of New Incentives' ideal future budget it will be unable to support with the funding it has or should expect to have available. We may then choose to either make or recommend grants to support those unfunded activities.

Room for more funding analysis

In general, we assess top charities' funding needs over a three-year period.127 We ask top charities to report their ideal budgets over the next three years, along with information about their current available funding and funding pipeline. The difference between a charity's three-year budget and the funding we project that it will have available to support that budget is the charity's "room for more funding."

The main components of our room for more funding analyses are: