Published: August 2020
Note: This page summarizes the rationale behind a GiveWell Incubation Grant to New Incentives. New Incentives reviewed this page prior to publication.
In April 2020, New Incentives received the first disbursement ($506,108) of a GiveWell Incubation Grant of up to $1,897,905 to support its program providing conditional cash transfers (CCTs) for infant vaccination in North West Nigeria. This grant supplements a larger GiveWell Incubation Grant that was made in November 2017 to support New Incentives' operations during a randomized controlled trial (RCT) of its program. The additional funding will sustain New Incentives' operations for the duration of the RCT, plus additional time for us to evaluate the results of the RCT and one extra month for New Incentives to plan its wind-down if we decide not to recommend funding for it in the future. From mid-May until our investigation is complete (between mid-July and December 2020), New Incentives will receive monthly disbursements of $253,054, covering its operating expenses for up to 7.5 months in total.
New Incentives offers conditional cash transfers (CCTs) to incentivize infant vaccination in North West Nigeria.1 There is strong evidence that expanding vaccination reduces illness and death among young children.2 CCTs could plausibly be a cost-effective way to increase vaccination rates among children in areas like North West Nigeria where vaccine supply is adequate but uptake is low.3 Our most recent estimates of the cost-effectiveness of New Incentives' program put it in the range of cost-effectiveness of programs we would consider recommending funding.4
(Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.)
More details on New Incentives' program can be found here.
Past grants to support New Incentives
New Incentives has received several GiveWell Incubation Grants to support different iterations of its program, starting in 2014. More information on past grants to New Incentives and related materials are available here.
November 2017 grant
In November 2017, New Incentives received a GiveWell Incubation Grant of $5,944,203. This grant was intended to cover:5
- Operating expenses for the duration of a randomized controlled trial (RCT), the results of which will be a significant factor in determining whether we recommend funding for New Incentives in the future.
- Additional operating expenses to support New Incentives' program while we make our decision on further funding for New Incentives.
- Exit funding in case we decide not to recommend further funding for New Incentives.
Our initial grant recommendation included an additional $745,074 to account for the possibility that a larger-than-expected number of clinics would be chosen for randomization; this funding was not disbursed.6
More information on the November 2017 grant is available here.
Rationale behind this grant
This grant covers all of the funding that we expect New Incentives to need before we finish processing the results from the RCT and decide whether or not to recommend additional funding to New Incentives.
The November 2017 grant was intended to cover New Incentives' operating expenses through September 2020.7 At the time the grant was made, we believed this would be sufficient time to allow for the completion of the RCT and data analysis, reporting of final results,8 and a wind-down period in case we decided not to recommend further funding.9 Since then, however, New Incentives has extended the timeline for the completion of the RCT, and we have revised our projection of the amount of time required to make our decision, hence the need for additional funding to sustain New Incentives' operations.10
What this grant covers
How much additional funding will be required depends on when the final RCT results are delivered and when we make our decision. Currently, we expect that IDinsight, the organization conducting the RCT, will deliver final or near-final RCT results in early July 2020, and that we will make our decision on whether to recommend future funding for New Incentives no later than the end of November 2020.11 At the time of this grant decision, New Incentives had enough funding to operate through mid-May 2020 before it would need to begin using exit funding;12 it therefore requires up to 6.5 additional months of program costs.
The current grant includes enough funding for up to 7.5 months of program costs, which takes into account the above timeline plus an additional month of wind-down time in case we decide not to recommend further funding (see below). New Incentives will receive monthly disbursements of $253,054 from mid-May until our investigation is complete.13
Updates since the previous grant
The factors contributing to the difference between our expectations for timeline/costs in 2017 and our current expectations are:
- New Incentives requested two months' additional ramp-up time to improve program implementation before starting the RCT. In March 2018, we approved a request by New Incentives to delay the start of the RCT by two months, until July 15, 2018.14 We and New Incentives thought that the additional time would ensure the program was running smoothly before the RCT began, thereby increasing the quality of the data generated.
Although the RCT was initially delayed by two months, New Incentives now needs funding for only two weeks of this extension period, for two reasons:
- IDinsight expects to need one month less for the preliminary data analysis than we expected in 2017, reducing the overall delay from two months to one month.
- New Incentives was able to use budget surplus to cover two weeks of the extension period.15
The current grant includes approximately $125,000 to cover operating expenses from this delay.16
- We have decided to wait on final RCT results to inform our decision, rather than use preliminary RCT results. In 2017, we planned to make our decision on recommending future funding for New Incentives based on preliminary RCT results, which IDinsight provided to us in early April 2020.17 We have since decided to wait for the final results to make our decision, as this will allow us to incorporate additional analyses that are not part of the preliminary results. This will also allow more time for IDinsight and an external reviewer to check for errors in the analyses.
Of the current grant, approximately $500,000 will go toward funding New Incentives for the approximately two-month period expected between preliminary and final RCT results.18
- We increased our expectation of how long it will take us to make a decision on future funding for New Incentives. In November 2017, we expected that it would take one month for us to decide whether to recommend funding for New Incentives in the future, and we allocated funding accordingly. Although we aim to decide as quickly as possible, we now estimate that we may need up to five months after the final RCT results are in to make this decision. The current grant includes up to approximately $1 million to cover New Incentives' operating expenses for an additional four months, if needed.
- New Incentives requested an extension of the wind-down period to allow it more time to inform its partners of plans for shutdown, if necessary. New Incentives requested that one more month's worth of operating expenses (approximately $250,000) be added to the wind-down period, allowing time for it to inform its partners and other stakeholders of the RCT results and plan out the program's shutdown in the event we decide not to recommend further funding.19 New Incentives has told us that it did not anticipate the need for this extra month in 2017.20
Next steps and decision timeline
As noted above, we expect to make a decision on whether to recommend further funding for New Incentives no later than the end of November 2020. The primary consideration in whether we recommend additional funding will be our assessment of how cost-effective New Incentives' program is compared with other programs for which we expect to recommend grants this year and in the next few years. As part of assessing cost-effectiveness, we have outlined our plans for assessing the internal and external validity of the RCT results in an addendum to the main pre-analysis plan. If we decide not to recommend further funding, New Incentives will likely make plans to shut down its program, using the portion of the November 2017 grant allocated for this purpose ($634,660)21 as well as the one month of additional funding included in the current grant.
For this grant, we are recording the following forecast:
75% chance that GiveWell will make a decision by September 30, 2020, about whether to recommend that Open Philanthropy and other donors continue to fund New Incentives.
"Conditional Cash Transfers (CCTs) are small sums of money that individuals in need can earn after meeting various education and health benchmarks such as school attendance or clinic visits. In the case of New Incentives, your donation gives poor mothers in rural West Africa small stipends on the condition that they vaccinate their children against deadly diseases. The stipends allow the women to afford transport to the clinic where the vaccinations are provided and food for their families." New Incentives webpage: How It Works
See GiveWell's 2009 report on Expanding Immunization Coverage for Children, "Program track record."
"There are a number of possible causes of low immunization coverage in Northern Nigeria. As the supply side continues to improve through donor support, it is becoming increasingly apparent that insufficient demand for vaccines is a major driver of low coverage rates." IDinsight's 2017 report of a site visit to New Incentives, Pg 3.
- See this spreadsheet, "New Incentives" tab, “Summary & Results” section, Median results, and this spreadsheet, "New Incentives" tab, “Summary & Results” section, Median results.
The November 2017 grant page notes that the goal of the grant was “to support New Incentives' program of CCTs for infant vaccination in North West Nigeria from November 2017 to May 2020, comprising the full expected duration of a randomized controlled trial (RCT) of the program. IDinsight has separately received funding to conduct the RCT as part of its partnership with GiveWell.”
- See this spreadsheet, "Expected timelines" sheet, top chart, for November 2017 expectations of the timeline that this grant would cover.
- See the budget section of the November 2017 grant page for expectations of what this grant would cover.
“GiveWell recommended a total grant amount of up to $6,689,277, intending for the second tranche of the grant to allow flexibility in the number of clinics to be randomized to receive the program.” New Incentives — General Support (November 2017), "Budget"
“GW Assessment and Combined Decision: March 2020 April 2020; Phaseout if Negative Result: May 2020, June 2020, July 2020, August 2020, September 2020.” New Incentives' November 2017-December 2020 budget, "Nov 2017 Grant: RCT Timeline" tab, rows 30-36.
"$634,660 (11%) to phase out the program in May-September 2020 in the event that the RCT finds a clearly negative result and/or GiveWell decides not to pursue the program further. If the program does not phase out, this will be put toward New Incentives' ongoing operating costs." New Incentives — General Support (November 2017), "Budget"
"In March 2018, GiveWell agreed on extending the ramp-up period by two months and internally approved $638,585." GiveWell's January 2020 grant extension questions for New Incentives and answers from New Incentives' founder and CEO, Svetha Janumpalli
"Shared Timeline for New Incentives RCT Endline Next Steps." Unpublished
“Budget surplus covers our field operations through 15-May-2020, instead of the original timeline of 30-Apr-2020.” GiveWell's January 2020 grant extension questions for New Incentives and answers from New Incentives' founder and CEO, Svetha Janumpalli
See this spreadsheet, "2019 Grant Request" tab, cell E37 for the monthly disbursement amount of $253,054 and cell D37 for the total grant request of $1,897,905, which is 7.5 times the monthly disbursement amount.
"RCT Timeline Outcomes: New Incentives, GiveWell, and IDinsight agreed that the RCT start date will be 15 July 2018." GiveWell's January 2020 grant extension questions for New Incentives and answers from New Incentives' founder and CEO, Svetha Janumpalli
“Using budget surplus, New Incentives has been able to cover an additional 8 RCT clinics and 2 weeks, leaving a need for 1.5 additional months of funding, totaling $379,581 (at $253,054/month).” GiveWell's January 2020 grant extension questions for New Incentives and answers from New Incentives' founder and CEO, Svetha Janumpalli
- Monthly program cost is $253,054 (GiveWell's December 2019 analysis of New Incentives' remaining funding needs), “Calc” tab, “NI, cost per month.” Half of the monthly cost of $253,054 is $126,527.
We plan to wait until the final RCT results are available to publicly share our takeaways from this research. We are not making the preliminary results public at this time.
New Incentives' monthly operating expenses currently average $253,054. See this spreadsheet, "2019 Grant Request" tab, cell E37.
“...we hope to have at least one month of lead time to continue normal operations while we communicate the results and develop an exit strategy with state stakeholders. If we receive a negative decision on July 31, 2020, we would commence phase-out on September 1, 2020.” Email thread between GiveWell and Svetha Janumpalli, Founder and CEO, New Incentives, June-December 2019
“We did not factor in the additional month that would be needed to officially review study results and phase-out plans with stakeholders once receiving a final decision. Stakeholder Relations is a domain in which our expertise has significantly improved since 2017.” GiveWell's January 2020 grant extension questions for New Incentives and answers from New Incentives' founder and CEO, Svetha Janumpalli
“In November 2017, New Incentives received a total of $5,944,203 toward its program through May 2020, which it plans to allocate as follows … $634,660 (11%) to phase out the program in May-September 2020 in the event that the RCT finds a clearly negative result and/or GiveWell decides not to pursue the program further. If the program does not phase out, this will be put toward New Incentives' ongoing operating costs.” New Incentives — General Support (November 2017), "Budget"