Published: June 2018
Note: This page summarizes the rationale behind a GiveWell Incubation Grant to Evidence Action Beta. Evidence Action staff reviewed this page prior to publication.
As part of GiveWell’s work to support the creation of future top charities, in March of 2018, Evidence Action Beta received a GiveWell Incubation Grant of $320,000 to scope a project to provide technical assistance for the Indian government's work on large-scale school-based iron and folic acid supplementation targeting children and adolescents.
Background on Evidence Action and the intervention
Evidence Action has built relationships with Indian policymakers through its work supporting the scale-up of deworming in India. This grant would allow Evidence Action to scope an opportunity to provide similar kinds of technical assistance for a different program: iron and folic acid (IFA) supplementation.
Evidence Action told us it has received requests from its contacts in the Indian government to work on this program, but that it needs funding to scope how it could add value to the program (e.g., how much it would cost to provide technical assistance, how much Evidence Action might be able to increase the coverage rate of the program, what data is available on baseline coverage rates, etc.).
This seems to us to potentially be an extremely cost-effective opportunity. Because the Indian government's programs affect a large number of people, improving its IFA program could have a very large impact. Our rough, early-stage estimate suggests that this kind of work may be about 12x as cost-effective as cash transfers.1
Our investigation process for this grant was relatively light, in part because we see Evidence Action as a highly-aligned partner, Evidence Action considers this work high-priority, and it seems intuitively plausible to us that providing support for Indian government programs could be highly cost-effective.
Core questions we focused on in investigating this grant included:
- How cost-effective is this program?
- How will the grant funds be spent?
- What will the key deliverables from this grant be to help us decide whether to provide further funding for the program in the future?
These questions are explored in more depth below.
How cost-effective is this program?
In brief, our early work suggests that Evidence Action's support for IFA supplementation in India could be extremely cost-effective (roughly 12x as cost-effective cash, or more)2, though we have much more uncertainty about most of the key parameters than we do for our top charities' cost-effectiveness analyses (CEAs).
Our current best guess is that directly paying for IFA supplementation in India would be about 9x as cost-effective as cash.3 This estimate relies on a number of uncertain judgments, including:
- How much it costs to deliver IFA for a year. We've estimated about $2 per person per year (about 2x the high-end cost estimate from a study that Evidence Action sent us on similar work in India).4
- How much weight in disability-adjusted life years (DALYs) to give anemia. We've modeled moderate and severe anemia as having substantial disability associated with them (DALY weights of ~0.05 and ~0.15, respectively), and mild anemia as having a fairly small DALY weight of ~.004.5
- The impact of iron supplementation on malaria risk. As we've discussed previously, there seems to be a possibility that iron supplementation can increase malaria risk.6 We believe this is a relatively small concern in India, where malaria death rates for children under five are about 22x lower than in sub-Saharan Africa.7 We include these potential small negative effects in our bottom-line estimate.
- How significant IFA supplementation's effect is on long-term income via cognitive benefits.8
Our rough model of the cost-effectiveness of Evidence Action's technical assistance work to support India's IFA program estimates it is about 12x as effective as cash.9 Key uncertainties in our model include:
- How much it costs. We've assumed an annual cost of roughly $500,000 to assist the median Indian state for 5 years, which is similar to the Deworm the World Initiative's costs.10 (Our model also includes the $320,000 cost of this grant.)
- How much Evidence Action can increase coverage. We've estimated a 5 percentage point coverage increase for 3 years, but this is very uncertain.11
- How much additional government spending toward IFA Evidence Action's activities might cause. We've estimated a 10% increase in government funding for 3 years from a low base of total spending,12 but believe this may be on the conservative side. Evidence Action is uncertain how much potential leverage its activities could have over government spending.
Our main takeaway from the above CEAs is that, despite remaining uncertainty, it seems unusually plausible that this program could be extremely cost-effective. We consider it worth funding Evidence Action to gather more information that will help us refine our cost-effectiveness estimates.
How cost-effective is this grant?
Our rough calculation, based on the predictions below, implies that this grant is roughly 23x as effective as cash if there's a 10% chance that it will cause us to reallocate $15 million to an opportunity that is 10x as effective as cash instead of an opportunity that is 5x as effective as cash (which is roughly the cost-effectiveness of current marginal spending on our top charities).13
These calculations assume that the cost of this grant alone ($320,000) will allow us to determine whether to reallocate $15 million toward this intervention in the future. In fact, we think we'll likely need to provide some additional funding before determining whether to reallocate a large amount of funding to this program, but we also expect to have a better sense in the future of the probability that this work is in fact highly cost-effective.
How will the grant funds be spent?
Evidence Action expects to spend this grant roughly as follows:
- ~$220,000 (~69%): Personnel costs
- ~$25,000 (~8%): Travel, data collection, etc. (Evidence Action has told us it wants to do some data collection on pre-existing coverage rates of India's IFA program; we aren't sure whether this line item includes other costs.)
- ~$30,000 (~9%): Regional costs
- ~$50,000 (~15.5%): Indirect costs (standard overhead)
What will the key deliverable from this grant be?
Evidence Action has mentioned the following as key deliverables:
- A funding proposal that describes how Evidence Action would improve India's IFA program via technical assistance, what this would cost, timelines for the project, etc.14
- Evidence Action's own CEA of how cost-effective this work could be, as well as key data that could help us refine our cost-effectiveness model, such as:
- The prevalence of anemia in various age groups in India (though Evidence Action only expects to be able to share existing data from the Government of India on this)
- Data on current IFA program gaps, particularly around trainings, drug supply, reporting procedures, and community awareness
- Estimates of the current coverage of India's IFA program (only in the 10 states in which Deworm the World already provides technical assistance)
- Estimates of additional coverage increases resulting from Evidence Action's technical assistance, based on identified gaps in India's Weekly Iron and Folic Acid Supplementation (WIFS) Programme
- Estimates of government costs and technical assistance costs
- Estimates of any potential cost-sharing opportunities with Deworm the World
- A concept note for the Indian government with a proposal for Evidence Action taking on this technical assistance.
- A document about other donors that might be able to fund this work.
Plans for follow-up
We plan to check in at the conclusion of the grant period (roughly in July) to learn how the scoping project went and determine whether it seems promising to provide additional funding. Key questions for follow-up include:
- What kinds of technical assistance might Evidence Action do to support India's IFA program? Would it be working at the national level or in particular states? How might it try to improve the existing program?
- What would IFA coverage rates and government funding for IFA likely be if Evidence Action did not provide technical assistance?
- How much does Evidence Action think it can increase coverage rates and funding for this program, and why? On what timeframe?
- If Evidence Action expects to increase funding from the Indian government for this program, what does it expect those funds would have been spent on otherwise?
- How much funding would Evidence Action need to work on this program in the future? If the program is successful, what is the total potential room for more funding?
- Is it worth GiveWell providing further funding for this work? Are there other potential funders for this work?
- How does more in-depth analysis affect our CEA of IFA?
For this grant, we are recording the following forecasts:
|50%||We believe direct funding of IFA in India is no more than 2x less cost-effective than we believe it is now (as discussed above, we currently estimate it's roughly 9x as cost-effective as cash transfers, using our January 2018 CEA as a baseline).||March 2019|
|75%||Evidence Action believes that it can add substantial value to India's IFA program and requests over $500,000 for a follow-up grant to move past scoping.||March 2019|
|40%||Conditional on Evidence Action getting funding to do technical assistance past the scoping stage, Evidence Action ultimately spends at least $6 million total on IFA technical assistance that GiveWell models as 10x as effective (or more) than cash transfers (using our January 2018 CEA as a baseline).||January 2023|
|20%||Conditional on Evidence Action getting funding to do technical assistance past the scoping stage, Evidence Action ultimately spends at least $15 million total on IFA technical assistance that we model as 10x as effective (or more) than cash transfers (using our January 2018 CEA as a baseline).||January 2023|
|~10%||Implied from other calculations: Evidence Action ultimately spends at least $15 million total on IFA technical assistance that we model as 10x as effective (or more) than cash transfers (using our January 2018 CEA as a baseline).||January 2023|
|Aguayo, Paintal, and Singh 2012||Source|
|GiveWell's Evidence Action India BOTEC||Source|
|GiveWell's rough IFA CEA||Source|
|GiveWell, Failure in international aid||Source|
GiveWell's Evidence Action India BOTEC, cell B48.
GiveWell's Evidence Action India BOTEC, cell B48.
GiveWell's rough IFA CEA, sheet "Main," cell B95.
"With the addition of other programme components…programme costs increased, ranging in most programmes from $US 0.30 to $US 0.90 per adolescent girl per annum." Aguayo, Paintal, and Singh 2012, pg. 1671
See GiveWell's rough IFA CEA, sheet "Main," cells B15:B17.
"A formal evaluation of an iron supplementation program [Sazawal 2006] found that supplements caused higher rates of hospital and admission and death; the authors concluded that the iron supplements had made children more vulnerable to malaria. This study led to a change in international guidelines for iron supplementation." GiveWell, Failure in international aid
See GiveWell's rough IFA CEA, sheet "Supplementary data," note in cell B16.
For our rough calculations, see GiveWell's rough IFA CEA, sheet "Main," rows 27 to 44.
GiveWell's Evidence Action India BOTEC, cell B48.
See GiveWell's Evidence Action India BOTEC, cell B5, and the explanation in cell C5.
See GiveWell's Evidence Action India BOTEC, cells B24:B25, and the descriptions in cells C24 and C25.
GiveWell's Evidence Action India BOTEC, cell B34.
See GiveWell's Evidence Action India BOTEC, rows 52 to 72.
This assumes that Evidence Action determines this work is promising; Evidence Action may decide, after its initial scoping, not to pursue the project further, in which case it would send us a document explaining why.