Published: August 2017
Note: this page summarizes the rationale behind a GiveWell Incubation Grant to Charity Science Health. Charity Science Health staff reviewed this page prior to publication.
Summary
As part of GiveWell's work to support the creation of future top charities, in July of 2017 Charity Science Health received a GiveWell Incubation Grant of $357,543 to support its SMS immunization reminder program in India.
We spoke with Katriel Friedman and Erik Hausen on March 6 and 11, 2019 for updates on this grant.
Table of Contents
The organization
Charity Science Health was founded following the principles of effective altruism, with the explicit goal of becoming "one of the most effective charities at saving lives in a cost-effective and evidence-based way."1 In 2016, GiveWell recommended a $200,000 GiveWell Incubation Grant to Charity Science Health to support its first year of activities. This new grant is intended to cover the total costs for its second year of activities.
The intervention
Charity Science Health is developing a program to send Short Message Service (SMS) immunization reminders to caregivers of infants aged 0-1 in India.
GiveWell's preliminary analysis has indicated that there is some limited evidence for the effectiveness of SMS reminders for vaccination within developing countries.2 While we cannot yet draw broad conclusions about the effectiveness of such interventions, we believe the program is promising and plan to continue our investigation.
Progress and challenges since last grant
Since receiving the initial GiveWell Incubation Grant in 2016, Charity Science Health’s primary challenge has been identifying a cost-effective method of collecting participant phone numbers so that they can receive SMS reminders.3 Charity Science Health spent its first year evaluating several different methods for phone number collection, and is still determining the best approach.4 Charity Science Health had hoped to be in a position to run a randomized controlled trial of its program by early 2018, but now expects this to be delayed until it has identified a cost-effective way to collect participant phone numbers.5
Cost-effectiveness
GiveWell's preliminary cost-effectiveness model estimates that Charity Science Health's program is approximately as cost-effective as the Against Malaria Foundation and approximately 2.5 times as cost-effective as GiveDirectly.6 We are highly uncertain about several inputs important to this cost-effectiveness analysis: the magnitude of effect on vaccination rates to expect in the population targeted by Charity Science Health, the baseline vaccination rate in this population, and disease incidence and mortality rates in this population. We are somewhat uncertain about the costs of implementing an SMS reminders for vaccination program: Charity Science Health expects a fully-loaded cost per enrollee of between $0.25 - $0.50 per participant.7
Grant details
This $357,543 grant is intended to cover the total budget for Charity Science Health’s second year of activities. Charity Science Health plans to allocate its budget roughly as follows:8
- $137,000 for staff costs, including full-time program staff (three people) and a small number of local part-time staff (e.g. to conduct hospital outreach, interact with the government).
- $158,000 for costs associated with participant phone number acquisition.
- $62,000 for overhead expenses (e.g. travel, office space, bank fees, contingency funds).
Goals for the grant
Charity Science Health’s primary goals for the next year include collecting between 8,000 - 10,000 participant phone numbers by the end of 2017, and collecting approximately 100,000 phone numbers by the end of July 2018.9
As with all GiveWell Incubation Grants, GiveWell's goal is to eventually evaluate Charity Science Health as a potential GiveWell top charity.
Remaining questions
Our primary open questions are around:
- Cost-effectiveness: Can Charity Science Health identify a cost-effective way to gather a large number of phone numbers?
- Evidence of effectiveness: Does Charity Science Health's program increase rates of timely vaccination?
Internal forecasts
For this grant, we are recording the following forecasts:
- 80% chance that we will recommend another GiveWell Incubation Grant to Charity Science Health by August 2018.
- 15% chance that Charity Science Health will be a GiveWell top charity by the end of 2019.
Sources
- 1
- "Charity Science Foundation operates as the umbrella organization under which the charities below fall. All of our charities are part of a larger team, have the ultimate goal of helping relieve as much suffering as possible, and were created following the principles of Effective Altruism." Charity Science Foundation homepage
- "Charity Science Health was founded July 2016 after substantial research conducted by the Charity Science team to find the world’s most effective charity intervention… The goal of our project is to become one of the most effective charities at saving lives in a cost-effective and evidence-based way." Charity Science Health blog, June 25 2017
- 2
"There is some limited evidence of the effectiveness of SMS reminders for vaccination within developing countries. Due to the diversity of settings and characteristics of available studies, we cannot draw broad conclusions about the effectiveness of SMS reminders for vaccination in developing countries...This program is promising, and we plan to continue our investigation." GiveWell, Intervention Report on SMS Reminders, June 2017
- 3"The main goal of the first year of the program was to find a viable, cost-effective means of collecting phone numbers at scale." GiveWell's non-verbatim summary of a conversation with Charity Science Health, May 17, 2017, p. 1.
- 4
"The program has considered five general models for collecting phone numbers and other data:
- Buying lists of numbers.
- Hiring staff to work directly in hospitals to collect numbers.
- Partnering with Indian non-profits already using SMS vaccination reminders.
- Partnering with data collection organizations that respond to a request for proposals (RfP). These would work with hospitals to collect phone numbers, then pass them to Charity Science Health.
- Partnering with state governments, which would take responsibility for number collection (by paying hospitals to do it) then send them to Charity Science Health.
" GiveWell's non-verbatim summary of a conversation with Charity Science Health, May 17, 2017, pp. 3-4.
- 5
"A randomized controlled trial (RCT) of the program was originally planned for mid-2018, but is now more likely to happen towards the end of that year. This is primarily because sample size calculations suggest that far more phone numbers will be needed than originally expected. To detect a 3-5 percentage point difference with 90% confidence, a sample of at least 50,000 numbers is necessary." GiveWell's non-verbatim summary of a conversation with Charity Science Health, May 17, 2017, p. 1.
- 6
Cost effectiveness estimate, median of contributors:
Charity Science Health vs Against Malaria Foundation: 0.8
Charity Science Health vs GiveDirectly: 2.7
Cost per equivalent outcome to an under-5 death averted: $3,156GiveWell's preliminary cost-effectiveness analysis of Charity Science Health, 2017, tab "Charity Science Health". - 7 "Costs: The program’s fully-loaded per-enrollee cost is not yet clear, and estimates have increased since the initial cost-effectiveness analysis. It will most likely be between $0.25 and $0.50, with only a small chance of rising above $0.60. It is very unlikely to fall below $0.25 unless the government is heavily involved." GiveWell's non-verbatim summary of a conversation with Charity Science Health, May 17, 2017, p. 2.
- 8
- 9"Charity Science Health hopes to collect 10,000 phone numbers by the end of 2017, though this number could be closer to 8,000. By the end of July 2018, the figure should be around 100,000." GiveWell's non-verbatim summary of a conversation with Charity Science Health, May 17, 2017, p. 1.