Deworm the World Initiative, Led By Evidence Action - 2013 Review | GiveWell

# Deworm the World Initiative, Led By Evidence Action - 2013 Review

We have published a more recent review of this organization. See our most recent report on Deworm the World.

The Deworm the World Initiative, led by Evidence Action, is one of our top-rated charities, and an organization that we feel offers donors an outstanding opportunity to accomplish good with their donations.

Published: November 2013

## Summary

What do they do? The Deworm the World Initiative, led by Evidence Action (DtWI), advocates to and supports governments in their implementation of mass school-based deworming programs. (evidenceaction.org/deworming)

Does it work? We believe that deworming is a program backed by relatively strong evidence. We believe that DtWI raises the probability that deworming programs in India go forward (based mostly on conversations with government officials as well as the fact that DtWI has been involved in all but one statewide deworming program currently operating in India), and that such programs are reasonably well executed (based on monitoring reports collected by DtWI, which we have limited confidence in). Our evidence on both fronts is highly limited.

What do you get for your dollar? We estimate that children are dewormed for a total of about $0.35 per child, with DtWI paying under 10% of these costs. We believe the per-child positive impact of deworming is likely much smaller than the impact implied by the strongest study on deworming, because compared to the study, the areas DtWI is working in or planning to work in generally have lower prevalence and intensity of worm infections (and a complete absence of schistosomiasis, one of the key infections treated in the study). Room for more funds: DtWI has told us that it has a funding gap of$2-3 million. DtWI seeks unrestricted funds to respond to unanticipated situations, including requests from governments for technical support as well as to experiment with ways to increase coverage and optimize their approach to reducing infections. Quantifying the marginal impact of these funds is extremely difficult. In addition, donors should note that DtWI is only one initiative of an organization, Evidence Action, whose other work we have not evaluated as closely.

DtWI is recommended because of its:

• Focus on a program with a strong track record and excellent cost-effectiveness (more).
• Standout transparency - it has shared significant, detailed information about its programs with us.
• Room for more funding - DtWI appears to have a need for additional funding (more).

Major unresolved issues include:

• The difficulty of assessing the impact of an organization whose role is advocacy and technical assistance, rather than direct implementation. There are substantial potential advantages to supporting such an organization, as it may be able to have more impact per dollar by influencing government policy than by simply carrying out programs on its own, but this situation also complicates impact assessment. While we believe DtWI to be impactful, our evidence is limited, and in addition, there is always a risk that future expansions will prove more difficult than past ones.
• The apparent fact that DtWI is working in states with existing disease control programs including some that administer the drug used in deworming.
• The marginal value of additional unrestricted funds for DtWI.
• The fact that we find the data on the quality of the deworming programs that DtWI has been involved in to be relatively limited.

### Our review process

To date, our investigation process has consisted of

• Conversations with Evidence Action Executive Director Alix Zwane, DtWI Director Grace Hollister, Evidence Action Director of Strategic Initiatives Christina Riechers, DtWI India Country Director Priya Jha, DtWI India Operations Director Ayan Chatterjee, DtWI Strategy and M&E Associate Director Sharad Barkataki in 2013, and with Deworm the World’s Managing Director Alissa Fishbane and Regional Director Karen Levy in 2012.
• Conversations with the Children's Investment Fund Foundation (CIFF), a funder of DtWI, about DtWI.
• Reviewing documents DtWI sent in response to our queries.
• Site visits:
• In November 2012, we visited DtWI's office in Nairobi, Kenya and met its staff there.
• In September 2013, we visited DtWI's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with DtWI.

Older content on DtWI:

## What do they do?

The Deworm the World Initiative is led by Evidence Action; its aim is to advocate for and provide technical assistance for implementation of deworming programs for school-aged children.

DtWI was founded in 2007,1 and is currently active in Kenya and India.

Its program in Kenya is fully supported by the Children’s Investment Fund Foundation (CIFF) and the Ending Neglected Diseases Fund through 2016,2 so we focus this review on its activities in India, the primary work that DtWI is hoping to expand in the near future.

Since 2009, DtWI has worked to (a) encourage Indian state governments to implement statewide school-based deworming programs,3 (b) design and conduct prevalence and intensity surveys to determine the need for deworming in specific areas,4 (c) provide technical assistance to state governments’ implementations of deworming programs, (d) provide additional support for the monitoring and evaluation of the programs.5

The bulk of program costs are paid for by Indian state governments; DtWI primarily pays for the activities listed above. We estimate the following breakdown for DtWI’s spending in India (note that all personnel costs are classified as program management, so non-program management lines represent non-personnel direct costs):6

 Program management: India personnel 30.6% Program management: Global personnel 22.9% Program management: Travel and other mgmt 18.9% Prevalence survey 9.7% Training 6.9% Program management: Indirect costs 4.4% Monitoring and evaluation 4.1% Awareness and mobilization 1.9%

### Activities by state in India

Historically, DtWI has supported deworming programs in Andhra Pradesh, Bihar, Delhi, and Rajasthan. As of October 2013, DtWI was actively supporting programs in Bihar, Delhi, and Rajasthan.7 Our review focuses on the four most recent deworming rounds that DtWI has assisted with (excluding rounds in Delhi and Rajasthan in October 2013 for which data is not yet available as of this writing): Bihar in 2011 and 2012, Delhi in 2012, and Rajasthan in 2012.

### DtWI and Evidence Action

In early 2013, Innovations for Poverty Action announced the formation of Evidence Action, a new organization, which is spinning out programs implemented at scale from Innovations for Poverty Action, and will continue to house and grow two such programs: DtWI and Dispensers for Safe Water.8 We focus this review on DtWI and discuss the room for more funding implications of DtWI’s being a program of a larger organization below.

Evidence Action projects a total budget of $11.8 million for 2014, of which$6.7m is DtWI.9 Of this, $3.0 million is allocated to Kenya (which is fully supported by CIFF and the END Fund, see above).10 ## Does it work? DtWI supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. There is a very strong case that mass deworming programs in general are effective in reducing infections. The evidence on the connection to positive quality-of-life impacts is less clear, but there is a possibility that deworming is strongly beneficial. Note that our intervention report focuses on combination deworming (i.e., programs that treat both schistosomiasis and soil-transmitted helminths (STH)). DtWI is expanding in India, where it treats only STH due to the extremely limited presence of schistosomiasis.11 Below, we discuss the considerations relevant to comparing deworming in India to Sub-Saharan Africa. Here, we focus on the following questions: • What proportion of children in areas DtWI works/will likely work are infected? What is the intensity of these infections? The areas in which DtWI works appear to qualify for deworming under WHO guidelines, though the prevalence and intensity of infections is generally substantially lower than it was in the study that serves as the best evidence of deworming's impact. More in our section on cost-effectiveness. • Do the programs DtWI supports successfully deworm children? What evidence of impact for its programs has DtWI shared? The best data we have on the quality of deworming programs comes from reports submitted by DtWI-compensated monitors, and suggests fairly strong (though far from perfect) implementation. However, we have substantial reservations about this data, as well as about the reliability of the government data we are using to estimate the number of children treated. (We understand that collecting reliable data at scale can be quite costly and that there is a trade-off between evidence quality and cost-effectiveness.) • DtWI is an advocacy/technical assistance organization, not a direct service provider. What impact does DtWI’s involvement have on program implementation? We have very limited evidence with which to answer this question. We believe it is highly likely that DtWI's work improves the likelihood of a given Indian state's going forward with deworming, based on speaking with government officials, speaking with a source on the general dynamics of Indian school health programs, and the observation that DtWI has been involved in all but one of the Indian statewide deworming programs to date. We are more uncertain as to whether DtWI improves the quality of deworming programs, feeling there are some reasons to believe it does and some reasons to believe it does not. ### What proportion of children are infected? What is the intensity of these infections? DtWI aims to conduct prevalence surveys for each Indian state in which it works before the first round of treatment to determine the prevalence and intensity of infections from each STH and therefore inform appropriate treatment strategies.12 These aim to survey a representative sample of the population. DtWI has shared the technical details of these surveys with us, and we have spoken with the researchers who are currently advising DtWI and its partners on future prevalence surveys; methodological details in this footnote.13 The following table provides results from the original prevalence surveys DtWI conducted in Bihar in 2010 and 2011, Delhi in 2011, and Rajasthan in 2012. Footnotes next to each state name provide additional detail on unique circumstances for the state’s surveys. STH (intensity)14 Bihar15 Delhi16 Rajasthan17 Hookworm prevalence (moderate/high) 0.7% 0.0% See footnote18 Ascaris prevalence (moderate/high) 8.8% 1.2% 0.1% Trichuris prevalence (moderate/high) 0.2% 0.3% 0.0% Any STH prevalence (moderate/high) 9.7% 1.5% 0.09%19 Hookworm prevalence (any) 42.2% 1.3% See footnote20 Ascaris prevalence (any) 52.1% 11.6% 12.6% Trichuris prevalence (any) 0.2% 5.6% 0.1% Any STH prevalence (any intensity) 67.5% 15.8% 12.6%21 Note that the WHO recommends the following treatment frequencies based on prevalence of any STH, which DtWI advises governments to follow:22 • Less than 20%: no mass treatment • 20%-49%: annual treatment • Greater than 50%: biannual treatment23 ### How many children were reached? As of October 2013, DtWI had assisted with seven rounds of deworming in four states in India. We have seen data on how many children received treatment in four of these rounds. Delhi and Rajasthan each held deworming days in October 2013 and coverage data was not yet available as of this writing (in October 2013). We did not ask to see coverage data from the first round of treatment DtWI assisted with, in Andhra Pradesh in 2009. All coverage figures are reported by the state governments. We report state-level figures here, though DtWI also shared more granular data (for example, at the district and block level).24 State Year # of school-aged children # of pre-school children # of adults % reporting Notes Bihar25 2011 16,775,003 - 361,408 Not reported - Bihar26 2012 16,265,432 - 601,956 97% of blocks - Delhi27 2012 1,954,155 756,266 159,235 82% of schools and pre-schools Adults figure includes teen girls Rajasthan28 2012 6,950,363 3,866,056 None reported 99% of blocks - #### Uncertainty in coverage figures The coverage figures above may over- or underestimate actual coverage. The number of children treated in each school is reported by school staff to government officials who aggregate data from many schools and pass a tally for their administrative area to a higher level of government. This process is repeated several times, and the lowest level at which the disaggregated data is preserved is generally blocks, which each contain hundreds of schools.29 It is not clear what error-checking, if any, is conducted by government officials at each level of aggregation, although some blocks may be erroneously counted as having done no deworming since the tally was never reported. Errors could theoretically be introduced at any stage, either inadvertently or purposefully, such as by officials inflating numbers or guessing when reports are low or missing. To monitor the quality of coverage figures, DtWI sends monitors to audit a sample of schools, generally a few days after the deworming days, and asks them to record the number of students dewormed according to class registers (on which teachers mark off students’ names when they receive pills) and according to school-level summaries of how many children were dewormed in each class (which are then used to create the block or district tallies).30 (More below about who these monitors are.) However, it seems to us that these audits cannot tell us about the quality of state-level data, because audits are performed at the school level while all the government data we have seen is at the block level or higher. More details on our review of this auditing data are in this footnote.31 Note that collecting reliable data at scale may be quite costly and that there is a potential trade-off between evidence quality and cost-effectiveness here. ### Were deworming programs of high quality? In each of the four deworming programs for which we have seen data, monitors visited randomly selected schools on the day of deworming and on the mop up day (when children who were absent on the first deworming day are treated) to observe how the program was carried out. #### Monitoring procedures Monitoring procedures varied somewhat among the states and rounds of deworming, but in most cases the process included:32 • Pre-program. Monitors visit a selection of schools to observe whether hygiene facilities are available, principal and teachers are prepared for deworming day, drugs and supplementary materials are available, teachers are knowledgeable on topics related to deworming, and students are aware of the upcoming deworming day. This part of the process is a more recent addition. To date, we have only seen data from pre-program monitoring from one deworming round, in Rajasthan in 2012.33 • Deworming day and mop up day. On the days in which deworming takes place, monitors visit schools to observe whether deworming is taking place, principal and teacher are prepared and knowledgeable on topics related to deworming, drugs are stored properly, teachers are adhering to deworming procedures (such as administering the drugs after the students have eaten and not giving drugs to children who appear sick), and children are swallowing the pills. • Post-deworming. Monitors are asked to visit schools a few days after deworming has concluded to check whether class registers match summary forms (discussed above) and whether students report having been given a tablet recently, what color it was and what it tasted like, and whether they swallowed it. In Bihar, monitors visited 5% of schools in 201134 and 1.75% of schools in 2012.35 In Delhi, monitors visited 2.2% of schools and pre-schools.36 In Rajasthan, monitors visited 1.1% of schools.37 #### Monitor selection and training • Bihar: Monitors for the 2011 program included both district-level officials and “the existing cadre of external monitors of the WHO’s polio programme.” The latter were "trained WHO monitors who DtWI recruited through a database of independent monitors provided by the WHO." 38 In Bihar's 2012 program, most of the monitors came from the same database and "14 of the monitors were non-WHO monitors who [DtWI] contracted through recommendations from partners."39 • Delhi: Monitors for the 2012 program were senior "government officials and monitors from the Health Department who had worked as monitors in the polio program."40 In Delhi's 2013 program, the "monitors were hired through a vendor. Each [2013] monitor had prior non-profit field experience, and was interviewed by [DtWI's] Delhi Team prior to selection."41 • Rajasthan: Monitors for the 2012 and 2013 program were "independent monitors… hired through a vendor with 15 years of experience doing similar work for NGOs... such as UNICEF."42 #### Monitoring results Monitors recorded observations and responses on a large number of questions. Here we present data from a small subset of questions. We did not choose these indicators systematically. Instead, we chose a few indicators that were fairly easy to interpret without much additional context, represented examples from each stage of the monitoring process, and more directly addressed the question ‘how well was the program implemented.’ For complete results, refer to the following documents: Bihar 201243 Delhi 201244 Rajasthan 201245 Sample of question asked46 Pre-deworming: percent of schools that report having received drugs47 96% (on deworming day) N/A 95% of schools, 55% of pre-schools "Did the school receive the Deworming tablets?"48 Pre-deworming: percent of teachers that correctly answer whether it is safe to give deworming medicine to children who may not have worms N/A N/A 49% in schools, 44% in pre-schools "Is it dangerous to treat children who are not infected?"49 Deworming day: monitors that observe deworming taking place at this portion of schools 90% of schools 74% of schools, 86% of pre-schools 83% of schools, 69% of pre-schools "Is the deworming activity clearly visible in the school?"50 Deworming day: monitors observe drugs administered after a meal at this portion of schools 98% of schools 97% of schools, 100% of pre-schools 97% of schools, 96% of pre-schools "Did the students eat tiffin/mid-day meal before deworming?"51 Deworming day: monitors do not observe drugs administered to any sick children at this portion of schools52 89% of schools 79% of schools, not reported for pre-schools 93% of schools, 90% of pre-schools "Did you observe any sick child eating the tablet?"53 Follow up: pecent of students that report receiving treatment 98% of schools 99% in schools, 97% in pre-schools 98% in schools, 95% in pre-schools "When were you given the tablet?"54 Follow up: percent of students that report correct color of tablet 67% of schools 92% in schools, n/a for pre-schools 94% in schools, n/a for pre-schools "What was the colour of the tablet?"55 For Bihar’s 2011 deworming, we have not seen aggregated results (only raw data which we have not analyzed).56 We believe there are a number of reasons to believe that monitoring data could be inaccurate, including: • Data is missing for many questions from many schools, presumably because monitors did not report this data. We have not analyzed how much data is missing, but looking through the data sets, there appears to be a large number of blank cells. It is our understanding that missing data has been excluded from the summarized results. • Monitors may be motivated to report more favorable results than they observe. We have limited information on who the monitors are and what their motivations are and it is our understanding that monitors receive minimal training for their role. We have not seen audits of monitors' work. • While some results are directly observed by monitors, some results are self-reported by principals, teachers, and students. Principals, teachers, and students may seek to reflect positively on their schools by hiding problems with the program. • Principals and teachers that are monitored on the deworming days may conduct higher quality programs because they know they are being monitored. We also note that we have little information about procedures for storing drugs, ensuring that they don’t go bad, etc. The bottles of albendazole we examined on our site visit had expiration dates five years out, and their only storage instructions were to avoid direct sunlight and high temperatures (we don’t recall the exact figure but believe it was the equivalent of about 80 degrees Fahrenheit), so we consider this a relatively minor concern. ### Did worm infection and intensity rates decline after deworming? If pills were stored and administered properly, we would expect, based on independent evidence of the efficacy of deworming drugs, that worm burdens would decrease following treatment. However, we do not have direct evidence of this for the programs that DtWI has assisted in India. DtWI plans to measure worm infection and intensity rates again after three years of deworming treatment in each Indian state, which would make Bihar the first to qualify in 2014.57 ### What impact do DtWI’s activities have on deworming programs? Because it is a technical assistance/advocacy organization and not a direct service organization, evaluating DtWI’s impact is not straightforward. There are substantial potential advantages to supporting such an organization, as it may be able to have more impact per dollar by influencing government policy than by simply carrying out programs on its own, but this situation also complicates impact assessment. DtWI may be having an impact in the following ways: 1. It may increase the likelihood that a state implements deworming 2. It may improve the quality of the deworming program (leading to more children dewormed) While we have seen limited evidence to support the above, we would guess that DtWI likely has some impact on the likelihood that a state implements deworming and on the quality of the deworming it implements. We find the former effect more important, both in terms of the strength of the case for DtWI’s impact and in terms of what it would mean for the magnitude of DtWI’s impact (since a modest impact on the likelihood that a state implements deworming could provide good value for money, due to the scale of the programs). We discuss each in more detail below. #### Does DtWI increase the chances a given state will implement a deworming program? We have seen limited evidence supporting the conclusion that DtWI directly causes states to implement deworming programs. In a nutshell, (a) DtWI has played a role in all but one of the mass school-based deworming programs currently in operation in India; (b) our limited understanding of how Indian states decide to implement and continue new programs points to a potentially important role for the type of work DtWI does; (c) the conversations we’ve had with government officials are broadly (though not unqualifiedly) consistent with the idea that DtWI has raised the probability of deworming’s being implemented. Detail on what we know: • Existing deworming programs in India and DtWI’s role. • DtWI and the Children’s Investment Fund Foundation (CIFF) have told us that the Indian government mandates that all states provide school-based deworming and provides funds for this program.58 The only states in India that appear to be implementing school-based deworming programs are Andhra Pradesh, Bihar, Delhi, Punjab, and Rajasthan.59 Excluding Punjab (where it was not involved), DtWI states that it (or others affiliated with it) played a key role in each of these states’ decisions to implement deworming.60 • Without involvement from DtWI, Assam initiated a vitamin A supplementation and deworming mass drug administration (MDA) in the 2010-2011 school year, but reported low and conflicting coverage for that year and no apparent deworming MDAs subsequently.61 In the 2012-2013 school year Assam planned to implement a deworming program through the School Health Program, although it was delayed until at least March 2013.62 However, they have recently approached DtWI to begin a program with DtWI's assistance.63 • On our site visit to DtWI in Rajasthan, India, we spoke with three government officials who were involved in the deworming program.64 Of these, two stressed the importance of DtWI’s technical assistance, and the person we perceive as having been most responsible (of the three) for the decision to go forward with deworming gave the impression that the availability of this technical assistance had been a key factor in deciding to go forward. It should be noted that one of the three gave the impression that DtWI’s help was not needed, and all three conversations took place with multiple DtWI representatives present. With that said, the highest ranking of the three officials gave what we felt to be a nuanced and realistic picture of DtWI’s impact that implied a substantial (while not determinative) role. She stated that the immediate availability of technical assistance improved her confidence that the program would proceed quickly and smoothly; that she wasn’t sure whether the program would have proceeded if not for DtWI; and that there is at least one category of other program that she would very much like to find a nonprofit technical assistance partner for, supporting the idea that nonprofit technical assistance can be a key factor in getting a program to go forward.65 • Reasons new health programs may be started or halted in India. • DtWI and CIFF have told us that Indian state governments often receive funds earmarked for broader health programs, but often fail to spend these funds.66 DtWI also encourages states to create a dedicated line-item in their budget for deworming programs to ensure funding and administrative stability; Bihar recently made this change.67 • We asked the Center for Global Development for a recommendation of someone to talk to in order to understand states’ decisions to initiate deworming programs, and CGD referred us to Professor Devesh Kapur.68 Dr. Kapur asserted that it takes significant internal political will or external stimulus (such as from a non-profit) to sufficiently overcome general bureaucratic inertia in India to sustain a new health program. This holds true even in cases where a national mandate exists for a program (as it does for school-based deworming). Dr. Kapur felt that in the majority of cases, nonprofit technical assistance was likely to increase the probability of a program’s going forward.69 • DtWI told us that it believes that school health programs have been stalled in Indian states due to negative media attention;70 one of DtWI’s goals is to prevent these reports in the states in which it works.71 The documentation that DtWI has sent us supports -- but does not fully demonstrate -- this interpretation of events in Assam. We have not attempted to independently verify it because we do not think we would likely be able to do so effectively. Nonetheless, it is plausible that were decision makers in Indian states to have the impression that negative reports of program impacts could cause a program to be halted, they might be less willing to move ahead, and DtWI’s assurances that it would help prevent these reports could increase the likelihood that a state agrees to implement deworming. #### Does DtWI’s work increase the quality of deworming programs? We have seen limited evidence to convince us that DtWI’s work directly improves the quality of the programs it supports. DtWI could improve program quality by: • Increasing the chances that the first deworming round in each state begins earlier than it otherwise would have and that subsequent rounds occur on schedule72 • Increasing the training quality by simplifying training material and creating a more robust training program for the resource persons, who in turn train representatives from each school.73 • Ensuring that support roles are staffed74 • Improving focus and attention to detail75 • Ensuring that community sensitization occurs when programs are implemented in order to increase community acceptance of mass treatment and the ability of a program to withstand publicity associated with adverse events or seemingly related adverse events.76 • Implementing monitoring systems for both worm prevalence surveying and drug coverage which may improve program quality by creating a mechanism through which implementers are held accountable or by generating lessons that DtWI and the government can use in future rounds of treatment. We believe our strongest piece of evidence in favor of the idea that DtWI improves quality to be the conversation we had with a Rajasthan nodal officer, who cited many of the factors above and made the case that DtWI had played an important role in improving the quality of deworming. On the other hand, (a) this conversation took place with multiple DtWI representatives present, and the nodal officer had worked closely with DtWI; (b) another government official (on the same visit) stated that he believed DtWI’s technical help was not needed and did not add value, as the state was accustomed to running school health programs and deworming is a relatively simple one. In addition, we have limited information on the quality of India’s deworming programs (see above), and we felt that the training we attended (the only part of DtWI’s work that we observed directly) had major limitations in terms of potential to improve program quality - much of our view that the dewormings are likely carried out appropriately comes down to a view that the program is inherently relatively simple to execute. (We note that we do not put strong weight on (b) alone, since the government official may have had other incentives to give the message he did and had not worked as closely with the program as the Nodal Officer, but in context of the other observations we find (b) worth noting.) ### Possible negative and offsetting impact • Concerns over whether treatment will be sustained: We believe it is important that deworming programs are sustained over time, as re-infection is rapid and a one-time treatment may have little long-term effect.77 Because DtWI is relatively young, we have limited information about the likelihood that the programs it supports continue long-term. In a conversation, Dr. Devesh Kapur (referred to us by the Center for Global Development as someone who could provide context on how Indian states decide to implement programs) told us that when elections transfer control of state leadership from one political party to another, the newly elected leaders often cancel programs enacted by the previous government.78 (Note that, as discussed below, DtWI is seeking funds partly to investigate the possibility of elimination-focused programs, which could potentially mitigate this issue.) • Replacement of government funding: Indian state governments provide most of the financial resources for deworming programs.79 We have limited information about whether state governments would pay for the parts of the program paid for by DtWI in its absence, though our impression is that they would not.80 We also have little information about what state governments would use deworming funds for if they did not choose to implement deworming programs. Our understanding is that they might spend those funds on other state-government expenses (not necessarily health programs) or return them to the central government.81 • Diversion of skilled labor: Drug distribution occurs only once or twice per year and is conducted by teachers in schools. Based on our site visit in Rajasthan, our impression is that the Nodal Officer (the state official who manages all state school-based programs), the Nodal Officer’s staff, and the people that the Nodal Officer manages throughout the state (Resource Persons and Community Development Project Officers) have significant capacity to take on additional programs, so their taking on this program doesn't impose a significant burden on their time.82 On the other hand, a principal we spoke with commented that he would prefer fewer school-based health programs because they take focus away from the school day.83 • Adverse effects and unintended consequences of taking deworming drugs. Our understanding is that possible side effects are minimal and there is little reason to be concerned that drug resistance is currently a major issue (more). ## What do you get for your dollar? This section addresses the following questions: • What is the cost per child treated of the deworming programs, and what percentage of this does DtWI bear? We estimate costs in the range of$.35 per child treated total, with DtWI bearing a little under 10% of these.
• Should these deworming programs be expected to have as much impact (per person treated) as programs in sub-Saharan Africa, on which the evidence for deworming's positive impact on life outcomes is based? We believe that the programs DtWI is involved with are likely to have substantially less impact on a per-person basis, due to relatively low prevalence and intensity of infections where DtWI works.
• Should we think of DtWI as "leveraging" Indian state funds, such that each dollar it spends mobilizes multiple dollars from other players? We could imagine that DtWI's funds have substantial leverage, but could also imagine that DtWI is covering costs that aren't crucial to the success (in terms of quality, sustainability and coverage) of deworming and are thus less impactful on a per-dollar basis than the average dollar spent on deworming. Because we believe that DtWI likely increases the probability of a given program's going forward, we believe that it likely does leverage other funds to some extent, though this is difficult to quantify.

DtWI has estimated the total cost of each of the four deworming rounds in India discussed in-depth above (two in Bihar and one in each Delhi and Rajasthan). These estimates include the cost of the following, which were paid by a combination of government, DtWI, and other partners (percentage breakdown excludes round 1 in Bihar):84

Cost Category % of total cost % of category paid by government and other partners % of category paid by DtWI
Deworming day support (includes value of school staff time)85 61.7% 100% 0%
Training86 18.6% 97% 3%
Drugs (including both donated and purchased)87 9.7% 99% 1%
Program management (includes government staff time and both DtWI's India-based and US-based staff time)88 6.6% 3% 97%
Public awareness and mobilization89 1.7% 91% 9%
Prevalence survey90 1.0% 18% 82%
Monitoring and evaluation91 0.5% 36% 64%

DtWI has estimated some of the above costs because it did not have access to full cost data for governments and other partners, and boundaries between cost categories for internal receipts is not always clear.92

The table below shows the costs of DtWI, its partners, and the government in Bihar, Delhi, and Rajasthan, as well as estimates of cost per child treated.93 As discussed above, we believe that the number of children treated figures may not be accurate; we would guess that this is the largest source of uncertainty in the cost-per-child-treated calculation.

 Government costs $14,253,194 Other partner costs$311,060 DtWI costs $1,304,941 Total cost$15,869,195 Reported children treated 45,823,425 Total cost per child treated $0.35 DtWI estimates that its Kenya deworming program costs around$0.40 per child treated and is refining those estimates.94 We have not examined the details behind that calculation, since our focus here is on the India program.

### Deworming in India vs. Sub-Saharan Africa

There are two differences we discuss below: (a) the prevalence of particular worms and the prevalence of high intensity infections and (b) the existence of other neglected tropical disease programs that may treat the same condition.

Difference in worm prevalence and intensity

Key pieces of evidence that we discuss in our report on deworming (Miguel and Kremer 2004 and Baird et al 2012) are from a deworming experiment conducted in Kenya in the late 1990s. The participants in that study had relatively high rates of moderate-to-heavy infections of schistosomiasis and all three soil-transmitted helminths (STH).95

The situation in India is different. Schistosomiasis is not present in the vast majority of India,96 and those DtWI has helped treat (thus far) have significantly lower intensity infections than those treated in the Miguel and Kremer experiment in Kenya. In the table below, we compare the prevalence, moderate-or-heavy prevalence (using definitions from Miguel and Kremer 2004), and average eggs per gram of Bihar, Rajasthan, and the population in the Miguel and Kremer experiment.

STH (intensity)97 Bihar98 Rajasthan99 Miguel and Kremer 2004 year 1100
Hookworm prevalence (moderate/high) 3.7% See footnote101 15%
Ascaris prevalence (moderate/high) 8.8% 0.1% 16%
Trichuris prevalence (moderate/high) 0.4% 0.0% 10%
Hookworm prevalence (any) 42.2% See footnote102 77%
Ascaris prevalence (any) 52.1% 12.6% 42%
Trichuris prevalence (any) 0.2% 0.1% 55%

Difference in pre-existing programs

India has an existing program to treat lymphatic filariasis (LF).103 Albendazole, the same drug used to treat STH, is sometimes used to treat LF.104 The treatment for LF in India is annual administration of DEC or DEC and Albendazole105 and the WHO recommends annual albendazole treatment for areas with 20-50% prevalence of STH.106 According to the Indian government's program website, the LF treatment program has high coverage across many states (e.g., 80-90% in states carrying out the program), including one of the three states in which DtWI is currently active.107 Therefore, it appears that in some cases the effect of DtWI's work may be to transition from annual to biannual treatment in appropriate areas, and we are unsure of how the impact of a such a transition would compare to the impact of transitioning from no treatment to some treatment.

### How leveraged are DtWI contributions?

We have written before about the complexity involved in trying to understand leverage in charity, and below we lay out the relevant issues worth considering in the case of DtWI.

Donors to DtWI support only the costs paid by DtWI, which are less than 10% of the overall cost of the program. The role these funds play in the program is an important consideration in determining the cost effectiveness of donations.

The role DtWI funds play, and the resulting cost effectiveness, could range between:

• High leverage, high cost effectiveness. If DtWI leverages government funds that otherwise would not have been spent or would have been spent on a less valuable program by causing the government to run deworming programs or by making those programs significantly better than they otherwise would be, then a small contribution from DtWI has a large impact on the number of children receiving treatment and DtWI is highly cost effective.
• Small marginal contribution to the quality of the program. If DtWI's involvement doesn't cause more infected children to be reached, then DtWI’s contribution is more appropriately considered a small marginal contribution to the program that would have happened without its help. It is possible in that case that DtWI contributes little to increasing the number of children who receive treatment, but may improve the functioning of the program in less obvious ways.

We would guess that DtWI has played a role in increasing the likelihood that governments conduct deworming programs, while also contributing to the quality of programs. We also believe that it is very unlikely that DtWI was causally necessary for each of the programs it has assisted.

We believe that DtWI's leverage could reasonably range from less than 1x (i.e., programs would take place without DtWI's involvement) to 4-5x. We do not think we have a reasonable way of refining this estimate further, but this footnote provides individual GiveWell staff estimates of the leverage they would guess DtWI has.108 We plan to revisit this calculation annually.

## Room for more funds

DtWI has told us it could effectively use $2-3 million to support its activities.109 DtWI's main need is for unrestricted funds to respond to unanticipated situations, including requests from governments for technical support. DtWI would also like the opportunity to explore how their programs might contribute to STH elimination, not only control, by expanding coverage and reaching some adults. Quantifying the marginal impact of these funds is extremely difficult. DtWI aims to raise the following types of funding: • Restricted funding to expand DtWI-led deworming programs into additional states in India • Unrestricted funding to give it more flexibility in responding to situational needs and working capital to improve the organization’s stability and efficiency ### Restricted funds for state-specific scale-up DtWI has recently received substantial financial support to enable it to significantly expand its activities in India. • In 2012, DtWI received a 5-year,$5.23 million grant from the United States Agency for International Development (USAID) to support its work in India. This grant is specifically earmarked to support DtWI expansion into new states.110
• DtWI is also in conversations with another funder with hopes of receiving $1 million in funding for state-specific expansion. In sum, DtWI hopes to have$2.67 million available for state-specific scale-up in 2014. Based on past expenditures of approximately $220,000 to$483,000 per state,111 these funds should enable DtWI to expand into 4-6 more states.112

### Unrestricted funding

Alix Zwane, Evidence Action’s Executive Director, told us that DtWI has a pressing need for unrestricted funds.113

These funds would support two goals:

• Give DtWI the flexibility to pursue new models of providing technical assistance to Indian states. For example, in 2013, Punjab approached DtWI for assistance in implementing a deworming program. According to DtWI, Punjab did not require DtWI’s standard, full engagement and DtWI considered providing less intensive consulting to Punjab. However, DtWI did not have the funds available to do so, and Punjab went ahead with deworming without DtWI’s assistance. DtWI has told us that it would like to explore this "consulting model" and hopes to allocate $200,000 in 2014 to explore this.114 Our understanding is that this consulting model is one example of a model DtWI could pursue with unrestricted funds, and with funds available it would be in a position to consider other opportunities as they arose. • Provide DtWI with "working capital" enabling it to be a more robust organization. DtWI has told us that it believes working capital of approximately 10% of its overall budget would provide it with more stability and flexibility. DtWI was explicit that this 10% figure was not chosen with substantial analysis but is just intended to be a rough guide of what it believes would help. DtWI has not given specific examples of how it would use "working capital" funds but believes that some working capital would enable it to reduce the need to periodically focus on short-term, pressing fundraising and instead devote more effort to its core work. For example, DtWI told us that in 2012, its senior staff focused primarily on fundraising for a substantial period of time to raise the funds needed to continue its work in Bihar.115 Quantifying the marginal impact of such funds would largely be an exercise in guesswork. We could see a case that most of DtWI’s impact has come from sparking interest in deworming in a few initial states, which will lead to more interest over time (in Rajasthan, our impression is that the largest factor in the deworming program’s being started was media coverage of the program in Delhi116), and that further funds will be far less impactful. We could also see a case that by allowing flexibility, unrestricted funds allow substantial upside for DtWI and could be more impactful than the bulk of DtWI’s funds, which are restricted. ### Deworm the World Initiative and Evidence Action The Deworm the World Initiative is an initiative led by Evidence Action,117 run by Executive Director, Alix Zwane. Evidence Action includes a second program: Dispensers for Safe Water (DSW).118 Donations to DtWI, even if restricted to DtWI, would likely change the actions staff take to fundraise (i.e., which grants they pursue, what type of funding they ask for). Therefore, we do not believe donors can realistically donate to DtWI and support only DtWI and not DSW or Evidence Action more broadly. Dr. Zwane has told us that DSW has a funding gap of$2 million for 2014 and $5 million for 2015.119 Dr. Zwane has told us that she is currently in conversations with donors to close this gap, but none of these donations are assured.120 We recommend that donors recognize this fact and support Evidence Action with unrestricted gifts (to reduce the administrative burden placed on Evidence Action); Dr. Zwane has told us that Evidence Action recognizes that funds Evidence Action receives via GiveWell are due to DtWI and are aimed at supporting it to the extent possible. As with all our recommendations, we will follow Evidence Action’s progress in the future and report publicly on how it spends all funds it receives. ## The organization We have limited observations on which to base an assessment of the organization: • Evidence Action hired a new Executive Director, Alix Zwane, in 2013. Dr. Zwane has communicated extremely clearly and directly with us and given thoughtful answers to our critical questions, and has shared significant, substantive information. This represents a change from previous interactions with Deworm the World, in which we’ve struggled to get clear answers to our questions and to receive clearance to share information. • We believe DtWI’s self-evaluation to have been less rigorous than that of our other top charities to date, though we are hopeful that it will improve going forward. More on how we think about evaluating organizations at our 2012 blog post. ## Sources Document Source Alix Zwane conversation August 30th 2013 Unpublished Alix Zwane conversation June 4th 2013 Source Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013 Unpublished Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013 Unpublished Assam 2010 guidelines for deworming Source Assam mid day meal report 2013 Source (archive) Assam reproductive and child health 2011-2012 Source (archive) Assam state programme implementation plan 2011-2012 Source (archive) Baird et al 2012 Source CIFF conversation September 10th 2013 Source Devesh Kapur conversation October 14th 2013 Unpublished DSW 2012 GiveWell site visit Source DtWI 2013 GiveWell site visit Source DtWI 2013 GiveWell government interviews Source DtWI Assam research 2013 Source DtWI Bihar 2011 cost data Source DtWI Bihar 2011 coverage data Source DtWI Bihar 2011 monitoring data for deworming day Source DtWI Bihar 2011 monitoring data for mop-up day Source DtWI Bihar 2011 Monitoring Form for Deworming Day Source DtWI Bihar 2011 monitoring form for mop-up day Source DtWI Bihar 2011 prevalence survey report Source DtWI Bihar 2011 program report Source DtWI Bihar 2012 cost data Source DtWI Bihar 2012 coverage data Source DtWI Bihar 2012 monitoring data for coverage validation Source DtWI Bihar 2012 monitoring report Source DtWI class register audits 2013 Source DtWI cost narrative 2013 Source DtWI cost per treatment summary 2013 Source DtWI Delhi 2012 cost data Source DtWI Delhi 2012 coverage data by anganwadi Source DtWI Delhi 2012 coverage data by school Source DtWI Delhi 2012 coverage report Source DtWI Delhi 2012 monitoring data Source DtWI Delhi 2012 monitoring form deworming day Source DtWI Delhi 2012 prevalence survey design Source DtWI Delhi 2012 prevalence survey report Source DtWI Delhi 2012 program report Source DtWI Rajasthan 2012 cost data Source DtWI Rajasthan 2012 coverage data for anganwadi Source DtWI Rajasthan 2012 coverage data for schools Source DtWI Rajasthan 2012 monitoring data for coverage validation in anganwadis Source DtWI Rajasthan 2012 monitoring data for coverage validation in schools Source DtWI Rajasthan 2012 monitoring form coverage day Source DtWI Rajasthan 2012 monitoring form pre-deworming day Source DtWI Rajasthan 2012 monitoring report Source DtWI Rajasthan 2012 prevalence survey report Source Evidence Action 2014 budget Unpublished Evidence Action cover letter 2013 Source Evidence Action launch announcement 2013 Source (archive) Evidence Action website 2013 Source (archive) Grace Hollister conversation June 19th 2013 Source Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 Unpublished Harvard Business School Kenya Case Study A 2010 Unpublished LF treatment coverage 2012 Source (archive) LF treatment drugs 2012 Source (archive) Miguel and Kremer 2004 Source Preventive chemotherapy in human helminthiasis 2006 Source (archive) Professor Devesh Kapur Biography 2013 Source (archive) WHO soil-transmitted helminthiases 2012 Source (archive) World Schistosomiasis Risk Chart 2012 Source (archive) ### Uncited supplementary materials Document Source DtWI Bihar 2011 monitoring forms phase I Source DtWI Bihar 2012 monitoring data deworming day anon Source DtWI Bihar 2012 monitoring data mop-up anon Source DtWI Kenya site visit 2012 Source DtWI Rajasthan 2012 monitoring data anganwadi deworming day anon Source DtWI Rajasthan 2012 monitoring data anganwadi mop-up anon Source DtWI Rajasthan 2012 monitoring data anganwadi prep anon Source DtWI Rajasthan 2012 monitoring data schools deworming day anon Source DtWI Rajasthan 2012 monitoring data schools mop-up anon Source DtWI Rajasthan 2012 monitoring data schools prep anon Source • 1. “In January 2007, the YGL launched the Deworm the World campaign with the goal of improving children’s health and education by massively expanding deworming programs.” Harvard Business School Kenya Case Study A 2010 Pg 7. • 2. “Deworm the World (DtW) is funded through 2015-2016 in Kenya, thanks to the Children’s Investment Fund Foundation (CIFF), and the Ending Neglected Diseases (END) Fund.” Alix Zwane conversation June 4th 2013, Pg 1. • 3. “As such, the states themselves have to make the decision to conduct a deworming campaign; DtW can only encourage that decision by showing that it can be done and offering assistance to help implement the program in a robust fashion that involves intensive monitoring of the program.” Alix Zwane conversation June 4th 2013, Pg 2. • 4. • “Six districts were identified within [Andhra Pradesh] where deworming would be piloted. DtW conducted prevalence surveys in those six districts, finding that worms existed in less than 20% of the population in the districts, which is the World Health Organization-recommended threshold for treating all children.” Grace Hollister conversation June 19th 2013, Pg 1. • “DtW did two stages of prevalence surveys between August 2010 and February 2011. They found that over 50% of school-aged children had worms, a level at which the World Health Organization (WHO) recommends deworming twice a year, rather than just once a year. Bihar already had a statewide albendazole treatment.” Grace Hollister conversation June 19th 2013, Pg 2. • “In 2011 DtW conducted a prevalence survey throughout the National Capital Territory. The average infection rate was below the 20% threshold, although there were large disparities in prevalence between different areas of the city.” Grace Hollister conversation June 19th 2013, Pg 3. • “DtW’s prevalence survey found that around 20% of the children were infected with at least one type of STH, particularly in the Western part of the state. Based on elevations and other climatic factors, it is estimated that hookworm is a lot more prevalent in the Eastern part of the state. Taken together, the data led DtW to recommend a mass treatment for the whole state once a year.” Grace Hollister conversation June 19th 2013, Pg 4. • 5. “Andhra Pradesh... Deworm the World [Initiative]’s contributions • Prevalence survey... • Operational support • Helped government develop operational plans and budgets • Coordinated cross-sectoral partners through the establishment of a State School Healthe Coordination Committee, bringing together health and education departments and other stakeholders (such as the microfinance partner SKS) • Coordinated drug donation made by Feed the Children • Designed a monitoring and evaluation (M&E) system • Created government tableau for community awareness • Trainings • Conducted a master training session for program • Designed training cascade for the master trainees to train the rest of the implementors • Designed training materials • Developed materials and campaigns for community sensitization Bihar… Deworm the World’s contributions to the deworming program in Bihar were similar to those in Andrha Pradesh (see above). In Bihar, DtW coordinated drug donations for Rounds 2 and 3 of the program through the WHO… Delhi… In addition to the standard contributions (see Andrha Pradesh, above), DtW helped set up a technical secretariat within the School Health Scheme of the Delhi government to support program monitoring. In Delhi, DtW coordinated drug donations for school-age children through Feed the Children. Rajasthan… DtW’s prevalence survey and recommendation to treat annually thus increased the efficiency of the program significantly, as well as decreasing the required government funding contribution. Additionally DtW successfully encouraged the government to include preschoolers in the program as well. DtW coordinated drug donations for school-age children through the WHO.” Grace Hollister conversation June 19th 2013, Pg 1-4. • 6. Data summary and primary sources in DtWI cost per treatment summary 2013, Sheet DtWI v partners. Includes costs from three rounds of deworming: Bihar 2012, Delhi 2012, and Rajasthan 2012. Excludes data from Bihar 2011 because we do not have a budget breakdown for DtWI for that deworming round. Descriptions of the categories are available in DtWI cost narrative 2013. DtWI notes that these campaign-oriented costs don't include any costs of running DtWI as a whole where they don't pertain to a particular campaign. • 7. Details on DtWI’s timeline in the states in which it has worked are included in the notes from our conversation with DtWI Director Grace Hollister. Since the conversation with Ms. Hollister, we have learned of the following recent and upcoming deworming rounds that DtWI is supporting: • Bihar: The third round is now scheduled for January 2014. • Delhi: conducted a second round of deworming with DtWI’s assistance on October 3rd, 2013. Results are pending. • Rajasthan: the October 2013 round of treatment was underway as of this writing. "Bihar Round 3 is scheduled for January 16, 2014." Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013. "Delhi round 2 occurred Thursday October 3rd, 2013, and Rajasthan round 2 occurred October 15th, 2013." DtWI 2013 GiveWell site visit, Pg 12. • 8. “Innovations for Poverty Action (IPA) is pleased to announce the launch of a new organization created with IPA’s support and dedicated to taking cost-effective programs to scale to improve the lives of millions in Africa and Asia. Evidence Action has been created to bridge the gap between evidence about what works to alleviate poverty around the world and what is actually implemented. The organization scales interventions based on rigorous evidence and crafts resilient business models for long run success. Two IPA initiatives that touch millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative – will spin off from IPA to be managed by Evidence Action.” Evidence Action launch announcement 2013 • 9. Evidence Action 2014 budget • 10. Evidence Action 2014 budget • 11. “INDIA: Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore).” World Schistosomiasis Risk Chart 2012, Pg 3. • 12. "Prevalence and intensity mapping to target treatment is what is recommended by the WHO and what is encouraged by DtW." Alix Zwane conversation June 4th 2013, Pg 3. • 13. • School and participant selection: "Pots were distributed to 5240 children aged 1-20. The sample was stratified into three groups MCD schools (1600), Government Schools (1600) and Slums (2040) and weighted in its distribution according to the populations and numbers of schools and slums in each district (and zone where appropriate). At all sites 40 children were sampled (with a target of 30 samples returned from each site and a total sample size of 3870)." Pg 1. We are unsure of whether the 40 students in each school were selected randomly in Delhi or by some other process. • Participation rate: Table One, on Pg 1, shows that 3668 out of 5240 pots were returned, a participation rate of 70%. In addition, "Of the 3668 children remaining, 441 were dropped from the data set due to missing parasitological data and a further 3 who lacked age data," leaving a sample of 3224 out of 5240 (62%). Pg 2. • The report did not provide details of the technique used to analyze the samples. • Selection of districts and schools: "Four districts of Bihar were selected to complement the existing STH model and twenty schools, five from each district were randomly selected from the state schools database." Pg 9. For more details see DtWI Delhi 2012 prevalence survey design • Selection of students: "Within each school 65 children aged 6 and above representing both sexes equally from class 1 to 6 were randomly selected, class by class, using random number tables." Pg 9. • Technique: "Screening of infection for STH was based on a double Kato-Katz smear of 41.7 mg prepared from fresh stool samples." Pg 11. • Participation rate: "From a total of 1,281 school children registered in the survey and provided with pots, 1,159 returned samples." (90.5%) Pg 11. • District selection: "Thirteen districts were randomly selected and compared with secondary data collected for all districts. This information was circulated to the task force constituted to oversee the prevalence survey, who noted a good spread of key factors expected to influence worm prevalence: elevation, population density, and climate.” Pg 6. • School selection: “Four schools were randomly selected per district from a complete list provided by the RCEE, excluding those with enrolment lower than 30 students, for a total of 52 schools… The targeted anganwadis were drawn from the one or two anganwadis nearest to the randomly selected school, as a comprehensive database of anganwadis was unavailable.” Pg 6. • Participant selection: “35 children (aged 6-15) were selected per school. Selection was undertaken by having children from each grade line up, separated by gender, and selecting students randomly from each queue while attempting to have a 50-50 male-female split. From each anganwadi, 24 children (aged 2-5) were randomly selected and invited to participate.” Pg 7. • Technique: “Samples were analyzed by Kato Katz technique using a World Health Organization (WHO) standard template.” Pg 9. • Participant selection:The report did not provide details on participation rate, but DtWI reports that it was approximately 56% and is verifying that number. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 • 14. using WHO definitions of moderate intensity: • "Ascarsis [sic] lumbricoides: 1 - 4,999 epg: light-intensity infections; 5,000 - 49,999: moderate-intensity infections; >50,000 epg: heavy intensity infections • Trichuris trichiura: 1 - 999 epg: light-intensity infections; 1,000 - 9,999: moderate-intensity infections; >10,000: heavy-intensity infections • Hookworms: 1 - 1,999: light-intensity infections; 2,000 - 3,999: moderate-intensity infections; >4,000: heavy-intensity infections" WHO soil-transmitted helminthiases 2012, Pg viii. • 15. • There is a pre-existing LF program that provides albendazole to a large portion of the population in Bihar annually. • Based on timing of the decision to deworm in Bihar, there was only time to collect prevalence survey from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet DtWI's typical standard of quality. • Before the DtWI-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report • Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness. • 16. • "A clustered, stratified and randomized sample is suggested for the Delhi prevalence study." DtWI Delhi 2012 prevalence survey design, Pg 1. • DtWI chose to encourage and support deworming in Delhi despite low prevalence since it plays a leadership role in India and might encourage other states to start deworming programs. • 17. • The prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. DtWI Rajasthan 2012 prevalence survey report. • "Results also reflect underlying uncertainty in the data, which one might expect to be very high as 20% of the schools sampled examined less than 10 children. This poor response introduces a strong possibility of a bias, which based on the common assumption that risk is higher in non-enrolled children would likely result in an underestimate of prevalence.” DtWI Rajasthan 2012 prevalence survey report. • "Areas in western Rajasthan are predicted to have a higher risk of Ascaris infection compared to eastern parts of the state. Figure 5 shows the probability that the predicted prevalence of Ascaris exceeds 20%, based on a model including EVI and altitude. While there is a high probability that areas in western Rajasthan will require at least annual MDA, there is likely to be a lower prevalence of Ascaris throughout the east of Rajasthan." DtWI Rajasthan 2012 prevalence survey report, Pg 14. • "The climatic thresholds for hookworm transmission limits in Asia have been previously modelled using an aridity index, mean and maximum land surface temperature (LST). The map of these limits is shown in Figure 6, which categorises areas as unstable transmission (Mean LSTH 5-10°C, Aridity Index <0.2) or beyond transmission (Max LST >42°C, Mean LSTH <5°C, Aridity Index <0.03). These limits are based on experimental data showing that the development of infectious stages of hookworm ceases at 40° Celsius, and observed relationships between prevalence and environmental covariates using data from the Global Atlas of Helminth Infections. Based on these limits, the prevalence of hookworm in Rajasthan would be expected to oppose that of Ascaris and be more prevalent in the east of the state. All hookworm positive cases identified from this survey originated from areas within transmission limits." DtWI Rajasthan 2012 prevalence survey report, Pg 15-16. • DtWI concluded, "Based on Ascaris prevalence levels in the surveyed districts, observed associations with environmental data, hookworm transmission limits and the possibility that data are an underestimate of the true prevalence, annual deworming throughout the state is recommended." DtWI Rajasthan 2012 prevalence survey report, Pg 17. • 18. As discussed in a footnote above, the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. • 19. This figure represents just Ascaris and Trichuris prevalence, not hookworm, as the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. • 20. As discussed in a footnote above, the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. • 21. This figure represents just Ascaris and Trichuris prevalence, not hookworm, as the prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. • 22. • 23. With respect to prevalence greater than 50%, The WHO adds that "If resources are available, a third drug distribution intervention might be added. In this case the appropriate frequency of treatment would be every 4 months" Preventive chemotherapy in human helminthiasis 2006, Pg 41. • 24. See: • 25. DtWI Bihar 2011 coverage data • 26. DtWI Bihar 2012 coverage data • 27. DtWI Delhi 2012 coverage report • 28. DtWI Rajasthan 2012 coverage data for schools DtWI Rajasthan 2012 coverage data for anganwadi • 29. In Rajasthan, for example, there are 257 blocks for 80,000 schools: "There are 257 blocks in Rajasthan." DtWI 2013 GiveWell site visit, Pg 3. "There are 80,000 public schools in Rajasthan." DtWI 2013 GiveWell site visit, Pg 4. • 30. • In Bihar in 2011, monitors visited a randomly selected set of schools on deworming day and mop up day. In one of the three phases of the program, monitors were asked to record the number of children dewormed on that day by asking the teacher conducting the deworming and checking the school’s forms. They were not asked to compare two sources of data. See DtWI Bihar 2011 Monitoring Form for Deworming Day, DtWI Bihar 2011 monitoring form for mop-up day, DtWI Bihar 2011 monitoring data for deworming day, and DtWI Bihar 2011 monitoring data for mop-up day. DtWI shared the data from this monitoring process with us, but there appears to be a problem with the data we received (the data in the column for “How many children were dewormed today” is yes/no rather than numerical). DtWI Bihar 2011 monitoring data for deworming day, worksheet "Phase 3". • In Bihar in 2012, monitors visiting randomly selected schools after the deworming days were asked to record the number of children dewormed according to the class registers and according to the school summary report. The class registers were available for examination in “almost all schools,” while the summary form was available in only 69% of schools. DtWI Bihar 2012 monitoring report, Pg 11. • In Delhi in 2012, monitors visiting randomly selected schools after the deworming days were asked to record the number of children dewormed according to the class registers and according to the school summary report. Data was available for comparison for 65% of classes visited. DtWI Delhi 2012 monitoring data • In Rajasthan in 2012, monitors visiting randomly selected "independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation." DtWI Rajasthan 2012 monitoring report, Pg 1. • 31. • In the monitoring report on Bihar’s 2012 deworming day, the median number of children dewormed per class according to the class registers was 35, while the median number according to the school summary forms was 38. DtWI Bihar 2012 monitoring report. Our own analysis finds a median of 38 for both, an average of 49 according to the class registers and 50 according to the summary reports, and an inflation rate of 1.5%. (Data missing for 31% of classes visited.) • In Delhi in 2012, the median number of children dewormed per class according to the class registers was 88, while the median number according to the school summary forms was 89.5. The average number from the registers was very slightly higher than from the summary forms (128 versus 127). The inflation rate on the summary forms was 1%.(Data missing for 38% of classes visited.) • In Rajasthan in 2012, the median number of children dewormed per class according to the class registers was 17, while the median number according to the school summary forms was 18. The average number from the registers was 25 and from the summary forms was 28. The inflation rate on the summary forms was 11.3%. The vast majority of the inflation comes from schools in which the inflation rate is several hundred percent. (Data missing for 20% of classes visited.) • Data summarized in DtWI class register audits 2013. • DtWI explains, "In those small percentage of cases where [class summaries report significantly higher coverage numbers than school attendance registers], we believe it is not monitor error, but a failure by the school teachers to tick the attendance register. On deworming day Round 2 in Rajasthan, DtWI staff observed that a teacher had directly filled out the school summary form with the number of children dewormed rather than entering it on the register. We speculate, based on our own experiences monitoring the process across all the states, that the main reason for this difference is the teachers either partially fill out the register and then go directly to the summary form, or they simply do not fill out the register on one of the two days. This is a common problem that we try to rectify through the training. In Bihar Round 2, similar to Rajasthan Round 1, there were some schools where the summary form totals were greater than the attendance registrar ticking. Given that the percentage of these errors is not large, and our own staff’s experiences of visiting schools where attendance registers were not ticked (rather the school form was filled out that day itself), we do not find it surprising that this discrepancy exists. We do not believe this discrepancy indicates a systematic bias in the reported deworming numbers." Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 • 32. • Bihar 2011: • Process: “Moreover, independent monitors were mobilized in each block (560 total) to visit 5% of all government schools on deworming and mop-up days to ensure that an adequate quantity of drugs were available, trained teachers were administrating drugs according to protocol, and community sensitization initiatives were successful in mobilizing non-enrolled children to attend. One auditor per district was in charge of supervising monitors to ensure quality monitoring of all schools.” DtWI Bihar 2011 program report, Pg 9. • What we’ve seen: Monitoring forms used for data collection and full monitoring data from deworming day and mop-up day. We have not seen a summary of this data and have not analyzed it ourselves. • Bihar 2012: • Process: “Independent monitors visited a randomly selected sample of schools over four days – Deworming Day, Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 1216 schools (1.75% of 69,299 schools in Bihar) targeted for monitoring. From each of the 38 districts in Bihar, 2 blocks were selected by simple random sampling, for a total of 76 blocks. 2 clusters were randomly selected from each of the 76 blocks, excluding clusters with fewer than 8 schools. In each of these 152 clusters, 8 schools were randomly selected for a total of 1216 schools. Each monitor was assigned a block comprising 16 schools to be monitored. From this list of 16 schools, the monitor could visit any 4 schools on Deworming Day, another 4 on Mop-Up Day, and 8 more schools over the two Coverage Validation days. Out of the total sample of 1216 schools, monitors were able to survey 1196 schools comprising 302 schools on Deworming Day, 296 on Mop-Up Day, and 598 on Coverage Validation days.” DtWI Bihar 2012 monitoring report, Pg 1. • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Bihar 2012 monitoring report). • Delhi 2012: • Process: “Monitoring visits were conducted by senior government officials. Random site visits by independent auditors occurred at a subset of training sessions and participating institutions. Coverage validation engaged independent auditors who conducted random site visits at a representative sample of schools and anganwadis to validate coverage statistics. This source of information was carefully compared with programme reports collated from each school and anganwadi to arrive at an accurate assessment of programme coverage.” DtWI Delhi 2012 program report, Pgs 7-8. • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Delhi 2012 program report, Pgs 17-22). • Rajasthan 2012: • Process: “In order to evaluate the efficacy of the deworming protocol and process, independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation. A multi-stage sampling strategy was used to select the 990 schools (1.1% of 90,488 schools in Rajasthan) targeted for monitoring. From each of the 33 districts in Rajasthan, 2 blocks were selected by simple random sampling. In each of these 66 blocks, 15 schools were randomly selected for a total of 990 schools. One monitor was assigned to each of these blocks. From the list of 15 schools in a block, the monitor could visit any 3 schools before Deworming Day for Preparation Monitoring, another 2 schools on Deworming Day, 2 more schools on Mop-Up Day, and 4 more schools over the two Coverage Validation days. The remaining 4 schools served as a buffer in case a particular school could not be visited. Hence, the actual sample size was 726 schools (0.8% of all schools).” DtWI Rajasthan 2012 monitoring report, Pg 1. • What we’ve seen: Monitoring forms used for data collection, full monitoring data from deworming day, mop-up day, and coverage validation days following deworming, and a summary report of results (DtWI Rajasthan 2012 monitoring report). • 33. DtWI Rajasthan 2012 monitoring report • 34. • 35. “A multi-stage sampling strategy was used to select the 1216 schools (1.75% of 69,299 schools in Bihar) targeted for monitoring.” DtWI Bihar 2012 monitoring report, Pg 1. • 36. Total schools: “The programme targeted 3,032 schools and 9,934 anganwadis [pre-schools].“ Pg 11. Deworming day: “This data analysis is based on 45 schools and 15 anganwadis." DtWI Delhi 2012 program report, Pg 17. Mop-up day: “This data was collected from 49 schools and 14 anganwadis." DtWI Delhi 2012 program report, Pg 19. Coverage validation: “This data was collected from 80 schools and 87 anganwadis.” DtWI Delhi 2012 program report, Pg 20. • 37. “A multi-stage sampling strategy was used to select the 990 schools (1.1% of 90,488 schools in Rajasthan) targeted for monitoring.” DtWI Rajasthan 2012 monitoring report, Pg 1. • 38. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 “On deworming days, district-level officials from both the Health and Education Departments visited randomly selected schools to confirm that children were being dewormed in accordance with protocols. Both government monitors and the existing cadre of external monitors of the WHO’s polio programme were trained to monitor the school-based deworming programme in Bihar.” DtWI Bihar 2011 program report, Pg 9. • 39. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 • 40. "Delhi: For Round 1, we used government officials and monitors from the Health Department who had worked as monitors in the polio program (mostly medical doctors)." Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 “Monitoring visits were conducted by senior government officials. Random site visits by independent auditors occurred at a subset of training sessions and participating institutions. Coverage validation engaged independent auditors who conducted random site visits at a representative sample of schools and anganwadis to validate coverage statistics.” DtWI Delhi 2012 program report, Pg 7. • 41. • 42. "In order to evaluate the efficacy of the deworming protocol and process, independent monitors visited a randomly selected sample of schools and anganwadis over five days – one day before deworming for preparation monitoring, on Deworming Day, on Mop-Up Day, and two days post-deworming allocated for coverage validation." DtWI Rajasthan 2012 monitoring report, Pg 1. • 43. DtWI Bihar 2012 monitoring report • 44. DtWI Delhi 2012 monitoring data • 45. DtWI Rajasthan 2012 monitoring report • 46. Some of the monitoring forms we received are not translated into English; we have not checked whether the questions were phrased the same in each round of treatment, or whether the translations to English are accurate. Sample questions here are taken from one of the monitoring forms with English translations. • 47. Monitors asked the principal of each school whether they had received the deworming tablets or not. • 48. DtWI Rajasthan 2012 monitoring form pre-deworming day, Pg 2. • 49. DtWI Rajasthan 2012 monitoring form pre-deworming day, Pg 4. • 50. DtWI Delhi 2012 monitoring form deworming day, Pg 1. • 51. DtWI Delhi 2012 monitoring form deworming day, Pg 4. • 52. Monitors were instructed to visit one or more classrooms while they were administering the deworming tablets and answer a handful of questions about the process. The monitors were not required to watch the entire process for each class visited, or every class in each school, so when the monitors report that they did not observe any sick children receiving the tablet, they are not reporting that no sick children received the tablet in that class or school. • 53. DtWI Delhi 2012 monitoring form deworming day, Pg 5. • 54. DtWI Rajasthan 2012 monitoring form coverage day, Pg 5. • 55. DtWI Rajasthan 2012 monitoring form coverage day, Pg 5. • 56. DtWI Bihar 2011 monitoring data for deworming day • 57. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 • 58. "National funds for deworming are part of the WIFS program and the national government mandates that states should adopt the WIFS program. Indian national policy indicates that deworming should take place biannually." DtWI 2013 GiveWell site visit, Pg 12. "Wherever possible, DtWI works to get deworming added as a line item in the budget. Otherwise, it is difficult to ensure that long-term funding will exist for the program." DtWI 2013 GiveWell site visit, Pg 10. "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2. • 59. Alix Zwane conversation August 30th 2013 Assam has also announced its intentions to move forward with its deworming program in March 2013: "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. As reported by NRHM, Assam any [sic] school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam mid day meal report 2013, Pg 11. According to DtWI, "No schools in Assam have been covered under the School Health Program to provide deworming in 2012/13. The programme was then re-scheduled to begin in March 2013. No evidence of this actually having taken place." DtWI Assam research 2013, Pg 1. "[DtWI] are also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2. • 60. "DtW has been involved in deworming programs in four different states, and is still actively involved of three of those. Of the states DtW has worked with in the past, none of them had school-based deworming programs before DtW's involvement." Grace Hollister conversation June 19th 2013, Pg 1. "In 2009, DtW and the World Bank had conversations with the Chief Minister of Andhra Pradesh, in which they advocated for a broad school-based deworming program, which hadn't happened before in the state. In a public announcement with health and education ministers following this interaction, the Chief Minister announced the plan to do so, and deworming became the flagship of the state’s school health program." Grace Hollister conversation June 19th 2013, Pg 1. "In January 2010 the Jameel Poverty Action Lab (J-PAL) hosted a regional development and policy conference, at which evidence on school-based deworming was presented, as well as experiences from Andhra Pradesh. Immediately following the conference, discussions started among the state of Bihar, J-PAL, led by members of the DtW Board of Directors, and DtW about the possibility of a deworming initiative there. In August a memorandum of understanding (MoU) was formalized between DtW and the relevant players in Bihar (School Health Society Bihar and Bihar Education Project Council) for program implementation." Grace Hollister conversation June 19th 2013, Pg 2. "DtW leveraged its networks to engage the support of the Minister of Health, Women, and Child Welfare [in Delhi]. Education stakeholders were also brought in so that the program could reach beyond preschool to older grades as well. DtW conducted a great deal of advocacy, maintaining continuous interactions and significant support among all relevant departments to bring the program to fruition Due to the complicated agency system, there were many other government stakeholders as well, requiring a high degree of coordination by DtW to ensure program objectives could be achieved." Grace Hollister conversation June 19th 2013, Pg 3. "After the deworming program launched in Delhi, Rajasthan saw the results generated by the DtW- supported program in Delhi and committed to doing a deworming program, allocating funding for it in their budget. In March 2012 they brought DtW in to help. In this case, the state already knew what it wanted and already had a school health program. They sought DtW's technical expertise, mapping ability, general program support, stakeholder coordination, etc. DtWI coordinated signature of a MoU between the Departments of Women and Child Development, Education, and Health, UNICEF and DtW to guide program implementation, and helped establish of a technical secretariat housed within the Education Department." Grace Hollister conversation June 19th 2013, Pg 3. "DtW has worked in Delhi, Bihar, Rajasthan, and Andhra Pradesh (AP). • In Bihar, according to documentation provided by DtW, the program began because of previous Poverty Action Lab (J-PAL) work in Bihar. DtW also noted that Rajasthan approached them because of the success (as highlighted in the media) of the Delhi program. DtW supported a pilot program and prevalence survey in AP, and though DtW didn't continue to engage with AP, the AP government seems to have recently conducted a school-based deworming program. • More recently, Punjab approached DtW, but DtW didn't have the funding to move forward with Punjab, and Punjab is now implementing deworming on its own. It's not possible to be certain whether or not DtW has caused deworming to happen that otherwise would not have, but it's important to note that DtW took the opportunities presented to help run better programs where possible as well as improve data quality and did so on a limited budget. They are also widely acknowledged by the deworming community to be the only technical assistance available in India." CIFF conversation September 10th 2013, Pg 2. • 61. "Keeping in view the problem of worm infestation of children in Assam it has been decided to conduct two rounds of De-worming every year in the month of March and September. The first round will be conducted during March’ 2010. On every Wednesday (VHND/Immunization Day) medicine for De-worming will be given to all children between the age group 1 to 5 years along with vitamin A supplementation." Assam 2010 guidelines for deworming, Pg 1. 976,192 "children below 5 years provided vitamin A syrup" as of November/December 2010 Assam reproductive and child health 2011-2012, Pg 73. The National Rural Health Mission reported the progress for 2010-2011 as 92,957 "Students given IFA/ de-worming tablets". Assam state programme implementation plan 2011-2012, Pg 21. Assam is "supposed to provide deworming to all children, but AWC services provided in Assam are some of the worst performers for deworming coverage." DtWI Assam research 2013, Pg 1. • 62. "Status of School Health Programme with special focus on provision of micronutrients, Vitamin-A, de-worming medicine, Iron and Folic acid, Zinc, distribution of spectacles to children with refractive error and recording of height, weight etc. NB. As reported by NRHM, Assam any school has not been covered under School Health Programme during 2012-13. The programme is being implemented from March/2013. They have completed the training of Multipurpose Worker (MPW) and Lady Health Visitor (LHV) for the purpose. Recruitment of dedicated Medical officer, Dental Surgeon and Block Health Programme officer have been made for implementation of the programme. The weekly Iron and Folic Acid Supplementation Programme among the adolescent students of Class VI to VIII is also being implemented from the March/ 2013. The training programme for District trainers of all the districts have been completed in Dec./12." Assam mid day meal report 2013, Pg 11. • 63. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 • 64. DtWI 2013 GiveWell government interviews • 65. "If DtWI had not been there, how would the deworming program be different? • Since DtWI have already implemented deworming programs in Delhi and Tamil Nadu, they brought experience. • If a state government decides to do something, nothing is impossible. However, DtWI's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have DtWI’s experience, there could have been more problems. • The government received important support from DtWI, particularly with distributing the deworming tablets, implementing the trainings, and developing training materials." "Have you ever wanted to do a program but been unable to find a technical partner? • Ms. Gupta is still looking for a technical partner on specific issues. For example, Rajasthan has a large population with special needs. It has generally been able to find partners for helping the visually impaired, but very few organizations work with autistic children, children with cerebral palsy, or children with mental retardation. If there were a partner to support the Rajasthan government in this area, this would be very welcome. The Rajasthan government has funds for this type of program, but are looking for a technical partner. • An NGO called Sight Savers works with blind children. Before Sight Savers, she was not aware of problems with low vision. Many children need magnifying glasses and large print books. Sight Savers helped them to identify doctors and hospitals to work with such children." • 66. "In India, DtW facilitates the expenditure of resources that the states already have available." Alix Zwane conversation June 4th 2013, Pg 2. "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively." CIFF conversation September 10th 2013, Pg 2. • 67. "Wherever possible, DtWI works to get deworming added as a line item in the budget. Otherwise, it is difficult to ensure that long-term funding will exist for the program." DtWI 2013 GiveWell site visit, Pg 10. "Recently, [Bihar] added the deworming program as a line item in the state budget, which makes the funding for deworming programs more secure." DtWI 2013 GiveWell site visit, Pg 2. • 68. Professor Devesh Kapur Biography 2013 • 69. Devesh Kapur conversation October 14th 2013 • 70. "Some school health programs, such as the WIFS program in Delhi, are rolled out too quickly and their trainings are carried out poorly. Then they receive negative media attention because of students’ adverse reactions to treatments. Negative media can cause delays or cancellation of school health programs. The WIFS program has also received negative media attention in the states of Haryana and Odisha." DtWI 2013 GiveWell site visit, Pg 10. • 71. "There have been very few adverse reactions to deworming treatments in DtWI-supported states because of the quality of DtWI’s trainings." DtWI 2013 GiveWell site visit, Pg 10. • 72. "DtWI's presence helped Rajasthan to roll out the program quickly and error-free. If the government did not have DtWI’s experience, there could have been more problems." Veenu Gupta, Principal Secretary to School Education Department, Government of Rajasthan DtWI 2013 GiveWell government interviews, Pg 7. • 73. "DtWI has helped to improve deworming trainings. Last year, DtWI helped to develop the content for the trainings. DtWI made the content more concise and easy to understand than the government would have done on its own. DtWI also improves trainings by tracking when people do not show up to trainings and following up with them and by determining aspects of training that could be improved in future years." Girish Bharbwag, Nodal Officer in Rajasthan DtWI 2013 GiveWell government interviews, Pg 4. • 74. The District Coordinator position, for example, is not always staffed in other school based health programs. "The state government paid for District Coordinators for the hand washing program in 2008. However, the payment for this position was low. In the first year, there were 15-16 independent monitors. This year, there are only 7 District Coordinators. The government is finding it difficult to fill these vacancies." DtWI 2013 GiveWell government interviews, Pg 5. "District Coordinators (temporary DtWI employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by DtWI that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 4. • 75. "The presence of District Coordinators, combined with the independent monitors hired by DtWI that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make." DtWI 2013 GiveWell government interviews, Pg 5. • 76. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013 • 77. "Single-dose oral therapies can kill the worms, reducing ... infections by 99 percent ... Reinfection is rapid, however, with worm burden often returning to eighty percent or more of its original level within a year ... and hence geohelminth drugs must be taken every six months and schistosomiasis drugs must be taken annually." Miguel and Kremer 2004, Pg 161. • 78. Devesh Kapur conversation October 14th 2013 • 79. Total government costs from Bihar 2011, Bihar 2012, Delhi 2012, Rajasthan 2012 are$14,253,193.
Total all-inclusive costs from the same programs are $15,935,050. See DtWI Bihar 2011 cost data, DtWI Bihar 2012 cost data, DtWI Delhi 2012 cost data, DtWI Rajasthan 2012 cost data See What do you get for your dollar? Section for more details. • 80. • "There's limited data available on current access to deworming in India because very few prevalence surveys have been done and because the deworming that does occur is not always reported, or, if it is, state-wide data is difficult to access. The poorest states are unlikely (in CIFF's view) to have the capacity to implement evidence-based statewide deworming programs on their own. CIFF notes that many parts of India are extremely poor with high percentages (60%) of the population practicing open defecation; limited access to sanitation services makes it likely that deworming is needed." CIFF conversation September 10th 2013, Pg 2. • "District Coordinators (temporary DtWI employees that play a monitoring and evaluation role) are important because they provide reliable feedback to the government about any problems with the deworming program. Typically, the government must rely on government officers to monitor school health programs. However, these officers often fix any problems that they see and then do not report them to the state government because they are worried that the existence of problems will reflect negatively on them. District Coordinators hired and managed by non-governmental organizations are more likely to report problems. The presence of District Coordinators, combined with the independent monitors hired by DtWI that were known to show up unannounced to inspect the program, makes everyone more careful and more likely to implement the program properly because they know that people are paying attention and that they will receive feedback about any mistakes that they make. The District Coordinators and DtWI’s tele-callers were valuable because they were able to confirm that schools received the appropriate amount of drugs and that teachers had been trained. DtWI called a random sample of 8,000 schools. The prevalence survey would not have happened without DtWI’s support." DtWI 2013 GiveWell government interviews, Pg 5. • 81. "India is an appealing place to implement a deworming program because the Indian government has policies in place mandating deworming and makes money available to states to implement deworming programs, but many states don't have the background or the expertise to implement deworming programs effectively. It is possible that DtW brings its expertise to the table and leverages these available resources." CIFF conversation September 10th 2013, Pg 2. • 82. "RPs tend to have enough capacity that adding further school health programs would not take away from the work they do for other school-based health programs." DtWI 2013 GiveWell site visit, Pg 3. • 83. "[The Nodal Head Master said] that most aspects of the program are excellent, but he had 2 suggestions: - Deworm students in private schools as well (even though they have more money and can often buy treatment, they will often not do so) Reduce the number of health programs throughout the year; it takes away from teaching time. His school has school health programs on 40 to 42 days each year." DtWI 2013 GiveWell site visit, Pg 6. • 84. DtWI provided breakdowns by cost category for round 2 in Bihar and the first rounds in Delhi and Rajasthan. For Bihar round 1, it provided total costs covered by each DtWI and the government, but did not provide a breakdown by cost category. For that reason, we include Bihar round 1 in the overall cost analysis, but exclude it from the breakdown by cost category. Numbers in the below table are calculated in DtWI cost per treatment summary 2013, Sheet DtWI v partners. • 85. "Deworming Day Support: School-based deworming programs require the time of teachers, principals and government officials. The time involved for each type of personnel to prepare for deworming, conduct training and/or sensitization, and record and report results is estimated. This time, along with estimates of the number of government officials and teachers involved and their respective average salaries, is used to estimate this cost component of the cost. These costs reflect the large number of schools involved in this program." DtWI cost narrative 2013, Pg 2. • 86. "Training: Training costs that were not incurred directly by the Deworm the World Initiative were imputed from market rates. Training material costs are based on estimates from experiences with printing in the state. Honoraria and travel allowances for training participants are based on estimates of daily stipends for trainers and attendees, and associated travel allowances. Venue related costs are estimated based on costs of venues of similar size across the state." DtWI cost narrative 2013, Pg 1. • 87. "Drugs: a. Procurement: In cases where donated drugs were provided to the b. Drug packaging, testing, and distribution: Direct costs incurred for programs at no cost, the value of those tablets is imputed based on market rates for those tablets in India at the time. When the Government directly procured the drugs, those costs were calculated on the basis of discussion with government officials and estimates based on market rates. These activities are included, along with in-kind support such as personnel for drug repackaging, for which market based estimates of those costs are used." DtWI cost narrative 2013, Pg 1. • 88. Program Planning and Management: The costs of overall programmatic support provided by governments are estimated based on market estimates of the salaries and time of the personnel involved." DtWI cost narrative 2013, Pg 2. • 89. "Public Awareness and Mobilization: Media campaign costs incurred by the government and partners are estimated based on market rates for TV, radio, electronic, and print media advertising across the state. Information on the number of the different types and modes of advertising was collected from government sources for these calculations." DtWI cost narrative 2013, Pg 1. • 90. "Prevalence Survey: These include all direct costs involved in the execution of worm prevalence and intensity surveys conducted in the field, as well as imputed costs for services provided in kind by the government. The latter may include the provision of laboratories, manpower in the form of technicians, and training support in terms of venue and manpower." DtWI cost narrative 2013, Pg 1. • 91. "Monitoring and Evaluation: a. Training of monitors: Where the government provided trainers, training materials and/or training facilities, these costs are estimated based on market rates. b. Monitoring personnel: Where monitors were provided by the government, the cost of their time and travel allowances are estimated. Costs associated with time taken by government officials for reporting purposes or collecting forms, is imputed." DtWI cost narrative 2013, Pg 1-2. • 92. For example, in order to account for the cost of centralized organizational costs such as fundraising expenses, DtWI increases its cost estimate for each campaign by a flat percentage. DtWI Rajasthan 2012 cost data, DtWI Delhi 2012 cost data, DtWI Bihar 2012 cost data. • 93. DtWI cost per treatment summary 2013, Summary. • 94. "We note that preliminary estimates of program costs in Kenya, which we can estimate with far greater precision, suggest programming costs there of about$0.40 per child." Evidence Action cover letter 2013, Pg 2.

• 95.

Details on our Reanalysis of the Miguel and Kremer deworming experiment page.

• 96.

"…absent from most of the country, risk exists only in restricted areas." World Schistosomiasis Risk Chart 2012, Pg 1.

"INDIA - Risk is limited to the area around Gimvi in Ratnagiri district (Maharashtra) in the hills along the Konkan coast south of Mumbai (approximately 16km from shore)." World Schistosomiasis Risk Chart 2012, Pg 3.

• 97. using Miguel and Kremer 2004 definitions of moderate intensity, which are not the same epg levels as WHO definitions for all worms
• "Following Brooker, Miguel, et al. (2000), thresholds for moderate infection are 250 epg for Schistosomiasis. mansoni and 5,000 epg for Roundworm, the WHO standards, and 750 epg for Hookworm and 400 epg for Whipworm, both somewhat lower than the WHO standard." Miguel and Kremer 2004, Pg. 167.
• 98.
• There is a pre-existing LF program that provides albendazole to a large portion of the population in Bihar annually.
• Based on timing of the decision to deworm in Bihar, there was only time to collect prevalence survey from two districts before the annual LF treatment in 2010. Results from that first stage of surveying were sufficient to convince the state to carry out a full deworming program 6 months after the LF treatment, although the methodology did not meet DtWI's typical standard of quality.
• Before the DtWI-managed deworming program in early 2011, a second stage prevalence survey was conducted in an additional four states. These states were selected based in part on their complementarity with the original two states. DtWI Bihar 2011 prevalence survey report, Pg 9.
• Since the quality of the second stage was higher, our analysis excludes results from the first stage of the survey. Since the second stage districts were selected in part based on complementarity with the first stage, there may be bias introduced into the statewide estimates from lack of randomness.
• 99.
• The prevalence survey in Rajasthan was unable to measure prevalence or intensity of hookworm infections. Hookworm prevalence is expected to be significant in Rajasthan, and DtWI conducted a second prevalence survey in the state in September 2013 which is intended to more accurately measure hookworm prevalence. DtWI Rajasthan 2012 prevalence survey report.
• "Results also reflect underlying uncertainty in the data, which one might expect to be very high as 20% of the schools sampled examined less than 10 children. This poor response introduces a strong possibility of a bias, which based on the common assumption that risk is higher in non-enrolled children would likely result in an underestimate of prevalence.” DtWI Rajasthan 2012 prevalence survey report.
• "Areas in western Rajasthan are predicted to have a higher risk of Ascaris infection compared to eastern parts of the state. Figure 5 shows the probability that the predicted prevalence of Ascaris exceeds 20%, based on a model including EVI and altitude. While there is a high probability that areas in western Rajasthan will require at least annual MDA, there is likely to be a lower prevalence of Ascaris throughout the east of Rajasthan." DtWI Rajasthan 2012 prevalence survey report, Pg 14.
• "The climatic thresholds for hookworm transmission limits in Asia have been previously modelled using an aridity index, mean and maximum land surface temperature (LST). The map of these limits is shown in Figure 6, which categorises areas as unstable transmission (Mean LSTH 5-10°C, Aridity Index <0.2) or beyond transmission (Max LST >42°C, Mean LSTH <5°C, Aridity Index <0.03). These limits are based on experimental data showing that the development of infectious stages of hookworm ceases at 40° Celsius, and observed relationships between prevalence and environmental covariates using data from the Global Atlas of Helminth Infections. Based on these limits, the prevalence of hookworm in Rajasthan would be expected to oppose that of Ascaris and be more prevalent in the east of the state. All hookworm positive cases identified from this survey originated from areas within transmission limits." DtWI Rajasthan 2012 prevalence survey report, Pg 15-16.
• DtWI concluded, "Based on Ascaris prevalence levels in the surveyed districts, observed associations with environmental data, hookworm transmission limits and the possibility that data are an underestimate of the true prevalence, annual deworming throughout the state is recommended." DtWI Rajasthan 2012 prevalence survey report, Pg 17.
• 100. Data from Miguel and Kremer 2004 (Table II, Pg 168.) is provided in this table for comparison, as this paper represents what we believe to be the strongest piece of evidence of long term damages from intestinal parasites (see discussion in this section). We provide data from year one of the study, which is the time period before el nino weather conditions set in and increased the prevalence and intensity of local infections.
• 101. Hookworm infections were not measured in DtWI's 2012 Rajasthan survey. DtWI Rajasthan 2012 prevalence survey report
• 102. Hookworm infections were not measured in DtWI's 2012 Rajasthan survey. DtWI Rajasthan 2012 prevalence survey report
• 103.

The National Vector Borne Disease Control Programme LF treatment coverage 2012

• 104.

"DEC + Albendazole in selected distt & DEC in other distt" LF treatment drugs 2012.

• 105.

"Annual Mass Drug Administration with single dose of DEC was taken up as a pilot project covering 41 million population in 1996-97 and extended to 74 million population. This strategy was to be continued for 5 years or more to the population excluding children below two years, pregnant women and seriously ill persons in affected areas to interrupt transmission of disease.
Interruption of transmission of filariasis by Annual MDA for 5 years or more to the population except:
- children below 2 years
- pregnant women
- seriously ill persons
(DEC + Albendazole in selected distt & DEC in other distt)" LF treatment drugs 2012.

• 106.
• 107.

The LF program has recently been active in Bihar, but not Rajasthan or Delhi. LF treatment coverage 2012

• 108.
• Timothy Telleen-Lawton: 3x. I expect that DtWI's leverage value for some states will be about 1x, and for some states will be close to 10x (when they are a critical factor in the program happening at all in a given year). With large uncertainties about how much leverage they deserve credit for even within the states they operate currently, I would guess a geometrically central estimate of 3x.
• Elie Hassenfeld: 3x. Very few deworming programs exist in India without DtWI's involvement, and I suspect we'll see the pattern continue in the future. I recognize that this may only be a reflection of the fact that states that decide to deworm seek assistance from DtWI even though they might have chosen to deworm without DtWI's help.
• Holden Karnofsky: 3x. I feel roughly the same way as Elie and Tim.
• Alexander Berger: 2x. I agree with Tim's general reasoning but my guess is that in most cases, the impact of DtWI funding is less than its proportion (i.e. a multiplier below 1x), but that in some states it garners considerable leverage. I also think that the degree of leverage is likely to decline relatively quickly as the organization grows, and may be less than one at the current margin (i.e. DtWI may be "being leveraged" at this point instead of "leveraging additional funding").
• 109.

Alix Zwane, DtWI Executive Director, email exchange with GiveWell, November 2013

• 110.

"The current funding comes from USAID ($5 million over 5 years), the Michael & Susan Dell Foundation, and The Douglas B. Marshall Jr. Family Foundation, as well as donations made by smaller, individual donors." Alix Zwane conversation June 4th 2013, Pg 1. • 111.$483,000 is reported as the estimated costs to DtW in Bihar's first round. DtWI Bihar 2011 cost data, "Summary of prog costs" worksheet.

\$220,345 is reported as the estimated costs to DtWI in Bihar's second round. DtWI Bihar 2012 cost data, "DtWI expense breakdown" worksheet.

• 112.

According to DtWI, outreach to additional states has been conducted only in one state so far, and consisted of several phone calls and meetings. The initial meeting focused on providing information about DtWI’s technical support and expertise, including track record of success in other states. The government indicated that it had initiated deworming but suspended it a couple of years ago. DtWI's understanding is that preschool children are currently treated through an integrated vitamin distribution program. Subsequent communication in May 2013 indicated that the Government needed time to consider the various facets of a school-based deworming program internally among various stakeholders. Grace Hollister, DtWI Director, email exchange with GiveWell, October 2013

• 113.

Alix Zwane conversation August 30th 2013

• 114.
• 115.

Alix Zwane conversation August 30th 2013

• 116.

"How the deworming program in Rajasthan began:
· Ms. Gupta was a Commissioner Associate when she first heard about deworming, via media coverage of the deworming program in Delhi.
· After she heard about deworming, she spoke with other staff members about the possibility of doing deworming in Rajasthan. Then they got in touch with DtWI, and DtWI was very responsive." DtWI 2013 GiveWell government interviews, Pg 6.

• 117.

"Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage.

• 118.

"Two programs evaluated and incubated within Innovations for Poverty Action which are currently making a difference in the lives of millions of people in Africa and Asia – Dispensers for Safe Water and the Deworm the World Initiative - are transitioning to Evidence Action." Evidence Action website 2013, homepage.

GiveWell conducted a site visit to DSW in November 2012 and published notes. DSW 2012 GiveWell site visit

See GiveWell's review on Water quality interventions.

• 119.

Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013

• 120.

Alix Zwane, DtWI Executive Director, phone call with GiveWell, November 2013