Evidence Action's Deworm the World Initiative – August 2022 version

Evidence Action’s Deworm the World Initiative was one of GiveWell’s top-rated charities from 2013 to 2022. We updated our criteria for top charities in August 2022, and due to these changes, Deworm the World is no longer one of our top charities, but remains eligible to receive grants from our All Grants Fund.

This does not reflect an update to our view of Deworm the World. The change was motivated by our desire to clarify our recommendations to donors, not by any shift in our thinking about Deworm the World. More information is available in this blog post. (See Evidence Action's response here.)

We are no longer accepting donations designated for Deworm the World. You can support Deworm the World by donating directly here. If you would like to support GiveWell's grantmaking you can do so here—we plan to continue to support deworming through the All Grants Fund.


Published: November 2021; Last Updated: April 2023 (November 2021 version)

Summary

What do they do? Evidence Action's Deworm the World Initiative (evidenceaction.org/deworm-the-world) advocates for, supports, and evaluates government-run school-based deworming programs. (More)

Does it work? We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost. Deworm the World sends monitors to schools during and, for most distributions, after deworming to determine whether the programs it supports have reached a large proportion of children targeted. We have reviewed data from each of its major programs, which overall indicate strong results. (More)

What do you get for your dollar? Our best guess is that deworming is generally highly cost-effective. We estimate that, in Kenya, the cost per child dewormed is about $0.66 per child, or $0.46 per child excluding in-kind contributions from governments. We estimate that the cost per child treated in India is roughly half of that. We expect the cost per treatment in other countries to be closer to that in Kenya than India, and may be more expensive in the early stages of a program. The number of lives significantly improved is a function of a number of difficult-to-estimate factors, which we discuss in detail in a separate report. (More)

Is there room for more funding? Our most recent analysis finds that Deworm the World currently has $6.4 million in room for more funding—prior to receiving any grants that GiveWell specifically makes or recommends—to support its deworming portfolio in 2022-24. (More)

Evidence Action's Deworm the World Initiative is recommended because of its:

  • Focus on a program with a strong track record and excellent cost-effectiveness. (More)
  • Strong process for assessing whether the deworming programs it supports are successfully deworming children. (More)
  • Standout transparency – it has shared significant, detailed information about its programs with us.
  • Room for more funding – we believe Deworm the World will be able to use additional funds to start or maintain deworming programs.

Table of Contents

Our review process

Our review process has consisted of:

  • Extensive conversations with Deworm the World Director Grace Hollister and other Deworm the World and Evidence Action staff since 2012.1
  • Reviewing documents Deworm the World sent in response to our queries.
  • Site visits:
    • In November 2012, we visited Deworm the World's office in Nairobi, Kenya and met its staff there. (Notes from our visit)
    • In October 2013, we visited Deworm the World's operations in Rajasthan, India, where we met with its local staff and with government officials who had worked with Deworm the World. (Notes from our visit)
  • In 2015, we retained two journalists to visit areas served by Deworm the World in Kenya. We published their report on our blog.
  • Conversations with the Children's Investment Fund Foundation (CIFF), a funder of Deworm the World.2

All content on Deworm the World, including past reviews, updates, blog posts, and conversation notes, is available here. We have also published a page with additional, detailed information on the program to supplement some of the sections below.

What do they do?

The Deworm the World Initiative is a program led by Evidence Action, an organization that focuses on scaling up interventions that it believes are cost-effective and evidence-based. Deworm the World advocates for and supports the implementation of government-run deworming programs for preschool- and school-age children.3 The support that Deworm the World provides is of two types: 1) offering technical assistance to governments implementing deworming, and 2) funding components of deworming programs.4

The deworming programs that Deworm the World supports are focused on executing school-based mass drug administrations (MDAs), in which the aim is to treat the entire population of children within a geographic area by distributing deworming pills.5 Deworm the World focuses on MDAs that treat children infected with soil-transmitted helminthiasis (STH).6 Where needed, these programs also include treatment for schistosomiasis.7 In the countries it works in, Deworm the World works primarily with government staff to implement deworming programs; as it has expanded to new countries, it has started to also collaborate with non-governmental partners to support governments.8

Deworm the World was founded in 2007.9 Since then, it has supported deworming programs in India, Kenya, Ethiopia, Nigeria, Vietnam, and Pakistan. As of 2020, it was active in India, Kenya, Nigeria, and Pakistan.10 On a separate page with additional information about Deworm the World, we discuss more details of its work by country.

Below, we discuss:

  • Deworm the World's role in government-led deworming programs
  • A breakdown of Deworm the World’s spending
  • Deworm the World's relationship to Evidence Action

Deworm the World’s role in government-led deworming programs

The deworming programs that Deworm the World supports are implemented by the governments with which it works.11 Below, we expand on Deworm the World's role in the programs it supports. Note that we use "Deworming Day" to mean the day on which the MDA takes place. Similarly, we use "Mop-Up Day" to refer to the day that occurs several days after Deworming Day and is when students who were absent or sick on Deworming Day are given their deworming pills (although note that not all countries have just one Deworming Day or include a Mop-Up Day in their program).12

The assistance that Deworm the World provides in each country varies based on what each partnering government needs.13 Historically, Deworm the World's role has included the following:14

  1. Advocacy. Deworm the World actively encourages national and large sub-national governments to implement mass school-based deworming programs.15 Our impression is that Deworm the World's advocacy consists of meeting with health and education officials in a government to discuss the benefits of deworming and how a deworming program might be implemented.16 Deworm the World has told us that it will not work with a government on a national deworming program until it has built a strong working relationship with that government via its advocacy.17 Deworm the World also participates in the broader "STH community"; that is, it works with other organizations advocating for and implementing activities that aim to further reduce or eliminate STH globally.18
  2. Prevalence surveys. Before Deworm the World helps launch a deworming program in a new area, it evaluates whether the prevalence of worm infections is sufficient to justify an MDA for the school-age population.19 If no prevalence surveys have been conducted recently, it generally commissions one.20 The results of prevalence surveys are used to determine the appropriate treatment strategy (in particular, MDA frequency) for a given location.21 It also conducts follow-up prevalence surveys periodically, so that it can track the impact of the MDAs and refine treatment strategies as needed, in accordance with WHO guidelines.22 Deworm the World generally contracts out work on prevalence surveys.23
  3. High-level program planning. Deworm the World has told us that it often assists governments with high-level operational decisions, such as developing the country's treatment strategy and operational guidelines24 and creating a budget for the program.25
  4. Drug procurement and protocols. Deworm the World assists governments in obtaining drugs, designing drug distribution and tracking processes, and developing adverse event protocols for cases where children react poorly to treatment.26 For example, Deworm the World has helped governments submit requests for deworming drugs (albendazole or praziquantel) to the World Health Organization (WHO) global drug donation program.27
  5. Program preparation: trainings and distribution of materials. Deworm the World has helped governments design and organize what it calls a "training cascade" (more detail in the footnote).28 Through the training cascade, teachers and other government staff learn how to implement a Deworming Day and receive materials necessary for implementation (such as reporting forms and drugs).29 In the past, Deworm the World has hired or trained staff to lead the trainings and developed materials for the trainings.30 In India, Deworm the World has also arranged tele-callers to reach out to schools to assess their preparedness and notify government officials of any problems before Deworming Day.31
  6. Community sensitization. Deworm the World supports community sensitization efforts, which aim to make local communities aware of Deworming Day and the benefits of deworming children.32 For example, via the training cascade, teachers are instructed to spread the word about Deworming Day to their communities.33 Deworm the World has also developed text message campaigns, organized public announcement events, and edited mass media materials to be more appropriate for local contexts.34
  7. Monitoring and evaluation. Deworm the World told us that it helps governments design or improve reporting and monitoring systems. It also collects monitoring data independently.35 Deworm the World focuses on assisting with the collection of three main types of monitoring data:36
    • Monitoring before and during deworming: Monitors hired by Deworm the World visit schools before and during Deworming Day and Mop-Up Day. They are meant to assess both a) how prepared schools and health systems are to implement deworming and b) the extent to which proper procedures are followed.37 Monitoring visits may include assessments of the quality of trainings, community sensitization efforts, and Deworming Day activities, depending on what Deworm the World and the government agree to monitor.38
    • Coverage reporting: On Deworming Day and Mop-Up Day, teachers are asked to mark the number of children that they deworm and schools complete specially designed reporting forms to tally the number of children treated. This data is then aggregated and reported by school staff to government officials. Our understanding is that data is generally aggregated stepwise by officials at several levels (e.g., in India: school, node, block, district, and state) to create a reported coverage estimate for a region.39 Deworm the World notes that, as of at least 2017, block officials in India have submitted coverage data online to the national government.40
    • Coverage validation: Approximately one to two weeks after deworming day (or Mop-Up Day) in India, Deworm the World sends independent monitors back to schools to check the coverage data and attendance records recorded at schools against the data submitted and ask students about whether or not they were dewormed. In Kenya, Nigeria, and Pakistan, monitors also visit households during coverage validation to interview children who are not enrolled in school or are not at school at the time of the school visit.41 This data can then be compared to the coverage data reported by the government.

    Typically, Deworm the World hires and trains third-party monitors to collect process monitoring and coverage validation data; the following footnote includes Deworm the World’s descriptions of the monitor selection process used in four states in India in 2015.42 In Kenya, Evidence Action (Deworm the World's parent organization) maintains a monitoring team year-round that Deworm the World makes use of; this team conducts some data collection, and trains and manages third-party firms that conduct the majority of the data collection.43 In Nigeria, all data collection is conducted by third-party firms that are trained and managed by Evidence Action staff. In Pakistan, all data is collected by third-party firms that are hired and trained by Evidence Action’s in-country partner.44

On a separate page, we detail Deworm the World's work by country.

Breakdown of Deworm the World’s spending

We summarize Deworm the World's spending from 2017 through 2021 in this spreadsheet.

In short:

  • Deworm the World spent a total of $11.3 million on deworming programs in 2021, up from $8.7 million in 2020.45 Our understanding is that the lower rate of spending in 2020 was largely explained by program disruptions caused by the COVID-19 pandemic, which included some rounds of deworming being missed,46 other rounds being modified,47 and monitoring activities being constrained.48 We have not asked Deworm the World for details on how modifications made to program delivery and monitoring affected program costs in 2020.
  • In 2021, about 16% of its spending was from funding sources that were restricted to a particular project.49 This restricted funding supported programs in India, Kenya, Nigeria, and Pakistan, which were funded by CIFF, the END Fund, EPIC Foundation UK and EPIC Foundation France, and Dubai Cares.50 The other 84% of funding was from funds that Deworm the World can allocate at its discretion (this includes GiveWell-directed funds).51 Deworm the World used this unrestricted funding to support a portion of program costs in India, Kenya, Nigeria, and Pakistan.
  • India, Kenya, Nigeria, and Pakistan each accounted for around 25% of Deworm the World's program spending.52

For information on spending in previous periods, see the previous versions of our review of Deworm the World.

Deworm the World and Evidence Action

In early 2013, Innovations for Poverty Action (IPA) announced the formation of Evidence Action to scale cost-effective and evidence-based programs. Two IPA initiatives, Deworm the World and Dispensers for Safe Water, were spun off from IPA to be managed by Evidence Action.53 Evidence Action subsequently built a department called Evidence Action Beta for investigating, testing, and considering new programs for scaling up;54 one program in the Beta portfolio was No Lean Season, which GiveWell also recommended as a top charity in 2017. We removed No Lean Season from our list of top charities in 2018; see this blog post for more detail. In 2019, Evidence Action replaced Evidence Action Beta with Evidence Action Accelerator, which will continue the work of developing new programs. We focus this review specifically on the Deworm the World program.

Does it work?

This section was last updated in November 2020. The information that Deworm the World has provided since that date is not yet reflected in this section.

We believe that there is strong evidence that administration of deworming drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.

We believe the evidence from Deworm the World's monitoring makes a relatively strong case that the programs it has supported have successfully dewormed children. Here we focus on Deworm the World's monitoring from Kenya, India, Nigeria, and Pakistan because those are its largest programs.55 Deworm the World's track record in Kenya and India is strong. Deworm the World's track record in Nigeria and Pakistan is more limited, but what we have seen seems fairly strong.

In the sections below, we focus on the following questions to understand whether Deworm the World’s activities are having the intended impact.

  • Are mass school-based deworming programs effective when implemented well?
  • Are Deworm the World's programs targeted at areas of need?
  • Are deworming pills delivered to and ingested by recipients?
  • How does Deworm the World affect program outcomes?
  • Are there any negative or offsetting impacts?

Are mass school-based deworming programs effective when implemented well?

Deworm the World supports mass school-based deworming programs, the independent evidence for which we discuss extensively in our intervention report on deworming programs. In short, we believe that there is strong evidence that administration of the drugs reduces worm loads but weaker evidence on the causal relationship between reducing worm loads and improved life outcomes; we consider deworming a priority program given the possibility of strong benefits at low cost.

There are some important differences between the type and severity of worm infections in the places Deworm the World works and the places where the key studies on improved life outcomes from deworming took place (which we discuss below). In particular, Deworm the World primarily provides support to mass drug administrations (MDAs) that treat populations in which fewer children are infected with soil-transmitted helminths and where the severity of infections tends to be lower (as compared to populations in the key studies we refer to above). In addition, some of the programs Deworm the World supports do not treat schistosomiasis because it is not endemic in the areas the programs support.56

Are Deworm the World's programs targeted at areas of need?

What is the likely impact per treatment in Deworm the World's programs compared with the independent studies on the impact of deworming?

In general, mass deworming programs treat everyone in a targeted demographic, regardless of whether each individual is infected (more). Because of this, the benefits (and therefore the cost-effectiveness) of a program are highly dependent on the baseline prevalence of worm infections.

In this section, we discuss how the disease burden in the areas Deworm the World works in compares to the places where the independent studies that form the evidence base for the impact of deworming were conducted. While it is our understanding that Deworm the World programs generally target areas that require mass treatment according to WHO guidelines,57 the disease burden in Deworm the World areas is on average lower than in the study areas, so our expectation is that the impact per child treated is lower in Deworm the World areas. We adjust our cost-effectiveness estimate accordingly (more below).

In this spreadsheet, we compare the worm prevalence in places where Deworm the World currently supports a program to the prevalence from the studies providing the best evidence for the benefits of deworming. The prevalences in the table may not be directly comparable to one another. Prevalence surveys were conducted in Madhya Pradesh and Chhattisgarh after multiple rounds of treatment, in Bihar and Rajasthan after one round of treatment, and in Uttar Pradesh after multiple rounds of treatment in some districts and one round in other districts.58 It is likely that prevalence was higher at baseline for these regions. Deworm the World notes that with a few exceptions, it conducts prevalence surveys prior to supporting treatment.59

Deworm the World notes that there are relevant methodological differences between the prevalence surveys, which makes them difficult to compare;60 we agree that this data is not ideal for our purposes but believe that it provides the best estimate we have and adjusting for baseline infection rates is an important part of our cost-effectiveness model.

Treatment for lymphatic filariasis

In some of the countries where Deworm the World works, there are existing programs to treat lymphatic filariasis (LF).61 Albendazole, the same drug used to treat STH, is usually used in combination with one additional drug to treat LF (and the same dosage is used for both treatments).62 For areas that have existing LF treatment programs, the effect of Deworm the World’s support may be to transition an area from once-per-year deworming treatment (for STH) to twice-per-year treatment.63

We detail what we know about the status of LF programs in the areas in which Deworm the World works on a separate page with additional information about Deworm the World.

Are deworming pills delivered to and ingested by recipients?

The information we have seen from monitors hired and trained by Deworm the World in India, Kenya, Nigeria, and Pakistan suggests that the programs successfully deliver pills to children, who then swallow them.

Additionally, prevalence surveys in Kenya, and to a lesser degree Bihar, India, show that the prevalence rates of STH and schistosomiasis have declined substantially since Deworm the World started supporting MDAs in those areas, providing additional evidence that the treatments are reaching recipients.64

Evidence from monitoring

Deworm the World conducts monitoring to assess the quality of program implementation ("process monitoring") and to evaluate government reports of what proportion of the target population was reached with deworming treatment in the previous MDA ("coverage validation"). We use results from past MDAs to understand the impact we should expect future MDAs to have. Specifically, we use coverage validation results about the proportion of targeted children reached, along with data on program spending, to estimate the cost of reaching a child with deworming. Our interpretation of these coverage validation results is informed by their comprehensiveness and the methodology used to collect them.

Comprehensiveness

See this spreadsheet for all process monitoring and coverage validation results we have seen from Deworm the World's programs in India, Kenya, Nigeria, Pakistan, Ethiopia, and Vietnam.

We focus our review on coverage validation results, which we believe provide the best indication of program impact. Specifically, we focus on results from India, Kenya, Nigeria, and Pakistan because those are Deworm the World's largest programs.65 We have seen coverage validation results from 2014-19 in Kenya and from 2015-19 in India; over the years, we have reviewed these results to varying degrees of depth due to their high degree of consistency across years. We have also reviewed coverage validation results from 2018-19 in Nigeria and 2019 in Pakistan, which was the first year that Deworm the World supported a program in that location. In general, in locations in Nigeria and Pakistan where Deworm the World has supported two deworming rounds per year,66 it has only conducted coverage validation for one of the two rounds, but conducted process monitoring for all rounds.67 We thus believe that we have seen a relatively thorough picture of the impact of Deworm the World's programs; we incorporate this assessment into our cost-effectiveness model.68

Methodology

For each of its programs, Deworm the World hires monitors (who are not associated with the government implementing the program) to:69

  • [Sometimes] Make calls to communities and schools before Deworming Day. In India, Deworm the World asks monitors to call, unannounced, a random selection of schools and/or communities before Deworming Day. During the calls, monitors interview teachers, headmasters, and other functionaries, asking a variety of questions to assess preparedness, such as whether or not the school has enough treatments for Deworming Day and if a representative from the school attended training. 70
  • Observe activities on Deworming Day and Mop-Up Day. In all of its programs, Deworm the World sends its monitors, unannounced, to observe a random sample of schools on Deworming Day and Mop-Up Day.71 At the schools, monitors interview teachers and school administrators to assess how prepared the school was for Deworming Day. For example, monitors often ask if the school has sufficient drugs for Deworming Day, whether or not a school representative attended training, and a variety of questions to test teachers' knowledge about the proper procedures for the MDA campaign, such as what the teacher should do if a child is feeling sick.72 Then, the monitors observe randomly selected classes, recording details about the Deworming Day activities, such as whether deworming is in progress, teachers are documenting who is dewormed, and teachers are watching to make sure that students swallow the pills.73 In Kenya, Nigeria, and Pakistan, monitors visit communities surrounding schools, select a sample of adults to interview, and ask questions to assess their awareness of Deworming Day. This helps Deworm the World determine how successful its community sensitization efforts were.74
  • Conduct a coverage validation exercise. In India and Kenya, Deworm the World has typically sent out monitors within 1 to 2 weeks (though sometimes longer) of Deworming Day and Mop-Up Day to conduct a coverage validation exercise. It has also done so in the two most recent program years in Nigeria and in its first program year in Pakistan.75

    In India, coverage validation takes place at a randomly selected sample of schools. In Kenya, Nigeria, and Pakistan, coverage validation takes place at households as well as schools in order to evaluate coverage for both enrolled and non-enrolled children.76 During household-based coverage validation, households are randomly selected, and then eligible children within households are interviewed about their experience on Deworming Day.77 During school-based coverage validation, monitors randomly select a small sample of students to interview, asking the children questions about their experience on Deworming Day, which are largely the same questions as those asked during household interviews.78 For example, they ask if the child received a pill and if the child swallowed the pill under supervision.79 In India, monitors also check class registers and record the number of students that were dewormed according to the school or anganwadi center's records.80

    In India, Deworm the World interviews only enrolled children who were present on either Deworming Day or Mop-up Day in schools where deworming happened on at least one of those days;81 attendance data is collected separately in India to assess the rate of absenteeism on Deworming Day and Mop-up Day.82 In Kenya, Nigeria, and Vietnam, Deworm the World interviews a sample of all children, regardless of whether they were present on Deworming Day or Mop-up Day or whether they are enrolled or non-enrolled.83

    While we believe that Deworm the World's monitoring is overall of high quality, we note a few ways in which its methodology may produce bias in the results:


  • Source URL: https://www.givewell.org/charities/deworm-world-initiative/August-2022-version