Aggregator

Summary

• Four of GiveWell's top charities support deworming---the mass distribution of medicines to children in poor countries to rid their bodies of schistosomiasis, hookworm, and parasites.
• GiveWell's recommendation relies primarily on research from western Kenya finding that deworming in childhood boosted income in adulthood. GiveWell has also placed weight on a study by Hoyt Bleakley of the hookworm eradication effort in the American South 100 years ago.
• I reviewed the Bleakley study and reach a different conclusion than he did: the deworming campaign in the American South did not coincide with breaks in long-term trends that would invite eradication as the explanation.
• GiveWell research staff took the conclusions of this post into account when updating their recommendations for the 2017 giving season. GiveWell continues to recommend deworming charities.
• I also reviewed a separate Bleakley study of the impacts of malaria eradication in the United States, Brazil, Colombia, and Mexico. My reading there is more supporting. I'm finalizing the write-ups now and will share them soon.

Read More

The post Questioning the evidence on hookworm eradication in the American South appeared first on The GiveWell Blog.

Summary

• Four of GiveWell’s top charities support deworming—the mass distribution of medicines to children in poor countries to rid their bodies of schistosomiasis, hookworm, and parasites.
• GiveWell’s recommendation relies primarily on research from western Kenya finding that deworming in childhood boosted income in adulthood. GiveWell has also placed weight on a study by Hoyt Bleakley of the hookworm eradication effort in the American South 100 years ago.
• I reviewed the Bleakley study and reach a different conclusion than he did: the deworming campaign in the American South did not coincide with breaks in long-term trends that would invite eradication as the explanation.
• GiveWell research staff took the conclusions of this post into account when updating their recommendations for the 2017 giving season. GiveWell continues to recommend deworming charities.
• I also reviewed a separate Bleakley study of the impacts of malaria eradication in the United States, Brazil, Colombia, and Mexico. My reading there is more supporting. I’m finalizing the write-ups now and will share them soon.

Introduction

After the latest refresh, GiveWell’s list of top charities includes four that support deworming—the mass distribution of medicines to children to rid their guts of certain parasites. Several dozen randomized studies measure the short-term effects of deworming programs (within a year or so) on everything from body weight to being in school.1The 2016 Campbell review finds 52 short-term studies with follow-up duration under five years. Most last one to two years. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); If intestinal worms were often fatal, then short-term gains against them might be measured in lives saved, which could on its own make a decisive case for deworming. But the symptoms are normally subtler. On the other hand, some research finds that the aftereffects last into adulthood. This is why the long-term effects of deworming dominate GiveWell’s estimates of the cost-effectiveness of charities that support it.

Unfortunately, only a handful of experimental studies assess deworming’s impacts over the long haul, and most of those are based on a single experiment in Kenya. For summaries, see this 2016 post, in the section entitled “The research on the long-term impacts of deworming.” This paucity of experimental evidence has led GiveWell to place weight on a non-experimental, historical study of deworming. Hoyt Bleakley‘s 2007 paper tracks the impacts of the campaign to eradicate hookworm from the American South a century ago.

As part of an ongoing effort to scrutinize the evidence on the long-term impacts of deworming (this, this), GiveWell worked over the past year to revisit the Bleakley study. With huge assists from Christian Smith, Zachary Tausanovitch, and Claire Wang, I have formed a fresh and critical assessment of the evidence. The hookworm eradication effort in the American South did not coincide with breaks in long-term trends that would invite eradication as the explanation. For example, after the eradication campaign, outcomes such as school attendance indeed rose faster for children in historically worm-endemic areas, which could be taken as a sign of success. But that trend began decades before eradication. The full write-up is in this new working paper.

As John D. Rockefeller, arguably the richest human in history, entered philanthropy just over a century ago, he was persuaded to back large-scale, scientifically informed public health campaigns—not unlike Bill Gates in our era. In 1910, he gave $1 million to create the Rockefeller Sanitary Commission for the Eradication of Hookworm Disease. Across eleven southern states from North Carolina to Texas, the RSC soon launched what today would be called the War on Worms. Drugs were dispensed to treat infected children. Doctors, teachers, and the public were educated about the importance of sanitation, especially the use of proper privies.

From a researcher’s point of view, the suddenness and success of the campaign, and its broad geographic sweep, offer hope for credible impact assessment. If, for example, school attendance rates jumped just as infection rates plunged, that could be a compelling sign of the knock-on effects of mass deworming of children. The Bleakley (2007) study recognizes and exploits this opportunity for impact assessment. Paralleling the modern research out of Kenya, the study finds that after the RSC campaign, children in formerly worm-afflicted areas went to school more (a short-term development) and earned more as adults (a long-term effect).

In this post, I’ll explain how the GiveWell reanalysis of the Bleakley (2007) hookworm research differs from Bleakley’s original. Then I will show you some graphs that tell most of the analytical story.

I have also reviewed the related Bleakley (2010) study of the impacts of malaria eradication in the United States, Brazil, Colombia, and Mexico. There, my conclusion is more positive. I hope to release and blog that review in the next few weeks.

What we did

The reanalysis of the Bleakley (2007) hookworm study included the following steps:

• Returning to primary sources to reconstruct the data set. The data and computer code for the study are not publicly available. In correspondence starting a year ago, Hoyt Bleakley stated that they are effectively inaccessible now. Re-gathering the data was a major undertaking because Bleakley culled nearly 50 variables from obscure, century-old books and articles. Some, such as the student-teacher ratio in each county of the eleven southern states, were found in state government reports that varied in completeness and reporting conventions. Christian Smith, Claire Wang, and, especially, Zachary Tausanovitch, poured many hours into this effort.
• Expanding the census data sets. Bleakley (2007) tracks outcomes such as school attendance, literacy, and income using U.S. census data. These come to us not from old books, but from the IPUMS online database. Until recently, all the IPUMS data sets were samples from a given year’s census records, taking, for example, one household from every fifth page of the enumeration. (Here’s a sample page from 1920 with my great-grandparents and family in rows 3–6.) When carrying out this research in 2003–05, Bleakley appears to have used the biggest sets then available, such as the 1-in-250 sample from the 1910 census and the 1-in-100 sample from 1920. No data were then to be had from 1930. The GiveWell reanalysis takes advantage of the newer, bigger samples, including preliminary 100% samples for 1910–40. In aggregate, the new data set is about 100 times larger than that in Bleakley (2007).
• Copying choices from one Bleakley (2007) table or figure to another. For example, one table in the paper estimates impacts on school enrollment, school attendance, and literacy. A corresponding figure, discussed soon, only depicts impacts on attendance. In the new paper, I rerun the figure for all three outcomes.
• Imposing an arguably tougher standard for proof of impact. I concur with Bleakley that after the eradication campaign swept through the South in 1911–14, prospects improved disproportionately for children born in areas historically prone to hookworm. This catch-up, or convergence, surfaces in the data whether comparing counties within the South (low-lying counties tended to have more hookworm than mountainous ones), or comparing southern states to other states. But that observation alone leaves me unconvinced that ridding children’s bellies of hookworm was the cause. What if the trend began well before eradication or continued well after? I therefore focus on this question: Did convergence temporarily accelerate in tandem with eradication? The Bleakley (2007) tables and figures do not approach this question so aggressively.

We shared drafts of the paper and this post with Hoyt Bleakley. This did not yield any additional insight into why our analysis differs from the original.

The short-term impact on schooling

The figure below, adapted from one edition of the Bleakley study, illustrates the finding that I just mentioned, that after eradication, school attendance surged among kids living where hookworm had been common.2Versions of Bleakley (2007) appeared in the Quarterly Journal of Economics, a World Bank report, and the site of the National Center for Biotechnology Information. They are nearly the same. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); I will convey the gist of the figure first, then explain it more precisely. You can see that the central red line stays essentially flat from 1870 to 1910. Then it jumps to about zero between 1910 and 1920, census years bracketing the Rockefeller campaign. Thereafter, the red line mostly again holds steady. The one-time jump looks like a fingerprint of eradication.

What does the red line mean exactly? For each census round with available data between 1870 and 1950, Bleakley (2007) computes the association within Southern counties between the school attendance rate of 8–16-year-olds and the hookworm infection rate as measured at the start of eradication, circa 1910.3The regressions for each census year control for the interactions of sex and race on the one hand and age on the other. They do not include the other Bleakley (2007) controls. Samples are restricted to eleven Southern states. The unit of observation is the State Economic Area, which is an aggregation of several counties. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); That the red line starts around –0.1 in 1870 means that on average, if a county’s child hookworm infection rate was 10 percentage points higher when measured around 1910, its school attendance rate in the 1870 census was 1 percentage point lower. More plainly, counties with more worms in kids had fewer kids in school. But between the 1910 and 1920 censuses, that bad-news association abruptly faded. As of 1920, a child in a historically high-hookworm county was no less likely to be in school. The black, dashed lines show confidence intervals for these census-by-census estimates—probably 95% confidence, but I cannot tell for sure.

Here is the best replication of that graph using the reconstructed data and code. I have drawn it differently to emphasize that we only have data from certain decennial censuses, and to depict the gradations of confidence within the 95% confidence intervals.4The 1890 census records were destroyed in a fire. 1930 records had not been digitized at the time Bleakley did this work. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

I discern a resemblance between the original graph and the reconstruction. In both, school enrollment rises especially quickly between 1910 and 1920 and then declines slightly. But there is a difference too, and it is more than cosmetic. Now it appears that children in hookworm-infested areas gained substantially on school attendance not just between 1910 and 1920 but between 1880 and 1900 as well—and maybe throughout 1880–1910. For lack of access to Bleakley’s data and code, I cannot explain the discrepancy between this reconstruction and the original. There could be an error in the new or the old, or some subtle difference in data or method.

The new graph’s ambiguous mix of confirmation and contradiction forces a question that is at once conceptual and practical. How do we systematically judge whether the signal of hookworm eradication is present amidst the noise of other influences? To what degree does the new graph confirm or contradict the old?

I think there is no one best way to answer that question. One approach that I took is depicted with the red lines in the reconstructed graph above, and in the p values in the bottom-left corner. I drew the red lines to connect the dots that surround the eradication campaign. I wanted to quantify how much the red contour bends upward in 1910 and downward in 1920—as in Bleakley’s graph—and with what statistical significance. That is: Suppose the education gains took place at a constant pace between 1900 and 1940 with no acceleration around the campaign in the early 1910s. (I would have substituted 1930 for 1940 were 1930 data available in this graph.) What is the chance that we would see as much bending in the red line as we do? The computer says that for the upward kink at 1910, the answer is 0.37, which is not very low. On the other hand, the deceleration around 1920 is quite hard to ascribe to pure chance, at p = 0.03. Still, the new graph casts doubt on the proposition that the campaign brought a big break with the past.

Having settled on an analytical approach, the next step was to add all the census data that has been digitized since Bleakley did his work. This brings an obvious change (see below; now that 1930 data are included, I extend the third red line only that far). Now it looks far more as though the high-hookworm parts of the South began closing the schooling gap with low-hookworm parts around 1880, some 30 years before the hookworm campaign:

In a final test, I recomputed the graph while incorporating all the Bleakley (2007) control variables. Hookworm eradication was hardly a clean experiment, in the sense that the geographic reach of the disease was not random going in. The South had it more than the rest of the country; within the South, the coastal plains had it most. If the beneficiaries of eradication differed systematically from the rest before eradication, they could continue to differ after for reasons having little to do with hookworm prevention, creating a false appearance of impact. Striving to statistically remove such initial differences, Bleakley (2007) introduces into some of the regressions an aggressive set of controls. They relate to education, health, agriculture, and race. The paper includes these controls in some of the schooling regressions reported in a table, but does not bring them to the schooling graph shown above. It turns out that doing so (in our expanded-data graph) removes most signs of any long-term gains:

The lack of upward trend here does not mean that the historically hookworm-burdened parts of the South did not after all close a schooling gap between 1880 and 1920. It does suggest that the closure was correlated with, and therefore potentially caused by, the non-hookworm factors that Bleakley sometimes controls for.5Consistent with this graph, while the Bleakley (2007) full-controls regressions continue to put a statistically significant coefficient on the treatment proxy, the reconstructions do not. This is one of the few cases where the original results are not recognizable in the reconstruction. See Table 6, panel B, of the new paper. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

In sum, I do not see robust evidence that schooling and literacy improved at an historically anomalous rate circa 1910, in a way naturally attributable to hookworm eradication.

The long-term impact on earnings

What the first half of the Bleakley (2007) study does for short-term impacts on schooling, the second does for long-term impacts on earnings. Here too, the conclusion is encouraging. “Long-term follow-up,” writes Bleakley, “indicates a substantial income gain as a result of the reduction in hookworm infection.” This finding resonates strongly with the GiveWell cost-effectiveness analysis, which makes a key assumption about how much deworming children boosts future income. The number we use for that impact comes from modern, experimental research in Kenya; yet Bleakley’s inference from American history had boosted our confidence in the Kenya number. (That said, GiveWell has discounted the Kenya number by 80–90% out of fear that it won’t replicate to other settings.6See the “Replicability adjustment for deworming” row of the “Parameters” tab of the cost-effectiveness analysis spreadsheet. jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });)

The Bleakley (2007) graph I will focus on draws together data from censuses as ancient as 1870 and modern as 1990. One problem with measuring impacts on income over this span is that not until 1940 did Census takers begin asking people how much money they made. For this reason, the IPUMS census database provides proxies for income that reach back farther. One is the occupational income score (OIS), which is, approximately speaking, the average income in 1950 associated with a person’s census-reported profession. Thus, if lawyers averaged $10,000 in income in 1950, then any self-described lawyer between 1870 and 1990 is taken to earn that much. The OIS is expressed in hundreds of dollars of 1950, and is an example of an index of “occupational standing.”

Before scrutinizing the evidence of long-term impacts on occupational standing, I need to describe a twist that Bleakley (2007) introduces in moving from short- to long-term. As one tries to follow people over longer periods of time, the analytical tack that Bleakley took for schooling starts to break down. For it looks at how the people in given places fared over time. The problem is that sometimes people move—across the state or across the country. And in this analytical set-up, the researcher does not follow them. If deworming gave children in coastal Georgia more agency in life—better health, more education—perhaps they exercised that agency by moving to Atlanta. If we only looked at the incomes of the people who stayed behind, we would miss the full story.

To minimize this attrition from migration, in studying long-term impacts, Bleakley (2007) groups census records not by place of residence at the time of census, but by place of birth. Then, if a person was born in Georgia in 1915, showed up in the census in 1940 as a bricklayer in Atlanta, in 1950 as a general contractor in Lexington, and in 1960 as the manager of a construction company in Phoenix, all three census records would be associated with Georgia in 1915. After organizing the data this way, Bleakley (2007) could study whether children born in certain areas after eradication went on to earn more than those born in the same places before eradication.

Reorganizing the data this way generates two ripple effects. First, while census takers record place of residence with extreme precision, they only record place of birth by state. We cannot differentiate people by whether they were born in hookworm-prone areas within, say, Mississippi. We can only differentiate by whether they were native to a historically high-hookworm state such as Mississippi or a low-hookworm one such as Michigan. Thus, while the short-term analysis compares counties within 11 southern states, the long-term analysis compares states across the continental U.S.

The second ripple effect is that the data come to us at higher temporal resolution: by birth year, not census decade. In response, Bleakley (2007) hypothesizes that how much hookworm depressed adult earnings depended on the percentage of one’s childhood spent where it was endemic. If we take eradication to have occurred in 1910 and assume with Bleakley (2007) that childhood lasts 19 years, then babies born in or before 1891 would have reached adulthood before eradication, too soon to benefit. Babies born in endemic areas in 1892 would have been helped for one year (between their 18th and 19th birthdays); in 1893 for two; and so on. Those born in 1910 or later stood to reap the full 19 years of benefit. Bleakley (2007) therefore hypothesizes that the impact of eradication follows a sort of diagonal step shape with respect to birth year. The step starts rising in 1891 and stops in 1910. Bleakley depicted that contour with dashed lines in this figure:

As you can see, Bleakley (2007) fit this contour to data, to see how well it could explain historical patterns. These dots are derived much as in the earlier Bleakley (2007) figure. For example, the leftmost dot is for the year 1825, and has a vertical coordinate of about –2. That means that among people born in 1825, being native to a state whose hookworm infection rate circa 1910 was 10 percentage points (0.1) higher corresponded to having an Occupational Income Score 0.20 lower. That means $20/year less income throughout adulthood, in the dollars of 1950. The graph shows that this association was generally negative in the mid-19th century and generally positive after 1910: formerly, coming from a hookworm zone depressed lifetime earnings. And the graph suggests that the transition followed the step pattern expected if the cause was hookworm eradication.

Below is my best reproduction of that graph. As before, I have plotted both the dots and their 95% confidence intervals. I have avoided superimposing the step-like contour the way Bleakley (2007) does because I worry that it tricks the eye into believing that the contour fits the data better than it really does. But I have marked the years when the contour kinks, 1891 and 1910:

Here is the same graph when using the 100-times-bigger census data sets now available7In addition to adding data, this version mimics the rest of the Bleakley (2007) analysis in adding blacks and in fitting directly to census microdata rather than aggregates, in order to include controls for race, sex, and their interaction. jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });:

And here is the graph when I copy Bleakley (2010) in incorporating all the controls for cross-state differences in health and health policy, education policy, and other traits8As I noted, when looking at short-term impacts on education, Bleakley (2007) does not plot a graph while incorporating all controls. But now, when looking at long-term impacts on occupational standing, Bleakley (2007) does also include such a graph. See the bottom right of this figure. jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });:

Does it look to you like the upward trends in these last two graphs accelerated around 1891 and decelerated around 1910, as predicted by the Bleakley (2007) theory? To me, I have to say, not much. The climbs look steady and longer-term.

Since “not much” is muddy, I moved once again to formalize my interpretation. In analogy with my earlier graphs for schooling, I fit lines to the data points in the 19 years between 1891 and 1910, as well as to the 19 years on either side. Then I checked whether any bending in 1891 and 1910 is statistically significant. The final two graphs fit lines to the dots in the previous two. The dots in these next graphs are the same as in the previous two. It doesn’t look that way because I erased the grey confidence bars in order to expand the vertical scales.

In the both graphs the trend looks quite straight over the three generations surrounding the eradication campaign. The p values, shown in the bottom-right of each plot, are high.

Conclusion

Reanalyzing the Bleakley (2007) study left me unconvinced that the children who benefited from hookworm eradication went to school more or earned more as adults. Conceivably, if I had access to the original data and code, the confrontation with the reconstructed versions would expose errors in the GiveWell version that would alter my view. But this seems unlikely. The new census data sets are much bigger than the old, which improves precision. And most of the differences probably do not arise from clear-cut errors on either side, but from minor differences in implementation, such as taking education spending from a different edition of an annual government report. If the conclusions swing on such modest and debatable discrepancies, then they are not robust and reliable.

Finally, even if the two versions of the data matched exactly, I might still disagree on interpretation, since I use tests, illustrated above, that focus more exclusively on whether the time trends contain the temporal fingerprint of hookworm eradication. For me, that fingerprint is characterized not merely by once-high-hookworm areas catching up with low-hookworm ones, but catch-up that accelerates and decelerates at times that fit the timing of the eradication campaign.

The data and code for this study are here (1.6 GB). The full write-up is here.

 

Notes   [ + ]

1. ↑ The 2016 Campbell review finds 52 short-term studies with follow-up duration under five years. Most last one to two years. 2. ↑ Versions of Bleakley (2007) appeared in the Quarterly Journal of Economics, a World Bank report, and the site of the National Center for Biotechnology Information. They are nearly the same. 3. ↑ The regressions for each census year control for the interactions of sex and race on the one hand and age on the other. They do not include the other Bleakley (2007) controls. Samples are restricted to eleven Southern states. The unit of observation is the State Economic Area, which is an aggregation of several counties. 4. ↑ The 1890 census records were destroyed in a fire. 1930 records had not been digitized at the time Bleakley did this work. 5. ↑ Consistent with this graph, while the Bleakley (2007) full-controls regressions continue to put a statistically significant coefficient on the treatment proxy, the reconstructions do not. This is one of the few cases where the original results are not recognizable in the reconstruction. See Table 6, panel B, of the new paper. 6. ↑ See the “Replicability adjustment for deworming” row of the “Parameters” tab of the cost-effectiveness analysis spreadsheet. 7. ↑ In addition to adding data, this version mimics the rest of the Bleakley (2007) analysis in adding blacks and in fitting directly to census microdata rather than aggregates, in order to include controls for race, sex, and their interaction. 8. ↑ As I noted, when looking at short-term impacts on education, Bleakley (2007) does not plot a graph while incorporating all controls. But now, when looking at long-term impacts on occupational standing, Bleakley (2007) does also include such a graph. See the bottom right of this figure. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Questioning the evidence on hookworm eradication in the American South appeared first on The GiveWell Blog.

Summary

• Four of GiveWell’s top charities support deworming—the mass distribution of medicines to children in poor countries to rid their bodies of schistosomiasis, hookworm, and parasites.
• GiveWell’s recommendation relies primarily on research from western Kenya finding that deworming in childhood boosted income in adulthood. GiveWell has also placed weight on a study by Hoyt Bleakley of the hookworm eradication effort in the American South 100 years ago.
• I reviewed the Bleakley study and reach a different conclusion than he did: the deworming campaign in the American South did not coincide with breaks in long-term trends that would invite eradication as the explanation.
• GiveWell research staff took the conclusions of this post into account when updating their recommendations for the 2017 giving season. GiveWell continues to recommend deworming charities.
• I also reviewed a separate Bleakley study of the impacts of malaria eradication in the United States, Brazil, Colombia, and Mexico. My reading there is more supporting. I’m finalizing the write-ups now and will share them soon.

Introduction

After the latest refresh, GiveWell’s list of top charities includes four that support deworming—the mass distribution of medicines to children to rid their guts of certain parasites. Several dozen randomized studies measure the short-term effects of deworming programs (within a year or so) on everything from body weight to being in school.1The 2016 Campbell review finds 52 short-term studies with follow-up duration under five years. Most last one to two years. jQuery("#footnote_plugin_tooltip_1").tooltip({ tip: "#footnote_plugin_tooltip_text_1", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); If intestinal worms were often fatal, then short-term gains against them might be measured in lives saved, which could on its own make a decisive case for deworming. But the symptoms are normally subtler. On the other hand, some research finds that the aftereffects last into adulthood. This is why the long-term effects of deworming dominate GiveWell’s estimates of the cost-effectiveness of charities that support it.

Unfortunately, only a handful of experimental studies assess deworming’s impacts over the long haul, and most of those are based on a single experiment in Kenya. For summaries, see this 2016 post, in the section entitled “The research on the long-term impacts of deworming.” This paucity of experimental evidence has led GiveWell to place weight on a non-experimental, historical study of deworming. Hoyt Bleakley‘s 2007 paper tracks the impacts of the campaign to eradicate hookworm from the American South a century ago.

As part of an ongoing effort to scrutinize the evidence on the long-term impacts of deworming (this, this), GiveWell worked over the past year to revisit the Bleakley study. With huge assists from Christian Smith, Zachary Tausanovitch, and Claire Wang, I have formed a fresh and critical assessment of the evidence. The hookworm eradication effort in the American South did not coincide with breaks in long-term trends that would invite eradication as the explanation. For example, after the eradication campaign, outcomes such as school attendance indeed rose faster for children in historically worm-endemic areas, which could be taken as a sign of success. But that trend began decades before eradication. The full write-up is in this new working paper.

As John D. Rockefeller, arguably the richest human in history, entered philanthropy just over a century ago, he was persuaded to back large-scale, scientifically informed public health campaigns—not unlike Bill Gates in our era. In 1910, he gave $1 million to create the Rockefeller Sanitary Commission for the Eradication of Hookworm Disease. Across eleven southern states from North Carolina to Texas, the RSC soon launched what today would be called the War on Worms. Drugs were dispensed to treat infected children. Doctors, teachers, and the public were educated about the importance of sanitation, especially the use of proper privies.

From a researcher’s point of view, the suddenness and success of the campaign, and its broad geographic sweep, offer hope for credible impact assessment. If, for example, school attendance rates jumped just as infection rates plunged, that could be a compelling sign of the knock-on effects of mass deworming of children. The Bleakley (2007) study recognizes and exploits this opportunity for impact assessment. Paralleling the modern research out of Kenya, the study finds that after the RSC campaign, children in formerly worm-afflicted areas went to school more (a short-term development) and earned more as adults (a long-term effect).

In this post, I’ll explain how the GiveWell reanalysis of the Bleakley (2007) hookworm research differs from Bleakley’s original. Then I will show you some graphs that tell most of the analytical story.

I have also reviewed the related Bleakley (2010) study of the impacts of malaria eradication in the United States, Brazil, Colombia, and Mexico. There, my conclusion is more positive. I hope to release and blog that review in the next few weeks.

What we did

The reanalysis of the Bleakley (2007) hookworm study included the following steps:

• Returning to primary sources to reconstruct the data set. The data and computer code for the study are not publicly available. In correspondence starting a year ago, Hoyt Bleakley stated that they are effectively inaccessible now. Re-gathering the data was a major undertaking because Bleakley culled nearly 50 variables from obscure, century-old books and articles. Some, such as the student-teacher ratio in each county of the eleven southern states, were found in state government reports that varied in completeness and reporting conventions. Christian Smith, Claire Wang, and, especially, Zachary Tausanovitch, poured many hours into this effort.
• Expanding the census data sets. Bleakley (2007) tracks outcomes such as school attendance, literacy, and income using U.S. census data. These come to us not from old books, but from the IPUMS online database. Until recently, all the IPUMS data sets were samples from a given year’s census records, taking, for example, one household from every fifth page of the enumeration. (Here’s a sample page from 1920 with my great-grandparents and family in rows 3–6.) When carrying out this research in 2003–05, Bleakley appears to have used the biggest sets then available, such as the 1-in-250 sample from the 1910 census and the 1-in-100 sample from 1920. No data were then to be had from 1930. The GiveWell reanalysis takes advantage of the newer, bigger samples, including preliminary 100% samples for 1910–40. In aggregate, the new data set is about 100 times larger than that in Bleakley (2007).
• Copying choices from one Bleakley (2007) table or figure to another. For example, one table in the paper estimates impacts on school enrollment, school attendance, and literacy. A corresponding figure, discussed soon, only depicts impacts on attendance. In the new paper, I rerun the figure for all three outcomes.
• Imposing an arguably tougher standard for proof of impact. I concur with Bleakley that after the eradication campaign swept through the South in 1911–14, prospects improved disproportionately for children born in areas historically prone to hookworm. This catch-up, or convergence, surfaces in the data whether comparing counties within the South (low-lying counties tended to have more hookworm than mountainous ones), or comparing southern states to other states. But that observation alone leaves me unconvinced that ridding children’s bellies of hookworm was the cause. What if the trend began well before eradication or continued well after? I therefore focus on this question: Did convergence temporarily accelerate in tandem with eradication? The Bleakley (2007) tables and figures do not approach this question so aggressively.

We shared drafts of the paper and this post with Hoyt Bleakley. This did not yield any additional insight into why our analysis differs from the original.

The short-term impact on schooling

The figure below, adapted from one edition of the Bleakley study, illustrates the finding that I just mentioned, that after eradication, school attendance surged among kids living where hookworm had been common.2Versions of Bleakley (2007) appeared in the Quarterly Journal of Economics, a World Bank report, and the site of the National Center for Biotechnology Information. They are nearly the same. jQuery("#footnote_plugin_tooltip_2").tooltip({ tip: "#footnote_plugin_tooltip_text_2", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); I will convey the gist of the figure first, then explain it more precisely. You can see that the central red line stays essentially flat from 1870 to 1910. Then it jumps to about zero between 1910 and 1920, census years bracketing the Rockefeller campaign. Thereafter, the red line mostly again holds steady. The one-time jump looks like a fingerprint of eradication.

What does the red line mean exactly? For each census round with available data between 1870 and 1950, Bleakley (2007) computes the association within Southern counties between the school attendance rate of 8–16-year-olds and the hookworm infection rate as measured at the start of eradication, circa 1910.3The regressions for each census year control for the interactions of sex and race on the one hand and age on the other. They do not include the other Bleakley (2007) controls. Samples are restricted to eleven Southern states. The unit of observation is the State Economic Area, which is an aggregation of several counties. jQuery("#footnote_plugin_tooltip_3").tooltip({ tip: "#footnote_plugin_tooltip_text_3", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] }); That the red line starts around –0.1 in 1870 means that on average, if a county’s child hookworm infection rate was 10 percentage points higher when measured around 1910, its school attendance rate in the 1870 census was 1 percentage point lower. More plainly, counties with more worms in kids had fewer kids in school. But between the 1910 and 1920 censuses, that bad-news association abruptly faded. As of 1920, a child in a historically high-hookworm county was no less likely to be in school. The black, dashed lines show confidence intervals for these census-by-census estimates—probably 95% confidence, but I cannot tell for sure.

Here is the best replication of that graph using the reconstructed data and code. I have drawn it differently to emphasize that we only have data from certain decennial censuses, and to depict the gradations of confidence within the 95% confidence intervals.4The 1890 census records were destroyed in a fire. 1930 records had not been digitized at the time Bleakley did this work. jQuery("#footnote_plugin_tooltip_4").tooltip({ tip: "#footnote_plugin_tooltip_text_4", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

I discern a resemblance between the original graph and the reconstruction. In both, school enrollment rises especially quickly between 1910 and 1920 and then declines slightly. But there is a difference too, and it is more than cosmetic. Now it appears that children in hookworm-infested areas gained substantially on school attendance not just between 1910 and 1920 but between 1880 and 1900 as well—and maybe throughout 1880–1910. For lack of access to Bleakley’s data and code, I cannot explain the discrepancy between this reconstruction and the original. There could be an error in the new or the old, or some subtle difference in data or method.

The new graph’s ambiguous mix of confirmation and contradiction forces a question that is at once conceptual and practical. How do we systematically judge whether the signal of hookworm eradication is present amidst the noise of other influences? To what degree does the new graph confirm or contradict the old?

I think there is no one best way to answer that question. One approach that I took is depicted with the red lines in the reconstructed graph above, and in the p values in the bottom-left corner. I drew the red lines to connect the dots that surround the eradication campaign. I wanted to quantify how much the red contour bends upward in 1910 and downward in 1920—as in Bleakley’s graph—and with what statistical significance. That is: Suppose the education gains took place at a constant pace between 1900 and 1940 with no acceleration around the campaign in the early 1910s. (I would have substituted 1930 for 1940 were 1930 data available in this graph.) What is the chance that we would see as much bending in the red line as we do? The computer says that for the upward kink at 1910, the answer is 0.37, which is not very low. On the other hand, the deceleration around 1920 is quite hard to ascribe to pure chance, at p = 0.03. Still, the new graph casts doubt on the proposition that the campaign brought a big break with the past.

Having settled on an analytical approach, the next step was to add all the census data that has been digitized since Bleakley did his work. This brings an obvious change (see below; now that 1930 data are included, I extend the third red line only that far). Now it looks far more as though the high-hookworm parts of the South began closing the schooling gap with low-hookworm parts around 1880, some 30 years before the hookworm campaign:

In a final test, I recomputed the graph while incorporating all the Bleakley (2007) control variables. Hookworm eradication was hardly a clean experiment, in the sense that the geographic reach of the disease was not random going in. The South had it more than the rest of the country; within the South, the coastal plains had it most. If the beneficiaries of eradication differed systematically from the rest before eradication, they could continue to differ after for reasons having little to do with hookworm prevention, creating a false appearance of impact. Striving to statistically remove such initial differences, Bleakley (2007) introduces into some of the regressions an aggressive set of controls. They relate to education, health, agriculture, and race. The paper includes these controls in some of the schooling regressions reported in a table, but does not bring them to the schooling graph shown above. It turns out that doing so (in our expanded-data graph) removes most signs of any long-term gains:

The lack of upward trend here does not mean that the historically hookworm-burdened parts of the South did not after all close a schooling gap between 1880 and 1920. It does suggest that the closure was correlated with, and therefore potentially caused by, the non-hookworm factors that Bleakley sometimes controls for.5Consistent with this graph, while the Bleakley (2007) full-controls regressions continue to put a statistically significant coefficient on the treatment proxy, the reconstructions do not. This is one of the few cases where the original results are not recognizable in the reconstruction. See Table 6, panel B, of the new paper. jQuery("#footnote_plugin_tooltip_5").tooltip({ tip: "#footnote_plugin_tooltip_text_5", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });

In sum, I do not see robust evidence that schooling and literacy improved at an historically anomalous rate circa 1910, in a way naturally attributable to hookworm eradication.

The long-term impact on earnings

What the first half of the Bleakley (2007) study does for short-term impacts on schooling, the second does for long-term impacts on earnings. Here too, the conclusion is encouraging. “Long-term follow-up,” writes Bleakley, “indicates a substantial income gain as a result of the reduction in hookworm infection.” This finding resonates strongly with the GiveWell cost-effectiveness analysis, which makes a key assumption about how much deworming children boosts future income. The number we use for that impact comes from modern, experimental research in Kenya; yet Bleakley’s inference from American history had boosted our confidence in the Kenya number. (That said, GiveWell has discounted the Kenya number by 80–90% out of fear that it won’t replicate to other settings.6See the “Replicability adjustment for deworming” row of the “Parameters” tab of the cost-effectiveness analysis spreadsheet. jQuery("#footnote_plugin_tooltip_6").tooltip({ tip: "#footnote_plugin_tooltip_text_6", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });)

The Bleakley (2007) graph I will focus on draws together data from censuses as ancient as 1870 and modern as 1990. One problem with measuring impacts on income over this span is that not until 1940 did Census takers begin asking people how much money they made. For this reason, the IPUMS census database provides proxies for income that reach back farther. One is the occupational income score (OIS), which is, approximately speaking, the average income in 1950 associated with a person’s census-reported profession. Thus, if lawyers averaged $10,000 in income in 1950, then any self-described lawyer between 1870 and 1990 is taken to earn that much. The OIS is expressed in hundreds of dollars of 1950, and is an example of an index of “occupational standing.”

Before scrutinizing the evidence of long-term impacts on occupational standing, I need to describe a twist that Bleakley (2007) introduces in moving from short- to long-term. As one tries to follow people over longer periods of time, the analytical tack that Bleakley took for schooling starts to break down. For it looks at how the people in given places fared over time. The problem is that sometimes people move—across the state or across the country. And in this analytical set-up, the researcher does not follow them. If deworming gave children in coastal Georgia more agency in life—better health, more education—perhaps they exercised that agency by moving to Atlanta. If we only looked at the incomes of the people who stayed behind, we would miss the full story.

To minimize this attrition from migration, in studying long-term impacts, Bleakley (2007) groups census records not by place of residence at the time of census, but by place of birth. Then, if a person was born in Georgia in 1915, showed up in the census in 1940 as a bricklayer in Atlanta, in 1950 as a general contractor in Lexington, and in 1960 as the manager of a construction company in Phoenix, all three census records would be associated with Georgia in 1915. After organizing the data this way, Bleakley (2007) could study whether children born in certain areas after eradication went on to earn more than those born in the same places before eradication.

Reorganizing the data this way generates two ripple effects. First, while census takers record place of residence with extreme precision, they only record place of birth by state. We cannot differentiate people by whether they were born in hookworm-prone areas within, say, Mississippi. We can only differentiate by whether they were native to a historically high-hookworm state such as Mississippi or a low-hookworm one such as Michigan. Thus, while the short-term analysis compares counties within 11 southern states, the long-term analysis compares states across the continental U.S.

The second ripple effect is that the data come to us at higher temporal resolution: by birth year, not census decade. In response, Bleakley (2007) hypothesizes that how much hookworm depressed adult earnings depended on the percentage of one’s childhood spent where it was endemic. If we take eradication to have occurred in 1910 and assume with Bleakley (2007) that childhood lasts 19 years, then babies born in or before 1891 would have reached adulthood before eradication, too soon to benefit. Babies born in endemic areas in 1892 would have been helped for one year (between their 18th and 19th birthdays); in 1893 for two; and so on. Those born in 1910 or later stood to reap the full 19 years of benefit. Bleakley (2007) therefore hypothesizes that the impact of eradication follows a sort of diagonal step shape with respect to birth year. The step starts rising in 1891 and stops in 1910. Bleakley depicted that contour with dashed lines in this figure:

As you can see, Bleakley (2007) fit this contour to data, to see how well it could explain historical patterns. These dots are derived much as in the earlier Bleakley (2007) figure. For example, the leftmost dot is for the year 1825, and has a vertical coordinate of about –2. That means that among people born in 1825, being native to a state whose hookworm infection rate circa 1910 was 10 percentage points (0.1) higher corresponded to having an Occupational Income Score 0.20 lower. That means $20/year less income throughout adulthood, in the dollars of 1950. The graph shows that this association was generally negative in the mid-19th century and generally positive after 1910: formerly, coming from a hookworm zone depressed lifetime earnings. And the graph suggests that the transition followed the step pattern expected if the cause was hookworm eradication.

Below is my best reproduction of that graph. As before, I have plotted both the dots and their 95% confidence intervals. I have avoided superimposing the step-like contour the way Bleakley (2007) does because I worry that it tricks the eye into believing that the contour fits the data better than it really does. But I have marked the years when the contour kinks, 1891 and 1910:

Here is the same graph when using the 100-times-bigger census data sets now available7In addition to adding data, this version mimics the rest of the Bleakley (2007) analysis in adding blacks and in fitting directly to census microdata rather than aggregates, in order to include controls for race, sex, and their interaction. jQuery("#footnote_plugin_tooltip_7").tooltip({ tip: "#footnote_plugin_tooltip_text_7", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });:

And here is the graph when I copy Bleakley (2010) in incorporating all the controls for cross-state differences in health and health policy, education policy, and other traits8As I noted, when looking at short-term impacts on education, Bleakley (2007) does not plot a graph while incorporating all controls. But now, when looking at long-term impacts on occupational standing, Bleakley (2007) does also include such a graph. See the bottom right of this figure. jQuery("#footnote_plugin_tooltip_8").tooltip({ tip: "#footnote_plugin_tooltip_text_8", tipClass: "footnote_tooltip", effect: "fade", fadeOutSpeed: 100, predelay: 400, position: "top right", relative: true, offset: [10, 10] });:

Does it look to you like the upward trends in these last two graphs accelerated around 1891 and decelerated around 1910, as predicted by the Bleakley (2007) theory? To me, I have to say, not much. The climbs look steady and longer-term.

Since “not much” is muddy, I moved once again to formalize my interpretation. In analogy with my earlier graphs for schooling, I fit lines to the data points in the 19 years between 1891 and 1910, as well as to the 19 years on either side. Then I checked whether any bending in 1891 and 1910 is statistically significant. The final two graphs fit lines to the dots in the previous two. The dots in these next graphs are the same as in the previous two. It doesn’t look that way because I erased the grey confidence bars in order to expand the vertical scales.

In the both graphs the trend looks quite straight over the three generations surrounding the eradication campaign. The p values, shown in the bottom-right of each plot, are high.

Conclusion

Reanalyzing the Bleakley (2007) study left me unconvinced that the children who benefited from hookworm eradication went to school more or earned more as adults. Conceivably, if I had access to the original data and code, the confrontation with the reconstructed versions would expose errors in the GiveWell version that would alter my view. But this seems unlikely. The new census data sets are much bigger than the old, which improves precision. And most of the differences probably do not arise from clear-cut errors on either side, but from minor differences in implementation, such as taking education spending from a different edition of an annual government report. If the conclusions swing on such modest and debatable discrepancies, then they are not robust and reliable.

Finally, even if the two versions of the data matched exactly, I might still disagree on interpretation, since I use tests, illustrated above, that focus more exclusively on whether the time trends contain the temporal fingerprint of hookworm eradication. For me, that fingerprint is characterized not merely by once-high-hookworm areas catching up with low-hookworm ones, but catch-up that accelerates and decelerates at times that fit the timing of the eradication campaign.

The data and code for this study are here (1.6 GB). The full write-up is here.

 

Notes   [ + ]

1. ↑ The 2016 Campbell review finds 52 short-term studies with follow-up duration under five years. Most last one to two years. 2. ↑ Versions of Bleakley (2007) appeared in the Quarterly Journal of Economics, a World Bank report, and the site of the National Center for Biotechnology Information. They are nearly the same. 3. ↑ The regressions for each census year control for the interactions of sex and race on the one hand and age on the other. They do not include the other Bleakley (2007) controls. Samples are restricted to eleven Southern states. The unit of observation is the State Economic Area, which is an aggregation of several counties. 4. ↑ The 1890 census records were destroyed in a fire. 1930 records had not been digitized at the time Bleakley did this work. 5. ↑ Consistent with this graph, while the Bleakley (2007) full-controls regressions continue to put a statistically significant coefficient on the treatment proxy, the reconstructions do not. This is one of the few cases where the original results are not recognizable in the reconstruction. See Table 6, panel B, of the new paper. 6. ↑ See the “Replicability adjustment for deworming” row of the “Parameters” tab of the cost-effectiveness analysis spreadsheet. 7. ↑ In addition to adding data, this version mimics the rest of the Bleakley (2007) analysis in adding blacks and in fitting directly to census microdata rather than aggregates, in order to include controls for race, sex, and their interaction. 8. ↑ As I noted, when looking at short-term impacts on education, Bleakley (2007) does not plot a graph while incorporating all controls. But now, when looking at long-term impacts on occupational standing, Bleakley (2007) does also include such a graph. See the bottom right of this figure. function footnote_expand_reference_container() { jQuery("#footnote_references_container").show(); jQuery("#footnote_reference_container_collapse_button").text("-"); } function footnote_collapse_reference_container() { jQuery("#footnote_references_container").hide(); jQuery("#footnote_reference_container_collapse_button").text("+"); } function footnote_expand_collapse_reference_container() { if (jQuery("#footnote_references_container").is(":hidden")) { footnote_expand_reference_container(); } else { footnote_collapse_reference_container(); } } function footnote_moveToAnchor(p_str_TargetID) { footnote_expand_reference_container(); var l_obj_Target = jQuery("#" + p_str_TargetID); if(l_obj_Target.length) { jQuery('html, body').animate({ scrollTop: l_obj_Target.offset().top - window.innerHeight/2 }, 1000); } }

The post Questioning the evidence on hookworm eradication in the American South appeared first on The GiveWell Blog.

If you’re thinking about where to give to charity this year and it would be helpful to speak with a member of GiveWell’s staff about your decision, please let us know. We’re happy to answer questions sent to info@givewell.org or to schedule a call via the form here.

We know we publish a lot of information, and we’re glad to provide a brief overview of our headline recommendations. We can also answer questions about our process for finding top charities, the strengths and weaknesses of our top charities, or how your personal values might point you toward one organization we recommend over another. Conversations like these also help us understand how people use our research and what questions they have.

Due to limited staff capacity, it’s possible we won’t be able to speak with everyone who requests a call, although based on past experience we hope to be able to connect with anyone who gets in touch.

We look forward to hearing from you!

The post Want to talk to someone at GiveWell about your giving decision? appeared first on The GiveWell Blog.

If you’re thinking about where to give to charity this year and it would be helpful to speak with a member of GiveWell’s staff about your decision, please let us know. We’re happy to answer questions sent to info@givewell.org or to schedule a call via the form here.

We know we publish a lot of information, and we’re glad to provide a brief overview of our headline recommendations. We can also answer questions about our process for finding top charities, the strengths and weaknesses of our top charities, or how your personal values might point you toward one organization we recommend over another. Conversations like these also help us understand how people use our research and what questions they have.

Due to limited staff capacity, it’s possible we won’t be able to speak with everyone who requests a call, although based on past experience we hope to be able to connect with anyone who gets in touch.

We look forward to hearing from you!

The post Want to talk to someone at GiveWell about your giving decision? appeared first on The GiveWell Blog.

This year, we added two new top charities, Evidence Action’s No Lean Season program and Helen Keller International’s vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action’s Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Our top charities and recommendations for donors, in brief

Top charities

We now have nine top charities. They are:

• Against Malaria Foundation
• Schistosomiasis Control Initiative
• Malaria Consortium’s seasonal malaria chemoprevention (SMC) program
• Evidence Action’s Deworm the World Initiative
• Helen Keller International (HKI)’s vitamin A supplementation (VAS) program
• Sightsavers’ deworming program
• END Fund’s deworming program
• Evidence Action’s No Lean Season program
• GiveDirectly

Standout charities

We also provide a list of standout charities. We believe they are implementing programs that are evidence-backed and may be extremely cost-effective. However, we do not feel as confident in the impact of these organizations as we do in our top charities.

• Development Media International
• Evidence Action’s Dispensers for Safe Water program
• Food Fortification Initiative
• Global Alliance for Improved Nutrition’s Universal Salt Iodization program
• Iodine Global Network
• Living Goods
• Project Healthy Children

Conference call to discuss recommendations

We are planning to hold a conference call at 1:30pm ET/10:30am PT on Thursday, November 30 to discuss our charity recommendations and answer your questions.

If you’d like to join the call, please register using this online form. If you can’t make this date, but would be interested in joining another call at a later date, please indicate this on the registration form.

Additional details and explanation

Below, we provide:

• An explanation of changes to our recommended charity list and of major changes to our review process in the past year that are not specific to any one organization. More
• A discussion of our approach to determining how much funding charities can use effectively (“room for more funding”) and our ranking of charities’ funding gaps. More
• Reasoning behind how we have ranked charities’ funding gaps. More
• Details about each of our new top charities, including an overview of what we know about their work and our understanding of their funding needs. More
• Details about each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year, and our understanding of their funding needs. More
• A brief overview of each of our standout charities. More
• The process we followed that led to these recommendations. More
• An update on giving to support GiveWell’s operations versus giving to our top charities. More

Major changes in the last 12 months

Major changes to our recommended charities list and review process over the past year include:

• Overall, we believe our top charities are able to absorb more funding than they could in previous years. This is a result both of recent additions to the top charities list with large funding gaps (particularly Malaria Consortium) as well as expansion by top charities that have been on the list for a longer time (particularly Deworm the World and AMF).

  We expect overall “room for more funding” to continue to expand as we gain more confidence in recently-added top charities and continue to add new top charities, particularly through GiveWell Incubation Grants, our program to grow the pipeline of potential future top charities and improve our understanding of our current top charities.

• We added two new programs to our list of top charities: vitamin A supplementation (VAS) and seasonal migration subsidies. We have not previously recommended charities that work on these programs.

  We had considered VAS a priority program for a number of years but had not found an organization that was able to answer our key questions. While we have some remaining questions, we can now make a strong case for supporting HKI’s work on VAS.

  We initially supported No Lean Season through GiveWell’s Incubation Grants program. No Lean Season is the first organization we have added to our top charity list through our Incubation Grants program.

• Last year, the charities we recommended on the margin were estimated to be about three times as cost-effective as unconditional cash transfers, the program implemented by top charity GiveDirectly. This year, we believe that the charities we are recommending on the margin are about six times as cost-effective as cash transfers. For the most part, this change was due to (a) a series of small adjustments to our cost-effectiveness model and (b) changes in which individuals contribute to the model and the values entered into the model by these and other contributors.

  We now feel fairly confident that there will be large amounts of room for more funding in this range. As more time has passed without identifying opportunities that are considerably more cost-effective than this, we have become more pessimistic about finding such opportunities. Our current best guess is that, if they exist, they will be in the area of policy advocacy in developing countries, on issues like lead regulation and tobacco taxation. We intend to do further research in those areas.

• We made a significant change to our cost-effectiveness analysis to more formally incorporate adjustments for the way in which our top charities’ funding affects funding from other sources by (a) attracting more resources to the programs they work on (e.g., governments contributing staff time to support implementation of the programs) or (b) displacing resources that would have otherwise supported the programs. We will be writing more about this in a future post.

• We continued to analyze the complex evidence base for deworming (treating intestinal parasites), the program implemented by four out of our nine top charities.

  At the end of 2016, David Roodman, a Senior Advisor to GiveWell, conducted a detailed review of the core evidence underlying our deworming recommendation (blog posts here and here).

  This year, we saw new follow-up results on the main study that leads us to recommend deworming, which continued to show similar long-term impacts of deworming on adult earnings as were estimated previously.

  Further investigation and updates based on new data led us to believe that two deworming studies (Croke 2014 and Bleakley 2007) no longer provide substantial support for the theory that deworming has long-term impacts. We plan to write more about this in the future. All together, this work led us to the same conclusion about deworming: that it is a reasonable bet to take based on its strong cost-effectiveness (which incorporates our uncertainty about the impact).

Room for more funding analysis

Types of funding gaps

In the last two years, we used a framework of “capacity funding” and “execution levels” to compare funding gaps (unfilled funding needs) across charities. This framework was intended to capture whether funding would enable a charity to expand or grow in important ways and how likely it was, in our estimation, that each top charity would be constrained by funding in the next year.

We developed this approach in response to a situation where we expected to direct more funding to several of our top charities than they would be able to use (commit or spend) in that year. We used capacity funding to describe opportunities to increase the amount of funding a charity might be able to absorb in the future (by, say, investing in expanding to a new location) and execution levels to describe the likelihood, down to the 5 percent level, that a charity would be able to make use of additional funding before encountering non-funding bottlenecks to their work.

This year, because we have added new top charities and most of our other top charities have more room for more funding than in previous years, we expect that the funding we will direct to each organization will not reach the level where they will encounter significant non-funding bottlenecks. As a result, we have moved away from describing capacity funding and execution levels.

Ranking funding gaps

The first million dollars to a charity can have a very different impact from the 20th million dollars. Accordingly, we have created a ranking of individual funding gaps that accounts for our best guess of the impact of additional funds at each level.

The below table lays out our ranking of funding gaps, up to $75.7 million in total funding. We expect Good Ventures to give $75 million to GiveWell’s top charities this year, so this table is our recommendation to Good Ventures, plus the allocation of funding that GiveWell holds to allocate at its discretion (currently $0.7 million). We then discuss our recommendation for all other donors.

The Open Philanthropy Project, which was incubated at GiveWell but is now a separate organization, plans to write more soon about the reasons for Good Ventures increasing its support of GiveWell top charities from $50 million last year to $75 million this year. In short, the amount was based on discussions about how to allocate funding across time and across cause areas. It was not set based on the total size of top charities’ funding gaps or the projection of what others would give.

Charity Description Amount (millions) All top charities Incentive grants: $2.5 million per charity 22.5 All standout charities Standout grants: $100,000 per charity 0.7 Deworm the World Funding gaps in India and Kenya over the next three years (including central costs) 3.0 Helen Keller International Funding gaps over three years in Burkina Faso, Mali, and Guinea—countries that have missed recent vitamin A campaigns due to lack of funding 4.7 No Lean Season Full funding gap over three years for implementing the program in Bangladesh 9.0 Deworm the World Three years of funding for a new program in Pakistan and reserves to protect against funding shortfalls in India 10.4 Malaria Consortium Part of the funding gap for SMC in Burkina Faso, Nigeria, and Chad over the next three years 25.4

In total, we are recommending that Good Ventures make the following grants:

• Malaria Consortium’s seasonal malaria chemoprevention program: $27.9 million
• Evidence Action’s Deworm the World Initiative: $15.2 million. We are also recommending that GiveWell’s Board of Directors grant the $0.7 million in discretionary funds that we currently hold from the third quarter (from donors who selected to give to “Grants to recommended charities at GiveWell’s discretion” on our donation form) to Deworm the World, bringing the total to $15.9 million.
• Evidence Action’s No Lean Season program: $11.5 million
• Helen Keller International’s vitamin A supplementation program: $7.2 million
• Schistosomiasis Control Initiative: $2.5 million
• Against Malaria Foundation: $2.5 million
• Sightsavers’ deworming program: $2.5 million
• END Fund’s deworming program: $2.5 million
• GiveDirectly: $2.5 million

Our recommendation to donors

For donors who are interested in directing funding to whichever recommended charity or charities GiveWell believes has the most pressing funding need at the time the funds are granted, we recommend giving to “Grants to recommended charities at GiveWell’s discretion.” These grants will respond to the greatest funding need we see; they may not match the recommended allocation outlined below.

For donors (other than Good Ventures) who are interested in donating directly to our top charities, we recommend splitting your donation as follows:

• 70 percent to the Against Malaria Foundation
• 30 percent to the Schistosomiasis Control Initiative

Why these recommendations?

Our recommendations to donors, including Good Ventures, are based on:

1. Overall cost-effectiveness of the charity. Our cost-effectiveness model is a key input into our decision-making process, and large differences in modeled cost-effectiveness impact our recommendations. We try not to put significant weight on relatively small differences in cost-effectiveness according to the model because many inputs are highly uncertain.

   Our model this year found relatively small differences between many top charities, with Deworm the World at ~12 times as cost-effective as cash transfers, four top charities in the ~6-10x cash transfers range, and three top charities in the ~3-5x cash transfers range. We consider differences between charities implementing the same intervention or interventions that have similar inputs and output in the model more meaningful (e.g., malaria nets and seasonal malaria chemoprevention) than differences between charities implementing quite different interventions.

   We have completed a sensitivity analysis of our cost-effectiveness analysis to get a better sense for which parameters are most sensitive. We are more hesitant to consider differences in the cost-effectiveness as meaningful when they rely on very sensitive inputs.

2. Cost-effectiveness of particular funding opportunities. Charities’ work can vary significantly in cost-effectiveness across locations due to different costs, disease burdens, uptake in the targeted population, or probability that other funders would step in in GiveWell’s absence. While not a part of our formal cost-effectiveness model, we ran supplementary analyses of cost-effectiveness for some locations for which our top charities were seeking additional funding and considered the output as part of our prioritization of funding gaps.
3. Qualitative factors not captured in our cost-effectiveness model. The main factors we focused on were:
   • Proportion of the global funding need for the program that is filled. We expect that funders will generally (but imperfectly) select the areas where cost-effectiveness is higher first, leaving the areas with higher costs, lower disease burden, lower cultural acceptance of the program, etc. for last. We believe we have captured some of the consequences of this in our cost-effectiveness analysis. For example, we use national level disease burden estimates for the countries in which each charity has worked and/or plans to work; charities working in higher burden countries are therefore modelled as more cost-effective. But we do not use sub-national estimates to distinguish the highest priority regions within a country; if charities are filling the lowest priority funding gaps within a county, they will likely be less cost-effective than our model suggests. This was an important consideration in comparing AMF and Malaria Consortium. We estimate that ~80 percent of the global funding need for nets (the program AMF implements) has been filled, and ~35 percent of the global funding need for seasonal malaria chemoprevention (the program Malaria Consortium implements).
   • Our level of knowledge about the organization. We have recommended AMF, Deworm the World, SCI, and GiveDirectly for many years. We know less about Malaria Consortium and No Lean Season and the least about HKI. We seek to be somewhat conservative about recommending large amounts of funding to organizations where there is a relatively high chance that additional research could lead us to believe the program was less cost-effective than we previously thought.
   • Ease of communication with the organization. It is important to us that we are able to learn over time about the charities we recommend, to enable us to improve our decisions. The ability to communicate effectively with an organization is a key factor in our ability to learn from the organization’s experiences.
   • Ongoing monitoring and likelihood of detecting future problems. Evaluating an organization’s monitoring processes and results is an important part of our charity reviews and for the most part is not captured in our cost-effectiveness analysis. As with ease of communication, we have more confidence in recommending funds to an organization if we believe that we will learn about how successful its work has been.

Summary of key considerations for top charities

The table below summarizes the key considerations for our nine top charities. More detail is provided below, as well as in the charity reviews.

Estimated cost-effectiveness (relative to cash transfers) Our level of knowledge about the organization Primary benefits of the intervention Ease of communication Ongoing monitoring and likelihood of detecting future problems Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation Other major considerations AMF ~6x High Deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars High proportion (~80%) of global gap for program is filled Malaria Consortium (SMC program) ~7x Moderate Under-5 deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars Relatively low proportion (~35%) of global gap for program is filled Helen Keller International (VAS program) ~9x Moderate Under-5 deaths averted Strong Moderate High: could absorb tens of millions of dollars Learning benefits Deworm the World ~12x High Possible increased income in adulthood Strong Strong Moderate: could absorb millions of dollars END Fund (deworming program) ~4x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars SCI ~10x High Possible increased income in adulthood Moderate Moderate High: could absorb tens of millions of dollars Sightsavers (deworming program) ~5x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars No Lean Season ~5x Moderate Immediate increase in consumption Strong Moderate Low: further funding would be used for different types of activities Potential upside GiveDirectly Baseline High Immediate increase in consumption and assets Strong Strong Very high: could absorb over 100 million dollars

Reasons for this funding gap ranking

Prioritization of funding that we have recommended to Good Ventures (we recommend Good Ventures fill the highest-priority funding needs first, to ensure these are funded):

1. We start by recommending that each top charity receive $2.5 million as an “incentive grant.” These grants are intended to be a major contribution to the charity’s work in recognition of the fact that they have met GiveWell’s criteria and have dedicated significant time to working with us to help us follow their progress and plans each year. We don’t want our top charity funding process to be winner-takes-all because we believe that charities would be less likely to want to participate in that case.
2. After incentive grants, we believe the next most valuable funding to provide is for Deworm the World’s work in Kenya and India over the next three years. Deworm the World’s work in Kenya and India is the most cost-effective opportunity we have found. We estimate that its work in Kenya is ~20x as cost-effective as cash transfers and in India is ~30x+ as cost-effective as cash transfers.

3. We rank providing funding to our two new top charities, Helen Keller International (HKI)’s VAS program and No Lean Season, next.

   We estimate that HKI could use $7.2 million over three years to support VAS campaigns in countries with high child mortality rates that have recently missed campaigns due to lack of funds. HKI’s cost-effectiveness is at the high end of the range for top charities (~9x cash transfers). We believe HKI could absorb more than $7.2 million in additional funding for VAS effectively but that this $7.2 million gap is likely more cost-effective than HKI’s average cost-effectiveness. Also, because HKI is a new top charity of ours, we expect this first part of its gap to have significant learning benefits for us: by giving this money, we’ll be better positioned to follow HKI’s work and review its monitoring, which we believe will make it more likely that we have a more accurate estimate of its impact in future years.

   We decided to recommend funding all of No Lean Season’s funding gap in Bangladesh for the next three years. While No Lean Season’s cost-effectiveness is at the lower end of our top charities (~5x cash transfers), we see additional reasons to prioritize this gap. We believe No Lean Season is the top charity where there is the strongest case to be made for “upside”; our cost-effectiveness analysis may not capture the potential impact of scaling a new program that could lead to greater visibility and funding for a novel type of program.

4. We think the next highest priority funding to provide is $10.4 million to Deworm the World. This funding would support a new program in Pakistan and provide reserve funding for programs supported with restricted funds. We estimate that the program in Pakistan will be roughly ~7x as cost-effective as cash transfers, though this estimate is very sensitive to estimates of worm burdens in the locations where Deworm the World plans to work.

   The reserve funding is intended to make it unlikely that the India program, which we believe is very highly cost-effective, will be interrupted—Deworm the World relies on restricted funding for this program and there is some chance that this funding will not be available in the future. It may use this GiveWell-directed funding for other opportunities if it is not needed to backstop restricted funding in India; we expect that it will have unfunded opportunities remaining in the next few years, particularly in Nigeria.

5. The last funding gap on our list of recommendations for Good Ventures is $23.6 million to Malaria Consortium for its work on SMC. When choosing which gap to recommend for the remainder of Good Ventures’ $75 million, we focused on the remaining funding needs for Malaria Consortium’s SMC program, AMF, and SCI, which we believe to have the next highest-value gaps. Our cost-effectiveness model indicates that SCI is the most cost-effective of these three organizations (~10x cash transfers, compared with ~6-7x cash transfers for AMF and Malaria Consortium), but when the difference in modelled cost-effectiveness between two charities is relatively small, we also put significant weight on qualitative factors. We believe that AMF and Malaria Consortium are stronger on some qualitative factors, particularly the likelihood that we will be able to learn about the programs’ performance through the monitoring they conduct. Between AMF and Malaria Consortium, we have prioritized Malaria Consortium’s funding gap primarily due to the qualitative considerations discussed above around the proportion of the global funding need that is filled. After following Malaria Consortium for a second year, we believe that Malaria Consortium and AMF are comparable on other major qualitative factors, such as quality of ongoing monitoring and likelihood of detecting future problems.

   The total amount we are recommending for Malaria Consortium’s SMC program represents a rough compromise between providing a high level of funding to a program that we prefer to the next funding gap on the list and not wanting to make too large of a bet on an organization that we have less experience with than some other top charities.

Prioritization for non-Good Ventures donors:

1. Our current recommendation for donors is to give to GiveWell for making grants to top charities at our discretion. Our goal is for SCI to receive $9 million, in addition to the $2.5 million incentive grant that we are recommending to Good Ventures, and AMF to receive the remainder of expected GiveWell-directed funding because AMF and SCI represent the next highest-value funding opportunities we see. Giving us funding to grant at our discretion allows GiveWell to better target this allocation, and to adapt if we learn new information about pressing, high-value funding needs at our top charities.

2. For donors who prefer to give directly to charities, we recommend giving 70 percent to AMF and 30 percent to SCI. These percentages are our best guess of what will achieve our target allocation given our projections of total donations driven by our recommendations.

   This allocation comes from a belief that, at these margins, it is difficult to distinguish between the quality of AMF and SCI’s funding gaps. SCI has better modeled cost-effectiveness, while AMF appears to be better on several qualitative factors, including monitoring of program performance. We have roughly targeted a two-to-one ratio between the two.

Details on new top charities

Helen Keller International (HKI) for work on vitamin A supplementation

Our full review of HKI’s work on vitamin A supplementation is here.

Overview

HKI (http://www.hki.org/) is a large organization with multiple programs focused on reducing malnutrition and averting blindness and poor vision. Our review focuses on HKI’s work on vitamin A supplementation (VAS) and our recommendation is specific to its VAS program. HKI provides technical assistance, engages in advocacy, and contributes funding to government-run VAS programs.

There is strong evidence from many randomized controlled trials (RCTs) conducted in the 1980s and 1990s that VAS can substantially reduce child mortality, but weaker evidence on how effective VAS is in the places HKI would work with additional funding in the next few years. In particular, there is little available information on current rates of vitamin A deficiency in areas where HKI works. We have adjusted our cost-effectiveness analysis for our best guess of how much less effective VAS is today (~25 percent as effective as in the trials in the 1980s and 1990s); the intervention remains cost-effective with that adjustment.

We feel that the monitoring data that we have seen from HKI’s programs gives us limited information on HKI’s past performance, but demonstrates the types of data HKI is able to collect on program performance. We have requested that HKI collect this monitoring data of all programs funded with GiveWell-directed funds.

Overall, we have not yet investigated HKI at the same level of depth as some of our other top charities, which we have recommended for several years. We have reviewed documents from HKI, had a number of conversations with their staff, and spent three days meeting with HKI and observing a VAS campaign in Guinea. We have remaining questions about HKI’s work that we will seek more information on in the future, but overall we believe this program is, like our other top charities, an excellent giving opportunity.

Funding gap

We believe that HKI’s VAS work is highly likely to be constrained by funding next year. HKI has provided details of VAS programs that it could support with additional funding of up to about $41.4 million in 2018-2020. HKI appears to have limited prospects for funding these programs from other sources.

Our understanding is that with additional funds, HKI would cause additional rounds of VAS to occur in some countries, while in other countries, HKI primarily aims to increase coverage rates in rounds of VAS that would take place regardless of its involvement. We have asked HKI to prioritize use of GiveWell-directed funding in countries where it expects to cause additional rounds of VAS to occur. HKI’s funding gap for countries that have recently missed VAS campaigns due to lack of funds is $7.2 million.

HKI’s VAS work was supported by the Canadian government in the past. That funding ended in 2016 and has not been renewed. Over the past year, several VAS campaigns have been skipped in countries HKI previously supported.

Evidence Action’s No Lean Season program

Our full review of No Lean Season is here.

Overview

No Lean Season (https://www.evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the season of income and food insecurity (‘lean season’) between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on a household member stating their intention to migrate to urban or other rural locations to seek short-term employment.

Several randomized controlled trials (RCTs) of subsidies to increase migration provide moderately strong evidence that such an intervention increases household income and consumption during the lean season. An additional RCT is ongoing. We estimate that No Lean Season is roughly five times as cost-effective as cash transfers (see our cost-effectiveness analysis).

Evidence Action has shared some details of its plans for monitoring No Lean Season in the future, but, as many of these plans have not been fully implemented, we have seen limited results. Therefore, there is some uncertainty as to whether No Lean Season will produce the data required to give us confidence that loans are appropriately targeted and reach their intended recipients in full; that recipients are not pressured into accepting loans; and that participants successfully migrate, find work, and are not exposed to major physical and other risks while migrating.

Funding gap

We expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Evidence Action is seeking funding beyond this level to allow it to build reserves for No Lean Season.

Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net (LLIN) distributions for protection against malaria in developing countries. AMF has conducted post-distribution surveys of all completed distributions to determine whether LLINs have reached their intended destinations and how long they remain in good condition. AMF’s post-distribution surveys have generally found positive results (with some exceptions); we believe they have some methodological limitations.

We estimate that AMF’s program is roughly six times as cost-effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

Prior to this year, we had seen results from AMF’s “post-distribution check ups” (PDCUs) from two countries, Malawi and the Democratic Republic of the Congo, and had significant uncertainties about the methodology used in each location. We have now also seen results from Ghana. We have more confidence in our understanding of AMF’s PDCUs than we did previously, though this work is ongoing. In particular, we commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings (see links). Further discussion of the strengths and weaknesses of PDCUs here.

In 2017, AMF signed relatively few new agreements to fund LLIN distributions and, as a result, has a balance of $58 million in uncommitted funds, or $35 million if distributions where AMF believes agreements are imminent are counted as committed. Our understanding is that many of AMF’s conversations with countries could not progress until decisions were made about how much Global Fund funding each country would allocate to LLIN distributions (as opposed to other malaria control efforts). This decision-making process extended into late 2017. Global Fund funding is allocated on three-year cycles and we do not expect this to continue to be a bottleneck for AMF in 2018.

Funding gap

We believe that AMF is very likely to be constrained by lack of funding. There is high uncertainty in the maximum amount of funding that AMF could use productively, though we expect the maximum to be much greater than what AMF is likely to receive. To fund all of the distributions that it is currently in detailed discussions about, AMF would need $50 million more than we project it will receive. The total funding gap for LLINs for 2018-2020 appears to be hundreds of millions of dollars.

With additional funding, AMF’s top priorities would be to fund a portion of the next round of distributions, in 2018-2020, in each of the countries in which it has recently funded distributions.

END Fund (for work on deworming)

Our full review of the END Fund’s work on deworming is here.

Background

The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming.

Slightly more than half of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list.

We have seen limited monitoring results on the number of children reached in END Fund-supported programs. In 2016, the END Fund began requiring that surveys be conducted to determine whether its programs have reached a large proportion of children targeted; we have seen coverage surveys for (a non-random sample of) 35 percent of its 2016 deworming grant portfolio. These studies leave us with some remaining questions about the program’s impact.

Important changes in the last 12 months

We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas that the END Fund works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. We also saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible.

Funding gap

We believe the END Fund could substantially increase its deworming grantmaking with additional funds. We roughly estimate that there is gap of $18 million between the amount of funding the END Fund will have available for grants for deworming and the amount of funding it would need to make all of the potential grants it has identified. Sources of major uncertainty in this estimate include whether the END Fund will encounter non-funding bottlenecks in some of its identified and early-stage opportunities, the amount of funding it will receive from other sources, the proportion of funding it will allocate to deworming, and costs other than grants.

Evidence Action’s Deworm the World Initiative

Our full review of Deworm the World is here.

Background

Evidence Action’s Deworm the World (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates deworming programs. Its main countries of operation are India, Kenya, and Nigeria, and it is considering expanding to Pakistan.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World overall is the most cost-effective charity we have found. We estimate that it is ~12 times as cost-effective as cash transfers, but note that, due to differences in worm burdens and costs across countries, there is significant variation in cost-effectiveness across the countries in which it works. We estimate that its work to date in India has been more than 30 times as cost-effective as cash transfers, while its planned work in Nigeria is around three times as cost-effective as cash transfers (though this estimate is based on low-quality information).

Important changes in the last 12 months

We estimate that Deworm the World could absorb considerably more funding this year than we estimated last year, due to opportunities it has identified to expand its geographic reach. (More in the next section.)

The quality of the monitoring that we have seen from Deworm the World has remained high. To date, we have seen limited monitoring from Nigeria, which is a new addition to Deworm the World’s portfolio and is expected to become a major portion of its work in the future. This is of minor concern given the strong monitoring track record elsewhere and how new the program is in Nigeria.

Funding gap

We believe that Deworm the World is very likely to be constrained by funding. We expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive over the next three years. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities.

With additional funding, Deworm the World would sustain its current work in Kenya and India, and would seek to expand its work in Nigeria and India to additional states and support the government in Pakistan to initiate a deworming program.

GiveDirectly

Our full review of GiveDirectly is here.

Background

GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82 percent overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people.

We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. We believe that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected. It continues to experiment heavily, with the aim of improving how its own cash transfer programs are run as well as those of governments. It has recently started work on a universal basic income trial and has started partnering with major funders on evaluations of cash transfers in new geographies with the aim of influencing the broader international aid sector to use its funding more cost-effectively.

We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost-effective than cash.

Important changes in the last 12 months

We had previously expressed reservations about GiveDirectly’s targeting strategy: that by excluding the least poor households in each village, the program might lead to negative reactions by non-recipients, increase costs per household reached, and exclude households that were still quite poor. In 2017, GiveDirectly largely switched to a “saturation” approach of making transfers to all households in selected villages. It will continue to use a targeted approach in Rwanda, where government regulations require such an approach, but the saturation approach will be used in Kenya and Uganda.

In 2016, GiveDirectly built up its operations in Uganda and Kenya with the anticipation of revenue growth in 2017. Revenue growth has been slower than expected and GiveDirectly had to lay off some staff as a result.

GiveDirectly launched its universal basic income project this month.

In 2015, Good Ventures made a grant of $25 million to GiveDirectly on GiveWell’s recommendation. GiveDirectly’s goals for the grant were to expand its ability to raise funds from donors not influenced by GiveWell’s recommendation and to collaborate with large aid institutions or governments to address their questions about cash transfers. We expect to write more about the performance of the grant in the future, but, in short, our impression is that fundraising has progressed slower than expected and collaborative projects have progressed more quickly than expected.

Funding gap

We believe that GiveDirectly is highly likely to be constrained by funding next year. It expects to use additional funding primarily for standard cash transfers and for additional collaborative projects. For collaborative projects, GiveDirectly’s potential partners require it to contribute funding, which the partner matches (at a one-to-one ratio, minimum). These projects would largely be in countries GiveDirectly has not worked in before and many are at an early stage of discussion. We estimate that GiveDirectly could use more than $200 million in additional funding in 2018-2019.

Malaria Consortium (for work on seasonal malaria chemoprevention)

Our full review of Malaria Consortium’s seasonal malaria chemoprevention program is here.

Background

Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria.

There is strong evidence that SMC substantially reduces cases of malaria. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately statistically powered to find an impact on mortality.

Malaria Consortium and its partners have conducted studies in all of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, though past surveys have been conducted after four rounds of SMC (SMC is given in a maximum of four treatment courses at monthly intervals) and may be subject to error due to the inaccurate recall or recordkeeping. Starting in 2017, Malaria Consortium is conducting coverage surveys after each round of SMC, to reduce recall error.

Important changes in the last 12 months

We have increased our confidence in Malaria Consortium’s monitoring, though we have not yet seen all of the research that Malaria Consortium expected to share in 2017 (in particular, tracking of malaria cases and deaths over time in areas where Malaria Consortium works). Coverage survey results from 2016 were generally positive, with a couple of outliers. The change from conducting coverage surveys after four treatment cycles to conducting them after each cycle will increase our confidence in the results.

Last year, we had only a rough estimate of how much additional funding Malaria Consortium could use productively. We have significantly improved our understanding of its room for more funding this year.

Funding gap

We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. It appears that there is a large remaining global need for additional funding for SMC programs and that Malaria Consortium is well-positioned to fill these gaps, if it has sufficient funding to do so.

Malaria Consortium estimates that it could spend $28-30 million per year on SMC in each of the next three years and that this level of funding would largely fill the global funding gap for SMC, with the exception of Nigeria, where the scale of the gap would be beyond Malaria Consortium’s operational capacity in the short term.

It appears to have limited prospects for major funding from other sources. The major grant for Malaria Consortium’s work on SMC previously, from Unitaid, is ending and Malaria Consortium told us that it will not be renewed.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify partner countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies to determine whether its programs have reached a large proportion of children targeted. These studies cover (a non-random sample of) about 40 percent of treatments SCI reports having delivered over the past few years. The studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is less cost-effective than Deworm the World and more cost-effective than Sightsavers and the END Fund. Given the uncertainty in our cost-effectiveness model, we are hesitant to say that SCI is more cost-effective than AMF and Malaria Consortium, though taken literally, SCI is 1.5 times as cost-effective as AMF and Malaria Consortium (~10x cash transfers vs. ~6-7x cash transfers).

Important changes in the last 12 months

We continued to follow SCI’s progress in 2017 and there have not been many major changes to its work. As in the past, SCI shared monitoring of deworming coverage levels for a portion of its programs with us; there continue to be several SCI-supported countries for which we have not seen monitoring results. In the past, we have noted that we had low confidence in the accuracy of the financial information that SCI provided and that SCI made significant improvements to its financial systems in 2016; our remaining concerns about SCI’s financial management and reporting are fairly minor.

In 2017, SCI allocated nearly all available funding to programs in its 2017-2018 budget year. This was a large increase in spending over the previous budget year ($9.6 million in 2016-2017 compared with $22.5 million in 2017-2018), driven in large part by a large increase in GiveWell-directed funding ($3.7 million in 2015 compared with $16.6 million in 2016). We believe this decision was due in part to a miscommunication with GiveWell—in a conversation with SCI in early 2017, we recommended that they treat the funds like a multi-year grant because of the risk of large fluctuations in GiveWell-directed funding, but we did not emphasize this point. SCI told us that it plans to allocate future funding over multiple years, noting that its funding allocation decisions in 2016-2017 were due to the desire to avoid allowing drugs to expire as well as a misunderstanding with GiveWell about how the funding was intended to be used.

Funding gap

We estimate that SCI could productively use about $30 million more than it expects to receive to deliver treatments to school-aged children over the next three years. It could use almost three times this amount if it were to follow World Health Organization guidelines, which include treating many adults; we are not recommending funding to treat adults because we haven’t seen sufficient evidence on the impact of treating adults.

The primary use of this funding, and SCI’s top priority, would be to sustain and expand work in current countries of operation. A smaller portion would be used to expand to up to four additional countries.

Sightsavers (for work on deworming)

Our full review of Sightsavers is here.

Background

Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and – more specifically – advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections.

Sightsavers has shared surveys for some of its past NTD programs that measure whether these programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact. We have seen very limited results from Sightsavers’ deworming programs specifically. For GiveWell-supported programs, Sightsavers has told us it will conduct coverage surveys for each round of deworming; we have reviewed one of those surveys to date.

Important changes in the last 12 months

In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results.

We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years.

Funding gap

We believe that Sightsavers’ deworming work is likely to be constrained by funding next year. Sightsavers has provided details of deworming programs that it could fund with additional funding of up to about $6.4 million in 2018 and 2019. Sightsavers appears to have limited prospects for funding these programs from other sources. We believe it is likely that Sightsavers could absorb funding beyond this amount to extend programs to 2020 and/or seek out additional opportunities to fund deworming programs.

Of the $6.4 million, $2.8 million would be used to add deworming to existing NTD programs and $3.7 million would be used to fund NTD programs that would treat several NTDs in addition to schistosomiasis and STH. We will request that Sightsavers prioritize the first set of opportunities, because we believe they will likely be more cost-effective.

Standout charities

In addition to our top charities, we recognize standout charities—organizations that support programs that may be extremely cost-effective and are evidence-backed but for which we have less confidence in their impact than we do for our top charities. We have reviewed their work and feel these groups stand out from the vast majority of organizations we have considered in terms of the evidence base for the program they support, their transparency, and their potential cost-effectiveness. These organizations offer additional giving options for donors who feel highly aligned with their work.

We’ve added one organization to the list this year: Evidence Action’s Dispensers for Safe Water.

We don’t follow standout organizations as closely as we do our top charities. We generally have one or two calls per year with representatives from each group and publish notes on our conversations. We provide brief updates on these charities below.

New addition to the standout list:

• Evidence Action’s Dispensers for Safe Water. The Dispensers for Safe Water program provides chlorine dispensers for decontamination of drinking water to prevent diarrhea and associated deaths of young children. We believe that there is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death. Our rough cost-effectiveness analysis of Dispensers for Safe Water suggests that the program is in a similar range of cost-effectiveness as unconditional cash transfer programs. Our review of Dispensers for Safe Water is here.

Organizations that have conducted randomized controlled trials of their programs:

• Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its child survival media campaign in Burkina Faso and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15 percent or more) and data from health facilities finding an increase in facility visits. (The results have not yet been published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. DMI is conducting an RCT of its family planning radio campaign in Burkina Faso and it is planning work on early child development in Burkina Faso and child survival in Mozambique. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI is here and notes from our most recent conversation with DMI are here.
• Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a RCT of its program and measured a 27 percent reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of GiveDirectly. It is conducting a second RCT of its program and results are expected in 2020. Living Goods recently expanded the number of family planning products it offers and is interested in expanding to a third country. Living Goods is scaling up its program and could scale up more quickly with additional funding. Our review of Living Goods is here and notes from our most recent conversation with Living Goods are here.

Organizations working on micronutrient fortification:

We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence.

• Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost-effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. FFI’s recent work includes advocating for legislation to mandate that rice imported to West Africa is fortified with vitamins and minerals. Our full review is here and notes from our most recent conversation are here.
• Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. GAIN has focused its recent USI work on Tanzania, Mozambique, Ethiopia, and Kenya, which it targeted based on relatively low levels of coverage of iodized salt and strong relationships with stakeholders. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN is here and notes from our most recent conversation are here.
• Iodine Global Network (IGN). Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-directed funding has made up a large part of its funding in recent years. It expects to have data from before and after its recent work in Madagascar, Lebanon, and possibly Israel by the end of 2018; this data may provide additional evidence of IGN’s impact. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN is here and notes from our most recent conversation here.
• Project Healthy Children (PHC)/Sanku. PHC/Sanku aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs and by enabling fortification among small-scale millers. PHC is scaling up its Sanku project, which equips small millers with a machine that enables them to fortify their flour with micronutrients; we have not done as much formal analysis of Sanku as of PHC’s core work on advocacy and technical assistance to countries to implement fortification. PHC/Sanku expects to be constrained by funding in the future. Our review of PHC/Sanku is here and notes from our more recent conversation are here.

Our research process in 2017

We plan to detail the work we completed this year in a future post as part of our annual review process. A major focus of 2017 was improving our recommendations in future years, in particular through our work on GiveWell Incubation Grants and prioritizing promising programs for further investigation.

Below, we highlight the key research that led to our current charity recommendations. This page describes our overall process.

• Following existing top charities. We followed the progress and plans of each of our 2016 top charities. We had several conversations by phone with each organization, met in person at least once with each top charity (including a three-day visit to Rwanda and the Democratic Republic of the Congo with the END Fund), and reviewed documents they shared with us.
• Identifying new top charities.
  • No Lean Season. We had recommended a series of Incubation Grants to No Lean Season beginning in 2014 and have followed its progress since then. This year, due to the scale at which No Lean Season was operating and the track record it had established, we decided that the No Lean Season program was at a stage of development where we could evaluate it as a potential top charity. In addition to extensive communications with No Lean Season staff over the phone and reviewing documents they shared with us, GiveWell staff spent five days visiting the program in Bangladesh.
  • Helen Keller International’s vitamin A supplementation program. Earlier this year, Research Analyst Chelsea Tabart began reaching out to organizations that might be a fit for our criteria, but with which we had limited or no previous contact with. As a result of that process, we reconnected with Helen Keller International (which we first considered as a potential top charity in 2007) and began to consider its vitamin A supplementation program as a potential top charity. In addition to extensive communications with HKI staff over the phone and reviewing documents they shared with us, GiveWell staff spent three days meeting with HKI staff in Guinea and observing a vitamin A supplementation program.
• Completing intervention reports on obstetric fistula surgery and measles vaccination campaigns; completing interim intervention reports on SMS reminders for vaccination, Sayana® Press (an injectable contraceptive), oral rehydration solution, and antiretroviral therapy for HIV/AIDS; and expanding our interim intervention report on seasonal malaria chemoprevention to a full intervention report.
• Staying up to date on the research for malaria nets, cash transfers, and deworming. We did not find major new research on cash transfers, nets, or deworming that affected our recommendation of GiveDirectly, AMF, or the organizations we recommend for their work on deworming. David Roodman published an in-depth review (parts 1 and 2) of the deworming studies that form the primary basis of our views on the impact of deworming (though much of this work was completed in 2016 and informed our top charity recommendations last year).
• Making extensive updates to our cost-effectiveness model and publishing several updates to the model over the course of the year. We instituted a process to track and report publicly on updates to the model to reduce the possibility of errors and make our process more transparent. This year, staff members have also provided substantially more detail in our cost-effectiveness file about why they have chosen particular inputs.

Giving to GiveWell vs. top charities

GiveWell is currently in a stable financial position. We project that our revenue and our expenses will be approximately equal in the future. However, this projection forecasts some growth in the level of operating support we receive.

In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations. We retain our “excess assets policy” to ensure that if we fundraise for our own operations beyond a certain level, we will grant the excess to our recommended charities.

We cap the amount of operating support we ask Good Ventures to provide to GiveWell at 20 percent, for reasons described here. We thus ask that donors who use GiveWell’s research consider the following:

• If you have supported GiveWell’s operations in the past, we ask that you maintain your support. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and to minimize staff time spent on fundraising for our operating expenses.
• If you have not supported GiveWell’s operations in the past, we ask that you designate 10 percent of your donation to help fund GiveWell’s operations. This can be done by selecting the option to “Add 10% to help fund GiveWell’s operations” on our credit card donation form or letting us know how you would like to designate your funding when giving another way.

We’re happy to answer questions in the comments below. Please also feel free to reach out directly with any questions.

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

This year, we added two new top charities, Evidence Action's No Lean Season program and Helen Keller International's vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action's Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Read More

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

This year, we added two new top charities, Evidence Action's No Lean Season program and Helen Keller International's vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action's Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Read More

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

This year, we added two new top charities, Evidence Action’s No Lean Season program and Helen Keller International’s vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action’s Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Our top charities and recommendations for donors, in brief

Top charities

We now have nine top charities. They are:

• Against Malaria Foundation
• Schistosomiasis Control Initiative
• Malaria Consortium’s seasonal malaria chemoprevention (SMC) program
• Evidence Action’s Deworm the World Initiative
• Helen Keller International (HKI)’s vitamin A supplementation (VAS) program
• Sightsavers’ deworming program
• END Fund’s deworming program
• Evidence Action’s No Lean Season program
• GiveDirectly

Standout charities

We also provide a list of standout charities. We believe they are implementing programs that are evidence-backed and may be extremely cost-effective. However, we do not feel as confident in the impact of these organizations as we do in our top charities.

• Development Media International
• Evidence Action’s Dispensers for Safe Water program
• Food Fortification Initiative
• Global Alliance for Improved Nutrition’s Universal Salt Iodization program
• Iodine Global Network
• Living Goods
• Project Healthy Children

Conference call to discuss recommendations

We are planning to hold a conference call at 1:30pm ET/10:30am PT on Thursday, November 30 to discuss our charity recommendations and answer your questions.

If you’d like to join the call, please register using this online form. If you can’t make this date, but would be interested in joining another call at a later date, please indicate this on the registration form.

Additional details and explanation

Below, we provide:

• An explanation of changes to our recommended charity list and of major changes to our review process in the past year that are not specific to any one organization. More
• A discussion of our approach to determining how much funding charities can use effectively (“room for more funding”) and our ranking of charities’ funding gaps. More
• Reasoning behind how we have ranked charities’ funding gaps. More
• Details about each of our new top charities, including an overview of what we know about their work and our understanding of their funding needs. More
• Details about each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year, and our understanding of their funding needs. More
• A brief overview of each of our standout charities. More
• The process we followed that led to these recommendations. More
• An update on giving to support GiveWell’s operations versus giving to our top charities. More

Major changes in the last 12 months

Major changes to our recommended charities list and review process over the past year include:

• Overall, we believe our top charities are able to absorb more funding than they could in previous years. This is a result both of recent additions to the top charities list with large funding gaps (particularly Malaria Consortium) as well as expansion by top charities that have been on the list for a longer time (particularly Deworm the World and AMF).

  We expect overall “room for more funding” to continue to expand as we gain more confidence in recently-added top charities and continue to add new top charities, particularly through GiveWell Incubation Grants, our program to grow the pipeline of potential future top charities and improve our understanding of our current top charities.

• We added two new programs to our list of top charities: vitamin A supplementation (VAS) and seasonal migration subsidies. We have not previously recommended charities that work on these programs.

  We had considered VAS a priority program for a number of years but had not found an organization that was able to answer our key questions. While we have some remaining questions, we can now make a strong case for supporting HKI’s work on VAS.

  We initially supported No Lean Season through GiveWell’s Incubation Grants program. No Lean Season is the first organization we have added to our top charity list through our Incubation Grants program.

• Last year, the charities we recommended on the margin were estimated to be about three times as cost-effective as unconditional cash transfers, the program implemented by top charity GiveDirectly. This year, we believe that the charities we are recommending on the margin are about six times as cost-effective as cash transfers. For the most part, this change was due to (a) a series of small adjustments to our cost-effectiveness model and (b) changes in which individuals contribute to the model and the values entered into the model by these and other contributors.

  We now feel fairly confident that there will be large amounts of room for more funding in this range. As more time has passed without identifying opportunities that are considerably more cost-effective than this, we have become more pessimistic about finding such opportunities. Our current best guess is that, if they exist, they will be in the area of policy advocacy in developing countries, on issues like lead regulation and tobacco taxation. We intend to do further research in those areas.

• We made a significant change to our cost-effectiveness analysis to more formally incorporate adjustments for the way in which our top charities’ funding affects funding from other sources by (a) attracting more resources to the programs they work on (e.g., governments contributing staff time to support implementation of the programs) or (b) displacing resources that would have otherwise supported the programs. We will be writing more about this in a future post.

• We continued to analyze the complex evidence base for deworming (treating intestinal parasites), the program implemented by four out of our nine top charities.

  At the end of 2016, David Roodman, a Senior Advisor to GiveWell, conducted a detailed review of the core evidence underlying our deworming recommendation (blog posts here and here).

  This year, we saw new follow-up results on the main study that leads us to recommend deworming, which continued to show similar long-term impacts of deworming on adult earnings as were estimated previously.

  Further investigation and updates based on new data led us to believe that two deworming studies (Croke 2014 and Bleakley 2007) no longer provide substantial support for the theory that deworming has long-term impacts. We plan to write more about this in the future. All together, this work led us to the same conclusion about deworming: that it is a reasonable bet to take based on its strong cost-effectiveness (which incorporates our uncertainty about the impact).

Room for more funding analysis

Types of funding gaps

In the last two years, we used a framework of “capacity funding” and “execution levels” to compare funding gaps (unfilled funding needs) across charities. This framework was intended to capture whether funding would enable a charity to expand or grow in important ways and how likely it was, in our estimation, that each top charity would be constrained by funding in the next year.

We developed this approach in response to a situation where we expected to direct more funding to several of our top charities than they would be able to use (commit or spend) in that year. We used capacity funding to describe opportunities to increase the amount of funding a charity might be able to absorb in the future (by, say, investing in expanding to a new location) and execution levels to describe the likelihood, down to the 5 percent level, that a charity would be able to make use of additional funding before encountering non-funding bottlenecks to their work.

This year, because we have added new top charities and most of our other top charities have more room for more funding than in previous years, we expect that the funding we will direct to each organization will not reach the level where they will encounter significant non-funding bottlenecks. As a result, we have moved away from describing capacity funding and execution levels.

Ranking funding gaps

The first million dollars to a charity can have a very different impact from the 20th million dollars. Accordingly, we have created a ranking of individual funding gaps that accounts for our best guess of the impact of additional funds at each level.

The below table lays out our ranking of funding gaps, up to $75.7 million in total funding. We expect Good Ventures to give $75 million to GiveWell’s top charities this year, so this table is our recommendation to Good Ventures, plus the allocation of funding that GiveWell holds to allocate at its discretion (currently $0.7 million). We then discuss our recommendation for all other donors.

The Open Philanthropy Project, which was incubated at GiveWell but is now a separate organization, plans to write more soon about the reasons for Good Ventures increasing its support of GiveWell top charities from $50 million last year to $75 million this year. In short, the amount was based on discussions about how to allocate funding across time and across cause areas. It was not set based on the total size of top charities’ funding gaps or the projection of what others would give.

Charity Description Amount (millions) All top charities Incentive grants: $2.5 million per charity 22.5 All standout charities Standout grants: $100,000 per charity 0.7 Deworm the World Funding gaps in India and Kenya over the next three years (including central costs) 3.0 Helen Keller International Funding gaps over three years in Burkina Faso, Mali, and Guinea—countries that have missed recent vitamin A campaigns due to lack of funding 4.7 No Lean Season Full funding gap over three years for implementing the program in Bangladesh 9.0 Deworm the World Three years of funding for a new program in Pakistan and reserves to protect against funding shortfalls in India 10.4 Malaria Consortium Part of the funding gap for SMC in Burkina Faso, Nigeria, and Chad over the next three years 25.4

In total, we are recommending that Good Ventures make the following grants:

• Malaria Consortium’s seasonal malaria chemoprevention program: $27.9 million
• Evidence Action’s Deworm the World Initiative: $15.2 million. We are also recommending that GiveWell’s Board of Directors grant the $0.7 million in discretionary funds that we currently hold from the third quarter (from donors who selected to give to “Grants to recommended charities at GiveWell’s discretion” on our donation form) to Deworm the World, bringing the total to $15.9 million.
• Evidence Action’s No Lean Season program: $11.5 million
• Helen Keller International’s vitamin A supplementation program: $7.2 million
• Schistosomiasis Control Initiative: $2.5 million
• Against Malaria Foundation: $2.5 million
• Sightsavers’ deworming program: $2.5 million
• END Fund’s deworming program: $2.5 million
• GiveDirectly: $2.5 million

Our recommendation to donors

For donors who are interested in directing funding to whichever recommended charity or charities GiveWell believes has the most pressing funding need at the time the funds are granted, we recommend giving to “Grants to recommended charities at GiveWell’s discretion.” These grants will respond to the greatest funding need we see; they may not match the recommended allocation outlined below.

For donors (other than Good Ventures) who are interested in donating directly to our top charities, we recommend splitting your donation as follows:

• 70 percent to the Against Malaria Foundation
• 30 percent to the Schistosomiasis Control Initiative

Why these recommendations?

Our recommendations to donors, including Good Ventures, are based on:

1. Overall cost-effectiveness of the charity. Our cost-effectiveness model is a key input into our decision-making process, and large differences in modeled cost-effectiveness impact our recommendations. We try not to put significant weight on relatively small differences in cost-effectiveness according to the model because many inputs are highly uncertain.

   Our model this year found relatively small differences between many top charities, with Deworm the World at ~12 times as cost-effective as cash transfers, four top charities in the ~6-10x cash transfers range, and three top charities in the ~3-5x cash transfers range. We consider differences between charities implementing the same intervention or interventions that have similar inputs and output in the model more meaningful (e.g., malaria nets and seasonal malaria chemoprevention) than differences between charities implementing quite different interventions.

   We have completed a sensitivity analysis of our cost-effectiveness analysis to get a better sense for which parameters are most sensitive. We are more hesitant to consider differences in the cost-effectiveness as meaningful when they rely on very sensitive inputs.

2. Cost-effectiveness of particular funding opportunities. Charities’ work can vary significantly in cost-effectiveness across locations due to different costs, disease burdens, uptake in the targeted population, or probability that other funders would step in in GiveWell’s absence. While not a part of our formal cost-effectiveness model, we ran supplementary analyses of cost-effectiveness for some locations for which our top charities were seeking additional funding and considered the output as part of our prioritization of funding gaps.
3. Qualitative factors not captured in our cost-effectiveness model. The main factors we focused on were:
   • Proportion of the global funding need for the program that is filled. We expect that funders will generally (but imperfectly) select the areas where cost-effectiveness is higher first, leaving the areas with higher costs, lower disease burden, lower cultural acceptance of the program, etc. for last. We believe we have captured some of the consequences of this in our cost-effectiveness analysis. For example, we use national level disease burden estimates for the countries in which each charity has worked and/or plans to work; charities working in higher burden countries are therefore modelled as more cost-effective. But we do not use sub-national estimates to distinguish the highest priority regions within a country; if charities are filling the lowest priority funding gaps within a county, they will likely be less cost-effective than our model suggests. This was an important consideration in comparing AMF and Malaria Consortium. We estimate that ~80 percent of the global funding need for nets (the program AMF implements) has been filled, and ~35 percent of the global funding need for seasonal malaria chemoprevention (the program Malaria Consortium implements).
   • Our level of knowledge about the organization. We have recommended AMF, Deworm the World, SCI, and GiveDirectly for many years. We know less about Malaria Consortium and No Lean Season and the least about HKI. We seek to be somewhat conservative about recommending large amounts of funding to organizations where there is a relatively high chance that additional research could lead us to believe the program was less cost-effective than we previously thought.
   • Ease of communication with the organization. It is important to us that we are able to learn over time about the charities we recommend, to enable us to improve our decisions. The ability to communicate effectively with an organization is a key factor in our ability to learn from the organization’s experiences.
   • Ongoing monitoring and likelihood of detecting future problems. Evaluating an organization’s monitoring processes and results is an important part of our charity reviews and for the most part is not captured in our cost-effectiveness analysis. As with ease of communication, we have more confidence in recommending funds to an organization if we believe that we will learn about how successful its work has been.

Summary of key considerations for top charities

The table below summarizes the key considerations for our nine top charities. More detail is provided below, as well as in the charity reviews.

Estimated cost-effectiveness (relative to cash transfers) Our level of knowledge about the organization Primary benefits of the intervention Ease of communication Ongoing monitoring and likelihood of detecting future problems Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation Other major considerations AMF ~6x High Deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars High proportion (~80%) of global gap for program is filled Malaria Consortium (SMC program) ~7x Moderate Under-5 deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars Relatively low proportion (~35%) of global gap for program is filled Helen Keller International (VAS program) ~9x Moderate Under-5 deaths averted Strong Moderate High: could absorb tens of millions of dollars Learning benefits Deworm the World ~12x High Possible increased income in adulthood Strong Strong Moderate: could absorb millions of dollars END Fund (deworming program) ~4x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars SCI ~10x High Possible increased income in adulthood Moderate Moderate High: could absorb tens of millions of dollars Sightsavers (deworming program) ~5x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars No Lean Season ~5x Moderate Immediate increase in consumption Strong Moderate Low: further funding would be used for different types of activities Potential upside GiveDirectly Baseline High Immediate increase in consumption and assets Strong Strong Very high: could absorb over 100 million dollars

Reasons for this funding gap ranking

Prioritization of funding that we have recommended to Good Ventures (we recommend Good Ventures fill the highest-priority funding needs first, to ensure these are funded):

1. We start by recommending that each top charity receive $2.5 million as an “incentive grant.” These grants are intended to be a major contribution to the charity’s work in recognition of the fact that they have met GiveWell’s criteria and have dedicated significant time to working with us to help us follow their progress and plans each year. We don’t want our top charity funding process to be winner-takes-all because we believe that charities would be less likely to want to participate in that case.
2. After incentive grants, we believe the next most valuable funding to provide is for Deworm the World’s work in Kenya and India over the next three years. Deworm the World’s work in Kenya and India is the most cost-effective opportunity we have found. We estimate that its work in Kenya is ~20x as cost-effective as cash transfers and in India is ~30x+ as cost-effective as cash transfers.

3. We rank providing funding to our two new top charities, Helen Keller International (HKI)’s VAS program and No Lean Season, next.

   We estimate that HKI could use $7.2 million over three years to support VAS campaigns in countries with high child mortality rates that have recently missed campaigns due to lack of funds. HKI’s cost-effectiveness is at the high end of the range for top charities (~9x cash transfers). We believe HKI could absorb more than $7.2 million in additional funding for VAS effectively but that this $7.2 million gap is likely more cost-effective than HKI’s average cost-effectiveness. Also, because HKI is a new top charity of ours, we expect this first part of its gap to have significant learning benefits for us: by giving this money, we’ll be better positioned to follow HKI’s work and review its monitoring, which we believe will make it more likely that we have a more accurate estimate of its impact in future years.

   We decided to recommend funding all of No Lean Season’s funding gap in Bangladesh for the next three years. While No Lean Season’s cost-effectiveness is at the lower end of our top charities (~5x cash transfers), we see additional reasons to prioritize this gap. We believe No Lean Season is the top charity where there is the strongest case to be made for “upside”; our cost-effectiveness analysis may not capture the potential impact of scaling a new program that could lead to greater visibility and funding for a novel type of program.

4. We think the next highest priority funding to provide is $10.4 million to Deworm the World. This funding would support a new program in Pakistan and provide reserve funding for programs supported with restricted funds. We estimate that the program in Pakistan will be roughly ~7x as cost-effective as cash transfers, though this estimate is very sensitive to estimates of worm burdens in the locations where Deworm the World plans to work.

   The reserve funding is intended to make it unlikely that the India program, which we believe is very highly cost-effective, will be interrupted—Deworm the World relies on restricted funding for this program and there is some chance that this funding will not be available in the future. It may use this GiveWell-directed funding for other opportunities if it is not needed to backstop restricted funding in India; we expect that it will have unfunded opportunities remaining in the next few years, particularly in Nigeria.

5. The last funding gap on our list of recommendations for Good Ventures is $23.6 million to Malaria Consortium for its work on SMC. When choosing which gap to recommend for the remainder of Good Ventures’ $75 million, we focused on the remaining funding needs for Malaria Consortium’s SMC program, AMF, and SCI, which we believe to have the next highest-value gaps. Our cost-effectiveness model indicates that SCI is the most cost-effective of these three organizations (~10x cash transfers, compared with ~6-7x cash transfers for AMF and Malaria Consortium), but when the difference in modelled cost-effectiveness between two charities is relatively small, we also put significant weight on qualitative factors. We believe that AMF and Malaria Consortium are stronger on some qualitative factors, particularly the likelihood that we will be able to learn about the programs’ performance through the monitoring they conduct. Between AMF and Malaria Consortium, we have prioritized Malaria Consortium’s funding gap primarily due to the qualitative considerations discussed above around the proportion of the global funding need that is filled. After following Malaria Consortium for a second year, we believe that Malaria Consortium and AMF are comparable on other major qualitative factors, such as quality of ongoing monitoring and likelihood of detecting future problems.

   The total amount we are recommending for Malaria Consortium’s SMC program represents a rough compromise between providing a high level of funding to a program that we prefer to the next funding gap on the list and not wanting to make too large of a bet on an organization that we have less experience with than some other top charities.

Prioritization for non-Good Ventures donors:

1. Our current recommendation for donors is to give to GiveWell for making grants to top charities at our discretion. Our goal is for SCI to receive $9 million, in addition to the $2.5 million incentive grant that we are recommending to Good Ventures, and AMF to receive the remainder of expected GiveWell-directed funding because AMF and SCI represent the next highest-value funding opportunities we see. Giving us funding to grant at our discretion allows GiveWell to better target this allocation, and to adapt if we learn new information about pressing, high-value funding needs at our top charities.

2. For donors who prefer to give directly to charities, we recommend giving 70 percent to AMF and 30 percent to SCI. These percentages are our best guess of what will achieve our target allocation given our projections of total donations driven by our recommendations.

   This allocation comes from a belief that, at these margins, it is difficult to distinguish between the quality of AMF and SCI’s funding gaps. SCI has better modeled cost-effectiveness, while AMF appears to be better on several qualitative factors, including monitoring of program performance. We have roughly targeted a two-to-one ratio between the two.

Details on new top charities

Helen Keller International (HKI) for work on vitamin A supplementation

Our full review of HKI’s work on vitamin A supplementation is here.

Overview

HKI (http://www.hki.org/) is a large organization with multiple programs focused on reducing malnutrition and averting blindness and poor vision. Our review focuses on HKI’s work on vitamin A supplementation (VAS) and our recommendation is specific to its VAS program. HKI provides technical assistance, engages in advocacy, and contributes funding to government-run VAS programs.

There is strong evidence from many randomized controlled trials (RCTs) conducted in the 1980s and 1990s that VAS can substantially reduce child mortality, but weaker evidence on how effective VAS is in the places HKI would work with additional funding in the next few years. In particular, there is little available information on current rates of vitamin A deficiency in areas where HKI works. We have adjusted our cost-effectiveness analysis for our best guess of how much less effective VAS is today (~25 percent as effective as in the trials in the 1980s and 1990s); the intervention remains cost-effective with that adjustment.

We feel that the monitoring data that we have seen from HKI’s programs gives us limited information on HKI’s past performance, but demonstrates the types of data HKI is able to collect on program performance. We have requested that HKI collect this monitoring data of all programs funded with GiveWell-directed funds.

Overall, we have not yet investigated HKI at the same level of depth as some of our other top charities, which we have recommended for several years. We have reviewed documents from HKI, had a number of conversations with their staff, and spent three days meeting with HKI and observing a VAS campaign in Guinea. We have remaining questions about HKI’s work that we will seek more information on in the future, but overall we believe this program is, like our other top charities, an excellent giving opportunity.

Funding gap

We believe that HKI’s VAS work is highly likely to be constrained by funding next year. HKI has provided details of VAS programs that it could support with additional funding of up to about $41.4 million in 2018-2020. HKI appears to have limited prospects for funding these programs from other sources.

Our understanding is that with additional funds, HKI would cause additional rounds of VAS to occur in some countries, while in other countries, HKI primarily aims to increase coverage rates in rounds of VAS that would take place regardless of its involvement. We have asked HKI to prioritize use of GiveWell-directed funding in countries where it expects to cause additional rounds of VAS to occur. HKI’s funding gap for countries that have recently missed VAS campaigns due to lack of funds is $7.2 million.

HKI’s VAS work was supported by the Canadian government in the past. That funding ended in 2016 and has not been renewed. Over the past year, several VAS campaigns have been skipped in countries HKI previously supported.

Evidence Action’s No Lean Season program

Our full review of No Lean Season is here.

Overview

No Lean Season (https://www.evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the season of income and food insecurity (‘lean season’) between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on a household member stating their intention to migrate to urban or other rural locations to seek short-term employment.

Several randomized controlled trials (RCTs) of subsidies to increase migration provide moderately strong evidence that such an intervention increases household income and consumption during the lean season. An additional RCT is ongoing. We estimate that No Lean Season is roughly five times as cost-effective as cash transfers (see our cost-effectiveness analysis).

Evidence Action has shared some details of its plans for monitoring No Lean Season in the future, but, as many of these plans have not been fully implemented, we have seen limited results. Therefore, there is some uncertainty as to whether No Lean Season will produce the data required to give us confidence that loans are appropriately targeted and reach their intended recipients in full; that recipients are not pressured into accepting loans; and that participants successfully migrate, find work, and are not exposed to major physical and other risks while migrating.

Funding gap

We expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Evidence Action is seeking funding beyond this level to allow it to build reserves for No Lean Season.

Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net (LLIN) distributions for protection against malaria in developing countries. AMF has conducted post-distribution surveys of all completed distributions to determine whether LLINs have reached their intended destinations and how long they remain in good condition. AMF’s post-distribution surveys have generally found positive results (with some exceptions); we believe they have some methodological limitations.

We estimate that AMF’s program is roughly six times as cost-effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

Prior to this year, we had seen results from AMF’s “post-distribution check ups” (PDCUs) from two countries, Malawi and the Democratic Republic of the Congo, and had significant uncertainties about the methodology used in each location. We have now also seen results from Ghana. We have more confidence in our understanding of AMF’s PDCUs than we did previously, though this work is ongoing. In particular, we commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings (see links). Further discussion of the strengths and weaknesses of PDCUs here.

In 2017, AMF signed relatively few new agreements to fund LLIN distributions and, as a result, has a balance of $58 million in uncommitted funds, or $35 million if distributions where AMF believes agreements are imminent are counted as committed. Our understanding is that many of AMF’s conversations with countries could not progress until decisions were made about how much Global Fund funding each country would allocate to LLIN distributions (as opposed to other malaria control efforts). This decision-making process extended into late 2017. Global Fund funding is allocated on three-year cycles and we do not expect this to continue to be a bottleneck for AMF in 2018.

Funding gap

We believe that AMF is very likely to be constrained by lack of funding. There is high uncertainty in the maximum amount of funding that AMF could use productively, though we expect the maximum to be much greater than what AMF is likely to receive. To fund all of the distributions that it is currently in detailed discussions about, AMF would need $50 million more than we project it will receive. The total funding gap for LLINs for 2018-2020 appears to be hundreds of millions of dollars.

With additional funding, AMF’s top priorities would be to fund a portion of the next round of distributions, in 2018-2020, in each of the countries in which it has recently funded distributions.

END Fund (for work on deworming)

Our full review of the END Fund’s work on deworming is here.

Background

The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming.

Slightly more than half of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list.

We have seen limited monitoring results on the number of children reached in END Fund-supported programs. In 2016, the END Fund began requiring that surveys be conducted to determine whether its programs have reached a large proportion of children targeted; we have seen coverage surveys for (a non-random sample of) 35 percent of its 2016 deworming grant portfolio. These studies leave us with some remaining questions about the program’s impact.

Important changes in the last 12 months

We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas that the END Fund works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. We also saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible.

Funding gap

We believe the END Fund could substantially increase its deworming grantmaking with additional funds. We roughly estimate that there is gap of $18 million between the amount of funding the END Fund will have available for grants for deworming and the amount of funding it would need to make all of the potential grants it has identified. Sources of major uncertainty in this estimate include whether the END Fund will encounter non-funding bottlenecks in some of its identified and early-stage opportunities, the amount of funding it will receive from other sources, the proportion of funding it will allocate to deworming, and costs other than grants.

Evidence Action’s Deworm the World Initiative

Our full review of Deworm the World is here.

Background

Evidence Action’s Deworm the World (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates deworming programs. Its main countries of operation are India, Kenya, and Nigeria, and it is considering expanding to Pakistan.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World overall is the most cost-effective charity we have found. We estimate that it is ~12 times as cost-effective as cash transfers, but note that, due to differences in worm burdens and costs across countries, there is significant variation in cost-effectiveness across the countries in which it works. We estimate that its work to date in India has been more than 30 times as cost-effective as cash transfers, while its planned work in Nigeria is around three times as cost-effective as cash transfers (though this estimate is based on low-quality information).

Important changes in the last 12 months

We estimate that Deworm the World could absorb considerably more funding this year than we estimated last year, due to opportunities it has identified to expand its geographic reach. (More in the next section.)

The quality of the monitoring that we have seen from Deworm the World has remained high. To date, we have seen limited monitoring from Nigeria, which is a new addition to Deworm the World’s portfolio and is expected to become a major portion of its work in the future. This is of minor concern given the strong monitoring track record elsewhere and how new the program is in Nigeria.

Funding gap

We believe that Deworm the World is very likely to be constrained by funding. We expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive over the next three years. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities.

With additional funding, Deworm the World would sustain its current work in Kenya and India, and would seek to expand its work in Nigeria and India to additional states and support the government in Pakistan to initiate a deworming program.

GiveDirectly

Our full review of GiveDirectly is here.

Background

GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82 percent overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people.

We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. We believe that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected. It continues to experiment heavily, with the aim of improving how its own cash transfer programs are run as well as those of governments. It has recently started work on a universal basic income trial and has started partnering with major funders on evaluations of cash transfers in new geographies with the aim of influencing the broader international aid sector to use its funding more cost-effectively.

We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost-effective than cash.

Important changes in the last 12 months

We had previously expressed reservations about GiveDirectly’s targeting strategy: that by excluding the least poor households in each village, the program might lead to negative reactions by non-recipients, increase costs per household reached, and exclude households that were still quite poor. In 2017, GiveDirectly largely switched to a “saturation” approach of making transfers to all households in selected villages. It will continue to use a targeted approach in Rwanda, where government regulations require such an approach, but the saturation approach will be used in Kenya and Uganda.

In 2016, GiveDirectly built up its operations in Uganda and Kenya with the anticipation of revenue growth in 2017. Revenue growth has been slower than expected and GiveDirectly had to lay off some staff as a result.

GiveDirectly launched its universal basic income project this month.

In 2015, Good Ventures made a grant of $25 million to GiveDirectly on GiveWell’s recommendation. GiveDirectly’s goals for the grant were to expand its ability to raise funds from donors not influenced by GiveWell’s recommendation and to collaborate with large aid institutions or governments to address their questions about cash transfers. We expect to write more about the performance of the grant in the future, but, in short, our impression is that fundraising has progressed slower than expected and collaborative projects have progressed more quickly than expected.

Funding gap

We believe that GiveDirectly is highly likely to be constrained by funding next year. It expects to use additional funding primarily for standard cash transfers and for additional collaborative projects. For collaborative projects, GiveDirectly’s potential partners require it to contribute funding, which the partner matches (at a one-to-one ratio, minimum). These projects would largely be in countries GiveDirectly has not worked in before and many are at an early stage of discussion. We estimate that GiveDirectly could use more than $200 million in additional funding in 2018-2019.

Malaria Consortium (for work on seasonal malaria chemoprevention)

Our full review of Malaria Consortium’s seasonal malaria chemoprevention program is here.

Background

Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria.

There is strong evidence that SMC substantially reduces cases of malaria. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately statistically powered to find an impact on mortality.

Malaria Consortium and its partners have conducted studies in all of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, though past surveys have been conducted after four rounds of SMC (SMC is given in a maximum of four treatment courses at monthly intervals) and may be subject to error due to the inaccurate recall or recordkeeping. Starting in 2017, Malaria Consortium is conducting coverage surveys after each round of SMC, to reduce recall error.

Important changes in the last 12 months

We have increased our confidence in Malaria Consortium’s monitoring, though we have not yet seen all of the research that Malaria Consortium expected to share in 2017 (in particular, tracking of malaria cases and deaths over time in areas where Malaria Consortium works). Coverage survey results from 2016 were generally positive, with a couple of outliers. The change from conducting coverage surveys after four treatment cycles to conducting them after each cycle will increase our confidence in the results.

Last year, we had only a rough estimate of how much additional funding Malaria Consortium could use productively. We have significantly improved our understanding of its room for more funding this year.

Funding gap

We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. It appears that there is a large remaining global need for additional funding for SMC programs and that Malaria Consortium is well-positioned to fill these gaps, if it has sufficient funding to do so.

Malaria Consortium estimates that it could spend $28-30 million per year on SMC in each of the next three years and that this level of funding would largely fill the global funding gap for SMC, with the exception of Nigeria, where the scale of the gap would be beyond Malaria Consortium’s operational capacity in the short term.

It appears to have limited prospects for major funding from other sources. The major grant for Malaria Consortium’s work on SMC previously, from Unitaid, is ending and Malaria Consortium told us that it will not be renewed.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify partner countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies to determine whether its programs have reached a large proportion of children targeted. These studies cover (a non-random sample of) about 40 percent of treatments SCI reports having delivered over the past few years. The studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is less cost-effective than Deworm the World and more cost-effective than Sightsavers and the END Fund. Given the uncertainty in our cost-effectiveness model, we are hesitant to say that SCI is more cost-effective than AMF and Malaria Consortium, though taken literally, SCI is 1.5 times as cost-effective as AMF and Malaria Consortium (~10x cash transfers vs. ~6-7x cash transfers).

Important changes in the last 12 months

We continued to follow SCI’s progress in 2017 and there have not been many major changes to its work. As in the past, SCI shared monitoring of deworming coverage levels for a portion of its programs with us; there continue to be several SCI-supported countries for which we have not seen monitoring results. In the past, we have noted that we had low confidence in the accuracy of the financial information that SCI provided and that SCI made significant improvements to its financial systems in 2016; our remaining concerns about SCI’s financial management and reporting are fairly minor.

In 2017, SCI allocated nearly all available funding to programs in its 2017-2018 budget year. This was a large increase in spending over the previous budget year ($9.6 million in 2016-2017 compared with $22.5 million in 2017-2018), driven in large part by a large increase in GiveWell-directed funding ($3.7 million in 2015 compared with $16.6 million in 2016). We believe this decision was due in part to a miscommunication with GiveWell—in a conversation with SCI in early 2017, we recommended that they treat the funds like a multi-year grant because of the risk of large fluctuations in GiveWell-directed funding, but we did not emphasize this point. SCI told us that it plans to allocate future funding over multiple years, noting that its funding allocation decisions in 2016-2017 were due to the desire to avoid allowing drugs to expire as well as a misunderstanding with GiveWell about how the funding was intended to be used.

Funding gap

We estimate that SCI could productively use about $30 million more than it expects to receive to deliver treatments to school-aged children over the next three years. It could use almost three times this amount if it were to follow World Health Organization guidelines, which include treating many adults; we are not recommending funding to treat adults because we haven’t seen sufficient evidence on the impact of treating adults.

The primary use of this funding, and SCI’s top priority, would be to sustain and expand work in current countries of operation. A smaller portion would be used to expand to up to four additional countries.

Sightsavers (for work on deworming)

Our full review of Sightsavers is here.

Background

Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and – more specifically – advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections.

Sightsavers has shared surveys for some of its past NTD programs that measure whether these programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact. We have seen very limited results from Sightsavers’ deworming programs specifically. For GiveWell-supported programs, Sightsavers has told us it will conduct coverage surveys for each round of deworming; we have reviewed one of those surveys to date.

Important changes in the last 12 months

In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results.

We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years.

Funding gap

We believe that Sightsavers’ deworming work is likely to be constrained by funding next year. Sightsavers has provided details of deworming programs that it could fund with additional funding of up to about $6.4 million in 2018 and 2019. Sightsavers appears to have limited prospects for funding these programs from other sources. We believe it is likely that Sightsavers could absorb funding beyond this amount to extend programs to 2020 and/or seek out additional opportunities to fund deworming programs.

Of the $6.4 million, $2.8 million would be used to add deworming to existing NTD programs and $3.7 million would be used to fund NTD programs that would treat several NTDs in addition to schistosomiasis and STH. We will request that Sightsavers prioritize the first set of opportunities, because we believe they will likely be more cost-effective.

Standout charities

In addition to our top charities, we recognize standout charities—organizations that support programs that may be extremely cost-effective and are evidence-backed but for which we have less confidence in their impact than we do for our top charities. We have reviewed their work and feel these groups stand out from the vast majority of organizations we have considered in terms of the evidence base for the program they support, their transparency, and their potential cost-effectiveness. These organizations offer additional giving options for donors who feel highly aligned with their work.

We’ve added one organization to the list this year: Evidence Action’s Dispensers for Safe Water.

We don’t follow standout organizations as closely as we do our top charities. We generally have one or two calls per year with representatives from each group and publish notes on our conversations. We provide brief updates on these charities below.

New addition to the standout list:

• Evidence Action’s Dispensers for Safe Water. The Dispensers for Safe Water program provides chlorine dispensers for decontamination of drinking water to prevent diarrhea and associated deaths of young children. We believe that there is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death. Our rough cost-effectiveness analysis of Dispensers for Safe Water suggests that the program is in a similar range of cost-effectiveness as unconditional cash transfer programs. Our review of Dispensers for Safe Water is here.

Organizations that have conducted randomized controlled trials of their programs:

• Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its child survival media campaign in Burkina Faso and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15 percent or more) and data from health facilities finding an increase in facility visits. (The results have not yet been published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. DMI is conducting an RCT of its family planning radio campaign in Burkina Faso and it is planning work on early child development in Burkina Faso and child survival in Mozambique. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI is here and notes from our most recent conversation with DMI are here.
• Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a RCT of its program and measured a 27 percent reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of GiveDirectly. It is conducting a second RCT of its program and results are expected in 2020. Living Goods recently expanded the number of family planning products it offers and is interested in expanding to a third country. Living Goods is scaling up its program and could scale up more quickly with additional funding. Our review of Living Goods is here and notes from our most recent conversation with Living Goods are here.

Organizations working on micronutrient fortification:

We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence.

• Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost-effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. FFI’s recent work includes advocating for legislation to mandate that rice imported to West Africa is fortified with vitamins and minerals. Our full review is here and notes from our most recent conversation are here.
• Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. GAIN has focused its recent USI work on Tanzania, Mozambique, Ethiopia, and Kenya, which it targeted based on relatively low levels of coverage of iodized salt and strong relationships with stakeholders. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN is here and notes from our most recent conversation are here.
• Iodine Global Network (IGN). Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-directed funding has made up a large part of its funding in recent years. It expects to have data from before and after its recent work in Madagascar, Lebanon, and possibly Israel by the end of 2018; this data may provide additional evidence of IGN’s impact. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN is here and notes from our most recent conversation here.
• Project Healthy Children (PHC)/Sanku. PHC/Sanku aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs and by enabling fortification among small-scale millers. PHC is scaling up its Sanku project, which equips small millers with a machine that enables them to fortify their flour with micronutrients; we have not done as much formal analysis of Sanku as of PHC’s core work on advocacy and technical assistance to countries to implement fortification. PHC/Sanku expects to be constrained by funding in the future. Our review of PHC/Sanku is here and notes from our more recent conversation are here.

Our research process in 2017

We plan to detail the work we completed this year in a future post as part of our annual review process. A major focus of 2017 was improving our recommendations in future years, in particular through our work on GiveWell Incubation Grants and prioritizing promising programs for further investigation.

Below, we highlight the key research that led to our current charity recommendations. This page describes our overall process.

• Following existing top charities. We followed the progress and plans of each of our 2016 top charities. We had several conversations by phone with each organization, met in person at least once with each top charity (including a three-day visit to Rwanda and the Democratic Republic of the Congo with the END Fund), and reviewed documents they shared with us.
• Identifying new top charities.
  • No Lean Season. We had recommended a series of Incubation Grants to No Lean Season beginning in 2014 and have followed its progress since then. This year, due to the scale at which No Lean Season was operating and the track record it had established, we decided that the No Lean Season program was at a stage of development where we could evaluate it as a potential top charity. In addition to extensive communications with No Lean Season staff over the phone and reviewing documents they shared with us, GiveWell staff spent five days visiting the program in Bangladesh.
  • Helen Keller International’s vitamin A supplementation program. Earlier this year, Research Analyst Chelsea Tabart began reaching out to organizations that might be a fit for our criteria, but with which we had limited or no previous contact with. As a result of that process, we reconnected with Helen Keller International (which we first considered as a potential top charity in 2007) and began to consider its vitamin A supplementation program as a potential top charity. In addition to extensive communications with HKI staff over the phone and reviewing documents they shared with us, GiveWell staff spent three days meeting with HKI staff in Guinea and observing a vitamin A supplementation program.
• Completing intervention reports on obstetric fistula surgery and measles vaccination campaigns; completing interim intervention reports on SMS reminders for vaccination, Sayana® Press (an injectable contraceptive), oral rehydration solution, and antiretroviral therapy for HIV/AIDS; and expanding our interim intervention report on seasonal malaria chemoprevention to a full intervention report.
• Staying up to date on the research for malaria nets, cash transfers, and deworming. We did not find major new research on cash transfers, nets, or deworming that affected our recommendation of GiveDirectly, AMF, or the organizations we recommend for their work on deworming. David Roodman published an in-depth review (parts 1 and 2) of the deworming studies that form the primary basis of our views on the impact of deworming (though much of this work was completed in 2016 and informed our top charity recommendations last year).
• Making extensive updates to our cost-effectiveness model and publishing several updates to the model over the course of the year. We instituted a process to track and report publicly on updates to the model to reduce the possibility of errors and make our process more transparent. This year, staff members have also provided substantially more detail in our cost-effectiveness file about why they have chosen particular inputs.

Giving to GiveWell vs. top charities

GiveWell is currently in a stable financial position. We project that our revenue and our expenses will be approximately equal in the future. However, this projection forecasts some growth in the level of operating support we receive.

In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations. We retain our “excess assets policy” to ensure that if we fundraise for our own operations beyond a certain level, we will grant the excess to our recommended charities.

We cap the amount of operating support we ask Good Ventures to provide to GiveWell at 20 percent, for reasons described here. We thus ask that donors who use GiveWell’s research consider the following:

• If you have supported GiveWell’s operations in the past, we ask that you maintain your support. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and to minimize staff time spent on fundraising for our operating expenses.
• If you have not supported GiveWell’s operations in the past, we ask that you designate 10 percent of your donation to help fund GiveWell’s operations. This can be done by selecting the option to “Add 10% to help fund GiveWell’s operations” on our credit card donation form or letting us know how you would like to designate your funding when giving another way.

We’re happy to answer questions in the comments below. Please also feel free to reach out directly with any questions.

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

This year, we added two new top charities, Evidence Action's No Lean Season program and Helen Keller International's vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action's Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Read More

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

This year, we added two new top charities, Evidence Action’s No Lean Season program and Helen Keller International’s vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action’s Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Our top charities and recommendations for donors, in brief

Top charities

We now have nine top charities. They are:

• Against Malaria Foundation
• Schistosomiasis Control Initiative
• Malaria Consortium’s seasonal malaria chemoprevention (SMC) program
• Evidence Action’s Deworm the World Initiative
• Helen Keller International (HKI)’s vitamin A supplementation (VAS) program
• Sightsavers’ deworming program
• END Fund’s deworming program
• Evidence Action’s No Lean Season program
• GiveDirectly

Standout charities

We also provide a list of standout charities. We believe they are implementing programs that are evidence-backed and may be extremely cost-effective. However, we do not feel as confident in the impact of these organizations as we do in our top charities.

• Development Media International
• Evidence Action’s Dispensers for Safe Water program
• Food Fortification Initiative
• Global Alliance for Improved Nutrition’s Universal Salt Iodization program
• Iodine Global Network
• Living Goods
• Project Healthy Children

Conference call to discuss recommendations

We are planning to hold a conference call at 1:30pm ET/10:30am PT on Thursday, November 30 to discuss our charity recommendations and answer your questions.

If you’d like to join the call, please register using this online form. If you can’t make this date, but would be interested in joining another call at a later date, please indicate this on the registration form.

Additional details and explanation

Below, we provide:

• An explanation of changes to our recommended charity list and of major changes to our review process in the past year that are not specific to any one organization. More
• A discussion of our approach to determining how much funding charities can use effectively (“room for more funding”) and our ranking of charities’ funding gaps. More
• Reasoning behind how we have ranked charities’ funding gaps. More
• Details about each of our new top charities, including an overview of what we know about their work and our understanding of their funding needs. More
• Details about each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year, and our understanding of their funding needs. More
• A brief overview of each of our standout charities. More
• The process we followed that led to these recommendations. More
• An update on giving to support GiveWell’s operations versus giving to our top charities. More

Major changes in the last 12 months

Major changes to our recommended charities list and review process over the past year include:

• Overall, we believe our top charities are able to absorb more funding than they could in previous years. This is a result both of recent additions to the top charities list with large funding gaps (particularly Malaria Consortium) as well as expansion by top charities that have been on the list for a longer time (particularly Deworm the World and AMF).

  We expect overall “room for more funding” to continue to expand as we gain more confidence in recently-added top charities and continue to add new top charities, particularly through GiveWell Incubation Grants, our program to grow the pipeline of potential future top charities and improve our understanding of our current top charities.

• We added two new programs to our list of top charities: vitamin A supplementation (VAS) and seasonal migration subsidies. We have not previously recommended charities that work on these programs.

  We had considered VAS a priority program for a number of years but had not found an organization that was able to answer our key questions. While we have some remaining questions, we can now make a strong case for supporting HKI’s work on VAS.

  We initially supported No Lean Season through GiveWell’s Incubation Grants program. No Lean Season is the first organization we have added to our top charity list through our Incubation Grants program.

• Last year, the charities we recommended on the margin were estimated to be about three times as cost-effective as unconditional cash transfers, the program implemented by top charity GiveDirectly. This year, we believe that the charities we are recommending on the margin are about six times as cost-effective as cash transfers. For the most part, this change was due to (a) a series of small adjustments to our cost-effectiveness model and (b) changes in which individuals contribute to the model and the values entered into the model by these and other contributors.

  We now feel fairly confident that there will be large amounts of room for more funding in this range. As more time has passed without identifying opportunities that are considerably more cost-effective than this, we have become more pessimistic about finding such opportunities. Our current best guess is that, if they exist, they will be in the area of policy advocacy in developing countries, on issues like lead regulation and tobacco taxation. We intend to do further research in those areas.

• We made a significant change to our cost-effectiveness analysis to more formally incorporate adjustments for the way in which our top charities’ funding affects funding from other sources by (a) attracting more resources to the programs they work on (e.g., governments contributing staff time to support implementation of the programs) or (b) displacing resources that would have otherwise supported the programs. We will be writing more about this in a future post.

• We continued to analyze the complex evidence base for deworming (treating intestinal parasites), the program implemented by four out of our nine top charities.

  At the end of 2016, David Roodman, a Senior Advisor to GiveWell, conducted a detailed review of the core evidence underlying our deworming recommendation (blog posts here and here).

  This year, we saw new follow-up results on the main study that leads us to recommend deworming, which continued to show similar long-term impacts of deworming on adult earnings as were estimated previously.

  Further investigation and updates based on new data led us to believe that two deworming studies (Croke 2014 and Bleakley 2007) no longer provide substantial support for the theory that deworming has long-term impacts. We plan to write more about this in the future. All together, this work led us to the same conclusion about deworming: that it is a reasonable bet to take based on its strong cost-effectiveness (which incorporates our uncertainty about the impact).

Room for more funding analysis

Types of funding gaps

In the last two years, we used a framework of “capacity funding” and “execution levels” to compare funding gaps (unfilled funding needs) across charities. This framework was intended to capture whether funding would enable a charity to expand or grow in important ways and how likely it was, in our estimation, that each top charity would be constrained by funding in the next year.

We developed this approach in response to a situation where we expected to direct more funding to several of our top charities than they would be able to use (commit or spend) in that year. We used capacity funding to describe opportunities to increase the amount of funding a charity might be able to absorb in the future (by, say, investing in expanding to a new location) and execution levels to describe the likelihood, down to the 5 percent level, that a charity would be able to make use of additional funding before encountering non-funding bottlenecks to their work.

This year, because we have added new top charities and most of our other top charities have more room for more funding than in previous years, we expect that the funding we will direct to each organization will not reach the level where they will encounter significant non-funding bottlenecks. As a result, we have moved away from describing capacity funding and execution levels.

Ranking funding gaps

The first million dollars to a charity can have a very different impact from the 20th million dollars. Accordingly, we have created a ranking of individual funding gaps that accounts for our best guess of the impact of additional funds at each level.

The below table lays out our ranking of funding gaps, up to $75.7 million in total funding. We expect Good Ventures to give $75 million to GiveWell’s top charities this year, so this table is our recommendation to Good Ventures, plus the allocation of funding that GiveWell holds to allocate at its discretion (currently $0.7 million). We then discuss our recommendation for all other donors.

The Open Philanthropy Project, which was incubated at GiveWell but is now a separate organization, plans to write more soon about the reasons for Good Ventures increasing its support of GiveWell top charities from $50 million last year to $75 million this year. In short, the amount was based on discussions about how to allocate funding across time and across cause areas. It was not set based on the total size of top charities’ funding gaps or the projection of what others would give.

Charity Description Amount (millions) All top charities Incentive grants: $2.5 million per charity 22.5 All standout charities Standout grants: $100,000 per charity 0.7 Deworm the World Funding gaps in India and Kenya over the next three years (including central costs) 3.0 Helen Keller International Funding gaps over three years in Burkina Faso, Mali, and Guinea—countries that have missed recent vitamin A campaigns due to lack of funding 4.7 No Lean Season Full funding gap over three years for implementing the program in Bangladesh 9.0 Deworm the World Three years of funding for a new program in Pakistan and reserves to protect against funding shortfalls in India 10.4 Malaria Consortium Part of the funding gap for SMC in Burkina Faso, Nigeria, and Chad over the next three years 25.4

In total, we are recommending that Good Ventures make the following grants:

• Malaria Consortium’s seasonal malaria chemoprevention program: $27.9 million
• Evidence Action’s Deworm the World Initiative: $15.2 million. We are also recommending that GiveWell’s Board of Directors grant the $0.7 million in discretionary funds that we currently hold from the third quarter (from donors who selected to give to “Grants to recommended charities at GiveWell’s discretion” on our donation form) to Deworm the World, bringing the total to $15.9 million.
• Evidence Action’s No Lean Season program: $11.5 million
• Helen Keller International’s vitamin A supplementation program: $7.2 million
• Schistosomiasis Control Initiative: $2.5 million
• Against Malaria Foundation: $2.5 million
• Sightsavers’ deworming program: $2.5 million
• END Fund’s deworming program: $2.5 million
• GiveDirectly: $2.5 million

Our recommendation to donors

For donors who are interested in directing funding to whichever recommended charity or charities GiveWell believes has the most pressing funding need at the time the funds are granted, we recommend giving to “Grants to recommended charities at GiveWell’s discretion.” These grants will respond to the greatest funding need we see; they may not match the recommended allocation outlined below.

For donors (other than Good Ventures) who are interested in donating directly to our top charities, we recommend splitting your donation as follows:

• 70 percent to the Against Malaria Foundation
• 30 percent to the Schistosomiasis Control Initiative

Why these recommendations?

Our recommendations to donors, including Good Ventures, are based on:

1. Overall cost-effectiveness of the charity. Our cost-effectiveness model is a key input into our decision-making process, and large differences in modeled cost-effectiveness impact our recommendations. We try not to put significant weight on relatively small differences in cost-effectiveness according to the model because many inputs are highly uncertain.

   Our model this year found relatively small differences between many top charities, with Deworm the World at ~12 times as cost-effective as cash transfers, four top charities in the ~6-10x cash transfers range, and three top charities in the ~3-5x cash transfers range. We consider differences between charities implementing the same intervention or interventions that have similar inputs and output in the model more meaningful (e.g., malaria nets and seasonal malaria chemoprevention) than differences between charities implementing quite different interventions.

   We have completed a sensitivity analysis of our cost-effectiveness analysis to get a better sense for which parameters are most sensitive. We are more hesitant to consider differences in the cost-effectiveness as meaningful when they rely on very sensitive inputs.

2. Cost-effectiveness of particular funding opportunities. Charities’ work can vary significantly in cost-effectiveness across locations due to different costs, disease burdens, uptake in the targeted population, or probability that other funders would step in in GiveWell’s absence. While not a part of our formal cost-effectiveness model, we ran supplementary analyses of cost-effectiveness for some locations for which our top charities were seeking additional funding and considered the output as part of our prioritization of funding gaps.
3. Qualitative factors not captured in our cost-effectiveness model. The main factors we focused on were:
   • Proportion of the global funding need for the program that is filled. We expect that funders will generally (but imperfectly) select the areas where cost-effectiveness is higher first, leaving the areas with higher costs, lower disease burden, lower cultural acceptance of the program, etc. for last. We believe we have captured some of the consequences of this in our cost-effectiveness analysis. For example, we use national level disease burden estimates for the countries in which each charity has worked and/or plans to work; charities working in higher burden countries are therefore modelled as more cost-effective. But we do not use sub-national estimates to distinguish the highest priority regions within a country; if charities are filling the lowest priority funding gaps within a county, they will likely be less cost-effective than our model suggests. This was an important consideration in comparing AMF and Malaria Consortium. We estimate that ~80 percent of the global funding need for nets (the program AMF implements) has been filled, and ~35 percent of the global funding need for seasonal malaria chemoprevention (the program Malaria Consortium implements).
   • Our level of knowledge about the organization. We have recommended AMF, Deworm the World, SCI, and GiveDirectly for many years. We know less about Malaria Consortium and No Lean Season and the least about HKI. We seek to be somewhat conservative about recommending large amounts of funding to organizations where there is a relatively high chance that additional research could lead us to believe the program was less cost-effective than we previously thought.
   • Ease of communication with the organization. It is important to us that we are able to learn over time about the charities we recommend, to enable us to improve our decisions. The ability to communicate effectively with an organization is a key factor in our ability to learn from the organization’s experiences.
   • Ongoing monitoring and likelihood of detecting future problems. Evaluating an organization’s monitoring processes and results is an important part of our charity reviews and for the most part is not captured in our cost-effectiveness analysis. As with ease of communication, we have more confidence in recommending funds to an organization if we believe that we will learn about how successful its work has been.

Summary of key considerations for top charities

The table below summarizes the key considerations for our nine top charities. More detail is provided below, as well as in the charity reviews.

Estimated cost-effectiveness (relative to cash transfers) Our level of knowledge about the organization Primary benefits of the intervention Ease of communication Ongoing monitoring and likelihood of detecting future problems Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation Other major considerations AMF ~6x High Deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars High proportion (~80%) of global gap for program is filled Malaria Consortium (SMC program) ~7x Moderate Under-5 deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars Relatively low proportion (~35%) of global gap for program is filled Helen Keller International (VAS program) ~9x Moderate Under-5 deaths averted Strong Moderate High: could absorb tens of millions of dollars Learning benefits Deworm the World ~12x High Possible increased income in adulthood Strong Strong Moderate: could absorb millions of dollars END Fund (deworming program) ~4x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars SCI ~10x High Possible increased income in adulthood Moderate Moderate High: could absorb tens of millions of dollars Sightsavers (deworming program) ~5x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars No Lean Season ~5x Moderate Immediate increase in consumption Strong Moderate Low: further funding would be used for different types of activities Potential upside GiveDirectly Baseline High Immediate increase in consumption and assets Strong Strong Very high: could absorb over 100 million dollars

Reasons for this funding gap ranking

Prioritization of funding that we have recommended to Good Ventures (we recommend Good Ventures fill the highest-priority funding needs first, to ensure these are funded):

1. We start by recommending that each top charity receive $2.5 million as an “incentive grant.” These grants are intended to be a major contribution to the charity’s work in recognition of the fact that they have met GiveWell’s criteria and have dedicated significant time to working with us to help us follow their progress and plans each year. We don’t want our top charity funding process to be winner-takes-all because we believe that charities would be less likely to want to participate in that case.
2. After incentive grants, we believe the next most valuable funding to provide is for Deworm the World’s work in Kenya and India over the next three years. Deworm the World’s work in Kenya and India is the most cost-effective opportunity we have found. We estimate that its work in Kenya is ~20x as cost-effective as cash transfers and in India is ~30x+ as cost-effective as cash transfers.

3. We rank providing funding to our two new top charities, Helen Keller International (HKI)’s VAS program and No Lean Season, next.

   We estimate that HKI could use $7.2 million over three years to support VAS campaigns in countries with high child mortality rates that have recently missed campaigns due to lack of funds. HKI’s cost-effectiveness is at the high end of the range for top charities (~9x cash transfers). We believe HKI could absorb more than $7.2 million in additional funding for VAS effectively but that this $7.2 million gap is likely more cost-effective than HKI’s average cost-effectiveness. Also, because HKI is a new top charity of ours, we expect this first part of its gap to have significant learning benefits for us: by giving this money, we’ll be better positioned to follow HKI’s work and review its monitoring, which we believe will make it more likely that we have a more accurate estimate of its impact in future years.

   We decided to recommend funding all of No Lean Season’s funding gap in Bangladesh for the next three years. While No Lean Season’s cost-effectiveness is at the lower end of our top charities (~5x cash transfers), we see additional reasons to prioritize this gap. We believe No Lean Season is the top charity where there is the strongest case to be made for “upside”; our cost-effectiveness analysis may not capture the potential impact of scaling a new program that could lead to greater visibility and funding for a novel type of program.

4. We think the next highest priority funding to provide is $10.4 million to Deworm the World. This funding would support a new program in Pakistan and provide reserve funding for programs supported with restricted funds. We estimate that the program in Pakistan will be roughly ~7x as cost-effective as cash transfers, though this estimate is very sensitive to estimates of worm burdens in the locations where Deworm the World plans to work.

   The reserve funding is intended to make it unlikely that the India program, which we believe is very highly cost-effective, will be interrupted—Deworm the World relies on restricted funding for this program and there is some chance that this funding will not be available in the future. It may use this GiveWell-directed funding for other opportunities if it is not needed to backstop restricted funding in India; we expect that it will have unfunded opportunities remaining in the next few years, particularly in Nigeria.

5. The last funding gap on our list of recommendations for Good Ventures is $23.6 million to Malaria Consortium for its work on SMC. When choosing which gap to recommend for the remainder of Good Ventures’ $75 million, we focused on the remaining funding needs for Malaria Consortium’s SMC program, AMF, and SCI, which we believe to have the next highest-value gaps. Our cost-effectiveness model indicates that SCI is the most cost-effective of these three organizations (~10x cash transfers, compared with ~6-7x cash transfers for AMF and Malaria Consortium), but when the difference in modelled cost-effectiveness between two charities is relatively small, we also put significant weight on qualitative factors. We believe that AMF and Malaria Consortium are stronger on some qualitative factors, particularly the likelihood that we will be able to learn about the programs’ performance through the monitoring they conduct. Between AMF and Malaria Consortium, we have prioritized Malaria Consortium’s funding gap primarily due to the qualitative considerations discussed above around the proportion of the global funding need that is filled. After following Malaria Consortium for a second year, we believe that Malaria Consortium and AMF are comparable on other major qualitative factors, such as quality of ongoing monitoring and likelihood of detecting future problems.

   The total amount we are recommending for Malaria Consortium’s SMC program represents a rough compromise between providing a high level of funding to a program that we prefer to the next funding gap on the list and not wanting to make too large of a bet on an organization that we have less experience with than some other top charities.

Prioritization for non-Good Ventures donors:

1. Our current recommendation for donors is to give to GiveWell for making grants to top charities at our discretion. Our goal is for SCI to receive $9 million, in addition to the $2.5 million incentive grant that we are recommending to Good Ventures, and AMF to receive the remainder of expected GiveWell-directed funding because AMF and SCI represent the next highest-value funding opportunities we see. Giving us funding to grant at our discretion allows GiveWell to better target this allocation, and to adapt if we learn new information about pressing, high-value funding needs at our top charities.

2. For donors who prefer to give directly to charities, we recommend giving 70 percent to AMF and 30 percent to SCI. These percentages are our best guess of what will achieve our target allocation given our projections of total donations driven by our recommendations.

   This allocation comes from a belief that, at these margins, it is difficult to distinguish between the quality of AMF and SCI’s funding gaps. SCI has better modeled cost-effectiveness, while AMF appears to be better on several qualitative factors, including monitoring of program performance. We have roughly targeted a two-to-one ratio between the two.

Details on new top charities

Helen Keller International (HKI) for work on vitamin A supplementation

Our full review of HKI’s work on vitamin A supplementation is here.

Overview

HKI (http://www.hki.org/) is a large organization with multiple programs focused on reducing malnutrition and averting blindness and poor vision. Our review focuses on HKI’s work on vitamin A supplementation (VAS) and our recommendation is specific to its VAS program. HKI provides technical assistance, engages in advocacy, and contributes funding to government-run VAS programs.

There is strong evidence from many randomized controlled trials (RCTs) conducted in the 1980s and 1990s that VAS can substantially reduce child mortality, but weaker evidence on how effective VAS is in the places HKI would work with additional funding in the next few years. In particular, there is little available information on current rates of vitamin A deficiency in areas where HKI works. We have adjusted our cost-effectiveness analysis for our best guess of how much less effective VAS is today (~25 percent as effective as in the trials in the 1980s and 1990s); the intervention remains cost-effective with that adjustment.

We feel that the monitoring data that we have seen from HKI’s programs gives us limited information on HKI’s past performance, but demonstrates the types of data HKI is able to collect on program performance. We have requested that HKI collect this monitoring data of all programs funded with GiveWell-directed funds.

Overall, we have not yet investigated HKI at the same level of depth as some of our other top charities, which we have recommended for several years. We have reviewed documents from HKI, had a number of conversations with their staff, and spent three days meeting with HKI and observing a VAS campaign in Guinea. We have remaining questions about HKI’s work that we will seek more information on in the future, but overall we believe this program is, like our other top charities, an excellent giving opportunity.

Funding gap

We believe that HKI’s VAS work is highly likely to be constrained by funding next year. HKI has provided details of VAS programs that it could support with additional funding of up to about $41.4 million in 2018-2020. HKI appears to have limited prospects for funding these programs from other sources.

Our understanding is that with additional funds, HKI would cause additional rounds of VAS to occur in some countries, while in other countries, HKI primarily aims to increase coverage rates in rounds of VAS that would take place regardless of its involvement. We have asked HKI to prioritize use of GiveWell-directed funding in countries where it expects to cause additional rounds of VAS to occur. HKI’s funding gap for countries that have recently missed VAS campaigns due to lack of funds is $7.2 million.

HKI’s VAS work was supported by the Canadian government in the past. That funding ended in 2016 and has not been renewed. Over the past year, several VAS campaigns have been skipped in countries HKI previously supported.

Evidence Action’s No Lean Season program

Our full review of No Lean Season is here.

Overview

No Lean Season (https://www.evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the season of income and food insecurity (‘lean season’) between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on a household member stating their intention to migrate to urban or other rural locations to seek short-term employment.

Several randomized controlled trials (RCTs) of subsidies to increase migration provide moderately strong evidence that such an intervention increases household income and consumption during the lean season. An additional RCT is ongoing. We estimate that No Lean Season is roughly five times as cost-effective as cash transfers (see our cost-effectiveness analysis).

Evidence Action has shared some details of its plans for monitoring No Lean Season in the future, but, as many of these plans have not been fully implemented, we have seen limited results. Therefore, there is some uncertainty as to whether No Lean Season will produce the data required to give us confidence that loans are appropriately targeted and reach their intended recipients in full; that recipients are not pressured into accepting loans; and that participants successfully migrate, find work, and are not exposed to major physical and other risks while migrating.

Funding gap

We expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Evidence Action is seeking funding beyond this level to allow it to build reserves for No Lean Season.

Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net (LLIN) distributions for protection against malaria in developing countries. AMF has conducted post-distribution surveys of all completed distributions to determine whether LLINs have reached their intended destinations and how long they remain in good condition. AMF’s post-distribution surveys have generally found positive results (with some exceptions); we believe they have some methodological limitations.

We estimate that AMF’s program is roughly six times as cost-effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

Prior to this year, we had seen results from AMF’s “post-distribution check ups” (PDCUs) from two countries, Malawi and the Democratic Republic of the Congo, and had significant uncertainties about the methodology used in each location. We have now also seen results from Ghana. We have more confidence in our understanding of AMF’s PDCUs than we did previously, though this work is ongoing. In particular, we commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings (see links). Further discussion of the strengths and weaknesses of PDCUs here.

In 2017, AMF signed relatively few new agreements to fund LLIN distributions and, as a result, has a balance of $58 million in uncommitted funds, or $35 million if distributions where AMF believes agreements are imminent are counted as committed. Our understanding is that many of AMF’s conversations with countries could not progress until decisions were made about how much Global Fund funding each country would allocate to LLIN distributions (as opposed to other malaria control efforts). This decision-making process extended into late 2017. Global Fund funding is allocated on three-year cycles and we do not expect this to continue to be a bottleneck for AMF in 2018.

Funding gap

We believe that AMF is very likely to be constrained by lack of funding. There is high uncertainty in the maximum amount of funding that AMF could use productively, though we expect the maximum to be much greater than what AMF is likely to receive. To fund all of the distributions that it is currently in detailed discussions about, AMF would need $50 million more than we project it will receive. The total funding gap for LLINs for 2018-2020 appears to be hundreds of millions of dollars.

With additional funding, AMF’s top priorities would be to fund a portion of the next round of distributions, in 2018-2020, in each of the countries in which it has recently funded distributions.

END Fund (for work on deworming)

Our full review of the END Fund’s work on deworming is here.

Background

The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming.

Slightly more than half of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list.

We have seen limited monitoring results on the number of children reached in END Fund-supported programs. In 2016, the END Fund began requiring that surveys be conducted to determine whether its programs have reached a large proportion of children targeted; we have seen coverage surveys for (a non-random sample of) 35 percent of its 2016 deworming grant portfolio. These studies leave us with some remaining questions about the program’s impact.

Important changes in the last 12 months

We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas that the END Fund works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. We also saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible.

Funding gap

We believe the END Fund could substantially increase its deworming grantmaking with additional funds. We roughly estimate that there is gap of $18 million between the amount of funding the END Fund will have available for grants for deworming and the amount of funding it would need to make all of the potential grants it has identified. Sources of major uncertainty in this estimate include whether the END Fund will encounter non-funding bottlenecks in some of its identified and early-stage opportunities, the amount of funding it will receive from other sources, the proportion of funding it will allocate to deworming, and costs other than grants.

Evidence Action’s Deworm the World Initiative

Our full review of Deworm the World is here.

Background

Evidence Action’s Deworm the World (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates deworming programs. Its main countries of operation are India, Kenya, and Nigeria, and it is considering expanding to Pakistan.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World overall is the most cost-effective charity we have found. We estimate that it is ~12 times as cost-effective as cash transfers, but note that, due to differences in worm burdens and costs across countries, there is significant variation in cost-effectiveness across the countries in which it works. We estimate that its work to date in India has been more than 30 times as cost-effective as cash transfers, while its planned work in Nigeria is around three times as cost-effective as cash transfers (though this estimate is based on low-quality information).

Important changes in the last 12 months

We estimate that Deworm the World could absorb considerably more funding this year than we estimated last year, due to opportunities it has identified to expand its geographic reach. (More in the next section.)

The quality of the monitoring that we have seen from Deworm the World has remained high. To date, we have seen limited monitoring from Nigeria, which is a new addition to Deworm the World’s portfolio and is expected to become a major portion of its work in the future. This is of minor concern given the strong monitoring track record elsewhere and how new the program is in Nigeria.

Funding gap

We believe that Deworm the World is very likely to be constrained by funding. We expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive over the next three years. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities.

With additional funding, Deworm the World would sustain its current work in Kenya and India, and would seek to expand its work in Nigeria and India to additional states and support the government in Pakistan to initiate a deworming program.

GiveDirectly

Our full review of GiveDirectly is here.

Background

GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82 percent overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people.

We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. We believe that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected. It continues to experiment heavily, with the aim of improving how its own cash transfer programs are run as well as those of governments. It has recently started work on a universal basic income trial and has started partnering with major funders on evaluations of cash transfers in new geographies with the aim of influencing the broader international aid sector to use its funding more cost-effectively.

We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost-effective than cash.

Important changes in the last 12 months

We had previously expressed reservations about GiveDirectly’s targeting strategy: that by excluding the least poor households in each village, the program might lead to negative reactions by non-recipients, increase costs per household reached, and exclude households that were still quite poor. In 2017, GiveDirectly largely switched to a “saturation” approach of making transfers to all households in selected villages. It will continue to use a targeted approach in Rwanda, where government regulations require such an approach, but the saturation approach will be used in Kenya and Uganda.

In 2016, GiveDirectly built up its operations in Uganda and Kenya with the anticipation of revenue growth in 2017. Revenue growth has been slower than expected and GiveDirectly had to lay off some staff as a result.

GiveDirectly launched its universal basic income project this month.

In 2015, Good Ventures made a grant of $25 million to GiveDirectly on GiveWell’s recommendation. GiveDirectly’s goals for the grant were to expand its ability to raise funds from donors not influenced by GiveWell’s recommendation and to collaborate with large aid institutions or governments to address their questions about cash transfers. We expect to write more about the performance of the grant in the future, but, in short, our impression is that fundraising has progressed slower than expected and collaborative projects have progressed more quickly than expected.

Funding gap

We believe that GiveDirectly is highly likely to be constrained by funding next year. It expects to use additional funding primarily for standard cash transfers and for additional collaborative projects. For collaborative projects, GiveDirectly’s potential partners require it to contribute funding, which the partner matches (at a one-to-one ratio, minimum). These projects would largely be in countries GiveDirectly has not worked in before and many are at an early stage of discussion. We estimate that GiveDirectly could use more than $200 million in additional funding in 2018-2019.

Malaria Consortium (for work on seasonal malaria chemoprevention)

Our full review of Malaria Consortium’s seasonal malaria chemoprevention program is here.

Background

Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria.

There is strong evidence that SMC substantially reduces cases of malaria. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately statistically powered to find an impact on mortality.

Malaria Consortium and its partners have conducted studies in all of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, though past surveys have been conducted after four rounds of SMC (SMC is given in a maximum of four treatment courses at monthly intervals) and may be subject to error due to the inaccurate recall or recordkeeping. Starting in 2017, Malaria Consortium is conducting coverage surveys after each round of SMC, to reduce recall error.

Important changes in the last 12 months

We have increased our confidence in Malaria Consortium’s monitoring, though we have not yet seen all of the research that Malaria Consortium expected to share in 2017 (in particular, tracking of malaria cases and deaths over time in areas where Malaria Consortium works). Coverage survey results from 2016 were generally positive, with a couple of outliers. The change from conducting coverage surveys after four treatment cycles to conducting them after each cycle will increase our confidence in the results.

Last year, we had only a rough estimate of how much additional funding Malaria Consortium could use productively. We have significantly improved our understanding of its room for more funding this year.

Funding gap

We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. It appears that there is a large remaining global need for additional funding for SMC programs and that Malaria Consortium is well-positioned to fill these gaps, if it has sufficient funding to do so.

Malaria Consortium estimates that it could spend $28-30 million per year on SMC in each of the next three years and that this level of funding would largely fill the global funding gap for SMC, with the exception of Nigeria, where the scale of the gap would be beyond Malaria Consortium’s operational capacity in the short term.

It appears to have limited prospects for major funding from other sources. The major grant for Malaria Consortium’s work on SMC previously, from Unitaid, is ending and Malaria Consortium told us that it will not be renewed.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify partner countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies to determine whether its programs have reached a large proportion of children targeted. These studies cover (a non-random sample of) about 40 percent of treatments SCI reports having delivered over the past few years. The studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is less cost-effective than Deworm the World and more cost-effective than Sightsavers and the END Fund. Given the uncertainty in our cost-effectiveness model, we are hesitant to say that SCI is more cost-effective than AMF and Malaria Consortium, though taken literally, SCI is 1.5 times as cost-effective as AMF and Malaria Consortium (~10x cash transfers vs. ~6-7x cash transfers).

Important changes in the last 12 months

We continued to follow SCI’s progress in 2017 and there have not been many major changes to its work. As in the past, SCI shared monitoring of deworming coverage levels for a portion of its programs with us; there continue to be several SCI-supported countries for which we have not seen monitoring results. In the past, we have noted that we had low confidence in the accuracy of the financial information that SCI provided and that SCI made significant improvements to its financial systems in 2016; our remaining concerns about SCI’s financial management and reporting are fairly minor.

In 2017, SCI allocated nearly all available funding to programs in its 2017-2018 budget year. This was a large increase in spending over the previous budget year ($9.6 million in 2016-2017 compared with $22.5 million in 2017-2018), driven in large part by a large increase in GiveWell-directed funding ($3.7 million in 2015 compared with $16.6 million in 2016). We believe this decision was due in part to a miscommunication with GiveWell—in a conversation with SCI in early 2017, we recommended that they treat the funds like a multi-year grant because of the risk of large fluctuations in GiveWell-directed funding, but we did not emphasize this point. SCI told us that it plans to allocate future funding over multiple years, noting that its funding allocation decisions in 2016-2017 were due to the desire to avoid allowing drugs to expire as well as a misunderstanding with GiveWell about how the funding was intended to be used.

Funding gap

We estimate that SCI could productively use about $30 million more than it expects to receive to deliver treatments to school-aged children over the next three years. It could use almost three times this amount if it were to follow World Health Organization guidelines, which include treating many adults; we are not recommending funding to treat adults because we haven’t seen sufficient evidence on the impact of treating adults.

The primary use of this funding, and SCI’s top priority, would be to sustain and expand work in current countries of operation. A smaller portion would be used to expand to up to four additional countries.

Sightsavers (for work on deworming)

Our full review of Sightsavers is here.

Background

Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and – more specifically – advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections.

Sightsavers has shared surveys for some of its past NTD programs that measure whether these programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact. We have seen very limited results from Sightsavers’ deworming programs specifically. For GiveWell-supported programs, Sightsavers has told us it will conduct coverage surveys for each round of deworming; we have reviewed one of those surveys to date.

Important changes in the last 12 months

In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results.

We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years.

Funding gap

We believe that Sightsavers’ deworming work is likely to be constrained by funding next year. Sightsavers has provided details of deworming programs that it could fund with additional funding of up to about $6.4 million in 2018 and 2019. Sightsavers appears to have limited prospects for funding these programs from other sources. We believe it is likely that Sightsavers could absorb funding beyond this amount to extend programs to 2020 and/or seek out additional opportunities to fund deworming programs.

Of the $6.4 million, $2.8 million would be used to add deworming to existing NTD programs and $3.7 million would be used to fund NTD programs that would treat several NTDs in addition to schistosomiasis and STH. We will request that Sightsavers prioritize the first set of opportunities, because we believe they will likely be more cost-effective.

Standout charities

In addition to our top charities, we recognize standout charities—organizations that support programs that may be extremely cost-effective and are evidence-backed but for which we have less confidence in their impact than we do for our top charities. We have reviewed their work and feel these groups stand out from the vast majority of organizations we have considered in terms of the evidence base for the program they support, their transparency, and their potential cost-effectiveness. These organizations offer additional giving options for donors who feel highly aligned with their work.

We’ve added one organization to the list this year: Evidence Action’s Dispensers for Safe Water.

We don’t follow standout organizations as closely as we do our top charities. We generally have one or two calls per year with representatives from each group and publish notes on our conversations. We provide brief updates on these charities below.

New addition to the standout list:

• Evidence Action’s Dispensers for Safe Water. The Dispensers for Safe Water program provides chlorine dispensers for decontamination of drinking water to prevent diarrhea and associated deaths of young children. We believe that there is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death. Our rough cost-effectiveness analysis of Dispensers for Safe Water suggests that the program is in a similar range of cost-effectiveness as unconditional cash transfer programs. Our review of Dispensers for Safe Water is here.

Organizations that have conducted randomized controlled trials of their programs:

• Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its child survival media campaign in Burkina Faso and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15 percent or more) and data from health facilities finding an increase in facility visits. (The results have not yet been published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. DMI is conducting an RCT of its family planning radio campaign in Burkina Faso and it is planning work on early child development in Burkina Faso and child survival in Mozambique. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI is here and notes from our most recent conversation with DMI are here.
• Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a RCT of its program and measured a 27 percent reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of GiveDirectly. It is conducting a second RCT of its program and results are expected in 2020. Living Goods recently expanded the number of family planning products it offers and is interested in expanding to a third country. Living Goods is scaling up its program and could scale up more quickly with additional funding. Our review of Living Goods is here and notes from our most recent conversation with Living Goods are here.

Organizations working on micronutrient fortification:

We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence.

• Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost-effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. FFI’s recent work includes advocating for legislation to mandate that rice imported to West Africa is fortified with vitamins and minerals. Our full review is here and notes from our most recent conversation are here.
• Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. GAIN has focused its recent USI work on Tanzania, Mozambique, Ethiopia, and Kenya, which it targeted based on relatively low levels of coverage of iodized salt and strong relationships with stakeholders. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN is here and notes from our most recent conversation are here.
• Iodine Global Network (IGN). Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-directed funding has made up a large part of its funding in recent years. It expects to have data from before and after its recent work in Madagascar, Lebanon, and possibly Israel by the end of 2018; this data may provide additional evidence of IGN’s impact. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN is here and notes from our most recent conversation here.
• Project Healthy Children (PHC)/Sanku. PHC/Sanku aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs and by enabling fortification among small-scale millers. PHC is scaling up its Sanku project, which equips small millers with a machine that enables them to fortify their flour with micronutrients; we have not done as much formal analysis of Sanku as of PHC’s core work on advocacy and technical assistance to countries to implement fortification. PHC/Sanku expects to be constrained by funding in the future. Our review of PHC/Sanku is here and notes from our more recent conversation are here.

Our research process in 2017

We plan to detail the work we completed this year in a future post as part of our annual review process. A major focus of 2017 was improving our recommendations in future years, in particular through our work on GiveWell Incubation Grants and prioritizing promising programs for further investigation.

Below, we highlight the key research that led to our current charity recommendations. This page describes our overall process.

• Following existing top charities. We followed the progress and plans of each of our 2016 top charities. We had several conversations by phone with each organization, met in person at least once with each top charity (including a three-day visit to Rwanda and the Democratic Republic of the Congo with the END Fund), and reviewed documents they shared with us.
• Identifying new top charities.
  • No Lean Season. We had recommended a series of Incubation Grants to No Lean Season beginning in 2014 and have followed its progress since then. This year, due to the scale at which No Lean Season was operating and the track record it had established, we decided that the No Lean Season program was at a stage of development where we could evaluate it as a potential top charity. In addition to extensive communications with No Lean Season staff over the phone and reviewing documents they shared with us, GiveWell staff spent five days visiting the program in Bangladesh.
  • Helen Keller International’s vitamin A supplementation program. Earlier this year, Research Analyst Chelsea Tabart began reaching out to organizations that might be a fit for our criteria, but with which we had limited or no previous contact with. As a result of that process, we reconnected with Helen Keller International (which we first considered as a potential top charity in 2007) and began to consider its vitamin A supplementation program as a potential top charity. In addition to extensive communications with HKI staff over the phone and reviewing documents they shared with us, GiveWell staff spent three days meeting with HKI staff in Guinea and observing a vitamin A supplementation program.
• Completing intervention reports on obstetric fistula surgery and measles vaccination campaigns; completing interim intervention reports on SMS reminders for vaccination, Sayana® Press (an injectable contraceptive), oral rehydration solution, and antiretroviral therapy for HIV/AIDS; and expanding our interim intervention report on seasonal malaria chemoprevention to a full intervention report.
• Staying up to date on the research for malaria nets, cash transfers, and deworming. We did not find major new research on cash transfers, nets, or deworming that affected our recommendation of GiveDirectly, AMF, or the organizations we recommend for their work on deworming. David Roodman published an in-depth review (parts 1 and 2) of the deworming studies that form the primary basis of our views on the impact of deworming (though much of this work was completed in 2016 and informed our top charity recommendations last year).
• Making extensive updates to our cost-effectiveness model and publishing several updates to the model over the course of the year. We instituted a process to track and report publicly on updates to the model to reduce the possibility of errors and make our process more transparent. This year, staff members have also provided substantially more detail in our cost-effectiveness file about why they have chosen particular inputs.

Giving to GiveWell vs. top charities

GiveWell is currently in a stable financial position. We project that our revenue and our expenses will be approximately equal in the future. However, this projection forecasts some growth in the level of operating support we receive.

In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations. We retain our “excess assets policy” to ensure that if we fundraise for our own operations beyond a certain level, we will grant the excess to our recommended charities.

We cap the amount of operating support we ask Good Ventures to provide to GiveWell at 20 percent, for reasons described here. We thus ask that donors who use GiveWell’s research consider the following:

• If you have supported GiveWell’s operations in the past, we ask that you maintain your support. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and to minimize staff time spent on fundraising for our operating expenses.
• If you have not supported GiveWell’s operations in the past, we ask that you designate 10 percent of your donation to help fund GiveWell’s operations. This can be done by selecting the option to “Add 10% to help fund GiveWell’s operations” on our credit card donation form or letting us know how you would like to designate your funding when giving another way.

We’re happy to answer questions in the comments below. Please also feel free to reach out directly with any questions.

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

This year, we added two new top charities, Evidence Action’s No Lean Season program and Helen Keller International’s vitamin A supplementation program, and retained our seven top charities from 2016. We also added Evidence Action’s Dispensers for Safe Water program to our list of standout charities.

We recommend that donors give to GiveWell for granting to top charities at our discretion so that we can direct the funding to the top charity or charities with the most pressing funding need. For donors who prefer to give directly to our top charities, we recommend giving 70 percent of your donation to the Against Malaria Foundation (AMF) and 30 percent to the Schistosomiasis Control Initiative (SCI) to maximize your impact. We expect Good Ventures, a foundation with which we work closely, to provide significant support to each top charity; our recommendation to give to AMF and SCI is based on how much good we believe additional donations can do.

Our top charities and recommendations for donors, in brief

Top charities

We now have nine top charities. They are:

• Against Malaria Foundation
• Schistosomiasis Control Initiative
• Malaria Consortium’s seasonal malaria chemoprevention (SMC) program
• Evidence Action’s Deworm the World Initiative
• Helen Keller International (HKI)’s vitamin A supplementation (VAS) program
• Sightsavers’ deworming program
• END Fund’s deworming program
• Evidence Action’s No Lean Season program
• GiveDirectly

Standout charities

We also provide a list of standout charities. We believe they are implementing programs that are evidence-backed and may be extremely cost-effective. However, we do not feel as confident in the impact of these organizations as we do in our top charities.

• Development Media International
• Evidence Action’s Dispensers for Safe Water program
• Food Fortification Initiative
• Global Alliance for Improved Nutrition’s Universal Salt Iodization program
• Iodine Global Network
• Living Goods
• Project Healthy Children

Conference call to discuss recommendations

We are planning to hold a conference call at 1:30pm ET/10:30am PT on Thursday, November 30 to discuss our charity recommendations and answer your questions.

If you’d like to join the call, please register using this online form. If you can’t make this date, but would be interested in joining another call at a later date, please indicate this on the registration form.

Additional details and explanation

Below, we provide:

• An explanation of changes to our recommended charity list and of major changes to our review process in the past year that are not specific to any one organization. More
• A discussion of our approach to determining how much funding charities can use effectively (“room for more funding”) and our ranking of charities’ funding gaps. More
• Reasoning behind how we have ranked charities’ funding gaps. More
• Details about each of our new top charities, including an overview of what we know about their work and our understanding of their funding needs. More
• Details about each of the top charities we are continuing to recommend, including an overview of their work, major changes over the past year, and our understanding of their funding needs. More
• A brief overview of each of our standout charities. More
• The process we followed that led to these recommendations. More
• An update on giving to support GiveWell’s operations versus giving to our top charities. More

Major changes in the last 12 months

Major changes to our recommended charities list and review process over the past year include:

• Overall, we believe our top charities are able to absorb more funding than they could in previous years. This is a result both of recent additions to the top charities list with large funding gaps (particularly Malaria Consortium) as well as expansion by top charities that have been on the list for a longer time (particularly Deworm the World and AMF).

  We expect overall “room for more funding” to continue to expand as we gain more confidence in recently-added top charities and continue to add new top charities, particularly through GiveWell Incubation Grants, our program to grow the pipeline of potential future top charities and improve our understanding of our current top charities.

• We added two new programs to our list of top charities: vitamin A supplementation (VAS) and seasonal migration subsidies. We have not previously recommended charities that work on these programs.

  We had considered VAS a priority program for a number of years but had not found an organization that was able to answer our key questions. While we have some remaining questions, we can now make a strong case for supporting HKI’s work on VAS.

  We initially supported No Lean Season through GiveWell’s Incubation Grants program. No Lean Season is the first organization we have added to our top charity list through our Incubation Grants program.

• Last year, the charities we recommended on the margin were estimated to be about three times as cost-effective as unconditional cash transfers, the program implemented by top charity GiveDirectly. This year, we believe that the charities we are recommending on the margin are about six times as cost-effective as cash transfers. For the most part, this change was due to (a) a series of small adjustments to our cost-effectiveness model and (b) changes in which individuals contribute to the model and the values entered into the model by these and other contributors.

  We now feel fairly confident that there will be large amounts of room for more funding in this range. As more time has passed without identifying opportunities that are considerably more cost-effective than this, we have become more pessimistic about finding such opportunities. Our current best guess is that, if they exist, they will be in the area of policy advocacy in developing countries, on issues like lead regulation and tobacco taxation. We intend to do further research in those areas.

• We made a significant change to our cost-effectiveness analysis to more formally incorporate adjustments for the way in which our top charities’ funding affects funding from other sources by (a) attracting more resources to the programs they work on (e.g., governments contributing staff time to support implementation of the programs) or (b) displacing resources that would have otherwise supported the programs. We will be writing more about this in a future post.

• We continued to analyze the complex evidence base for deworming (treating intestinal parasites), the program implemented by four out of our nine top charities.

  At the end of 2016, David Roodman, a Senior Advisor to GiveWell, conducted a detailed review of the core evidence underlying our deworming recommendation (blog posts here and here).

  This year, we saw new follow-up results on the main study that leads us to recommend deworming, which continued to show similar long-term impacts of deworming on adult earnings as were estimated previously.

  Further investigation and updates based on new data led us to believe that two deworming studies (Croke 2014 and Bleakley 2007) no longer provide substantial support for the theory that deworming has long-term impacts. We plan to write more about this in the future. All together, this work led us to the same conclusion about deworming: that it is a reasonable bet to take based on its strong cost-effectiveness (which incorporates our uncertainty about the impact).

Room for more funding analysis

Types of funding gaps

In the last two years, we used a framework of “capacity funding” and “execution levels” to compare funding gaps (unfilled funding needs) across charities. This framework was intended to capture whether funding would enable a charity to expand or grow in important ways and how likely it was, in our estimation, that each top charity would be constrained by funding in the next year.

We developed this approach in response to a situation where we expected to direct more funding to several of our top charities than they would be able to use (commit or spend) in that year. We used capacity funding to describe opportunities to increase the amount of funding a charity might be able to absorb in the future (by, say, investing in expanding to a new location) and execution levels to describe the likelihood, down to the 5 percent level, that a charity would be able to make use of additional funding before encountering non-funding bottlenecks to their work.

This year, because we have added new top charities and most of our other top charities have more room for more funding than in previous years, we expect that the funding we will direct to each organization will not reach the level where they will encounter significant non-funding bottlenecks. As a result, we have moved away from describing capacity funding and execution levels.

Ranking funding gaps

The first million dollars to a charity can have a very different impact from the 20th million dollars. Accordingly, we have created a ranking of individual funding gaps that accounts for our best guess of the impact of additional funds at each level.

The below table lays out our ranking of funding gaps, up to $75.7 million in total funding. We expect Good Ventures to give $75 million to GiveWell’s top charities this year, so this table is our recommendation to Good Ventures, plus the allocation of funding that GiveWell holds to allocate at its discretion (currently $0.7 million). We then discuss our recommendation for all other donors.

The Open Philanthropy Project, which was incubated at GiveWell but is now a separate organization, plans to write more soon about the reasons for Good Ventures increasing its support of GiveWell top charities from $50 million last year to $75 million this year. In short, the amount was based on discussions about how to allocate funding across time and across cause areas. It was not set based on the total size of top charities’ funding gaps or the projection of what others would give.

Charity Description Amount (millions) All top charities Incentive grants: $2.5 million per charity 22.5 All standout charities Standout grants: $100,000 per charity 0.7 Deworm the World Funding gaps in India and Kenya over the next three years (including central costs) 3.0 Helen Keller International Funding gaps over three years in Burkina Faso, Mali, and Guinea—countries that have missed recent vitamin A campaigns due to lack of funding 4.7 No Lean Season Full funding gap over three years for implementing the program in Bangladesh 9.0 Deworm the World Three years of funding for a new program in Pakistan and reserves to protect against funding shortfalls in India 10.4 Malaria Consortium Part of the funding gap for SMC in Burkina Faso, Nigeria, and Chad over the next three years 25.4

In total, we are recommending that Good Ventures make the following grants:

• Malaria Consortium’s seasonal malaria chemoprevention program: $27.9 million
• Evidence Action’s Deworm the World Initiative: $15.2 million. We are also recommending that GiveWell’s Board of Directors grant the $0.7 million in discretionary funds that we currently hold from the third quarter (from donors who selected to give to “Grants to recommended charities at GiveWell’s discretion” on our donation form) to Deworm the World, bringing the total to $15.9 million.
• Evidence Action’s No Lean Season program: $11.5 million
• Helen Keller International’s vitamin A supplementation program: $7.2 million
• Schistosomiasis Control Initiative: $2.5 million
• Against Malaria Foundation: $2.5 million
• Sightsavers’ deworming program: $2.5 million
• END Fund’s deworming program: $2.5 million
• GiveDirectly: $2.5 million

Our recommendation to donors

For donors who are interested in directing funding to whichever recommended charity or charities GiveWell believes has the most pressing funding need at the time the funds are granted, we recommend giving to “Grants to recommended charities at GiveWell’s discretion.” These grants will respond to the greatest funding need we see; they may not match the recommended allocation outlined below.

For donors (other than Good Ventures) who are interested in donating directly to our top charities, we recommend splitting your donation as follows:

• 70 percent to the Against Malaria Foundation
• 30 percent to the Schistosomiasis Control Initiative

Why these recommendations?

Our recommendations to donors, including Good Ventures, are based on:

1. Overall cost-effectiveness of the charity. Our cost-effectiveness model is a key input into our decision-making process, and large differences in modeled cost-effectiveness impact our recommendations. We try not to put significant weight on relatively small differences in cost-effectiveness according to the model because many inputs are highly uncertain.

   Our model this year found relatively small differences between many top charities, with Deworm the World at ~12 times as cost-effective as cash transfers, four top charities in the ~6-10x cash transfers range, and three top charities in the ~3-5x cash transfers range. We consider differences between charities implementing the same intervention or interventions that have similar inputs and output in the model more meaningful (e.g., malaria nets and seasonal malaria chemoprevention) than differences between charities implementing quite different interventions.

   We have completed a sensitivity analysis of our cost-effectiveness analysis to get a better sense for which parameters are most sensitive. We are more hesitant to consider differences in the cost-effectiveness as meaningful when they rely on very sensitive inputs.

2. Cost-effectiveness of particular funding opportunities. Charities’ work can vary significantly in cost-effectiveness across locations due to different costs, disease burdens, uptake in the targeted population, or probability that other funders would step in in GiveWell’s absence. While not a part of our formal cost-effectiveness model, we ran supplementary analyses of cost-effectiveness for some locations for which our top charities were seeking additional funding and considered the output as part of our prioritization of funding gaps.
3. Qualitative factors not captured in our cost-effectiveness model. The main factors we focused on were:
   • Proportion of the global funding need for the program that is filled. We expect that funders will generally (but imperfectly) select the areas where cost-effectiveness is higher first, leaving the areas with higher costs, lower disease burden, lower cultural acceptance of the program, etc. for last. We believe we have captured some of the consequences of this in our cost-effectiveness analysis. For example, we use national level disease burden estimates for the countries in which each charity has worked and/or plans to work; charities working in higher burden countries are therefore modelled as more cost-effective. But we do not use sub-national estimates to distinguish the highest priority regions within a country; if charities are filling the lowest priority funding gaps within a county, they will likely be less cost-effective than our model suggests. This was an important consideration in comparing AMF and Malaria Consortium. We estimate that ~80 percent of the global funding need for nets (the program AMF implements) has been filled, and ~35 percent of the global funding need for seasonal malaria chemoprevention (the program Malaria Consortium implements).
   • Our level of knowledge about the organization. We have recommended AMF, Deworm the World, SCI, and GiveDirectly for many years. We know less about Malaria Consortium and No Lean Season and the least about HKI. We seek to be somewhat conservative about recommending large amounts of funding to organizations where there is a relatively high chance that additional research could lead us to believe the program was less cost-effective than we previously thought.
   • Ease of communication with the organization. It is important to us that we are able to learn over time about the charities we recommend, to enable us to improve our decisions. The ability to communicate effectively with an organization is a key factor in our ability to learn from the organization’s experiences.
   • Ongoing monitoring and likelihood of detecting future problems. Evaluating an organization’s monitoring processes and results is an important part of our charity reviews and for the most part is not captured in our cost-effectiveness analysis. As with ease of communication, we have more confidence in recommending funds to an organization if we believe that we will learn about how successful its work has been.

Summary of key considerations for top charities

The table below summarizes the key considerations for our nine top charities. More detail is provided below, as well as in the charity reviews.

Estimated cost-effectiveness (relative to cash transfers) Our level of knowledge about the organization Primary benefits of the intervention Ease of communication Ongoing monitoring and likelihood of detecting future problems Room for more funding, after expected funding from Good Ventures and donors who give independently of our recommendation Other major considerations AMF ~6x High Deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars High proportion (~80%) of global gap for program is filled Malaria Consortium (SMC program) ~7x Moderate Under-5 deaths averted and possible increased income in adulthood Strong Strong High: could absorb tens of millions of dollars Relatively low proportion (~35%) of global gap for program is filled Helen Keller International (VAS program) ~9x Moderate Under-5 deaths averted Strong Moderate High: could absorb tens of millions of dollars Learning benefits Deworm the World ~12x High Possible increased income in adulthood Strong Strong Moderate: could absorb millions of dollars END Fund (deworming program) ~4x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars SCI ~10x High Possible increased income in adulthood Moderate Moderate High: could absorb tens of millions of dollars Sightsavers (deworming program) ~5x Moderate Possible increased income in adulthood Moderate Moderate Moderate: could absorb millions of dollars No Lean Season ~5x Moderate Immediate increase in consumption Strong Moderate Low: further funding would be used for different types of activities Potential upside GiveDirectly Baseline High Immediate increase in consumption and assets Strong Strong Very high: could absorb over 100 million dollars

Reasons for this funding gap ranking

Prioritization of funding that we have recommended to Good Ventures (we recommend Good Ventures fill the highest-priority funding needs first, to ensure these are funded):

1. We start by recommending that each top charity receive $2.5 million as an “incentive grant.” These grants are intended to be a major contribution to the charity’s work in recognition of the fact that they have met GiveWell’s criteria and have dedicated significant time to working with us to help us follow their progress and plans each year. We don’t want our top charity funding process to be winner-takes-all because we believe that charities would be less likely to want to participate in that case.
2. After incentive grants, we believe the next most valuable funding to provide is for Deworm the World’s work in Kenya and India over the next three years. Deworm the World’s work in Kenya and India is the most cost-effective opportunity we have found. We estimate that its work in Kenya is ~20x as cost-effective as cash transfers and in India is ~30x+ as cost-effective as cash transfers.

3. We rank providing funding to our two new top charities, Helen Keller International (HKI)’s VAS program and No Lean Season, next.

   We estimate that HKI could use $7.2 million over three years to support VAS campaigns in countries with high child mortality rates that have recently missed campaigns due to lack of funds. HKI’s cost-effectiveness is at the high end of the range for top charities (~9x cash transfers). We believe HKI could absorb more than $7.2 million in additional funding for VAS effectively but that this $7.2 million gap is likely more cost-effective than HKI’s average cost-effectiveness. Also, because HKI is a new top charity of ours, we expect this first part of its gap to have significant learning benefits for us: by giving this money, we’ll be better positioned to follow HKI’s work and review its monitoring, which we believe will make it more likely that we have a more accurate estimate of its impact in future years.

   We decided to recommend funding all of No Lean Season’s funding gap in Bangladesh for the next three years. While No Lean Season’s cost-effectiveness is at the lower end of our top charities (~5x cash transfers), we see additional reasons to prioritize this gap. We believe No Lean Season is the top charity where there is the strongest case to be made for “upside”; our cost-effectiveness analysis may not capture the potential impact of scaling a new program that could lead to greater visibility and funding for a novel type of program.

4. We think the next highest priority funding to provide is $10.4 million to Deworm the World. This funding would support a new program in Pakistan and provide reserve funding for programs supported with restricted funds. We estimate that the program in Pakistan will be roughly ~7x as cost-effective as cash transfers, though this estimate is very sensitive to estimates of worm burdens in the locations where Deworm the World plans to work.

   The reserve funding is intended to make it unlikely that the India program, which we believe is very highly cost-effective, will be interrupted—Deworm the World relies on restricted funding for this program and there is some chance that this funding will not be available in the future. It may use this GiveWell-directed funding for other opportunities if it is not needed to backstop restricted funding in India; we expect that it will have unfunded opportunities remaining in the next few years, particularly in Nigeria.

5. The last funding gap on our list of recommendations for Good Ventures is $23.6 million to Malaria Consortium for its work on SMC. When choosing which gap to recommend for the remainder of Good Ventures’ $75 million, we focused on the remaining funding needs for Malaria Consortium’s SMC program, AMF, and SCI, which we believe to have the next highest-value gaps. Our cost-effectiveness model indicates that SCI is the most cost-effective of these three organizations (~10x cash transfers, compared with ~6-7x cash transfers for AMF and Malaria Consortium), but when the difference in modelled cost-effectiveness between two charities is relatively small, we also put significant weight on qualitative factors. We believe that AMF and Malaria Consortium are stronger on some qualitative factors, particularly the likelihood that we will be able to learn about the programs’ performance through the monitoring they conduct. Between AMF and Malaria Consortium, we have prioritized Malaria Consortium’s funding gap primarily due to the qualitative considerations discussed above around the proportion of the global funding need that is filled. After following Malaria Consortium for a second year, we believe that Malaria Consortium and AMF are comparable on other major qualitative factors, such as quality of ongoing monitoring and likelihood of detecting future problems.

   The total amount we are recommending for Malaria Consortium’s SMC program represents a rough compromise between providing a high level of funding to a program that we prefer to the next funding gap on the list and not wanting to make too large of a bet on an organization that we have less experience with than some other top charities.

Prioritization for non-Good Ventures donors:

1. Our current recommendation for donors is to give to GiveWell for making grants to top charities at our discretion. Our goal is for SCI to receive $9 million, in addition to the $2.5 million incentive grant that we are recommending to Good Ventures, and AMF to receive the remainder of expected GiveWell-directed funding because AMF and SCI represent the next highest-value funding opportunities we see. Giving us funding to grant at our discretion allows GiveWell to better target this allocation, and to adapt if we learn new information about pressing, high-value funding needs at our top charities.

2. For donors who prefer to give directly to charities, we recommend giving 70 percent to AMF and 30 percent to SCI. These percentages are our best guess of what will achieve our target allocation given our projections of total donations driven by our recommendations.

   This allocation comes from a belief that, at these margins, it is difficult to distinguish between the quality of AMF and SCI’s funding gaps. SCI has better modeled cost-effectiveness, while AMF appears to be better on several qualitative factors, including monitoring of program performance. We have roughly targeted a two-to-one ratio between the two.

Details on new top charities

Helen Keller International (HKI) for work on vitamin A supplementation

Our full review of HKI’s work on vitamin A supplementation is here.

Overview

HKI (http://www.hki.org/) is a large organization with multiple programs focused on reducing malnutrition and averting blindness and poor vision. Our review focuses on HKI’s work on vitamin A supplementation (VAS) and our recommendation is specific to its VAS program. HKI provides technical assistance, engages in advocacy, and contributes funding to government-run VAS programs.

There is strong evidence from many randomized controlled trials (RCTs) conducted in the 1980s and 1990s that VAS can substantially reduce child mortality, but weaker evidence on how effective VAS is in the places HKI would work with additional funding in the next few years. In particular, there is little available information on current rates of vitamin A deficiency in areas where HKI works. We have adjusted our cost-effectiveness analysis for our best guess of how much less effective VAS is today (~25 percent as effective as in the trials in the 1980s and 1990s); the intervention remains cost-effective with that adjustment.

We feel that the monitoring data that we have seen from HKI’s programs gives us limited information on HKI’s past performance, but demonstrates the types of data HKI is able to collect on program performance. We have requested that HKI collect this monitoring data of all programs funded with GiveWell-directed funds.

Overall, we have not yet investigated HKI at the same level of depth as some of our other top charities, which we have recommended for several years. We have reviewed documents from HKI, had a number of conversations with their staff, and spent three days meeting with HKI and observing a VAS campaign in Guinea. We have remaining questions about HKI’s work that we will seek more information on in the future, but overall we believe this program is, like our other top charities, an excellent giving opportunity.

Funding gap

We believe that HKI’s VAS work is highly likely to be constrained by funding next year. HKI has provided details of VAS programs that it could support with additional funding of up to about $41.4 million in 2018-2020. HKI appears to have limited prospects for funding these programs from other sources.

Our understanding is that with additional funds, HKI would cause additional rounds of VAS to occur in some countries, while in other countries, HKI primarily aims to increase coverage rates in rounds of VAS that would take place regardless of its involvement. We have asked HKI to prioritize use of GiveWell-directed funding in countries where it expects to cause additional rounds of VAS to occur. HKI’s funding gap for countries that have recently missed VAS campaigns due to lack of funds is $7.2 million.

HKI’s VAS work was supported by the Canadian government in the past. That funding ended in 2016 and has not been renewed. Over the past year, several VAS campaigns have been skipped in countries HKI previously supported.

Evidence Action’s No Lean Season program

Our full review of No Lean Season is here.

Overview

No Lean Season (https://www.evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the season of income and food insecurity (‘lean season’) between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on a household member stating their intention to migrate to urban or other rural locations to seek short-term employment.

Several randomized controlled trials (RCTs) of subsidies to increase migration provide moderately strong evidence that such an intervention increases household income and consumption during the lean season. An additional RCT is ongoing. We estimate that No Lean Season is roughly five times as cost-effective as cash transfers (see our cost-effectiveness analysis).

Evidence Action has shared some details of its plans for monitoring No Lean Season in the future, but, as many of these plans have not been fully implemented, we have seen limited results. Therefore, there is some uncertainty as to whether No Lean Season will produce the data required to give us confidence that loans are appropriately targeted and reach their intended recipients in full; that recipients are not pressured into accepting loans; and that participants successfully migrate, find work, and are not exposed to major physical and other risks while migrating.

Funding gap

We expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Evidence Action is seeking funding beyond this level to allow it to build reserves for No Lean Season.

Details on top charities we are continuing to recommend

Against Malaria Foundation (AMF)

Our full review of AMF is here.

Background

AMF (againstmalaria.com) provides funding for long-lasting insecticide-treated net (LLIN) distributions for protection against malaria in developing countries. AMF has conducted post-distribution surveys of all completed distributions to determine whether LLINs have reached their intended destinations and how long they remain in good condition. AMF’s post-distribution surveys have generally found positive results (with some exceptions); we believe they have some methodological limitations.

We estimate that AMF’s program is roughly six times as cost-effective as cash transfers (see our cost-effectiveness analysis). This estimate seeks to incorporate many highly uncertain inputs, such as the effect of mosquito resistance to the insecticides used in nets on how effective they are at protecting against malaria, how differences in malaria burden affect the impact of nets, and how to discount for displacing funding from other funders, among many others.

Important changes in the last 12 months

Prior to this year, we had seen results from AMF’s “post-distribution check ups” (PDCUs) from two countries, Malawi and the Democratic Republic of the Congo, and had significant uncertainties about the methodology used in each location. We have now also seen results from Ghana. We have more confidence in our understanding of AMF’s PDCUs than we did previously, though this work is ongoing. In particular, we commissioned IDinsight, an organization with which we are partnering as part of our Incubation Grants program, to observe post-distribution surveys in Malawi and Ghana and report their findings (see links). Further discussion of the strengths and weaknesses of PDCUs here.

In 2017, AMF signed relatively few new agreements to fund LLIN distributions and, as a result, has a balance of $58 million in uncommitted funds, or $35 million if distributions where AMF believes agreements are imminent are counted as committed. Our understanding is that many of AMF’s conversations with countries could not progress until decisions were made about how much Global Fund funding each country would allocate to LLIN distributions (as opposed to other malaria control efforts). This decision-making process extended into late 2017. Global Fund funding is allocated on three-year cycles and we do not expect this to continue to be a bottleneck for AMF in 2018.

Funding gap

We believe that AMF is very likely to be constrained by lack of funding. There is high uncertainty in the maximum amount of funding that AMF could use productively, though we expect the maximum to be much greater than what AMF is likely to receive. To fund all of the distributions that it is currently in detailed discussions about, AMF would need $50 million more than we project it will receive. The total funding gap for LLINs for 2018-2020 appears to be hundreds of millions of dollars.

With additional funding, AMF’s top priorities would be to fund a portion of the next round of distributions, in 2018-2020, in each of the countries in which it has recently funded distributions.

END Fund (for work on deworming)

Our full review of the END Fund’s work on deworming is here.

Background

The END Fund (end.org) manages grants, provides technical assistance, and raises funding for controlling and eliminating neglected tropical diseases (NTDs). We have focused our review on its support for deworming.

Slightly more than half of the treatments the END Fund has supported have been deworming treatments, while the rest have been for other NTDs. The END Fund has funded SCI, Deworm the World, and Sightsavers. We see the END Fund’s value-add as a GiveWell top charity as identifying and providing assistance to programs run by organizations other than those we separately recommend, and our review of the END Fund has excluded results from charities on our top charity list.

We have seen limited monitoring results on the number of children reached in END Fund-supported programs. In 2016, the END Fund began requiring that surveys be conducted to determine whether its programs have reached a large proportion of children targeted; we have seen coverage surveys for (a non-random sample of) 35 percent of its 2016 deworming grant portfolio. These studies leave us with some remaining questions about the program’s impact.

Important changes in the last 12 months

We significantly improved our understanding of the END Fund’s cost per treatment and the baseline prevalence in areas that the END Fund works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years. We also saw some monitoring from END Fund programs; previously our recommendation of the END Fund was based on specific monitoring plans that we found credible.

Funding gap

We believe the END Fund could substantially increase its deworming grantmaking with additional funds. We roughly estimate that there is gap of $18 million between the amount of funding the END Fund will have available for grants for deworming and the amount of funding it would need to make all of the potential grants it has identified. Sources of major uncertainty in this estimate include whether the END Fund will encounter non-funding bottlenecks in some of its identified and early-stage opportunities, the amount of funding it will receive from other sources, the proportion of funding it will allocate to deworming, and costs other than grants.

Evidence Action’s Deworm the World Initiative

Our full review of Deworm the World is here.

Background

Evidence Action’s Deworm the World (evidenceaction.org/#deworm-the-world) advocates for, supports, and evaluates deworming programs. Its main countries of operation are India, Kenya, and Nigeria, and it is considering expanding to Pakistan.

Deworm the World retains or hires monitors who visit schools during and following deworming campaigns. We believe its monitoring is the strongest we have seen from any organization working on deworming. Monitors have generally found high coverage rates and good performance on other measures of quality.

As noted above, we believe that Deworm the World overall is the most cost-effective charity we have found. We estimate that it is ~12 times as cost-effective as cash transfers, but note that, due to differences in worm burdens and costs across countries, there is significant variation in cost-effectiveness across the countries in which it works. We estimate that its work to date in India has been more than 30 times as cost-effective as cash transfers, while its planned work in Nigeria is around three times as cost-effective as cash transfers (though this estimate is based on low-quality information).

Important changes in the last 12 months

We estimate that Deworm the World could absorb considerably more funding this year than we estimated last year, due to opportunities it has identified to expand its geographic reach. (More in the next section.)

The quality of the monitoring that we have seen from Deworm the World has remained high. To date, we have seen limited monitoring from Nigeria, which is a new addition to Deworm the World’s portfolio and is expected to become a major portion of its work in the future. This is of minor concern given the strong monitoring track record elsewhere and how new the program is in Nigeria.

Funding gap

We believe that Deworm the World is very likely to be constrained by funding. We expect Deworm the World to have opportunities to spend $18.9 million more than we expect it to receive over the next three years. Funding beyond this level would allow Deworm the World to build its reserves and take advantage of unanticipated opportunities.

With additional funding, Deworm the World would sustain its current work in Kenya and India, and would seek to expand its work in Nigeria and India to additional states and support the government in Pakistan to initiate a deworming program.

GiveDirectly

Our full review of GiveDirectly is here.

Background

GiveDirectly (givedirectly.org) transfers cash to households in developing countries via mobile phone-linked payment services. It targets extremely low-income households. The proportion of total expenses that GiveDirectly has delivered directly to recipients is approximately 82 percent overall. We believe that this approach faces an unusually low burden of proof, and that the available evidence supports the idea that unconditional cash transfers significantly help people.

We believe GiveDirectly to be an exceptionally strong and effective organization, even more so than our other top charities. It has invested heavily in self-evaluation from the start, scaled up quickly, and communicated with us clearly. We believe that GiveDirectly has been effective at delivering cash to low-income households. GiveDirectly has one major randomized controlled trial (RCT) of its impact and took the unusual step of making the details of this study public before data was collected. It continues to experiment heavily, with the aim of improving how its own cash transfer programs are run as well as those of governments. It has recently started work on a universal basic income trial and has started partnering with major funders on evaluations of cash transfers in new geographies with the aim of influencing the broader international aid sector to use its funding more cost-effectively.

We believe cash transfers are less cost-effective than the programs our other top charities work on, but have the most direct and robust case for impact. We use cash transfers as a “baseline” in our cost-effectiveness analyses and only recommend other programs that are robustly more cost-effective than cash.

Important changes in the last 12 months

We had previously expressed reservations about GiveDirectly’s targeting strategy: that by excluding the least poor households in each village, the program might lead to negative reactions by non-recipients, increase costs per household reached, and exclude households that were still quite poor. In 2017, GiveDirectly largely switched to a “saturation” approach of making transfers to all households in selected villages. It will continue to use a targeted approach in Rwanda, where government regulations require such an approach, but the saturation approach will be used in Kenya and Uganda.

In 2016, GiveDirectly built up its operations in Uganda and Kenya with the anticipation of revenue growth in 2017. Revenue growth has been slower than expected and GiveDirectly had to lay off some staff as a result.

GiveDirectly launched its universal basic income project this month.

In 2015, Good Ventures made a grant of $25 million to GiveDirectly on GiveWell’s recommendation. GiveDirectly’s goals for the grant were to expand its ability to raise funds from donors not influenced by GiveWell’s recommendation and to collaborate with large aid institutions or governments to address their questions about cash transfers. We expect to write more about the performance of the grant in the future, but, in short, our impression is that fundraising has progressed slower than expected and collaborative projects have progressed more quickly than expected.

Funding gap

We believe that GiveDirectly is highly likely to be constrained by funding next year. It expects to use additional funding primarily for standard cash transfers and for additional collaborative projects. For collaborative projects, GiveDirectly’s potential partners require it to contribute funding, which the partner matches (at a one-to-one ratio, minimum). These projects would largely be in countries GiveDirectly has not worked in before and many are at an early stage of discussion. We estimate that GiveDirectly could use more than $200 million in additional funding in 2018-2019.

Malaria Consortium (for work on seasonal malaria chemoprevention)

Our full review of Malaria Consortium’s seasonal malaria chemoprevention program is here.

Background

Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. Our review has focused exclusively on its seasonal malaria chemoprevention (SMC) programs, which distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria.

There is strong evidence that SMC substantially reduces cases of malaria. The randomized controlled trials on SMC that we considered showed a decrease in cases of clinical malaria but were not adequately statistically powered to find an impact on mortality.

Malaria Consortium and its partners have conducted studies in all of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found positive results, though past surveys have been conducted after four rounds of SMC (SMC is given in a maximum of four treatment courses at monthly intervals) and may be subject to error due to the inaccurate recall or recordkeeping. Starting in 2017, Malaria Consortium is conducting coverage surveys after each round of SMC, to reduce recall error.

Important changes in the last 12 months

We have increased our confidence in Malaria Consortium’s monitoring, though we have not yet seen all of the research that Malaria Consortium expected to share in 2017 (in particular, tracking of malaria cases and deaths over time in areas where Malaria Consortium works). Coverage survey results from 2016 were generally positive, with a couple of outliers. The change from conducting coverage surveys after four treatment cycles to conducting them after each cycle will increase our confidence in the results.

Last year, we had only a rough estimate of how much additional funding Malaria Consortium could use productively. We have significantly improved our understanding of its room for more funding this year.

Funding gap

We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. It appears that there is a large remaining global need for additional funding for SMC programs and that Malaria Consortium is well-positioned to fill these gaps, if it has sufficient funding to do so.

Malaria Consortium estimates that it could spend $28-30 million per year on SMC in each of the next three years and that this level of funding would largely fill the global funding gap for SMC, with the exception of Nigeria, where the scale of the gap would be beyond Malaria Consortium’s operational capacity in the short term.

It appears to have limited prospects for major funding from other sources. The major grant for Malaria Consortium’s work on SMC previously, from Unitaid, is ending and Malaria Consortium told us that it will not be renewed.

Schistosomiasis Control Initiative (SCI)

Our full review of SCI is here.

Background

SCI (imperial.ac.uk/schisto) works with governments in sub-Saharan Africa to create or scale up deworming programs. SCI’s role has primarily been to identify partner countries, provide funding to governments for government-implemented programs, provide advisory support, and conduct research on the process and outcomes of the programs.

SCI has conducted studies to determine whether its programs have reached a large proportion of children targeted. These studies cover (a non-random sample of) about 40 percent of treatments SCI reports having delivered over the past few years. The studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact.

As noted above, we believe that SCI is less cost-effective than Deworm the World and more cost-effective than Sightsavers and the END Fund. Given the uncertainty in our cost-effectiveness model, we are hesitant to say that SCI is more cost-effective than AMF and Malaria Consortium, though taken literally, SCI is 1.5 times as cost-effective as AMF and Malaria Consortium (~10x cash transfers vs. ~6-7x cash transfers).

Important changes in the last 12 months

We continued to follow SCI’s progress in 2017 and there have not been many major changes to its work. As in the past, SCI shared monitoring of deworming coverage levels for a portion of its programs with us; there continue to be several SCI-supported countries for which we have not seen monitoring results. In the past, we have noted that we had low confidence in the accuracy of the financial information that SCI provided and that SCI made significant improvements to its financial systems in 2016; our remaining concerns about SCI’s financial management and reporting are fairly minor.

In 2017, SCI allocated nearly all available funding to programs in its 2017-2018 budget year. This was a large increase in spending over the previous budget year ($9.6 million in 2016-2017 compared with $22.5 million in 2017-2018), driven in large part by a large increase in GiveWell-directed funding ($3.7 million in 2015 compared with $16.6 million in 2016). We believe this decision was due in part to a miscommunication with GiveWell—in a conversation with SCI in early 2017, we recommended that they treat the funds like a multi-year grant because of the risk of large fluctuations in GiveWell-directed funding, but we did not emphasize this point. SCI told us that it plans to allocate future funding over multiple years, noting that its funding allocation decisions in 2016-2017 were due to the desire to avoid allowing drugs to expire as well as a misunderstanding with GiveWell about how the funding was intended to be used.

Funding gap

We estimate that SCI could productively use about $30 million more than it expects to receive to deliver treatments to school-aged children over the next three years. It could use almost three times this amount if it were to follow World Health Organization guidelines, which include treating many adults; we are not recommending funding to treat adults because we haven’t seen sufficient evidence on the impact of treating adults.

The primary use of this funding, and SCI’s top priority, would be to sustain and expand work in current countries of operation. A smaller portion would be used to expand to up to four additional countries.

Sightsavers (for work on deworming)

Our full review of Sightsavers is here.

Background

Sightsavers (sightsavers.org) is a large organization with multiple program areas that focuses on preventing avoidable blindness and supporting people with impaired vision. Our review focuses on Sightsavers’ work to prevent and treat neglected tropical diseases (NTDs) and – more specifically – advocating for, funding, and monitoring deworming programs. Deworming is a fairly new addition to Sightsavers’ portfolio; in 2011, it began delivering some deworming treatments through NTD programs that had been originally set up to treat other infections.

Sightsavers has shared surveys for some of its past NTD programs that measure whether these programs have reached a large proportion of children targeted. These studies have generally found moderately positive results, but leave us with some remaining questions about the program’s impact. We have seen very limited results from Sightsavers’ deworming programs specifically. For GiveWell-supported programs, Sightsavers has told us it will conduct coverage surveys for each round of deworming; we have reviewed one of those surveys to date.

Important changes in the last 12 months

In 2017, as expected, we learned relatively little about the performance of Sightsavers’ deworming programs, because programs funded with GiveWell-directed funds were at early stages. We did not expect to receive any monitoring results from programs funded with GiveWell-directed funds; however, Sightsavers shared a coverage survey from Guinea with us earlier than expected. The survey found middling coverage results.

We significantly improved our understanding of Sightsavers’ cost per treatment and the baseline prevalence in areas where Sightsavers works (which is used in our cost-effectiveness analysis), though we continue to have lower confidence in our estimates than we do for the deworming organizations that we have recommended for several years.

Funding gap

We believe that Sightsavers’ deworming work is likely to be constrained by funding next year. Sightsavers has provided details of deworming programs that it could fund with additional funding of up to about $6.4 million in 2018 and 2019. Sightsavers appears to have limited prospects for funding these programs from other sources. We believe it is likely that Sightsavers could absorb funding beyond this amount to extend programs to 2020 and/or seek out additional opportunities to fund deworming programs.

Of the $6.4 million, $2.8 million would be used to add deworming to existing NTD programs and $3.7 million would be used to fund NTD programs that would treat several NTDs in addition to schistosomiasis and STH. We will request that Sightsavers prioritize the first set of opportunities, because we believe they will likely be more cost-effective.

Standout charities

In addition to our top charities, we recognize standout charities—organizations that support programs that may be extremely cost-effective and are evidence-backed but for which we have less confidence in their impact than we do for our top charities. We have reviewed their work and feel these groups stand out from the vast majority of organizations we have considered in terms of the evidence base for the program they support, their transparency, and their potential cost-effectiveness. These organizations offer additional giving options for donors who feel highly aligned with their work.

We’ve added one organization to the list this year: Evidence Action’s Dispensers for Safe Water.

We don’t follow standout organizations as closely as we do our top charities. We generally have one or two calls per year with representatives from each group and publish notes on our conversations. We provide brief updates on these charities below.

New addition to the standout list:

• Evidence Action’s Dispensers for Safe Water. The Dispensers for Safe Water program provides chlorine dispensers for decontamination of drinking water to prevent diarrhea and associated deaths of young children. We believe that there is strong evidence that chlorination is biochemically effective at inactivating most diarrhea-causing microorganisms, but weaker evidence on the causal relationship between water chlorination programs and reductions in under-5 diarrhea and death. Our rough cost-effectiveness analysis of Dispensers for Safe Water suggests that the program is in a similar range of cost-effectiveness as unconditional cash transfer programs. Our review of Dispensers for Safe Water is here.

Organizations that have conducted randomized controlled trials of their programs:

• Development Media International (DMI). DMI produces radio and television programming in developing countries that encourages people to adopt improved health practices. It conducted a randomized controlled trial (RCT) of its child survival media campaign in Burkina Faso and has been highly transparent, including sharing preliminary results with us. The results of its RCT were mixed, with a household survey not finding an effect on mortality (it was powered to detect a reduction of 15 percent or more) and data from health facilities finding an increase in facility visits. (The results have not yet been published.) We believe there is a possibility that DMI’s work is highly cost-effective, but we see no solid evidence that this is the case. DMI is conducting an RCT of its family planning radio campaign in Burkina Faso and it is planning work on early child development in Burkina Faso and child survival in Mozambique. It is our understanding that DMI will be constrained by funding in the next year. Our full review of DMI is here and notes from our most recent conversation with DMI are here.
• Living Goods. Living Goods recruits, trains, and manages a network of community health promoters who sell health and household goods door-to-door in Uganda and Kenya and provide basic health counseling. They sell products such as treatments for malaria and diarrhea, fortified foods, water filters, bednets, clean cookstoves, and solar lights. Living Goods completed a RCT of its program and measured a 27 percent reduction in child mortality. Our best guess is that Living Goods’ program is less cost-effective than our top charities, with the possible exception of GiveDirectly. It is conducting a second RCT of its program and results are expected in 2020. Living Goods recently expanded the number of family planning products it offers and is interested in expanding to a third country. Living Goods is scaling up its program and could scale up more quickly with additional funding. Our review of Living Goods is here and notes from our most recent conversation with Living Goods are here.

Organizations working on micronutrient fortification:

We believe that food fortification with certain micronutrients can be a highly effective intervention. For each of these organizations, we believe they may be making a significant difference in the reach and/or quality of micronutrient fortification programs but we have not yet been able to establish clear evidence of their impact. The limited analysis we have done suggests that these programs are likely not significantly more cost-effective than our top charities—if they were, we might put more time into this research or recommend a charity based on less evidence.

• Food Fortification Initiative (FFI). FFI works to reduce micronutrient deficiencies (especially folic acid and iron deficiencies) by doing advocacy and providing assistance to countries as they design and implement flour and rice fortification programs. We have not yet completed a full evidence review of iron and folic acid fortification, but our initial research suggests it may be competitively cost-effective with our other priority programs. Because FFI typically provides support alongside a number of other actors and its activities vary widely among countries, it is difficult to assess the impact of its work. FFI’s recent work includes advocating for legislation to mandate that rice imported to West Africa is fortified with vitamins and minerals. Our full review is here and notes from our most recent conversation are here.
• Global Alliance for Improved Nutrition (GAIN) – Universal Salt Iodization (USI) program. GAIN’s USI program supports national salt iodization programs. We have spent the most time attempting to understand GAIN’s impact in Ethiopia. Overall, we would guess that GAIN’s activities played a role in the increase in access to iodized salt in Ethiopia, but we do not yet have confidence about the extent of GAIN’s impact. GAIN has focused its recent USI work on Tanzania, Mozambique, Ethiopia, and Kenya, which it targeted based on relatively low levels of coverage of iodized salt and strong relationships with stakeholders. It is our understanding that GAIN’s USI work will be constrained by funding in the next year. Our review of GAIN is here and notes from our most recent conversation are here.
• Iodine Global Network (IGN). Like GAIN-USI, IGN supports (via advocacy and technical assistance rather than implementation) salt iodization. IGN is small, and GiveWell-directed funding has made up a large part of its funding in recent years. It expects to have data from before and after its recent work in Madagascar, Lebanon, and possibly Israel by the end of 2018; this data may provide additional evidence of IGN’s impact. It is our understanding that IGN will be constrained by funding in the next year. Our review of IGN is here and notes from our most recent conversation here.
• Project Healthy Children (PHC)/Sanku. PHC/Sanku aims to reduce micronutrient deficiencies by providing assistance to small countries as they design and implement food fortification programs and by enabling fortification among small-scale millers. PHC is scaling up its Sanku project, which equips small millers with a machine that enables them to fortify their flour with micronutrients; we have not done as much formal analysis of Sanku as of PHC’s core work on advocacy and technical assistance to countries to implement fortification. PHC/Sanku expects to be constrained by funding in the future. Our review of PHC/Sanku is here and notes from our more recent conversation are here.

Our research process in 2017

We plan to detail the work we completed this year in a future post as part of our annual review process. A major focus of 2017 was improving our recommendations in future years, in particular through our work on GiveWell Incubation Grants and prioritizing promising programs for further investigation.

Below, we highlight the key research that led to our current charity recommendations. This page describes our overall process.

• Following existing top charities. We followed the progress and plans of each of our 2016 top charities. We had several conversations by phone with each organization, met in person at least once with each top charity (including a three-day visit to Rwanda and the Democratic Republic of the Congo with the END Fund), and reviewed documents they shared with us.
• Identifying new top charities.
  • No Lean Season. We had recommended a series of Incubation Grants to No Lean Season beginning in 2014 and have followed its progress since then. This year, due to the scale at which No Lean Season was operating and the track record it had established, we decided that the No Lean Season program was at a stage of development where we could evaluate it as a potential top charity. In addition to extensive communications with No Lean Season staff over the phone and reviewing documents they shared with us, GiveWell staff spent five days visiting the program in Bangladesh.
  • Helen Keller International’s vitamin A supplementation program. Earlier this year, Research Analyst Chelsea Tabart began reaching out to organizations that might be a fit for our criteria, but with which we had limited or no previous contact with. As a result of that process, we reconnected with Helen Keller International (which we first considered as a potential top charity in 2007) and began to consider its vitamin A supplementation program as a potential top charity. In addition to extensive communications with HKI staff over the phone and reviewing documents they shared with us, GiveWell staff spent three days meeting with HKI staff in Guinea and observing a vitamin A supplementation program.
• Completing intervention reports on obstetric fistula surgery and measles vaccination campaigns; completing interim intervention reports on SMS reminders for vaccination, Sayana® Press (an injectable contraceptive), oral rehydration solution, and antiretroviral therapy for HIV/AIDS; and expanding our interim intervention report on seasonal malaria chemoprevention to a full intervention report.
• Staying up to date on the research for malaria nets, cash transfers, and deworming. We did not find major new research on cash transfers, nets, or deworming that affected our recommendation of GiveDirectly, AMF, or the organizations we recommend for their work on deworming. David Roodman published an in-depth review (parts 1 and 2) of the deworming studies that form the primary basis of our views on the impact of deworming (though much of this work was completed in 2016 and informed our top charity recommendations last year).
• Making extensive updates to our cost-effectiveness model and publishing several updates to the model over the course of the year. We instituted a process to track and report publicly on updates to the model to reduce the possibility of errors and make our process more transparent. This year, staff members have also provided substantially more detail in our cost-effectiveness file about why they have chosen particular inputs.

Giving to GiveWell vs. top charities

GiveWell is currently in a stable financial position. We project that our revenue and our expenses will be approximately equal in the future. However, this projection forecasts some growth in the level of operating support we receive.

In the long term, we seek to have a model where donors who find our research useful contribute to the costs of creating it, while holding us accountable to providing high-quality, easy-to-use recommendations. We retain our “excess assets policy” to ensure that if we fundraise for our own operations beyond a certain level, we will grant the excess to our recommended charities.

We cap the amount of operating support we ask Good Ventures to provide to GiveWell at 20 percent, for reasons described here. We thus ask that donors who use GiveWell’s research consider the following:

• If you have supported GiveWell’s operations in the past, we ask that you maintain your support. Having a strong base of consistent support allows us to make valuable hires when opportunities arise and to minimize staff time spent on fundraising for our operating expenses.
• If you have not supported GiveWell’s operations in the past, we ask that you designate 10 percent of your donation to help fund GiveWell’s operations. This can be done by selecting the option to “Add 10% to help fund GiveWell’s operations” on our credit card donation form or letting us know how you would like to designate your funding when giving another way.

We’re happy to answer questions in the comments below. Please also feel free to reach out directly with any questions.

The post Our top charities for giving season 2017 appeared first on The GiveWell Blog.

In a previous blog post, we described how we use cost-effectiveness analyses when deciding which charities to recommend to donors.

Today, we published a report that discusses how GiveWell and other actors, such as governments and global health organizations, approach one of the most subjective and uncertain inputs into cost-effectiveness analyses: how to morally value different good outcomes.

For example, GiveDirectly, one of GiveWell’s seven top charities, increases recipients’ consumption, while the primary benefit we see from our top charity the Against Malaria Foundation is that it averts the deaths of young children. How can one make a direct comparison between the amount of “good” achieved by each of these charities?

GiveWell does this by assigning quantitative “moral weights” to different outcomes in our cost-effectiveness analyses. As a check on how sensitive our recommendations are to our moral assumptions, we investigated how others typically answer these questions in their cost-effectiveness analyses.

For a full discussion of the findings from our investigation, see our detailed report.

The summary of the report is:

We focus on the following questions:

• Why does GiveWell explicitly include moral weights in our cost-effectiveness analyses, and how do we decide on moral weights?
• Is there a “standard” approach to moral weights in cost-effectiveness analyses? How do other actors, such as governments and the World Health Organization, make these judgments?
• How much would GiveWell’s cost-effectiveness analyses change if we took a “standard” approach to moral weights?

In brief:

• We include moral weights in our cost-effectiveness analyses because they are an important part of any giving decision and we think it is valuable to be transparent about them. The moral weights that drive our cost-effectiveness estimates are based on our staff’s personal values.
• Governments and other prominent actors often use “value of a statistical life” estimates to compare the value of improving health relative to raising incomes. These estimates often imply that a year of healthy life is roughly 2-3x as valuable as a year of doubling someone’s income. However, there is little relevant research to inform such estimates in low- and middle-income country (LMIC) contexts; we understand that how income is valued relative to health may shift when a population is much poorer.
• There does not seem to be a standard approach for comparing the value of life at different ages; the most commonly used framework that we have seen (the disability-adjusted life year framework) explicitly does not provide judgments on this topic. Nevertheless, most other analyses that we have seen assume that averting death during childhood is about 1-2x more valuable than averting death during adulthood.
• Our initial analysis suggests that using relatively “standard” moral weight assumptions (i.e., the assumptions in the previous two bullet points) instead of our staff’s moral weights would not change our overall view of the relative cost-effectiveness of our current top charities. It may affect how we view some interventions in the future, particularly those that disproportionately focus on averting deaths for young children or adults. We plan to include explicit comparisons between staff moral weights and relatively “standard” moral weights in our analyses going forward.

For more detail, see the full report here.

The post How GiveWell and mainstream policymakers compare the “good” achieved by different programs appeared first on The GiveWell Blog.

In a previous blog post, we described how we use cost-effectiveness analyses when deciding which charities to recommend to donors.

Today, we published a report that discusses how GiveWell and other actors, such as governments and global health organizations, approach one of the most subjective and uncertain inputs into cost-effectiveness analyses: how to morally value different good outcomes.

For example, GiveDirectly, one of GiveWell’s seven top charities, increases recipients’ consumption, while the primary benefit we see from our top charity the Against Malaria Foundation is that it averts the deaths of young children. How can one make a direct comparison between the amount of “good” achieved by each of these charities?

GiveWell does this by assigning quantitative “moral weights” to different outcomes in our cost-effectiveness analyses. As a check on how sensitive our recommendations are to our moral assumptions, we investigated how others typically answer these questions in their cost-effectiveness analyses.

For a full discussion of the findings from our investigation, see our detailed report.

The summary of the report is:

We focus on the following questions:

• Why does GiveWell explicitly include moral weights in our cost-effectiveness analyses, and how do we decide on moral weights?
• Is there a “standard” approach to moral weights in cost-effectiveness analyses? How do other actors, such as governments and the World Health Organization, make these judgments?
• How much would GiveWell’s cost-effectiveness analyses change if we took a “standard” approach to moral weights?

In brief:

• We include moral weights in our cost-effectiveness analyses because they are an important part of any giving decision and we think it is valuable to be transparent about them. The moral weights that drive our cost-effectiveness estimates are based on our staff’s personal values.
• Governments and other prominent actors often use “value of a statistical life” estimates to compare the value of improving health relative to raising incomes. These estimates often imply that a year of healthy life is roughly 2-3x as valuable as a year of doubling someone’s income. However, there is little relevant research to inform such estimates in low- and middle-income country (LMIC) contexts; we understand that how income is valued relative to health may shift when a population is much poorer.
• There does not seem to be a standard approach for comparing the value of life at different ages; the most commonly used framework that we have seen (the disability-adjusted life year framework) explicitly does not provide judgments on this topic. Nevertheless, most other analyses that we have seen assume that averting death during childhood is about 1-2x more valuable than averting death during adulthood.
• Our initial analysis suggests that using relatively “standard” moral weight assumptions (i.e., the assumptions in the previous two bullet points) instead of our staff’s moral weights would not change our overall view of the relative cost-effectiveness of our current top charities. It may affect how we view some interventions in the future, particularly those that disproportionately focus on averting deaths for young children or adults. We plan to include explicit comparisons between staff moral weights and relatively “standard” moral weights in our analyses going forward.

For more detail, see the full report here.

The post How GiveWell and mainstream policymakers compare the “good” achieved by different programs appeared first on The GiveWell Blog.

In 2016, we tracked a total of $91.6 million given to our top charities as a direct result of our research.

In addition to this $91.6 million, we also directed $13.3 million to our Incubation Grants program.

A note about this report:

We have yet to complete all of the work necessary to publish our 2016 metrics report. There are a number of reasons the report has been difficult to produce this year. In our view the main factors responsible for this delay were the increasing number and complexity of the data sources used to track donors giving to our recommended charities and competing priorities that required the attention of the staff member who produced the 2016 report. The delay was not a result of a decreased dedication to transparency.

In mid-September, we committed to publishing an update on our key metrics (money moved and web traffic) by October 1st even if we were unable to complete our report by then. We have failed to complete our full report, and, today, are publishing an interim update consistent with our commitment.

Unfortunately, we don’t have an updated estimate about when we’ll publish our full metrics report. It is possible that it will take us several months or more to complete it. We view this as a major failing on our part, and we plan to correct this in the future.

For the purpose of this report, please note:

• We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
• We differentiate between our traditional charity recommendations, the work of the Open Philanthropy Project, and our work aiming to support the development of future GiveWell top charities. GiveWell and Open Philanthropy are now separate legal organizations, but during 2016 Open Philanthropy was part of GiveWell, so we report its grantmaking here.
• More context on the relationship between Good Ventures, Open Philanthropy, and GiveWell can be found here.

Summary of influence:

In 2016, GiveWell influenced charitable giving in several ways. The following table includes (a) donations from donors who cited our research when donating to a third party (or cited a source that recommended our top charities because of our recommendation), (b) donations to GiveWell that we granted to top charities and standout organizations, and (c) grants made on our recommendation, through GiveWell Incubation Grants and the Open Philanthropy Project.

Total money moved:

In 2016, GiveWell tracked $91.6 million in money moved to our recommended charities. Our money moved figure only includes donations that we are confident were influenced by our recommendations. In our full metrics report, we plan to include our best guess of the total funding that was given to our top and standout charities due to our research; the numbers in this blog post include only donations that we could specifically track as being due to our research. The methodology used to generate the numbers in this post was very similar to what is described in our 2015 metrics report, with the exception that, for 2016, we counted more donations through partner organizations such as Giving What We Can. We now believe that we should have included those donations last year.

Open Philanthropy gave an additional $13.3 million to the GiveWell Incubation Grants program, to support the development of future top charities, and GiveWell granted $400,000 in participation grants to organizations that applied for a top charity recommendation, from funding provided by Good Ventures for this purpose. We do not count either of these grant types in our headline money moved figure.

Money moved by charity:

Our seven top charities received the majority of our money moved. Our six standout charities received a total of about $3.1 million.

The post Interim update on GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

In 2016, we tracked a total of $91.6 million given to our top charities as a direct result of our research.

In addition to this $91.6 million, we also directed $13.3 million to our Incubation Grants program.

A note about this report:

We have yet to complete all of the work necessary to publish our 2016 metrics report. There are a number of reasons the report has been difficult to produce this year. In our view the main factors responsible for this delay were the increasing number and complexity of the data sources used to track donors giving to our recommended charities and competing priorities that required the attention of the staff member who produced the 2016 report. The delay was not a result of a decreased dedication to transparency.

In mid-September, we committed to publishing an update on our key metrics (money moved and web traffic) by October 1st even if we were unable to complete our report by then. We have failed to complete our full report, and, today, are publishing an interim update consistent with our commitment.

Unfortunately, we don’t have an updated estimate about when we’ll publish our full metrics report. It is possible that it will take us several months or more to complete it. We view this as a major failing on our part, and we plan to correct this in the future.

For the purpose of this report, please note:

• We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
• We differentiate between our traditional charity recommendations, the work of the Open Philanthropy Project, and our work aiming to support the development of future GiveWell top charities. GiveWell and Open Philanthropy are now separate legal organizations, but during 2016 Open Philanthropy was part of GiveWell, so we report its grantmaking here.
• More context on the relationship between Good Ventures, Open Philanthropy, and GiveWell can be found here.

Summary of influence:

In 2016, GiveWell influenced charitable giving in several ways. The following table includes (a) donations from donors who cited our research when donating to a third party (or cited a source that recommended our top charities because of our recommendation), (b) donations to GiveWell that we granted to top charities and standout organizations, and (c) grants made on our recommendation, through GiveWell Incubation Grants and the Open Philanthropy Project.

Total money moved:

In 2016, GiveWell tracked $91.6 million in money moved to our recommended charities. Our money moved figure only includes donations that we are confident were influenced by our recommendations. In our full metrics report, we plan to include our best guess of the total funding that was given to our top and standout charities due to our research; the numbers in this blog post include only donations that we could specifically track as being due to our research. The methodology used to generate the numbers in this post was very similar to what is described in our 2015 metrics report, with the exception that, for 2016, we counted more donations through partner organizations such as Giving What We Can. We now believe that we should have included those donations last year.

Open Philanthropy gave an additional $13.3 million to the GiveWell Incubation Grants program, to support the development of future top charities, and GiveWell granted $400,000 in participation grants to organizations that applied for a top charity recommendation, from funding provided by Good Ventures for this purpose. We do not count either of these grant types in our headline money moved figure.

Money moved by charity:

Our seven top charities received the majority of our money moved. Our six standout charities received a total of about $3.1 million.

The post Interim update on GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

In 2016, we tracked a total of $91.6 million given to our top charities as a direct result of our research.

In addition to this $91.6 million, we also directed $13.3 million to our Incubation Grants program.

A note about this report:

We have yet to complete all of the work necessary to publish our 2016 metrics report. There are a number of reasons the report has been difficult to produce this year. In our view the main factors responsible for this delay were the increasing number and complexity of the data sources used to track donors giving to our recommended charities and competing priorities that required the attention of the staff member who produced the 2016 report. The delay was not a result of a decreased dedication to transparency.

In mid-September, we committed to publishing an update on our key metrics (money moved and web traffic) by October 1st even if we were unable to complete our report by then. We have failed to complete our full report, and, today, are publishing an interim update consistent with our commitment.

Unfortunately, we don’t have an updated estimate about when we’ll publish our full metrics report. It is possible that it will take us several months or more to complete it. We view this as a major failing on our part, and we plan to correct this in the future.

For the purpose of this report, please note:

• We report on “metrics years” that run from February through January; for example, our 2016 data cover February 1, 2016 through January 31, 2017.
• We differentiate between our traditional charity recommendations, the work of the Open Philanthropy Project, and our work aiming to support the development of future GiveWell top charities. GiveWell and Open Philanthropy are now separate legal organizations, but during 2016 Open Philanthropy was part of GiveWell, so we report its grantmaking here.
• More context on the relationship between Good Ventures, Open Philanthropy, and GiveWell can be found here.

Summary of influence:

In 2016, GiveWell influenced charitable giving in several ways. The following table includes (a) donations from donors who cited our research when donating to a third party (or cited a source that recommended our top charities because of our recommendation), (b) donations to GiveWell that we granted to top charities and standout organizations, and (c) grants made on our recommendation, through GiveWell Incubation Grants and the Open Philanthropy Project.

Total money moved:

In 2016, GiveWell tracked $91.6 million in money moved to our recommended charities. Our money moved figure only includes donations that we are confident were influenced by our recommendations. In our full metrics report, we plan to include our best guess of the total funding that was given to our top and standout charities due to our research; the numbers in this blog post include only donations that we could specifically track as being due to our research. The methodology used to generate the numbers in this post was very similar to what is described in our 2015 metrics report, with the exception that, for 2016, we counted more donations through partner organizations such as Giving What We Can. We now believe that we should have included those donations last year.

Open Philanthropy gave an additional $13.3 million to the GiveWell Incubation Grants program, to support the development of future top charities, and GiveWell granted $400,000 in participation grants to organizations that applied for a top charity recommendation, from funding provided by Good Ventures for this purpose. We do not count either of these grant types in our headline money moved figure.

Money moved by charity:

Our seven top charities received the majority of our money moved. Our six standout charities received a total of about $3.1 million.

The post Interim update on GiveWell’s money moved and web traffic in 2016 appeared first on The GiveWell Blog.

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our June 2017 open thread here.

The post September 2017 open thread appeared first on The GiveWell Blog.

Our goal with hosting quarterly open threads is to give blog readers an opportunity to publicly raise comments or questions about GiveWell or related topics (in the comments section below). As always, you’re also welcome to email us at info@givewell.org or to request a call with GiveWell staff if you have feedback or questions you’d prefer to discuss privately. We’ll try to respond promptly to questions or comments.

You can view our June 2017 open thread here.

The post September 2017 open thread appeared first on The GiveWell Blog.

Many donors who give through GiveWell’s website choose to donate to support “Grants to recommended charities at GiveWell’s discretion,” rather than selecting a specific recommended charity or charities as the target of their gift.

We periodically grant these “discretionary funds” to what we see as the highest-value funding opportunities among our top charities. We last granted discretionary funds in April; then, we granted $4.4 million to the Against Malaria Foundation and $0.5 million to the Deworm the World Initiative.

Since we last allocated funds, we received an additional $1.25 million in discretionary funds that we recently granted out. We also hold roughly $1 million in discretionary funds that we plan to grant out in the next month or two. We plan to grant all of this funding to the Deworm the World Initiative.

Recommendation for donors

We continue to recommend that donors give 100% of their donation to the Against Malaria Foundation (AMF). In other words, although we’re choosing to grant discretionary funds to Deworm the World, we don’t believe that donors who rely on our recommendations should adjust their giving at this time. We explain the rationale for this below.

Summary

This post will discuss:

• Why we believe Deworm the World has a pressing funding need.
• The benefits of granting discretionary funds to Deworm the World today.
• The risks of granting discretionary funds to Deworm the World today.
• Why we’re continuing to recommend that donors give 100% of their donation to AMF.

Deworm the World’s funding gap

Deworm the World recently told us that they have a pressing funding need. We model Deworm the World as the most cost-effective of our top charities (~10x as cost-effective as cash transfers). Last November, we recommended what we estimated was enough funding to make it 80 percent likely that Deworm the World would not be constrained by funding in the next year. (Due to donors following GiveWell’s recommendations, this recommendation generally tracks with what charities receive as a result.) The existence of a funding gap today thus surprised us.

We understand Deworm the World’s funding gap is driven by two primary factors. First, our estimate of Deworm the World’s room for more funding made assumptions about how much funding it would receive from other funders, and so far this year revenue has been lower than projections.

Second, global costs (salaries, office space, travel, etc. for staff based outside of countries where programs operate) were erroneously left out of our cost-effectiveness analysis for Deworm the World last year. Deworm the World did not include these costs in its list of ways it might spend money and we did not recognize that they were not built into the budget.

The existence of this gap, and some time-sensitive considerations discussed below, led us to decide to recommend the current discretionary funds go to Deworm the World. We also considered granting them to AMF, which is our current top recommendation for donors; we believe AMF has significant room for more funding. However, we decided that Deworm the World’s funding need was more pressing. We don’t expect this decision to change the total amount that AMF and Deworm the World each receive as a result of GiveWell’s recommendation in 2017, so the cost of not choosing AMF today is delaying when AMF receives this funding.

Benefits of granting discretionary funds to Deworm the World now

Deworm the World is currently in discussions with the government of Kenya to sign a memorandum of understanding (MOU) to continue its deworming program in Kenya for the next five years. Deworm the World had enough funding committed to the program to fund two years and believes that if it had a larger funding commitment, the government would see the MOU as a higher priority and might speed up the process to finalize the MOU. When we spoke with Deworm the World about additional funding in June and July, of particular concern were the upcoming elections in Kenya on August 8. Because elections tend to take government officials away from other work, Deworm the World feared that delaying the MOU process could delay its deworming work and, in a worst-case scenario, cause the 2017-2018 round of deworming in Kenya to be missed.

Though we understand from Deworm the World that the MOU was not signed prior to the elections, our expectation is that receiving this funding several months earlier than it otherwise would have will decrease the likelihood of Kenya missing a round of deworming.

In addition, Deworm the World may have opportunities this year to fund work in Pakistan and additional work in India (it currently works in only a portion of the states in India). Receiving additional funding now may allow Deworm the World to accelerate its work in those locations.

We have not completed a full room for more funding analysis of Deworm the World since last October and are unsure how much additional room for more funding Deworm the World has.

Risks of granting discretionary funds to Deworm the World now

We believe that granting GiveWell’s discretionary funds to Deworm the World now has the following risks:

• We may later come to believe that Deworm the World needed less than $2.25 million this year.

  We have not seen complete financial information from Deworm the World and we are unsure how much additional funding would be needed to bring them to the level of being 80 percent likely that they won’t be bottlenecked by funding (we generally target our most cost-effective top charities receiving this level of funding).

  Discretionary funds allocated now will reduce our estimation of Deworm the World’s 2017 end-of-year room for more funding, which we incorporate into our annual year-end recommendations. It’s possible, although we think highly unlikely, that once we see Deworm the World’s full financial information (which we expect to before our end-of-year recommendation decisions), we will conclude that the amount we want to recommend to them this year is less than $2.25 million.

  We don’t think this is likely; our best guess is that granting funds to Deworm the World today will accelerate when they receive funds from GiveWell but not change the overall amount of funding they receive in 2017 due to our recommendation.

  It’s also possible that receiving GiveWell discretionary funds now will, for example, allow Deworm the World to lay more groundwork in Pakistan this year and create new opportunities to fund deworming in Pakistan next year, thus increasing our estimate of Deworm the World’s room-for-more funding at the end of the year.

• If we decide at the end of the year that Deworm the World is no longer a top charity, we may not think it should have received $2.25 million a few months before. We think this is highly unlikely.
• Funding for and implementation of Kenya’s deworming program is complex. It involves the government of Kenya, multiple funders (including GiveWell-influenced donors, the Children’s Investment Fund Foundation, and the END Fund), and the Deworm the World Initiative, among others. We’d recommend any donor considering major gifts to Deworm the World before we publish our updated review of Deworm the World in November contact us one-on-one to discuss these considerations.

Recommendation to individuals

We continue to recommend that donors give to our current top recommendation: the Against Malaria Foundation. We are not changing the recommendation to donors because:

1. We are unsure how much additional funding we would like to see Deworm the World receive this year. Making it our new recommendation for donors could drive enough funding in the next few months to overshoot the total Deworm the World could effectively use in the near future.
2. The main benefit of giving to Deworm the World now, rather than at the end of the year, is to accelerate the timeline for the MOU in Kenya and possibly work in Pakistan and India. We anticipate that much of the funding driven by a GiveWell recommendation to donors (e.g., as we currently recommend giving 100% to AMF) would not reach Deworm the World until about the time when we will have completed a full room for more funding analysis of Deworm the World’s work, so we see limited benefit in changing our recommendation now, while we do see the cost noted above.

The post Why we’re allocating discretionary funds to the Deworm the World Initiative appeared first on The GiveWell Blog.