2020 Cost-Effectiveness Analysis Changelog

This page provides details about changes that were made to our cost-effectiveness analysis (CEA) in 2020. For past versions of our CEA, see this page.

Version 2 -- Published November 19, 2020

Link to the cost-effectiveness analysis (CEA) file: 2020 CEA — version 2

For full details on the impact of each of the following changes on our cost-effectiveness estimates, see this spreadsheet.

Change 1: Updated cost per child covered with seasonal malaria chemoprevention (SMC) analysis for Malaria Consortium

Our 2019 cost per child covered with SMC analysis relied on information from Malaria Consortium's programs between 2015 and 2018. The updated 2020 analysis incorporates information on costs and SMC coverage of Malaria Consortium's programs in 2019.

Our overall cost per child covered with SMC estimate decreased from $6.80 in our 2019 analysis to $6.59 in our 2020 analysis.1

Change 2: Revised adjustments for excluded effects

Our CEAs include core models for each top charity and a set of additional adjustments outside of the core models.2 One of the types of adjustments we use are adjustments for excluded effects, which account for our best guesses about the impact of intervention-level factors excluded from our core CEA models.3

We reviewed and made edits to several of our adjustments for excluded effects (details in footnote).4

Change 3: Updated SCI Foundation's cost per person dewormed per year

Our 2020 cost per person dewormed per year analysis for SCI Foundation is our first update since our 2018 analysis. Our 2020 analysis incorporates information from SCI Foundation's 2018-19 financial year.5

Our overall cost per person dewormed estimate for SCI moved from $0.99 in our 2018 analysis to $1.03 in our 2020 analysis.6

Change 4: Updated Sightsavers' cost per child dewormed per year

Our 2019 cost per child dewormed per year analysis relied on information on Sightsavers' deworming programs in 2017 and 2018. The updated 2020 analysis also incorporates information on Sightsavers' deworming programs in 2019.7

Our overall cost per child dewormed estimate moved from $0.91 in our 2019 analysis to $0.92 in our 2020 analysis.8

Change 5: Updated Global Fund's cost-effectiveness estimate

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

Our leverage and funging calculations rely on cost-effectiveness estimates of the programs other organizations or governments would support if they were not contributing to programs supported by our top charities ("counterfactual cost-effectiveness").9 With this change, we have updated our estimate of the counterfactual cost-effectiveness of spending by the Global Fund to Fight AIDS, Tuberculosis, and Malaria (Global Fund), which we use in our leverage and funging adjustment calculations for the Against Malaria Foundation (AMF) and Malaria Consortium.10 Our updated estimate is based on data the Global Fund shared with us from its unfilled quality demand (UQD) registry, which is a list of programming that it would like to fund but does not have the resources to when it makes its initial allocations.11 We do not have permission to make the data the Global Fund shared with us public, but our calculations based on that data are available in this spreadsheet.

Change 6: Removed "User inputs" sheet

Our past CEAs have used medians of inputs from members of our research staff as default values for some parameters, including discount rates and our replicability adjustment for deworming.12 For each of these parameters, one or two GiveWell staff were designated "parameter owners," responsible for conducting research on the parameter and suggesting input choices.13

We noticed in our most recent CEA that most individual GiveWell staff members were following the input recommendations of the parameter owners.14 We expect that the benefits of continuing to ask individual GiveWell staff members to update their inputs for these parameters are limited, so with this change we have removed the "User inputs" sheet.15

In most cases, we have chosen the previous parameter owner's value from the 2020 GiveWell cost-effectiveness analysis — version 1 as the default value in our CEAs for each top charity (details in footnote).16

Change 7: Corrected an error in our insecticide resistance adjustment for the Against Malaria Foundation (AMF)

Our CEA for AMF includes an adjustment for the effect of insecticide resistance on the impact of long-lasting insecticide-treated net (LLIN) distributions.17 We noticed that our 2019 insecticide resistance spreadsheet included an adjustment to account for expected increases in insecticide resistance between the time that insecticide resistance studies had been conducted and the current year.18 We now believe it is more appropriate to adjust for expected increases in insecticide resistance between the date of insecticide resistance studies and the date in which we expect bed nets funded with donations this year would actually be delivered. We have updated our 2020 insecticide resistance spreadsheet and our CEA accordingly.19

Change 8: Updated Malaria Consortium's marginal spending by country

Our CEAs include both country-specific and overall cost-effectiveness estimates. Overall cost-effectiveness estimates are based on the proportion of marginal spending we expect each organization to spend in each country.20

We have updated our expectations of the proportion of marginal spending Malaria Consortium would allocate to each country it operates in, based on our 2020 room for more funding analysis for Malaria Consortium (before accounting for our recommendations to Open Philanthropy).21

Change 9: Updated Helen Keller International's marginal spending by country

We have updated our expectations of the proportion of marginal spending Helen Keller International (Helen Keller) would allocate to each country it operates in, based on our 2020 room for more funding analysis for Helen Keller (before accounting for our recommendations to Open Philanthropy).22

Change 10: Updated GiveWell's moral weights

GiveWell uses moral weights to make cost-effectiveness comparisons between interventions achieving different types of outcomes (e.g. averting the death of a child from malaria vs. doubling consumption for a year for a very poor individual). See this page for more discussion on how GiveWell uses moral weights.

Before November 2019, we asked GiveWell staff members to set their moral weights individually.23 In 2019, we began using a set of moral weights based in part on a survey conducted by IDinsight of individuals broadly representative of the populations our top charities aim to serve. The survey included questions about the relative value these individuals place on different good outcomes.

In 2020, we aggregated information from GiveWell staff's past individual-level moral weights, IDinsight's survey, the disability-adjusted life year (DALY) framework, and a survey of a set of GiveWell donors we conducted to create a new set of moral weights. We describe our process in more detail in this document.

Change 11: Updated worm burden adjustments for SCI Foundation

Our recommendation of mass deworming programs primarily relies on a series of follow-up studies to the experiment described in Miguel and Kremer 2004.24

Our cost-effectiveness analyses for deworming programs use adjustments to account for differences in the prevalence and intensity of worm infections between the population studied in Miguel and Kremer 2004 and populations reached by our top charities that implement deworming programs. Our worm burden adjustment model was previously based on average intensities of worm infections.25 Our updated worm burden adjustment model is instead based on the prevalence of moderate and heavy intensity worm infections. The updated worm burden adjustment model is available in this spreadsheet, and we describe our model in more detail in this document.

With this change, we have applied our updated worm burden adjustment model to our CEA for SCI Foundation.26

Change 12: Updated marginal spending by country, added new countries, added worm burden adjustments for new countries, and updated proportion of deworming going to children for SCI Foundation

We have updated our expectations of the proportion of marginal spending SCI Foundation would allocate to each country it operates in, based on our 2020 room for more funding analysis for SCI Foundation (before accounting for our recommendations to Open Philanthropy).27 We now expect that SCI Foundation will spend marginal funding in some countries that were not included in the CEA preceding this change (including Sudan, Niger, Mauritania, and Burundi).28 We have also applied our updated worm burden adjustment model (described above) to these countries.29

Our CEA for SCI Foundation also includes an adjustment to account for the proportion of deworming treatments that are delivered to children by country. With this change, we have updated this input based on our 2020 cost per person dewormed per year analysis for SCI Foundation.30

Change 13: Updated worm burden adjustments for the END Fund

We applied our updated worm burden adjustment model, described above, to the END Fund's deworming programs.31

Change 14: Updated worm burden adjustments for Deworm the World

We applied our updated worm burden adjustment model, described above, to Deworm the World.32

Change 15: Updated marginal spending by country and removed countries for Deworm the World

We have updated our expectations of the proportion of marginal spending Deworm the World would allocate by country, based on our 2020 room for more funding analysis for Deworm the World (before accounting for our recommendations to Open Philanthropy).33

Change 16: Updated worm burden adjustments for Sightsavers

We applied our updated worm burden adjustment model, described above, to Sightsavers' deworming programs.34

Change 17: Updated marginal spending by country, added new countries, and added worm burden adjustments for new countries for Sightsavers

We have updated our expectations of the proportion of marginal spending Sightsavers would allocate by country, based on our 2020 room for more funding analysis for Sightsavers (before accounting for our recommendations to Open Philanthropy).35 We now expect that Sightsavers will spend marginal funding in some countries that were not included in the version of the CEA preceding this change (including Senegal and Chad).36 We have also applied our updated worm burden adjustment model (described above) to these countries.37

Change 18: Updated leverage and funging adjustments for AMF and Malaria Consortium and removed countries for AMF

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

As part of our leverage and funging adjustment calculations, we estimate the probability of several scenarios that might occur in absence of philanthropic support for a particular program (e.g. "government costs would replace philanthropic costs" or "distributions would go unfunded").38 We have revised most of our probability estimates used in leverage and funging calculations for Malaria Consortium and AMF's programs.39 Our reasoning for our updated probability estimates is described in this document.

We also removed several countries in our CEA of AMF in which we did not expect AMF to spend marginal funding.40

Change 19: Updated our cost per supplement analysis for Helen Keller International

Our 2019 cost per supplement analysis for Helen Keller International relied on information on Helen Keller's past spending and supplements delivered in 2018 and on projected spending and supplements delivered for 2019.41

The 2020 analysis incorporates information on Helen Keller's actual spending and supplements delivered in 2019 (rather than projections).42 More information on our 2020 analysis of Helen Keller's cost per supplement is available in this document.

Our overall cost per supplement estimate for Helen Keller moved from $1.23 in our 2019 analysis to $1.10 in our 2020 analysis.43

Change 20: Updated Deworm the World's cost per child dewormed per year

Our 2019 cost per child dewormed per year analysis for Deworm the World relied on information on costs and treatments delivered in Deworm the World's programs between 2014 and 2018.44

The updated 2020 analysis also incorporates information on costs and treatments in 2019 in Pakistan and Nigeria (but not India or Kenya).45

Change 21: Updated leverage and funging adjustments for Helen Keller International

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

Our previous calculations for Helen Keller only accounted for leverage and funging of domestic governments.46 We have updated our calculations to account for Helen Keller's spending also leveraging or funging with spending on vitamin A supplementation (VAS) programs by other philanthropic actors.47

We have also revised the probability estimates used in leverage and funging calculations for Helen Keller's programs.48

Change 22: Updated downside adjustments

Our CEAs include core models for each top charity and a set of additional adjustments outside of the core models.49 One of the types of adjustments we use are downside adjustments, which account for the risk of "wastage" (e.g. double treatment with vitamin A supplements or deworming tablets), quality of monitoring and evaluation, and confidence in funds being used for their intended purpose.50

We reviewed and made edits to several of our downside adjustments for AMF, Malaria Consortium, and Helen Keller International.51

Change 23: Added additional adjustment for baseline mortality rate and vitamin A deficiency (VAD) prevalence estimates in Kenya for Helen Keller International

For our CEA of Helen Keller in Kenya, we added an additional downward adjustment to our cost-effectiveness estimate.52 This downward adjustment is intended to account for two factors: relatively low baseline mortality rates in Kenya (compared to mortality rates among populations studied in trials of VAS) and a micronutrient survey in Kenya finding considerably lower VAD prevalence than our primary data source (Global Burden of Disease 2017).53

Change 24: Added adjustment for treatment ingestion for Malaria Consortium

Our understanding is that Malaria Consortium's coverage surveys are designed to assess whether children received the first day of SMC, but that they are not designed to measure the degree to which those drugs are fully ingested by program participants. (Vomiting may be a side effect of SMC for some children.)54 We have added a rough adjustment for "Failure to ingest first day of treatment of SMC" to our CEA for Malaria Consortium.55

Change 25: Updated AMF's marginal spending by country

We have updated our expectations of the proportion of marginal spending AMF would allocate to each country it operates in, based on our 2020 room for more funding analysis for AMF (before accounting for our recommendations to Open Philanthropy).56

Change 26: Added November 2020 recommendations to Open Philanthropy and allocation of Q3 2020 Maximum Impact Fund

We added a "List of funding gaps" sheet, which includes our November 2020 recommendations to Open Philanthropy for allocating $70 million across our top charities and standout charities, along with an allocation of $3.8 million from our Maximum Impact Fund from the third quarter of 2020. This version of our CEA is available here.

Change 27: Updated list of funding gaps

We updated the "List of funding gaps" sheet after taking into account our November 2020 recommendations to Open Philanthropy and our allocation from our Maximum Impact Fund from the third quarter of 2020 (see change 26 above). This version of our CEA is available here.

Version 1 — Published September 11, 2020

Link to the cost-effectiveness analysis (CEA) file: 2020 CEA — Version 1

For the impact of each of the following changes on our cost-effectiveness estimates, see this spreadsheet.

Change 1: Corrected errors in our cost per seasonal malaria chemoprevention (SMC) treatment estimates for Malaria Consortium

We found and corrected two errors in our 2019 cost per SMC treatment estimate for Malaria Consortium.57 The corrections led to our cost per SMC treatment estimate for Chad changing from $8.77 to $8.17 and our estimate for Nigeria changing from $7.70 to $7.79.58

Change 2: Updated data on population distributions by age for our CEA of the Against Malaria Foundation (AMF)

Our CEA for AMF relies on estimates of the distribution of populations by age for our calculations of "Mortality reduction in children under 5," "Mortality reduction in individuals 5+ years old," and "Income increases — ages 14 and under."59 We previously used multiple sources for data on population distributions by age.60

For consistency with our CEA of Malaria Consortium's SMC program, we now use data from the Institute for Health Metrics and Evaluation’s Global Burden of Disease Study 2017 Population Estimates for all countries in our CEA for AMF.61

Change 3: Corrected an error in our leverage and funging adjustments for Helen Keller International (HKI)

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. Our leverage and funging adjustments are intended to account for the impact of these spending changes. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

We noticed and corrected a formula error in our leverage and funging adjustment calculations in our CEA for HKI's vitamin A supplementation program:

Change 4: Corrected an error in our estimates of AMF's marginal spending by country

We noticed that the estimates of the proportion of marginal donations that we expect to be allocated to each country in our CEA for AMF did not match the values we calculated in our 2019 room for more funding analysis for AMF. We have corrected the error.62

Change 5: Updated our leverage and funging calculations for AMF

Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

Our leverage and funging adjustments for AMF are calculated on a non-public spreadsheet.63 We made the same changes described in Change 2 and Change 4 above in our AMF leverage and funging spreadsheet.

Change 6: Corrected an error in our cost breakdown in our CEA of HKI

We noticed that some of the values in our cost breakdown in our CEA of HKI (the proportion of total costs covered by HKI, other philanthropic actors, and domestic governments) did not match the values calculated in our 2019 cost per supplement analysis for HKI. We have corrected the error.64

Change 7: Corrected errors in our leverage and funging adjustments and country weighting methods

We noticed and corrected a set of errors related to the relative weights countries receive in our overall cost-effectiveness estimates. These errors were caused by mismatches in spending totals between the main section of each CEA and the leverage and funging sections.

Background

Our top charities often work in partnership with other organizations or governments.65 Our top charities' spending may lead other organizations or governments to spend more on programs supported by our top charities than they otherwise would have ("leverage") or less than they otherwise would have ("funging"). Our leverage and funging adjustments are intended to account for the impact of these spending changes. For a full introduction to GiveWell's views on leverage and funging, see this blog post.

Our CEA also includes country-specific cost-effectiveness and overall cost-effectiveness estimates for each top charity.66 Overall cost-effectiveness estimates account for the percentage of marginal donations we expect each top charity to allocate to programs in each country. For AMF in our 2020 CEA — version 1, this means that our cost-effectiveness estimate for the Democratic Republic of the Congo (where we expect AMF to allocate 68% of marginal donations) has more influence on our overall cost-effectiveness estimate than Guinea (where we expect AMF to allocate 7% of marginal donations).

Errors we identified

We noticed mismatches in our 2019 CEA — version 6 between the spending totals reported in the core CEA models and in the leverage and funging sections. For Malaria Consortium, for example, we list the (arbitrary) total spending on the program in the main CEA section at $100,000. But in the leverage and funging section, we state that the overall spending total is ~$110,000. These mismatches lead to incorrect weighting by country in our overall cost-effectiveness estimates in cases where our top charity does not cover the same proportion of total costs in each country (details in footnote).67

These types of errors applied to all of our top charities (excluding GiveDirectly, for which we do not estimate country-specific cost-effectiveness).68

How we corrected the errors

We have addressed these errors with the following changes:

  • We have added a "Total amount spent on the program by all contributors" row to the core model of our CEAs for each top charity (in order to match the spending total listed in the leverage and funging section).69
  • We have edited all calculations of "units of value per $100,000 donation" to "units of value per dollar spent."
    • We needed to make this change because we are no longer assuming that $100,000 is spent in total on each program in the core CEA model for each top charity. We instead assume that our top charity is spending $100,000, and that other actors spend proportionally to their contributions to total costs.70

Change 8: Corrected an error in our cost per net estimates for AMF

We noticed and corrected an error in our cost per net analysis spreadsheet for AMF. This spreadsheet is not public because we do not have permission to publish country-specific cost estimates we received from the Global Fund to Fight AIDS, Tuberculosis, and Malaria, which our calculations rely on. See our review of AMF for more details.

Correcting the error led to slight changes (less than $0.01) in our cost per net estimates for Papua New Guinea and Nigeria.

Change 9: Corrected an error in importing data for our CEA of AMF

Our CEA for AMF relies on data on malaria prevalence and mortality by country from the Institute for Health Metrics and Evaluation's Global Burden of Disease (GBD). We noticed and corrected an error in importing data from GBD for Nigeria in our CEA of AMF.71

Change 10: Reordered adjustment calculations

Our CEAs include a core model for each top charity and three additional types of adjustments:72

  • Downside adjustments: Adjustments to account for the risk of "wastage" (e.g., double treatment with vitamin A supplements or deworming tablets), quality of monitoring and evaluation, and confidence in funds being used for their intended purpose.73
  • Adjustments for excluded effects: Adjustments to account for our best guesses about the impact of intervention-level factors excluded from our core CEA model.74
  • Leverage and funging adjustments: Our top charities' spending may lead other organizations or governments to spend more ("leverage") or less ("funging") on programs implemented by our top charities than they otherwise would have. For a full introduction to our approach to leverage and funging adjustments, see this blog post.

We previously made adjustments for leverage and funging first, followed by downside adjustments and adjustments for excluded effects.75 Following this change, we implement downside adjustments first, followed by adjustments for excluded effects and leverage and funging adjustments.76

We reordered these calculations in order to more easily calculate "cost per life saved" estimates in our CEAs of AMF, HKI, and Malaria Consortium (details in footnote).77

Change 11: Updated development effects estimate for vitamin A supplementation programs

Some health programs for children may positively impact development, leading to lasting increases in their productivity and earnings in adulthood ("development effects").78

Our updated development effects estimate for HKI's vitamin A supplementation programs is based on a comparison of the evidence base for factors that may be correlated with development effects (such as impacts on cognition, child growth, and anemia) for both seasonal malaria chemoprevention (SMC) and vitamin A supplementation. See this document for more details (the method we previously used to estimate development effects for vitamin A supplementation programs is discussed here.)79

Change 12: Updated external validity adjustment for vitamin A supplementation programs

We use an external validity adjustment in our CEA for HKI's vitamin A supplementation program to account for differences between the populations studied in randomized controlled trials (RCTs) of vitamin A supplementation programs and populations reached by HKI's programs today.80 Our 2020 external validity adjustment calculations are in this spreadsheet (our previous version from 2019 is available here.)

Our main updates in 2020 to our external validity adjustment calculations for HKI are as follows:

  • Our 2019 external validity spreadsheet compared cause-specific child mortality rates between populations in 1990 in countries where HKI works and populations in those countries today.81 Our 2020 analysis instead compares cause-specific mortality rates between populations studied in RCTs of vitamin A supplementation included in Imdad et al. 2017, a Cochrane Systematic Review, and targeted populations in countries where HKI works today, which we believe is a more appropriate adjustment.82
  • Our 2019 external validity spreadsheet used cause-specific mortality data for children under five years of age.83 We updated our 2020 external validity spreadsheet to use mortality data for children aged 6 months to 59 months, the target population for vitamin A supplementation programs.84
  • Our 2019 external validity calculations included adjustments for both "changes in VAD prevalence" and "changes in vitamin A-susceptible child mortality."85 In 2020, we added a rough additional adjustment to account for possible non-independence of these two factors.86

Compare the archived version of our CEA preceding this change here to the version following this change here.

Change 13: Updated AMF's marginal spending by country

We updated our estimates of the distribution of AMF's marginal spending across countries.87 Our updates were based on our 2020 room for more funding analysis for AMF.88

Change 14: Updated AMF's cost per net

In 2019 and 2020, we updated our analysis of AMF's cost per net but did not publish an updated version of the spreadsheet with the data we used and our calculations because we have not received permission to publish country-specific cost estimates we received from the Global Fund to Fight AIDS, Tuberculosis, and Malaria, which our calculations rely on.89

In 2019, our overall cost per net estimate for AMF was $4.59. Following updates in 2020, our overall cost per net estimate is $4.95. For updated overall and country-level cost per net estimates, see the version of our CEA following this change here.

As part of our 2020 cost per net update for AMF, we also updated our estimates of the proportion of nets AMF plans to purchase that incorporate piperonyl butoxide (PBO), which impacted the insecticide resistance adjustment in our CEA.90

Change 15: Updated our estimate of equivalent coverage years per net

Our CEA for AMF relies on evidence from randomized controlled trials (RCTs) using conventionally treated nets (CTNs) to calculate the effectiveness of nets at averting child mortality.91 However, the CTNs used in these trials differ in key ways from the long-lasting insecticide-treated nets (LLINs) used today.92

Using field durability monitoring data, we've evaluated the physical survival of today's LLINs and roughly compared it to the expected decay of CTNs in terms of equivalent coverage years.93 "Equivalent coverage years" refers to the average duration for which we estimate an LLIN from an AMF distribution offers protection equivalent to conventionally-treated nets (CTNs) re-treated every 6 mos as used in RCTs, over the 36-month interval between net distributions. Our current best guess is that an LLIN from an AMF distribution confers 2.11 equivalent coverage years.94 Our full report on estimating equivalent coverage years for LLINs is available here.

Prior to our analysis this year, we estimated the lifespan of nets using the decay model described on this page.

Change 16: Updated malaria mortality and malaria case incidence estimates for AMF

Our assessment of the effectiveness of insecticide-treated net distributions relies heavily on Lengeler 2004, a Cochrane Systematic Review of randomized controlled trials (RCTs), as discussed in our intervention report here.

In 2020, we reviewed Pryce, Richardson, and Lengeler 2018, an updated version of Lengeler 2004.95 Our review of Pryce, Richardson, and Lengeler 2018 led us to update our malaria mortality and malaria case incidence estimates in our CEA of AMF.96

Change 17: Corrected an error in our adjustments for excluded effects for HKI

Our CEAs for each top charity include core models and a set of additional adjustments outside the core models (see above).

We noticed an inconsistency in our adjustments for excluded effects—we included an adjustment for "investment of income increases" for other organizations, but did not include the adjustment for HKI. We have now included this adjustment for HKI as well.97

  • 1

    See the "Summary" sheets of our 2019 cost per child covered with SMC analysis and our 2020 analysis for overall and country-specific estimates.

  • 2

    See this changelog entry for a description of all three types of additional adjustments.

  • 3

    See the "Inclusion/Exclusion" sheet in our 2020 CEA — version 2 for a list of these adjustments.

  • 4
    • Compare highlighted portions of the "Inclusion/Exclusion" sheet in the CEA version following this change here to the version preceding this change here.
    • We removed the adjustment for "Difference in net types" for the Against Malaria Foundation (AMF). See this adjustment before it was removed here. This effect is no longer excluded from our core model—in change 15 of our 2020 CEA — version 1, we began incorporating information on the brands of nets AMF purchases in our estimate of the number of equivalent coverage-years per net.
    • We removed the adjustment for "other NTD prevention" from SCI Foundation and Deworm the World (see the adjustment here). When we created this adjustment in 2018, our original intention was to only apply it to Sightsavers and END Fund. Both Sightsavers and END Fund support multiple types of neglected tropical disease programs. Beginning with 2019 GiveWell cost-effectiveness analysis — version 6, we mistakenly began applying this adjustment to Deworm the World and SCI Foundation as well. See "Inclusion/exclusion — Deworming" sections on the "Deworming - DtW" and "Deworming - SCI" sheets in 2019 GiveWell cost-effectiveness analysis — version 6.
    • We removed the "Spillover effects prevent child deaths <3 months, as well as adult deaths" adjustment for Malaria Consortium, which was based on the values GiveWell placed on averting deaths at different ages (see here). This adjustment had not been updated with GiveWell's moral weights updates since 2018 (see here and here). Our previous version of this adjustment made a very small (less than one percent) difference in our cost-effectiveness estimate for Malaria Consortium. We have not prioritized updating this adjustment based on our updated moral weights because we do not expect that it would make a meaningful difference in our cost-effectiveness estimates for Malaria Consortium.
    • We updated our effect size estimate for "Short-term consequences of reduced infectious disease morbidity" for Helen Keller International (Helen Keller) from 35% to 15% (see original adjustment here and new adjustment here). Our updated judgment is that the size of morbidity effects for Helen Keller's programs are likely to be similar to the size of morbidity effects for AMF's and Malaria Consortium's programs (also set at 15%, see the "Malaria morbidity" and "Short-term anemia effects" rows of the "AMF" and "SMC" sections here).
    • In addition to estimating our best guess at the effect size of excluded effects on charities' cost-effectiveness, we also rate each excluded effect from zero to three on three criteria: "Can it be objectively justified?", "Ease of modeling," and "Consistency." See these columns. Our ratings across these criteria impact the weight that each effect receives in our overall adjustment for excluded effects for each charity. See these columns.

      We have updated our subjective ratings for these three criteria for several adjustments. Compare highlighted portions of the "Inclusion/Exclusion" sheet in the CEA version following this change here to the version preceding this change here:
      • We changed our "Consistency" rating for "Malaria adult mortality" for AMF from two to three. "Malaria adult mortality" is also included (and rated three for consistency) for Malaria Consortium.
      • We changed our "Consistency" rating for "Investment of income increases" from three to two for Malaria Consortium, to match the consistency ratings used for "Investment of income increases" for other organizations.
      • We changed our "Can it be objectively justified?" rating for "Vision benefits" for Helen Keller from one to two. This is only an updated subjective best guess at this rating—we have not conducted additional research on this topic to inform this judgment.

  • 5

    See "Per-year analysis" sheet in 2020 cost per person dewormed per year estimate for SCI Foundation.

  • 6

    Compare the "Summary" sheets of our 2020 cost per person dewormed per year estimate for SCI Foundation and our 2018 analysis.

  • 7

    See our 2020 analysis of Sightsavers' cost per child dewormed per year, "Sightsavers' costs by country and year" sheet.

  • 8

    See "Summary" sheets in our 2019 cost per child dewormed per year analysis and our 2020 analysis.

  • 9

    See the "Counterfactual non-philanthropic" sheet in the version of the CEA following this change here.

  • 10

    Compare the "Counterfactual non-philanthropic," "AMF," and "Malaria Consortium" sheets in the version of the CEA preceding this change here to the version following this change here.

  • 11

    "Countries are encouraged to include with their funding application to the Global Fund a request to fund priority programs that should be fully funded but cannot be because of limited resources. This 'prioritized above allocation request' is reviewed by the Technical Review Panel. Strategically focused and technically sound interventions are then registered as 'unfunded quality demand.'" Global Fund, Register of Unfunded Quality Demand for 2020-2022 Funding Cycle.

  • 12

    See 2020 GiveWell cost-effectiveness analysis — version 1, "User inputs" sheet.

  • 13

    See cell notes in 2020 GiveWell cost-effectiveness analysis — version 1, "User inputs" sheet.

  • 14

    See cell notes in 2020 GiveWell cost-effectiveness analysis — version 1, "User inputs" sheet.

  • 15

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 16
    • In the 2020 GiveWell cost-effectiveness analysis — version 1, five parameters were listed on the "User Inputs" sheet: "Discount rate," "Replicability adjustment for deworming," "Percentage of transfers invested" (for cash transfers), "Return on investment" (for cash transfers), and "Duration of investment benefit (in years)" (for cash transfers).
    • In the version of the CEA following this change, we use a discount rate of 4%. See the "Moral weights and discount rate" sheet here. We have described our reasoning for choosing this discount rate in this document. In the version of our CEA preceding this change, we used a median of staff inputs for discount rates, which resulted in a very slightly different rate of 3.995% (see this cell). This change resulted in small changes in some of our cost-effectiveness estimates following the removal of our "User inputs" sheet (see here).
    • For each of the four remaining parameters on the "User inputs" sheet, we have chosen the previous parameter owner's value as the default in our CEAs, which was also equal to the median of staff inputs. Compare the "User inputs" sheet in the version of the CEA preceding this change here to the version of the CEA following this change ("Replicability adjustment for deworming" input here; "Percentage of transfers invested," "Return on investment," and "Duration of investment benefits (in years)" inputs here).

  • 17

    See the version of our CEA following this change here.

  • 18

    See these rows of our 2019 insecticide resistance spreadsheet.

  • 19
    • See these rows in our 2020 insecticide resistance spreadsheet.
    • See the version of our CEA following this change here.

  • 20
    • For example, see the "Funding allocation" section of the "Malaria Consortium" sheet in the version of the CEA following this change here.
    • Our overall cost-effectiveness estimates are based on sums of outcomes estimated in each country. See this row for our calculations of the number of malaria-attributable deaths averted in the treated population for Malaria Consortium, for example.

  • 21

    Compare the "Funding allocation" section of the "Malaria Consortium" sheet in the version of the CEA preceding this change here to the version following this change here.

  • 22
    • See above for a description of how our estimates of organizations' marginal spending by country impacts our overall cost-effectiveness estimates.
    • Compare the "Funding allocation" section of the "HKI" sheet in the version of the CEA preceding this change here to the version following this change here.

  • 23
    • See this section of our page on moral weights.
    • Versions of our cost-effectiveness analyses published before November 2019 are available on this page.

  • 24

    For further discussion, see:

  • 25
    • Our previous worm intensity adjustment workbook is available here.
    • Adjustments calculated in that workbook were used in the 2020 GiveWell cost-effectiveness analysis -- version 1. See the "Worm intensity adjustment" row on the "Deworm the World," "END Fund," "SCI Foundation," and "Sightsavers" sheets.

  • 26

    Compare the version of our CEA preceding this change here to the version following this change here.

  • 27
    • See above for a description of how our estimates of organizations' marginal spending by country impacts our overall cost-effectiveness estimates.
    • Compare the version of the CEA preceding this change here to the version following this change here.

  • 28

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 29

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 30
    • Compare the version of the CEA preceding this change here to the version following this change here.
    • Our spreadsheet calculating the proportion of children receiving deworming treatments by country, based on information in our 2020 cost per person dewormed per year analysis for SCI Foundation, is available here.
    • Note that we include this adjustment because "development effects" are the primary benefit of mass deworming programs in our cost-effectiveness model, which we only expect to occur in children. See this section of our intervention report on mass deworming programs for more details.

  • 31

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 32

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 33
    • See above for a description of how our estimates of organizations' marginal spending by country impacts our overall cost-effectiveness estimates.
    • Compare the version of the CEA preceding this change here to the version following this change here.
    • Note that we have removed states in India from our CEA of Deworm the World with this change (see links in bullet point above) because we do not currently expect Deworm the World to spend marginal funds it receives for work in 2021-2023 in India (see our 2020 room for more funding analysis for Deworm the World, "RFMF projections" sheet).

  • 34

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 35
    • See above for a description of how our estimates of organizations' marginal spending by country impacts our overall cost-effectiveness estimates.
    • Compare the version of the CEA preceding this change here to the version following this change here.

  • 36

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 37

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 38

    For example, see the leverage and funging section of our CEA for Malaria Consortium here.

  • 39

    Compare the leverage and funging sections of the "AMF" and "Malaria Consortium" sheets in the version of the CEA preceding this change here to the version following this change here.

  • 40

    We removed Ghana, Papua New Guinea (PNG), Zambia, and Nigeria. In the version of the CEA preceding this change, the "percentage of funding to be allocated to each country with marginal donations" for each of these countries was set at 0%. The version of the CEA following this change is available here.

  • 41

    See our 2019 cost per supplement analysis for Helen Keller, "Cost per supplement Guinea 2018-2019," "Cost per supplement Mali 2018-2019," and "Cost per supplement Burkina Faso 2018-2019" sheets.

  • 42

    See our 2020 cost per supplement analysis for Helen Keller International, "Cost per supplement Guinea 2018-2019," "Cost per supplement Mali 2018-2019," "Cost per supplement Niger 2018-2019, and "Cost per supplement Burkina Faso 2018-2019" sheets.

  • 43

    See "Summary" sheets in our 2019 cost per supplement analysis for Helen Keller and our 2020 cost per supplement analysis for Helen Keller.

  • 44

    See our 2019 cost per child dewormed per year analysis for Deworm the World, "Costs by treatment round" sheet.

  • 45
    • See our 2020 cost per child dewormed analysis for Deworm the World, "Costs by treatment round" sheet.
    • We have not incorporated information from deworming rounds in 2019 in India or Kenya into our cost per child dewormed estimates. For both countries, we have already analyzed several years of cost and treatment data (see link above). We do not expect that the additional GiveWell staff time required to analyze data from India or Kenya in 2019 would result in substantially different cost per child treated per year estimates.

  • 46

    See the leverage and funging section of the "HKI" sheet in the version of the CEA preceding this change here.

  • 47
    • See the leverage and funging section of the "HKI" sheet in the version of the CEA following this change here.
    • In order to calculate leverage and funging adjustments for other philanthropic actors, we use an estimate of the cost-effectiveness of the programs those actors would spend funds on if they were not spending funding on VAS. As a rough best guess, we assume that the counterfactual cost-effectiveness of these actors is the same as our estimate of the cost-effectiveness of spending on health programs by domestic governments (see here).

  • 48
    • Compare the version of the Helen Keller CEA preceding this change here with the version following this change here.
    • We provide our reasoning for choices in our probability estimates for Helen Keller in this document.

  • 49

    See this changelog entry for a description of all three types of additional adjustments.

  • 50

    For example, see here for our downside adjustments for Malaria Consortium.

  • 51
    • Compare highlighted rows and cell notes in the version of the CEA following this change for AMF here to the version preceding this change here.
    • Compare highlighted rows and cell notes in the version of the CEA following this change for Malaria Consortium here to the version preceding this change here.
    • Compare highlighted rows and cell notes in the version of the CEA following this change for Helen Keller here to the version preceding this change here.

  • 52

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 53

    For more details, see this document.

  • 54
    • Christian Rassi, SMC programme director, Malaria Consortium, email to GiveWell, September 28, 2020 (unpublished)
    • For more information on Malaria Consortium's coverage surveys, see this section of our review.

  • 55

    Compare the version of the CEA preceding this change here to the version following this change here.

  • 56
    • See above for a description of how our estimates of organizations' marginal spending by country impacts our overall cost-effectiveness estimates.
    • Compare the "Funding allocation" section of the "AMF" sheet in the version of the CEA preceding this change here to the version following this change here.

  • 57
    • One error was in the cell references in the formula used in this cell.
    • The second error was in the cell references in the formula used in these cells.

  • 58

    Compare our 2019 GiveWell Cost-Effectiveness Analysis — Version 6 (public), "SMC" sheet, Cost per equivalent child treated with all 4 cycles of SMC for Chad and Nigeria,
    to 2020 GiveWell cost-effectiveness analysis — version 0 [change 1 archive], "Malaria Consortium" sheet, Cost per equivalent child treated with all 4 cycles of SMC for Chad and Nigeria.

  • 59

    See 2020 GiveWell cost-effectiveness analysis — version 0 [change 2 archive], "AMF" sheet.

  • 60

  • 61

    See 2020 GiveWell cost-effectiveness analysis — version 0 [change 2 archive], "AMF" sheet, "Age distribution of net coverage" section and "Malaria Consortium" sheet, "Income increases — ages 14 and under" section.

  • 62
    • Our calculations of the proportion of marginal donations that we expect to be allocated to each country for AMF are calculated in our 2019 room for more funding analysis here.
    • Compare the archived version of our CEA preceding this change here to the version following this change here.

  • 63

    We have calculated our leverage and funging adjustments for AMF on a non-public spreadsheet because we do not have permission to publish country-specific cost estimates we received from the Global Fund to Fight AIDS, Tuberculosis, and Malaria, which our calculations rely on. See our review of AMF for more details.

  • 64

    Compare the archived version of our CEA preceding this change here to the version following this change here.

  • 65

    For example, see our review of AMF for a discussion of the roles played by AMF and its partners in net distributions.

  • 66

    See the sheets for each top charity in 2020 GiveWell cost-effectiveness analysis — version 1. The drop-down settings on the "Results" sheet can also be used to toggle between overall and country-specific cost-effectiveness estimates. To use drop-down settings, make a copy of the CEA linked above using "File," then "Make a copy."

  • 67

    Using Malaria Consortium as an example:

    • The calculations in the core CEA model for Malaria Consortium use our total cost per child treated estimates (including costs covered by actors other than Malaria Consortium). See our CEA preceding this change here.
    • The CEA also uses our expectations of Malaria Consortium's spending by country.
    • Because Malaria Consortium does not cover the same proportion of total costs in each country, using our expectations of Malaria Consortium's spending by country and total cost per treatment estimates led to errors in our calculations of the number of children expected to be treated by country. Compare the version of the CEA preceding this change here to the version following this change here.

  • 68

    See 2020 GiveWell cost-effectiveness analysis — version 1, "GiveDirectly" sheet.

  • 69

    Compare the top charity sheets in the version of our CEA preceding this change to the version of our CEA following this change. For Malaria Consortium, for example, this row was added.

  • 70

    Compare the top charity sheets in the version of our CEA preceding this change to the version of our CEA following this change.

  • 71

    Compare the archived version of our CEA preceding this change here to the version following this change here.

  • 72

    Our 2020 GiveWell cost-effectiveness analysis — version 1 reports the following cost-effectiveness results for each top charity, found here (to toggle among these choices, make a copy of the CEA file using "File," then "Make a copy"):

    • Core model only
    • Include downside adjustments
    • Include downside adjustments and adjustments for excluded effects
    • Include leverage/funging and all other adjustments

  • 73

    See here for an example of how we apply these adjustments in our CEA for Malaria Consortium.

  • 74
    • See the "Inclusion/exclusion" sheet here for details on how we calculate these adjustments.
    • See here for an example of how we apply these adjustments in our CEA for Malaria Consortium.

  • 75

    See the archived copy of the CEA preceding this change here.

  • 76

    See the archived copy of the CEA following this change, here.

  • 77
    • Our cost-effectiveness analyses of AMF, HKI, and Malaria Consortium include multiple types of benefits, including deaths averted and long-term income increases for some program participants. See our "AMF," "HKI," and "Malaria Consortium" sheets in 2020 GiveWell cost-effectiveness analysis — version 1 for more details.
    • We sometimes want to report "cost per life saved" estimates for these organizations, which exclude other types of benefits of their programs.
    • We have found these calculations simpler to implement when leverage and funging adjustments are applied after downside adjustments and adjustments for excluded effects. That is because our cost per life saved estimates use all downside adjustments in our model, but only account for selected adjustments for excluded effects and adjustments for funging (but not leverage). For more details, see the calculations and cell notes for Malaria Consortium here. The same calculations are implemented at the bottom of our "HKI" and "AMF" sheets.

  • 78

    For example, we discuss potential development impacts of mass deworming programs in this section of our intervention report.

  • 79
    • See this section of our cost-effectiveness analysis of HKI for our updated development effects estimate.
    • For comparison, see the archived version of our cost-effectiveness analysis preceding this change here.

  • 80

    See our review of HKI and our intervention report on vitamin A supplementation for further discussion on RCTs of vitamin A supplementation.

  • 81

    See our 2019 HKI VAS - information for country-level cost-effectiveness spreadsheet, "External validity by country" sheet.

  • 82

    See our 2020 HKI VAS - information for country-level cost-effectiveness spreadsheet, "External validity by country" sheet.

  • 83

    See our 2019 HKI VAS - information for country-level cost-effectiveness spreadsheet, "External validity by country" sheet.

  • 84

  • 85

    See our 2019 HKI VAS - information for country-level cost-effectiveness spreadsheet, "External validity by country" sheet, this row.

  • 86

    See our 2020 external validity spreadsheet, "External validity by country" sheet, this row.

  • 87

    Compare the archived version of the CEA following this change here to the version preceding this change here.

  • 88

    As of the date of the publication of this changelog, our 2020 room for more funding analysis for AMF is not yet published. See our most up-to-date version of our AMF room for more funding analysis in this section of our review of AMF.

  • 89

    See this section of our review of AMF for additional information.

  • 90
    • We discuss insecticide resistance in depth in this report. PBO nets are discussed in this section.
    • Our updated insecticide resistance adjustment following this change is available here, and our adjustment preceding the change is available here.
    • We do not have permission from AMF to share the specific proportions of PBO nets it plans to purchase by country.

  • 91

  • 92

    See the "What is LLIN distribution and how does it target malaria?" and "What is the evidence regarding the general effectiveness of LLIN distributions?" sections of our intervention report on mass distribution of long-lasting insecticide-treated nets (LLINs) for more details.

  • 93

  • 94
    • Compare the archived version of our CEA preceding this change here to the version following this change here.
    • For all countries except Democratic Republic of the Congo, the value of 2.11 comes from this calculation. In our review of AMF, we note that "the data we have seen from AMF's distributions in DRC provide some evidence that LLINs decayed considerably more quickly than expected." We roughly adjust our LLIN lifespan estimate for DRC downwards by 10% to account for this difference.

  • 95

    See the "History" section of Pryce, Richardson, and Lengeler 2018, Pg 65.

  • 96

    Compare the "AMF" sheet in the archived version of the CEA preceding this change here to the highlighted cells in the version of the CEA following this change here.

  • 97

    See the "Inclusion/exclusion" sheet on the version of the CEA preceding this change here compared to the version following this change here.