You are here

2018 Cost-Effectiveness Analysis Changelog

This page provides details about changes that were made to our cost-effectiveness analysis (CEA) in 2018.

Version 8 — Published September 6, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 8

Change 1: Updated the adjustment for insecticide resistance in the nets model

We changed the value for the parameter labeled Efficacy reduction due to insecticide resistance from 26% to 35%. This updated value reflects our latest views on insecticide resistance.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Deworm the World 12.2x 12.2x 0.0%
Schistosomiasis Control Initiative 9.7x 9.7x 0.4%
Sightsavers 4.8x 4.8x 0.6%
Against Malaria Foundation 5.4x 4.7x -12.1%
Malaria Consortium 7.0x 6.9x -0.9%
The END Fund 4.2x 4.2x 0.2%

Change 2: Removed short-term health benefits of deworming from the model

We removed the short-term health benefits of deworming from our model. Short-term health benefits represented only a small portion of the overall benefits of deworming according to our model. We felt that our method for assessing short-term health benefits had too many issues to be worth keeping in the CEA. The short-term health benefits of deworming are now listed as "Excluded" in the "Inclusion/Exclusion" tab.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Deworm the World 12.2x 12.0x -2.1%
Schistosomiasis Control Initiative 9.7x 9.5x -2.6%
Sightsavers 4.8x 4.7x -2.2%
The END Fund 4.2x 4.1x -2.6%

Change 3: Updated END Fund's cost per treatment

An error was found in our 2017 analysis of END Fund's cost per treatment. We corrected this error in our updated analysis. The cost per person deworming moved from $0.94 to $0.81.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
The END Fund 4.1x 4.8x 17.6%

Change 4: Added a parameter to account for long-term benefits of Vitamin A supplementation

We added a parameter labeled Value of development benefits from a year of VAS coverage relative to a year of deworming in Miguel and Kremer. By default, this parameter has been set to a value of zero for all individuals. Accordingly, this addition has not yet had any effect on the model's results. We expect that individuals contributing inputs to the CEA may change the values used for this parameter in the future. We added this parameter so that individuals have the option to include potential development benefits of VAS; this makes the VAS CEA more consistent with our CEAs for interventions related to malaria and deworming, but we are highly uncertain whether VAS leads to development benefits.

Change 5: Updated the "Inclusion/Exclusion" tab

We made several additions and adjustments to content in the "Inclusion/Exclusion" tab. These changes did not affect the model's results.

Version 7 — Published August 24, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 7

Change 1: Added a "Country selection" tab to the CEA

We added a new "Country selection" tab to the CEA spreadsheet. If users select specific countries from drop-down menus on this new tab, country-specific parameter values will be used in the CEA.

By default, the CEA will continue to use overall figures that are calculated by aggregating across countries. Selecting specific countries will lead to changes in CEA results. When the default figures are used, results in this version of the CEA are identical to results in the previous version.

Version 6 — Published August 10, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 6

Change 1: Updated the cost per child covered in Helen Keller International's vitamin A supplementation program

We updated the cost per child covered in each round of Helen Keller International's vitamin A supplementation program from $0.75 to $1.35. We made this update to incorporate the results of our 2018 analysis of Helen Keller International's cost per supplement.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Helen Keller International 8.7x 5.1x -41.6%

Version 5 — Published July 23, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 5

Change 1: Updated the cost per child targeted in Malaria Consortium's SMC program

We updated the cost per child targeted in Malaria Consortium's seasonal malaria chemoprevention (SMC) program from $4.33 to $4.54. We made this update to incorporate the results of our 2018 analysis of Malaria Consortium's cost per treatment.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Malaria Consortium 7.1x 6.8x -3.8%

Change 2: Updated estimates of coverage rates in Malaria Consortium's SMC program

We updated estimates of coverage rates in Malaria Consortium's SMC program. We made this update to incorporate the results of coverage surveys that occurred in 2017.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Malaria Consortium 6.8x 7.0x 2.9%

Version 4 — Published July 16, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 4

Change 1: Updated the Cost per ITN parameter

We updated the Cost per ITN parameter in the "Nets" tab from a value of $4.22 to $4.53. We made this update to incorporate the results of our 2018 analysis of AMF's cost per net.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Schistosomiasis Control Initiative 9.7x 9.7x 0.3%
Sightsavers 4.7x 4.8x 0.3%
Against Malaria Foundation 5.7x 5.4x -6.5%
Malaria Consortium 7.1x 7.1x -0.5%
The END Fund 4.2x 4.2x 0.1%

Change 2: Made a new explanatory document available

We added links in the CEA spreadsheet to a document that includes:

  • A high-level overview of why we use CEAs and how we structure them
  • Conceptual guidelines on how to create CEAs
  • Technical guidelines related to CEAs

While the target audience of the document is members of the GiveWell research team, the content may be useful to others. The new document is intended to replace the out-of-date FAQ & Parameter Details document.

Version 3 — Published May 3, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 3

Change 1: Updated the Transfers as a percentage of total cost parameter

We updated the Transfers as a percentage of total cost parameter in the "Cash" tab from a value of 81.5% to 83.0%. We made this update to incorporate information about GiveDirectly's spending from August 2017 through February 2018. The calculations behind the new value can be seen here.



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Deworm the World 12.4x 12.2x -1.8%
Schistosomiasis Control Initiative 9.8x 9.7x -1.8%
Sightsavers 4.8x 4.7x -1.8%
Against Malaria Foundation 5.8x 5.7x -1.8%
Malaria Consortium 7.2x 7.1x -1.8%
Helen Keller International 8.8x 8.7x -1.8%
The END Fund 4.2x 4.2x -1.8%
No Lean Season 5.3x 5.2x -1.8%

Version 2 — Published April 26, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 2

Change 1: Renamed the "Parameters" tab to "User inputs"

This change did not affect the model's results.

Change 2: Replaced items on the "User inputs" tab with default values

We removed the majority of items from the "User inputs" tab and began using a single default value for these parameters. The parameters and default values are listed below, separated by intervention. Items marked "Removed" were deleted from the CEA. Removing values was mathematically equivalent to setting their values to 100%. Our reasoning behind the default values can be found in the cell notes associated with each item. These changes are intended to make the CEA easier to engage with.

Cash parameters

  • Percent of investment returned when benefits end [20%]



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Against Malaria Foundation 6.0x 6.1x 0.8%
Malaria Consortium 6.9x 6.9x 0.6%
Helen Keller International 8.9x 8.9x 0.2%


Migration subsidies parameters
  • Internal validity adjustment — Migration subsidies [80%]
  • External validity adjustment — Migration subsidies [80%]
  • Further adjustment for uncertainty of evidence for benefits in the second year of remigration [Removed]
  • Adjustment for uncertainty of evidence regarding benefits in lesser lean seasons [Removed]
  • Adjustment to the utility of migration, migrants' nonmigrating household members [100%]



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
No Lean Season 4.9x 5.3x 8.2%


VAS parameters
  • Relative risk of all-cause mortality for children aged 6- to 59-months participating in VAS programs [0.76]
  • Internal validity adjustment — VAS [95%]



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Helen Keller International 8.9x 8.8x -0.5%


SMC parameters
  • Internal validity adjustment — SMC [95%]
  • External validity adjustment — SMC [100%]
  • Ratio of the reduction in malaria mortality to the reduction in malaria incidence [100%]



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Malaria Consortium 6.9x 7.2x 4.2%

Nets parameters
  • Relative efficacy of ITNs for reducing mortality of individuals age 5 and older [80%]
  • Net use adjustment [90%]
  • Proportion of mortality attributed to malaria in areas AMF works vs. the contexts of trials in Lengeler 2004 [100%]
  • Internal validity adjustment — Nets [95%]



Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Deworm the World 12.4x 12.4x 0.0%
Schistosomiasis Control Initiative 9.8 9.8x 0.4%
Sightsavers 4.8x 4.8 0.5%
Against Malaria Foundation 6.1x 5.8x -3.6%
Malaria Consortium 7.2x 7.2x 0.5%
The END Fund 4.2x 4.2x 0.2%


Deworming parameters
  • Treatment effect of deworming on ln(consumption) [0.14]
  • Multiplier for resource sharing within households [2.0]
  • Duration of long-term benefits of deworming (in years) [30]
  • Proportion of dewormed children that benefited from long-term gains in Baird et al. [Removed]
  • Adjustment for varying treatment frequency (all deworming charities) [Removed]

Changes to deworming parameters moved results for some individuals, but did not alter the median CEA result.

Change 3: Added "Inclusion/Exclusion" tab

We added a tab titled "Inclusion/Exclusion" to the CEA. This tab lists different items that we considered accounting for in our cost-effectiveness analysis. Each item is evaluated with a rubric assessing how easy it would to include the item in our model and how much including an item might move our bottom line estimates.

This change did not affect the model's results.

Change 4: Removed leverage/funging adjustments from the "User inputs" tab

This change was intended to make the CEA easier to engage with. The change did not affect the model's results.

Change 5: Added a set of "Conventional" moral weights to the CEA

We added a column labeled "Conventional" to the "Moral weights" tab of the CEA. This set of moral weights is an attempt to represent normative values commonly used in analyses conducted by other organizations in the global health and development field; see more information here. We don't believe there's a single set of normative values that can accurately represent the typical view in the global health and development community, so these moral weights should be interpreted cautiously. The new set of moral weights is combined with the median values for items on the "User inputs" tab to arrive at final cost-effectiveness figures in the "Conventional" column. These final figures were not accounted for when calculating the median results among individuals included in our CEA.

Change 6: Revised content on the "Notes" tab

Content on the "Notes" tab was revised to reflect the changes above. This change did not affect the model's results.

Version 1 — Published January 24, 2018

Link to the cost-effectiveness analysis file: 2018 CEA — Version 1

Change 1: Rolled out a new update process

With this update, we transitioned to a new process for releasing updates to the cost-effectiveness model. The new process involves both internal changes in how GiveWell works on updates and an external change to the structure of release notes. We hope that this change will make updating the CEA easier.

This change did not affect the model's results.

Change 2: Changed the summary effect in the seasonal malaria chemoprevention CEA

We moved the parameter labeled "Relative risk for malaria cases, intention to treat effect" from the "Parameters" tab to the "SMC" tab. The parameter was set to a default value of 0.25, the relative risk suggested by our meta-analysis of seasonal malaria chemoprevention trials.


Charity
Median [Charity]
vs. cash before
Median [Charity]
vs. cash after
Percent
change
Malaria Consortium 6.6x 6.9x 4.2%