Progress on Issues We Identified During Top Charities Red Teaming

Published: December 2025

Progress on issues we flagged as mistakes in our original report (either because they led to important errors or because they worsened the credibility of our research)

Failure to account for individuals receiving interventions from other sources

From our top charities red teaming report:

Bottom line:

We looked into this issue by analyzing survey data and speaking with experts to better understand how many people receive interventions from non-campaign sources. We found we were underestimating the percentage of people who would receive nets without our funding. For nets, we now estimate approximately 25-30% of children under 5 would use nets obtained outside of campaigns (compared to our previous assumption of 5-10% counterfactual net access2 ), which reduces cost-effectiveness by 20-25%. This did not change any grant decisions.

We've recommended a grant to improve routine coverage of VAS (continuous distribution through existing health channels). For nets, we haven’t done much additional exploration of funding routine distribution, but have heard since red teaming that this is an area worth looking into. We plan to look into this further.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Understand how common it is for individuals to receive nets outside of campaigns in DRC and Nigeria, two countries where we’re considering large nets grants. What is our best guess for coverage from continuous distribution or other channels currently in both DRC and Nigeria?
  • We incorporated this concern into our estimated cost-effectiveness of nets campaigns in DRC, Nigeria, Chad, and Zambia based on survey data and discussions with stakeholders.3 We now estimate that 23% in Nigeria, 28% in DRC, 28% in Chad, and 29% in Zambia of children under 5 would use nets without campaigns. This reduced cost-effectiveness by ~20-25% but didn't change our grant decisions.4
• Explore whether and how to fund additional surveys of Vitamin A deficiency in countries where we expect to consider large VAS grants.
  • We haven't funded additional VAD surveys yet because we think the “value of information” per dollar spent is too low, given high survey costs relative to VAS funding opportunities per country.5 We're exploring cheaper options including testing existing blood samples or studying multiple nutritional deficiencies simultaneously.
• Update our estimates of counterfactual vaccination coverage for New Incentives' program to account for the increase in vaccination rates over time. We will compare our revised estimates to the actual coverage achieved in areas where New Incentives operates to check the accuracy of our assumptions.
  • We analyzed counterfactual vaccination coverage trends using DHS/MICS survey data from 2013-2023. We revised our estimate of annual changes in the unvaccinated population from -5% to -2%. This increased our estimate of the program's cost-effectiveness by 8-20% across states.6
• Explicitly state our assumptions on the percentage of individuals who would receive interventions like nets from other sources without campaigns. (We’ve already incorporated this into our updated top charity pages.7 )
  • We had already addressed this at the time we published our red teaming report.
• Talk to experts about the extent to which campaigns for health commodities we fund interact with routine distribution, and consider whether we should be supporting routine distribution, at least in some areas. Is our funding inadvertently leading to the weakening of routine coverage systems? What would the consequences of this be? Are there areas where supporting routine distribution would be cost-effective?
  • VAS: In September 2025, we recommended a grant that supports the transition from VAS campaigns to routine delivery of VAS in Cote d'Ivoire and two states in Nigeria. We plan to continue to explore more targeted approaches to routine VAS delivery in the future.8 We also published a lookback on a previous VAS grant in Nigeria, finding cost-effectiveness was significantly lower than our initial estimate, partly due to increased routine coverage.9
  • Nets: At conferences10 and through conversations11 with other malaria experts since red teaming, we've learned that routine net distribution through antenatal care (ANC) and immunization (EPI) visits could be an area worth looking into further. "We also learned about a complex "swap mechanism" in the DRC where leftover campaign nets flow into routine channels and are later reimbursed for use in future mass campaigns. AMF's perspective is that this mechanism allows efficient use of nets by reducing transport costs, though routine needs consistently exceed supply.12 We haven’t yet considered grants specifically for routine distribution of nets.

Failure to sense check raw data in burden calculations

From our top charities red teaming report:

Bottom line:

We examined discrepancies in burden data across all our priority programs and found several cases where relying on a single source led to significant inaccuracies. By incorporating multiple data sources and conducting systematic reviews of burden data, we ended up making $34m in grants we probably wouldn’t have otherwise and not making $14m in grants we probably would have otherwise.13

We've also published guidance on burden estimation and created a more standardized approach across our teams, and we’ve identified and accounted for similar issues with parameters like anemia rates, population estimates, and program coverage.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Audit analyses of other programs to understand where else this issue is present. We guess this is much more prevalent than just the examples we’ve cited.
  • We conducted additional audits across multiple program areas and found several instances where burden data needed further scrutiny.14
  • Some of these led to changes in our grants:
    • Directing over $29 million in new funding to malaria interventions (SMC and nets) in Chad after finding IHME data significantly underestimated malaria burden15
    • Deciding not to renew a $20 million nets campaign in Anambra, Nigeria after finding burden was overestimated16
    • Updating our approach to estimating mortality in Cameroon, which led us to renew a $5 million grant to Helen Keller International.17
  • Some of these have not yet led to changes in grants but we think improved estimates and/or may change grants in the future.
    • For maternal mortality in Nigeria, we discovered a roughly 3x difference between IHME and WHO/UNICEF estimates. This finding has prompted us to explore interventions targeting maternal mortality that may be more cost-effective than we previously thought. Moving forward, we plan to adopt an averaging approach across burden sources.18
    • For child mortality in Burkina Faso, we found IHME's estimates differed significantly from other sources including UN IGME, a recent DHS survey, and control group data from recent randomized controlled trials. In response, we developed a weighted average approach that incorporates multiple data sources, resulting in an estimate of 9.4 deaths per 1,000 person-years for children aged 1-59 months.19
    • For our vitamin A supplementation grants, we observed that cost-effectiveness estimates fell substantially when using IHME's Global Burden of Disease (GBD) 2021 data compared to GBD 2019. We now incorporate multiple data sources (GBD 2019, GBD 2021, IGME 2021, and others when available) for both all-cause and VAS-preventable mortality. The effect of this change varied across countries.20
• Investigate the discrepancies between IHME and UN IGME estimates of under-5 mortality rates in Nigerian states. We will consult with experts in the field and examine the methodologies used by each organization to better understand the reasons for the differences. In the meantime, we will consider using an average of state-level and zonal estimates to reduce the impact of potential noise in the data.
  • We spoke with experts at IHME and the IGME Technical Advisory Group to investigate these discrepancies.
  • We now make adjustments of IHME GBD’s under-5 mortality rates in Nigeria to incorporate estimates from both UN IGME and recent survey data from the 2021 UNICEF Multiple Indicator Cluster Survey (“MICS”).
  • We do not adjust UN or GBD state-level estimates towards a regional mean. We think their models should smooth out noise from each individual survey, but we’re not sure exactly whether and how they do this in practice.
  • For MICS estimates, we start with zonal estimates (a zone is a group of ~6 states) and then adjust them in each state based on average state-level differences within zones over 3 recent national surveys. This is because we think MICS state-level estimates are likely to be noisy.21
• Conduct a systematic review of the data sources and methodologies used to estimate the malaria burden across the countries where we fund malaria interventions. We will engage with experts and compare our estimates to those from other reputable organizations to identify any inconsistencies and potential areas for improvement. In the meantime, we’ll include a note in our nets and SMC CEAs to explain how to deal with specific countries like Chad with burden data that’s very different from other data sources.
  • We looked into this issue in Chad specifically, where we concluded we were substantially underestimating malaria burden.22
• Develop a standardized process for scrutinizing key data inputs in our cost-effectiveness analyses, including comparing estimates from multiple sources, engaging with experts, and documenting any discrepancies and the reasons for our chosen inputs.
  • For burden estimates, we’ve created guidance on how to incorporate multiple, noisy inputs. In short: “Disease burden estimates, such as child mortality rates, are a key input in our cost-effectiveness analyses. Historically, for consistency and convenience, we've primarily relied on a single source for these estimates. Going forward, we plan to consider multiple sources for burden estimates, apply a higher level of scrutiny to these estimates, and adjust for potential biases or inaccuracies, like we do when estimating other parameters in our models.”
  • We’ve identified other cases where this is an issue in our work. For example, we flagged concerns regarding anemia burden measures in our iron grantmaking red teaming report.23 We’ve also noticed a similar issue with population estimates,24 vaccination rates,25 and coverage data.26

Failure to estimate the interactions and overlap between programs

From our top charities red teaming report:

By focusing on the delivery of a specific health program, we're susceptible to ignoring interactions or overlap with other programs. We found a few cases where we think this could be an issue... In areas like Northern Nigeria, GiveWell or others are funding insecticide-treated nets (ITNs), seasonal malaria chemoprevention (SMC), vaccines, oral rehydration solution (ORS), increased access to clean water, azithromycin, and other programs. If these programs interact or address the same health issues, we could be incorrectly estimating their combined impact.

Bottom line:

While we haven’t done a comprehensive analysis of this issue across all programs, we think we’ve identified the main cases where this could significantly reduce cost-effectiveness, and we’ve begun accounting for these in our analysis. For example, we've adjusted our vitamin A supplementation model to account for its overlap with azithromycin distribution (reducing estimated effectiveness by 20%) and found that the roll out of malaria vaccines is unlikely to have a major impact on cost-effectiveness of SMC and ITNs programs, at least in the near term.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Develop an approach to model overlapping effects of programs and address this issue in upcoming grant investigations where overlap is most likely (e.g., azithromycin and VAS).
  • We’ve created a rough framework for understanding where overlapping programs could change our decisions and addressed this in what we think are the cases where it’s most likely to make a difference. We now think we’re accounting for places where this overlap is likely to have meaningful effects on cost-effectiveness, but we still have several uncertainties.
  • The framework focuses on three questions:
    • Do other programs reduce the health problems our program aims to solve? For example, in Burkina Faso, the eligible population for malaria vaccination accounts for 60% of under-5 mortality, the vaccines are expected to reach about 60% of eligible children during the grant period, and we think they will reduce their mortality by 40%. This meant that before our nets campaign even started, vaccines would already be preventing approximately 14% of potential malaria deaths.27 Without accounting for this, we would have overestimated the impact of our nets. Our approach now is to explicitly adjust for these kinds of overlaps when they're expected to reduce the treatable burden by more than 10%.28
    • Do programs make each other more or less effective? Sometimes programs can work together to enhance (or occasionally reduce) each other's effectiveness. For instance, we found evidence that PBO nets (a newer type of mosquito net) might increase malaria in areas using certain kinds of indoor residual spraying.29 We’ll continue to adjust our models when we have specific evidence of these kinds of interactions.
    • Can programs be delivered together to save money? We've found significant opportunities to reduce costs by delivering multiple health interventions at the same time. For example, adding vitamin A supplementation to existing seasonal malaria chemoprevention campaigns in Nigeria reduced the cost of delivering vitamin A by about 23% (from $0.65 to $0.50 per supplement). We’ll continue to look for these opportunities, especially when programs have high fixed costs that can be shared.
  • We've already applied this framework in a couple contexts:
    • We completed an adjustment for the interaction between vitamin A supplementation and azithromycin distribution.30 This showed that failing to account for this interaction could lead to approximately 15% overestimation of VAS cost-effectiveness.31
    • We include an adjustment for malaria vaccine roll-out in our malaria prevention CEAs.32
  • We also conducted a rough initial analysis of how overlapping health programs might reduce the impact of net distributions in places like Nigeria, where we and others fund multiple preventive health interventions at the same time. We think we’re already accounting for multiple programs affecting malaria directly (which we think have the biggest effects), while programs that indirectly prevent malaria deaths likely reduce net cost-effectiveness by ~5% individually. While we think it's possible that collectively these programs reduce cost-effectiveness up to 20%, we don't currently recommend modeling these indirect effects due to high uncertainty and limited impact on health program rankings (though this could matter more for health vs. livelihoods comparisons). We may revisit this in the future.33
• In the future, publish our view on why we typically support vertical over horizontal programs in order to solicit feedback and pushback.
  • We haven't yet published our position on vertical programs (those focused on delivering a specific intervention) versus horizontal programs (those strengthening overall health systems). However, we've begun analyzing this question internally.
  • This has become more relevant in light of recent changes to US foreign aid, and we expect to explore opportunities that provide more horizontal support than we typically have in the past.34

Failure to more frequently engage with outside experts

From our top charities red teaming report:

Failure to more frequently engage with outside experts. We've identified some issues in our analyses through recent interactions with external experts (e.g., overestimation of net durability and underestimation of routine net distribution). This suggests we may be missing important perspectives by not engaging with implementation experts, researchers, and other stakeholders more frequently.

Bottom line:

We’ve substantially increased our engagement with outside experts. In 2024, we spent ~240 days at conferences or site visits, compared to ~60 in 2023. We spoke to experts more regularly as part of grant investigations, and tried a few new approaches to getting external feedback. While it’s tough to establish impact, we think this led to four smaller grants we might not have made otherwise (totalling ~$1 million) and led us to deprioritize a ~$10 million grant we might’ve made otherwise.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• More regularly attend conferences with experts in areas in which we fund programs (malaria, vaccination, etc.).
  • In 2024, our research team attended 16 conferences, or ~140 days, compared to ~40 days at conferences in 2023.35
  • We think these conferences helped us build relationships with experts and identify new grant opportunities. Two examples:
    • A conversation with another funder at a conference led us to re-evaluate our assumptions on HPV coverage and ultimately deprioritize a roughly $10 million grant we may have made otherwise.36
    • We learned about Uduma, a for-profit rural water utility, at a conference and made a $480,000 grant to them in November 2024.37
  • We also made more site visits. In 2023, we spent approximately 20 days on site visits. In 2024, the number was approximately 100 days.38
• Reach out to experts more regularly as part of grant investigations and intervention research. We’ve always consulted with program implementers, researchers, and others through the course of our work, but we think we should allocate more relative time to conversations over desk research in most cases.
  • Our research team has allocated more time to expert conversations. A few examples:
    • Our 2024 grants for VAS to Helen Keller International relied significantly on conversations with program experts. Excluding conversations with the grantee, we had 15 external conversations.
    • We’ve set up longer-term contracts with individuals who provide us regular feedback. For example, our water and livelihoods team has engaged Daniele Lantagne and Paul Gunstensen for input on grant opportunities and external review of our research.
    • We spoke with other implementers about programs we’re considering. For example, we discussed our 2024 grant to support PATH’s technical assistance to support the rollout of malaria vaccines with external stakeholders in the space.39
  • This led to learning about some new grant opportunities. For example:
    • The $150,000 grant to International Rescue Committee (IRC) for desk-based scoping of programs to increase vaccination coverage stemmed from a conversation with an expert.
    • We connected with Zipline at a conference, and made a $55,000 grant to them for desk-based scoping of drones to increase vaccination coverage in December 2024.
    • We are currently considering a $4 million grant that we learned about through an expert conversation.40
• Experiment with new approaches for getting feedback on our work.
  • In addition to the above, we tried a few other approaches we hadn’t (or hadn’t extensively) used before. Three examples:
    • Following our red teaming of GiveWell’s top charities, we decided to review our iron grantmaking to understand what were the top research questions we should address as we consider making additional grants in the near future. We had three experts review our work in parallel to internal red teaming, so we could get input and ask questions along the way.41 We did not do this during our top charities red teaming, in the report of which we wrote “we had limited back-and-forth with external experts during the red teaming process, and we think more engagement with individuals outside of GiveWell could improve the process.”
    • We made a grant to Busara to collect qualitative information on our grants to Helen Keller International's vitamin A supplementation program in Nigeria.42
    • We funded the Center for Global Development to understand why highly cost-effective GiveWell programs aren’t funded by other groups focused on saving lives. This evaluation was designed to get external scrutiny from an organization with expertise in global health and development, and by other funders and decision-makers in low- and middle-income countries.

Progress on issues that we had been aware of at the time of red teaming and had deprioritized but that we thought were worth looking into following red teaming

Insufficient consideration of other ways to support uptake of effective interventions

From our top charities red teaming report:

We have focused mainly on direct delivery methods for interventions like vitamin A supplementation and nets. However, fortification may be a more effective way to increase vitamin A uptake in some regions, or there may be other opportunities to increase the use of nets (e.g., market shaping to lower prices, research and development into new types of nets). We haven’t investigated these in-depth, which means we may be missing out on more cost-effective funding opportunities.

Bottom line:

Since publishing the red teaming report, we’ve done limited work to explore alternative ways to promote the uptake of top charity interventions like ITNs or VAS.

However, we have continued to explore alternative approaches for other programs. We've launched an RFI for water chlorination programs, funded evidence generation through RCTs, tested novel delivery platforms, shared research findings with other funders, and developed more responsive funding mechanisms for urgent needs. This work was already underway prior to red teaming, so we guess these weren’t causally affected by red teaming.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Assess the cost-effectiveness and funding gaps for vitamin A fortification programs, to determine whether this approach could complement or partially replace vitamin A supplementation in some regions.
  • We have conducted limited research comparing vitamin A fortification to supplementation programs. Our current assessment suggests supplementation is more promising in the short term due to evidence gaps regarding fortification effectiveness.43 However, fortification remains potentially promising, as some evidence indicates that more frequent vitamin A delivery (at consistent dosage) may yield greater mortality reduction effects.44 We plan to prioritize evidence generation on the effect of vitamin A fortification on vitamin A status.
• Learn more about other ways philanthropic funding could support more uptake of nets or more effective nets by talking to other funders, malaria experts, and implementers.
  • We are currently funding Malaria Consortium and Behavioral Insights Team’s “Be In A Net” research project to explore and pilot novel ways of increasing net use in Nigeria and Uganda. While we think this is a good example of finding alternative ways to increase uptake of nets, the decision to fund this project was underway before red teaming.
  • We may explore other approaches in the future.
• In the future, beyond our top charities, look for additional ways to support effective programs beyond just, for example, direct delivery. Are there alternative ways to support uptake of oral rehydration solution (ORS), chlorination, etc., via technical assistance, evidence generation, or market shaping? Where is direct delivery the bottleneck, and where are other approaches likely to be more impactful?
  • We’ve continued to explore alternative grants beyond our top charities. Some examples:
    • In February 2025, we made a $250,000 grant to CHAI to quickly address time-sensitive malaria funding gaps resulting from a USAID stop work order. Rather than investigating each small funding gap individually (which would cause harmful delays), we leveraged CHAI's expertise to identify and fill the most critical needs for interventions like seasonal malaria chemoprevention and insecticide-treated net distribution.
    • In January 2025, we launched our first public RFI seeking organizations interested in implementing water chlorination programs in high-priority countries. This approach aims to expand the number of implementers of large-scale chlorination programs beyond Evidence Action, our primary partner for water quality interventions that doesn't operate in some places where chlorination could be highly cost-effective (such as Francophone Africa). The RFI is designed to accommodate various implementers including non-profits, for-profit water utilities, and multilateral organizations.
    • In May 2024, we made a $676,857 grant to Wageningen University to add chlorine, ORS, and zinc to an existing study of door-to-door health service delivery in remote communities in Sierra Leone. This research will help us understand how household-level provision of these products increases usage when delivered door-to-door in remote communities.
    • In May 2024, we made a $1 million grant to Dimagi to pilot their CommCare Connect mobile health platform in Nigeria. CommCare Connect aims to increase the take-up of specific healthcare commodities, such as vitamin A supplementation and oral rehydration solution and zinc, by providing payments to frontline health workers through local healthcare organizations to deliver these commodities through household visits in areas with low coverage.

Insufficient investigation of how our funding decisions affect other funders

From our top charities red team report:

We think we could do more to understand how much our funding of top charities crowds in or out other funders. We might do this by, for example, commissioning an external review of our assumptions to get a more objective view on these questions, looking at how long it took for programs we decided not to fund to get funded, or trying to find specific funding opportunities where we’re especially likely to crowd in funding from others.

Bottom line:

We've made limited progress understanding how our funding decisions affect other funders. While we've commissioned an external review through the Center for Global Development to examine why highly effective programs we fund aren't already funded by other organizations, most of our planned investigations in this area haven't begun. Though we've added these items to our research pipeline, we haven't prioritized them for immediate action.

We decided to deprioritize this work this year given significant changes in the US aid funding landscape, which are likely to affect whether our funding crowds out other funders and alters our understanding of how additive our funding is and how we might influence others.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Commission an external review of our short- and long-term funging assumptions. Since this red teaming exercise, we’ve made a grant to the Center for Global Development for Justin Sandefur to talk to other funders about how they decide which programs to fund and how GiveWell’s funding may influence their decisions. This project will include speaking to Ministry of Health officials in countries where GiveWell’s top charities operate, multilateral and bilateral funders, and other large philanthropic funders.
  • See this grant page.
• Look back at past “close calls” (i.e. cases where we strongly considered funding a campaign, but didn’t end up funding it) to better understand what happens when we don’t fund campaigns. Did these eventually get funded? How long did it take?
• Look for opportunities where we think there’s a good chance our funding could crowd in other funders. Are there new programs we could support that, if successful, would be a natural fit for funders like the Global Fund or Gavi? Is it possible to distinguish these cases from those where we’re likely to crowd out other funders in the long term?
• Decide whether and how to account for long-term funging explicitly in our cost-effectiveness estimates. How much does this affect programs? How should this change the cost-effectiveness of our top charities relative to other programs we might fund?
• In the future, consider other approaches to learn more about how we influence funding from other donors. This could include:
  • Analysis of how other funders’ support of our top charities’ programs has changed once we start funding them. We’ve done some initial, unpublished analysis to understand how Global Fund support for nets has changed since we started funding nets in specific countries. We could extend this analysis to other interventions and keep it up-to-date as we begin funding in new countries.
  • Historical case studies on funding in specific countries or by other funders. E.g., how has malaria funding evolved in Nigeria over the past 10 years? What is the history of funding for SMC programs in Sub-Saharan Africa?

We've made some progress in understanding how other funders make decisions, though this progress has been limited. Some examples of progress:

• Malaria programs: Our teams report stronger relationships with key malaria funders than 18 months ago, suggesting improved capability to understand funding dynamics, though recent disruptions in U.S. government funding to major organizations like PMI and the Global Fund have complicated this landscape.
• Vitamin A supplementation: Conversations with UNICEF, Nutrition International, and the Government of Canada have confirmed there are few major VAS funders besides ourselves and the Canadian government, limiting opportunities to crowd in additional funding. Our mapping of regional coverage patterns shows that UNICEF tends to prioritize emergency zones and areas where VAS can be integrated with other health programs, while HKI fills remaining coverage gaps.
• Vaccines: We are monitoring potential advocacy opportunities for increased funding around the malaria vaccine through networking conversations.

Questions about displacing other funders have become especially relevant as we've considered responses to recent aid cuts. We have recently made recoverable grants that include verbiage such that if USAID funding resumes during the pre-campaign period, implementing partners would return any unused funds to GiveWell.45

Insufficient attention to some fundamental drivers of intervention efficacy

From our top charities red team report:

For example, we think our assumptions about insecticide resistance and net durability are outdated, and we’ve postponed research on vaccine efficacy in Nigeria and other countries we fund. Doing these checks not only informs cost-effectiveness estimates but may also reveal new grant opportunities (e.g., support for improving vaccine efficacy if we find it’s low, support for next-generation nets less affected by insecticide resistance).

Bottom line:

We've done some follow-up on this issue but have left significant work undone. For nets, we've updated our assumptions on durability and attrition, and begun triangulating our estimates against others. For vaccines, we've had initial conversations about concerns regarding vaccine efficacy. For SMC, we've considered funding a study on efficacy of different drug combinations outside the Sahel, where resistance is especially concerning. In these cases, our updates haven't substantially changed our grant decisions. For VAS, we've done less follow-up work overall.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Update our assumptions about insecticide resistance and net durability by reviewing the most recent available evidence and incorporating up-to-date estimates into our cost-effectiveness analyses. We will also consider whether to prioritize the adoption of new net technologies in areas with high levels of resistance.
  • We previously assumed that chemical decay (the loss of insecticide in a net) was similar for campaign nets and nets used in the studies we rely on. We therefore didn’t make any additional adjustments for insecticide washout. We now add an adjustment of -20% and -40% to account for declines in piperonyl butoxide (PBO) content reducing the effectiveness of PBO nets in the second and third year following a distribution, respectively.46 We made no changes to account for insecticide washout in chlorfenapyr nets. This reduced cost-effectiveness of programs by 1% to 10%.
  • We revamped our ITN durability analysis. The analysis was expanded to include a wider variety of durability studies beyond just the Permanet 2.0 brand, and new parameters were added to explicitly model how much physical protection nets retain as they physically deteriorate. This changed cost-effectiveness of programs from -5% to 12%.
  • We’re currently soliciting feedback from experts on work we’ve done to understand the discrepancies between our assumptions on net durability vs. other sources. We think we should put some weight on modelling studies (in addition to field durability studies) to account for people overreporting net use due to their knowledge of being in a trial. We think this will reduce cost-effectiveness of programs by 0% to 10%.47
  • We haven't updated the insecticide resistance estimates more generally. That's because we think the CFP nets that we have skewed toward purchasing overcome a lot of the concerns we had previously.48
• Prioritize research on vaccine efficacy in the contexts where we fund vaccination programs. This may include commissioning additional biomarker studies for a range of diseases and engaging with experts and other organizations to better understand potential explanations for the observed discrepancies in efficacy.
  • We have significantly reduced our uncertainty around our observed discrepancies in vaccine efficacy in areas we fund programs. We've conducted desk research, compared our estimates to others49 , and had conversations with experts on the poor results from the "IDInsight" biomarkers pilot.50 We expect to continue conversation on vaccine effectiveness as we talk to more modelers.51
• Consider funding monitoring of these types of issues alongside upcoming grants. For example, talk to Malaria Consortium about funding studies of drug resistance to SMC alongside upcoming SMC distributions.
  • We recently made a $69,050 grant to the Liverpool School of Tropical Medicine to collect data that will inform the design of a potential randomized control trial (RCT) to look at the effectiveness of different combinations of chemoprevention drugs outside of the Sahel.
• Consider whether this broader issue applies to other aspects of our top charity interventions. Who reviews the quality of SMC or VAS? Who checks whether nets, SMC, or VAS have reached their expiration dates?
  • We haven’t prioritized additional research on this issue for SMC, nets, or VAS.
  • We’re fairly confident that for New Incentives’ programs, vaccines are in a usable condition and have not expired.52
• In the future:
  • Develop a process for regularly reviewing and updating key parameters in our cost-effectiveness analyses, particularly those that are likely to change rapidly over time (like increases in insecticide resistance or the spread of the anopheles stephensi mosquito53 ), to ensure that our assumptions remain accurate and up-to-date.
  • Look for cases where program effectiveness is likely to change from one grant to the next and prioritize research on those cases that are likely to be decision-relevant.
  • Consider funding opportunities that would keep us more up-to-date on these parameters.

We haven’t prioritized additional research on these issues.

Insufficient attention to inconsistency across CEAs

From our top charities red team report:

For example, our assumptions about “indirect deaths” (deaths prevented by, say, a malaria prevention program that aren’t directly attributable to malaria) range from 0.75 for SMC and nets to over 2 for water chlorination programs. We haven’t done the work to think through whether these differences are plausible.

Bottom line:

We’ve begun doing consistency checks across CEAs and corrected inconsistencies in development effects. We have not documented any grant decision changes resulting from consistency checks. We have not done further work on indirect effects, though we have made adjustments to our estimates in cases where values were implausibly large.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Conduct more research on indirect effects. We’re uncertain about these estimates in general, both because they vary across programs and because the evidence we have for them has substantial limitations.54 It’s possible we end up concluding that it’s plausible indirect deaths are meaningfully different across programs.
  • We haven’t prioritized further systematic work on indirect effects but have made adjustments to our estimates in cases where our estimates seem implausible. For example, we noticed that in some states in Nigeria, our indirect deaths assumptions implied malaria-related deaths were close to 100% of all deaths. To address this, we modified our indirect deaths adjustment to be smaller in areas with higher direct malaria-attributable mortality. In Lagos, this means we only adjust upward by about 40% instead of 75%, so we now estimate that something closer to 75% of deaths are directly or indirectly malaria-attributable.55
• Correct our development effects estimates to be more consistent across programs.
  • Note from red teaming report: We’ve completed this update since red teaming: our current method for estimating development effects is to use our estimate of the development effects of malaria, calculated here, and adjust it based on how the current program compares to an anti-malaria program. See our VAS estimate and New Incentives estimate. Development effects now account for 20-32% of benefits across Top Charities, taking averages across locations.56 )
• Conduct more regular consistency checks across our programs to ensure that, at a high level, we understand why programs are more or less cost-effective and, for certain key parameters, we understand whether they differ in plausible ways across programs.
  • Note from red teaming report: Since red teaming, we’ve begun implementing consistency checks as a regular part of reviewing CEAs. We describe these checks in more detail in our CEA consistency guidelines. Our consistency checks have not resulted in any changes to grant decisions.

Insufficient focus on simplicity in cost-effectiveness models

From our top charities red team report:

Our CEAs have become too complex, with models like the main CEA for ITNs spanning 225 lines. We think this complexity led to at least two errors. More broadly, high complexity makes it difficult to understand and critique our models.

Bottom line:

We’ve begun including “simple CEAs” for all of our programs that provide a quick overview of main drivers of cost-effectiveness. However, we’ve made limited progress beyond this. We haven’t documented any grant changes directly related to simplifying our models.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Develop simplified, 15-25 line CEAs as a starting point when creating or reviewing cost-effectiveness models. This will help ensure that the core assumptions and calculations are easily understandable and can be quickly assessed for accuracy.
  • From our red teaming report: We’ve now incorporated these into our top charity reports57 and all new grant pages and intervention reports have these simple CEAs.58
• Re-examine our assumption about reducing the interval between campaigns, and explain this adjustment and the key assumptions underlying it more legibly.
  • We simplified this adjustment to address inaccuracies under certain circumstances. We now use a 33% adjustment for a speed up of approximately 1 year, instead of calculating location-specific adjustments.59
• In the future, we plan to apply these same legibility principles and level of scrutiny to all important and complex parameters.
  • We haven’t made much progress on these. We’ve been exploring ways to simplify our nets CEA but have not made major adjustments yet.

Insufficient investigation of some factors our models may miss

From our top charities red team report:

Our models include only rough, ad hoc adjustments for issues like transmission dynamics, and it's possible more complex models by epidemiologists would show meaningfully different cost-effectiveness estimates. If we were to add these types of considerations, we'd want to do it in a way that still preserves legibility and consistency across our models.

Bottom line:

At the time of red teaming, our main question here was about transmission dynamics. We’ve made limited progress on understanding how much we should account for transmission dynamics across programs.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Reach out to vaccine experts, including epidemiologists and disease modelers, to discuss our current approach to modeling the transmission effects of vaccines in the New Incentives CEA. We will seek their input on how we can improve our model to better capture the dynamic impacts of vaccination programs.
  • We've spoken to several modelers and modeling groups (a combination of disease-specific and more general experts), conducted light desk research into transmission effects, and are planning an update to the CEA that will explicitly model these transmission effects. Prior to this work, we thought there was a moderately high chance that we were significantly underestimating the indirect benefits of vaccinations (i.e. we thought we might be understanding benefits by 40-100%), but we now think there is a smaller chance we are missing major upsides (our best guess now is that indirect effects would be roughly a 10-25% benefit on top of mortalities directly averted through vaccination).60 We consider this a first pass, and we'll likely continue discussing indirect effects with more disease-specific modelers, in particular for measles and PCV where we have seen the most conflicting evidence.61
• Consider whether similar transmission questions apply to other programs. Are transmission effects of malaria or diseases averted by VAS similar?
  • We haven’t conducted additional research on this issue. Since red teaming, we’ve incorporated spillovers into our analysis of cash transfers and in the future hope to incorporate this into our consideration of spillovers for other programs. This raises a similar question: Are we consistent in spillover effects across health and non-health programs? We have not investigated this question in depth.
• In the future, get more regular feedback from experts on our modeling assumptions. This will require making our models legible to these experts.
  • We’re continuing to seek regular feedback from experts on challenging questions in our model.62 We’ve begun piloting with other ways to get feedback (e.g., asking experts to input their own parameters) but have found this challenging so far.

Insufficient consideration of what happens when conditional cash transfers for vaccination are stopped

From our top charities red team report:

At the time of red teaming, we had deprioritized investigation of what would happen if we stopped funding New Incentives’ program in a given area. I.e., what would happen to vaccination rates in Northern Nigeria if we removed conditional cash transfers for vaccination? We now think this was a mistake and that we should look more into the potential effects of ending a program we fund.

Bottom line:

We’ve looked into this further and lowered our estimated cost-effectiveness of New Incentives programs by 10%-30% in states we’re considering funding in the future. This has made us somewhat less likely to support expansion of New Incentives’ program into new areas. We’ve also begun considering broader work on what happens after our top charities (or other programs we fund) stop delivering programs.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Conduct additional desk research on the effects of removing cash incentives for vaccinations on long-term vaccination rates. We’ve started this work and are planning to update our intervention report. We may consider funding this research if it doesn’t exist.
  • We’ve updated our analysis. Our best guess is that removing conditional cash transfers for vaccination leads to a small decline in vaccination rates, relative to what would’ve happened if there had been no cash transfer program. We estimate this lowers cost-effectiveness by 11%-30% in states where New Incentives’ program does not currently operate.63
• In the future, consider other opportunities to learn what happens after our top charities (or other major programs we fund) stop delivering their programs. Three examples:
  • We were funding SMC in Chad through Malaria Consortium and recently decided to provide “exit funding” to transition out of these programs by 2024.64 A few years from now, we could look back to ask: What happened after we exited? Is there anything we could’ve done differently to make for a smoother transition? Do we still think exiting was the right choice?
  • If there are negative effects of removing cash incentives for vaccinations, are there approaches that can mitigate these negative effects?
  • Are there negative effects of removing other programs like vouchers for chlorination, free ORS and zinc, and mobile conditional cash transfers for vaccination programs?
    • We have made limited progress on the specific examples mentioned above, but have spent time considering the negative effects of removing VAS programs:
      • We received feedback from an expert who expressed concern that removing VAS from one of the annual Maternal, Newborn, and Child Health Weeks (MNCHW) rounds and moving it to Seasonal Malaria Chemoprevention (SMC) campaigns (through our grant to support co-delivery of VAS and SMC in Nigeria) could be harmful in the long run because it might be difficult to reintegrate VAS back into government platforms if SMC campaigns were to end. However, we have not made additional progress in investigating the long-term impacts of removing VAS from the MNCHW at this point.
      • We're considering tracking what happens to VAS coverage following the end of Helen Keller funding as part of a potential exit grant for programs in Cote d’Ivoire and Benue and Ebonyi states in Nigeria.

Insufficient follow-up on potentially concerning monitoring and costing data

From our top charities red team report:

For example, in our VAS cost-effectiveness analysis, we used aggregate data instead of country-specific costs for four countries where the data seemed implausibly low. While this conservative approach avoids relying on potentially unreliable data, it misses an opportunity to understand how some countries may be achieving lower costs or whether this is indicative of lower quality monitoring and program implementation in these countries.

Bottom line:

We've made some progress on addressing concerns about monitoring and costing data. We've investigated specific data quality issues with AMF's Zambia campaign and found that data systems have improved since the 2018 distribution. For Helen Keller's VAS programs, we've analyzed cost variation across countries and implemented improvements to their coverage surveys. We've completed two program lookbacks (for VAS and New Incentives). These checks have generally made us more confident in potential grants, but we don’t think any of this work has led to changes in grant decisions.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• More regularly review monitoring data we receive from grantees and ask questions about any irregularities or lower-than-expected coverage figures. For example, are potential issues in the Zambia nets campaign likely to apply to other campaigns? What drove lower-than-expected coverage in these cases?
  • We looked into this during the most recent AMF investigation. AMF argued that poor-quality distribution data resulted from poor data systems rather than significant, widespread issues with the distribution itself and we found this to be reasonably credible.65 It appears that the data systems in Zambia have improved since the 2018 campaign (i.e. data from the 2023 Zambia distribution indicates that 97% of purchased nets were distributed).66 The 2018 data quality incident did not affect our evaluation of the Zambia grant and we did not incorporate this into our cost-effectiveness analysis.67
  • We haven’t found any other irregularities but also have not done a systematic review to find them. We plan to conduct a more systematic review of monitoring and evaluation across our top charities in the near future.
• Publish updates on how successful previous programs we’ve funded were. For example, for large nets campaigns, how many nets were distributed, compared to expectations? What can we learn about the successes or failures of previous net distributions?
  • We’ve published two lookbacks so far on VAS and New Incentives.
  • We have not yet published lookbacks on ITNs or SMC.
• Talk to Helen Keller about the extent to which program costs vary across countries and whether variation in costs capture genuine differences or just noise.
  • We’ve spoken to Helen Keller about key drivers of costs of its VAS programs and run our own analysis on the same. Our best guess is that cost differences across countries are genuine and capture variation in population size (higher population means cost per person are lower since fixed costs are spread over more recipients), ability to co-deliver with immunization campaigns (co-delivering is cheaper because it leverages costs), and lower payments or lump-sum incentives for some areas.68
  • In 2024, we discussed with Helen Keller improvements to its coverage surveys that would address some of our concerns about incentives and potential bias. Since Helen Keller agreed to make updates in response to these concerns in February 2024, we have seen initial indicators that they have updated their processes, such as having independent monitors re-visit 10% of clusters and conduct a household census,69 though we have not fully vetted these changes. Going forward, continued improvement in monitoring will be important for our ongoing support. We also plan to consider making a grant to a third party evaluator who can triangulate Helen Keller's monitoring.
• In the future, publish more regularly on how the actual costs of distributions compared to the costs we’ve budgeted. How far off were our estimates? If they changed, what drove that? How can we apply this to future distributions?
  • We have included this in our first lookbacks on VAS70 and New Incentives71 .

Insufficient sideways checks on coverage, costs, and program impact

From our top charities red team report:

We haven't regularly cross-checked our data on program coverage, costs, and impact against external sources like DHS surveys or other organizations' reports. While initial checks during red teaming didn't reveal major discrepancies, we think more regularly doing these checks could increase confidence in our work and surface errors.

Bottom line:

We’ve continued to look out for ways to sideways check coverage, effect size, burden, costs, and other parts of our work. While we haven’t found any issues that overturned the cases for or against grants, we think these checks make our work more robust than relying on single (often fragile) data points or perspectives.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Look for additional opportunities to use DHS surveys or other sources to confirm coverage estimates.
  • A few cases where we’ve prioritized additional work since red teaming:
    • We have conducted sideways checks on vaccination coverage and treatment effects for New Incentives’ program. These checks have not materially changed our decision making. We compared the 2023-24 DHS state-level results against New Incentives’ baseline surveys, and generally observed changes in coverage that were smaller than those found by the New Incentives coverage surveys. However, the DHS state-level results are limited by small sample size and in many cases, we do not think the state-level results are very representative of New Incentives’ program because they operate in only select areas within a state. We have prioritized doing additional work (having more conversations with stakeholders who have more local knowledge and looking for other data to triangulate and assess our estimates) in states where there was the largest divergence between DHS and New Incentives run surveys.
    • We now incorporate DHS coverage surveys directly into our ITN usage estimates.72
    • We’ve made a $926k grant to The Development Innovation Lab and Innovations for Poverty Action for external chlorine surveys to provide an independent coverage estimate for Evidence Action's safe water programs in Malawi and Uganda. We think this survey will produce a coverage estimate that we have more confidence in than Evidence Action's internal M&E, and inform our view about the promisingness of Evidence Action's whole portfolio of water programs. It will also be an interesting datapoint about grantee-collected vs independent data more broadly. We expect to do this for other programs, too.
  • Look for additional opportunities to sense check other parameters, such as burden estimates, program effectiveness, or costs.
    • Some examples of work done beyond red teaming:
      • Sense-checking cost-effectiveness against peer programs. For example, we looked into how oral rehydration solution and zinc distribution through New Incentives compares to other interventions GiveWell thinks are promising.
      • Triangulating high-level estimates of cost per death averted. For example, we calculated some rough estimates of deaths averted by global malaria funding and sense checked those against others’ estimates to understand if we disagree and, if we do, why.73
      • Sense-checking effect sizes. We estimate effect sizes of programs like SMC, New Incentives, VAS, or nets on mortality by relying on RCTs. We’ve incorporated sections on this evidence in all of our top charity intervention reports.
      • Triangulating program cost data. See the above section.
      • Triangulating sources on disease burden. See the above section.

We plan to continue to look for these checks so our decisions are based less on one, complex CEA and more on multiple perspectives.

Insufficient attention to high uncertainty regarding VAS

From our top charities red team report:

We think the case for VAS is categorically more uncertain than our top charities, given fundamental questions about VAS’s effect on child mortality. We need to consider how this uncertainty will influence our grantmaking.

Bottom line:

We continue to fund VAS despite high uncertainty, and we're exploring additional research to address key uncertainties, including exploring the possibility of funding RCTs on VAS effectiveness, conducting additional VAD surveys, and investigating vitamin A fortification as an alternative approach.

More detail on what we said we’d do to address this issue and what we found (text in italics is drawn from our original report):

• Consider how much funding we’re willing to direct to VAS programs in light of these uncertainties.
  • While we think there are several uncertainties about VAS, we still plan to consider grants that meet our cost-effectiveness bar. Alongside, we plan to do work to address key uncertainties where we can, including:
    • Exploring the possibility of funding randomized controlled trials to better measure VAS effectiveness, potentially using stepped-wedge designs to address ethical concerns about withholding vitamin A.
    • Investigating ways to conduct additional vitamin A deficiency (VAD) surveys, which are currently limited by high costs ($1.5-$4 million per country).
    • Investigating vitamin A fortification, as discussed (above).
  • Publish a stance on how we approach this type of uncertainty in our grantmaking.
    • We haven't published a specific position on this issue. However, our general approach is reflected in how we structure our giving funds. Our Top Charities Fund supports programs where we have high confidence in the evidence base, while our All Grants Fund allows us to support promising opportunities with more uncertainty that could still be highly impactful.

We believe it's important to investigate and make grants even in areas with substantial uncertainty, particularly when these programs could be highly impactful and when grants might help reduce that uncertainty over time.

Sources

• 1

  Our rough estimates are in this spreadsheet. Attributing specific grant changes directly to red teaming is challenging, since some of these changes might have happened without red teaming, and some issues were already being addressed before red teaming highlighted them. In these estimates, we’ve tried to assign “credit” due to red teaming (0%-100%). For example, if we decided against a $10m grant and estimated that red teaming deserves 70% of the credit, we count that as $7m attributed to red teaming.

• 2
  • We previously assumed that people were no more or less likely to obtain nets from other sources than they were during original studies testing the impact of ITNs on malaria, which were predominantly conducted in the 1980s and 1990s. In these trials, around 5% to 10% of people had access to nets in the control group. By using these control groups as a stand-in for the counterfactual, we effectively assumed that 5% to 10% of people would got nets in the absence of any mass distribution campaign. GiveWell, Mass Distribution of Insecticide-Treated Nets (ITNs), April 2024 version.
    • In particular, baseline coverage in the control group was “very low” in Marbiah 1997; 6% in Nevill et al. 1996, 9% in Sexton et al. 1990, and 12% in Sochantha et al. 2006. See here.
  • Our new estimates (25-30%) do not account for net durability/retention, but account for usage, and hence represent counterfactual net usage. Our old estimates (5-10%) do not account for durability, retention, nor usage, and hence represent counterfactual net access. Hence, these estimates are not perfectly comparable.

• 3

  We discuss the mechanisms of this adjustment in our forthcoming Nets intervention report update.

• 4

  We are yet to publish the CEA associated with our December 2024 grant to Against Malaria Foundation for net distributions in DRC, Nigeria, Chad and Zambia, that contains these figures.

• 5

  Our initial value of information calculations suggest VAD surveys likely wouldn't meet our 10x cost-effectiveness threshold. However, we may revisit this in the future.

  • VAD survey costs can be substantial and vary significantly based on geography, population density, topography, survey design, and required precision of estimates. GiveWell in conversation with Global Alliance for Vitamin A (GAVA), August 2024. (Unpublished)
  • GiveWell's best guess is that survey costs can range from $1.5-$4 million.
  • While we have not funded VAD surveys since red teaming, we have updated our estimated VAD rate in Nigerian children 6-59 months from 21.6% to 13.5%. This is in response to new evidence we reviewed. See our write-up on this change here.

• 6

  The decrease in our adjustment is due to the most recent DHS survey showing decreases (or smaller increases) in vaccination coverage. We plan on revisiting this analysis as new data becomes available, and will compare our estimates to actual coverage achieved in areas where New Incentives operates to check the accuracy of our assumptions. We are planning to publish a detailed update on this change in the near future.

• 7

  For examples of explicitly stating our assumptions about underlying coverage rates, see this section of our report on mass distribution of ITNs, this section of our report on seasonal malaria chemoprevention (SMC), this section of our report on conditional cash transfers for routine vaccination, and this section of our report on vitamin A supplementation.

• 8

  We will publish details on this grant in a forthcoming grant page.

• 9

  “We now estimate that the program is significantly less cost-effective than we originally estimated (about three to eleven times as cost-effective as unconditional cash transfers, varying by state, down from 24 times in Nigeria as a whole; we did not have state-specific cost-effectiveness estimates in 2021).” GiveWell, “Lookback: Expansion Grant for Helen Keller International’s Vitamin A Supplementation Program in Nigeria.”

• 10

  GiveWell attended the Alliance on Malaria Prevention (AMP) 2025 Conference where routine distribution of nets was discussed.

• 11

  Conversations between GiveWell and various malaria experts, April-May 2024. (Unpublished)

• 12

  GiveWell in conversation with AMF, March 2024. (Unpublished)

• 13

  Total grant amounts are adjusted for the % credit we attribute to red teaming in our grant decision making. See this spreadsheet.

• 14

  We outline some of these instances in this forum post.

• 15

  See the “Implications for our grants” section of GiveWell, “Guidance on Burden.”

• 16

  We discuss this decision in a forthcoming nets lookback report.

• 17

  Our updated approach to mortality estimation in Cameroon brought our cost-effectiveness estimate for the program above our 8x bar, leading to a decision to renew approximately $5 million in funding for the program. It is unlikely that we would have funded the program in Cameroon without this change.

• 18

  GiveWell, “Maternal Mortality Ratio Estimates in Nigeria - 2024,” (Unpublished). We are yet to formally write/publish our plan to incorporate multiple maternal mortality sources moving forward, which we plan to do alongside quick evidence analysis of interventions in the space.

• 19

  We are using this approach in a current grant investigation. We are yet to publish details on this, as we do not typically share details on ongoing investigations.

• 20

  See these calculations here.

• 21

  See these calculations in GiveWell's Analysis of All-Cause Under-5 Mortality in Nigeria (NDHS23/24 update)

• 22

  Incorporating UN IGME estimates led us to increase our estimate of malaria mortality by about 70% in Chad. This caused us to make a $3 million grant to seasonal malaria chemoprevention (SMC) in Chad and $25 million grant in insecticide-treated nets (ITNs) in Chad that we likely would not have made otherwise. See the “Implications for our grants” section of GiveWell, “Guidance on Burden.”

• 23

  “Insufficient attention on anemia burden. We estimate ~60-70% of children under 5 in India have anemia, based on data from the Institute for Health Metrics and Evaluation’s (IHME) Global Burden of Disease (GBD) study. While this estimate aligns roughly with India's National Family Health Survey (NFHS), we still have some concerns. For example, one expert noted that capillary blood draws might systematically overstate anemia rates if blood samples are diluted through improper collection techniques. There may be other data sources we should triangulate against, too.” GiveWell, What We Learned From Red Teaming Our Iron Grantmaking.

• 24

  For example, during our investigation into supporting MiracleFeet, we initially used their World Bank based population projections to estimate that approximately 3,100 children were born with clubfoot annually in the Philippines. However, MiracleFeet has since learned that the government of the Philippines reports significantly lower actual birth rates than those projected by the World Bank, resulting in a reduced projection closer to 1,700.

• 25

  We use vaccination rates from DHS and MICS surveys to estimate baseline coverage in locations where New Incentives has not conducted baseline surveys, to estimate measles vaccination coverage, and to understand how coverage would have changed over time in Nigeria. The state-level results we rely on (e.g. here) are often based on very small sample sizes and have wide confidence intervals. Some of the state-to-state and year-to-year variation may be due to statistical noise rather than true differences in coverage. We are in the process of gathering more qualitative input on the vaccination landscape and reviewing the survey methods in more depth to better understand their reliability and comparability.

• 26

  For example, for some of our technical assistance grants, we've noticed some discrepancies between coverage data reported by our grantee and coverage data reported by the government. We plan to publish on this separately.

• 27

  kids who are eligible account for ~60% of deaths during the period, 60% of those eligible kids get vaccinated, and the weighted average efficacy is around 40%. 60%*60%*40%=~14%.

• 28

  For example, see this row in a recent version of our cost-effectiveness analysis for seasonal malaria chemoprevention.

• 29

  “Conversely, mortality was significantly reduced in the combinations of pirimiphos-methyl IRS with pyrethroid-PBO ITNs (55–59%) compared to pirimiphos-methyl IRS alone (77–78%, p < 0.001), demonstrating evidence of an antagonistic effect when both interventions are applied in the same household.” Syme et al., 2022.

• 30

  See our write-up here.

• 31

  “15% of the VAS mortality benefit is expected to be displaced by aMDA.” GiveWell, External validity adjustment for azithromycin mass drug administration, September 2024

• 32

  For example, see this row in a recent version of our cost-effectiveness analysis for seasonal malaria chemoprevention.

• 33

  See this write-up for our initial analysis.

• 34

  In an April 2025 podcast, GiveWell’s Director of Research Teryn Mattox noted "GiveWell has historically kind of been in this comfortable position of being a funder more at the margin... assuming that a lot of these basic healthcare services are being provided, how can we support at the margin to enhance those programs to save more lives. That has meant that we have not looked as systematically at more integrated health programming that now is very much at risk." GiveWell, “Making Cost-Effective Grants Amid Uncertainty,” April 23, 2025.

• 35
  • Malaria:
    • SMC Alliance meeting in Abuja, Nigeria
    • Alliance for Malaria Prevention Annual Partners Meeting in Nairobi, Kenya
    • Multilateral Initiative on Malaria in Kigali, Rwanda;
    • The American Society of Tropical Medicine and Hygiene (ASTMH) in New Orleans.
  • Vaccines:
    • World Vaccine Congress in Washington, D.C
    • Gates HPV Convening in Seattle
  • Water and livelihoods:
    • World Water Week in Stockholm, Sweden
    • World Water Forum in Bali, Indonesia
      A conference on Economic Growth in LMICs in Washington, D.C.
  • New Areas
    • Global Health Market Shaping conference in Barcelona, Spain
    • Skoll Forum in Oxford
    • Gates Azithromycin conference in Nigeria
    • REACH Network Annual Regional Meeting in Abuja, Nigeria.
  • Cross-cutting
    • UN General Assembly in New York
    • Center for Global Development Market Shaping Accelerator in Washington, D.C.
    • Y-RISE conference in Cayman Islands.

• 36

  GiveWell, internal analysis (Unpublished)

• 37

  GiveWell, “Uduma — In-line Chlorination Pilot (November 2024).”

• 38

  In 2024, our team made site visits to Mozambique, Kenya, DRC, India, Tanzania and Uganda.

• 39

  “In investigating this grant, we relied primarily on conversations with PATH and with other expert stakeholders, including people in the countries targeted by this grant” GiveWell, “PATH — Technical Assistance to Support Malaria Vaccines Rollout (March 2024).”

• 40

  We are yet to publish details on this potential grant, as we do not typically share details on ongoing investigations.

• 41

  See the “What did experts say?” section of GiveWell, “What We Learned From Red Teaming Our Iron Grantmaking.”

• 42

  This grant was made in February 2025, since it was after our 2024 calendar year (which runs through January 2025). We’re including in this lookback because it was informed in large part by red teaming and the grant was made close to the end of 2024.

• 43

  GiveWell, Impact of vitamin A fortification on under-5 all-cause mortality (Unpublished)

• 44

  "Supplementation frequency may be a meaningful predictor of the impact of VAS on child mortality. VAS trials administered vitamin A to children at different frequencies ranging from weekly to every ten months.185 We have identified a possible relationship between supplementation frequency and VAS effectiveness, with more frequent supplementation yielding a larger reduction in child mortality. This relationship is not statistically significant in our initial analyses including the trials from the Imdad et al. 2022 meta-analysis (p = 0.17; R2 = 0.12),186 but we nevertheless believe it is probably true because it is supported by biological plausibility187 and two additional fortification and diet advice trials with large effect sizes not included in Imdad et al. 2022.188 The relationship suggests that vitamin A fortification is more than twice as effective as VAS every six months, per cumulative unit of vitamin A delivered. We are following up on this finding, and we plan to adjust our VAS CEA for it in the future. In contrast, in the same set of trials we did not identify a relationship between cumulative vitamin A dose and reduction in child mortality (p = 0.72; R2 = 0.0008).189” See this section of our report on Vitamin A supplementation.

• 45

  For example, see GiveWell, Short-Term SMC Campaign Gaps in Guinea, Mali, Cameroon, Côte d'Ivoire, Togo, and Benin (April 2025)

• 46

  See this adjustment in our cost-effectiveness analysis here.

• 47

  We are yet to publish this analysis, and plan to in the future.

• 48
  • GiveWell, “Chlorfenapyr vs. PBO nets BOTEC,” September 2022. (Unpublished). This analysis relied on evidence from a trial in Tanzania in 2022 (Mosha et al. 2022)
  • GiveWell in conversation with an expert on ITNs, April 11, 2024, (Unpublished).

• 49

  “We've also lightly reviewed other studies in Nigeria that looked at whether infants developed the biomarkers associated with immunity post-immunization.” GiveWell, “New Incentives (Conditional Cash Transfers to Increase Infant Vaccination).”

• 50

  “In 2024, we spoke with an expert on measles biomarkers tests who thought the most likely explanation for the poor results was inadequate sample collection and difficulties around interpreting the results from commercial testing kits (more details in the footnotes). Based on this discussion, we think it’s most likely that the poor results were a result of the test failing to detect immunity rather than a failure of the vaccines. We continue to place some weight on these results because we can’t definitively rule out the possibility that the results were indicative of problems with vaccine quality.” GiveWell, “New Incentives (Conditional Cash Transfers to Increase Infant Vaccination).”

• 51

  “As of October 2024, we’ve spoken with several other modelers and conducted light checks of our model against others such as Carter et al. 2024. This work is still ongoing, but we’ve identified several common divergences between our analysis and others that we are investigating further.” GiveWell, “New Incentives (Conditional Cash Transfers to Increase Infant Vaccination).”

• 52

  “Field Officers conduct vaccine vial monitor (VVM) checks at partner clinics to ensure that vaccines are in a usable condition and have not expired. Field Officers randomly select one vial of each directly incentivized vaccine during immunization days and check it is in a usable condition. Where any vaccines are not usable, Field Officers request that clinic staff check all the vaccines at the clinic. They also record data on these checks in a log so that supply side officers can verify the VVM assessments and escalate issues to respective government agencies when warranted.” GiveWell, “New Incentives.”

• 53

  “Anopheles stephensi is a mosquito species that is capable of transmitting both Plasmodium falciparum and P. vivax malaria parasites. Unlike the other main mosquito vectors of malaria, it thrives in urban and man-made environments. Originally native to parts of South Asia and the Arabian Peninsula, An. stephensi has been detected, to date, in 7 countries in the African continent.”
  World Health Organization, “WHO initiative to stop the spread of Anopheles stephensi in Africa, 2023 update.”

• 54

  See the “Indirect malaria mortality” section of our report on ITNs, the
  “Non-malaria deaths indirectly averted” section of our report on SMC, and the “Adjustment for all-cause mortality effect” section of our report on New Incentives.

• 55

  See more about this adjustment in the “Example comparing malaria-attributable mortality to all-cause mortality” section of GiveWell, “Guidance on Burden.”

• 56

  We estimated the percentage of benefits from development effects for each top charity intervention by averaging across relevant locations included in our analysis. However, this does not always cover every location we analyze for that intervention. This is because not all locations are representative: some locations have not received funding from us, or we decided to discontinue funding in those areas. For clarity, we rely on our intervention reports to determine the relevant locations and provide links to explanations in those reports.
  For mass distribution of nets, 31.5% of benefits come from development effects on average across DRC, Guinea, Nigeria (Global Fund states), Nigeria (PMI states), Togo, and Uganda. See this row of our nets CEA for the figures averaged and footnote 23 of our report on nets for the reasoning behind using these locations.
  For SMC, 26% of benefits come from development effects on average across Burkina Faso, Togo, and various states in Nigeria. See this row of our SMC CEA for the figures averaged and footnote 25 of our report on SMC for the reasoning behind using these locations.
  For VAS, 20% of benefits come from development effects on average across Burkina Faso, Cameroon, Côte d'Ivoire, DRC, Guinea, Madagascar, Mali, Niger, and five states in Nigeria: Adamawa, Benue, Ebonyi, Nasarawa, and Taraba (for Helen Keller) and Chad (for Nutrition International). See this row of our VAS CEA for the figures averaged and footnote 10 of our report on VAS for the reasoning behind using these locations. NB: We use an estimate of 20% across the board for VAS.
  For New Incentives, 20% of benefits come from development effects on average across Bauchi, Gombe, Jigawa, Kano, Katsina, Kebbi, Sokoto, and Zamfara. See this row of our New Incentives CEA for the figures averaged and footnote 14 of our report on New Incentives for the reasoning behind using these locations.
  Previously, these values ranged from 10% on average for current New Incentives sites to 43% on average for nets. See GiveWell, “What We Learned From Red Teaming Our Top Charities.”

• 57

  The simple CEAs are included in the following linked sections of our nets report, SMC report, New Incentives report, and VAS report. The simple CEAs are also included in the broader CEA spreadsheets: nets simple CEA, SMC simple CEA, New Incentives simple CEA, and VAS simple CEA.

• 58

  For examples of simple CEAs in new non-top charity grant pages, see our grant to IRD Global for a breastfeeding reminders pilot, our grant to ALIMA for a malnutrition treatment in Chad, and our grant to the Clinton Health Access Initiative for oral rehydration solution/zinc co-delivery.
  For an example of simple CEAs in our intervention reports, see our intervention report on oral rehydration solution and zinc. For all intervention reports, see this table.

• 59

  This adjustment was derived from taking a rough average of the “Adjustment for program impact being to move distributions closer together” in DRC, Nigeria and South Sudan from this version of our CEA (((-29% + -35% + -37%)/3) = -33.7%) We are yet to publish a version of the CEA that applies the new blanket -33% adjustment.

• 60

  This 10-25% includes benefits we have explicitly modeled for other unvaccinated children under 5 as well as rough adjustments we apply for other populations.

• 61

  We plan to discuss these findings in our forthcoming intervention report update.

• 62

  Some examples include:

  • We’re currently soliciting feedback from experts on work we’ve done to understand the discrepancies between our assumptions on net durability vs. other sources. (Unpublished)
  • Our vaccines team sought expert input on modeling spillovers in our programs. (Unpublished)
  • In March and April 2025, we collected forecasts on how US global health spending will change over the next few years. We think these forecasts may help us make more informed decisions about where to allocate our funding and identify potential gaps that may need to be filled. GiveWell, Forecasts on US Global Health Funding, May 2025.

• 63

  See the “What happens to vaccination rates if the program ends?” section of GiveWell, “New Incentives (Conditional Cash Transfers to Increase Infant Vaccination)”

• 64
  • “This new $87.5 million grant extends the program’s funding runway in all four countries. It consists of… $6.5 million for Chad (7% of the grant). This amount, intended to be exit funding, is based on adding one additional year of funding at the current scale (as of 2022) of Malaria Consortium's support to the program in Chad.” GiveWell, “Malaria Consortium — SMC Renewal in Nigeria, Burkina Faso, Chad, and Togo (January 2023)”, April 2023.
  • See also the “Risks of discontinuing funding in Chad” section of the same page.

• 65

  Correspondence from AMF to GiveWell, November 2024. (Unpublished)

• 66
  • AMF monitoring data for Zambia, 2023. (Unpublished)
  • We find stakeholders’ claims that data systems have been substantially changed to be credible, particularly given that digitalization represents a substantial shift to data-collection systems.

• 67

  We believe that the 2026 campaign is significantly more likely to resemble the 2023 campaign than the 2018 campaign in terms of the quality of the distribution as well as the data. As a result, the 2018 data quality incident does not affect our evaluation of this potential Zambia grant, and is not incorporated into our cost-effectiveness analysis.

• 68

  Correspondence from Helen Keller to GiveWell, November 2024. (Unpublished)

• 69

  See this footnote in our lookback for more detail.

• 70

  “Spending has been largely in line with expectations. Roughly assuming even spending over the grant period, we would have predicted that Helen Keller would have spent around $5 million from mid-2021 to mid-2023. In actuality, we estimate that Helen Keller spent $5.5 million over this period." GiveWell, “Lookback: Expansion Grant for Helen Keller International’s Vitamin A Supplementation Program in Nigeria,” 2025.

• 71

  “The increase in our estimate of cost-effectiveness is driven primarily by a large drop in cost per child enrolled (from ~$39 in 2020 to ~$18 in 2024). We think this is due to economies of scale, efficiency efforts by New Incentives, and the devaluation of the naira.” GiveWell, “Lookback: Grants to New Incentives' Conditional Cash Transfer Program for Childhood Vaccination,” 2025.

• 72

  See GiveWell’s analysis of ITN usage following mass campaigns for more detail.

• 73

  We triangulated our estimates with those derived by the Center for Global Development and did not find meaningful discrepancies. We have not published this analysis.