In a nutshell
Between 2016 and 2022, GiveWell recommended three grants totaling ~$13 million to support Results for Development’s (R4D’s) program to improve treatment for children with pneumonia in Tanzania. The key objectives of these grants were to:
- Improve the supply of amoxicillin-DT (a drug used to treat childhood pneumonia) in government health facilities
- Transition the financing and procurement of amoxicillin-DT to the government of Tanzania.
With funding from GiveWell, R4D directly procured amoxicillin-DT in the early stages of these grants, and then shifted toward providing technical assistance. This work included helping the government with demand forecasting, procurement, and supply chain monitoring. (more)
This page provides a lookback on those grants. We’ve assessed this grant’s performance against initial expectations and identified key lessons learned. This shallow assessment fits into a broader cross-cutting assessment of our technical assistance grantmaking, which you can read about here.
Overall, we think we would make these grants again, as we think the grants were plausibly as cost-effective as we modeled at the time of making them. Though there are still wide uncertainty intervals, we generally feel more confident that this grant’s implementation targets were met compared to other technical assistance grants, as monitoring data from the program are relatively consistent and we have better (albeit still speculative) ways to investigate the counterfactual.
Reasons we think implementation has gone well:
- Based on independent coverage surveys commissioned by R4D, the percentage of Tanzanian government health facilities with amoxicillin-DT in stock increased from 48% (in 2017) to 90% (in 2022). This seems broadly consistent with government data – though we haven’t been able to access government data from before 2020, nor do we have a reliable independent measure of coverage before 2017 (when R4D’s work started). (more)
- Via independent coverage surveys, R4D also tracked the availability of other commodities over time, which we used to approximate what amoxicillin-DT would have been in the absence of R4D’s support. This data implies that the availability of amoxicillin-DT increased faster than other commodities (e.g., zinc or oral rehydration salts) – suggesting a counterfactual improvement in coverage – though we think there are limits to what we can learn from this. (more)
- Based on data R4D has sent us, as of 2024, the procurement of amoxicillin-DT has 100% transitioned to resources mobilized by the government (either through domestic resources or funds raised from other donors) (more)
- When we visited this program in Tanzania in September 2024, we spoke to the Ministry of Health official responsible for overseeing this program. He generally spoke positively of R4D’s support, and also mentioned benefits that we had not initially anticipated, such as R4D helping to improve the government’s demand forecasting methodology for commodities beyond amoxicillin-DT (e.g., bed nets and antiretroviral drugs) (more)
- Given the changes in the percentage of health facilities that had amoxicillin-DT in stock after we made these grants, we think these grants were plausibly around our cost-effectiveness threshold (~10x our benchmark). However, these estimates have wide confidence intervals and we haven’t prioritized making other updates to assumptions in our CEA. (more)
Broader takeaways and lessons for our grantmaking: (more)
- We should anticipate programs evolving differently than expected, and not place too much stock in a given theory of change playing out as expected. Practically, when we investigate these grants, we think this means we should spend less time on cost-effectiveness modeling (which can realistically only capture a handful of pathways to impact), and more time investigating factors like the strength of a grantees relationship with the government, which we’d expect to more robustly predict impact in a wider range of scenarios
- We had to make an unexpected $1 million top-up (exit) grant due to unforeseen disruption caused by COVID-19. While we think this decision was reasonable, we’ve had to make top-up grants for other TA grants too, and think this suggests we should build more contingency into our budgets
Published: January 2026
Background
Between 2016 and 2022, GiveWell recommended three grants totaling ~$13 million to support Results for Development’s (R4D’s) amoxicillin-DT program in Tanzania. We made a ~$6 million grant in 2016, ~$6 million grant in 2019, and a ~$1 million (exit) grant in 2022. These grants had two key objectives: to increase the supply of amoxicillin-DT in government health facilities in Tanzania and to transition the financing and procurement of amoxicillin-DT to the government.1
Amoxicillin-DT is recommended by the WHO as the first-line treatment against childhood pneumonia.2 By sustainably increasing the supply of this drug in health facilities, we thought these grants could reduce the number of pneumonia deaths in Tanzania.
Our early grants were mostly geared towards commodity financing (i.e., purchasing amoxicillin-DT) and technical assistance on forecasting and procurement, as the government faced a funding shortage for amoxicillin-DT procurement following the expiry of a UNICEF supply contract in 2016.3 Our later grants were more geared towards transition and technical assistance – e.g., funding consultants to work with the government to improve and strengthen their processes and systems related to demand forecasting, procurement, and management of the amoxicillin-DT supply chain.4
Would we have made these grants again, knowing what we know now?
Yes, though with important caveats about our confidence level. We think these grants were plausibly as cost-effective as our current cost-effectiveness threshold (~10x), based on monitoring and evaluation (M&E) data we’ve seen and conversations with government stakeholders (more).
How did implementation go?
Coverage and financing estimates
Coverage
In our grants to R4D, we earmarked funding for R4D to outsource data collection to an independent firm (EDI Global).5
Throughout 2017, enumerators visited a randomly selected, nationally representative sample of health facilities in Tanzania, where they asked the facility staff whether amoxicillin-DT was in-stock and directly observed whether amoxicillin-DT was on the shelves. For the second phase of the facility surveys from 2020-2022, data collection transitioned to phone surveys to save costs and to more feasibly collect data with greater frequency. EDI would call up health facilities and ask whether they had amoxicillin-DT in-stock. The same sample of facilities was maintained across all survey rounds from 2017-2022.
This data shows that the percentage of health facilities with amoxicillin-DT increased over time, from 48% in March 2017 to 90% in March 2022 (when data was last collected).6
One limitation of this data is that we don’t have a good measure of amoxicillin-DT availability before R4D’s work started in 2016. The first shipment of GiveWell-funded, R4D-procured amoxicillin DT arrived in Tanzania in December 2016.7 We could have delayed the distribution of this shipment while we established a baseline availability measure through our facility survey, but we chose not to as it didn’t feel ethical to delay procurement for the sake of establishing this baseline measure. Another limitation of the data is that the transition to phone surveys might have led to biased responses, as we were no longer able to physically verify stock.
Data from these surveys are generally in agreement with the government’s data on amoxicillin-DT availability. Rather than surveys, these estimates were calculated from the Ministry of Health’s database for supply chain monitoring, the Electronic Logistics Management Information System (eLMIS), which calculates availability from health facility reports submitted every month reporting their stock on hand.8 These data imply slightly higher availability of amoxicillin-DT compared to the surveys, though the difference is fairly small (less than 10 percentage points).
For the independent surveys, we also asked R4D to track the availability of other commodities that weren’t targeted by this grant, to approximate what might have happened to amoxicillin-DT coverage had we not funded this work. Generally, amoxicillin-DT coverage appears to have increased faster than these other commodities, though from a lower initial base.
| Mar '17 | Jul '17 | Nov '17 | May '20 | Aug '20 | Nov '20 | Feb '21 | May '21 | Aug '21 | Nov '21 | Mar '22 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Amox DT | 48% | 52% | 61% | 80% | 69% | 72% | 76% | 86% | 81% | 86% | 90% |
| Peds Amox | 60% | 60% | 66% | 83% | 73% | 76% | 79% | 88% | 83% | 88% | 93% |
| Co-trim OS | 73% | 70% | 71% | 70% | 60% | 72% | 82% | 90% | 84% | 90% | 92% |
| ALU | 97% | 99% | 91% | 92% | 95% | 96% | 99% | 99% | 100% | 99% | 100% |
| Zinc/ORS Co-pack | 69% | 62% | 43% | 44% | 40% | 41% | 51% | 59% | |||
| FeFol | 76% | 72% | 59% | 80% | 92% | 85% | 82% | 90% | |||
| Benzylpenicillin Injection | 84% | 77% | 49% | 49% | 88% | 88% | 92% | 90% | |||
| Paracetamol | 62% | 74% | 63% | 82% | 87% | 88% | 93% | 99% | |||
| Observations | 624 | 622 | 624 | 247 | 247 | 249 | 248 | 249 | 249 | 249 | 248 |
Notes: We made our first grant in May 2016.
One mistake we made was not pre-specifying in advance which commodity we thought would most reliably proxy the counterfactual. R4D have told us that the zinc and oral rehydration salts co-packs share some relevant characteristics to serve as a comparator. Like amoxicillin-DT, external funding for these co-packs also dried up with the expiry of the UNICEF contract in 2016, and so this may approximate what would have happened to amoxicillin-DT availability had R4D not been involved.
This data shows that zinc/ORS co-pack availability slightly declined over time, which might suggest this grant had a positive counterfactual impact on amoxicillin-DT availability.9 However, we think there are limits to what we can learn from this comparison. First, we did not pre-specify zinc/ORS coverage as our estimator of the counterfactual, and it also looks more favorable than the other medicines.10 Second, although zinc/ORS shares some relevant characteristics, there are other reasons it isn’t a perfect proxy. One thing that undermines its role as a counterfactual is that it’s more easily substitutable with other products than amoxicillin-DT. The WHO only recommends amoxicillin-DT in tablet form; they recommend zinc-ORS in co-pack form (i.e. ORS sachet and zinc tablets packaged together) as well as separately (zinc tablets and ORS sachets distributed separately).11 When the UNICEF contract expired for co-packs, the Tanzanian government may have been less incentivized to increase the supply as they would have been for amoxicillin-DT, as they could substitute towards the sachets and tablets separately. If so, using the supply of zinc-ORS co-pack to approximate the counterfactual supply of amoxicillin-DT would overstate the impact of our grant.
Another uncertainty we have about this data concerns the likelihood of coverage increases being sustained once R4D exits. While we don’t have independent survey data beyond 2022, R4D have sent us government data on amoxicillin-DT availability up to late 2024. This data suggests amoxicillin-DT availability dipped through the year as R4D transitioned the funding for the medicine to fully domestic resources and moved towards a more light-touch technical assistance approach. Both GiveWell and R4D plan to monitor this situation in 2025. If coverage increases aren’t sustained, we think this grant could still plausibly have been above our cost-effectiveness bar, but not as cost-effective as they would have been had these increases been more sustained.
Financing
Beyond increasing coverage, another key objective of this grant was transitioning financing and procurement of amoxicillin-DT in Tanzania away from GiveWell (via R4D) and towards resources mobilized by the government.12
Data sent to us by R4D indicates that this was accomplished in 2024, as 100% of the amoxicillin-DT procured by the government was procured using funds they had mobilized themselves.
There was a dip in progress in 2020, when the COVID-19 pandemic squeezed the government health budget. The Ministry of Health asked if we could redirect grant funding earmarked for TA towards direct procurement of amoxicillin-DT, which we agreed to. Since 2020, funding has gradually transitioned to the government. This funding could include procurement financed by domestic resources (i.e., revenue collection, central government funds earmarked for commodities), or other donor funding the government of Tanzania has identified. When we asked R4D about this, they said it was mostly likely the former, as they’d expect to have been made aware if another donor was financing amoxicillin-DT procurement in Tanzania.
Discussions with local stakeholders
We visited Tanzania in September 2024 to meet with R4D and government stakeholders involved in the program. When discussing monitoring and evaluation (M&E) results, R4D gave several examples of specific bottlenecks to amoxicillin-DT coverage that they thought their TA had helped to resolve:
- Demand quantification: In the early stages of the project, R4D’s claims to have improved the government’s process for forecasting demand for amoxicillin-DT. Previously, the government was using an entirely consumption-based approach – they would look at how much amoxicillin-DT had been dispensed in health facilities (this data is tracked in an e-procurement/supply chain monitoring system called ELMIS) and forecast demand based on this. If there are lots of stockouts, they would lead to a reduction in consumption, so the forecasts are going to be an under-estimate of the true need. R4D helped fix this by adapting the existing Excel-based demand forecasting model the government was using.13 The Ministry staff kept consumption as an input, but also used other inputs such as combining population estimates with pneumonia prevalence estimates. As a result, they forecasted 35% higher actual demand across the country. According to both R4D and a government official we spoke to, the government followed R4D’s recommendation and likely ended up procuring more amoxicillin-DT as a result.14
- Market shaping: according to R4D, before the project, there was just one approved supplier of amoxicillin-DT in Tanzania; as of 2025, there are six.15 R4D claims to have played a role in this by supporting the government to identify suppliers (mostly private pharmaceutical companies in India) by reviewing proposals and recommending specific firms. More importantly, the first phase of this project helped them establish a volume guarantee. R4D agreed to purchase large amounts of amoxicillin-DT between 2016-2020, which incentivized private pharmaceutical companies to agree to manufacturing contracts.
We also spoke with the chief pharmacist responsible for amoxicillin-DT procurement in the Ministry of Health, who was R4D’s main point of contact throughout the course of this grant. We met with the official without R4D present. He generally spoke favourably about R4D’s work, and specifically mentioned their help with revising the demand quantification tools in the early stages of this project. He also said that the government had gotten a lot of value out of the later stages of the project, when the systems were relatively well-established for amoxicillin-DT procurement/distribution and R4D was able to help with other requests. In particular, he mentioned their support in helping the Ministry transition to “bottom-up” forecasting for other health commodities, including antiretrovirals, antimalarial commodities, and tuberculosis medicines.
How cost-effective do we think these grants were?
Based on a retrospective cost-effectiveness model, we think these grants were plausibly around our current cost-effectiveness threshold (10x), though with wide uncertainty intervals.
| Cost-effectiveness we modeled at the time | Retrospective cost-effectiveness estimates | |||
|---|---|---|---|---|
| 25th percentile16 | Best-guess | 75th percentile | ||
| R4D amoxicillin-DT | 5x17 | 2x | 9x | 13x |
To model cost-effectiveness, we fitted our 2018 CEA into a template we’ve built for modeling technical assistance grants. Key assumptions in this model include:
- Effect of program on coverage: We put some weight on data reported by the government and some weight on amoxicillin-DT coverage implied by the coverage surveys. These two data points triangulate fairly well.
- How coverage would have changed: We estimate that amoxicillin-DT availability would have increased without our grant to R4D, though at a slower rate.18 This is partly because the availability of other commodities in Tanzanian health facilities increased slower than amoxicillin-DT.19 To estimate counterfactual impact, we take the ‘wedge’ between: i) our estimated effect of the program on amoxicillin-DT availability and ii) how we expect amoxicillin-DT availability would have changed otherwise. These assumptions are illustrated below.
- Program costs: We take the sum of our ~$6 million 2016 grant, ~$6 million 2019 grant, and ~$1 million 2022 grant (~$13 million total).
We did not revisit the downstream assumptions in our amoxicillin-DT model – e.g., the percentage of children in Tanzania dying from lower respiratory tract infections, or the effectiveness of amoxicillin-DT in reducing the fatality risk of these infections.
Did we set ourselves up to learn?
Funding data collection on the availability of amoxicillin-DT and non-target commodities helped us to investigate the counterfactual impact of this grant, even though we don’t think this data lends itself to strong conclusions.
One thing that would have made evaluation easier is if we’d established a baseline measure of amoxicillin-DT availability before R4D’s work started. We chose not to do this because we’d heard that the supply of amoxicillin-DT was critically low, and it didn’t feel ethical to delay distribution in order to establish a baseline measure. We think this was the right decision in hindsight, and our future M&E will also be shaped by ethical considerations like these.
Another thing we could have done better was to lay out more explicit milestones when making the grant lay out more explicit milestones when making the grant. In our grant pages, we weren’t clear about exactly when we expected certain activities to transition to the government (e.g., when amoxicillin-DT procurement would be handed over). In future, we plan to be more explicit about our theory of change for our TA grants, which we think will be helpful in looking back on how calibrated we are in predicting what activities need to happen for targeted outcomes to be achieved.
How calibrated were our forecasts?
We made three forecasts related to our 2019 grant, all of which were realized as “no”.
| Forecast | Confidence (%) | Realization date | What happened? |
|---|---|---|---|
| R4D or an R4D program is a top charity | 40% | 12/31/2023 | No |
| R4D or an R4D program is a top charity and we estimate that donations to that program are at least half as cost-effective as the most cost-effective unfunded giving opportunity among top charities (i.e., where we recommend donors give on the margin) | 35% | 12/31/2023 | No |
| R4D or an R4D program is a top charity and we estimate that donations to that program are at least twice as cost-effective as the most cost-effective unfunded giving opportunity among top charities (i.e., where we recommend donors give on the margin) | 5% | 12/31/2023 | No |
Sources
- 1We discuss these objectives in more detail in this section of our 2016 grant page.
- 2
From the WHO fact sheet on pneumonia: “Pneumonia should be treated with antibiotics. The antibiotic of choice for first line treatment is amoxicillin dispersible tablets. Most cases of pneumonia require oral antibiotics, which are often prescribed at a health centre. These cases can also be diagnosed and treated with inexpensive oral antibiotics at the community level by trained community health workers. Hospitalization is recommended only for severe cases of pneumonia.”
- 3Source: GiveWell site visit notes from conversations with R4D and Tanzania Ministry of Health staff on amoxicillin-DT TA - September 1-12 2024 (unpublished)
- 4See this section of our page on our 2019 grant where activities were more focused on advocacy and assistance rather than procurement.
- 5
This was not explicitly stated in our grant page at the time, but was discussed in our conversations with R4D.
- 6
Source: GiveWell’s analysis of monitoring data provided by R4D (unpublished)
- 7
See our grant page for our 2016 grant to R4D here.
- 8
Source: GiveWell site visit notes from conversations with R4D and Tanzania Ministry of Health staff on amoxicillin-DT TA - September 1-12 2024 (unpublished)
- 9
Zinc/ORS co-pack availability was highest in May 2020, the first month for which we have data, and declined to 59% in March 2022, the last month for which we have data. Meanwhile amoxicillin-DTavailability increased from 80% availability to 90% availability over the same time period.
- 10
For instance, iron and folic acid (FeFol) coverage increased from 76% to 90% from May 2020 to March 2022, a greater increase than for amoxicillin-DT over the same period.
- 11
- 12
On our 2019 grant page, we listed advocating to increase the amount of funding the government of Tanzania allocates to purchasing amoxicillin-DT as one of the planned activities of the grant. (more)
- 13
Source: GiveWell site visit notes from conversations with R4D and Tanzania Ministry of Health staff on amoxicillin-DT TA - September 1-12 2024 (unpublished)
- 14
Source: GiveWell site visit notes from conversations with R4D and Tanzania Ministry of Health staff on amoxicillin-DT TA - September 1-12 2024 (unpublished)
- 15
Source: GiveWell site visit notes from conversations with R4D and Tanzania Ministry of Health staff on amoxicillin-DT TA - September 1-12 2024 (unpublished)
- 16In our CEA, we’ve modeled three scenarios with different assumptions about the impact of Evidence Action’s program on coverage:
Our best guess of the effect
A 25th percentile guess, representing what we consider a conservative estimate of program impact
A 75th percentile guess, representing what we consider an optimistic estimate of program impact - 17
December 2018 Amoxicillin-DT CEA
- 18
Our comparison of how we think amoxicillin-DT availability would have changed under the counterfactual scenario (without R4D support) and in the TA scenario (with R4D’s support) can be seen here.
- 19
See this table for data on how availability of other health commodities changed in comparison to amoxicillin-DT.