The Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM) | GiveWell

# The Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM)

GiveWell aims to find the best giving opportunities we can and recommend them to donors. We tend to put a lot of investigation into the organizations we find most promising, and de-prioritize others based on limited information. When we decide not to prioritize an organization, we try to create a brief writeup of our thoughts on that charity because we want to be as transparent as possible about our reasoning. The following write-up should be viewed in this context: it explains why we determined that we wouldn't be prioritizing the organization in question as a potential top charity. This write-up should not be taken as a "negative rating" of the charity. Rather, it is our attempt to be as clear as possible about the process by which we came to our top recommendations.

The last time we examined The Global Fund was in November 2010. In our latest open-ended review of charities, we determined that it was unlikely to meet our criteria based on our past examination of it, so we did not revisit it. We invite all charities that feel they meet our criteria to apply for consideration. The content we created in November 2010 appears below. This content is likely to be no longer fully accurate, both with respect to what it says about The Global Fund and with respect to what it implies about our own views and positions. With that said, we do feel that the takeaways from this examination are sufficient not to prioritize re-opening our investigation of this organization at this time.

Published: November 2010

## Summary

The Global Fund is an extremely large funding mechanism, supporting a broad variety of programs to combat HIV/AIDS, tuberculosis and malaria throughout the developing world. The Global Fund's commitment to transparency is outstanding, and its activities are mostly proven and cost-effective. We do not have a clear enough sense of its project-by-project (or aggregate) outcomes to be fully confident in its impact or to give a cost-effectiveness estimate. Previous reports on the Global Fund to Fight AIDS, Tuberculosis and Malaria:

## What do they do?

The Global Fund funds programs - primarily through developing-world governments - aiming to control HIV/AIDS, TB, or malaria.1 Funding is determined by a grant application process; applications are reviewed and approved (or rejected) by the Global Fund's Technical Review Panel.2 Grantees negotiate an agreement with the Global Fund, which "identifies specific, measurable results to be tracked using a set of key indicators."3 Local Fund Agents are assigned to each grant as independent auditors, to monitor and verify information submitted by grantees.4 Getting a full picture of what activities are funded by the Global Fund has proven difficult. It publishes individual grant proposals, but the large number of grants (761 at our last check5) and the large variety of proposal types make it impractical to gain a bird's-eye view from these documents. Based on a recent progress report, we roughly estimate the following overall allocations:6
• HIV/AIDS (39%)
• Malaria (21%)
• Tuberculosis (9%)
• 8% tuberculosis treatment, both DOTS and treatment of multidrug-resistant tuberculosis
• 1% TB/HIV collaboration
• Other (32%)
• 17% supportive environment, "including policy development, civil society strengthening, stigma reduction efforts, and management and administration."8
• 15% health system strengthening, "including community systems strengthening, information system and operational research, infrastructure, human resources, procurement and supply management, and M&E."9

## Does it work?

We do not have the information necessary to give an overall verdict on the Global Fund's effectiveness. We do feel that its general commitment to monitoring, evaluation and transparency far exceeds that of most charities we have seen, and for this reason alone we believe it is more accountable - and thus likely more effective - than most charities.

### Evidence of program effectiveness

It is difficult to get an overall sense of the Global Fund's effectiveness because of the wide range of programs it funds. Some of these programs (particularly antiretroviral treatment, malaria treatment, insecticide-treated net distribution, and DOTS) have strong evidence bases, leading us to believe that they would likely be successful if implemented appropriately. However, large amounts of funds (see above) go to other activities whose exact nature is unclear; examples we have seen include direct support for orphans and vulnerable children10 education aiming to change behavior,11 policy development, and stigma reduction. We would need a large amount of information about the outcomes of these programs to feel confident in their impact.

### Monitoring and evaluation

On-going monitoring and evaluation: The Global Fund does appear to consistently set concrete metrics for its projects, and monitor and evaluate outputs, though not outcomes/impacts of the projects, with the help of Local Fund Agents (see above). We have requested examples of completed Local Fund Agent data verification reports, but have not received them. We reviewed grant proposals and grant performance reports for the largest five grants for each of the three diseases, by disbursement to date (as of August 2010). Of the fifteen grants, fourteen set outcome/impact goals (such as reduction in disease prevalence/incidence), two reported data on progress toward the goals they set out, another four reported partially, and another one had received its first disbursement recently. All reported on outputs of the program (such as number of people trained or number of people treated).12 The Global Fund appears to stop or decrease funding to at least some of its under-performing grants. The Global Fund initially funds grants for a two-year period (Phase 1) after which it evaluates progress and determines whether or not to commit additional funding for the remaining three years of the five year grant (Phase 2).13 Grants rated 'C' (the lowest rating) received 26% of the funding approved for them in Phase 2 of the grant, compared to 96% and 94% for 'A' and 'B1' rated grants, respectively, and 73% for grants rated 'B2.'14 Independent evaluation: In 2009, the Global Fund released a large-scale five-year evaluation.15 The evaluation credits the Global Fund with increasing funding and coverage of medical interventions but stops short of directly attributing impact,16 noting inconsistent grant evaluation practices (in line with the findings of our analysis of the Global Fund's top grants),17 the challenge of attributing impact to the Global Fund as opposed to other actors,18 thin data, a large variety of projects, and the relatively short time window over which the Global Fund has been active.19 The conclusions are at best preliminarily encouraging, not instilling strong confidence in impact; at the same time, the extent to which this report is open and honest about its criticisms of the Global Fund is far beyond what we are accustomed to from charities' materials.

### Transparency

The Global Fund is extremely transparent in publishing documentation, sharing (a) all documents related to each country's grant,20 (b) external evaluations of the Global Fund,21 and (c) documents from all board meetings.22

### Possible negative or offsetting impact

We believe the potential for negative or offsetting impact is relatively high. The Global Fund gives funding directly to governments, as noted above, and could potentially be interfering with or substituting for their responsibilities and accountability, even strengthening harmful governments in some cases. In addition, many of the interventions promoted by the Global Fund - such as antiretroviral treatment - may often require highly skilled labor. We see a serious risk of the Global Fund funding's distorting the allocation of labor, if expenses are not calibrated extremely carefully (and given the Global Fund's scale, it seems intuitively likely that they are often quite rough.)

## Financials/other

• What are the specifics of the monitoring and evaluation conducted by Local Fund Agents? As discussed above, Local Fund Agents are independent auditors who vet information reported by grantees and perform site visits. We have not seen the specifics detailing what particular Local Fund Agents have monitored in respect to individual grants.
• To what extent is the Global Fund expanding vs. reallocating health care coverage? The Global Fund is a large grantmaker which funds countries and NGOs based on their proposals and holds them accountable to agreed upon outcomes. As far as we know, the Global Fund does not try to evaluate any effects it might have beyond the scope of its grants. For example, do Global Fund grants reduce capacity for other healthcare activities like immunization or deworming programs?

## Sources

• 1. 51% of cumulative funds (through 2008) were granted to governments, 35% to non-governmental organizations (i.e., nonprofits). The rest went to multilateral institutions and the private sector. Global Fund, "Innovation and Impact (2008)," Pg 26, Figure 2.4.
• 2. "The Technical Review Panel reviews all eligible proposals for technical merit and makes one of four recommendations to the Global Fund Board: (1) fund; (2) fund if certain conditions are met; (3) encourage resubmission; and (4) do not fund. Step 5 The Board approves grants based on technical merit and availability of funds." Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 12.
• 3. Global Fund, "Innovation and Impact (2010)," Pg 103.
• 4. "The Global Fund does not have a country-level presence outside its offices in Geneva. Instead, it relies on independent advice from local auditors referred to as Local Fund Agents. In the initial stage, a Local Fund Agent assesses the capacity of a nominated Principal Recipient to administer grant funds and be responsible for implementation. During the life of a grant, the Local Fund Agent will also verify the Principal Recipient's periodic disbursement requests and progress updates. In addition, Local Fund Agents are responsible for reviewing the Principal Recipient's annual report and advising the Global Fund on matters involving disbursements and other actions." Global Fund, "Innovation and Impact (2010)," Pg 102. There are 9 Local Fund Agents. Number in parentheses indicates the number of countries an LFA is responsible for: PriceWaterhouseCoopers (67), Swiss THP (17), KPMG (16), UNOPS (14), Cardno EM (7), Deloitte (3), Crown Agents (3), Finconsult (1), and Grant Thornton (1). Global Fund, "LFAs Selected."
• 5. Global Fund, "Grant Report (July 31, 2010)".
• 6. We calculated these figures in the following way using data from Global Fund, "Innovation and Impact (2010)."
• Pg 25, Figure 2.2: Allocation of Cumulative Expenditure by Service Delivery Area for Three Diseases (Through 2008 Expenditure Reporting Cycle).
• Pg 20, Table 2.6: Disbursement Total by Disease (as of December 2009). We multiplied these by the numbers from Figure 2.2 to estimate (very roughly) overall proportion of funds for each broad intervention type.
Pg 19, Table 2.3 serves as a rough guide to which specific interventions correspond to some of these broad intervention types. Calculations in GiveWell, "Global Fund Funding Breakdown (2009)."
• 7. Global Fund, "Innovation and Impact (2010)," Pg 26.
• 8. The description refers to supportive environment expenditure for HIV, but it seems reasonable, given its generality, to assume that it describes 'supportive environment' expenditure for malaria and TB as well. Global Fund, "Innovation and Impact (2010)," Pg 26.
• 9. The description refers to supportive environment expenditure for HIV, but it seems reasonable, given its generality, to assume that it describes 'health system strengthening' expenditure for malaria and TB as well. Global Fund, "Innovation and Impact (2010)," Pg 26.
• 10. "4.5 million basic care and support services were provided to orphans and other vulnerable children through Global Fund-financed programs – 1.3 million in 2009 alone." Global Fund, "Innovation and Impact (2010)," Pg 16.
• 11. "More than 138 million outreach activities were undertaken, such as behavior change communication interventions targeting at-risk populations for HIV, TB and malaria." Global Fund, "Innovation and Impact (2010)," Pg 18.
• 12. See GiveWell, "Analysis of Global Fund Top Grants."
• 13. "Proposals are approved for five years, but the Global Fund initially commits funds for a two-year period only (Phase 1). Funding for the remainder of the period covered by the proposal (Phase 2) is based on performance during Phase 1 and contextual factors." Global Fund, "Local Fund Agent Phase 2 Report."
• 14. Global Fund, "Performance-Based Funding In Action." The relationship between a grant's rating and the disbursement it receives is explained at Global Fund, "Grant Performance Rating and Disbursement Decision-making Methodology": "Deciding on a disbursement amount: Each performance rating category (A1, A2, B1, B2, C) has an indicative disbursement range, calculated in order to ensure the relationship between results achieved and funds disbursed by the Global Fund. These disbursement ranges are, nevertheless, only indicative and serve as a starting point for the disbursement decision. Ultimately, the final disbursement amount is based on: 1) overall grant performance; 2) contextual factors (force majeure, political and civil issues, etc.); 3) real budget needs in the context of spending ability and (4) actions needed to address identified weaknesses in management capacity."
• 15. Global Fund, "The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis, and Malaria: Synthesis of Study Areas 1, 2 and 3."
• 16. "Collective efforts are showing major changes in the availability and coverage of interventions. Although current data sources are not complete enough to measure disease impact, we can conclude from stepwise analysis that the increased funding is resulting in better availability and utilization of services which ultimately will have an impact on disease burden." Global Fund, "The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis, and Malaria: Synthesis of Study Areas 1, 2 and 3," Pg 18.
• 17. "As shown by the review of selected country grants, there is frequent shifting of indicators and targets during the different stages of the Global Fund grant. Impact and outcome indicators are usually included, but there is limited investment in the measurement, and baselines and targets often are no more than guesses or estimates based on outdated data. In several instances, the indicators are simply the wrong choice because the data need modeling (e.g., TB incidence data) or have considerable measurement issues (e.g., condom use at last sex with nonregular partner), which render them unsuitable for PBF [performance-based funding]. Therefore, PBF relies almost entirely on a range of process and output indicators that require solid quality control systems. The measures taken by the Global Fund to control quality, such as indicator guidelines, verification by the local fund agent in country, and ad hoc country visits, have not been adequate, as can be deduced from the actual country system and the data used in the grant performance reports. Regular independent assessment of data quality through further analysis, facility assessment, and population-based data collection is essential but may still not guarantee a satisfactory system of rating performance." Global Fund, "Global Fund Five-Year Evaluation: Study Area 3," Pgs ES-12 – ES-13.
• 18. "All of these factors affect the ability to attribute changes in general efforts to scaling up and to the Global Fund itself. From a scientific perspective, impact evaluation requires both measuring changes in health status and disease epidemiology, as well as attributing observed changes to specific interventions and actors." Global Fund, "Global Fund Five-Year Evaluation: Study Area 3," Pg 1-4.
• 19. "There are scientific, technical, and practical challenges to evaluating impact. Major challenges in this particular evaluation study are the relatively short duration of the scaling up of funding, the time lag between funding and implementation, the large number of interventions that have been implemented with the funding, the large number of partners involved in funding and implementation, the lack of reliable trend data on outputs and outcomes of scaling up for many interventions, and the difficulties of measuring changes in health outcomes... The present evaluation study will add to the experience in the evaluation of large international programs, but its contribution will necessarily be limited by the short time period that funds have effectively been available to design and implement appropriate intervention programs." Global Fund, "Global Fund Five-Year Evaluation: Study Area 3," Pgs 1-4 - 1-5.
• 20. These documents include (a) the country's original proposal, (b) the final grant agreement, and (c) reports on the grant's success at reaching its indicators. These documents are available on the Global Fund's website. Global Fund, "Program Search."
• 21. Global Fund, "The Global Fund Evaluation Library." This page includes a critical evaluation of the Local Fund Agent process (United States Government Accountability Office 2007), giving us some confidence that the Global Fund is sharing all documents, not just positive reviews.
• 22. Global Fund, "Board Decisions."
• 23. Global Fund, "Resource Scenarios (2011-2013)," Pg 5.
• 24. Global Fund, "Resource Scenarios (2011-2013)," Pg 5.
• 25. Global Fund, "Third Voluntary Replenishment (2011 - 2013)."
• 26. Wendt 2009.
• 27. Global Fund, "IRS Form 990 (2002-2008)" and Global Fund, "Annual Report (2009)."
• 28. Global Fund, "IRS Form 990 (2002-2008)" and Global Fund, "Annual Report (2009)."
• 29. Global Fund, "Annual Report:"
• 2004, Pg 32.
• 2005, Pg 93.
• 2006, Pg 105.
• 2007, Pg 6.
• 2009, Pg 55.
This data does not appear to have been included in the 2008 Annual Report.
• 30. Global Fund, "IRS Form 990 (2002-2008)."