The Global Fund to Fight AIDS, Tuberculosis and Malaria - July 2009 review

GFATM is an extremely large funding mechanism, supporting a broad variety of programs to combat HIV/AIDS, tuberculosis and malaria throughout the developing world. GFATM's commitment to transparency is outstanding, and its activities are mostly proven and cost-effective. We do not have a clear enough sense of its project-by-project (or aggregate) outcomes to be fully confident in its impact or to give a cost-effectiveness estimate.

What do they do?

The Global Fund funds programs - primarily through developing-world governments - aiming to control HIV/AIDS, TB, or malaria.1 Funding is determined by a grant application process; applications are reviewed and approved (or rejected) by the Global Fund's Technical Review Panel.2 Grantees negotiate an agreement with the Global Fund, which "identifies specific, measurable results to be tracked using a set of key indicators."3 Local Fund Agents are assigned to each grant as independent auditors, to monitor and verify information submitted by grantees.4 Getting a full picture of what activities are funded by GFATM has proven difficult. It publishes individual grant proposals, but the large number of grants (714 at our last check5) and the large variety of proposal types make it impractical to gain a bird's-eye view from these documents. Based on a recent progress report, we very roughly estimate the following overall allocations:6

• 21% HIV/AIDS treatment (antiretroviral therapy)
• 17% HIV/AIDS prevention programs (including, but not limited to, condom distribution and promotion programs)
• 13% malaria treatment
• 12% malaria prevention, both insecticide-treated nets and spraying of pesticides
• 5% tuberculosis treatment, both DOTS and treatment of multidrug-resistant tuberculosis
• 5% tuberculosis prevention (specific intervention unclear)
• 5% care and support for HIV/AIDS
• 22% classified as "Other"

Does it work?

It is difficult to get an overall sense of the Global Fund's effectiveness because of the wide range of programs it funds. Some of these programs (particularly antiretroviral treatment, malaria treatment, insecticide-treated net distribution, and DOTS) have strong evidence bases, leading us to believe that they would likely be successful if implemented appropriately. However, large amounts of funds (see above) go to other activities whose exact nature is unclear; examples we have seen include direct support for orphans and vulnerable children7 and community-based HIV education programs.8 We would need a large amount of information about the outcomes of these programs to feel confident in their impact. The Global Fund does appear to consistently set concrete metrics for its projects, and monitor and evaluate them with the help of Local Fund Agents (see above). In addition, it is extremely transparent in publishing documentation, sharing (a) all documents related to each country's grant,9 (b) external evaluations of the Global Fund,10 and (c) documents from all board meetings.11 In 2009, the Global Fund released a large-scale five-year evaluation.12 The evaluation credits GFATM with increasing funding and coverage of medical interventions but stops short of directly attributing impact,13 noting inconsistent evaluation practices,14 the challenge of attributing impact to GFATM as opposed to other actors,15 thin data, a large variety of projects, and the relatively short time window over which GFATM has been active.16 The conclusions are at best preliminarily encouraging, not instilling strong confidence in impact; at the same time, the criticality and honesty of this report is far beyond what we are accustomed to from charities' materials. We do not have the information necessary to give an overall verdict on GFATM's effectiveness. We do feel that its general commitment to monitoring, evaluation and transparency far exceeds that of most charities we have seen, and for this reason alone we believe it is more accountable - and thus likely more effective - than most charities.

Possible negative or offsetting impact

We believe the potential for negative or offsetting impact is relatively high. GFATM gives funding directly to governments, as noted above, and could potentially be interfering with or substituting for their responsibilities and accountability, even strengthening harmful governments in some cases. In addition, many of the interventions promoted by GFATM - such as antiretroviral treatment - may often require highly skilled labor. We see a serious risk of GFATM funding's distorting the allocation of labor, if expenses are not calibrated extremely carefully (and given GFATM's scale, it seems intuitively likely that they are often quite rough.)

What do you get for your dollar?

Some of the programs funded by the Global Fund (particularly insecticide-treated net distribution, and DOTS) are estimated to be extremely cost-effective, in the range of $1000 or less per life saved. Antiretroviral treatment, which accounts for 21% of its expenses (see above), is far less cost-effective. Other programs could vary widely, and in some cases may have zero or negative impact. With the information we have, we cannot usefully give a cost-effectiveness range for the Global Fund as a whole.

Room for more funds?

The Global Fund states that "the initial contributions ... to the Fund represent only part of the US $15 billion experts estimate is needed each year to prevent and treat HIV, TB and malaria effectively on a global scale."17 Total pledges through 2010 are in the range of $1.5-$3 billion a year18, far less than the $15 billion figure. However, it is not clear to us (a) how this $15 billion figure was estimated, and (b) whether the amount "needed to treat HIV, TB and malaria effectively on a global scale" is equal to the amount that can be productively used. A key question seems to be the extent to which the Global Fund has more quality proposals than it can fund; as such, we would be interested in seeing examples of proposals that were rejected (or only partially funded) due primarily to limited funding rather than poor proposal quality. We note that a blog post by the Center for Global Development states, "the board has ”˜in-theory' approved US$2.21 billion for round 9 proposals and US$0.43 billion in support of NSAs. This would cover the first two years of funding for approved grants. Unfortunately the Global Fund does not have the money needed to cover US$2.6 billion over the next two years, so the secretariat has come up with some creative ways to address the funding gap."19 Another potential concern is that the Global Fund may be reallocating rather than expanding medical coverage. If there are a limited number of doctors, providing more funding for HIV/AIDS treatment may not be net beneficial even if it involves a net increase in funding.

Financials/other

Revenue and expense growth (about this metric): GFATM's revenues and expenses grew consistently between 2002 and 2007. Revenues grew substantially faster than expenses in 2008, and then this reversed in 2009.20 Assets-to-expenses ratio (about this metric): GFATM maintains a reasonable assets:expenses ratio with approximately 1 year's worth of reserves.21 Expenses by program area (about this metric): GFATM splits its funds between its three targeted diseases, focusing most heavily on HIV/AIDS.22 Expenses by IRS-reported category (about this metric): As a large grantmaker, GFATM has very low administrative and fundraising costs.23

Unanswered questions

• What is the complete set of program types the Global Fund supports, and how much funding goes to each? The breakdown we have (see above) is very limited; approximately 22% of funds (an estimate of ours based on limited information), for example, support programs simply designated as "Other" within one of the three disease areas (HIV/AIDS, tuberculosis, malaria). We seek a more detailed breakdown for each of the past several years.
• What are the specifics of the monitoring and evaluation conducted by Local Fund Agents? As discussed above, Local Fund Agents are independent auditors who vet information reported by grantees and perform site visits. There is no pre-defined set of activities or checks that Local Fund Agents must perform; instead, they use their judgment about which aspects of a grantee to evaluate. We have not seen the specifics detailing what particular Local Fund Agents have monitored in respect to individual grants.
• How often does the Global Fund assess impacts as opposed to outputs? The Global Fund provides a list of "Top Ten Outcome/Impact Indicators" that include direct measurement of a program's impact, measuring, for example, HIV prevalence or malaria-associated deaths.24 However, grantees do not always use these indicators to report results.25 For what portion of its grants does the Global Fund track impact indicators?
• How frequently does the Global Fund stop funding poorly performing grants?
• Does a lack of money prevent the Global Fund from funding more proposals, or do they have enough money, but a lack of proposals worth funding?
• To what extent is the Global Fund expanding vs. reallocating health care coverage? The Global Fund is a large grantmaker which funds countries and NGOs based on their proposals and holds them accountable to agreed upon outcomes. As far as we know, the Global Fund does not try to evaluate any effects it might have beyond the scope of its grants. For example, do Global Fund grants reduce capacity for other healthcare activities like immunization or deworming programs?

Sources

• Bernescut, Beatrice. Global Fund Communications Officer. Phone conversation with GiveWell, May 18, 2009.
• GiveWell. Anti-retroviral therapy to treat HIV/AIDS.
• GiveWell. Condom distribution and promotion.
• GiveWell. Distribution of insecticide-treated nets (ITNs) to prevent malaria.
• GiveWell. Guide to GiveWell's financial metrics.
• GiveWell. Treatment of malaria.
• GiveWell. Tuberculosis case finding and treatment ("DOTS" approach).
• Global Fund. Annual report (2004) (PDF).
• Global Fund. Annual report (2005) (PDF).
• Global Fund. Annual report (2006) (PDF).
• Global Fund. Annual report (2007) (PDF).
• Global Fund. Annual report (2008) (PDF).
• Global Fund. Annual report (2009) (PDF).
• Global Fund. Board decisions. http://www.theglobalfund.org/en/board/decisions/?lang=en, (accessed April 16, 2010). Archived by WebCite® at http://www.webcitation.org/5p2XJFXSM.
• Global Fund. Evaluation library. http://www.theglobalfund.org/en/library/specific_evaluations/?lang=en (accessed April 16, 2010) Archived by WebCite® at http://www.webcitation.org/5p2WtajXF.
• Global Fund. Global Fund five-year evaluation: Study area 3. The impact of collective efforts on the reduction of the disease burden of AIDS, tuberculosis, and malaria (PDF).
• Global Fund. Grant Report (April 15, 2010) (XLS).
• Global Fund. Innovation and impact (2010) (PDF).
• Global Fund. IRS Form 990:
  • 2002 (PDF).
  • 2003 (PDF).
  • 2004 (PDF).
  • 2005 (PDF).
  • 2006 (PDF).
  • 2007 (PDF).
  • 2008 (PDF).
• Global Fund. Pledges (March 31, 2010) (XLS).
• Global Fund. Program search. http://www.theglobalfund.org/en/portfolio/?lang=en (accessed April 16, 2010). Archived by WebCite® at http://www.webcitation.org/5p2WhQ4Av.
• Global Fund. Resource mobilization. http://www.theglobalfund.org/en/mobilization/ (accessed April 16, 2010). Archived by WebCite® at http://www.webcitation.org/5p2Z43zHw.
• Global Fund. Scaling up for impact: Results report (2008) (PDF).
• Global Fund. The five-year evaluation of the Global Fund to Fight AIDS, Tuberculosis, and Malaria: Synthesis of study areas 1, 2 and 3 (PDF).
• Wendt, David. How the Global Fund is dealing with more demand than supply (accessed April 16, 2010). Global Health Policy Blog, November 17, 2009. Archived by WebCite® at http://www.webcitation.org/5p2ZLV7hS.

• 1. 56% of funds in 2007 were granted to governments, 32% to non-governmental organizations (i.e., nonprofits). The rest went to multilateral institutions and the private sector. Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 29, Figure 11.
• 2. "The Technical Review Panel reviews all eligible proposals for technical merit and makes one of four recommendations to the Global Fund Board: (1) fund; (2) fund if certain conditions are met; (3) encourage resubmission; and (4) do not fund. Step 5 The Board approves grants based on technical merit and availability of funds." Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 12.
• 3. Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 12.
• 4. "The Principal Recipient submits periodic disbursement requests with updates on programmatic and financial progress. The Local Fund Agent verifies information submitted and recommends disbursements based on demonstrated progress. Lack of progress triggers a request by the Secretariat for corrective action. The Principal Recipient submits a fiscal-year progress report and an annual audit of program financial statements to the Secretariat through the Local Fund Agent." Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 12.
• 5. Global Fund, "Grant Report (April 15, 2010)".
• 6. We calculated these figures in the following way using data from Global Fund, "Scaling up for Impact: Results Report (2008):"
  • Pg 29, Figure 12: Proportion of funds allocated to each broad intervention type, broken down by disease. 2007 calendar year.
  • Pg 30, Figure 13: Proportion of funds allocated to each disease, overall. We multiplied these by the numbers from Figure 12 to estimate (very roughly) overall proportion of funds for each broad intervention type.
  Pg 32, Figure 16 serves as a guide to which specific interventions correspond to these broad intervention types.
• 7. "3.2 million orphans and vulnerable children have been provided with basic care and support." Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 4.
• 8. In one example in Eritrea, the Global Fund supported a program that "Strengthened community-based activities: community health workers, diagnosis and education programs." Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 61.
• 9. These documents include (a) the country's original proposal, (b) the final grant agreement, and (c) reports on the grant's success at reaching its indicators. These documents are available on the Global Fund's website. Global Fund, "Program Search."
• 10. Global Fund, "The Global Fund Evaluation Library." This page includes a critical evaluation of the Local Fund Agent process (United States Government Accountability Office 2007), giving us some confidence that the Global Fund is sharing all documents, not just positive reviews.
• 11. Global Fund, "Board Decisions."
• 12. Global Fund, "The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis, and Malaria: Synthesis of Study Areas 1, 2 and 3."
• 13. "Collective efforts are showing major changes in the availability and coverage of interventions. Although current data sources are not complete enough to measure disease impact, we can conclude from stepwise analysis that the increased funding is resulting in better availability and utilization of services which ultimately will have an impact on disease burden." Global Fund, "The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis, and Malaria: Synthesis of Study Areas 1, 2 and 3," Pg 18.
• 14. "As shown by the review of selected country grants, there is frequent shifting of indicators and targets during the different stages of the Global Fund grant. Impact and outcome indicators are usually included, but there is limited investment in the measurement, and baselines and targets often are no more than guesses or estimates based on outdated data. In several instances, the indicators are simply the wrong choice because the data need modeling (e.g., TB incidence data) or have considerable measurement issues (e.g., condom use at last sex with nonregular partner), which render them unsuitable for PBF [performance-based funding]. Therefore, PBF relies almost entirely on a range of process and output indicators that require solid quality control systems. The measures taken by the Global Fund to control quality, such as indicator guidelines, verification by the local fund agent in country, and ad hoc country visits, have not been adequate, as can be deduced from the actual country system and the data used in the grant performance reports. Regular independent assessment of data quality through further analysis, facility assessment, and population-based data collection is essential but may still not guarantee a satisfactory system of rating performance." Global Fund, "Global Fund Five-Year Evaluation: Study Area 3," Pgs ES-12 – ES-13.
• 15. "One of the things that this impact evaluation was not able to do effectively was to document the impact of other contextual variables that may have had an effect on the ability of countries to apply for and successfully use funds from the Global Fund to implement programs in malaria control. However, a strong case can be made that a critical factor in the success of the Global Fund in addressing malaria was the launching of the RBM Partnership in 1998." Global Fund, "Global Fund Five-Year Evaluation: Study Area 3," Pg 7-43.
• 16. "There are scientific, technical, and practical challenges to evaluating impact. Major challenges in this particular evaluation study are the relatively short duration of the scaling up of funding, the time lag between funding and implementation, the large number of interventions that have been implemented with the funding, the large number of partners involved in funding and implementation, the lack of reliable trend data on outputs and outcomes of scaling up for many interventions, and the difficulties of measuring changes in health outcomes. All of these factors affect the ability to attribute changes in general efforts to scaling up and to the Global Fund itself. From a scientific perspective, impact evaluation requires both measuring changes in health status and disease epidemiology, as well as attributing observed changes to specific interventions and actors ... The present evaluation study will add to the experience in the evaluation of large international programs, but its contribution will necessarily be limited by the short time period that funds have effectively been available to design and implement appropriate intervention programs." Global Fund, "Global Fund Five-Year Evaluation: Study Area 3," Pgs 1-4 - 1-5.
• 17. Global Fund, "Resource Mobilization."
• 18. Global Fund, "Pledges (March 31, 2010)."
• 19. Wendt 2009.
• 20. Global Fund, "IRS form 990 (2002-2008)" and Global Fund, "Annual Report (2009)."
• 21. Global Fund, "IRS form 990 (2002-2008)" and Global Fund, "Annual Report (2009)."
• 22. Global Fund, "Annual Report:"
  • 2004, Pg 32.
  • 2005, Pg 93.
  • 2006, Pg 105.
  • 2007, Pg 6.
  • 2009, Pg 55.
  This data does not appear to have been included in the 2008 Annual Report.
• 23. Global Fund, "IRS form 990 (2002-2008)."
• 24. Global Fund, "Scaling up for Impact: Results Report (2008)," Pg 98, Annex 2.
• 25. Beatrice Bernescut, phone conversation with GiveWell, May 18, 2009.