SDI Update - March 2015

Current as of: March 2015

This page gives an update on the Service Delivery Indicators Program, which is being carried out by the World Bank in partnership with the African Economic Research Consortium and the African Development Bank. Good Ventures contributed to this project based on a joint assessment with GiveWell.

Summary

In March 2014, Good Ventures made a $500,000 grant to the Service Delivery Indicators (SDI) program. Since then, SDI has continued to make progress on its surveys. Key updates are:

  • SDI survey results have received some attention and use. Its data has been used by blogs, news agencies, governments, and the World Bank (more).
  • SDI's expenses are greater than it originally predicted, with surveys costing $700,000 instead of $500,000. SDI needs approximately $4 million in order to reach its goal of surveying 10-15 countries by the end of 2015 (more).
  • SDI continues to seek funding to reach its target number of countries, and is considering options for sustainable funding (more).

Table of Contents

SDI’s progress

Data collection, analysis, and publication

As of 2013, SDI aimed to complete assessments for 10-15 countries by the end of 2015.1 SDI still believes it has the capacity to reach that goal with sufficient funding.2 (More on funding needs below.)

Eventually, SDI hopes to be working with 20-25 countries in Africa.3 Once it is surveying this many countries, it will have covered all of the countries in Africa where it believes the political environment is sufficiently open to use its information.4 However, there is a growing interest in SDI’s surveys from other continents.5

SDI’s pilot programs began in 2010.6 As of July 2014, SDI had:

  • Released six complete sets of survey results (Tanzania and Senegal in April 2012, Kenya in July 2013, Uganda in October 2013, Nigeria in June 2014, and Togo in July 2014).7
  • Completed the fieldwork of gathering data in two additional countries (Tanzania’s second round and Mozambique’s first round).8
  • Made preparations to collect a first round of data in four further countries (Niger, Mali, and the Democratic Republic of the Congo).9
  • Started discussions with two new countries about future rounds of data collection (Senegal and Cote d’Ivoire). The latter would be SDI’s first collaborative survey with the African Development Bank.10

Public use of the indicators

As we wrote in our January 2014 overview of the SDI program, SDI monitors its intermediate success using two indicators:

    1. "Public debate on education and health service delivery [is] initiated and/or informed."
    2. "Stakeholders (policymakers, media, NGOs, CSO) reporting use of SDI analysis within 6 months after any of the SDI dissemination events."11

Usage in public debate

SDI has told us that there has been public debate involving SDI’s data in Uganda and Kenya, where both of these countries were in the midst of teachers’ strikes when SDI’s survey results were released.12 SDI claims that its data fueled debates on education, particularly in Uganda, and that its findings are now discussed at many of Uganda’s and Kenya’s education policy forums.13

Usage by stakeholders

Tanzania
On the policy side, SDI believes its greatest success so far has been in Tanzania. SDI has told us that in 2011 SDI results were presented in Tanzania to a closed cabinet meeting.14 Afterwards, the Minister of Education began to visit schools unannounced in order to confirm SDI’s data.15 Then, in April 2013, SDI claims that its data was used during planning for the "Big Results Now" initiative.16 The "Big Results Now" initiative is part of Tanzania’s effort to become a middle-income country; it focuses on improving six areas of the economy, one of which is education.17 Several blogs began to reference SDI's Tanzania data, especially its data on teacher absenteeism.18 In 2013, SDI’s data appeared in a report on Tanzania’s education system.19 Finally, SDI tells us that in 2014 its data was used to further track the progress of the "Big Results Now" initiative.20

The World Bank
Another policy accomplishment was the World Bank’s decision to use SDI’s data to help appraise programs that are requesting funding.21 The World Bank will agree to fund a country’s program if the program is shown to be achieving certain performance targets, and in both Tanzania and Uganda, the World Bank is now using SDI’s data as a tool to measure whether or not programs are reaching their targets.22 This could end up making SDI’s task more difficult, because if $10-15 million were dependent on the outcomes of SDI’s surveys, that could incentivize people to inflate the countries’ performance metrics, meaning SDI would have to increase its quality-assurance efforts.23

In addition, the World Bank and the Gates Foundation are looking to start a global initiative focused on primary health care. SDI surveys may be used in the future as part of a "Primary Health Care Performance Initiative."24

Media and other
Articles in high-profile news sources (e.g., the Economist and BBC) are beginning to discuss the data that SDI is collecting.25 SDI is not mentioned in these articles explicitly, but it claims to be the only possible source of the information discussed.26 SDI’s results have also been presented at fora such as:27

  • The 10th Anniversary of the 2004 World Bank Development Report (February 2014)
  • The Oxford University Conference on African Studies (March 2014)
  • The World Health Organization and Spain Colloquium on Water and Sanitation in Health Facilities (April 2014)
  • The third Uganda Economic Update on Decentralization and Service Delivery (June 2014)
  • The World Health Organization Consultative Meeting on Health Facility Surveys (June 2014)

SDI has also developed its own website and is working to improve data visualizations for visitors.28

Building capacity

SDI aims to build up countries’ capacities to gather and analyze data by having local researchers work on its surveys.29 Therefore, SDI and its partners strongly prefer to use national research institutes to implement the surveys.30 Pursuing this goal of building capacity, a grant in the amount of $840,270 was successfully mobilized to support capacity development activities implemented by the African Economic Research Consortium (AERC), one of SDI’s partners.31 With funds from the grant, AERC successfully launched first training workshop in Kenya on the use of SDI data by national researchers.32

Funding updates

Program Costs

Each round of surveys costs SDI $700k (this includes both the health and the education surveys).33 This is higher than SDI's original predictions of $400k-$500k per round of surveys.34 SDI underestimated the costs of contracts with survey implementers, the costs of additional oversight for quality assurance, and the costs of disseminating its findings.35

Room for more funding

SDI does not have enough funding for the last 5-6 countries it would like to survey to reach its 2015 goal of 10-15 countries surveyed; thus, its immediate funding gap was roughly $4 million in July 2014.36 It was hoping to have this funding by July 2015 in order to start the processes needed to finish the surveys by the end of 2015.37 If SDI is unable to find the funds it needs, it will simply survey fewer countries.38

Sustainability

SDI is also considering plans for its long-term sustainability. So far, SDI has been opportunistic about leveraging sources of funding within the countries it has surveyed.39 For example, it has used funds from the Department for International Development (DFID) and the Gates Foundation on a country by country basis.40 This allows SDI’s trust funds to last longer.41 However, SDI would prefer to have a more sustainable funding model; its current model is costly because of the many transactions and different sources of funding that SDI must keep track of.42

In thinking about how to create a new model, it is important to SDI that it remain neutral and transparent.43 Given the public good nature of the data generated through SDI, the preferred model is where most of its funding is split between governments and development partners, with some additional funding coming from the World Bank.44

Sources

Document Source
Big Results Now, The United Republic of Tanzania Prime Minister’s Office Source (archive)
DFID blog Source (archive)
Education: NKRA Lab Report Source
Gayle Martin, World Bank Senior Economist, email to GiveWell, June 1, 2014 Unpublished
GiveWell’s non-verbatim summary of a conversation with Gayle Martin, World Bank Senior Economist, on July 30, 2014 Source
Project Appraisal Document on a Proposed Grant to the Government of Uganda Source
Program Appraisal Document on a Proposed IPA Credit to the Republic of Tanzania Source
SDI PowerPoint slide sent to GiveWell on August 8, 2014 Source
SDI Program Document 2011 Source
SDI Senegal Source (archive)
SDI Tanzania Source (archive)
SDI Timeline Source (archive)
World Bank, Service Delivery Indicators, email to GiveWell, March 25, 2015 Unpublished

Source URL: https://www.givewell.org/service-delivery-indicators/March-2015-update