Schistosomiasis Control Initiative - August 2015 Update | GiveWell

You are here

Schistosomiasis Control Initiative - August 2015 Update

Summary

  • We continue to recommend SCI for its track record implementing a cost-effective program. (More in our full review)
  • We learned relatively little about SCI in the first half of 2015. We continued to have some challenges communicating with SCI. (More)
  • We have not significantly improved our understanding of SCI's finances (more) or seen additional monitoring results (more). Several monitoring reports were expected by this time but have been delayed.
  • SCI's budget differs from its previous plans in several ways, and it is not clear to us what caused these shifts. In particular, it appears that SCI has allocated all of its unrestricted funding (~$8 million) to deliver deworming treatments during its current budget year (April 2015 - March 2016) rather than holding reserves to maintain programs in future years. SCI told us that it expects to rely on funds received in 2015 to support programs in 2016; we do not know why SCI changed its strategy. Two programs are costing significantly more than expected. (More)
  • Recently, SCI had another conversation with us and shared some additional documents. These updates have not generally been incorporated into this report.

Published: August 2015

Note: though this update was published in August 2015, it is based primarily on information that we had received as of late April 2015. Since then, SCI shared three additional documents with us (SCI financial statements, 2013-14 and 2014-15, SCI's ICOSA Mid-Year Report to DFID, 2015 and SCI impact and coverage survey plans (May 2015 update)). We have mentioned these documents in this update where they may be related to the topics being discussed. We plan to analyze these documents more thoroughly before the end of this year.

Ongoing communication challenges

On several occasions in 2015 the information we received from SCI was inconsistent or substantially different from what SCI had previously told us. We have so far been unable to get clear answers on what changed. Details:

  • Reserves strategy: SCI previously told us that it would save significant reserves for future years to ensure the sustainability of its programs.1 In its current budget, it does not seem to plan to save unrestricted reserves.2 When we asked SCI about what had changed, SCI told us its reserve strategy was not fully worked out yet.3 It is therefore unclear to us how we should interpret the budget SCI provided.
  • Unrestricted funds held as of March 2015: In April 2015, SCI shared a draft budget for 2015 that indicated it had a funding gap of $1.5 million (~20% of its unrestricted funding balance).4 In July 2015, it sent us a funding update that appears to show that it held $1.6 million more in unrestricted funding as of March 2015 than it previously reported. We asked SCI for clarification by email and have not yet received more information. (More)
  • Budget revisions: In October 2014, SCI shared plans for how it would use additional funding in its next budget year.5 In its current budget, SCI has allocated substantially more to some programs than expected. In April 2015, we asked SCI about the increase in Ethiopia (its largest program), and it told us the additional funding would support more program staff and would not result in additional treatments.6 It is not clear to us what caused this change in plans. (More)

Note that we have faced similar challenges evaluating SCI in the past.7

Financial update

SCI's financial management

We wrote in our November 2014 review that we had not seen complete information on how SCI has spent funds in the past. We continue to know relatively little about SCI's finances.

Last year, SCI worked with two consulting firms to identify ways it could improve its operations. Both firms identified financial management as an area SCI should improve.8 SCI hired a new financial manager in January 2015.9 She told us that she has been learning SCI's financial systems and that due to the complexity of the system, she did not expect to be able to provide a detailed financial update until September 2015.10

Past spending

In July 2015 (after we had drafted this update), SCI shared spending and income data for April 2013 to March 2015 (SCI financial statements, 2013-14 and 2014-15). We expect to need to analyze this data in depth and discuss it with SCI before having confidence in it.11 We plan to discuss this data with SCI in our next conversation.12

Unrestricted funding

In December 2014, we wrote that we hoped SCI would receive $1 million from individuals as a result of our recommendation (in addition to $3 million from Good Ventures and $1 million from another donor, both of which had already been committed). We expected this would result in SCI having about $7 million in unrestricted funding (including its existing reserves and funds it raised from other sources).13

In April 2015, we noted that we had met our target; we had tracked $1.1 million in donations from individuals to SCI due to our recommendation. In March 2015, SCI reported to us that it held $6.5 million in unrestricted funding.14 In July 2015, SCI sent us a financial report that shows that its unrestricted funding balance as of March 2015 was $8.1 million.15

We do not know what the cause of this discrepancy is. It is unclear to us how much unrestricted funding SCI currently holds.

Monitoring and evaluation

We have seen limited new monitoring and evaluation studies assessing the impact of programs that SCI supports since our last update in November 2014. SCI shared an update on its plans for completing monitoring and evaluation studies.16 Many of its plans have been delayed.

New monitoring and evaluation

  • Remapping study from Yemen: The study concluded that the schistosomiasis control program in Yemen had significantly reduced infection rates after 2-3 rounds of treatment.17 We have not yet examined this study in depth. It is our understanding that Yemen accounts for a small portion of SCI's past spending.18
  • Remapping study from Malawi: The study analyzed schistosomiasis and STH mapping data in five districts in Malawi in 2013. These districts had received four rounds of treatment by the time of the study. The study does not compare infection rates to a baseline.19 Without baseline data, it does not appear that this study provides additional evidence of SCI's impact in Malawi. (Note that assessing SCI's impact was not the goal of the study.)

SCI's monitoring and evaluation plans

Previously, SCI had told us that it expected to complete reports on three additional studies by January 2015. In May 2015, SCI told us that the expected completion dates for these reports had been delayed:20

  • A report on a coverage survey in Uganda had been expected in September 2014. As of May 2015, it was expected in June 2015.21
  • A report on a coverage survey in Malawi had been expected in November 2014. As of May 2015, it was expected in July 2015.22
  • A report on a baseline prevalence survey in Tanzania had been expected in January 2015. As of May 2015, it was expected in September 2015.

Other monitoring plans also appear to have been delayed.23 We have not yet discussed the updated timeline with SCI and do not know what caused the delays.

According to the updated timeline, several reports had been expected in the next few months:24

  • Coverage surveys: Uganda in June 2015 and Malawi (second survey) in July 2015
  • Baseline impact surveys: Mozambique, Tanzania, and Niger in September 2015
  • Follow up impact surveys: Malawi (second follow up) in August 2015 and Liberia in September 2015

We would guess that the studies that are scheduled to be available in the next few months will have a limited effect on our view of SCI's impact. This is because:

  1. We have significantly more information on the impact of the Malawi program than on the impact of other country programs.25 Additional reports from Malawi will be of more limited value in understanding SCI's overall performance than reports from countries we know little about. We would also guess that the Malawi program is particularly well-run (as evidenced by the number of studies it has completed).
  2. The Liberia program is also somewhat exceptional because it was put on hold during the Ebola outbreak.26
  3. SCI previously told us that the Uganda coverage survey found low coverage rates.27
  4. Baseline reports are primarily useful for understanding SCI's potential impact (high baseline prevalence means higher potential impact) rather than its actual impact.

Additional monitoring

In an attempt to learn more about the impact of SCI’s programs, we are planning a project with IDinsight and SCI to do additional monitoring of SCI-supported programs. We have published a few documents from the preliminary planning for the project:

Cost per treatment

We do not have an updated estimate of SCI's cost per treatment delivered. Given our uncertainty about both the number of SCI-supported treatments (discussed below) and SCI's total spending (discussed above), we elected not to update our estimate of the cost per treatment.

Treatments in last fiscal year

SCI sent us preliminary data reporting that programs that it supported delivered a total of 26.1 million schistosomiasis treatments during its last budget year (April 2014 - March 2015).28 In addition, SCI estimated that an additional 6.7 million treatments were scheduled to be delivered in April 2015 (3.3 million treatments in Mozambique and 3.4 million treatments in Ethiopia).29 Data was not yet available for some countries.30

We do not have details on the sources of this data or whether there were any procedures for checking the accuracy of the reported numbers.

We compared the treatments that SCI has preliminarily reported to the number of treatments that it had planned to deliver last year, as of September 2014. We have seen both planned treatments and reported treatments for twelve of its programs.31 Across these programs, SCI planned 32.5 million treatments and supported an estimated 25.6 million treatments (including the 6.7 million April 2015 treatments in Mozambique and Ethiopia).32

SCI's Planned and Reported Treatments (millions)33

Country Projections Reported Difference
Mozambique 7.0 7.6 0.6
Malawi 5.6 4.3 -1.3
Ethiopia 3.7 3.4 -0.3
Cote d'Ivoire 5.4 2.8 -2.6
Tanzania 0.6 2.1 1.5
DRC 1.7 1.6 -0.1
Zanzibar 2.6 1.6 -1.0
Zambia 2.6 1.0 -1.6
Niger 2.1 1.0 -1.1
Madagascar 0.3 0.2 -0.1
Uganda 0.4 0.0 -0.4
Liberia 0.5 0.0 -0.5
Total 32.5 25.6 -6.9

SCI told us that treatments were delayed in Liberia (due to the Ebola outbreak) and Côte d'Ivoire (delayed from November 2014 to May 2015).34 We have not discussed with SCI what caused the changes in other countries.

Spending plans

March 2015 budget

In March 2015, SCI shared a draft budget for April 2015 to March 2016.35 SCI expects to spend about $15 million in total, using $7 million in restricted funding and about $8 million in unrestricted funding.36

For more details on how SCI is allocating its funds across its country programs, see SCI draft budget 2015-2016. Within each country program, we do not know how funds will be allocated across different activities (e.g., mass drug administrations, monitoring, and training staff) nor the breakdown between restricted and unrestricted funding.

How do SCI's current plans compare to its room for more funding analysis from November 2014?

We believe there is significant uncertainty in understanding SCI's plans, and we track how its plans change over time in order to better understand (a) the factors that can cause SCI to make adjustments and (b) the reliability of SCI's predictions. Quickly changing circumstances can make it difficult to predict in which countries SCI will be able to work; in other cases, the reasons for changes in SCI's plans have not been clear to us.37

There are a few notable differences between SCI’s planned budget (as of March 2015) and our prior understanding (as of November 2014) about how SCI would spend unrestricted funding:

  1. SCI appears to have changed its strategy for holding reserves. The budget indicates that SCI expects to spend all of its unrestricted funding in the 2015-2016 budget year, rather than holding a portion of funds received as reserves for future years.38 In November 2014, we expected that about a third of new unrestricted funding that SCI received would be held in reserve for 2016 to help ensure that programs could be maintained in the following year.39 In August 2015, SCI told us that it now expects to rely on funds received in 2015 to support programs in 2016;40 we do not know why SCI changed its strategy.
  2. SCI appears to have underestimated its funding need for certain programs. For example, we had expected that SCI would spend about $1.4 million this year in Ethiopia, but it is now allocating about $2.6 million to Ethiopia (SCI's largest allocation this year).41 Similarly, we had expected that SCI would spend about $800,000 this year in Tanzania, but it is now allocating about $2.1 million there.42 SCI told us that the updated Ethiopia budget will fund the same number of treatments that was planned initially.43 We do not know if other increases in programs' budgets are due to (a) previous estimates being too low or (b) changes in the number of treatments SCI expects to deliver.
  3. In our November 2014 report, we listed several countries where SCI would consider working or expanding its work if it had more funding. From discussions with SCI, we understood that SCI was likely to encounter political or logistical constraints in some countries. We expected that, if SCI were to receive sufficient funding for all of the work that we described in our November report as "probable" or "possible" funding opportunities, it would be able to move forward with some but not all of the opportunities. The budget SCI shared in early 2015 implies that SCI expects to be able to allocate funding to all of the "probable" and "possible" opportunities.44 We are uncertain whether (a) our expectations were wrong, (b) SCI has encountered fewer non-funding constraints than it expected, or (c) the budget should only be interpreted as possible allocations rather than expected spending.

This spreadsheet compares our November 2014 analysis to SCI's draft budget for each program.

Sources

Document Source
Accenture Development Partnerships, Programme and Financial Management Workshop (June 2014) Unpublished
Alan Fenwick and Blandine Labry, SCI Director and Finance and Operations Manager, conversation with GiveWell, February 5, 2015 Unpublished
Alan Fenwick, SCI Director, conversation with GiveWell, October 14, 2014 Unpublished
Alan Fenwick, SCI Director, email to GiveWell, April 14, 2015 Unpublished
Alan Fenwick, SCI Director, email to GiveWell, August 11, 2015 Unpublished
Deloitte Draft Internal Audit Report 2012/2013 for SCI (January 2014) Unpublished
GiveWell analysis of SCI preliminary treatment data 2014-2015 Source
GiveWell's non-verbatim summary of a conversation with Alan Fenwick and Blandine Labry, April 27, 2015 Source
Mapping of Schistosomiasis and Soil-Transmitted Helminths in Yemen, and the Push for Elimination Source
Schistosomiasis Prevalence and Intensity in Relation to the Proximity of Lake Malawi Source
SCI advisory board financial report (June 2014) Source
SCI draft budget 2015-2016 Source
SCI financial statements, 2013-14 and 2014-15 Source
SCI impact and coverage survey plans (May 2015 update) Source
SCI report to GiveWell (September 2014) Source
SCI's ICOSA Mid-Year Report to DFID, 2015 Source

Additional documents

SCI shared additional documents with us, some of which are internal reports rather than formal publications.

Document Source
Knipes et al. 2014 (abstract only) Source
Longitudinal Cohort Study for Monitoring and Evaluation of the Malawi National Schistosomiasis and STH Control Programme Source
SCI budget allocation meeting minutes (March 2015) Source
SCI Control of schistosomiasis and soil-transmitted helminths in Ethiopia, internal report (November 2014) Source
SCI Cote d'Ivore coverage survey protocol (2014) Source
SCI Cote d'Ivore coverage survey protocol, French (2014) Source
SCI Cote d'Ivore mapping protocol, French Source
SCI Final Report to Gates Foundation, excerpt (February 2012) Source
SCI Funded control of NTDs in Madagascar, internal report (2014) Source
SCI Malawi coverage survey protocol, draft (October 2012) Source
SCI Malawi mapping protocol Source
SCI Newsletter (December 2014) Source
SCI organogram, internal report (January 2015) Source
SCI Schistosomiasis Control in Yemen Progressing from Control of Morbidity to Elimination as a Public Health Problem (July 2014) Source
Worrell and Mathieu 2012 Source (archive)