Malaria Consortium – Seasonal Malaria Chemoprevention | GiveWell

# Malaria Consortium – Seasonal Malaria Chemoprevention

Malaria Consortium's seasonal malaria chemoprevention program is one of our top-rated charities and we feel that it offers donors an outstanding opportunity to accomplish good with their donations.

Published: November 2016

## Summary

What do they do? Malaria Consortium (malariaconsortium.org) works on preventing, controlling, and treating malaria and other communicable diseases in Africa and Asia. We have only reviewed its seasonal malaria chemoprevention (SMC) program, which aims to distribute preventive anti-malarial drugs to children 3-months to 59-months old in order to prevent illness and death from malaria; our recommendation is just for this part of Malaria Consortium's work. (More)

Does it work? There is strong evidence that SMC substantially reduces cases of malaria. Malaria Consortium has conducted studies in most of the countries where it has worked to determine whether its programs have reached a large proportion of children targeted. These studies have generally found that about 87% of children received at least one month of SMC treatment (out of four possible months), about 66% received three months of treatment, and about 46% received all four months of treatment. However, we have some remaining questions about the methodology behind these estimates. (More)

What do you get for your dollar? SMC programs appear to be in the range of cost-effectiveness of our other priority programs. We do not yet have a robust cost-per-treatment estimate for Malaria Consortium's SMC work, but we provide preliminary estimates below. (More)

Is there room for more funding? We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. It appears that there is a large remaining global need for additional funding for SMC programs, but we have not yet done a careful analysis of the funding landscape and other potential funders. (More)

Malaria Consortium's seasonal malaria chemoprevention program is recommended because:

• SMC is a program with a strong evidence base and strong cost-effectiveness. (More)
• Track record – Malaria Consortium has experience with supporting large-scale SMC programs. (More)
• Room for more funding – we believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. (More)

Major open questions include:

• We have not investigated Malaria Consortium at the same level of depth as some of our other top charities: we have not completed a full intervention report on SMC, done a site visit, requested budgets from SMC programs, or spent as much time communicating with Malaria Consortium staff. Our general experience has been that more investigation leads us to learn about more limitations and uncertainties associated with the impact of a program and charity.
• Will Malaria Consortium-supported SMC programs be able to achieve high levels of coverage in the future? Will Malaria Consortium's monitoring provide high-quality evidence of its programs' impact?
• Will there be limited room for more funding for SMC, either because other large funders fill the remaining funding gaps or because of inadequate supply of SMC treatments?

## Our review process

Malaria Consortium approached us to ask about applying for a recommendation in January 2016.

Over the next several months we tried to determine which of Malaria Consortium's programs we should prioritize evaluating for a possible recommendation, and we ultimately settled on seasonal malaria chemoprevention (SMC). The other programs that we investigated included: bed nets, deworming, dengue control, injectable artesunate for severe malaria, integrated community case management (ICCM), micronutrient powders, malnutrition management, neglected tropical diseases morbidity management, integration of nutrition with SMC, diagnosis of malaria, and diagnosis of pneumonia. We may do a deeper investigation of another Malaria Consortium-supported program in the future.

To date, we have:

• Had multiple conversations with Malaria Consortium staff.1
• Reviewed documents that Malaria Consortium shared with us.

## What do they do?

• ACCESS-SMC: UNITAID awarded up to $67 million to Malaria Consortium to lead a project to reach up to 7 million children per year in seven countries in the Sahel region of Africa.33 In May 2014, Malaria Consortium wrote that this was the "largest-yet global programme" for SMC.34 Malaria Consortium wrote that ACCESS-SMC would be led by Malaria Consortium, with Catholic Relief Services as the "primary sub-grantee" and support from many other organizations, including impact evaluation from the London School of Hygiene & Tropical Medicine.35 Malaria Consortium told us that its role in ACCESS-SMC included leading implementation of SMC in three of the seven ACCESS-SMC countries (Burkina Faso, Chad, and Nigeria), overseeing budgets and planning for all ACCESS-SMC activities, overseeing research (including methodology and presentation), and supporting training and operations at the local level.36 Our impression is that ACCESS-SMC pays for almost all aspects of program implementation and monitoring, including medicines and supplies, per diems for CHWs, training for CHWs, trainers, supervisors, and health facility workers, and research.37 Malaria Consortium shared the following source with us to help explain its staff structure: ACCESS-SMC, organizational structure. We have also seen mention of Malaria Consortium supporting SMC in Jigawa state in Nigeria and possibly increasing its scale in Katsina state; Malaria Consortium told us that this work is distinct from the work done under the BMGF grant mentioned above and was relatively limited.38 Malaria Consortium told us that the above programs represent the bulk of Malaria Consortium's work on SMC so far.39 ### Malaria Consortium's spending on SMC programs We have not yet seen a budget from Malaria Consortium for this program. We have seen the following documents, which analyze spending in Malaria Consortium-supported programs: We also provide some information on the estimated cost-per-treatment of Malaria Consortium-supported SMC programs below. We have not yet had detailed discussions about how many Malaria Consortium staff work on SMC programs and what roles they play. ## Does it work? Malaria Consortium-supported SMC programs are focused on delivering treatments that have been independently studied in rigorous trials and found to be effective. Malaria Consortium and its partners have conducted monitoring to determine what proportion of children targeted by its SMC programs receive treatments. We have seen headline results from coverage surveys for 2015 distributions in all seven ACCESS-SMC countries and results from coverage surveys for two of four LGAs included in Malaria Consortium's northern Nigeria SMC program. A weighted average of the ACCESS-SMC surveys finds that about 87% of children received at least one month of SMC treatment, about 66% received three months of treatment, and about 46% received all four months of treatment.40 We have not yet resolved all of our major remaining questions about these studies, so we are unsure whether we should expect these results to be a fully accurate representation of ACCESS-SMC’s coverage (we believe it is possible that these results could over- or underestimate actual coverage). We have also considered evidence that Malaria Consortium has collected about the impact of its SMC programs in northern Nigeria on malaria incidence. These results seem to be consistent with the impacts of SMC found in randomized controlled trials of the program, but due to remaining questions that we have about the studies we do not yet consider them to be strong additional evidence for the impact of SMC programs. We have not yet seen substantial monitoring of the health impact of ACCESS-SMC programs. Malaria Consortium has told us that it expects monitoring from its ACCESS-SMC programs (including estimates of health impact, estimates of adverse events, additional coverage surveys, and more) to be available in 2017. Details follow. ### Is SMC an effective intervention? SMC appears to have strong evidence of effectiveness. Seven randomized controlled trials provide strong evidence that SMC substantially reduces cases of malaria. We have not thoroughly investigated all of our major remaining questions about the evidence base. More details are available at our interim intervention report on SMC. ### Is Malaria Consortium working in areas suitable for SMC? We have not reviewed the evidence in detail, but we would guess that Malaria Consortium is working in areas that are suitable for SMC because it is working in countries with high malaria burdens and where it seems that malaria is seasonal.41 More information on our estimates of malaria burden in the countries where Malaria Consortium is working is available in our cost-effectiveness analysis. ### Are targeted children being reached? We have seen coverage survey results from the bulk of Malaria Consortium-supported SMC programs so far. A weighted average of these surveys finds that about 87% of children received at least one month of SMC treatment, about 66% received three months of treatment, and about 46% received all four months of treatment.42 #### Two methods for estimating coverage It appears that Malaria Consortium uses two different methods to measure the coverage rate that it is achieving with its programs:43 1. "Administrative" coverage: These coverage estimates are generated by collecting information from CHWs’ records about how many treatments they directly observed.44 Malaria Consortium provided a summary of the process for collecting and verifying this information, which we have included in the following footnote.45 We have not yet tried to assess these processes. 2. Coverage surveys: Data collectors visit a random sample of households in the target region, ask to review eligible children's caretakers' SMC record cards, and ask a variety of other questions to understand the quality of the program and the likely coverage of SMC. More details below. It appears that there are sometimes large discrepancies in the estimates of coverage between these two measures, usually with administrative coverage estimates finding higher levels of coverage.46 Malaria Consortium told us that possible reasons for discrepancies between administrative coverage estimates and coverage surveys include:47 • In some countries, large temporary migrant populations move into SMC treatment areas during the rainy season (sometimes to do agricultural work) and then leave before they would be interviewed by coverage surveys. • A substantial number of treatments are likely going to children who are roughly six to seven years old.48 • CHWs may be treating some people whom live outside of the target coverage areas. • There could be errors either in the number of treatments reported (the numerator) or in the assumed target population (the denominator). Malaria Consortium told us that in late 2016 it began doing rapid assessment surveys at the end of some treatment cycles to better understand why there were discrepancies between coverage estimates.49 Because this work began recently, we have not yet seen documentation from it. Below, we focus on analyzing Malaria Consortium's coverage surveys because we believe that coverage surveys are more likely to be accurate. For reference, all of the results from the administrative coverage estimates that we have seen are in the following footnote.50 It appears that according to administrative coverage estimates, ACCESS-SMC’s overall coverage rate of its target population was about 91%.51 We have not vetted this estimate. #### Coverage surveys ACCESS-SMC methodology The methodology for coverage surveys described in the ACCESS-SMC monitoring and evaluation plan is:52 • The London School of Hygiene and Tropical Medicine (LSHTM) will conduct coverage surveys within about one month after the last SMC cycle is delivered each year in a particular country.53 Our understanding is that one coverage survey will be done in each country each year.54 • The survey will use two-stage cluster sampling; we understand this to be a random sampling scheme that aims to be representative of the full population receiving the program.55 The survey would aim for a sample size of at least about 600 children aged 3-months to 59-months old.56 • The survey will aim to assess the proportion of eligible children who received 0, 1, 2, 3, or 4 SMC treatments.57 Surveyors are instructed to estimate coverage first by asking to review caretakers' SMC record cards and, if the record card is not available, then by asking the caretaker to recall how many doses their child received.58 Surveyors are also instructed to ask a variety of other questions to attempt to understand how many SMC treatments were received, such as:59 • "Do you know how many days should the child be given the drugs?" • "What type of drug was given out?" • "Do you know the recommended method to crush the tablets?" • "In your opinion, how many children under five in this community have taken the medicine that prevents malaria during the rainy season?" • "Did (NAME) take / swallow the drugs on day 2 and day 3 at home?" Our remaining questions include: • We are unsure how accurately caretakers (and CHWs) are recording SMC doses on the SMC Record Cards; systematic errors could lead to over- or underestimates of coverage.60 However, it may be that more reliable methods would be infeasible to implement at this scale. • We have not yet seen audits on the quality of the survey data. Malaria Consortium told us that an audit is planned to happen in Chad by December 2016.61 • We don't know whether "don't know" responses were counted as "no" or excluded from the sample. ACCESS-SMC results We have seen headline coverage survey results from all seven ACCESS-SMC countries for 2015 and a slightly more detailed report on the coverage survey results from Nigeria. A weighted average of these surveys finds that about 87% of children received at least one month of SMC treatment, about 66% received three months of treatment, and about 46% received all four months of treatment.62 The Nigeria coverage survey's methodology appeared to be broadly similar to the generic ACCESS-SMC coverage survey methodology described above.63 The survey included 1,112 children who were eligible for all 4 SMC cycles.64 Its findings included:65 • About 77% of children received at least one cycle of SMC, about 61% received at least three cycles, and about 42% received all four cycles. • About 61% of children 6-7 years old (i.e., non-eligible children) received SMC at least once. • About 75% among those issued an SMC card retained it by the time of the survey, and "Recall was an important source of information in Nigeria, because the SMC card was often incomplete." For the headline results of the coverage surveys from each ACCESS-SMC country in 2015, see GiveWell summary of SMC coverage information, November 2015. Malaria Consortium notes that, to put these results in context, "at each cycle, a child may not receive SMC, if they have malaria or are very sick at the time of the distribution, even though they are seen by the SMC health worker or community drug distributor. And at each cycle there is a chance that a child may miss SMC if they are away from the home (where SMC is delivered door-to-door) or are unable to attend at the SMC distribution point. 88% of children received an SMC card, generally issued at the first cycle. If the probability of receiving SMC at each cycle is 88%, the expected percentage who would receive at least three treatments is [67%], and the expected percentage who would receive four treatments is 60%."66 The fact that surveys identified relatively low coverage in some cases increases our confidence in their reliability. We have not yet seen results from ACCESS-SMC countries on many of the other questions listed in the ACCESS-SMC, Questionnaire for SMC coverage survey, such as what type of drug was given out and whether caretakers knew the recommended method to crush the tablets. Malaria Consortium told us that it will be able to share results from these other questions in the future.67 Northern Nigeria methodology and results We have not carefully reviewed the methodology of Malaria Consortium's surveys in northern Nigeria, but they appear to be broadly similar to its ACCESS-SMC surveys, primarily relying on information from SMC cards but asking caretakers to recall the number of treatments given if the card is not available.68 It conducted its survey in a random sample of about 750 households in two of the four LGAs where it delivered SMC in northern Nigeria in 2014.69 This survey appears to have been done by Malaria Consortium (not LSHTM, which did the ACCESS-SMC surveys). Its survey found that about 84% of children surveyed received at least one dose of SMC and that about 62% of children surveyed received at least 3 doses of SMC.70 It also found that about 86% of caretakers surveyed knew that 2 types of drug were given, 67% said that the first dose was directly observed by CHWs, and 86% correctly stated that doses should be given for each of 2 days after the first dose.71 ### Have malaria rates decreased in targeted populations? We have not yet seen detailed reports on ACCESS-SMC’s impact on health outcomes; Malaria Consortium told us that these reports will be available in 2017.72 We have seen three types of analyses of the impact of Malaria Consortium's northern Nigeria program on malaria indicators. These results seem to be consistent with the impacts of SMC found in randomized controlled trials of the program, but due to our remaining questions about the studies we do not yet see them as strong additional evidence for the impact of SMC programs. The three types of analyses of Malaria Consortium's northern Nigeria program are: 1. A (random sample) household survey that compared the same two districts in consecutive years before and after they received SMC, and assessed what proportion of children had malaria in each year by measuring whether they had a fever and whether they tested positive for malaria according to a malaria rapid diagnostic test (mRDT).73 This survey found large, statistically significant declines in fever and testing positive for malaria (details in footnote).74 One limitation of this survey is that the baseline survey was done during peak malaria season in 2013 (before SMC was distributed) while the endline survey was done shortly following the peak malaria season in 2014 (after SMC was distributed) in order to avoid measuring the immediate protective effect of SMC, which could have influenced the result.75 Another remaining uncertainty we have when interpreting these results is the possibility that trends unrelated to SMC (e.g., differences in weather patterns between years that led to different malaria rates) caused or contributed to the observed effect. 2. Difference-in-difference comparisons between routine sentinel site facility data from three health facilities in one LGA (Sandamu) that did not receive SMC versus data from five health facilities in two LGAs (Baure and Mashi) that received SMC for two years (see following footnote for charts).76 3. Before-and-after SMC comparisons of routine sentinel site facility data in all four LGAs that received SMC in northern Nigeria (see following footnote for charts).77 We do not yet have a strong understanding of the rigor of the latter two analyses. Our current understanding is that the data comes from "sentinel sites" (health facilities) in each LGA, but we do not know how these sites were chosen or the quality of their data; all of the methodological information we have seen about sentinel sites is in the following footnote.78 A remaining uncertainty we have when interpreting these studies is that it seems that cases of malaria do not need to be confirmed (they could simply be suspected) in order to be counted as a 'case of clinical malaria' by sentinel sites.79 It is unclear whether this would significantly affect the data, but due to lack of blinding, it may be that sentinel site staff would be less likely to suspect malaria cases in areas with SMC. ### Are there any negative or offsetting impacts? • Drug resistance: Mass-delivery of SMC medicines could contribute to increased drug resistance. Malaria Consortium has written that it is doing further research to evaluate whether this is a major issue, but we have not yet reviewed this research in detail.80 • Possible "rebound" effects: There is a potential concern that SMC could reduce the natural development of immunity to malaria so that after children turn five years old, and are no longer eligible to receive SMC, they could lack immunity and be more susceptible to malaria. We have not yet investigated this concern in-depth. • Side effects of SMC drugs: Our impression is that the most common side effect of SMC drugs is vomiting, but we do not have a strong sense of how common this is.81 Also, children are supposed to take the SMC drugs again within 30 minutes if they vomit; we are unsure whether caretakers typically request extra drugs from CHWs when this happens to ensure their child gets a full treatment regimen. Malaria Consortium told us that it expects vomiting to be less common with a new dispersible, better-tasting formulation of the SMC drugs that ACCESS-SMC began using in 2016; evidence on this point is not yet available.82 Our impression is that other side effects from these drugs are rare but include diarrhea, itching, headache, mild abdominal pain, and rash.83 We have not yet carefully reviewed the evidence on side effects of SMC drugs. Malaria Consortium shared two resources that report very low severe adverse event rates from SMC drugs (available in the following footnote).84 We have not yet reviewed these. • Drug quality and dosage: Malaria Consortium told us that it is only able to procure products from suppliers that meet internationally recognized quality assurance standards.85 We have not yet asked Malaria Consortium for the details of this process. If there were issues with drug quality or dosage, it could reduce the effectiveness of the intervention and lead to more rapid development of drug resistance. ### Other planned monitoring and evaluation Malaria Consortium has written that ACCESS-SMC plans to collect and share more information about its impact, including: • Case-control studies to measure the efficacy of SMC treatments.86 • Malaria sentinel surveillance studies to measure the health impact of ACCESS-SMC.87 • Molecular markers surveys to measure the prevalence of resistance to SMC drugs.88 • Additional coverage surveys for treatments delivered in 2016. • "Pharmacovigilance" studies to detect adverse events related to SMC drugs.89 Malaria Consortium told us that detailed reports from much of the above monitoring and evaluation should be available in April 2017.90 ## What do you get for your dollar? ### Cost for SMC coverage There are at least two major perspectives from which to evaluate the costs of SMC coverage. One perspective is to consider the marginal cost required to reach additional children with SMC in a country where basic program infrastructure and routine monitoring systems are in place. One estimate from this perspective finds that an additional 4 person-months of SMC coverage costs about$3.45 to $6.07.91 This estimate is based on a study of ACCESS-SMC conducted by Management Sciences for Health (see ACCESS-SMC DRAFT multi-country cost analysis, August 2016). We have not yet vetted this estimate. Alternatively, in order to make program costs comparable across all of our top charities, we aim to estimate the total cost to all actors of supporting a given program. Our estimate of cost-per-treatment for SMC includes research costs, start-up costs (which may not recur), and costs incurred by actors such as governments, which makes this estimate somewhat conservative. Our estimates also rely on coverage survey estimates to approximate the number of people reached rather than administrative coverage estimates. With these assumptions, we estimate that the total cost to achieve the equivalent of four person-months of SMC coverage is about$9.37.92 Full details in GiveWell SMC cost per treatment estimate, November 2015.

Note that we prefer to include all costs incurred to carry out a project, not just those that the charity in question pays for itself. We believe that this gives the best view of what it costs to achieve a particular impact (such as saving a life) and also avoids the lack of clarity and complications of leverage in charity. Our estimate of the cost to distribute SMC treatments aims to include both costs paid by Malaria Consortium and costs paid by others.

### Cost-effectiveness

SMC programs appear to be in the range of cost-effectiveness of our other priority programs. See our most recent cost-effectiveness model for estimates of the cost per equivalent life saved through Malaria Consortium-supported SMC programs.

Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. For example, our model does not include the short-term impact of non-fatal cases of malaria prevented. It also does not include possible offsetting impacts or other harms.93

There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

## Is there room for more funding?

We believe that Malaria Consortium could productively use more funding than it expects to receive to scale up its SMC activities. However, we have not yet had in-depth discussions with Malaria Consortium about its finances, so we are less confident that the below information is comprehensive and accurate than we are for our other top charities.

In short:

• Estimated needs: We have not yet attempted to estimate Malaria Consortium's maximum room for more funding. We would guess that Malaria Consortium could productively use at least an additional $30 million to scale up its SMC activities over the next three to four years. We have a general understanding of where additional funds would be used but have not yet asked for a high level of detail on potential bottlenecks to scaling up, so we are unsure about the precise amount of additional funding that Malaria Consortium could use. (More.) • Cash on hand: We have not yet had in-depth discussions with Malaria Consortium about its finances, but based on the information that we have seen, we do not believe Malaria Consortium has substantial unrestricted funding available for scaling up its support of SMC programs.94 • Other sources of funds: Malaria Consortium told us about other funds that it expects to receive to support its SMC programs, including from donors such as UNITAID. We have incorporated these expectations into our room for more funding analysis and believe that substantial funding gaps remain. (More.) • Past spending: It appears that in the recent past unrestricted expenditure has made up about 10%-14% of Malaria Consortium's total expenditure (the remaining expenditures were restricted) and that it has not used a substantial amount of unrestricted funding to support its SMC programs.95 • Additional considerations: Malaria Consortium spent roughly$57.7 million in its last budget year and works on a wide variety of programs.96 We have asked Malaria Consortium to use any GiveWell-influenced donations to specifically support SMC programs. Other relevant considerations to room for more funding include: possible lack of SMC drug supply, possible requirements for multi-year commitments to expand SMC programs, and potential delays in when funds would be spent. (More.)

### Available and expected funds

Our understanding is that all of Malaria Consortium's support for SMC programs is currently coming from UNITAID (which funds ACCESS-SMC) and that it does not have other potential funders that it expects to support its SMC work.97 As noted above, Malaria Consortium seems not to have allocated much unrestricted funding to its SMC programs in the past.98

Malaria Consortium is in the process of talking with UNITAID about how much funding it will provide for ACCESS-SMC in 2017 and 2018. These discussions are ongoing, but Malaria Consortium told us that its current best guess is that:99

• ACCESS-SMC will operate at roughly the same scale in 2017 as it did in 2016 in Nigeria, Chad, and Burkina Faso. In 2018, UNITAID will likely scale down funding for these countries by roughly 25-50%.100 It is unclear what funding for these countries will be provided beyond 2018.
• ACCESS-SMC will not continue providing treatments in Niger, Guinea, the Gambia, or Mali in 2017 and beyond unless it receives unexpected funding for these activities.
• Malaria Consortium told us that a major reason that UNITAID is scaling down its support for ACCESS-SMC is that part of its strategy is to provide ‘catalytic’ funding for relatively new programs (such as SMC) to prove that they work at a large scale with the goal of having other large funders (and/or domestic resources) fill the gaps for those programs in the future.101 We have not yet spoken with UNITAID about its thinking.

Malaria Consortium told us that its plans for how and where to use additional funding would depend on how much funding it receives.

If it received less than $4 million, Malaria Consortium told us that it would prioritize reaching additional children in countries where it is easiest to provide SMC in new districts on a short timeline.102 It told us that it would likely prioritize achieving full coverage in Burkina Faso first (which may cost roughly$2-3 million), and with remaining funds would try to reach children in Chad and/or Nigeria.103

If it received $4 million or more, Malaria Consortium told us that it would prioritize multi-year commitments to expand its programs into new states or regions.104 It told us that expanding into new states in Nigeria would be its highest priority because it expects Nigeria to have the largest unmet needs for SMC in the next few years and that Nigeria has a high malaria burden.105 One factor influencing funding needs in Nigeria is that in May 2016 the Global Fund announced that some of the funds it sent to Nigeria had been embezzled and that it would suspend disbursements to the National Malaria Elimination Program; these developments may lead the Global Fund to provide substantially less funding to Nigeria in the next few years.106 Malaria Consortium told us that it could use at least about$2M-$10M per year in Nigeria and that it would prefer to have at least two years of funding committed before expanding a program (for total room for more funding of at least$20M).107 It also told us that it could use at least an additional $2M-$5M per year to expand its programs in Chad (for total room for more funding gap of at least $10M).108 ### Additional considerations relevant to assessing Malaria Consortium's room for more funding • Lack of drug supply: Malaria Consortium told us that in 2015 ACCESS-SMC was limited to purchasing about 15 million treatments due to lack of supply of SMC drugs.109 In 2016, supply substantially expanded and ACCESS-SMC said it was able to purchase about 30 million treatments (and that it was not constrained by lack of drug supply).110 However, currently only one producer is supplying all global SMC drugs (Malaria Consortium told us that in 2016 about 60 million treatments were purchased in total).111 Malaria Consortium told us that it does not expect lack of drug supply to be a bottleneck to delivering more SMC treatments in 2017 and 2018 and that it is optimistic that supply of SMC treatments will expand in 2018; it told us that a new firm is applying for its products to be qualified so that it can provide SMC drugs in 2018.112 However, it seems that the level of expected purchases is uncertain, and we are uncertain about whether lack of drug supply will limit scale-up of SMC in the future.113 • Multi-year funding: Malaria Consortium told us that in order to expand its SMC programs to a new region it would need enough funding to commit to multiple years of delivering treatments.114 It told us that it would be irresponsible to scale up SMC in a new region for only one year because a) it would be less likely that the program would be established enough for other funders or the government to support it in the longer term, and b) there is a possibility that delivering treatments for only one year would leave many children susceptible to malaria "rebound"—i.e., higher levels of malaria in a subsequent year due to lack of immunity.115 We are unsure whether two or three years of funding would need to be committed before Malaria Consortium would be open to expanding to a new region. • Timeline of distributions: Malaria Consortium told us that it generally needs about 10 months to plan a SMC distribution after it has received the funds for the distribution.116 We are unsure whether funds that Malaria Consortium receives before the end of 2016 would be used to deliver SMC treatments during the 2017 rainy season or the 2018 rainy season. Malaria Consortium told us that if it had a strong sense of how much funding it would receive before January 2017 it may be able to deliver additional treatments in 2017, but otherwise it would use additional funding for the 2018 season.117 • Ability to scale: One reason that we believe that Malaria Consortium could likely use a large amount of additional funds relatively quickly (i.e., in the next three to four years) is that it supported the scale-up of ACCESS-SMC, which aimed to deliver about 45 million treatments. Malaria Consortium told us that it took about ten months of preparation before ACCESS-SMC began delivering treatments in 2015.118 • Fungibility: Since Malaria Consortium works on a variety of programs, it is possible that receiving additional funds for its SMC work could lead it to reallocate unrestricted funds or other organizational resources toward other programs, so that additional dollars donated to Malaria Consortium would not fully support additional SMC work. However, as discussed above, we do not see this as a major concern because we do not believe that Malaria Consortium has substantial unrestricted funding available, and it seems that Malaria Consortium has not allocated substantial unrestricted funding to SMC work in the past.119 ### Global need for treatment It seems that there is potentially a large amount of global need remaining for additional SMC treatments. In brief: • The WHO estimates that about 23.7 million children are eligible for SMC, mostly in the Sahel region of Africa.120 Malaria Consortium told us that ACCESS-SMC aimed to reach nearly 7 million children in 2016, that it supported about 50% of all global treatments, and that it would likely reach fewer children in 2017 and 2018 due to lack of funds.121 This implies that in 2016 roughly 14 million children were being reached (depending on coverage of other programs). We have not vetted these estimates. • ACCESS-SMC estimated that in the seven countries in which it was working in 2016, about 54% of eligible children would not be reached and the total cost of filling the unmet need for 2016 was approximately$49 million.122 We have not vetted this estimate.
• We have not yet done a detailed landscape of other likely funders of SMC work. Malaria Consortium told us that other major funders of this work include the Global Fund, the World Bank, the President's Malaria Initiative, UNICEF, and country governments.123 Malaria Consortium provided us with estimates of how much funding for SMC these other funders would provide in the next few years in this document—ACCESS-SMC, SMC donor mapping 2016 (with GiveWell calculations)—and it appears that it expects global funding for SMC to decline slightly in 2017 and 2018 relative to 2016.124 We have not vetted these estimates.
• Malaria Consortium told us that it is hopeful that in the longer term the Global Fund will be able to fully fund SMC in all countries that need it, but that this is unlikely to happen soon and that country governments need to be convinced of the value of SMC to prioritize using Global Fund funding to support it.125 Malaria Consortium told us that it believes an important part of its role is collecting evidence and operational knowledge that could persuade governments and other funders to support SMC in the future.126 Malaria Consortium told us that the Global Fund is planning to take over funding SMC work in some of the districts that ACCESS-SMC previously supported in Mali, Guinea, Niger, and the Gambia.127 However, Malaria Consortium noted that one potential limitation with other funders supporting this work is that they may be less likely to do detailed monitoring on important questions such as whether drug resistance is emerging.128 Also, it said that long-term commitments remain uncertain: Global Fund support in ACCESS-SMC districts is only confirmed for 2017, not yet beyond, but is in many of the Global Fund concept notes for 2018-20.129
• It is possible that lack of drug supply or other non-monetary bottlenecks, such as lack of distribution system capacity or lack of government interest, could limit scale-up of SMC in the future.130

## Major remaining questions

Our major remaining questions include:

• How cost-effective are SMC programs after fully reviewing the evidence base for those programs?
• How were coverage surveys carried out in practice?
• To best estimate the likely coverage of SMC programs, should we factor in administrative coverage rates in addition to using results from coverage surveys?
• How likely is it that Malaria Consortium will be able to use 30 million or more to scale up SMC programs over the next three to four years? Could Malaria Consortium absorb more than this level of funding? • Will other funders such as the Global Fund fill the funding gap for SMC treatment? Will lack of drug supply limit the scale-up of SMC in the future? • What will the upcoming studies on ACCESS-SMC show about its impact and the coverage rates that it achieved in 2016? • Will our estimates of cost per treatment shift substantially after learning more about SMC programs? • How much time will there be before drug resistance develops?131 • How does Malaria Consortium decide about whether to allocate unrestricted funding to SMC work? ## Malaria Consortium as an organization We have spent significantly less time investigating Malaria Consortium and have substantially less insight into its activities and track record than we do for top charities which we have followed for several years. As such, we have a limited view on the qualities below. • Track record: Malaria Consortium has experience with supporting large-scale SMC programs, particularly through its work on ACCESS-SMC, though we have only seen monitoring from one year of this work. • Self-evaluation: Malaria Consortium's self-evaluation is strong compared to the vast majority of organizations we have considered. • Communication: We have not spent as much time communicating with Malaria Consortium as we have with charities that we have recommended for several years. To date, Malaria Consortium has generally communicated clearly with us. • Transparency: Malaria Consortium is transparent. It has allowed us to publish most of the information it has shared with us. More on how we think about evaluating organizations at our 2012 blog post. ## Sources Document Source ACCESS-SMC DRAFT multi-country cost analysis, August 2016 Source ACCESS-SMC fact sheet Source (archive) ACCESS-SMC M&E strategy, November 2015 Source ACCESS-SMC Presentation, "Progress in scale-up of SMC in the Sahel," November 2016 Source ACCESS-SMC project brochure Source (archive) ACCESS-SMC Register Template Source ACCESS-SMC website, "The Project" Source (archive) ACCESS-SMC, 2016 presentation on safety monitoring Source ACCESS-SMC, adverse events cycle 1 in Nigeria 2015 Source ACCESS-SMC, Health impact data Unpublished ACCESS-SMC, Nigeria coverage summary report 2015 Source ACCESS-SMC, organizational structure Source ACCESS-SMC, presentation on cost and impact, June 2016 Source (archive) ACCESS-SMC, Questionnaire for SMC coverage survey Source ACCESS-SMC, Reported severe adverse events Source ACCESS-SMC, SMC donor mapping 2016 (with GiveWell calculations) Source ACCESS-SMC, summary administrative coverage spreadsheet Source ACCESS-SMC, summary of serious adverse events 2015 Source Cost analysis of the seasonal malaria chemoprevention project in Katsina state, Nigeria Source Gates Foundation, "Malaria Consortium" Source (archive) GiveWell SMC cost per treatment estimate, November 2015 Source GiveWell summary of SMC coverage information, November 2015 Source GiveWell's non-verbatim summary of a conversation with Malaria Consortium Staff, August 25, 2016 Unpublished GiveWell's non-verbatim summary of a conversation with Malaria Consortium Staff, November 23, 2016 Unpublished GiveWell's non-verbatim summary of conversations with Malaria Consortium staff, November 7 and November 9, 2016 Unpublished Global Fund, Executive Director statement on Nigeria, May 2016 Source (archive) Global Fund, Investigation in Nigeria, May 2016 Source (archive) Malaria Consortium emails (unpublished), November 23, 2016 Unpublished Malaria Consortium Quiz Answer Key Source Malaria Consortium training, competency checklist Unpublished Malaria Consortium training, daily evaluation Unpublished Malaria Consortium training, Final evaluation Unpublished Malaria Consortium training, pre- and post- test scores Unpublished Malaria Consortium website, "Seasonal Malaria Chemoprevention" Source (archive) Malaria Consortium website, "Who We Are" Source (archive) Malaria Consortium, "Seasonal Malaria Chemoprevention Programme Start-Up Guide, Nigeria" Source (archive) Malaria Consortium, "Support Scale up of Seasonal Malaria Chemoprevention (SMC)" Source (archive) Malaria Consortium, 2013 operational data Source Malaria Consortium, ACCESS-SMC announcement, May 8, 2014 Source (archive) Malaria Consortium, ACCESS-SMC page Source (archive) Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention Source Malaria Consortium, Annual Reviews Source (archive) Malaria Consortium, drug resistance Katsina study Source Malaria Consortium, excerpts of results and analyses in Katsina Source Malaria Consortium, M&E Plan Nigeria Source Malaria Consortium, Monitoring and evaluation summary Nigeria Source Malaria Consortium, Nigeria SMC evaluation report, 2014 Source Malaria Consortium, Project Brief: Seasonal malaria chemoprevention, Katsina Source (archive) Malaria Consortium, restricted and unrestricted expenditure analysis 2016 Source Malaria Consortium, sentinel data from Dutsi and Mai'adua Source Malaria Consortium, sentinel site surveillance data in Katsina Source Malaria Consortium, SMC presentation host page Source (archive) Malaria Consortium, SMC presentation, May 6, 2014 Source (archive) Malaria Consortium, summary of SMC donors Unpublished Malaria Consortium, SuNMaP Final Report Source Malaria Consortium, Trustees' report and financial statements 2015 Source (archive) Malaria Consortium, Trustees' report and financial statements 2016 Source Ndiaye et al. 2016 Source (archive) SMC Record Card Template 2016 Source Strachan et al. 2016 Source (archive) Timothy Rubashembusya Trip Report from Burkina Faso, August 2016 Source Wikipedia, "Cluster sampling" Source (archive) World Health Organization, "Seasonal Malaria Chemoprevention: A Field Guide" Source (archive) World Health Organization, "Seasonal Malaria Chemoprevention" Source (archive) • 1. • 2. "Established in 2003, Malaria Consortium is one of the world’s leading non-profit organisations specialising in the prevention, control and treatment of malaria and other communicable diseases among vulnerable populations. Our mission is to improve lives in Africa and Asia through sustainable, evidence-based programmes that combat targeted diseases and promote child and maternal health." Malaria Consortium website, "Who We Are". • 3. "Established in 2003, Malaria Consortium is one of the world’s leading non-profit organisations specialising in the prevention, control and treatment of malaria and other communicable diseases among vulnerable populations. ...With 95 percent of our staff working in malaria endemic areas, we currently have programmes and projects in 12 countries across Africa and Southeast Asia. Our local insight, embedded technical expertise and practical skills give us the agility to respond to critical challenges quickly and effectively." Malaria Consortium website, "Who We Are". • 4. • 5. "In March 2012, the World Health Organisation (WHO) issued a policy recommendation for a new intervention against Plasmodium falciparum malaria - seasonal malaria chemoprevention (SMC), previously referred to as intermittent preventive treatment in children (IPTc), in children under five years old. SMC is defined as the intermittent administration of full treatment courses of an anti-malarial treatment combination during the malaria season to prevent illness and death from the disease. The objective is to maintain therapeutic anti-malarial drug concentrations in the blood throughout the period of greatest risk. This will reduce the incidence of both simple and severe malaria disease and the associated anaemia and result in healthier, stronger children able to develop and grow without the interruption of disease episodes. SMC has been shown to be effective, cost effective and feasible for the prevention of malaria among children in areas where the malaria transmission season is no longer than four months." Malaria Consortium website, "Seasonal Malaria Chemoprevention". • 6. • 7. • 8. • 9. • 10. • 11. • 12. "How is SMC delivered? Local announcements each month will inform the community about the date of SMC, which will be delivered by community health workers at pre-arranged locations in the community, or by visiting each household. Health workers will receive appropriate training before the intervention begins and will be supervised by nurses and the district health team."ACCESS-SMC project brochure, Pg 3. • 13. • For Malaria Consortium's SMC program in Nigeria: "House to house delivery method was the most used approach, as reported by 88.2 percent of the respondents. This was similar across the LGAs [local government areas] though Maiadua had slightly higher numbers receiving through the fixed point delivery approach. The duration spent in receipt of drugs in the home was 20 minutes, half the time spent in receipt of drugs from a fixed point which was 47 minutes. Knowledge of the different types of SMC drugs and dose duration was high at over 80 percent. This highlights house to house delivery of SMC as a quicker and most preferred delivery mechanism by the caregivers. There is need for costing the two delivery mechanisms to assess if home based delivery still remains a cost effective delivery channel." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 38. • For ACCESS-SMC delivery methods, see the annexes on Pgs 36-69 of ACCESS-SMC DRAFT multi-country cost analysis, August 2016. Sample quote: "A combination of 6,500 trained community distributors and 355 health facility staff (e.g. nurses and midwives) distributed SMC by way of two distribution methods: door-to-door (two-person teams) and at fixed points located at health facilities that worked as referral centers (one-person teams). It was estimated that 90% of SMC was distributed by door-to- door teams and 10% was distributed at fixed points. To ensure the acceptability of SMC and high rates of coverage within communities, 3,483 trained community mobilizers sensitized communities on the benefits of SMC prior to and during each distribution cycle." ACCESS-SMC DRAFT multi-country cost analysis, August 2016, Pg 36. • 14. • 15. • 16. • 17. Malaria Consortium has shared some examples of tools used during training with us, but we have not yet reviewed them carefully. Since these sources include individuals' names, we cannot publish them. The examples of tools we have seen include: • 18. • 19. • 20. • 21. • See Figure 1, Pg 8, Malaria Consortium, "Seasonal Malaria Chemoprevention Programme Start-Up Guide, Nigeria". "DOT" stands for "Directly Observed Treatment". • "How long should the Role Model Caregiver observe each child after giving SMC medicines? a) 10 minutes, b) 15 minutes, c) 30 minutes, d) 1 hour, [Correct answer: C]" Malaria Consortium Quiz Answer Key. • "What should the Role Model Caregiver advise the child’s caregiver after giving the first dose of the SMC medicines? a) When to take the second and third dose of Amodiaquine (AQ) at home, b) The importance of adherence to giving the two doses of Amodiaquine (AQ) home, c) What to do if the child vomits, d) How to mark the SMC Record Card after giving each dose and to bring the card back for the next SMC cycle, e) When to go to the health facility if the child gets a fever or very sick, f) All of above, [Correct Answer: F.]" Malaria Consortium Quiz Answer Key. • "The child’s SMC Record Card is very important because: a) It shows the Role Model Caregiver the name and register number of the child, b) The child’s caregiver should always take it with them if they need to go to the health facility, c) It shows how many times the child received the SMC medicines each month, d) It is made of thick paper and is in a plastic packet, e) a, b and c, f) All of the above, [Correct answer: E.]" Malaria Consortium Quiz Answer Key. • 22. • "The child’s SMC Record Card is very important because: a) It shows the Role Model Caregiver the name and register number of the child, b) The child’s caregiver should always take it with them if they need to go to the health facility, c) It shows how many times the child received the SMC medicines each month, d) It is made of thick paper and is in a plastic packet, e) a, b and c, f) All of the above, [Correct answer: E.]" Malaria Consortium Quiz Answer Key. • We have seen a few versions of templates for "SMC Record Cards." The latest version that we have seen (from 2016) is here: SMC Record Card Template 2016. • 23. • 24. • 25. • "Who should NOT get SMC medicines? a) Any child with a fever or who is severely ill, b) Any child who is currently taking a sulfa medication such as co-trimoxazole (Septrin, or Bactrim), c) A child who has received a dose of either Amodiaquine (AQ) and sulfadoxine / pyrimethamine (SP) during the past month, d) A child who is allergic to sulfa medication such as co-trimoxazole, Septrin, or Bactrim, e) A child who is allergic to either Amodiaquine (AQ) and sulfadoxine / pyrimethamine (SP), f) a,b,ande, g) All of the above. [Correct answer is G.]" Malaria Consortium Quiz Answer Key. • "What important questions must the Role Model Caregiver ask about each child before giving SMC medicines? a) The child’s age, b) If the child has taken any medicines in the past 28 days, and which ones, c) If the child has any allergies, d) If the child has a fever or is sick, e) a, b, and d, f) All of the above, [Correct answer is F.]" Malaria Consortium Quiz Answer Key. • "What should the Role Model Caregiver do if a child has a fever on the day SMC medicines are being given? a) Complete the SMC Referral Form, b) Refer the child to the nearest health facility for a malaria test, c) Complete the SMC Register with the reason for the referral, d) Give the child Coartem, e) Give the child the SMC medicines to treat the fever, f) a, b, and c, g) All of the above, [Correct answer is F.]" Malaria Consortium Quiz Answer Key. • 26. • 27. • "SMC medicines come in two colour packets for different age children. What age group should get the medicines in the YELLOW packets? ... [correct answer:] b) 3 to 12 months " Malaria Consortium Quiz Answer Key. • "SMC medicines come in two colour packets for different age children. What age group should get the medicines in the BLUE packets? ... [correct answer:] b) 12 to 59 months" Malaria Consortium Quiz Answer Key. • 28. "Malaria Consortium notes that there are several layers of supervision that provide spot-check control of how well the administration process is followed: • There is in general a “proximity supervisory team” (CHWs with higher education or literacy skills / experience or a junior health worker) who follow a number of distribution teams (for instance, in Burkina Faso it’s 1 for every 5 CHW teams, in Nigeria, 1 for every 3 community drug distributor (CDD) teams). They work 3 to 4 days per cycle. • Health workers from one health facility carry out spot-check corrective supervision over all CDDs/CHWs and all supervisors in their catchment area; they work 3 to 4 days per cycle. • District health officials carry out supervisory visits over both distributors (to check quality of administration) and supervisors (to check on the quality / frequency of supervision). They work 3 to 4 days per cycle. • Regional health officials carry supervisory spot-check visits over all the levels below, to check on the quality of administration and of supervision and provide corrective guidance. They work 2 to 4 days per cycle. • National Malaria Control Program (NMCP) supervisors carry out supervisory spot-check visits over all the levels below (as above). They work 3 to 4 days per cycle. • Malaria Consortium and partner’s country supervisors / teams carry out spot-check visits (independently or jointly with health authorities, as above). They work 4 days per cycle on supervision and implementation monitoring. • AfRO Malaria Consortium and partner staff carry out regular country field visits including spot-check supervision to control all of the above. About 2 visits per country during the SMC round at minimum. The system is quite extensive and well-rehearsed, since it is used across several child survival, mass prevention campaigns. While it is likely that there will be some mistakes at this scale (in particular, age misclassification and eligibility are common issues for all mass campaigns, with no easy, inexpensive solutions around it), major mistakes in administration are likely to be caught. While these have not been quantified, in nearly 10,000 estimated instances of spot-check supervision across the region, anecdotal instances of reported gross negligence in administration of drugs, that could be identified in 2015, were within the hundreds, and promptly corrected: these mostly related to lack of observation of hygiene practices, excessive use of water, ignoring directly observed therapy (DOT) protocols, severe age misclassification, non-respect of observation time, and premature administration of second DOT after vomiting. Most of the other problems identified were, instead, around appropriate administration record-keeping, due to low literacy levels of volunteers in many countries. Please mind that no other mass campaign outside operational research is known to keep detailed individual records of recipients. In most mass campaigns that are known to be safe and effective (measles, polio, pneumonia, vitamin A, deworming, NTD MDAs, just to name a few), only simple tally sheets and sample in-process monitoring are used. Due to the research needs of a new intervention, SMC is being held to extremely high accountability standards as compared to other interventions. It is unlikely that individualized records and these layered levels of supervision are sustainable in the long term and/or at an increased scope, because of the implication on resources needed and costs to local budgets." Malaria Consortium emails (unpublished), November 23, 2016. • 29. • 30. • "Budget: 1,694,339.00 (USD)" Malaria Consortium, "Support Scale up of Seasonal Malaria Chemoprevention (SMC)". We also searched the Gates Foundation's grant database to see whether it made any additional grants to Malaria Consortium for SMC work, but only saw this grant (grant page available at Gates Foundation, "Malaria Consortium"). • "Malaria Consortium is implementing and assessing the feasibility of a community-based seasonal malaria chemoprevention (SMC) project in Katsina state, northern Nigeria, with funding from the Bill & Melinda Gates Foundation. Following new World Health Organisation policy recommendations on SMC, this project administers full antimalarial treatments during the malaria season in areas with highly seasonal malaria transmission, to prevent illness among children under five." Malaria Consortium, Project Brief: Seasonal malaria chemoprevention, Katsina, Pg 1. • "The project’s objectives are: • To design, in consultation with key local stakeholders, an appropriate community-based delivery system for SMC in Katsina state based on formative research, which will review aspects relating to feasibility, community acceptability, effectiveness and cost • To launch and execute SMC delivery according to the selected delivery system and collect data on process indicators and costs • To evaluate community acceptability, costs and effectiveness of the delivery system for SMC • To inform future national and state plans for SMC continuation/ scale up by disseminating findings and sharing experiences with key stakeholders" Malaria Consortium, Project Brief: Seasonal malaria chemoprevention, Katsina, Pg 2. • It appears that this project may have also been related to another major (£89 million) project that Malaria Consortium was working on in Nigeria called "Support to National Malaria Programme (SuNMaP)". • "Support to National Malaria Programme (SuNMaP) is an £89 million UK aid funded project that works with the government and people of Nigeria to strengthen the national effort to control malaria. The programme began in April 2008 and [ended] in March 2016. Led by Malaria Consortium, SuNMaP was jointly managed by a consortium, including lead partners Health Partners International and GRID Consulting, with nine other implementing partners. SuNMaP was implemented in 10 states across Nigeria, including Anambra, Kano, Niger, Katsina, Ogun, Lagos, Jigawa, Enugu, Kaduna and Yobe. SuNMaP worked with the Nigerian government's National Malaria Elimination Programme (NMEP) to harmonise donor efforts and funding agencies around national policies and plans for malaria control. Project targets were aligned with the National Malaria Strategic Plan and Global Malaria Action Plan. The project aimed to improve national, state and local government level capacity for the prevention and treatment of malaria." Malaria Consortium, SuNMaP Final Report, Pg 38. • "July 2013: Result of SuNMaP study on efficacy of sulphadoxine‐pyrimethamine (SP) for intermittent treatment against malaria in pregnancy published. SuNMaP commences seasonal malaria chemoprevention in Katsina State." Malaria Consortium, SuNMaP Final Report, Pg 16. • 31. • 32. • "The project’s objectives are: ...To inform future national and state plans for SMC continuation/scale up by disseminating findings and sharing experiences with key stakeholders," Malaria Consortium, Project Brief: Seasonal malaria chemoprevention, Katsina, Pg 1. • Strachan et al. 2016 Abstract: • "Background: Experience of seasonal malaria chemoprevention (SMC) is growing in the Sahel sub-region of Africa, though there remains insufficient evidence to recommend a standard deployment strategy. In 2012, a project was initiated in Katsina state, northern Nigeria, to design an appropriate and effective community-based delivery approach for SMC, in consultation with local stakeholders. Formative research (FR) was conducted locally to explore the potential feasibility and acceptability of SMC and to highlight information gaps and practical considerations to inform the intervention design. • Methods: The FR adopted qualitative methods; 36 in-depth interviews and 18 focus group discussions were conducted across 13 target groups active across the health system and within the community. Analysis followed the ‘framework’ approach. The process for incorporating the FR results into the project design was iterative which was initiated by a week-long ‘intervention design’ workshop with relevant stakeholders. • Results: The FR highlighted both supportive and hindering factors to be considered in the intervention design. Malaria control was identified as a community priority, the community health workers were a trusted resource and the local leadership exerted strong influence over household decisions. However, there were perceived challenges with quality of care at both community and health facility levels, referral linkage and supportive supervision were weak, literacy levels lower than anticipated and there was the potential for suspicion of ‘outside’ interventions. There was broad consensus across target groups that community-based SMC drug delivery would better enable a high coverage of beneficiaries and potentially garner wider community support. A mixed approach was recommended, including both community fixed-point and household-to-household SMC delivery. The FR findings were used to inform the overall distribution strategy, mechanisms for integration into the health system, capacity building and training approaches, supportive interventions to strengthen the health system, and the social mobilization strategy. • Conclusions: Formative research played a valuable role in exploring local socio-cultural contexts and health system realities. Both opportunities and challenges for the introduction of SMC delivery were highlighted, which were appropriately considered in the design of the project." • 33. • "UNITAID has awarded up to67 million to Malaria Consortium to oversee the largest-yet global programme to increase seasonal malaria chemoprevention (SMC) across the Sahel region of Africa, where malaria remains the leading cause of severe illness and mortality in young children...This grant will help increase capacity and reduce prices for SMC products in the seven target countries and is expected to supply an estimated 30 million treatments every year to protect 7.5 million children, preventing around 50,000 deaths." Malaria Consortium, ACCESS-SMC announcement, May 8, 2014.
• "ACCESS-SMC is a UNITAID-funded project, led by Malaria Consortium in partnership with Catholic Relief Services, which is scaling up access to seasonal malaria chemoprevention (SMC) across the Sahel to save children’s lives. This three year project is supported by London School of Hygiene & Tropical Medicine, Centre de Support de Santé International, Management Sciences for Health, Medicines for Malaria Venture, and Speak Up Africa. It will provide up to 30 million SMC treatments annually to 10 million children less than five years of age [specifically, children ages 3 to 59 months] in Burkina Faso, Chad, Guinea, Mali, Niger, Nigeria and The Gambia, potentially averting 49,000 deaths due to malaria." Malaria Consortium, ACCESS-SMC page.
• "ACCESS-SMC is working with all seven supported National Malaria Control Programs (NMCPs) to create SMC delivery pathways. We are providing support in a range of areas, including planning and management, supply chain, health worker training and communications and social mobilization. The project is also procuring almost 15m doses of SMC drugs in 2015, and up to 30m in 2016. Working together with NMCPs, we will reach 3.3m children in 2015 and up to 6.6m children by 2016 in Burkina Faso, Chad, Guinea, Mali, Niger, Nigeria and The Gambia." ACCESS-SMC website, "The Project".
• Clarifications around number of children targeted annually from Malaria Consortium emails (unpublished), November 23, 2016.
• 34.

"UNITAID has awarded up to $67 million to Malaria Consortium to oversee the largest-yet global programme to increase seasonal malaria chemoprevention (SMC) across the Sahel region of Africa, where malaria remains the leading cause of severe illness and mortality in young children." Malaria Consortium, ACCESS-SMC announcement, May 8, 2014. • 35. • "Malaria Consortium’s three year project, known as ACCESS SMC, will run across seven African countries: Burkina Faso, Chad, Guinea Conakry, Mali, Niger, Nigeria and The Gambia. ACCESS-SMC will be led by Malaria Consortium, with Catholic Relief Services as the primary sub-grantee. It will be supported by London School of Hygiene & Tropical Medicine, Management Sciences for Health, Medicines for Malaria Venture, Speak Up Africa and Centre de Support en Sante International." Malaria Consortium, ACCESS-SMC announcement, May 8, 2014. • "The ACCESS-SMC partnership: • Malaria Consortium is leading the ACCESS-SMC project, tracking its impact, managing the procurement of SMC drugs and supporting malaria control programmes to implement SMC in Burkina Faso, Chad and Nigeria. • Catholic Relief Services is the lead-subrecipient and contributing to tracking the reach and impact of the project and supporting malaria control programmes to implement SMC in Guinea, Mali, Niger and The Gambia. • London School of Hygiene & Tropical Medicine is generating evidence on drug resistance, strengthening pharmacovigilance and measuring SMC’s public health impact. • Medicines for Malaria Ventures is supporting manufacturers to develop a child friendly, dispersible formulation and ensuring accurate drug forecasting. • Management Sciences for Health is measuring the cost of SMC and working with countries to optimize the SMC supply chain. • Speak Up Africa is creating an integrated health communications and advocacy campaign. The complementary nature of the partnership, which includes non-profit and academic institutions, allows substantial geographic reach and ensures that good evidence will guide future SMC implementation." ACCESS-SMC fact sheet, Pg 2. • 36. • 37. • 38. • 39. • 40. See Cells B14, C14, and D14 in GiveWell summary of SMC coverage information, November 2015. • 41. For example, see: • "Malaria is still a serious public health concern in Nigeria, with fevers presumed to be malaria accounting for 60 percent of outpatient visits to health facilities, 30 percent of childhood deaths, 25 percent of deaths in children under one year and 11 percent of maternal deaths. In northern Nigeria, where malaria transmission is highly seasonal, malaria prevalence is also comparatively higher than other areas during the rainy seasons. The implementation of SMC in Katsina is specifically intended to reduce mortality in children under five living in areas with seasonal malaria, and strengthen health systems at the state and national levels." Malaria Consortium, Project Brief: Seasonal malaria chemoprevention, Katsina, Pg 1. • Under "Why Katsina State" in Malaria Consortium, SMC presentation, May 6, 2014: • "Katsina State is within the Sahel Region; rainy season and peak malaria incidence from July to October • 2012 estimated population of 6,916,641 • 1,383,328 under-5 years • 600,281 cases of malaria (2008) • 4,103 malaria related deaths • Katsina overall under-5 mortality rate 180 per 1,000 live births • Also, Malaria Consortium told us that “eligible districts are established by the National Malaria Programmes in the countries applying the WHO rainy season seasonality rules.” Malaria Consortium emails (unpublished), November 23, 2016. • 42. See Cells B14, C14, and D14 in GiveWell summary of SMC coverage information, November 2015. • 43. A few sources that seem to discuss the two different methods are: • Timothy Rubashembusya Trip Report from Burkina Faso, August 2016 includes a table that shows discrepancies in estimated coverage rates between "MC data" and "LSHTM survey". See Pg 4. • Timothy Rubashembusya Trip Report from Burkina Faso, August 2016 writes: "Propose data quality assessments for 2015 distribution data: The background of this was due to major discrepancies of coverage rates arising between the LSHTM 2015 End of Cycle survey and the administrative data submitted. Whereas these are not expected to be exactly the same, there were some major discrepancies in Cycle 4 coverage rates arising from the LSHTM survey in relation to the data reported through our routine reports. Examining further the results from the LSHTM reports presents some questions about the numbers presented which will only be ascertained by further evaluation of where the problem arose from. For example, the LSHTM reports that the rate of children who received all the 4 cycles was 86.4%. This isn’t realistic as both Cycle 4 and Cycle 3 from the same survey showed rates less than 86.4% (ie. 68.4% and 82.0%, respectively). i.e the number of those who received all the 4 cycles would need to be less or equal to the least cycle coverage rate to be valid – which is not the case. Recommendation: These discrepancies will be assessed further in relation to other countries to enable propose a clear strategy. However in the interim, I have proposed to plan a data verification exercise for 3 randomly selected districts out of the 11 which were part of the 2015 SMC distribution. This verification exercise would only focus on the fourth cycle – given that this is where we see a major gap. Out of these 3 districts, we would need to validate the records available at 3 (also randomly selected) CSPSes by comparing these records with those submitted in the MC reports. I will collaborate directly with Damiba Jean-Dieudonne to see how this can be achieved and in what timeframe." Timothy Rubashembusya Trip Report from Burkina Faso, August 2016, Pgs 4-5. • "SMC coverage surveys: All eligible children in endemic areas are expected to receive SMC every month during the high transmission season. Although output level data will provide information on the number of children who received SMC during each distribution cycle, the real coverage at population level can only be measured through representative household surveys, similar to coverage assessment of the Expanded Programme of Immunization (EPI). Indeed, sampling households at community level will guarantee the inclusion of eligible children potentially missed by the distribution teams and the exclusion of outsiders to the targeted community." ACCESS-SMC M&E strategy, November 2015, Pg 8. • 44. • We came to this understanding through comments from Malaria Consortium emails (unpublished), November 23, 2016. • Relevant quotes include: • "Routine monitoring data included SMC distribution data, extracted from SMC registers and selected routine health facility Health Management Information System (HMIS) data obtained from selected sentinel sites. Routine SMC distribution data was extracted from community health workers (CHWs) at the end of each distribution cycle. This data was aggregated at settlement, ward and LGA levels and compared with the estimated eligible population totals to establish SMC coverage at each distribution cycle. SMC coverage, estimated from distribution data is highlighted in figure 1 below. A total of 487,353 treatment courses were delivered in two LGAs over three treatment cycles in the first round of SMC representing an average of 115% coverage over the three cycles. In 2014, a total of 1,078,440 treatments were provided across four LGAs over four cycles with an average of 115% administrative coverage." Malaria Consortium, Monitoring and evaluation summary Nigeria, Pg 2. • "Table 1: Routine data and sources...Children reached…[Routine data source:] CHWs & HF [Health Facility] SMC drug registers" See Table 1, Malaria Consortium, M&E Plan Nigeria, Pg 10. • We have seen a few versions of templates for registers that CHWs may use when distributing SMC drugs. The latest version that we have seen is here: ACCESS-SMC Register Template. • We have seen a few versions of templates for "SMC Record Cards." The latest version that we have seen (from 2016) is here: SMC Record Card Template 2016. • Regarding data management, Malaria Consortium writes: "The project will set up a data management system to track all indicators both routine and non-routine. All data collected by the project from the different data sources mentioned above will systematically be entered into a database resident at the project offices in Katsina. The system will have in built data cleaning, validation and reporting modules to allow timely extraction of performance reports based on the data submitted. Under the management of project research officer, the data management system will be routinely used for the following purposes: • Produce reports for feedback to providers of the data on project progress e.g. SMC coverage, trends in malaria presentation • Produce indicator performance statistics to aid project management and decision making regarding project implementation." Malaria Consortium, M&E Plan Nigeria, Pg 17. • We have not yet seen reports from the above data management system. • 45. • "Each CDD/CHW team fills – among other things – one tally sheet per day with one directly observed treatment per child successfully treated – among other information recorded, by age, and which they submit to their reference health facility (to the head nurse or doctor, as applicable). • The health workers consolidate tally sheets each day, and transmit information by SMC or calls or other means to district. • District data managers (or equivalent role) consolidate all tally sheet data from their facilities and submit for validation / consolidation to regions and NMCP / central administration. • This is how data for any mass campaign are managed, with slight variations in roles, tools, layers of control / validation, quality assurance elements." Malaria Consortium emails (unpublished), November 23, 2016. • 46. For example, see: • It appears that about 62% of targeted children received at least 3 rounds of SMC in 2014 according to coverage surveys. See Table 5.3, Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 31. However, it appears that administrative coverage showed coverage estimates of over 100%: "A total of 487,353 treatment courses were delivered in two LGAs over three treatment cycles in the first round of SMC representing an average of 115% coverage over the three cycles. In 2014, a total of 1,078,440 treatments were provided across four LGAs over four cycles with an average of 115% administrative coverage." Malaria Consortium, Monitoring and evaluation summary Nigeria, Pg 2. The figure 62% refers to the proportion of targeted children receiving at least 3 rounds of treatment according to coverage, while 115% refers to the average administrative rate of coverage in each round of treatment. While these numbers are not directly analogous, we believe they demonstrate the discrepancy between these two types of coverage estimates. • ACCESS-SMC: "Percent coverage of target (3-59 months) [who received four cycles]" versus "Percent of children (3-59 months) receiving four full cycles of SMC, based on LSHTM coverage survey" in Table 2, ACCESS-SMC DRAFT multi-country cost analysis, August 2016, Pg 18. • Timothy Rubashembusya Trip Report from Burkina Faso, August 2016 includes a table that shows discrepancies in estimated coverage rates between "MC data" and "LSHTM survey". See Pg 4. • 47. • 48. For estimates of how many children who were six to seven years old received at least one round of SMC in ACCESS-SMC countries in 2015, see Column H in GiveWell summary of SMC coverage information, November 2015. The estimates suggest that, among children in this age group whose caregivers were interviewed, 17%-85% received at least one round of treatments. • 49. • 50. Administrative coverage estimates often seem to find coverage rates ranging from about 70% to about 130%: • Summary information from Malaria Consortium-supported programs in northern Nigeria in 2013 and 2014: • "A total of 487,353 treatment courses were delivered in two LGAs over three treatment cycles in the first round of SMC representing an average of 115% coverage over the three cycles. In 2014, a total of 1,078,440 treatments were provided across four LGAs over four cycles with an average of 115% administrative coverage." Malaria Consortium, Monitoring and evaluation summary Nigeria, Pg 2. • Chart of coverage: • Summary information from ACCESS-SMC programs: • See Table 2, ACCESS-SMC DRAFT multi-country cost analysis, August 2016, Pg 18. • Regarding methodology, it notes: "The numbers of children who received the four cycles of SMC were calculated by dividing the total number of SMC drugs administered, by four months. These figures do not represent the actual number of children who received four cycles. The SMC coverage percentages were calculated by dividing the estimated numbers of children reached by the estimated target populations. The percent coverage ranged from 69.97% in Niger to 105.93% in Burkina Faso (see Table 2). SMC coverage rates of more than 100% were likely due to a combination of an initial underestimation of the target population or children who came from outside the catchment area who received SMC. In general, the demand for SMC increased between the first and fourth cycles with the exception of Mali where SMC coverage decreased after the second cycle. The coverage surveys conducted by the London School of Hygiene and Tropical Medicine (LSHTM) indicate that many of the children received fewer than four cycles of SMC, which presumably demonstrates that more children were reached in total but some received less than the recommended four cycles of SMC (i.e. three cycles or less). For example, in Chad and Mali, as per LSHTM coverage surveys, only 22.7% and 33.7% of children (3-59 months), respectively, received all four cycles of SMC (see Table 2)." ACCESS-SMC DRAFT multi-country cost analysis, August 2016, Pg 17. • There appears to be some administrative data in ACCESS-SMC, summary administrative coverage spreadsheet, but we are not sure of the methodology underlying it or what year it is from. • 51. • 52. This information is primarily drawn from Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pgs 13-14. • 53. • 54. • 55. • "A two-stage cluster sample survey will be used to provide estimates of the proportion of eligible children who received 0,1,2,3 or 4 SMC treatments, representative of the areas where SMC is implemented." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 13. • "Two-stage cluster sampling, a simple case of multistage sampling, is obtained by selecting cluster samples in the first stage and then selecting sample of elements from every sampled cluster. Consider a population of N clusters in total. In the first stage, n clusters are selected using ordinary cluster sampling method. In the second stage, simple random sampling is usually used.[3] It is used separately in every cluster and the numbers of elements selected from different clusters are not necessarily equal. The total number of clusters N, number of clusters selected n, and numbers of elements from selected clusters need to be pre-determined by the survey designer. Two-stage cluster sampling aims at minimizing survey costs and at the same time controlling the uncertainty related to estimates of interest.[4] This method can be used in health and social sciences. For instance, researchers used two-stage cluster sampling to generate a representative sample of the Iraqi population to conduct mortality surveys.[5] Sampling in this method can be quicker and more reliable than other methods, which is why this method is now used frequently." Wikipedia, "Cluster sampling". • "Sampling for the coverage survey: From a list of all communities (villages or census enumeration areas) in the areas that received SMC, about 60 communities will be selected with PPES [Probability proportional to estimated size]. In each selected community, a sample of an approximately constant number of households will be made from village lists or using area sampling. All eligible children resident in the household at the time of the survey should be included. If any children are away or the parent is not present to interview, a call-back visit should be arranged before documenting a non-response for that child or household." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 14. • 56. "Sample size: About 10 children per community are required for a sample size of 600 this will give good precision on the overall estimates of coverage while permitting reasonable precision on sub- regional estimates. To allow for the fact that children up to 7 yrs of age will be surveyed, the sample size is increased to 14 per cluster (total 60x14=840). In vaccination coverage surveys in the Gambia, the rate of homogeneity (roh) for DPT3 was about 0.2, this gives a design effect of Deff=1+(b- 1).roh=2.8 for a cluster size b=10. A survey of 600 then corresponds to a sample size of 600/2.8=214 if we were to use simple random sampling and this would give, if the coverage was 70%, a margin of error (95% confidence interval) of +/-6%, if the coverage was 50%, +/-7%. And for sub-regional estimates, if we divide the area into three with 200 sampled in each of these three areas, the precision would be 50% +/-12% or 70% +/-11%. For estimating coverage in the out-of-range children (>5yr and 7<yrs), if we have about 4 of these per cluster, a total of 60x4=240, we would have a margin of error of +/-8% if the coverage was 20%." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 14. • 57. • "A two-stage cluster sample survey will be used to provide estimates of the proportion of eligible children who received 0,1,2,3 or 4 SMC treatments, representative of the areas where SMC is implemented." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 13. • Regarding the definition of "eligible children": "Study populations: SMC is aimed at children aged 3 to 59 months old. Children who receive SMC at the first cycle of the year, but have their 5th birthday before the last SMC cycle, should continue to receive SMC that year, but will not be eligible next year. Therefore, in the survey, eligible children are defined as those who are aged at least 4 months at the time of the survey (hence would have been at least 3 months old when the 4th SMC cycle took place), and are aged less than 5 yrs and 4 months. However, it is possible that older children are receiving SMC and it will be useful to capture this. Therefore, in the survey, all children aged less than 7 years at the time of the survey will be included, and the date of birth and age carefully noted to allow estimates of coverage in the target age range to be made. The sample size will be increased to ensure the target number in the eligible age range is obtained. Care should be taken to obtain the correct date of birth, this should be noted from the child’s health card, where this is not available, the child’s age should be recorded as accurately as possible using an event calendar, and referring to children in the compound of similar age whose date of birth is known." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pgs 13-14. • 58. • Regarding how SMC Record Cards are used: Our understanding is that CHWs are expected to record the first dose of each cycle and the date they gave the dose on a family's Record Card and that the family is expected to record all other doses. We have seen a few versions of templates for "SMC Record Cards." The latest version that we have seen (from 2016) is here: SMC Record Card Template 2016. • "Children in endemic areas should receive SMC every month for up to 4 months in the transmission season. A key indicator that should be reported is the percentage of children that receive all of their scheduled SMC treatments. Provided SMC doses are accurately recorded on a family-held record card, this can be done through household surveys using a similar methodology to that employed for vaccination coverage surveys, but will need to be supplemented by asking mothers about treatments." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 7. • "Q62: Was (Name) given a card to keep record of the drugs given?...If No or DK => Q64." "Q64: Did the child take the first dose in the presence of the caregivers for each distribution cycle? Q65: Did you continue to give the tablets at home the following days?" ACCESS-SMC, Questionnaire for SMC coverage survey, Pgs 14-15. • "SMC doses: Dates of SMC doses will be recorded from the child’s SMC card. In countries where administration of SMC doses was recorded electronically, a hand-held device will be used to read the QR code on the child’s card. The mother or carer should also be asked about receipt of SMC doses to cross-check the information on the card and to provide information about doses where the card has been lost. A short questionnaire, including reasons for missed SMC cycles, and adherence to the supervised and home doses, occurrence of fever in the child in the last 2 weeks and any treatment seeking for that fever, and the use of bednets by the child, will be completed. Where possible, a cross-check on SMC dates should be made from the SMC register." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 14. • 59. Examples of questions are pulled from ACCESS-SMC, Questionnaire for SMC coverage survey and from Malaria Consortium emails (unpublished), November 23, 2016. • 60. Our understanding is that CHWs are expected to record the first dose of each cycle and the date they gave the dose on a family's Record Card and that the family is expected to record all other doses. • 61. • 62. See Cells B14, C14, and D14 in GiveWell summary of SMC coverage information, November 2015. • 63. Details of the Nigeria coverage survey's methodology (from ACCESS-SMC, Nigeria coverage summary report 2015): • "Background: Seasonal Malaria Chemoprevention supported by the ACCESS-SMC project was introduced in 17 LGAs in Nigeria in 2015 (10 in Sokoto State and 7 in Zamfara state). In 2015, the first of four monthly cycles of SMC was carried out in early August, and the final cycle took place in late October/ early November. It is important the SMC programme is monitored to ensure the intervention is delivered effectively reaching the children that need it. WHO recommends that children should receive all four monthly cycles, and should adhere to the treatment dose each month, in order to maximise protection and minimise selection for drug resistance. As part of the monitoring of the SMC programme in Nigeria, a coverage survey was carried out in mid-November 2015 by ERIC with support from the London School of Hygiene & Tropical Medicine, to determine the proportion of children that received each monthly treatment, and to ask about adherence to the treatment doses, and reasons for missed treatments." • "Methods: Within each LGA implementing SMC in 2015 supported by the ACCESS-SMC project, between 3 and 5 settlements were selected with probability proportional to size (PPS sampling). Within each settlement, a random sample of approximately 20 children was then taken. The survey included eligible children (SMC is given to children aged at least 3 months, and who were less than 5 years old at the first cycle). The survey also included older children up to 7 years of age in order to determine how many of these older children were being treated. Call-backs were done if the caregiver was absent. The caregiver was asked about the number of SMC treatments the child received and in which months SMC was given, and the SMC record card was inspected to record the dates of treatments on the card. Data were collected using tablet PCs." • "More details on Key indicators: 1380 children were included in the survey, 1112 of these were eligible to receive all 4 cycles of SMC in 2015 based on their age at the time of the first SMC cycle. Around 84% of eligible children received an SMC card. If an SMC card was received, approximately 75% of children retained the card for inspection. Based on the card (where available, and on recall of the number of blister packs received otherwise), 70% of eligible children received at least 3 cycles of SMC, and 48% received all 4 cycles. Recall was an important source of information in Nigeria, because the SMC card was often incomplete. Coverage of the individual cycles is difficult to estimate because this relies on documentation of dates on the card, which was not always done, so may lead to an underestimate. However, even allowing for these issues, it appears that coverage of the third and fourth cycles of SMC, was lower than the first 2 cycles, with coverage of the final cycle particularly low. Reported adherence to the three day course of SMC was good, with 84% of those who had received SMC reporting that the full course was given. Of 75 children aged between 6 and 7 years who should not receive SMC, 40 (53%) had received an SMC card. For 46 (61%) of these older children, caregivers reported that the child had received at least one SMC cycle, with about 49% reporting that the child had received at least 3 cycles." • 64. "1380 children were included in the survey, 1112 of these were eligible to receive all 4 cycles of SMC in 2015 based on their age at the time of the first SMC cycle." ACCESS-SMC, Nigeria coverage summary report 2015, Pg 2. • 65. All findings drawn from ACCESS-SMC, Nigeria coverage summary report 2015. Slightly more detailed results are on Pg 3 of ACCESS-SMC, Nigeria coverage summary report 2015. • 66. This quote is pulled from an unpublishable source: ACCESS-SMC, Health impact data with edits from Malaria Consortium emails (unpublished), November 23, 2016. The original source wrote that if the probability of receiving SMC at each cycle is 88%, the expected percentage who would receive at least three treatments is 93%. However, assuming that the probability of receiving each round is independent of whether the child received other rounds we believe that the correct figure here would be 67%. See Cells AF4 and AF9 in GiveWell summary of SMC coverage information, November 2015. • 67. • 68. • "The survey was conducted in the 2nd phase LGAs which had received one round of SMC in from July to October 2014. Information on SMC receipt at each cycle was predominantly obtained from SMC cards given to the caregivers and from verbal responses of sampled caregivers in cases where the card could not be obtained." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 30. • 69. • "At each survey round, the survey was designed using a two-stage cluster sampling for a sample size of 750 households from 30 clusters (30 clusters X 25 households) selected with probability proportionate to size." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 7. • "Sampling and sample size: The survey employed a two stage cluster sample design. However, only sampling at the second stage was done. There was no re-sampling of clusters (first stage) as similar clusters sampled at baseline were used." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 4. • "Stage one: Stage one involved sampling of the primary sampling units (clusters) which were the enumeration areas (EA), usually settlements. At each of the survey rounds, a total of 30 clusters were selected with probability proportionate to size from which both the household and child health components of the surveys evaluation were conducted." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 4. • "Stage two: The second stage involved initially listing all households per cluster (settlement) for all the 30 clusters. From this, 25 households were selected per cluster using systematic sampling. A household was defined as a group of people who usually take their meals together. For enumeration areas of more than 150 households, an equal size section approach was used, i.e. the cluster was divided into two to four sections of approximately equal number of households, and one section was selected using simple random sampling. All households in that section were then mapped and the sample drawn as described above. The sample size of children under five that would give 95 percent power to detect a difference between 10 and 15 percent in malaria prevalence would be 690 in each survey. Respondents for the household survey were the heads of the household, from whom information relating to the household were obtained. Caregiver of children age under five years were the main respondents for information around knowledge of malaria prevention and health seeking behaviour for febrile children and SMC. Both household and caregiver questionnaires were based on the respective Malaria Indicator Survey (MIS) modules to ensure a standard approach[6]. All caregivers who were either permanent residents of the households sampled or visitors in the households on the night before the survey were eligible to be interviewed in the survey. All children 3-59 months who were listed in the household were eligible for the malariometric and anthropometric component of the survey." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 4. • "Table 2.1 below shows the response rates for both surveys. Both surveys had very good response rates with baseline response rate of 94 percent and 98 percent at endline. Interviews were conducted at the first visit to the household in more than 90 percent of the households at all survey rounds. Analysis was not weighted due to the invariability of information on cluster sizes." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 7. • "The survey was conducted in the 2nd phase LGAs which had received one round of SMC in from July to October 2014. Information on SMC receipt at each cycle was predominantly obtained from SMC cards given to the caregivers and from verbal responses of sampled caregivers in cases where the card could not be obtained." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 30. • "Table 3.1 below shows the geographic distribution of the households visited. A similar sample size was planned at both baseline and endline in the two LGAs. Results show that this was achieved with an equal sharing between Dutsi and Maiadua at the two survey rounds. Using Principle component analysis, households were classified into five relatively equal wealth categories based on their household possessions. Effort was made to use the same household characteristics at both survey rounds to enable comparability between the survey rounds." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 8. • 70. • See Table 1, Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 7. • "Overall reported coverage of SMC (receipt of at least one dose of SMC) was very high at 83.9 percent after one round of distribution. The coverage was deemed equitable by social economic status and gender as coverage was similar across each category. Dutsi LGA had a significantly higher coverage (94.8 percent) compared to Mai’adua. Coverage per cycle slightly declined at subsequent cycles from 75 percent in cycle one through to 55 percent at cycle four. This is in cases where information could be obtained. SMC distribution at each cycle was equitable. 61.8 percent of all children for whom data was collected received at least 3 cycles of SMC." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 30. • 71. • 72. As part of Malaria Consortium emails (unpublished), November 23, 2016, we received a presentation that included some high-level results from ACCESS-SMC but have not yet had the time to carefully review it. That presentation is available here: ACCESS-SMC Presentation, "Progress in scale-up of SMC in the Sahel," November 2016. • 73. • The methodology of this survey is largely described above: it is the same as the household coverage survey in northern Nigeria. • "The baseline and endline surveys were cross-sectional representative household surveys with a malariometric and anthropometric component for children under five years. A baseline survey was conducted to establish estimates before the intervention, on which performance targets and implementation scale can be based. The baseline survey was conducted in the two LGAs (Dutsi and Mai’Adua) at the peak of the transmission season before SMC was rolled out. The endline survey was timed at one month following the end of the rainy season to avoid the immediate prophylactic effect of the last round of drug ingestion, while remaining close to the usual period of highest prevalence. Figure 3 illustrates the timing of the surveys." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 4. • "The surveys included a malariometric component in which all children age 3-59 months listed on the household listing form during the household interviews invited for the malariometric and anthropometric assessments. Caregivers were taken through the consenting procedures and a short interview concerning history of fever in the last three days. Each participant received a record form and unique ID number. All health workers recruited to conduct the survey received standardized training to perform finger pricks for anaemia and malaria parasitaemia. In addition, standard operating procedures (SOPs) were developed along the MIS guidelines for their guidance while in the field." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 6. • "A malariometric component was conducted as part of the survey at both baseline and endline. During this process, children were asked if they felt fever at the time of the visit. none the less, fever was measured for all children and a finger prick done to take a sample. Those found to be positive were treated for malaria using artemisinin based combination therapies. At baseline, 9.7 percent of the children reported to be having fever at the time of the visit, this significantly reduced to 3.8 percent at endline. The same trend is observed for the measured fever i.e. having a temperature above 37.50c. 21.9 percent of children with temperature taken at baseline were found to have fever, a proportion that was only 6.8 percent at endline. Results from the mRDT tests conducted showed an mRDT positivity rate of 76.9 percent at baseline, declining to 47.8 percent at endline. mRDT positivity was higher in Dutsi LGA, and reduced among children from a higher social economic status." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 34. • 74. • 75. • "A baseline survey was conducted to establish estimates before the intervention, on which performance targets and implementation scale can be based. The baseline survey was conducted in the two LGAs (Dutsi and Mai’Adua) at the peak of the transmission season before SMC was rolled out. The endline survey was timed at one month following the end of the rainy season to avoid the immediate prophylactic effect of the last round of drug ingestion, while remaining close to the usual period of highest prevalence. Figure 3 illustrates the timing of the surveys." Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 4. • It appears that the endline survey was done roughly three weeks later in the year than the baseline survey. See Figure 3, Malaria Consortium, Nigeria SMC evaluation report, 2014, Pg 4. • 76. • 77. • 78. • "Sentinel Health facilities...These are the selected health facilities where HMIS strengthening activities will be conducted and where monitoring of malaria incidence, adverse events and RDT/Slide positivity rates will conducted." Malaria Consortium, M&E Plan Nigeria, Pg 27. • "Health facility routine HMIS data: Routine HMIS data will be a major data source for the project. Katsina state maintains a functional HMIS system through which all the health facilities in the state report. At state level, an electronic data (DHISs) is kept, where all monthly data is entered and quick summaries run. At health facility level, a compendium of HMIS forms is completed routinely. The existing system collects a wealth of information within the public health facilities on malaria. This includes all malaria cases suspected, diagnosed and treated, severe cases and malaria related deaths. Measurement of malaria incidence will be monitored from both the outpatient and inpatient malaria cases at the health facilities. Since confirmation of presenting fever cases as malaria is not consistent across the facilities, incidence will be measured through both suspected cases (reported as malaria or fever) and confirmed cases (by RDT or microscopy) among children under five reported at health facilities, over the two delivery phases In facilities where confirmation of malaria cases is done, RDT positivity rates data will also be monitored to assess trends pre and post SMC delivery. The project will work closely with the LGA and facility level HMIS focal persons in liaison with the State HMIS focal person to support and strengthen so as to enable effective monitoring of clinical malaria cases and the drugs dispensed during the study period. In doing so, the project will be working within and strengthening the existing structures as opposed to setting up parallel systems." Malaria Consortium, M&E Plan Nigeria, Pg 16. • "HMIS data was collected from three selected sentinel sites in each of the LGAs in which SMC was to be implemented namely Baure, Mashi, Maiadua and Dutsi. Three sentinel sites were also selected from a comparison LGA which was not planned to receive SMC. HMIS data on total OPD attendance and malaria cases in children less than 5 years was collected for the three year period from 2012 through to 2014. Results from figure 2 below indicate that there was a substantial decline in the number of malaria cases in LGAs that received SMC in 2013 and 2014 rounds. These trends were not observed in LGAs that did not receive any SMC, in fact an increase in malaria cases was observed in 2014," Malaria Consortium, Monitoring and evaluation summary Nigeria, Pg 2. • ""The indicator definition table shows the conceptual understanding of the indicators identified by the project to illustrate results and measure performance. This defines indicators in terms of numerator and denominator and data sources. Malaria Incidence: Number of confirmed and suspected cases of Malaria in children under five years per 1000 <5 population of children under five years. Measured over the transmission season. [Data Source: Ward HMIS & sentinel HF records], Disaggregation: Tests done, Frequency of collection/reporting: Weekly and Monthly, Target: 40% reduction." See Table 4.2, Malaria Consortium, M&E Plan Nigeria, Pg 19. • 79. "Since confirmation of presenting fever cases as malaria is not consistent across the facilities, incidence will be measured through both suspected cases (reported as malaria or fever) and confirmed cases (by RDT or microscopy) among children under five reported at health facilities, over the two delivery phases," Malaria Consortium, M&E Plan Nigeria, Pg 16. • 80. • 81. • 82. • 83. "It is very rare that children become sick after taking SMC medicines, but some children may feel a bit sick for a short while; what are some symptoms children may have? a) diarrhoea, b) itching, c) headache, d) mild abdominal pain, e) rash, f) a,b,andc, g) d,e,andf, h) All of the above, [Correct answer: H.]" Malaria Consortium Quiz Answer Key. • 84. • 85. • 86. "Efficacy of SMC treatments will be measured using the case control approach. Malaria cases, and controls who do not have malaria, will be recruited concurrently, and the dates of the doses of SMC they received noted. The efficacy of SMC can then be calculated as a function of the time since treatment using case-control analysis. It is essential that dates of SMC doses are accurately documented, and that malaria cases are parasitologically confirmed. Controls will be selected from the community, in the neighbourhood where the case lived at the time they had malaria. Trained fieldworkers will collect information about cases and controls, and make home visits to record bednet use and other household factors that may act as confounders. Microscopy will be used to confirm cases and to measure parasite density. Controls will be confirmed to be negative for P.falciparum, by RDT." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 10. • 87. • "Settings: Health facilities (outpatient clinics and hospitals) located in the areas targetted to receive SMC through the ACCESS-SMC programme starting in 2015 and 2016, and adjacent areas not targeted for SMC, will be selected as sentinel surveillance sites, in order to monitor changes in the incidence of malaria as SMC is introduced." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 14. • "Assessment of impact: Impact will be estimated from the difference in incidence with and without SMC and with SMC, using comparisons between SMC and control areas, and data on incidence before and after SMC implementation. Data on the incidence in older age groups will be used to help control for effects of year to year variation." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 15. • 88. "A survey will be conducted before the start of SMC, and again after two years of SMC, either in June 2015 and June 2017, or if timing is not possible for the first survey, in December 2015 and December 2017, to measure the prevalence of molecular markers associated with resistance to SMC drugs. The key markers to be monitored are the dhfr/dhps quintuple mutation, pfcrt76 and pfmdr1-86." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pg 11. • 89. "Context: The incidence of severe adverse events to SMC drugs has been very low but when SMC is implemented on a very large scale severer adverse drug reactions will occur and they should be properly managed and documented. A better understanding of associated predisposing factors may allow such events to be avoided in future. When severe events occur, communities need to be reassured that safety is being properly monitored and children with side effects are adequately cared for. National pharmacovigilance systems are weak and SMC is an opportunity to strengthen them." Malaria Consortium, Annex III: evaluation of seasonal malaria chemoprevention, Pgs 15-16. • 90. • 91. "In 2015 the ACCESS-SMC project delivered SMC to 3.12 million children (3 to 59 months) during the peak four months of malaria transmission in 7 countries in the Sahel for an average recurrent cost of between approximately USD 3.45 and USD 6.07 per child." ACCESS-SMC DRAFT multi-country cost analysis, August 2016, Pg 33. • 92. • 93. See our most recent model, "SMC" and "Results" sheets. • 94. • It appears that at the end of its fiscal year ending in March 2016, Malaria Consortium had about 8 million GBP in "Cash at bank and in hand." See Malaria Consortium, Trustees' report and financial statements 2016, Pg 18. We have not yet discussed with Malaria Consortium how it uses its cash on hand, but we would expect that it holds a substantial amount of funding as reserves. To get an understanding of how much unrestricted funding Malaria Consortium would be likely to spend on its SMC programs, we primarily analyzed how it spent unrestricted funds in the past. • In 2017, Malaria Consortium's projection suggests that it will spend a similar amount of unrestricted funding from March 2016 to March 2017 as it did in previous years (5,963 GBP in the year ending March 2017 compared to 5,886 GBP and 6,183 GBP for the years ending March 2016 and March 2015 respectively). This would represent about 10% of its overall spending in the 2017 fiscal year (5963/58,718 = 0.102). See Malaria Consortium, restricted and unrestricted expenditure analysis 2016. • Malaria Consortium told us that it often uses unrestricted funding to support overhead that is needed for some of the restricted grants it receives (sometimes funders' requirements do not allow it to spend on overhead), so these unrestricted funds are not available to support new programs and activities. GiveWell's non-verbatim summary of conversations with Malaria Consortium staff, November 7 and November 9, 2016. • For a more detailed breakdown of how Malaria Consortium has spent unrestricted funds in the past, see Malaria Consortium, restricted and unrestricted expenditure analysis 2016. It seems possible that some of the funds spent on "Monitoring & surveillance of malaria" and "Malaria strategy support" supported aspects of SMC programs, but we are uncertain. It does not seem that a large amount of unrestricted funds were directly allocated to SMC programs in the past. • 95. • 96. In March 2015-March 2016, it appears that Malaria Consortium's total expenditures were about 41,189,000 GBP, or about$57.7 million USD. See Malaria Consortium, Trustees' report and financial statements 2016, Pg 16. We used Google to convert from British pounds to US dollars. The value of GBP relative to USD varied substantially over this period; as a rough approximation we assumed that 1 British pound = $1.40. 41,189,000*1.4 = 57,664,600 dollars. • 97. • 98. • 99. • 100. • 101. • 102. • 103. • 104. • 105. • 106. • "An Office of the Inspector General (OIG) investigation found evidence of systematic embezzlement, fraudulent practices and collusion between 2010 and 2014 by a sub-recipient of Global Fund grants. The investigators concluded that a total of US$3,816,766 in expenditure by the Nigerian Government’s Department of Health Planning, Research and Statistics (DPRS) was non-compliant and is proposed for recovery. The expenditure was mainly related to training for a web-based health information system." Global Fund, Investigation in Nigeria, May 2016.
• "After learning the audit report’s specifics on weaknesses at two federal agencies in Nigeria, the Global Fund suspended all disbursements to both of them: the National Agency for Control of AIDS (NACA) and the National Malaria Elimination Program (NMEP). The Global Fund also executed an Additional Safeguards Policy." Global Fund, Executive Director statement on Nigeria, May 2016.
• Malaria Consortium told us that it expected there would be large funding gaps for malaria-related work in Nigeria in the next few years. GiveWell's non-verbatim summary of conversations with Malaria Consortium staff, November 7 and November 9, 2016.
• 107.
• 108.
• 109.
• 110.
• 111.
• 112.
• 113.

It appears that global demand for SMC treatments may decline in 2017 and 2018 relevant to 2016. See ACCESS-SMC, SMC donor mapping 2016 (with GiveWell calculations). However, we are unsure whether we should expect this donor mapping document to be comprehensive, and it may be that unexpected donors will support SMC in the future.

• 114.
• 115.
• 116.
• 117.
• 118.
• 119.
• 120.

"Some 23.7m children are eligible for SMC, the vast majority of who live in the Sahel region." ACCESS-SMC website, "The Project". Malaria Consortium told us that this estimate came from the WHO. Malaria Consortium emails (unpublished), November 23, 2016.

• 121.
• 122.

Slides 11-12, ACCESS-SMC, presentation on cost and impact, June 2016.

• 123.
• 124.

See Cells J17, K17, and L17 in ACCESS-SMC, SMC donor mapping 2016 (with GiveWell calculations).

• 125.
• 126.
• 127.
• 128.
• 129.
• 130.

"Why is SMC not more widely available?
While 23.7 million children are eligible for SMC, currently only 3.4% of them benefit from this intervention. Health ministries, donors and communities are have shown great interest in implementing SMC and scaling it up, but five critical barriers persist:

• Countries do not yet have the systems in place to distribute SMC drugs and administer them to children, including trained staff, logistics, tools, plans and communications materials.
• Global production of quality SMC drugs falls far short of demand, and current formulations of SMC drugs are complicated for community health workers to administer.
• The delivery of SMC drugs, which are themselves relatively inexpensive, is considered to be costly, while a lack of analysis and cost benchmarking means SMC’s cost effectiveness is poorly evidenced.
• Although evidence of safety and efficacy of SMC drugs has been demonstrated in trials, there is limited evidence at scale. This uncertainty prevents drug manufacturers and implementers from making large-scale investment in SMC.
• Health ministries, donors and the private sector currently dedicate insufficient financial resources to expanding SMC, meaning resources for scale-up are not available." ACCESS-SMC website, "The Project".
• 131.

[Under "Other issues for consideration":] "How long will the combination of AQ/SP last before drug resistance develops?" Slide 23, Malaria Consortium, SMC presentation, May 6, 2014.