IPA Independent Monitoring and Evaluation of MSI’s Family Planning Outreach Program (December 2025)

Organization Innovations for Poverty Action (IPA)
Location Sierra Leone and Kwara and Enugu states, Nigeria
Amount $1,155,000
Approval date December 2025
Funded by Unrestricted funds designated for grantmaking

Summary

This grant will fund Innovations for Poverty Action (IPA) to conduct independent monitoring and evaluation (M&E) of MSI Reproductive Choice’s family planning outreach program in Nigeria and Sierra Leone, which GiveWell has funded with a separate grant. We expect this independent data collection to reduce our uncertainty about how much of the contraception delivered by MSI would have been unavailable in the program's absence and how much of that contraception is used as a result of free and informed choice.

Published: September 2026

The challenge

Roughly 80 million women in low- and middle-income countries (LMICs) had unmet demand for contraception in 2024, and a further 136 million report wanting to avoid a pregnancy but not using a modern contraceptive method.1 MSI’s outreach program is designed to close part of this gap by sending mobile teams into remote and underserved communities to provide counseling and a full mix of contraceptive methods, including long-acting reversible contraceptives such as implants and intrauterine devices (IUDs).2 See our intervention report on family planning mobile outreach for more information.

GiveWell is currently funding MSI's outreach program in two states in Nigeria and five districts in Sierra Leone.3 Two questions are central to our decision about whether to continue or expand that funding:

  • How much of the contraception MSI provides is counterfactually additional—that is, used by women who would not otherwise have obtained modern contraception from another provider?
  • How much of that contraception is wanted—that is, the product of full, free, and informed choice?

We currently have limited evidence to answer these questions. For example, MSI conducts client exit interviews and "mystery client" visits but as far as we are aware does not systematically gather information from people who are not MSI clients.4 In addition, we believe that despite the best efforts of our grantee partners, there are many ways monitoring and evaluation processes conducted by implementing organizations can be biased.5 There are a small number of academic studies of contraceptive coercion in low- and middle-income countries, but none of them examine MSI's program.6 Without better, independent evidence, we face high uncertainty about both the program’s impact and its risks, which limits our ability to direct funding confidently.

How this grant could help

This grant funds IPA to collect and analyze data that GiveWell can use to refine the assumptions in our cost-effectiveness analysis and to monitor the risk of unwanted contraception. The grant will be used primarily for staff time (for research design and data analysis) and data collection.7 IPA's work, which will take place concurrently with the three-year grant to MSI, has five main components:

  • Household surveys with women of reproductive age in the communities where MSI works. IPA will survey roughly 1,500 women (942 in Nigeria and 547 in Sierra Leone) before the program begins and again in year 3, after MSI's grant is complete.8 The sample size was chosen to ensure sufficient statistical power to detect a 5-percentage-point change in modern contraceptive use between baseline and endline.9 The survey will measure contraceptive use, contraceptive autonomy, and community characteristics that could affect the program's impact, such as prior use of contraception, knowledge of how to access it, and fertility preferences.10 For about 5% of respondents in Nigeria and 10% of respondents in Sierra Leone, IPA will add open-ended questions to gather more detailed, qualitative information about women's perceptions of and access to contraception.11
  • Surveys with recent clients of MSI's outreach program. IPA will survey about 1,000 recent clients in year 1 and follow up with the same clients in year 3.12 These surveys will focus on clients' current and past contraceptive use, whether they knew of other ways to obtain contraception, whether they felt pressured toward a particular method, whether they were informed about side effects, their satisfaction with the method they received, and their perceptions about the impact of MSI’s outreach on their contraceptive use and autonomy.13
  • Unannounced visits to outreach sites. IPA will conduct around 80 unannounced visits during which IPA enumerators will observe the group counseling sessions that precede private consultations and interview providers.14 The enumerators will record information about service quality (which we expect to include things like what information is given about methods and side effects, and whether the providers encourage clients to use a particular method).15 IPA will also conduct rapid client surveys during these visits.16
  • "Mystery client" visits. IPA will conduct 72 visits during which trained enumerators will pose as clients and report on provider behavior.17 The enumerators will track which methods are offered, what information is given, and whether providers press clients toward contraception.18 Because the enumerators are not identified as evaluators, this approach seeks to avoid any changes in provider behavior that could occur when staff know they are being observed.
  • Data analysis using two complementary approaches. IPA will conduct a transportability analysis, which predicts how the results of a study conducted in one setting would apply to a different population by adjusting for differences in the characteristics that modify the treatment's effect.19 IPA will draw on an ongoing randomized controlled trial (RCT) of MSI's outreach program in the Democratic Republic of the Congo and two Nigerian states GiveWell is not funding and combine its results with the community-level data from the GiveWell-funded sites.20 Alongside this, IPA will use a more informal triangulation approach, checking the various data it collects against the key assumptions in the program's theory of change.21

Our reasoning

We are recommending this grant for several reasons:

  • First, we believe it will provide important information for our family planning grantmaking, which is in its early stages. MSI's outreach program is one of the more promising opportunities we have found, but we are unsure whether we would keep funding the program after the initial grants without additional information. MSI implements its program in more than 30 countries, so we think there is likely room to absorb substantially more cost-effective funding if the evidence supports it.22 We also expect the M&E to produce lessons that could inform other family planning opportunities and our broader approach to monitoring such programs.
  • Second, we think the M&E reduces the risk that the program generates unwanted contraception. By collecting client exit surveys, longitudinal follow-ups, mystery-client visits, and site observations, the grant provides us more insight into the potential risk. We have told MSI that we will weigh evidence about contraceptive autonomy in future funding decisions and do not want the program to maximize measured output at the expense of clients' choices. Because the data collection is spread across all three years rather than concentrated at the start or end, it strengthens this incentive and lets us flag early concerns in time for MSI to adjust.
  • Third, we think IPA is well-positioned to do this work. We selected IPA after soliciting proposals from four organizations. IPA has country offices in both Nigeria and Sierra Leone, it has prior experience conducting mystery-client visits at family planning clinics, and it has extensive experience collecting data for rigorous family planning studies and large randomized trials.23
  • Finally, we think the transportability analysis has learning value beyond this grant. We are unsure whether it will produce precise estimates, but we believe that testing it may tell us whether observational data from the sites we fund can be combined with data from earlier RCTs to estimate how a program's effects transfer across settings. If so, such an approach could inform how we adjust for external validity in other cost-effectiveness analyses.

Creating cost-effectiveness estimates for value of information grants is more challenging than for direct delivery programs; however, we estimate this grant to be 10 times as cost-effective as our benchmark, based on several subjective assumptions.24 In addition, it is based on the chance that what we learn leads us to direct more funding to MSI's Outreach program later; it does not include the program's other potential benefits, such as providing information to inform our broader family planning grantmaking or our approach to M&E.

What we could be wrong about

Our main uncertainties about this grant are:

  • The grant activities may reduce our uncertainty less than we hope. Our two central questions—how much of the contraception MSI provides is counterfactually additional, and how much is wanted—are inherently hard to answer without an intensive RCT. The design tries to work around this by leveraging an ongoing RCT and collecting several complementary types of data, but there is a risk that these point in different directions or raise more questions than they answer.
  • We are uncertain whether the transportability analysis will be feasible and informative. Although IPA treats it as the primary analytical approach, we think it could fail at several points. For example, the underlying RCT may be delayed or underpowered, or the community characteristics collected in the two settings may not overlap enough for the method to work. IPA has told us they will pivot toward other approaches if the transportability analysis proves infeasible, and will share the cleaned, de-identified data so that we can run our own analyses.25
  • This M&E grant is relatively expensive in comparison to the implementation grant; it is roughly 20% as large as our grant to the MSI program itself.26 We judged that the information value justified the expense and that our time was better spent moving forward than negotiating a lower cost, but this may not have been the right trade-off.
  • We have substantial uncertainty about several key parameters in our cost-effectiveness estimate. The value-of-information estimate depends on subjective judgments, such as the probability that we continue funding MSI with and without this M&E and the amount and duration of any future funding,27 and reasonable alternative assumptions would move the estimate substantially.

This page summarizes our research at the time of approval and was reviewed by the grant recipient.

Sources

Document Source
Canning and Karra 2023 Source
GiveWell, Family Planning Mobile Outreach, August 2026 Source
GiveWell, GiveWell's Cost-Effectiveness Analyses, September 2026 Source
GiveWell, GiveWell's "Excess Assets" Policy, 2024 Source
GiveWell, How Do You Value Information?, October 2024 Source
GiveWell, IPA M&E of MSI's Family Planning Outreach Program BOTEC, 2025 (public) Source
GiveWell, MSI Reproductive Choices - Family Planning Outreach Programs in Nigeria & Sierra Leone, 2025 Source
GiveWell, Podcast Episode 25: Following the Data on Dispensers for Safe Water Source
Guttmacher Institute, Adding It Up 2024: Investing in Sexual and Reproductive Health in Low- and Middle-Income Countries, 2025 Source (archive)
IPA, "Studies" Source (archive)
IPA, "Where We Work" Source (archive)
IPA, Budget: MSI Outreach Service Evaluation, December 2025 Source
IPA, Proposal to GiveWell: Estimating the impact of MSI’s Outreach Service on contraceptive usage and autonomy, 2026 Source
IPA, Research Protocol: MSI Outreach Service Evaluation (redacted), 2026 Source
MSI Reproductive Choices, "Where We Work." Source (archive)
MSI, Delivering Cost Effective Impact: An Investment in Reproductive Health in Nigeria, 2025 Source
MSI, Delivering Cost Effective Impact: An Investment in Reproductive Health in Sierra Leone, 2025 Source
Onyango et al. 2022 Source
Senderowicz 2019 Source
Senderowicz et al. 2023 Source
Track20, "Projected Trends in mCPR, Nigeria" Source (archive)
Track20, "Projected Trends in mCPR, Sierra Leone" Source (archive)