Note: This page summarizes the rationale behind a GiveWell grant to IDinsight. IDinsight staff reviewed this page prior to publication.
In a nutshell
In January 2025, GiveWell recommended a $299,083 grant to IDinsight for a project to review recent literature on revealed preferences between health, income, subjective well-being, and contraception, and to pilot new approaches for eliciting how people in low-income countries trade-off these outcomes. Most (~90%) of the grant will go towards funding these pilots. (more)
We are recommending this grant because:
- The values we assign to increased consumption vs. reduced mortality risk – what we call our "moral weights" – are a very important part of our cost-effectiveness models. Changing these weights would change our cost-effectiveness estimates for health-focused vs. livelihoods-focused programs (for instance).
- Our current weights are partially based on a survey we ran of potential beneficiaries in Kenya and Ghana in 2019, which asked people how they traded off consumption gains vs. reduced mortality risk. Since then, some new research has been published on methods to elicit these trade-offs. We’d also like to explore the potential for using surveys to elicit values over other outcomes, such as morbidity states and access to contraception. We’d like to pilot these approaches first before deciding whether we ought to fund another beneficiary preferences survey.
- We expect the literature on revealed preferences has changed since we last revisited our moral weights in 2020.
- IDinsight has relevant expertise and networks from conducting our previous beneficiary survey, and we believe they're well-placed to execute this literature review and pilot. (more)
Our main reservation about this grant is that survey experiments may not be the optimal method for eliciting these trade-offs, or we might not significantly improve upon our previous survey findings. (more)
Published: April 2025
The organization
IDinsight provides analytical support to governments, funders, and other clients with the goal of generating rigorous evidence to improve social impact.1 They conducted GiveWell's previous beneficiary preferences survey in 20192 and have significant experience in discrete choice experiments and preference elicitation.3 The work will be conducted under IDinsight's Dignity Initiative.
GiveWell has worked with IDinsight since 2014. Additional details on our relationship can be found on this page.
The grant
This grant will enable IDinsight to review research on and pilot methods for an updated beneficiary preference research survey.4
In 2019, GiveWell funded IDinsight to conduct a beneficiary preferences survey to understand how potential beneficiaries trade off between increased income versus reduced mortality risk.5
While informative, this survey had some limitations:
- About 38% of respondents chose averting child deaths regardless of the consumption benefits offered (up to $10 million).6 How we choose to incorporate these "never-switchers" is very impactful to our moral weight model.7 Taken at face-value, the moral weight we put on averting deaths would be inflated significantly. However, eliciting preferences over mortality risk requires an understanding of small probabilities, which can be especially challenging in settings with poor education. This survey could have also been influenced by social desirability bias – the tendency for respondents to answer survey questions in a way they believe others will see as virtuous or desirable – which have also impacted the results.8
- Additionally, the survey did not collect preferences on outcomes like morbidity reduction or access to contraception.9
Since the last survey, there has been more research published in the revealed preferences literature.10 There have also been papers published recommending new best practices for eliciting these trade-offs via survey experiments.11
The values we assign to these trade-offs – what we call our "moral weights" – are a very important part of our cost-effectiveness models. Changing these weights would change our cost-effectiveness estimates.
Given this, we want to fund pilots of new approaches to elicit these trade-offs via surveys, and revisit the literature on revealed preferences. In the pilots, IDinsight will test different discrete choice experiments and willingness-to-pay approaches to elicit stated preferences around consumption gains vs. mortality reductions, as well as additional outcomes like morbidity and access to contraception.12 While the exact design of these experiments will be finalized during the project, we broadly expect these experiments to take the following formats:
- Willingness to pay: asking people how much they would be willing to pay for mortality risk reductions / avoiding morbidity states / access to contraception
- Discrete choice experiments: asking people whether they would prefer cash transfers vs. improved health / access to contraception etc.
These tests will be run in small pilots of fewer than twenty participants, and the most promising strategies will be run in two full pilots of 100 participants each.13
The expected outputs of this grant are:14
- A summary of the revealed preferences literature, and a recommendation on whether we should use this approach for surveying beneficiaries in low and middle-income countries.
- A technical guide on how to collect preferences for any future data collection efforts, including recommendations on collection and analysis methods.
Budget for grant activities
The total grant amount is $299,083. A breakdown of the budget is as follows:15
- Personnel: $253,083
- Travel: $18,600
- Sub-grants: $26,000
- Other direct costs: $1,400
We expect approximately 10% of the grant amount ($30,000) to go towards IDinsight staff time for the literature review, with the remaining 90% funding the in-country pilots.
The case for the grant
We are recommending this grant because:
- We think we should revisit our moral weights. We last ran a beneficiary preferences survey in 2019, and last revisited our moral weights in 2020. Since then, new literature has been published on revealed preferences and methodologies for eliciting stated preferences. We’ve also tried to come up with moral weights for more outcomes (e.g. access to family planning), which we plan to publish about separately. It’s possible that revisiting this literature will update our current view.
- It seems important to try and ground these trade-offs in the preferences of program recipients. Ideally, we would like these moral weights driven by the people our programs affect – mostly poor people in low-income countries. This was the core motivation for the previous beneficiaries survey we ran in 2019. While this did meaningfully update our views, we think this survey had limitations which make us reluctant to put all our weight on it.16 We think it’s possible we’ll be able to improve upon our previous approach, and it feels important to test this hypothesis via piloting.
- We believe that piloting is a good idea prior to conducting a full survey. This grant will help us assess whether to conduct another full beneficiary preferences survey, and allow us to test improved methods before committing to a larger study.17 By piloting different methods for improving participant comprehension and question framing, we can identify the most promising approaches for a potential full-scale survey.
- IDinsight is a strong learning partner. IDinsight conducted our previous beneficiary preferences survey and maintains a network of academic experts in this space, and we believe they're well-situated to execute this literature review and pilot.18
Risks and reservations
Our main reservation about this grant is that results may be as difficult to interpret as our previous beneficiary preference survey,19 potentially leading us to conclude that further surveys cannot meaningfully address these challenges.
There are major difficulties with eliciting preferences over these outcomes via survey experiments – eliciting preferences over mortality risk requires an understanding of small probabilities (which can be especially challenging in settings with poor education), respondents may find it difficult to imagine their true preferences in hypothetical scenarios, and social desirability bias and cultural factors might cause people state preferences that are different from how they would actually behave if faced with real choices. It is possible that we conclude experimental approaches are not useful for eliciting these preferences after running these pilots. We are funding a smaller grant before committing to a larger survey due to these concerns.
Plans for follow up
We plan to have regular check-ins with IDinsight throughout the project, and may observe pilot implementation through a field visit. We may make the decision to fund a follow-up beneficiary preferences survey on the basis of these pilots if the pilots show promising results.
Internal forecasts
For this grant, we are recording the following forecasts:
| Confidence | Prediction | By time | Resolution |
|---|---|---|---|
| 5% | The literature review ends up swinging our current under-five death averted vs. doubling consumption moral weight ratio (116:1) by >50% | Jul 2025 | - |
| 70% | We make the decision to fund a follow-up beneficiary preferences survey on the basis of these pilots | Dec 2025 | - |
| 15% | Conditional on us funding it, this survey ends up swinging our current under-five death averted vs. doubling consumption moral weight ratio (116:1) by >50% | Dec 2026 | - |
Our process
Our investigation included:
- A review of IDinsight's proposal and budget, involving GiveWell staff who worked on the previous IDinsight survey
- Conversations with IDinsight on timeline and budget, and with IDinsight and Open Philanthropy on potential areas for survey improvement
Sources
- 1
"IDinsight is a global advisory, data analytics, and research organization that helps development leaders maximize their social impact. We tailor a wide range of data and evidence tools, including randomized evaluations and machine learning, to help decision-makers design effective programs and rigorously test what works to support communities. We work with governments, multilaterals, foundations, and innovative non-profit organizations in Asia and Africa. We work across a wide range of sectors, including agriculture, education, health, governance, sanitation, and financial inclusion." IDinsight, About us.
- 2
Read more about that grant here, and see our analysis of the results here.
- 3
See IDinsight's page on preference elicitation here.
- 4
"It is possible that new sources exist that capture the type of preference data required or that work is unpublished. We would conduct the ground work (dataset review, expert interviews) to identify and evaluate these opportunities. Second, the RFP highlights new questions of interest that could be answered with adapted versions of our previous choice experiments…We suggest an iterative design and piloting project." IDinsight, GiveWell Preferences Proposal (unpublished).
- 5
Read more about that grant here, and see our analysis of the results here.
- 6
"In the choice experiments focused on the value of life versus cash transfers, about 38% of respondents always chose the program that saved children's lives over any number of cash transfers offered (up to the presented maximum of $10 million)." GiveWell, Research on Moral Weights - 2019.
- 7
"These estimates are very imprecise and heavily affected by the proportion of the sample that always chose programs that saved children's lives." GiveWell, Research on Moral Weights - 2019.
- 8
"Respondents may tell surveyors what they think they should say, rather than their true preference. For example, perhaps people believed they would appear callous if they opted for receiving cash transfers instead of saving lives." GiveWell, Research on Moral Weights - 2019.
- 9
The 2019 beneficiary preferences survey focused on preference weightings between averting deaths among individuals of different ages and increasing income. See GiveWell, IDinsight — Beneficiary Preferences Survey (2019).
- 10
For example, this paper looks at governments’ response to the COVID-19 pandemic to infer how they traded off consumption outcomes (e.g. foregone GDP) vs. mortality risks.
- 11
For example, this paper recommends randomizing the ascendency vs. descendancy and ordering of choices in discrete choice experiments. This wasn’t done in our previous preferences survey; see IDinsight, Beneficiary Preferences 2019, "Method Overview" section, p. 60.
- 12
- “Improvements and themes under consideration for each stage of this work, that we will explore further, are discussed below. Revealed preferences…Discrete choice experience design” IDinsight, GiveWell Preferences Proposal (unpublished).
- “We propose to examine trade-offs between income and contraception, subjective wellbeing and morbidity.” IDinsight, GiveWell Preferences Proposal (unpublished).
- 13
- "Phase 2: Small pilots…Each pilot will involve fewer than 20 participants, with qualitative data collection to assess participant comprehension and explore the moral reasoning behind their responses, employing cognitive interview techniques." IDinsight, GiveWell Preferences Proposal (unpublished).
- "Phase 3: Full pilots…This phase will focus on testing the most promising elicitation strategies and evaluating their feasibility and relevance using internal and external consistency metrics developed in Phase 2. The full pilot will be conducted across 2 different contexts (likely Kenya and India), with sample sizes of 100 per context." IDinsight, GiveWell Preferences Proposal (unpublished).
- 14
“Over the course of this project we would produce the following deliverables: 1. Technical manual on recommended data collection and analysis methods for future revealed or stated preferences research by GiveWell. 2. Guidance on incorporating current and future preferences research into existing moral
Weights. 3. Supporting literature review and report on the data collected during the pilot phases. 4. If supported by the evidence, a detailed proposal for future research, or recommendations and opportunities for any future grantmaking in this area.” IDinsight, GiveWell Preferences Proposal (unpublished). - 15
IDinsight, GiveWell Preferences Proposal (unpublished).
- 16
For a discussion of the previous survey, and how it updated our views, refer to this page.
- 17
“We suggest an iterative design and piloting project to 1) assess feasibility of alternative DCE designs in LMIC contexts, and 2) collect enough data to establish likelihood that GiveWellʼs model could meaningfully change if the research were scaled.” IDinsight, GiveWell Preferences Proposal (unpublished).
- 18
"Through our work on this topic so far, we have built up a strong network of additional experts whom we would propose to consult, beyond the core research team." IDinsight, GiveWell Preferences Proposal (unpublished).
- 19
For more on the previous survey on beneficiary preferences, see GiveWell, Research on Moral Weights - 2019.