Published: October 2015
[Added December 19, 2016: GiveWell's experimental work is now known as GiveWell Incubation Grants.]
Note: this page summarizes the rationale behind a grant to Evidence Action made by Good Ventures. Evidence Action staff reviewed this page prior to publication.
Summary
As part of our general effort to support the creation of future GiveWell top charities, in March of 2014 Good Ventures granted $250,000 to Evidence Action to support the development of additional programs. Evidence Action used these funds to support preliminary work scaling up and conducting additional research on its "No Lean Season" program in Bangladesh.
In March of 2015, Good Ventures granted Evidence Action an additional $170,792 for further work on this project.
This page reports on progress from March 2014 through February 2015.
In March 2016, Good Ventures granted Evidence Action an additional $812,352 for further work on this project.
We have had the following conversations about updates on this grant:
- Mushfiq Mobarak and Karen Levy on 10/22/2015
Table of Contents
The intervention
Evidence Action's "No Lean Season" program offers grants to low-income agricultural workers in Rangpur, Bangladesh as incentive to migrate during the famine season to urban areas where higher wages can be earned.1 We previously completed a shallow investigation on seasonal migration.
Our process
Towards the end of 2013, we approached Evidence Action to express our interest in supporting the creation of new GiveWell top charities, either by funding additional studies of programs that had shown promising results in a single randomized controlled trial or by funding the scale-up of promising programs.2 In March 2014, Good Ventures made a $250,000 grant to Evidence Action to support the investigation and possible scale-up of promising programs.3 This grant was used to support research into scaling up Evidence Action's "No Lean Season" program in Bangladesh. We published notes on conversations with Evidence Action held on April 18 2014, July 24 2014, and January 22 2015. In March of 2015, Good Ventures granted Evidence Action an additional $170,792 for further work on this program. We are in the process of following up on the post-analysis results of the study funded by these two grants. This page reports on progress from March 2014 through February 2015.
Grant details
Good Ventures granted $250,000 to Evidence Action in March 2014 and an additional $170,792 in March 2015.
Activities supported by the March 2014 grant
The cost of the "No Lean Season" pre-scale project from June 15, 2014 to January 31, 2015 was $433,279.4 Good Ventures and one other outside donor provided $260,000, and Evidence Action's core funds contributed $173,279.5
Evidence Action used these funds to:
- $299,030: (August 2014) Collect data in Bangladesh to explore scaling up;6 and (October 2014) support a 3,600 household study (sometimes informally rounded to "a 4,000 household study") in northern Bangladesh examining any potential unintended consequences of migration and the feasibility of scaling up.7 This study is a collaboration between Evidence Action, Innovations for Poverty Action (IPA), and RDRS, an NGO in Bangladesh which is implementing the intervention.8
- $81,126: (April 2014) Pay for staff time and fringe costs, including hiring a graduate student to examine the feasibility of a migration-type seasonal income support project in Malawi and Zambia.9
- $53,123: Fund travel to Bangladesh ($6,700) and overhead costs ($46,422.75) incurred as part of the above activities.10
Data collection for the 3,600 household study concluded in February/March 2015.11 We are in the process of following up on the post-analysis results of this study.
Projected activities: February-September 2015
Good Ventures’ March 2015 grant of $170,792 covers the "No Lean Season" pre-scale project's projected costs through September 2015.12 Evidence Action expects to use these funds to:
- $92,493: Fund Evidence Action staff members who will support the scale-up of the program.13
- $50,000: Explore the feasibility of starting new seasonal income support projects in Zimbabwe, Ghana, Bihar, and a fourth location (likely Indonesia).14
- $28,299: Fund travel to Bangladesh ($10,000) and overhead costs ($18,299) incurred as part of the above activities.15
Track record and cost-effectiveness
As we have previously written, a 2008 randomized control trial (RCT) in Rangpur, Bangladesh found that offering a modest incentive (the price of a bus ticket, either as a grant or a limited-liability loan) was associated with a substantive increase in migration rate, as well as in total household expenditures and other metrics of household earnings and consumption in the year of incentivization. This effect persisted in the year after incentivization, as some households incentivized to migrate in 2008 chose to remigrate in 2009 with no further incentive.16 Follow-up surveys have continued to track the long-term effects of the 2008 incentives on migration rates and household expenditures. We are in the process of examining these results. As of this writing, we are uncertain about the duration and total magnitude of the effects of offering a one-time incentive to migrate. Evidence Action's October 2014 3,600 household study did not collect household consumption data.17
We have completed an interim cost-effectiveness analysis (GiveWell’s interim CEA) which we expect to substantially update once we have examined results from follow-up surveys and new incentives offered in 2011-2014. At this interim stage we believe that the cost-effectiveness of incentivizing seasonal migration in northern Bangladesh may be competitive with that of our top charities. However, this estimate of cost-effectiveness depends on several factors about which we are highly uncertain. We are especially uncertain about the duration and magnitude of the benefits of a one-time incentive to migrate, whether treatment households also migrate and experience associated consumption benefits in the lesser lean season,18 and the costs of implementing the program. We are also uncertain about the burdens associated with seasonal labor migration, such as potentially harsh labor and living conditions for migrants, and the effect of temporary separation of family members.
Risks of the grant
Because this is a small grant to an organization that we've worked with before, we believe the risks of this grant are low. It is possible that Evidence Action's 3,600-household RCT (partially supported by this grant) may have adverse effects. The study is investigating possible adverse effects of incentivizing migration. It is possible that this study, by incentivizing larger migration rates, may have negative economic effects on the rural region of out-migration or on the urban migration destinations (such as suppressing wages). It is possible that the study may have negative effects on the well-being of participants.
Plans for follow-up
We are in the process of following up with Evidence Action and Professor Mobarak on the results of the 3,600-household RCT and on plans for scaling up the program. We plan to publish a summary of what we learn from the RCT funded by this grant. Evidence Action has told us that it would like to scale up the "No Lean Season" program in Bangladesh if the results of the current study are promising.19 Our future plans for continuing to follow Evidence Action's "No Lean Season" program and potential other seasonal migration projects depend on the result of this RCT and on Evidence Action's future plans.
In the long term, we hope that this project demonstrates the feasibility of, or develops into, a GiveWell top charity working on incentives for seasonal migration in regions where migration support is cost-effective.
Sources
- 1
"Seasonal hunger is a problem in many regions of the world for the very poor during the period between planting food crops and harvest time. For example, northern Bangladesh is at risk of widespread famine during three months leading to the winter harvest. Researchers working in this region identified a simple and effective solution that took advantage of the relative abundance of employment opportunities outside of the famine-prone north during this lean season. They provided households a travel subsidy for work-migration during the lean season, allowing them to send a member away to generate income that would otherwise not have been possible. This resulted in significant improvements in household welfare (including consumption and nutrition) during the lean season, an effect that held even in subsequent years. Based on this evidence, providing these travel subsidies is a promising way to avert seasonal famine risk." Evidence Action Beta Homepage.
- 2
In a February 2014 blog post, we wrote: "A major obstacle we face in our traditional work is the lack of charities that will likely be able to meet our criteria. In light of this, we are working on several projects this year that are relatively experimental and may eventually lead to new top charities… We intend to speak with organizations that conduct research (including Innovations for Poverty Action, among others) as well as Evidence Action (whose mission is to scale up evidence-based programs) to determine whether there are any interventions for which this is the case. If we identify strong contenders, we (in partnership with Good Ventures) will consider funding this research." 2014 plan for GiveWell's traditional ("top charities") work.
- 3
Good Ventures grant: Evidence Action - Planning for Future Programs.
- 4
"Total costs incurred for this project stand at $433,279 from 15 June, 2014 up to 31 January, 2015 including fringe and overhead costs." Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 1.
- 5
"The following contributions are available for the Bangladesh Seasonal Income Support Pre-Scale Project:
- GiveWell: $250,000
- [Name redacted]: $10,000
- Evidence Action Core Funds: $173,279"
Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 1.
- 6
"In August 2014, Evidence Action will collect data in Bangladesh to investigate the possible scale-up of a program to provide seasonal income support to people who wish to migrate from rural to urban areas. In particular, it aims to collect data that will inform how the program's effects may be different at larger scale than in the initial study." GiveWell’s non-verbatim summary of a conversation with Alix Zwane and Christina Riechers on July 24, 2014, p. 3.
- 7
- "Evidence Action is supporting a 4,000 household study in northern Bangladesh in order to explore further the potential of scaling up a migration subsidy program:
- In October, Evidence Action’s local implementing partner, RDRS, distributed a $12 travel loan to participating households ($48,000 distributed in total)."
GiveWell’s non-verbatim summary of a conversation with Evidence Action on January 22, 2015, p. 1.
- "This study is still tracking the amount of remittances migrants send home, but the focus is on the unintended consequences of migration (at both origin and destination)." GiveWell’s non-verbatim summary of a conversation with Evidence Action on January 22, 2015, p. 2.
- "Assuming that the current study indicates that Evidence Action can expand its seasonal income support program without sacrificing impact, it would like to scale up its program significantly in 2015-2016." GiveWell’s non-verbatim summary of a conversation with Evidence Action on January 22, 2015, p. 2.
- "Innovations for Poverty Action in Bangladesh (IPA-BD) has a total budget of $299,030 for the Bangladesh Seasonal Income Support Pre-Scale (BSISP) project. The appendix contains a breakdown of this cost." Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p.1.
- "Appendix: Details of IPA-BD Budget
- A.1. Direct Survey Costs: $90,200
Of this amount, $90,200 was budgeted for direct expenditures on surveying (High Frequency Origin Survey, High Frequency Migrant Survey and Endline Survey). At this stage we have cancelled the High Frequency Migrant Survey and will use those funds to bolster Endline survey work. Originally we had budgeted for a relatively "light weight" endline survey but we are adding some items to it that may require some extra money. Once the Endline Survey instrument is finalized we will know exactly how much of the High Frequency Migrant Survey budget has been saved. - A.2. IPA-BD Research Costs: $109,628
Next, $109,628 was budgeted for IPA-BD’s direct research expenditures including staff time, materials, travel and related expenses. - A.3. Travel Grant Costs: $69,100
RDRS is the local partner NGO that handed out the travel grants to 3,800 eligible households. This amount included the actual travel grant, which amounted to $13.5 (BDT 1,000) per grantee, and RDRS’ overheads. - A.4. Indirect Expenses: $30,102
IPA-BD has budgeted $30,102 for overhead and contingency."
Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 3.
- A.1. Direct Survey Costs: $90,200
- "Evidence Action is supporting a 4,000 household study in northern Bangladesh in order to explore further the potential of scaling up a migration subsidy program:
- 8
"We are investigating several critical questions to pressure-test the hypothesis that a well-timed travel grant may be a cost effective means of providing seasonal income support in Bangladesh and elsewhere. We are working in collaboration with Innovations for Poverty Action, the original research team, and our local partner, RDRS, in Rangpur, Bangladesh." Evidence Action Blog: Testing the Path to Scale.
- 9
- "We hired a graduate student ([name redacted]) to conduct a discovery exercise for us in Malawi and Zambia. The purpose of the exercise was to gauge the potential for a migration-type seasonal income support project in the two countries. The total cost for this exercise and the cost of staff time adjusted for fringe costs (Evidence Action has a 23% fringe benefits rate) was $81,126.25." Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 1.
- "Evidence Action is also collecting qualitative data on seasonal income support in Malawi. In Africa, the decision to supplement income by migrating is more complex, because even extremely poor people have some crops and land. Thus working outside agriculture in one season involves a tradeoff, because it reduces the ability of a subsistence farmer to invest in his farm and increase his crop yields. Kelsey Jack did a randomized controlled trial in Zambia in which farmers were given interest-free loans of maize, which allowed them to optimize between working outside agriculture and investing in their farms, while having guaranteed access to food during the non-growing season. Seasonal income support may be politically difficult to implement, while in-kind loans to farmers may be more politically appealing." GiveWell’s non-verbatim summary of a conversation with Alix Zwane and Christina Riechers on July 24, 2014, p. 3.
- The results of this qualitative feasibility study are available at Seasonal Migration Feasibility Analysis: Malawi and Zambia.
- 10
- "Two trips have been made to Bangladesh for this project at a cost of $6,700."
- "Evidence Action has a 12% overhead rate. The overhead on the subtotal ($386,856.25) is $46,422.75."
Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 1.
- 11
- "Over the next two months, there will be three more rounds of data collection (one in January and two in February).
- A final survey will be completed by the end of February.
- Some data will be available for analysis by the middle of February; final survey data will be ready by the middle of March.
- Complete results should be ready for regression analysis by April."
GiveWell’s non-verbatim summary of a conversation with Evidence Action on January 22, 2015, p. 2.
- 12
"Estimated costs for the project are $170,792 for the period 1 February, 2015 to 30 September, 2015." Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 1.
- 13
"Mushfiq Mobarak serves as Principal Investigator on the project; a Research Assistant has been budgeted for him. An Evidence Action postdoctoral fellow has been assigned to the project to assist in design, development and implementation. Another staff member will work part time to negotiate our partnership with RDRS and support fundraising. The total cost of these staff members adjusted for fringe costs (Evidence Action has a 23% fringe benefits rate) has been budgeted at $92,493." Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 1.
- 14
"We intend to conduct a similar seasonal income support feasibility exercise as was conducted for Malawi and Zambia for Zimbabwe, Ghana, the Indian state of Bihar and a fourth setting (likely Indonesia). A total cost of $50,000 has been estimated for this activity." Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 2. As of this writing, Evidence Action has told us that it expects to begin work on these feasibility studies in Q1 2016.
- 15
- "We anticipate three more trips to Bangladesh (split between [names redacted]) at an estimated cost of $10,000."
- "Evidence Action has a 12% overhead rate. The overhead on the subtotal direct costs ($152,493) is $18,299."
Evidence Action Bangladesh Seasonal Income Support Pre-Scale Project Expenditure Update, p. 2.
- 16
"We randomly assign an $8.50 incentive to households in Bangladesh to out-migrate during the lean season, and document a set of striking facts. The incentive induces 22% of households to send a seasonal migrant, consumption at the origin increases by 30% (550-700 calories per person per day) for the family members of induced migrants, and follow-up data show that treated households continue to re-migrate at a higher rate after the incentive is removed. The migration rate is 10 percentage points higher in treatment areas a year later, and three years later it is still 8 percentage points higher." Bryan, Chowdhury, and Mobarak 2011, p. 1.
- 17
"Evidence Action is not attempting to document again the positive consumption effect. Measuring this effect is expensive and Evidence Action is confident it will persist." GiveWell’s non-verbatim summary of a conversation with Evidence Action on January 22, 2015, p. 2. As of this writing, Evidence Action has told us that future evaluation will aim to re-evaluate the effect on consumption.
- 18
Bryan, Chowdhury, and Mobarak 2011 also consider migration "just prior to the second (minor) lean season that corresponds to the pre-harvest period for a second rice harvest known as Boro." (p. 33). We are still in the process of evaluating the results of this further study.
- 19
"Assuming that the current study indicates that Evidence Action can expand its seasonal income support program without sacrificing impact, it would like to scale up its program significantly in 2015-2016." GiveWell’s non-verbatim summary of a conversation with Evidence Action on January 22, 2015, p. 2.