You are here

Evidence Action's No Lean Season

Donate

Evidence Action's No Lean Season is one of our top-rated charities and we feel that it offers donors an outstanding opportunity to accomplish good with their donations.

More information: What is our evaluation process?


Published: November 2017; Last Updated: June 2018

Summary

What do they do? Evidence Action's No Lean Season (evidenceaction.org/beta-no-lean-season/) provides no-interest loans to poor rural households during the time of seasonal income and food insecurity ("lean season") between planting and the major rice harvest in rural northern Bangladesh. Loans are conditional on an adult household member stating their intent to temporarily migrate to urban or other rural locations to seek short-term employment. The program is implemented by RDRS Bangladesh (http://www.rdrsbangla.net/). Evidence Action (https://www.evidenceaction.org) provides strategic direction for No Lean Season, mobilizes resources, provides technical assistance, conducts program monitoring independent of RDRS, and coordinates partnerships on the program. (More)

Does it work? We believe there is strong evidence for the positive impact of seasonal migration subsidies on household income and consumption. The program has been evaluated in several randomized controlled trials (RCTs) in Bangladesh, and an additional RCT is ongoing. We have remaining questions about whether the monitoring processes that Evidence Action is developing will be able to detect ways in which program implementation differs from the program as implemented in the trials. (More)

What do you get for your dollar? We estimate that the program will cost No Lean Season, on net, about $18 per participating household in 2018-2020. This estimate relies on a number of uncertain assumptions. Our cost-effectiveness analysis estimates the impact per dollar of the program on household income. (More)

Is there room for more funding? We believe that No Lean Season is very likely to be constrained by funding. Over the next three years, we expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive in that time to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and to do further research in Bangladesh and other locations. Funding beyond this level would allow Evidence Action to build reserves for No Lean Season. (More) December 2017 update: In November 2017, we recommended that Good Ventures give $11.5 million to Evidence Action's No Lean Season program, which we expect to fill its room for more funding for its work in Bangladesh over the next three years.

No Lean Season is recommended because of its:

  • Focus on a program with a strong evidence base and strong cost-effectiveness.
  • Standout transparency – it has shared significant, detailed information with us.
  • Room for more funding – we believe No Lean Season will be able to use additional funds to scale and implement its program in northern Bangladesh and to explore whether its model would be effective in other contexts.

Major open questions include:

  • We have somewhat limited information on how the program will be monitored to answer questions such as whether loans are targeted at low-income households, whether funds reach the intended households in full, whether recipients are pressured into participation, and whether participants find work when they migrate. We expect to see additional data from RDRS and Evidence Action in the future, but note that we put less weight on monitoring plans than on a demonstrated track record of producing high-quality monitoring. Our confidence in Evidence Action's plans is increased by its track record of producing high-quality monitoring for the Deworm the World Initiative, another GiveWell top charity.
  • The program as implemented in 2017 and going forward differs in some ways from the program as evaluated in previous RCTs (more). For example, some of the RCT evidence is from programs in which subsidies were given as grants rather than loans and only a portion of eligible households were invited to participate. We expect the 2017 RCT to provide additional evidence on the impact of the program in its current form.

Our review process

To date, our review process has consisted of:

  • Making several grants to No Lean Season as part of our work to support the development of potential future GiveWell top charities (now known as GiveWell Incubation Grants). For more information, see our reports on a March 2014 grant to No Lean Season and several subsequent grants.
  • Conversations with Evidence Action staff who work on No Lean Season and with Mushfiq Mobarak, a lead researcher on the studies that have informed our view of the No Lean Season program.1
  • Reviewing documents No Lean Season sent in response to our queries.
  • In September 2017, GiveWell visited No Lean Season in Bangladesh.

All content on No Lean Season, including past grants, blog posts, and conversation notes, is available here. We have also published a page with additional, detailed information on No Lean Season to supplement some of the sections below.

What do they do?

No Lean Season provides no-interest loans to poor rural households between planting and the major rice harvest in rural northern Bangladesh. This annual period, from approximately September to November, is often the time of greatest income insecurity and food insecurity for poor households (the "lean season").2 The No Lean Season loan is contingent on an adult member of the household engaging in temporary labor migration and is intended to increase employment opportunity, income, and household consumption. The loan amount of 1500 Bangladeshi taka (approximately $18.70 USD) is intended to be sufficient for round-trip transportation and to set aside some money for family members staying behind.3

No Lean Season was developed and is managed by Evidence Action, which provides strategic direction and technical assistance, mobilizes resources, conducts independent monitoring, and coordinates partnerships.4 The program is implemented by RDRS Bangladesh.5 The program is fairly new. The first randomized controlled trial (RCT) of conditional subsidies for seasonal labor migration in northern Bangladesh was conducted in 2008. Additional RCTs were conducted in 2011, 2013, and 2014; there is an ongoing RCT beginning in 2017.6 The program operated on a small scale in 2016 (out of 15 RDRS branch offices)7 and grew in 2017 to 52 branch offices.8 The No Lean Season program has only operated in and been evaluated in Rangpur Division.9

See also our report on conditional subsidies for seasonal labor migration in northern Bangladesh.

This section describes the roles played by organizations involved in the program, details of how the program is implemented, and how funding for the program is spent. Note that when we refer to program activities in a particular year, this refers to the implementation of the program targeting the lean season in the fall of that year, with program activities extending into the following calendar year. For example, loan recovery for the No Lean Season program in 2017 extends into March 2018.

Partners and staffing

No Lean Season is implemented by RDRS Bangladesh and most staff who work on No Lean Season are employed by RDRS. Evidence Action developed No Lean Season and plays a range of ongoing roles in the program, described below. Innovations for Poverty Action conducts evaluation activities, of which Yale professor Mushfiq Mobarak is the lead Principal Investigator.

RDRS Bangladesh

No Lean Season is implemented by RDRS Bangladesh, a microfinance and development agency that works primarily in Rangpur Division in northwest Bangladesh.10 As of March 2016, RDRS operated in every upazila (subdistrict) of every district in Rangpur Division and operated 171 total local branch offices.11 RDRS is expanding both within and beyond Rangpur: ten new branch offices were opened in 2016, three of which were in Rangpur Division.12 No Lean Season plans to scale to additional existing RDRS branch offices in future years, and may also scale to additional RDRS branch offices as they open.13

RDRS Bangladesh provides the operational infrastructure for the No Lean Season program in Bangladesh, and hires, trains and manages field staff.14 RDRS also conducts process and performance monitoring of the No Lean Season program.15

Evidence Action

Evidence Action is developing No Lean Season as a part of its Beta department (more below). Evidence Action leads the strategic direction and program management of No Lean Season, provides technical assistance, mobilizes resources, and conducts program monitoring independent of RDRS.16 Evidence Action also coordinates research partnerships for ongoing experimental evaluation of the program and for further optimization of the program design.17 Two Evidence Action staff members are based in Bangladesh and work full time on No Lean Season.18 In 2017, Evidence Action hired a team of part-time monitors.19 (See below for more on Evidence Action's monitoring of No Lean Season.) Evidence Action is in the process of exploring whether the program could scale in other contexts (more).

Evidence Action was created out of Innovations for Poverty Action (IPA) in 2013 with a mission to scale cost-effective and evidence-based programs. Two IPA initiatives, the Deworm the World Initiative (one of GiveWell's top charities) and Dispensers for Safe Water (one of GiveWell's standout charities), were spun off from IPA to be managed by Evidence Action.20 It has since built a department for investigating, testing, and considering new programs for scaling up called Evidence Action Beta;21 one program in the Beta portfolio is No Lean Season. We focus this review on No Lean Season and discuss the room for more funding implications of No Lean Season being a program of a larger organization below.

More on Evidence Action as an organization below.

Staffing

RDRS employs the following types of employees for the No Lean Season program:

  • Migration Organizers are the No Lean Season staff who are primarily responsible for interacting with program participants at each stage of the program. Two Migration Organizers are assigned to each RDRS branch office that is delivering the No Lean Season program, and divide the targeted villages in that office's catchment area between them, such that each Migration Organizer is, on average, responsible for six villages per year.22 Migration Organizers deliver the program in several phases: conducting village surveys and household surveys, informing prospective migrants about seasonal migration in the local lean season, making loan offers to eligible households, assisting in loan disbursement, monitoring migration compliance of the loan recipients, and following up with households to recover loans.23 Migration Organizers participate in RDRS-led training prior to each phase of the program, from approximately September to December of each year.24

    Evidence Action Beta's monitoring plans include administering a phone-based test to a sample of trainees to monitor the effectiveness of staff training and of staff's knowledge of how to implement the program.25 We have not evaluated the details of training materials, the protocols for the phone-based evaluations, or results from phone-based evaluations to date; instead, we focus below on overall program effectiveness.

    RDRS employed 30 Migration Organizers in 2016 and 102 in 2017.26 21 (70%) of the 2016 Migration Organizers were retained in 2017.27 See footnote for more details on RDRS's hiring process.28

  • Monitoring Officers. In addition to Migration Organizers, RDRS hires, trains, and manages a smaller number of Monitoring Officers. Monitoring Officers monitor Migration Organizers' performance, including field observations of work, training replacement staff, answering questions and providing technical support.29
  • Management staff. RDRS field management, i.e. Branch Managers, Area Managers, and Program Managers, provide overall management support for the program. RDRS has a dedicated Project Manager for No Lean Season. RDRS senior management staff such as the Director of Microfinance and four Microfinance Coordinators also provide support for the program.30

Evidence Action employs the following types of employees for the No Lean Season program:

  • Global support staff. Several Evidence Action Beta staff members based outside of Bangladesh provide full and part-time strategic, management, technical, and monitoring support for No Lean Season.31
  • Monitoring staff. At the time of GiveWell's 2017 site visit to No Lean Season, Evidence Action had hired a team of nine part-time staff to monitor the quality of program implementation by RDRS. All of these staff are masters students in local Bangladeshi universities.32 We describe Evidence Action's monitoring plans for each phase of program implementation below.
  • Local management and support staff. In 2016, Evidence Action Beta employed one employee in Bangladesh working full-time on No Lean Season.33 In 2017, another full-time staff member was hired to lead independent monitoring efforts and otherwise support the project.

Mobile phone data management system

No Lean Season field staff are provided with a mobile phone and trained in use of a mobile program implementation and data collection application.34 The mobile application includes modules for each stage of the program (described below): targeting surveys (village survey and household survey), invitations to offer meetings, making loan offers to households, loan applications and disbursement, migration compliance monitoring, loan repayment, and the migration debrief survey.35 The mobile application includes scripts and checklists to remind Migration Organizers of key information and procedures to communicate to program participants, as well as answers to Frequently Asked Questions based on field experiences to date.36

Evidence Action has backend access to the mobile application's database, allowing it to observe program implementation and program statistics in close to real time.

When there are problems with the mobile data collection application, Migration Organizers use pre-developed paper forms as a backup.37


Implementation phases of the program

In brief, the program involves the following stages:

  • Targeting: Selection of villages and households within selected villages, based on proxies for level of poverty
  • Offer: Communications with eligible households to explain the conditions of the loans (no interest and conditional on stated intent to migrate by an adult member of the household)
  • Disbursement: Loan disbursement to the interested eligible households
  • Migration Compliance Monitoring: Visits to households to determine whether they have sent an adult migrant and whether the migrant found work at their destination
  • Recovery and debrief: Loan recovery from returning successful migrants and administration of a migration debrief survey

Details are on our page with additional information about No Lean Season.

Breakdown of No Lean Season's spending

We summarize No Lean Season's 2016 spending in GiveWell's summary of No Lean Season's 2016 spending.38

In short:

  • No Lean Season spent approximately $930,000 in 2016.
  • Of that spending, approximately 19% was loan capital, 73% was spent on program management and implementation, and 8% was spent on ongoing research into the program's effectiveness.
  • Staffing costs made up a little less than half of management and implementation spending.

We note that 2016 was an early-stage version of the No Lean Season program. We therefore expect this cost breakdown to change as the program scales. One significant difference we anticipate in future years is that repaid loan capital will be "rolled over" to support the program's implementation in the subsequent year.

We have seen budget projections of No Lean Season's spending for 2017 and future years.39

Does it work?

We discuss the evidence for the impact of conditional subsidies for seasonal labor migration in northern Bangladesh in a separate report. We believe that there is strong evidence that conditional subsidies for seasonal labor migration in the region in which No Lean Season operates increase household income and consumption during the lean season in which participants receive subsidies and for 1 to 3 years following.

Additionally, the 2017 cycle of the program is being evaluated with an RCT, for which preliminary results are expected in 2018.

The program has produced limited independent monitoring data to date. We have seen 2016 data from RDRS on what proportion of loan recipients migrated, what proportion of migrants found work, and what wages migrants received. Evidence Action is currently developing and piloting monitoring procedures and has shared fairly detailed plans with us. We have seen initial results of Evidence Action's monitoring of 2017 household targeting. Given the limited monitoring data to date, we have remaining questions about whether the monitoring processes that Evidence Action is developing will be able to detect ways in which the program could fail to have the expected impact or have a negative impact.

What is the evidence for the effect of conditional subsidies for seasonal labor migration on household income and consumption?

We discuss the evidence for the impact of conditional subsidies for seasonal labor migration in northern Bangladesh in a separate report. We believe that there is strong evidence that conditional subsidies for seasonal labor migration increase household income and consumption during the lean season in which participants receive subsidies. We also believe there is fairly strong evidence that the program causes higher rates of migration in the 1 to 3 years following program participation, without further subsidies, and that migration in subsequent years increases household consumption. This conclusion is based on three randomized controlled trials (RCTs) conducted in the region in Bangladesh in which No Lean Season operates, two of which found an effect on income or consumption.

How is the program different from the intervention studied?

As described in our report on conditional subsidies for seasonal labor migration, we rely on the following studies to inform our view of the effectiveness of the intervention: a 2008 RCT with follow-up in 2009, 2011, and 2013; an unpublished 2013 RCT about which we have relatively little information; a 2014 RCT with follow-up regarding remigration in 2015; and a forthcoming 2017 RCT. In this section, we discuss how similar the program is to the intervention in these studies, with an emphasis on the 2008 and 2014 RCTs, the published studies taking place in the lean season. We believe that the No Lean Season program is highly similar in context and operation to the intervention in the key studies that inform our evaluation.

  • Geographic context. All of the studies that we rely on took place in Rangpur Division, Bangladesh.40 Because RDRS' presence is predominantly in Rangpur Division, we expect that No Lean Season predominantly plans to scale up within Rangpur Division, though it may plan to scale up in other areas of Bangladesh in the future.41 Our understanding is that there is a seasonal decline in consumption for low-income households throughout Rangpur Division.42 We are uncertain about the heterogeneity of relevant factors within Rangpur Division, such as the degree of seasonality in consumption, or the baseline prevalence of seasonal migration. We have not examined the village-level heterogeneity of the RCT results.
  • Eligibility criteria. The eligibility criteria in the studies that inform our evaluation were somewhat different from the eligibility criteria used in the No Lean Season program; the eligibility criteria of the No Lean Season program are strictly more inclusive.43
  • Subsidy type and amount. No Lean Season offers a conditional no-interest loan of 1500 taka (approximately 18.70 USD).44 The 2014 study offered a conditional cash transfer of 1000 taka (approximately 13.00 USD).45 The 2008 study included a study arm in which households were offered a conditional cash transfer of 600 taka (approximately 8.50 USD) and a study arm in which households were offered the same amount as a conditional no-interest loan; in both arms, migrants could earn an additional 200 taka (approximately 3 USD) for checking in with study staff at their migration destination, for a total subsidy of 8.50 - 11.50 USD.46
  • Targeting intensity. In each village where No Lean Season works, the program offers a conditional no-interest loan to all households which meet the eligibility criteria. In the 2014 study, one "low-intensity" study arm offered conditional cash subsidies to approximately 14% of the eligible population, while another "high-intensity" arm offered conditional cash subsidies to approximately 70% of the eligible population.47 In the 2008 RCT, the intensity of treatment was similar to the low-intensity 2014 treatment (approximately 14% of the eligible population).48 The results from the 2014 RCT using this variation in treatment intensity suggest that the program's effect on migration rate per offer increases with treatment intensity, but that the program's effect on total income per household offered is not higher at higher treatment intensity.49
  • Additional conditionalities. In the 2008 experiment, some households were assigned a migration destination or a migration group.50 This is not the case in the No Lean Season program.
  • Implementing partner. No Lean Season is implemented by RDRS Bangladesh. RDRS was the sole implementing partner for the 2014 RCT.51 The 2008 RCT was implemented by a number of NGOs, including RDRS.52

Is No Lean Season effectively targeting people who meet its criteria?

Households eligible for the No Lean Season program in 2017 are those households which report land ownership of 50 decimals53 or less or report any incidence of skipped meals in the two weeks prior to the household targeting census conducted by Migration Organizers.54

Evidence Action's monitoring plans include employing independent monitors to survey a sample of households in a sample of villages targeted by the program in order to verify the completeness and accuracy of the Household Targeting Survey implemented by RDRS.55 On our 2017 site visit we spoke with these monitors, who had recently conducted this monitoring activity.56 We have seen preliminary results of this monitoring for 2017.57 This monitoring activity provides evidence as to how effectively No Lean Season targets people who meet its criteria:

  • Does the Household Targeting Survey reach every household? The Household Targeting Survey is intended to be a complete village census and to collect data relevant to No Lean Season's eligibility criteria from every household.58 Evidence Action's monitoring provides verification of the comprehensiveness of the Household Targeting Survey by randomly sampling households in a village independently from the list of households visited by RDRS Migration Organizers,59 thereby estimating the proportion of households in a village that have been missed by the Household Targeting Survey. According to preliminary verification survey results, out of 748 randomly selected households visited by monitoring staff, 747 (99.9%) had also been captured in the Household Targeting Survey. It is possible that particularly hard-to-reach households would be skipped by both the Household Targeting Survey and the verification survey.60
  • Are households' self-reports of land ownership and skipped meals accurate? Evidence Action's verification survey provides a check on the reliability of households' self-report of land ownership by comparing households' responses in the Household Targeting Survey to responses in the verification survey. According to preliminary results of the verification survey, 64% of households in the verification survey reported exactly or nearly exactly the same amount of owned land as they reported in the Household Targeting Survey.61 Because Evidence Action's verification survey takes place at a later date, and reported incidence of skipped meals may be prone to recall error, it is not feasible to verify the consistency of households' reported incidence of skipped meals with this verification survey. We also expect that it is infeasible to verify households' land ownership or incidence skipped meals in a way that does not rely on self-reports.
  • How reliable is the determination of program eligibility? Evidence Action's verification survey provides a check on the reliability of the determination of program eligibility. Based on preliminary results of the verification survey on self-reported land ownership, discrepancies in recorded self-reported land ownership affected the eligibility status of 8% of households reached in the verification survey: 5% were deemed ineligible according to the Household Targeting Survey but would have been eligible based on the verification survey ("false negatives") while 3% were deemed eligible according to the Household Targeting Survey but would have been ineligible based on the verification survey ("false positives").62

We note that Migration Organizers, who conduct the surveys that determine eligibility, do not decide which households are eligible.63 RDRS told us that Migration Organizers are not told the program's eligibility criteria.64 Migration Organizers collect household data in the household eligibility survey. Evidence Action staff then analyze the data and create a list of eligible households that is sent back to Migration Organizers.65

Do loans reach the people targeted?

In the majority of cases, loans are disbursed at RDRS branch offices, and three RDRS staff are present during all disbursements. For villages that are far from a branch office, RDRS organizes disbursement days in the village; we do not know how these are supervised.66

We think it is fairly likely that we would find out about cases in which intended loan recipients do not receive the funds or receive the incorrect amount. Evidence Action plans to conduct household loan disbursement verification surveys, which will ask participating households whether they received a maximum of one loan of the correct amount and verify that the recipient matches the photo and ID in RDRS's records.67 We have not yet seen this questionnaire or any results indicating whether beneficiaries report receiving their loans (as this piece of monitoring is underway as of November 2017).

We are less confident that we would learn about cases in which Migration Organizers or other RDRS staff collect more than the loan amount in loan repayments, for example, by failing to record in the system that the loan has been repaid and asking recipients to repay the loan twice, or telling migrants that they are expected to repay the loan with interest.

Do recipients understand the conditions of the loan? Are they pressured into accepting or repaying the loan?

Evidence Action plans to collect the following monitoring data on whether beneficiaries understand the terms of the loan and whether they are pressured into accepting or paying back the loan:

  1. Observing a sample of offer meetings and interviewing participants from eligible households to check whether they understand the key information about the loan and to monitor for coercive practices by Migration Organizers.68 We have not yet seen this questionnaire or any results. Considering that Migration Organizers may change their behavior when Evidence Action monitors are present at an offer meeting, Evidence Action's post-meeting survey samples households from observed offer meetings and from unobserved offer meetings.69
  2. Making random, unannounced visits to branch offices during the disbursement process to verify that the loan disbursement procedures are being followed and to interview recipients to determine whether they have an accurate understanding of the terms of loan repayment.70
  3. Surveying a sample of households to verify repayment rates and to monitor whether recipients felt pressured into paying back the loan.71

Our understanding is that Migration Organizers are on salary and do not receive financial incentives to achieve higher take-up or repayment rates.72 We have not seen information on whether Migration Organizers who achieve higher take-up or repayment rates are evaluated more favorably or are more likely to be retained.

A 2016 small focus group study commissioned by Evidence Action found that some beneficiaries reported feeling some pressure to repay the loan (including disliking the frequency of visits from Migration Organizers), but no one in the sample reported being coerced to accept the loan.73 Other respondents reported positive experiences with loan repayment.74 Some respondents had been briefed by RDRS staff prior to focus group discussions.75 We put limited weight on the results of this small study.

Do people who receive loans successfully migrate and find work?

RDRS conducts a migration compliance survey to determine whether people who received loans have migrated76 and surveys migrants after they return to measure the percentage of migrants who are able to find work at their destination.77 In 2016, 87% of loan recipients migrated and 99.2% of recipients who migrated were able to find work, according to data from RDRS.78 RDRS reported following up with over 99% of households for the 2016 lean season.79 Migrants reported an average daily wage of 385 taka. The 25th percentile wage was 300 taka and the 75th percentile was 450 taka.80 No Lean Season has shared the questionnaire for this survey and the underlying data;81 we are not aware of any audits that were done to check the accuracy of the 2016 data.

These numbers may be inflated due to reporting bias; since stated intent to migrate is a condition of the loan, recipient households have an incentive to falsely report that someone migrated. On the other hand, some migrants may feel more inclined to report unsuccessful migration attempts in order to seek loan forgiveness. RDRS has told us that as a check, Migration Organizers ask up to three neighbors whether the loan recipient migrated; we are uncertain about the details of this procedure.82 RDRS and Evidence Action staff told us that neighbors are very likely to know whether a loan recipient has migrated; it seems possible to us that, since loan conditions are likely well-known in the village, neighbors might falsely report that a loan recipient has migrated.

Evidence Action plans to also survey households on migration compliance as a check on the data it receives from RDRS. This monitoring plan is under development and involves surveying both eligible and ineligible households in order to better understand migration throughout the village.83 Evidence Action also plans to conduct this migration debrief survey in villages not reached by the intervention in a given year in order to assess the rate of re-migration in the absence of an additional subsidy.84

Are people who migrate exposed to greater physical and other risks during migration than at home?

We have seen limited information addressing this question. The migration debrief survey that RDRS Migration Organizers conduct after the migrant returns home includes questions about whether the migrant experienced an illness, injury, or family emergency that prevented them from migrating or working.85 We have seen data from the 2016 migration debrief survey.86

Evidence Action's monitoring plans include loan recovery verification activities which aim to verify repayment rates and to check that households were not unduly pressured into paying back the loan, as well as migration debrief verification, which includes questions about the migrant's migration experience.87 We have not seen the questionnaires for these surveys and we have not seen how Evidence Action specifically plans to ask about negative experiences during migration.

Are there additional negative or positive effects of the program?

We discuss possible negative and positive effects of the program in a separate report. In short, we speculate that there may be additional impacts of the program, but we have little or no evidence about these possible additional impacts.

What do you get for your dollar?

In this section, we discuss the cost per participating household for the No Lean Season program.

See our most recent cost-effectiveness model for more detail on how we evaluate the cost-effectiveness of No Lean Season, and for how the program's cost-effectiveness compares to that of our other top charities. We discuss major judgement calls or uncertainties in our model here.

Cost per participating household

We estimate that, on average in 2018-2020, the net cost per loan made by No Lean Season will be $18.44.88

There are a number of sources of uncertainty in this estimate:

  • When possible, we prefer to rely on past results over projected future results, as projections can be overly optimistic. For No Lean Season, we have used projected costs and numbers of households reached because our understanding is that the cost structure of the program in the past was considerably different from what is expected in the future.89
  • No Lean Season's projections assume that 90% of the loan capital from each cycle will be repaid and available for the following cycle.90 If the loan repayment rate falls in the future, net costs would increase.
  • No Lean Season's projections assume that 77% of households in selected villages will be eligible for the program91 and 60% of those eligible will choose to participate.92 Lower eligibility and participation rates could lead to higher costs per participating household as Migration Organizers could need to visit additional villages to enroll the same number of households or as fixed costs of the program could be spread across fewer households.
  • Some research costs may be incurred by other organizations. We have not yet determined the amount of such costs or how to account for them in order to be consistent across our top charities.

Is there room for more funding?

We believe that No Lean Season could productively use more funding than it expects to receive and that it is very likely to be constrained by funding.

In short:93

  • Total opportunities to spend funds productively: No Lean Season projects that it could use up to $11.9 million for implementing and monitoring the program in Bangladesh in 2018-2020, and an additional $4.5 million over that period to conduct additional research on the program and explore the possibility of expansion to other countries.
  • Cash on hand: Evidence Action expects to have $400,000 available for No Lean Season in Bangladesh for 2018. In addition, we project that RDRS will have repaid loan capital available for 2018, which has been subtracted from the projected costs of the program. Evidence Action expects to have $657,750 available in matching funds for research outside of Bangladesh in 2018.
  • Expected additional funding: Evidence Action told us that it has limited fundraising prospects for supporting No Lean Season in the next few years.
  • Additional considerations: Below, we also discuss implications of No Lean Season being a program within the larger Evidence Action organization.

In sum, we expect No Lean Season to have opportunities to spend $11.5 million more than we expect it to receive over the next three years to implement and monitor the program in Bangladesh. We expect it to have a further $3.9 million in opportunities to expand to other countries and do further research, in Bangladesh and other locations. Funding beyond this level would allow Evidence Action to build reserves for No Lean Season.

Details in this spreadsheet.

December 2017 update: In November 2017, we recommended that Good Ventures give $11.5 million to Evidence Action's No Lean Season program, which we expect to fill its room for more funding for its work in Bangladesh over the next three years.

Available and expected funds

To date, Evidence Action has received funding for No Lean Season from the Global Innovation Fund (GIF; one grant for $2.5 million), Y Combinator (one-off grant of $100,000), and GiveWell-directed funds. It expects to have $400,000 from the GIF grant remaining to support the program in Bangladesh in 2018. It has also received $657,750 from GIF for a pilot and RCT in Indonesia, with the expectation that it will raise matching funding.94

RDRS will also hold some funding at the end of the 2017 cycle from repaid loans that will be available for funding the program in 2018. The budget for the program in 2018-2020 projects that $1.5, $1.7, and $2.8 million will be available in 2018, 2019, and 2020, respectively, from the previous year's loan repayments.95

Evidence Action told us that it has only speculative or early-stage prospects for raising funding for No Lean Season from sources other than GiveWell-directed funding. No Lean Season has not yet been invited to apply for additional funding from GIF and, given the length of the application process, it is unlikely that additional funding from GIF will be available for the 2018 and 2019 cycles in Bangladesh. Evidence Action told us that there are a few other institutional funders that could potentially be interested in funding No Lean Season, but that it is not in active conversations with any funders currently. It is also interested in fundraising from individual donors, but does not yet have specific plans for this work.96

Uses of additional funding

No Lean Season estimates that it could use up to $11.9 million for its work in Bangladesh in 2018-2020, and an additional $4.5 million over that period to conduct additional research on the program and explore the possibility of expansion to other countries.97

We ask top charities to consider GiveWell-directed funds to be multi-year grants. The amount of GiveWell-directed funding that a top charity receives can vary greatly from year to year, and spending the funds over two to three years can help smooth these fluctuations.

Details of how No Lean Season expects to spend additional funding are in this spreadsheet, sheet "Spending opportunities." In short:

  • No Lean Season projects that it could scale its work in Bangladesh to roughly 92,000 loans disbursed in 2018 ($2.5 million total budget), 151,000 in 2019 ($3.7 million), and 235,000 in 2020 ($5.7 million), up from an expected maximum of 84,000 in 2017 ($2.7 million). Evidence Action notes that it does not expect to expand No Lean Season in Bangladesh significantly in 2018 because it would like to see the results of the 2017 cycle RCT and spend time setting up an office in the country.98
  • At $11.9 million for its work in Bangladesh in 2018-2020, No Lean Season would budget $470,000 for research on the program, including completing an RCT begun in 2017.99 No Lean Season notes that this budget would "include all monitoring and evaluation costs related to assessing the path-to-scale and ongoing delivery-at-scale of the program in Bangladesh" and exclude "costs that are related to exploration of, adaptation to, and expansion into other geographies" and "costs that are related to broader learning agendas about the theory behind the program."100
  • If No Lean Season received sufficient funding for its work in Bangladesh, it would use additional funding for work including "identifying and exploring new geographic contexts [including pilot programs and monitoring and evaluation], global policy engagement and communications, [and] broadening the donor base in anticipation of future growth."101 It estimates that it could use up to $4.5 million for this additional work in 2018-2020.
    December 2017 update: In November 2017, we recommended that Good Ventures give $11.5 million to Evidence Action's No Lean Season program, which we expect to fill its room for more funding for its work in Bangladesh over the next three years.

The total of the above is $16.4 million.

In addition, No Lean Season – and Evidence Action more broadly – would like to increase its reserves. Evidence Action has drawn down some of its reserves in the past year, and aims to hold more funding in reserve to protect against underestimation of costs and overestimation of revenue from other sources.102

We believe that there is some risk that No Lean Season will encounter limitations to its ability to scale, other than funding. For example, the ability of the program to scale successfully is dependent on RDRS's capacity and ability to hire high-quality staff. Our understanding is that this is relatively unlikely to be a limitation. We note that No Lean Season's plan is to scale in current RDRS branches only through 2021 (scaling projections for 2022 onward assume that RDRS would expand, opening additional branch offices).103

The budget also makes assumptions about the rate of eligible households (77% of all households)104 and take up in targeted villages (60% of eligible households).105 We note that in 2017, No Lean Season underestimated the number of households that would be eligible to receive loans, and possibly also underestimated take-up rate. GiveWell recommended funding and a funding guarantee to cover cost overruns to ensure that the program could meet its commitments without drawing down unrestricted funding.106 The budget also assumes that 90% of loan capital from each year will be available for further loans in the subsequent year, based on 98% loan recovery in 2016.107

No Lean Season as a program of Evidence Action

No Lean Season is a program of Evidence Action. Evidence Action supports other programs in addition to No Lean Season, including GiveWell top charity Deworm the World Initiative.108

This has some implications relevant to No Lean Season's room for more funding: donations to Evidence Action, even if restricted to No Lean Season, might change the actions that staff take to fundraise (i.e., which grants they pursue and what type of funding they ask for). We've seen some evidence that this has been the case in the past and provide details on our page with additional information about Deworm the World.

Evidence Action as an organization

We believe that Evidence Action is a strong organization:

  • Track record: Evidence Action's track record of funding and monitoring the No Lean Season program is fairly limited. We have separately discussed its track record of implementing the Deworm the World program.
  • Self-evaluation: Evidence Action has demonstrated a commitment to evaluating No Lean Season with RCTs—it is supporting an RCT in 2017 to build upon the evidence collected in multiple previous trials. No Lean Season has described fairly detailed ongoing monitoring plans, but we have so far seen limited monitoring from the program. We have separately discussed Evidence Action's commitment to monitoring in the context of its Deworm the World program.
  • Communication: No Lean Season staff (Evidence Action staff and RDRS staff) have generally communicated clearly and directly with us, given thoughtful answers to our critical questions, and shared significant, substantive information.
  • Transparency: Evidence Action has consistently been strong in its commitment to transparency. It has provided the information we've asked for and has not hesitated to share it publicly (unless it had what we felt was a good reason).

More on how we think about evaluating organizations at our 2012 blog post.

Sources

Document Source
Akram, Chowdhury and Mobarak 2017 Source (archive)
Bryan, Chowdhury and Mobarak 2014 Source (archive)
EA Fund Payout Report, September 2017: No Lean Season Grant Source (archive)
Evidence Action 2016-2017 revenue and expenditures Source
Evidence Action memo to GiveWell, 23 October 2017 Unpublished
Evidence Action No Lean Season - 2016 Program Round Actual Expense Breakdown (Unpublished) Unpublished
Evidence Action webpage: "What We Do" Source (archive)
Evidence Action webpage: The solution: No Lean Season Source (archive)
Evidence Action webpage: What is Evidence Action Beta? Source (archive)
GiveWell's 2012 site visit to Dispensers for Safe Water Source
GiveWell's 2017 site visit to No Lean Season Unpublished
GiveWell's summary of No Lean Season's 2016 spending Source
Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh Unpublished
Information System of the Laws of Bangladesh Source (archive)
Information System of the Laws of Bangladesh: Bangladeshi Stamp Act of 1899 Source (archive)
Information System of the Laws of Bangladesh: Chapter II section 3 of the Stamp Act of 1899 Source (archive)
Information System of the Laws of Bangladesh: Schedule I of the Stamp Act of 1899 Source (archive)
Innovations for Poverty Action webpage: Evidence Action launch press release Source (archive)
Khandker 2012 Source (archive)
Khandker and Mahmud 2012 Source (archive)
Mobarak, Reimão and Shenoy 2017 Source
No Lean Season 2013 note about hartals and low migration rates Source
No Lean Season 2016 Analysis Plan for Selecting Target Households Source
No Lean Season 2016 Household Targeting Questionnaire Source
No Lean Season 2016 Loan Application Form Source
No Lean Season 2016 Migration Debrief Survey Source
No Lean Season 2016 program costs Unpublished
No Lean Season 2016 Program Data Codebook Source
No Lean Season 2016 summary of data Source
No Lean Season 2016 summary of data (guide) Source
No Lean Season 2016 Survey Guidelines & FAQ for the Household Targeting Questionnaire Source
No Lean Season 2017 CommCare Module Translation Unpublished
No Lean Season 2017 Offer Meeting Script Source
No Lean Season 2017 Preliminary Monitoring Results: Household Targeting Source
No Lean Season 2017 Process and Performance Monitoring Framework Source
No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet Source
No Lean Season Budget Projections Source
No Lean Season CommCare Design Graphic Source
No Lean Season Household Survey Verification Questionnaire Source
No Lean Season Research Timeline Source
RDRS Bangladesh 2016 Annual Report Source (archive)
RDRS Bangladesh webpage: RDRS in Brief Source (archive)
RDRS Bangladesh webpage: Where We Work Source (archive)
  • 1.

    We have had additional conversations from which we have not published conversation notes.

  • 2.
    • "Although the change in the rice crop cycle has helped reduce the seasonal spread in rice prices, the traditional lean season preceding the aman harvest and spreading from September to November has changed little in its characteristics. It is the season of the least crop-related activity with no major crops planted or harvested, as can be seen from the crop calendar shown in figure 3.4. Although the wheat-growing season starts in November, the crop is grown in only about 5 percent of the cultivated land area. Moreover, the aftermaths of natural disasters like floods, drought, or excessive rains are usually felt most severely in this season.

      "The cropping patterns in Bangladesh are delicately balanced within the natural cycles of rains and annual floods. Thus, farmers' production options and perception of risk are often determined by the physical environment: the degree of seasonal flooding, the timing and quantity of rainfall, and the soil characteristics (Mahmud, Rahman, and Zohir 1994, 2000)." Khandker and Mahmud 2012, Pgs 40-41. See especially figure 3.4, "Crop Calendar of Major Crops," which illustrates that no other major crop activities occur while the aman rice crop is in its growing period from September to November.

    • For background on seasonal variation in consumption in rural northern Bangladesh, see Khandker 2012 and Khandker and Mahmud 2012.
    • In particular, Khandker and Mahmud 2012 discuss seasonal patterns of income, expenditures, and poverty on Pgs 43-61, based on the authors' analysis of data from the Household Income and Expenditure Survey of the Bangladesh Bureau of Statistics, showing marked dips in income and consumption in the lean season. See especially data on seasonal employment patterns on Pg 60, showing that household employment in the farm sector varies seasonally from a peak of approximately 17 person-days per month to a low of approximately 8 person-days per month in the lean season.
    • Bryan, Chowdhury and Mobarak 2014, Pg 1676 shows seasonal variation in total expenditures per capita, food expenditure per capita, price of rice, and quantity of rice per household based on the Household Income and Expenditure Survey of the Bangladesh Bureau of Statistics.
  • 3.
    • "No Lean Season addresses seasonal poverty by offering a migration subsidy of about $20 to participating households, covering the cost of a round-trip bus fare to nearby urban areas that do not experience the same seasonal fluctuations in job opportunities." Evidence Action webpage: The solution: No Lean Season.
    • From conversations with Evidence Action Beta staff, it is our understanding that the costs of transportation may vary and that the loan amount is intended to be large enough to additionally allow migrants to set aside some money for family members staying behind.
    • "Following the success of the loan provision for seasonal migrants, we have piloted the project NLS in 15 branches for 7000-9,000 seasonally-unemployed adults who can borrow Tk 1,500 each on an interest-free basis." RDRS Bangladesh 2016 Annual Report, Pg 49.
    • The loan amount of 1500 taka is built in to No Lean Season Budget Projections, unpublished tab.
    • We use the currency conversion rate 1 USD / 80.22 Bangladeshi taka, as used in No Lean Season Budget Projections, unpublished tab.
  • 4.

    GiveWell's 2017 site visit to No Lean Season

  • 5.

    GiveWell's 2017 site visit to No Lean Season

  • 6.

    No Lean Season Research Timeline illustrates the history of research studies related to the No Lean Season project prior to 2017.

  • 7.

    "In 2016, we decided to work with 15 of 164 Branch Offices (as agreed in the MOU with RDRS). This was an increase from previous years, but a number that felt appropriate in terms of overall budget and implementation capacity." No Lean Season 2016 summary of data (guide), Pg 2.

  • 8.
  • 9.
    • In 2016, No Lean Season operated out of 15 RDRS branch offices "geographically dispersed throughout the target program area in northern Bangladesh." No Lean Season 2016 summary of data (guide), Pg 2.
    • 2014 RCT:
      • "The landless poor supplying agricultural labor on others' farms are especially affected when demand for agricultural labor falls. They constitute around 56% of the population in our sample area, and are the target of the seasonal migration encouragement intervention that we design. Our sampling frame is representative of this landless population in the Rangpur region of Northern Bangladesh. According to the Bangladesh Bureau of Statistics, there are roughly 15.8 million such inhabitants in Rangpur (BBS, 2011)." Akram, Chowdhury and Mobarak 2017, Pg5.
      • "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur," Akram, Chowdhury and Mobarak 2017, Pg 7.
    • 2008 RCT (and its follow-up surveys): "Our experiments were conducted in 100 villages in two districts (Kurigram and Lalmonirhat) in the seasonal-famine prone Rangpur region of northwestern Bangladesh." Bryan, Chowdhury and Mobarak 2014, Pg 1675.
  • 10.
    • "Established in 1972 to assist with relief and rehabilitation of greater Rangpur-Dinajpur region immediately following the War of Independence of Bangladesh in 1971, the RDRS program evolved into a sectoral then comprehensive effort. Formerly the Bangladesh field program of the Geneva-based Lutheran World Federation/Department for World Service, RDRS became a national development organization in 1997... RDRS is now a major multisectoral development agency focused on disadvantaged northwest Bangladesh." RDRS Bangladesh webpage: RDRS in Brief
    • "RDRS is unusual is maintaining a concentrated geographic program, focusing on 8 districts and 57 Upazilas (sub-districts) mainly in deprived north-west Bangladesh in Rangpur Division, far from the overcentralized economic and political powerbase of Dhaka, Chittagong, even Rajshahi." RDRS Bangladesh webpage: Where We Work, which also includes a map of areas where RDRS works, showing a geographic emphasis on Rangpur Division.
    • No Lean Season operates under the microfinance arm of RDRS: "No lean season (NLS); Following the success of the loan provision for seasonal migrants, we have piloted the project NLS in 15 branches for 7000-9,000 seasonally-unemployed adults who can borrow Tk 1,500 each on an interest-free basis." RDRS Bangladesh 2016 Annual Report, Pg 49, under the heading Microfinance.
  • 11.
  • 12.

    "As our Sylhet projects are doing well, we have had to open 7 new branches in this tea-growing area and 3 more branches were opened in Dinajpur." RDRS Bangladesh 2016 Annual Report, Pg 49. Sylhet District, Sylhet Division is in northeastern Bangladesh. Dinajpur District is in Rangpur Division.

  • 13.

    No Lean Season Budget Projections, tab "Budget Projections" shows projected growth in "Total # of Branch Offices." The number of branch offices in which No Lean Season projects operating by 2022 exceeds 171, the total number of RDRS branch offices as of 2016 (see table row "Total Branches," RDRS Bangladesh 2016 Annual Report, Pg 53), and continues to grow at a rate of 10 per year.

  • 14.

    See GiveWell's summary of No Lean Season's 2016 spending for a summary of RDRS's 2016 budget, including line items for "Salary and allowances" and other operating costs.

  • 15.

    "Both process and performance monitoring involve the collection of two interrelated sets of data:

    • Administrative data is collected by the RDRS in the course of implementation. This includes training attendance, household listing, offer attendance, migration compliance checks, loan recovery etc. Some key forms are digitized and collected using electronic data collection.
    • Verification data is collected by Evidence Action from a sample of respondents or site observations in order to confirm that various program process have been performed effectively and in adherence to protocol. The sampling plan, which is still under development, details the sampling approach to be used for verifying each phase of the program and is summarized on the third tab here."

    No Lean Season 2017 Process and Performance Monitoring Framework, Pg 2.

  • 16.

    GiveWell's 2017 site visit to No Lean Season

  • 17.

    GiveWell's 2017 site visit to No Lean Season

  • 18.

    We met these two staff members during GiveWell's 2017 site visit to No Lean Season.

  • 19.

    GiveWell's 2017 site visit to No Lean Season

  • 20.

    "Two IPA initiatives that touch millions of people in Africa and Asia - Dispensers for Safe Water and the Deworm the World Initiative - will spin off from IPA to be managed by Evidence Action." Innovations for Poverty Action webpage: Evidence Action launch press release

  • 21.

    "We identify and pressure-test evidence-based innovations that reduce the burden of poverty, and then design and build the most cost-effective programs for massive scale." Evidence Action webpage: What is Evidence Action Beta?

  • 22.
  • 23.

    "RDRS' Migration Organizers deliver the program at the last mile. They conduct village and household surveys, inform prospective migrants about the benefits and risks of migration, extend offers to eligible households and participate in loan recovery." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 2.

  • 24.
  • 25.
    • "To monitor the effectiveness of the trainings, Evidence Action will conduct phone-based tests after each training session using a sample of Migration Organizers. These tests will enable us to measure the impact of training on Migration Organizers' knowledge, and to assess the knowledge retention of Migration [Organizers] following training." No Lean Season 2017 Process and Performance Monitoring Framework, Pgs 2-3.
    • No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tabs "FullFramework" and "Sampling plan" show the dates and targets for these knowledge verification surveys. In 2017, Evidence Action aims to sample 41 Migration Organizers (40%) per survey (tab "Sampling plan"), and considers the target for program success to be 90% accuracy in Migration Organizer understanding of key program elements (tab "FullFramework").
  • 26.
  • 27.

    "In 2016 there were 30 Migration Organizers, 21 of whom were invited to continue this year." GiveWell's 2017 site visit to No Lean Season, Pg 16.

  • 28.

    "When recruiting for Migration Organizers, RDRS initially selects more candidates than the number of positions it is trying to fill so that positions can be filled more quickly (vacancies are typically filled within 1-2 days). In 2017, RDRS hired 102 Migration Organizers for No Lean Season out of about 1,000 applicants and left roughly one candidate in the reserve pool for every candidate hired. The monitoring officer observes each new recruit one-on-one for 1-3 days. If a Migration Organizer is performing poorly, they will be observed and evaluated.

    "So far in 2017, RDRS has replaced seven of the 102 Migration Organizers it hired this year. Since Migration Organizers are skilled workers, the usual reason for replacing them is because they leave for a better job. In 2016 there were 30 Migration Organizers, 21 of whom were invited to continue this year."

    GiveWell's 2017 site visit to No Lean Season, Pg 16.

  • 29.

    GiveWell's 2017 site visit to No Lean Season

  • 30.

    Evidence Action provided us with this list of management staff. We have seen RDRS budgets for No Lean Season including salary & allowance line items for Senior Management, a Project Manager, Branch Managers, and Area Managers (Evidence Action No Lean Season - 2016 Program Round Actual Expense Breakdown (Unpublished), "2016 report for GiveWell" tab). We met several of these individuals during GiveWell's 2017 site visit to No Lean Season.

  • 31.

    See No Lean Season Budget Projections, tab "EA MgmtEval for BD":

    • "Ongoing management and leadership costs at the country, regional, and global levels necessary to support ongoing program delivery and growth as per three year plan. Focus on establishing and building scale at minimum marginal cost in existing context; maximizing impact and efficiency; mitigating risk and minimizing unintended consequences"
    • "Ongoing communications, donor management, policy engagement, and other operational costs related to the three-year plan in Bangladesh."
    • The table shows to what extent various staffing categories are full-time staff or part-time staff.
    • "The budget should include everything we would spend on the program to deliver, monitor, assess, and manage it in Bangladesh -- the fully-loaded costs of the program that delivers the impact and corresponds to the creation of benefits as assessed in the model.
      "The budget should also include all monitoring and evaluation costs related to assessing the path-to-scale and ongoing delivery-at-scale of the program in Bangladesh, assuming a runway / decision-making time horizon of ~3 years."
  • 32.

    GiveWell's 2017 site visit to No Lean Season

  • 33.

    GiveWell's 2017 site visit to No Lean Season

  • 34.

    No Lean Season uses CommCare (https://www.dimagi.com/products/), an open source mobile data collection and case management platform. In 2017, Evidence Action Beta worked with AgImpact (https://agimpact.org/) to develop a No Lean Season mobile data collection app on the CommCare platform. CommCare was selected for its features such as low reliance on data connectivity and support for the Bengali alphabet. GiveWell's 2017 site visit to No Lean Season

  • 35.

    No Lean Season CommCare Design Graphic. Although this graphic does not include targeting surveys, Evidence Action has told us that the CommCare mobile application also includes modules for targeting surveys (village survey and household survey).

  • 36.

    No Lean Season CommCare Design Graphic
    No Lean Season 2017 CommCare Module Translation

  • 37.

    During GiveWell's 2017 site visit to No Lean Season, we heard from several people about a recent CommCare outage lasting approximately three days, in which data collection reverted to the paper system. Sohel Rana, senior Evidence Action staff member in Bangladesh, described this as a good, though unintentional, test of the backup paper system. GiveWell's 2017 site visit to No Lean Season

  • 38.

    GiveWell's summary of No Lean Season's 2016 spending is based on No Lean Season 2016 program costs.

  • 39.

    No Lean Season Budget Projections

  • 40.
    • 2014 RCT:
        "The landless poor supplying agricultural labor on others' farms are especially affected when demand for agricultural labor falls. They constitute around 56% of the population in our sample area, and are the target of the seasonal migration encouragement intervention that we design. Our sampling frame is representative of this landless population in the Rangpur region of Northern Bangladesh. According to the Bangladesh Bureau of Statistics, there are roughly 15.8 million such inhabitants in Rangpur (BBS, 2011)." Akram, Chowdhury and Mobarak 2017, Pg 5.
      • "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur." Akram, Chowdhury and Mobarak 2017, Pg 7.
    • 2008 RCT (and its follow-up surveys): "Our experiments were conducted in 100 villages in two districts (Kurigram and Lalmonirhat) in the seasonal-famine prone Rangpur region of northwestern Bangladesh." Bryan, Chowdhury and Mobarak 2014, Pg 1675.
    • We are also aware of an unpublished study from 2013, which we discuss in our intervention report.
  • 41.
    • No Lean Season Budget Projections, tab "Budget Projections" shows projected growth in "Total # of Branch Offices." The number of branch offices in which No Lean Season projects operating by 2022 exceeds 171, the total number of RDRS branch offices as of 2016 (see table row "Total Branches," RDRS Bangladesh 2016 Annual Report, Pg 53), and continues to grow at a rate of 10 per year.
    • RDRS works predominantly in Rangpur: "RDRS is unusual is maintaining a concentrated geographic program, focusing on 8 districts and 57 Upazilas (sub-districts) mainly in deprived north-west Bangladesh in Rangpur Division, far from the overcentralized economic and political powerbase of Dhaka, Chittagong, even Rajshahi." RDRS Bangladesh webpage: Where We Work, which also includes a map of areas where RDRS works, showing a geographic emphasis on Rangpur Division.
  • 42.

    For background on seasonal variation in consumption in rural northern Bangladesh, see Khandker 2012 and Khandker and Mahmud 2012.
    In particular, Khandker and Mahmud 2012 discuss seasonal patterns of income, expenditures, and poverty on Pgs 43-61, based on the authors' analysis of data from the Household Income and Expenditure Survey of the Bangladesh Bureau of Statistics, showing marked dips in income and consumption in the lean season. See especially the figures in this section, showing seasonal trends in both Rangpur and the rest of Bangladesh.

  • 43.
    • Evidence Action has confirmed for us that households were eligible for the No Lean Season program in 2017 if they reported owning 50 or fewer decimals of land or reported any incidence of skipped meals in the two weeks preceding the eligibility survey.
    • Eligibility criteria in the 2014 RCT: "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur. We first conducted village censuses to identify all households that would be 'eligible' to receive this intervention in each of these villages. A household was deemed eligible if (1) it owned less than 0.5 acres of land, and (2) it reported back in 2008 that a member had experienced hunger (i.e., skipped meals) during the 2007 monga season. We focused on landownership because land is the most important component of wealth in rural Bangladesh, and it is easily measurable and verifiable. We used the second question on skipping meals to avoid professional, non-agricultural households (who may not own much land, but who are comparatively well off). Our census data suggest that about 57% of households in these villages were eligible to receive the intervention after applying these two criteria." Akram, Chowdhury and Mobarak 2017, Pg 7.
    • Eligibility criteria in the 2008 RCT: "The two districts where the project was conducted (Lalmonirhat and Kurigram) represent the agro-ecological zones that regularly witness the monga famine. We randomly selected 100 villages in these two districts and first conducted a village census in each location in June 2008. Next, we randomly selected 19 households in each village from the set of households that reported (a) that they owned less than 50 decimals of land, and (b) that a household member was forced to miss meals during the prior (2007) monga season. [footnote] Seventy-one percent of the census households owned less than 50 decimals of land, and 63% responded affirmatively to the question about missing meals. Overall, 56% satisfied both criteria, and our sample is therefore representative of the poorer 56% of the rural population in the two districts." Bryan, Chowdhury and Mobarak 2014, Pgs 1677-78.
  • 44.
  • 45.

    "The basic form of our intervention was the offer of a cash grant worth 1,000 Taka ($13.00 USD) to rural households in northern Bangladesh to cover the round-trip cost of travel to nearby cities where there are job opportunities during the lean season. This was a conditional transfer, where the subsidy is conditional on one person from the household agreeing to out-migrate during the lean season. As offers were made, we let households know that they may have a better chance of finding work outside of their village, but we did not offer to make any connections to employers. No requirement was imposed on who within the household had to migrate, or what city they had to go to." Akram, Chowdhury and Mobarak 2017, Pgs 6-7.

  • 46.

    "Seven hundred three households in 37 randomly selected villages were offered cash of 600 Taka (∼US$8.50) at the origin conditional on migration, and an additional bonus of 200 Taka (∼US$3) if the migrant reported to us at the destination during a specified time period. We also provided exactly the same information about jobs and wages to this group as in the information-only treatment. Six hundred Taka covers a little more than the average round-trip cost of safe travel from the two origin districts to the four nearby towns for which we provided job information. We monitored migration behavior carefully and strictly imposed the migration conditionality, so that the 600 Taka intervention was practically equivalent to providing a bus ticket. The 589 households in the final set of 31 villages were offered the same information and the same Tk. 600 + Tk. 200 incentive to migrate, but in the form of a zero-interest loan to be paid back at the end of the monga season. The loan was offered by our partner micro-credit NGOs that have a history of lending money in these villages. There is an implicit understanding of limited liability on these loans since we are lending to the extremely poor during a period of financial hardship." Bryan, Chowdhury and Mobarak 2014, Pg 1678.

  • 47.

    "We randomly assigned the 133 villages into three groups: (a) Low Intensity – 48 villages where we targeted migration subsidies to roughly 14% of the eligible (landless, poor) population. This is comparable to the Bryan et al. (2014) treatment. (b) High Intensity – 47 villages where we targeted roughly 70% of the eligible population with migration subsidy offers. (c) Control – 38 randomly selected villages where nobody was offered a migration subsidy.
    "The high vs. low intensity design was chosen to generate significant variation in the size of the emigration shock, but the precise target (14% vs 70%) varied a little across villages within treatment arms. This is because our village population estimates were dated (from 2008) for most (100) villages, and imprecise in the 33 other villages, which made it difficult for us to precisely estimate the ratio (offers/eligible population) in each village." Akram, Chowdhury and Mobarak 2017, Pg 7.

  • 48.

    "(a) Low Intensity – 48 villages where we targeted migration subsidies to roughly 14% of the eligible (landless, poor) population. This is comparable to the Bryan et al. (2014) treatment." Akram, Chowdhury and Mobarak 2017, Pg 7, referring to the treatment in the 2008 experiment published in Bryan, Chowdhury and Mobarak 2014.

  • 49.

    In 2014 villages treated with cash at low intensity, migration rate was 24.8 percentage points higher than the average migration rate in the control group (34.2%). In villages treated at high intensity, migration rate among those offered was 39.8 percentage points higher than the average migration rate in the control group. Table 1, Akram, Chowdhury and Mobarak 2017, Pg 36. For comparison: in the 2008 RCT, in treatment villages (all at low-intensity), migration rate among those offered was 23.0 percentage points higher in the cash treatment group than in the pure control group (36.0%) and 20.8 percentage points higher in the credit treatment group than in the pure control group. Bryan, Chowdhury and Mobarak 2014, Pg 1683.
    In 2014 villages treated with cash at low intensity, total income among those offered was 2,589 taka higher than the average in the pooled control group (23902.688 taka). In villages treated at high intensity, total income among those offered was 2,105 taka higher than the average in the pooled control group. Table 5, Akram, Chowdhury and Mobarak 2017, Pg 41.

  • 50.

    "In the 68 villages where we provided monetary incentives for people to seasonally out-migrate (37 cash + 31 credit villages), we sometimes randomly assigned additional conditionalities to subsets of households within the village. A trial profile in Figure 2 provides details. Some households were required to migrate in groups, and some were required to migrate to a specific destination. These conditionalities created random within-village variation, which we use as instrumental variables to study spillover effects from one person to another." Bryan, Chowdhury and Mobarak 2014, Pgs 1678-79.

  • 51.

    "All of the implementation activities – the offers and marketing, grant disbursement, and monitoring to ensure adherence to the conditionality, were conducted by RDRS, a local NGO with 40 years of engagement in Rangpur, and substantial presence in the region." Akram, Chowdhury and Mobarak 2017, Pg 8.

  • 52.
    • "In August 2008, we randomly allocated the 100 villages into four groups: Cash, Credit, Information, and Control. These treatments were subsequently implemented on the 19 households in each village in collaboration with PKSF through their partner NGOs with substantial field presence in the two districts." Bryan, Chowdhury and Mobarak 2014, Pgs 1677-78.
    • GiveWell's 2017 site visit to No Lean Season
  • 53.

    50 decimals is roughly equal to half an acre.

  • 54.

    Evidence Action confirmed these 2017 eligibility criteria.

  • 55.

    "For household survey, we will use two stage sampling where we first select the village and then the household. Samples for these events will be selected to meet 95% confidence level (CL) and 5% margin of error (MoE)." No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, tab "Sampling plan."

  • 56.

    In September 2017, GiveWell spoke with Evidence Action's nine monitoring staff members in Bangladesh. They had recently completed the Household Targeting Verification Survey and described the process for this monitoring activity. Evidence Action randomly selected a sample of households from the full list compiled in the course of the household eligibility census, and communicated this list to the monitoring staff. Monitors were required to begin sampling at each of the randomly selected households, then to follow a "left hand rule" to count houses to the left of the starting house, and to subsequently sample the fifth and tenth houses. Evidence Action monitors that houses are counted as clearly visible buildings (if multiple households live in one structure, they would still be counted as one house for the purposes of counting towards the next house to sample). Evidence Action monitors told us that they are required to sample the first household (the household selected by Evidence Action), and hence will return if no adult household member is present at the time of the survey; they may ask neighbors or children over 12 when an adult household member will return. However, if no one is present in the fifth house, they are permitted to instead sample the sixth house (and so on), and subsequently to begin counting five more houses from that point in order to sample a third house in the sampling series. Evidence Action monitors described to us that when counting left-hand houses, if no more houses remain in a village cluster, the monitor turns and from that point begins counting on their new left side (formerly their right side). Evidence Action monitors told us that if they are unsure whether a house belongs to a village cluster (for example, it is remotely visible from other houses), they will inquire with nearby neighbors, and told us that neighbors are generally clearly aware of which houses are part of their village cluster or not. If an adult household member declines to participate in the survey, monitors move on to sample the next household to the left. One male monitor told us that he had experienced a small number (3-4 out of 118) of refusals, because adult household members present during the survey are more likely to be women than men. The female monitors told us they had never experienced refusal to participate in the survey. Evidence Action monitors told us that each survey takes 5-6 minutes. GiveWell's 2017 site visit to No Lean Season

  • 57.

    "At the beginning of the program year, we selected 575 villages in 52 branches to be included in No Lean Season 2017," No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 1. "Verification data was collected in a random subset of villages (n = 123) and a random subset of households within these villages (n = 748)," Pg 5.

  • 58.

    "The Household Targeting Survey is meant to be a complete census of each village selected," No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 1.

  • 59.

    "In each of the villages selected for verification activities, we would wait until the household targeting activity was complete to randomly select one household as starting point for the verification survey. The enumerator then used the left hand rule to interview two additional households skipping 4 households from the one interviewed. After conducting interviews in three households in the same area of the village, the enumerators then moved to the nearest para (i.e. Bangladesh administrative unit within villages) and randomly selected one households to be his starting point. For villages that did not have Paras, the enumerators walked for about 500 meters and then selected a starting household at random. In total, 748 households were visited during the households survey verification exercise." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 5.

  • 60.

    Evidence Action notes this possibility: "It may be the case that particularly 'hard to reach' households or sets of households would be similarly skipped by both the original survey and the verification survey. In the future we may use GPS data and satellite imagery to identify missing clusters of households and small villages. But overall, the results of this verification survey give us a relatively high level of confidence that the Household Targeting Survey successfully achieved extremely high coverage of households within participating villages." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 2.

  • 61.

    "Collecting reliable eligibility criteria, in particular the amount of cultivable land owned, is critical to the program's integrity and impact. We verified the administrative data collected on this criteria using verification data. Results from the analysis of this cross-check show that 64% of the randomly selected households reported exact or near-exact data on cultivable land owned to the independent enumerators." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 4. A footnote on Pg 5 specifies that "Near‐exact means less than a 8 decimal difference in reporting." The table on Pg 4 shows that 480 out of 747 households surveyed reported exact or near-exact amounts of cultivable land owned.

  • 62.

    "Of the 480 observations that were exact or near-exact, 421 were exact matches. For the 36% of households (267 households) with differences, we analyzed whether the discrepancies led to any changes in eligibility between surveys. In this analysis, 24% (63 households) of households' eligibility status would have changed -- 41 households 'false negatives' and 22 were 'false positives'. In total, false positives and false negatives account for 8% of the total sample (63 out of 747 households). In the context of the program goals, 'false negatives' are more of a concern than 'false positives'. This is because providing access to a travel subsidy for a non-eligible household may be slightly inefficient, but it is not reducing the potential impact of the program. On the other hand, not serving a household that should have been eligible is a greater concern. The verification survey shows that this is an issue that we will need to monitor over time, but that the incidence of false negatives is quite low -- only 41 out of 747 cases, or 5%." No Lean Season 2017 Preliminary Monitoring Results: Household Targeting, Pg 4.

  • 63.

    Evidence Action determines household eligibility using the process laid out in this document: No Lean Season 2016 Analysis Plan for Selecting Target Households.

  • 64.

    GiveWell's 2017 site visit to No Lean Season

  • 65.

    We have seen an outline of this process from 2016: No Lean Season 2016 Analysis Plan for Selecting Target Households.

  • 66.
    • "We can wait for your decision until November 30 this year, unless you live in a village that is far from a Branch Office. If that is the case, we will organize another group meeting in your village where you will be able to get the loan instead of having to travel to a Branch Office.
      "However, those who have taken a decision already, please tell me your name. I will make sure that we have a faster disbursement for you.
      "If you are interested, you are welcome to come to our Branch Office <branch name and location> between <dates> to receive the loan. Before that, however, I will come back to your village again to fill out a short loan application form for those who are interested. When you come to the Branch Office to collect the loan, you must bring your National ID Card with you.
      "Anytime between August and November, you can come to the Branch Office to get the loan amount. But just ONE DAY before coming, please give me a phone call so that I can arrange the money for you.
      ...
      "If you want this migration loan for another working member of your family (instead of yourself), please bring him/her with you to the Branch Office. Please make sure that s/he is 18 or more years old and s/he brings a copy of his/her National ID Card with him/her too. If that member doesn't have any National ID Card, please ask him/her to bring his/her Birth Certificate/ Voter Number document. The money will be disbursed only to the person who will apply and ideally, the same person would migrate afterwards. It is not possible to get the money on someone else's behalf."
      No Lean Season 2017 Offer Meeting Script, Pg 3.
    • GiveWell's 2017 site visit to No Lean Season
  • 67.

    "The household-level verification survey will check that each eligible household received a maximum of one stipend of the correct amount, and that the recipient matches the photo and ID in RDRS records." No Lean Season 2017 Process and Performance Monitoring Framework, p 4.

  • 68.

    "Evidence Action will observe a sample of offer meetings, and then will interview a sub-sample of participants from the list of eligible households post-meeting. Evidence Action will assess offer recipients' understanding of key information about the subsidy and check that they were not coerced into accepting the subsidy." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

  • 69.

    "Offer meetings were randomly sampled for the observation survey. During observations, Evidence Action enumerators collect data on whether the Migration Organizer followed protocol and covered key aspects of the script for the offer meeting, as well as collect data on attendance to ascertain who and how many people are attending offer meetings. Additionally, four eligible households were sampled per Migration Organizers for the post-meeting survey; two participants from the observed offer meeting and two other eligible households that were scheduled to attend another offer meeting, but where the offer meeting is unobserved. Evidence Action uses these surveys to assess offer recipients' understanding of key information about the subsidy and checks that they were not pressured into accepting the subsidy. In total, this year, Evidence Action randomly sampled 104 offer meetings for observation and 416 eligible households for post-meeting surveys." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

  • 70.

    "Evidence Action will make random, unannounced visits to branch offices during the disbursement process. During this visit, loan disbursement procedures will be verified, and loan recipients will be interviewed to assess whether they have a correct understanding about the terms of the loan repayment." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 4.

  • 71.

    "It is important that households do not feel pressured or coerced into re-paying the subsidy amount. Evidence Action verifies loan repayment for a sample of households to verify repayment rates and to check that households were not unduly coerced into paying back the loan. This component is under development." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 5.

  • 72.

    GiveWell's 2017 site visit to No Lean Season

  • 73.
    • Some beneficiaries reported feeling some pressure to repay the loan, including disliking the frequency of visits by Migration Organizers:
      • "Some people talked about this type of force from both sites. Apart from this kind of force, all respondents mentioned that they received the reminder from RDRS staff within a month of receiving the loan. Many respondents mentioned that they received frequent reminders from RDRS staffs. Some of them told that they were asked to re-pay in a certain date of the month." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 15.
      • "I took another loan of 1000 BDT to pay off this loan. Poor people are unable to save money. the staff forced me in many ways. He told me that 'I would lose the job if I delay paying the amount. He also told that it is not my duty to see how did you spent the amount, you have taken the loan and you have to pay off the loan'. I felt like, I have to migrate even if I die. I then lend it from my neighbor and paid it. " Excerpt from a day laborer from Kurigram district, focus group discussion. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 15.
      • "The staff told me that 'uncle, I may lose my job if you delay to pay off the loan'. I replied I will not cause you that damage and paid off the loan by taking another loan." Excerpt from a farmer from Kurigram district, focus group discussion. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 15.
      • "Some people mentioned that staffs frequent visited the loan recipients household that created a stress to repay the loan. After taking out loan, program organizers visited respective household frequently that many loan recipients did not appreciate. Loan recipients saw it as a type of indirect force to repay the loan. They saw it as a lack of trust between loan recipients and implementers. Since people took out loan after submitting relevant documents with a formal agreement, they need not to create any mental pressure for the loan recipients. Frequent visit by the implementer was not appreciated by many respondents and even by the female members of the family." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pgs 15-16.
      • "If someone [RDRS staff] faces any problem because of me and he visits my home every day for money, it does not feel good. It creates a pressure for us to pay the loan as soon as possible. I then questioned myself that why have I taken the loan? I took a loan from another person and returned the loan of RDRS. " Excerpt from discussion with an agricultural day laborer from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 16.
      • "How many times do you have to visit us, we already have returned the loan. If every time people start coming to our house, how could we live in the house. We have many things to do and are very busy with rice plantation. Why do you come repeatedly?" Excerpt from an informal discussion with the wife of a recipient from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 16.
      • "Program staffs need to think about developing an effective mechanism of monitoring loan recipients avoiding annoyance among the recipients. It was clear from the field research that loan recipients did not like the tendency of RDRS staff who visited them frequently to monitor. Although RDRS staff mentioned them that they just came to see them not to monitor them, people took it as a negative activity of the program that was discussed in the findings section in detail. Therefore, the frequency of visiting the household can be reduced, especially when the household head is away, because people from the community do not like outsiders to interact with their female counterpart frequently." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pgs 25-26.
      • "An incident was found during the field trip where the son of a loan recipient did not like the behavior of one staff of RDRS. A younger son of a loan receiver reported that the PO was forcing them to return the loan without giving extra time. The person was sick on that day and requested to extend the time but the program organizers did not listen to their request. In addition, it was only one month of getting the loan, they added. This was the only incident that we experienced during the field visit that shows a clear dissatisfaction." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 16 and described again: "No one reported about malpractice by the program organizers of RDRS in the Sonahar of Panchagar District. However, one incident was found where a son of a loan receiver reported that the Program Organizer was forcing them to return the loan without giving extra time and later on, we discovered many more incidents like this. The person was sick on that day and requested to extend the time but the program organizers did not listen to their request. In addition, it was only one month of getting the loan, they added." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 25.
  • 74.
    • "Most of the people told the research team that they had returned the loan right after coming back to their village from their destination place." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 14.
    • "It was in my head that I have to pay back the loan when I would go back home. I started to save 20-30 BDT everyday from my daily wage for paying back the loan, because if I go back home without saving money for this loan, it would be very difficult [he can spend the entire amount for household tasks] for me to arrange money. I saved about 800-900 BDT. After working couple of days more, I arranged the entire amount to pay off the loan. However, I did not return the entire amount at a time, but by two installments. The staff of RDRS listened to my request." Excerpt from an informal discussion with a day laborer in oil production from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 14.
    • "I found it very difficult to pay off the loan, as I did not go out to work. I did not find suitable work to do during this time. I took some loan from my neighbor and saved some money by working here and there to return the loan. The staffs of RDRS were very kind to me not to pressurize for paying off the loan." Excerpt from an informal discussion with a 35-year-old small business man from Kurigram district. Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pg 17.
  • 75.

    "A different perception was notice during the second field trip where a few people shared their views about some activities of the implementers, where implementers told some loan recipients to talk about good side of the program. A respondent from Sonahar, Panchagar, mentioned in this context that "RDRS staff told that someone from Dhaka would come to talk to us soon. If I talk to them in a good manner and say good things about the project, I would get loan again." - A Day Laborer, Sonahar, IDI, Panchagar" This is the only response that we discovered from the field research that can be seen as an isolated event. The researchers conducted interviews in absence of RDRS staff. The research team found that RDRS staff briefed many loan recipients prior to the visit of the research team. It did not affect the findings of the research as the research team engaged with them for a longer time building a good rapport. Development of a good rapport helped the team to discover this behavioral pattern of some RDRS staff in the field. This was not discovered in the first field trip, the research team found this type of comments during the second trip. The research team did not informed RDRS staff before hand of the second field trip." Improving Design and Delivery of Migration Stipend Program in Northern Bangladesh, Pgs 17-18.

  • 76.

    "Migration Organizers use the migration compliance survey to ascertain whether subsidy recipients actually migrate.

    "This key performance metric will be verified by Evidence Action by visiting a sample of households and conducting a supplementary interview to assess the incidence of migration. In this stage, we may draw the sample from the entire set of HHs surveyed (both eligible and ineligible households) in order to understand migration rates across the entire community. In steady state, we would also like to estimate the rate at which beneficiaries of the program re-migrate in the years following their receipt of the subsidy, even without the program intervention."

    No Lean Season 2017 Process and Performance Monitoring Framework, Pgs 4-5.

  • 77.

    No Lean Season 2017 Process and Performance Monitoring Framework Spreadsheet, "FullFramework" sheet.

  • 78.

    No Lean Season 2016 summary of data, "Migration" sheet.

  • 79.

    No Lean Season 2016 summary of data, "Migration" sheet.

  • 80.

    More detail here: No Lean Season 2016 summary of data, "Migration" sheet.

  • 81.
  • 82.

    GiveWell's 2017 site visit to No Lean Season

  • 83.

    "Migration Compliance: Migration Organizers make regular visits to the households that receive the subsidy to check migration compliance. They use the migration compliance survey to ascertain whether subsidy recipients actually migrate.
    "This key performance metric is verified by Evidence Action by visiting a sample of households and conducting a supplementary monitoring survey to assess the incidence of migration, verify loan and migrant details, and detect any coercion from Migration Organizer. In this stage, we draw a random sample from the entire set of HHs surveyed in the household targeting survey (including both eligible and ineligible households) in order to understand migration rates across the entire community. In steady state, we would also like to estimate the rate at which beneficiaries of the program re-migrate in the years following their receipt of the subsidy, even without the program intervention.
    "This year, a total of 748 households were randomly sampled for the migration compliance verification survey using a two-stage random sampling, similar to that of the Household Targeting verification survey. In other words, we randomly sample 123 villages and randomly select 6 households per village. After calculating the sample size, we then create three groups from which to sample for this survey (2 households from each group in each village). These three groups are: 'Accepted Loan' (households that took out the loan); 'Eligible, Non-Loan' (households that were eligible for the loan but did not take the loan yet); and, 'Not eligible' (households that were not eligible for the loan'). This design allows us to capturing the rate of the migration in these communities, regardless of whether the household took out a No Lean Season loan." No Lean Season 2017 Process and Performance Monitoring Framework, Pgs 5-6.

  • 84.

    "Migration Organizers make regular visits to the households that receive the subsidy to check migration compliance. They use the migration compliance survey to ascertain whether subsidy recipients actually migrate.

    "This key performance metric will be verified by Evidence Action by visiting a sample of households and conducting a supplementary interview to assess the incidence of migration. In this stage, we may draw the sample from the entire set of HHs surveyed (both eligible and ineligible households) in order to understand migration rates across the entire community. In steady state, we would also like to estimate the rate at which beneficiaries of the program re-migrate in the years following their receipt of the subsidy, even without the program intervention." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 5.

  • 85.

    "Rigorous Investigations on the Emergency Cases … Was this migrant sick, injured or dealing with a family emergency which restricted him/her from migration/work? … Please provide details of reason for restriction (If available take photos of any necessary documentation)." No Lean Season 2017 CommCare Module Translation, sheet "module8_form1," cells B77, B78, and B81.

  • 86.

    See No Lean Season 2016 summary of data and No Lean Season 2016 summary of data (guide). We have also seen unpublished underlying data from this survey.

  • 87.

    "It is important that households do not feel pressured into re-paying the subsidy amount. Evidence Action plans to assess loan repayment activities to verify repayment rates and to check that households were not unduly pressured into paying back the loan... In addition to the loan recovery verification activities, Evidence Action plans to conduct migration debrief verification. Upon migrants' return, Migration Organizers follow-up at the household to collect loan repayments and to conduct a debriefing about the household's migration experience. This debrief survey conducted by Migration Organizers collects data on migration dates, employment, wages, and costs of migration. The migration debrief verification survey will verify the data collected by the Migration Organizers is consistent and reliable. It is planned that these activities will accompany the loan recovery verification as these two steps are implemented together by RDRS, and verification surveys should be carried out in a similar manner." No Lean Season 2017 Process and Performance Monitoring Framework, Pg 6.

  • 88.

    For calculations, see our cost-effectiveness model, "Migration subsidies" sheet, cell B110.

  • 89.

    No Lean Season's program structure was considerably different in previous years. In 2014, as part of a research agenda, in some villages the program enrolled at "high intensity" (enrolling approximately 70% of eligible households) and in other villages at "low intensity" (enrolling approximately 14% of eligible households):

    • "The experiment was conducted in 133 randomly selected villages in Kurigram and Lalmonirhat districts of Rangpur. We first conducted village censuses to identify all households that would be 'eligible' to receive this intervention in each of these villages. A household was deemed eligible if (1) it owned less than 0.5 acres of land, and (2) it reported back in 2008 that a member had experienced hunger (i.e., skipped meals) during the 2007 monga season. We focused on landownership because land is the most important component of wealth in rural Bangladesh, and it is easily measurable and verifiable. We used the second question on skipping meals to avoid professional, non-agricultural households (who may not own much land, but who are comparatively well off). Our census data suggest that about 57% of households in these villages were eligible to receive the intervention after applying these two criteria." Akram, Chowdhury and Mobarak 2017, p. 7.
    • "We randomly assigned the 133 villages into three groups: (a) Low Intensity – 48 villages where we targeted migration subsidies to roughly 14% of the eligible (landless, poor) population. This is comparable to the Bryan et al. (2014) treatment. (b) High Intensity – 47 villages where we targeted roughly 70% of the eligible population with migration subsidy offers. (c) Control – 38 randomly selected villages where nobody was offered a migration subsidy." Akram, Chowdhury and Mobarak 2017, p. 7.

    In 2016, the program operated at a very small scale: "In 2016, we decided to work with 15 of 164 Branch Offices (as agreed in the MOU with RDRS). This was an increase from previous years, but a number that felt appropriate in terms of overall budget and implementation capacity." No Lean Season 2016 summary of data (guide), Pg 2.

    Spending data from 2017 is not yet available because disbursements are ongoing as of this writing.

  • 90.

    No Lean Season Budget Projections, unpublished tab.

  • 91.

    No Lean Season Budget Projections, tab "Budget Projections," column "2017" (unpublished), ratio of "Total # of eligible households" to "Total # of HHs per year" (households targeted by the eligibility survey): 140,061 / 183,000 = 77%. These figures are estimates made by No Lean Season.

  • 92.

    No Lean Season Budget Projections, tab "Budget Projections," column "2018," ratio of "Total # of Disbursements" to "Total # of eligible households": 92,440 / 154,067 = 60%. These figures are estimates made by No Lean Season.

  • 93.
  • 94.

    See this spreadsheet, "Available and expected funding" tab.

  • 95.

    No Lean Season Budget Projections, unpublished tab.

  • 96.

    Karen Levy, Evidence Action Director of Global Innovation, conversation with GiveWell, November 6, 2017 (unpublished).

  • 97.
  • 98.

    Karen Levy, Evidence Action Director of Global Innovation, conversation with GiveWell, November 6, 2017 (unpublished).

  • 99.

    No Lean Season Budget Projections, tab "EA MgmtEval for BD," rows "Research collaborators (PI, RA)" and "RCT and learning agenda."

  • 100.

    No Lean Season Budget Projections, tab "EA MgmtEval for BD."

  • 101.

    No Lean Season Budget Projections, tab "RFMF."

  • 102.
    • No Lean Season Budget Projections
    • "As greater visibility has been obtained into the organizational financial position as part of the clean up process, it has become evidence that the level of available and uncommitted unrestricted funding is very low– at the end of June 2017, unrestricted funds covered approximately 4 weeks of the organization's annual expenses. Unrestricted funding serves as the sole organizational backstop with the exception of Deworm the World reserves. Moreover maintaining a robust level of flexible unrestricted reserve is seen as particularly important during this period when we are still dealing with the implications of the issues discovered during the ongoing clean-up. Looking forward, Evidence Action is seeking to develop a further organizational backstop through the development and institutionalization of an organization-wide reserve policy." Evidence Action 2016-2017 revenue and expenditures, tab "Unrestricted commitments."
    • "After further internal discussion we would prefer to convert all reserves to organization-wide reserves. By maintaining central organizational reserves we anticipate we will actually require a lower amount of cumulative total reserves over time, since we diversify risk across programs. [...] It should be noted that we expect few funders besides GiveWell to contribute to reserves based on our knowledge of current donors." Evidence Action memo to GiveWell, 23 October 2017, unpublished.
    • Kanika Bahl and Grace Hollister, Evidence Action CEO and Deworm the World Director, conversation with GiveWell, October 16, 2017 (unpublished).
  • 103.

    No Lean Season Budget Projections, tab "Budget Projections" shows projected growth in "Total # of Branch Offices." The number of branch offices in which No Lean Season projects operating by 2022 exceeds 171, the total number of RDRS branch offices as of 2016 (see table row "Total Branches," RDRS Bangladesh 2016 Annual Report, Pg 53), and continues to grow at a rate of 10 per year.

  • 104.

    No Lean Season Budget Projections, tab "Budget Projections," column "2017" (unpublished), ratio of "Total # of eligible households" to "Total # of HHs per year" (households targeted by the eligibility survey): 140,061 / 183,000 = 77%. These figures are estimates made by No Lean Season.

  • 105.

    No Lean Season Budget Projections, tab "Budget Projections," column "2018," ratio of "Total # of Disbursements" to "Total # of eligible households": 92,440 / 154,067 = 60%. These figures are estimates made by No Lean Season.

  • 106.
    • In September 2017, Elie Hassenfeld, Executive Director of GiveWell and manager of the EA Fund for Global Health and Development, recommended that the EA Fund grant $150,000 to Evidence Action for No Lean Season.
    • "No Lean Season had underestimated the number of people who would take No Lean Season's loan offer, and needed funding to prevent having to turn interested borrowers away. This was particularly important because No Lean Season is currently running a randomized controlled trial of its program and wanted the program to reach as many people as possible to increase statistical power and increase the likelihood of measuring a statistically meaningful result.

      "I recommended a grant of $150,000 because Karen told me that this amount would allow No Lean Season to keep operating its program without interruption."

      EA Fund Payout Report, September 2017: No Lean Season Grant

    • In October 2017, GiveWell recommended that Good Ventures backstop up to $1.5 million above No Lean Season's existing level of funding to ensure that it was able to disburse loans to enrolled households for the 2017 season.
  • 107.

    No Lean Season Budget Projections, unpublished tab.

  • 108.
    • "Our current programs are:
      • Dispensers for Safe Water is an entrepreneurial program for more than 4.7 million people in rural Eastern and Southern Africa. It is a proven innovation that dramatically expands access to water treatment at an extremely low cost.
      • Deworm the World Initiative scales up school-based deworming programs worldwide to improve children's health, education, and long-term development. It currently supports deworming of more than 200 million children annually.
      • Evidence Action Beta explores what programs with proven impact might work next for millions of people. Similar to beta testing for software, we are pressure-testing promising interventions to determine their suitability for massive scale-up."

      Evidence Action webpage: "What We Do"

    • GiveWell conducted a site visit to Dispensers for Safe Water in November 2012 and published notes: GiveWell's 2012 site visit to Dispensers for Safe Water.