GiveDirectly is one of our top-rated charities
, and an organization that we feel offers donors an outstanding opportunity to accomplish good with their donations.
Published: November 2012
GiveDirectly (www.givedirectly.org) transfers cash to households in the developing world via the M-PESA mobile phone-based payment service. It targets extremely low-income households and aims to deliver at least 90 cents directly to recipients for every $1.00 in total expenses. (More
GiveDirectly is a recommended organization because of:
- Its focus on cash transfers: delivering 90% of all expenses directly to extremely low-income people in the developing world. We feel that this intervention faces an unusually low burden of proof, though donors' intuitive reactions to it may vary widely. The evidence that such transfers increase short-term consumption, especially of food, is very strong, and there is more limited evidence that such transfers may be invested at very high rates of return (e.g. ~20% annually). (More)
- Its documented success, to date, in reaching its 90% target, and what we perceive as a strong (and evolving) process for ensuring that cash is well-targeted and efficiently delivered. (More)
- Its strong transparency and commitment to self-evaluation. Among other things, GiveDirectly has two high-quality studies of its impact ongoing, and has taken the unusual step of making the details of these studies public before data is collected. (More)
- Room for more funding - we believe that GiveDirectly can use substantial additional funding productively, both for its core work of delivering cash and for investigative work on refining its (new and unusual) approach. (More)
Our full review, below, discusses our full assessment of GiveDirectly, including what we see as its strengths and weaknesses as well as issues we have yet to resolve.
Our investigation process
To date, our investigation process has consisted of
- Conversations with GiveDirectly co-founders and board members Paul Niehaus (also serving as CEO), Rohit Wanchoo (also serving as CFO), Michael Faye (Chairman of the Board), and Jeremy Shapiro as well as GiveDirectly COO Piali Mukhopadhyay.
- Reviewing documents GiveDirectly sent in response to our queries.
In November 2012, we visited GiveDirectly's operations in Kenya where we met with beneficiaries of its work and spoke with its local field staff (including its field monitor and one of 8 staff members responsible for enrolling new recipients).
Older content on GiveDirectly:
What do they do?
GiveDirectly transfers cash to poor households in the developing world via the M-PESA mobile phone-based payment service. It is currently active in Kenya, with plans to expand into an additional country during 2013.
Its current model involves grants of $1,000 (USD) over 9 months, after which recipients become ineligible, though it has stated that it is considering experimenting with other grant size/duration approaches (more
). GiveDirectly aims to help the poorest households, targeting those that are in "acute poverty." Recipient households are currently identified by (a) selecting poor villages based on factors such as number of boreholes, proximity to schools, roads, etc.; (b) determining eligibility based on the materials that houses are made of (more
). It aims to deliver at least 90 cents directly to recipients of every $1.00 in total expenses.
- Discuss the structure of GiveDirectly's transfers.
- Summarize GiveDirectly's transfers and other expenses to date.
- Discuss GiveDirectly's process for identifying recipient households and delivering cash transfers.
GiveDirectly's current model involves grants of $1,000 (USD) over 9 months, after which recipients become ineligible. This is a very different approach from the approach we've seen in government cash transfer programs
. One way of putting the difference (which has been reflected in GiveDirectly's communications with us) is that government programs aim for "income transfers" (small supplements to income over many years) whereas GiveDirectly aims for "wealth transfers" (large, one-off transfers that hopefully give people more flexibility to make large purchases and investments).
Note that household size varies substantially in the data we've reviewed (from the first round of Siaya transfers): while the mean household size is ~4.7 and the median size is 4, 16% of households have 1 or 2 people, ~20% have 6 or more, and the maximum household size is 16. We estimate that the average family receives $288 per capita from GiveDirectly, which is 121% of baseline annual consumption per capita.
Transfers and other expenses to date
GiveDirectly is a relatively young organization. It was founded in 2008 when its founders were graduate students in economic development; Paul Niehaus, GiveDirectly's CEO and co-founder, is now an Assistant Professor of Economics at the University of California, San Diego and its other co-founders and board members work in the private sector. GiveDirectly delivered its first cash transfers in 2011 (see table below). It has had one full-time staff member working since January 2011.
Thus far, it has worked on 4 sets of transfers in two districts within Western Kenya: Rarieda and Siaya.
- Rarieda transfers: the initial transfers GiveDirectly delivered were given to the treatment groups of a randomized controlled trial of the effects of unconditional cash transfers.
- Siaya I transfers: one set of transfers was initiated in June 2012.
- Siaya II transfers: another set of transfers initiated in November 2012.
- Nike transfers: additional transfers to young women in Siaya district as part of a project funded by the Nike Foundation. This set of transfers also includes a randomized controlled trial evaluation.
Details of each of these transfers is provided in this table:
|Set of transfers||Number of recipients||Dates of transfers||Size of transfers||Current status (as of 11/21/2012)|
|Rarieda, group A||358||July 2011-November 2012||$300||34 people have yet to receive full transfers ($6,056 left to disburse)|
|Rarieda, group B||140||July 2011-November 2012||$1,000||17 people have yet to receive full transfers ($6,824 left to disburse)|
|Siaya I||199||July 2012-January 2013||$1,000||First $500 transfer completed for all but 1 household; second transfer scheduled to be sent in January|
|Siaya II||Target of 900||December 2012-August 2013||$1,000||Enrollment in process as of November 2012|
|Nike, group A||41||Sept 2012 - ongoing||$500||Have started transferring to 19 girls, have not initiated transfers for the other 22|
|Nike, group B||37||Sept 2012 - ongoing||$1,000||Have started transferring to 22 girls, have not initiated transfers for the other 15|
In the fiscal year 9/1/11-8/31/12, GiveDirectly reports paying out $333,994 in cash transfers (not including transfer expenses and the expenses of the below process). Direct cash transfers have accounted for ~89.7% of GiveDirectly's total expenses in that fiscal year. GiveDirectly projects a final ratio of ~92% for its four ongoing projects. Below we break down GiveDirectly's full organizational costs incurred to deliver these transfers:
|Type of expense||Incurred||Future||Total cost||Per HH||% of total|
GiveDirectly's current process, which it is following in November 2012 as it enrolls households in Siaya, is laid out in the document "Operational Process Overview" (GiveDirectly 2012). We describe this process below; we note differences between its current process and past processes and provide details in footnotes.
The steps are as follows:
- Selection of a region. GiveDirectly told us that it initially chose to work in Western Kenya based on poverty statistics. We have not yet reviewed this data.
- Selection of counties and villages. GiveDirectly states that its staff (COO level) uses data on poverty, population density, security, and presence of poverty-focused NGOs (with the goal of avoiding overlapping with these) to select counties, then uses a predictive model of income to select villages. GiveDirectly has shared the full details of its village selection process, including data for each village and the method for weighting the different factors used to select villages; we have not done a similar examination of its county selection process. For details on how GiveDirectly has targeted villages historically, see this footnote.
- Selection of households. After selecting a village and meeting with local officials (a function performed by GiveDirectly's COO), GiveDirectly retains staff to create a census of all households in the village (this step was different in past transfer rounds), noting which are eligible for transfers (the criterion for eligibility at this stage is that the home is made of mud and thatch) and collecting other data including GPS coordinates. We have not reviewed census data. (This process was different in the project funded by the Nike Foundation because the proscribed target population of that study is young women.)
- Enrollment and eligibility checks. GiveDirectly retains a separate set of staff to visit households marked as eligible and give them a SIM card (if they do not already have an M-PESA account), which is used to transfer funds through the M-PESA system, and collect other data including GPS coordinates which can be checked against the initial data set (with discrepancies being flagged for audit). This process was different in past transfer rounds.
We have reviewed (and made public) a set of data collected on these households, with deletions to preserve anonymity. GiveDirectly has told us that in the above enrollment and eligibility checks and the further checks and audits described below, it errs on the side of excluding rather than including a recipient if there is reason to believe that the potential recipient does not meet GiveDirectly's eligibility criteria.
- Further checks and audits. GiveDirectly performs a series of checks and audits to ensure eligibility:
- GiveDirectly staff "check that the data collected in enrollment and the village census match, that GPS coordinates show an eligible house, that we have a valid photograph showing recipients in front of an eligible house, etc." Discrepancies are flagged for audit. We have reviewed (and made public) a set of data from these checks, with deletions to preserve anonymity. Note that in enrollment rounds completed before November 2012, GiveDirectly did not use all of these steps as "hard checks." Potential recipients could remain eligible even if they failed one of these steps.
- A village meeting is held (different from past rounds) "to answer questions anyone may have about the program, clarify that we aren’t affiliated with a political party, etc."
- GiveDirectly retains a third set of staff to conduct "back-checks" following the meeting, re-verifying eligibility and asking whether households were asked to pay a bribe to enroll. Discrepancies are flagged for follow-up phone calls or audits. We have reviewed (and made public) a set of data from these checks, with deletions to preserve anonymity. GiveDirectly aims to identify all households and does not disqualify a household merely because a potential recipient is not home.
- GiveDirectly's COO "audits 15% of recipients including any who were flagged in any of the checks and including some of the work done by each field worker." We have reviewed (and made public) a set of data from these checks, with deletions to preserve anonymity.
- GiveDirectly staff "call recipients to check that they know what they’re doing and ask if they had any problems / had to pay anyone to enroll." This includes verifying that their SIM card number matches the number they provided during enrollment. Staff make a first call soon after GiveDirectly transfers an initial, small installment of funds (approximately 5% of the total transfer) to recipients. This is different from past transfer processes. GiveDirectly staff make an additional call to recipients 1-2 weeks after the transfer is sent to confirm that it was received, ask how recipients spent their funds, and ask again whether they experienced any problems. We have reviewed two sets of aggregated data from these calls.
Key differences in some past distributions were (a) the lack of a "census" (instead, GiveDirectly asked village officials to take them to eligible households, and thus conducted two in-person checks of each house rather than three); (b) the lack of a village meeting.
In the past, GiveDirectly has used a somewhat informal "preponderance of evidence" approach to determining eligibility: while it has conducted all the checks above, it has been open to declaring people eligible despite, e.g., the lack of a photo or GPS match. It has expressed an interest in more rigid adherence to formal criteria in the future. We discuss the details of how the "preponderance of evidence approach" was implemented below.
Does it work?
This section discusses the following questions:
- Generally speaking, are unconditional cash transfers a promising approach to helping people? We believe that this approach faces an unusually low burden of proof, and that the available evidence is consistent with the idea that unconditional cash transfers help people.
- Does GiveDirectly have a reasonable and effective process for selecting recipients and getting cash to them? We believe GiveDirectly's process is a reasonable way to identify low-income people, and it appears that the process has been relatively effective so far.
- Are the people meeting GiveDirectly's criteria low-income? The evidence we have suggests that they are.
- Do the cash transfers cause problems and complications that offset their positive impact? GiveDirectly has taken some measures to address this question, and we believe that the evidence so far suggests that - while the cash transfers do raise some problems - these problems are relatively minor in the scheme of things. This is an area where we would particularly like to see more information in the future.
- Are the size and structure of the cash transfers well-thought-through and appropriate? We find GiveDirectly's approach to be defensible, but hope to see more experimentation with different approaches in the future.
Generally speaking, are unconditional cash transfers a promising approach to helping people?
We discuss this question more extensively at our report on cash transfers
. In brief:
- Cash transfers are amongst the best-studied development interventions, though questions remain. These studies generally show substantial increases in short-term consumption, especially food, and little evidence of negative impacts (e.g. increases in alcohol or tobacco consumption).
- There is also some evidence that recipients are able to invest cash transfers at high rates of return (e.g. ~20% per year), leading to long-term increases in consumption.
- We feel that this intervention faces an unusually low burden of proof, though donors' intuitive reactions to it may vary widely.
Does GiveDirectly have a reasonable and effective process for selecting recipients and getting cash to them?
GiveDirectly's process is described above.
We find it to be generally reasonable, in that
- We would guess that most people living in mud and thatch homes are low-income, particularly in Kenyan villages that available data has identified as high-poverty. (Our site visit in late 2012 was consistent with this.)
- The process involves multiple visits by different staff to each recipient home, as well as spot-checks and remote checks by higher-level staff, in order to confirm that recipients meet criteria. (The idea being that if someone tries to temporarily occupy a mud-and-thatch home in order to be enrolled, they are unlikely to be sure of being present for future re-checks.)
- The process also involves some procedures for catching other issues, in particular the phone surveys administered after each installment of the transfer.
We have examined data collected by GiveDirectly from its enrollment process, back-checks, remote checks and audits in Siaya, Kenya. We have also examined a summary of the results of its post-cash-transfer phone surveys.
The enrollment and back-check file lists 226 individuals who were initially enrolled in the program. 27 of these were ultimately deemed ineligible and transfers were made to 199 of them.
GiveDirectly's CEO told us that ineligibility was determined by a "preponderance of the evidence." We have not seen documentation of GiveDirectly's process in reaching these conclusions. GiveDirectly told us that in most cases where a household was deemed ineligible, GiveDirectly came to believe during its back-check based on information from other villagers that the ineligible individuals either did not live in the village GiveDirectly was targeting or lived in metal-roof houses and had tried to game the system by pretending to live in thatch-roof houses. (GiveDirectly believes many of these ineligible households were introduced because GiveDirectly allowed the Village Elder to lead staffers to households, enabling the Village Elder to help his friends or family. For this reason, GiveDirectly has changed its process so as not to rely on the Village Elder as a guide.) The file also shows that the 199 eligible households were checked and met GiveDirectly's eligibility criteria.
Are the people meeting GiveDirectly's criteria low-income?
GiveDirectly has collected data on poverty levels of the people receiving cash transfers as part of its studies
(these people were selected using the process detailed above). GiveDirectly has shared the full survey form used to interview participants, as well as its own summary of the data collected so far, as of March 2012:
Well over half of adults skip meals, less than half of household members eat until they are content, people commonly go to sleep hungry and a paltry 18% report having enough food for tomorrow in their household. Those living in eligible homes are even worse-off than the average household, consuming less and holding fewer assets. Overall, mean and median daily per capita consumption among eligible households are $0.65 and $0.55 at nominal rates, and 74% are below the Kenyan poverty line, indicating a very poor population.
GiveDirectly also provided charts that show a clear difference in the consumption, expenditures, and assets of households in mud and thatch homes compared to those in cement homes, but fairly small differences between those living in mud and thatch homes and those living in mud and iron roof homes.
GiveDirectly shared mid-line results from the RCT that Innovations for Poverty Action
is conducting for its transfers in Rarieda. While it shared all the results with us, the study’s Principal Investigators asked that we not publish the document at this time. GiveDirectly also notes that these results are preliminary as (a) recipients had only received $200 of the transfer at the time they were surveyed and may spend the first dollars they receive differently from later dollars; (b) at the mid-line, researchers only surveyed recipients on a subset of the variables they thought would ultimately be most important; and (c) respondents were surveyed less than a year after beginning to receive transfers so these results may not fully reflect seasonal variations due to the agricultural cycle, school fees, and so forth. With these caveats in mind, the mid-line results (for those in the control group who met the program's eligibility criteria but were not selected to receive transfers) show adults (26%) and children (39%) reporting that they have gone whole days without meals in the past month. Only 69% of households report eating until content each day. Other results related to food consumption are measured as well which are, in our view, consistent with the notion that recipients are extremely poor. We list all estimated treatment effects in this footnote.
Anecdotal evidence from GiveWell's site visit to Kenya
In November 2012, GiveWell staff visited Kenya to view GiveDirectly's program in the field. See our notes and photographs from the site visit
. We visited 5 locations (3 in Siaya and 2 in Rarieda) where GiveDirectly had transferred funds or was in the process of enrolling recipients to receive funds. We visited approximately 15 households across the 5 locations (including 2 non-recipient households that had metal roofs and cement walls and floors and did not qualify for GiveDirectly's program). For details on how homes we visited were selected, see this footnote.
We would characterize the ~15 households we visited (as well as other households we saw while walking but did not speak with directly) as extremely poor. We summarize characteristics of these households as follows:
- Most homes are made up of three rooms.
- The main room is a sitting area. In the homes we visited, this room varied in size from approximately 10'x12' to 12'x20'. This room has 3 doors: one to the outside; one to what appeared to be a storage room; one to a bedroom. The husband and wife sleep in the bedroom and the children sleep in the storage room or in the kitchen.
- The kitchen is often a separate structure, most often thatched-roof (even for homes that have tin roofs). Some households have no kitchen structure but just cook outside. Others have a small kitchen in place of a storage room.
- There are no doors in between rooms in the house, just hung curtains.
- The living room has many chairs and couches for sitting. They often almost fully cover the wall area (aside from doors). There are also coffee tables in the middle of the room. Poorer homes had less furniture; one home we saw had a single chair and a single broken table.
- People have wall hangings for decoration. The most common hanging we saw was old calendars (e.g., from 2003) that have pictures and can be used for decoration.
- Most houses had 1-2 kerosene lamps that provide light since they don't have electricity. One home (a non-recipient we visited) had two electric lights, which were powered with a solar panel.
- People we spoke with reported walking 5-20 minutes daily to obtain drinking water, which one recipient reported doing 1-2x per day.
- Most households owned a bicycle.
- Some households have radios (both of the non-recipients we saw had radios; one had a TV). Of the others, 3/14 had radios. These were most often powered with what looks like a car battery .
- Most people owned one cell phone pre-GiveDirectly.
- Households tend to own some livestock. Most commonly, we saw 1-2 cows and 4-5 chickens each. They say they use the milk from the cows for themselves and also sell it and many mentioned being able to sell their cows in the future when they need money for kids to go to secondary school.
Note that among the households we visited, many had already received part or all of their transfer from GiveDirectly, so these characterizations are based on a selection of households that include some newly-built or renovated structures in addition to older structures. Given that some of the recipients we met used transfers to build larger houses or buy livestock, our observations would likely over-estimate the assets of each household pre-transfer.
In addition, the homes we saw from afar in villages we visited and homes we passed while driving in the area appeared to be at a similar level of extreme poverty.
How do recipients spend their cash, and how does this spending impact their lives?
GiveDirectly provided us with data from responses recipients gave to an informal survey conducted by GiveDirectly staff for Rarieda and Siaya, concerning how recipients used their cash transfers. During our site visit in Kenya, GiveDirectly shared a document that reported the amount that each recipient in the Siaya I transfer round had spent on various items. The following were the primary uses of funds: (Note that GiveDirectly also explicitly asks respondents about other uses of funds, which we don't report below.)
- 67%: home improvement (e.g., replacing a thatched roof with an iron-sheet roof)
- 9%: other (during our site visit, we met with people who had spent funds on school uniforms, a motorcycle, among other things)
- 9%: livestock
- 4%: food
In our site visit to GiveDirectly recipients in Kenya, we asked about the value of these items. Recipients reported that their thatched-roofs frequently leak when it rains and require replacement every 3-4 months at a cost of 1,000 Kenyan shillings ($11.68 based on the exchange rate as of November 15, 2012) as well as time/labor. One recipient also reported that when it rains, she moves her family and their belongings into other structures to stay dry. Recipients reported buying livestock as an investment/savings device, hoping that they could (a) use the milk from the cow or goat for additional income and (b) sell the cow or goat and any offspring in the future if/when they needed additional funds (for e.g., secondary school fees for their children which are approximately 15,000 Kenyan shillings per year ($175.13 based on the exchange rate as of November 15, 2012).
GiveDirectly also provided us with mid-line results from the RCT that Innovations for Poverty Action
is conducting for its transfers in Rarieda. While it shared all the results with us, the study’s Principal Investigators asked that we not publish the document at this time. GiveDirectly also notes that these results are preliminary as (a) recipients had only received $200 of the transfer at the time they were surveyed and may spend the first dollars they receive differently from later dollars; (b) at the mid-line, researchers only surveyed recipients on a subset of the variables they thought would ultimately be most important; and (c) respondents were surveyed less than a year after beginning to receive transfers so these results may not fully reflect seasonal variations due to the agricultural cycle, school fees, and so forth. With these caveats in mind, the mid-line results find significant (and statistically significant) increases in spending on food, housing, household durables, livestock, land purchases, and travel. It also finds statistically significant impacts on other variables though the amounts spent are smaller. It does not find a statistically significant effect on alcohol or tobacco expenditure. Of 43 total estimated treatment effects, 17 are significant at the 1% level, 22 at the 5% level, and 23 at the 10% level.
We do not currently have information concerning further-out impacts such as nutritional or educational status. GiveDirectly's ongoing study
in Rarieda may provide such information; however, because cash transfers may have benefits that are spread across many different outcomes, it may be difficult for a study to pick up an impact on a particular outcome.
Do the cash transfers cause problems and complications that offset their positive impact?
- Does distribution to some community members and not others result in jealousy, conflict, or related issues? GiveDirectly surveys recipients (post-transfer) on the following questions:
GiveDirectly sent us results from these surveys for Rarieda and Siaya.
For Rarieda, 31.3% of respondents reported complaining or upset people (note that this doesn't mean 31.3% of villagers were upset, merely that 31.3% noticed someone who was complaining or upset). Most of these complaints appeared to be connected with people who felt they were eligible but not receiving transfers. There were 4 reports (0.8% of respondents) of shouting or angry arguments, 5 reports (1% of respondents) of conflict within the household, and 1 report (0.2%) of violence or crime, out of 458-463 total people responding.
For Siaya, 44% of respondents (70/159) reported complaining or upset people in the community. There were no reports of shouting or angry arguments, conflict within the household, violence or crime.
Note that GiveDirectly surveys only cash recipients, not non-recipients.
- Have there been any complaints or upset people in your community related to these transfers? Of the people who were complaining or upset, who were they upset with? Who has been complaining or upset about the transfers?
- Has there been any shouting or angry arguments among people in your village about these transfers? If yes, describe.
- Has there been any violence or crime in your village related to these transfers? If yes, describe.
- Do grants distort incentives and decisionmaking? We have not seen information on the question of whether individuals who live in the areas served by GiveDirectly change their behavior in order to increase their chances of receiving transfers - for example, by spending more time at home to increase their chances of being at home when GiveDirectly staff visit, or by choosing to live in poorer quality housing in hopes of receiving transfers. The one-off nature of transfers (recipients are not eligible for a second round of transfers) may help to mitigate these effects among past and current recipients. We also have seen some very limited evidence (mostly pointing to no distortion) in the broader literature on cash transfers.
Another way in which grants may distort decisionmaking is if they are promised and not delivered in time (causing people to make plans that cannot be executed). We do not have data directly addressing this issue, but GiveDirectly did provide statistics on the speed with which transfers were received for Rarieda and Siaya. GiveDirectly told us that the key factor determining when a recipients receives funds is when he or she registers for M-PESA, which is something the recipient controls. In Rarieda, while 67% (359 of 536) of recipients waited less than a month and 84% (448 of 536) waited 3 months or less, 6% (34 of 536) waited 6 months or more. In Siaya (a later group), 188 of 193 recipients waited less than a month, and the remaining 5 waited 2-3 months. GiveDirectly comments:
We were able to accelerate [the time it took for recipients to register for M-PESA] significantly for two reasons: (a) we gave clearer instructions, and (b) we let recipients designate which household member they wanted to receive the transfers, which gives them flexibility to choose someone who already has a National ID; in the Rarieda round we could not do this as we were randomizing recipient gender. I expect the Nike cohort will take longer to register as that project focuses on 18-19 year old women, many of whom will not yet have IDs.
- Do grants distort local markets? It seems possible to us that a large infusion of cash into an area could alter economic opportunities for both recipients and non-recipients. Such effects could be positive (for example by spurring investment and job creation or by increasing the availability of retail goods) or negative (for example, by leading primarily to local inflation). We do not currently have information to address this issue (aside from some very limited evidence, pointing to no distortion, in the broader literature on cash transfers), but GiveDirectly's future evaluation plans (discussed below) may include a study that would provide information on the matter.
We also spoke with recipients and non-recipients during our site visit to GiveDirectly's operations in Kenya in November 2012. For more, see our site visit notes
Do the funds GiveDirectly transfers fully reach recipients?
GiveDirectly transfers funds to recipients through M-PESA, a system that allows users to receive, send, deposit, and withdraw funds on their mobile phones. Recipients must present ID along with their mobile phone number and user-specified M-PESA pin code minimizing the ability of agents to defraud clients of funds.
Nonetheless, Lydia Tala, the GiveDirectly staff member responsible for making post-transfer phone calls to recipients, reports that one of the most common client complaints is the belief that M-PESA agents are overcharging or stealing funds. Ms. Tala believes that these reports are incorrect and that approximately 10% of recipients are not fully aware of how to use M-PESA, withdrawing funds without checking their balance and ultimately being surprised when they have drawn down their account. GiveDirectly told us that it recognizes this issue and tries to provide these recipients with assistance to help them navigate the M-PESA system.
Does GiveDirectly divert skilled labor away from other areas?
GiveDirectly's field staff consists of a Field Assistant to the COO who has been employed fairly continuously since GiveDirectly distributed its Siaya I round of transfers in mid-2012, and sets of census takers, enrollers, and back checkers who gather information on a household-by-household basis.
GiveDirectly reports that it receives approximately 10x the number of resumes as open positions, and that it hires one out of every three candidates it interviews. Successful candidates generally have a college education. They are paid approximately $8 per day in addition to expenses for travel and lodging while working.
Are the size and structure of the cash transfers well-thought-through and appropriate?
GiveDirectly's current model is to make transfers of $1000 per household over 9 months, after which point recipients become ineligible. It gives the following rationale for the size of its transfers:
GiveDirectly sends each recipient household $1,000, or $200 per person for an average household. These payments are spaced out in time to respect limits imposed by the M-PESA system and to give recipients time to plan for them, but should be thought of as wealth and not income transfers. GiveDirectly sized transfers at this level to ensure that they are fair, well-understood,
and potentially transformative
- Fair. Transfers are calibrated to be large enough to enable eligible households to raise their incomes to the level of their least well off but ineligible neighbors. This calculation was made using baseline data from our ongoing impact evaluation and assuming a 25% annual rate of return. (Our estimate of the return on capital was triangulated using average micro-credit loan charges, academic studies on the returns to capital in developing countries and interviews with recipients.) Calculations based on equalizing net worth, as opposed to income streams, led us to a similar ballpark figure. [GiveDirectly further notes, "'fair' is a subjective concept and we are not arguing for a particular concept of 'fairness' per se but rather that we think many would consider it 'unfair' to transfer so much to eligible [households] that we re-order the wealth distribution. We do not make the claim that non-recipients or particular donors agree that any particular transfer policy is fair."]
- Well-understood. Transfers are sized to be within the range of transfers issued by other well-studied cash transfer programs. Examples of transfer sizes from other well-known programs include
If anything we would lean towards transferring more than these programs do, since they serve people starting from a higher level of wealth.
- $406 per household per year for participants in Progressa / Opportunidades, and up to $4,059 in total over ten years.
- $524 per household per year for participants in Bolsa Familia (Brazil) in 2011, and a maximum of $7,855 in total over five years.
- Potentially transformative. Because cash transfers are flexible by design there are a number of relevant ways to think about what they could do for a recipient.
- If invested at a 25% real rate of return, the transfer would allow the average recipient to permanently increase his/her annual consumption by $0.14 over a baseline level of $0.65, a 22% increase.
- The transfer is enough to purchase
- 5.5 years of secondary schooling (estimated returns on a year of education for rural Kenya are around 15%)
- 5.2 years of basic food requirements for one adult.
- 1.2 acres of land, which is 1.8 times average baseline landholdings among eligible households.
- Tin roofs for 4 houses (estimated financial rate-of-return: 17%, not including health and comfort benefits.)
We have reservations about the above reasoning:
- Regarding "fair." Pre-cash-transfer wealth/income differences between eligible and ineligible recipients may exist for a number of reasons; we don't believe it's warranted to assume that a "fair" world would see the two groups with the same wealth/income due to an equalizing transfer, and more to the point, we don't believe that the ineligible households are likely to see the situation as "fair." In addition, we are concerned that by aiming to equalize eligible and ineligible households, GiveDirectly takes on a substantial risk of its calculations being off in a way that leads to eligible households becoming systematically better off than ineligible households, which could distort incentives and lead to conflict.
Strong evidence that the cash transfers do not lead to conflict and jealousy would reduce the weight we put on this concern, though as discussed above, it currently appears that GiveDirectly's cash transfers are leading to a fair amount of tension.
- Regarding "well-understood." GiveDirectly notes that its transfers are similar - in dollar terms - to those of government programs, but that they are likely much larger in "percentage of income" terms. We note that the cash transfer programs that have been studied to date seem to be in the range of 9-27% of recipients' annual consumption; by contrast, if GiveDirectly's clients average $0.65 in daily per capita consumption and receive an average of $288 per person over the course of a year (see above), this implies that people receive an average of 121% of their annual consumption in the year in which they receive the transfer. The quote above states that the lower level of initial income is an argument for making the cash transfer larger, but to us, it means that the risks of distorting incentives, causing conflict, etc. are likely to be greater than those of previously-studied programs, since the transfers are a substantially greater percentage of consumption. This issue is somewhat mitigated by the fact that GiveDirectly's transfers are designed as "wealth transfers" rather than as "income transfers": recipients receive funds over the course of a year and then become ineligible, whereas the government programs it alludes to have longer periods of eligibility. GiveDirectly has also told us that its decision to make larger transfers over a shorter period of time is based on recipients' reported preferences. But psychologically, the impact of GiveDirectly's transfers may be very different from those of government programs.
Our reservations about GiveDirectly's targeting strategy
We have reservations about the approach of targeting people based on the materials their houses are made of.
GiveDirectly is currently conducting a randomized controlled trial
in which eligible households are selected by lottery to receive cash transfers. These transfers were made in Rarieda (more information above) in 2011-12. It has publicly provided its plan for collecting and analyzing data
to determine the impact of these transfers. We have already reviewed some data from this study, and hope that the endline data will shed further light on questions about potential impacts - both positive and negative - of cash transfers, as well as helping to identify the most promising directions for future research and data collection.
GiveDirectly also began another set of transfers that will be evaluated in a randomized controlled trial. These transfers target young women aged 18-19 and are being funded by the Nike Foundation. GiveDirectly has shared the survey instrument it plans to use with us. We are also hoping that GiveDirectly will share an analysis plan, as it did for its other ongoing study.
GiveDirectly is also potentially interested in conducting an additional study in which it would randomize villages
to receive cash transfers, deliver cash transfers to all households in a chosen village
, and then look for village-level effects (such as potential inflation, discussed above).
What do you get for your dollar?
What percentage of GiveDirectly's expenses end up in the hands of recipients?
GiveDirectly aims to deliver at least 90 cents directly to recipients for every $1.00 in total expenses.
As discussed above
, data from GiveDirectly's distributions to date imply that it has been hitting this target. In addition, its unaudited financials for its most recent completed fiscal year (September 2011 through August 2012) show $339,620 in direct grants to households out of $379,240 in total expenses, for a ratio of ~90%.
Note that most of the costs of GiveDirectly's ongoing study
are excluded from its budget. GiveDirectly has estimated that the experiment budget has covered $4,000 worth of costs that GiveDirectly would have paid in the absence of the RCT. This $4,000 is included in the financials referred to in the previous paragraph.
Also excluded from GiveDirectly's costs are its unpaid CEO's time, which he estimates at 20 hours per week as well as approximately $30,000 spent by its board of directors on IT, outreach and legal expenses. Other board members have done some minor work (e.g., reviewing financial documents) but in our view this appears to be similar to the type of work carried out for most non-profits by their boards of directors.
Does GiveDirectly offer a large amount of humanitarian impact per dollar?
We have not conducted a cost-effectiveness analysis that attempts to quantify the benefits of cash transfers in humanitarian terms. Instead, in comparing cash transfers to the interventions conducted by our two other top charities
, we have attempted to monetize some of the benefits of the latter, in particular the “developmental effects” of deworming
. (In the case of the comparison with bednets, for instance, this means quantifying the estimated impact of bednets on later-in-life income of children, through a comparison with the effects of deworming, and then subjectively comparing the cost per life saved with the value of that amount of money as a cash transfer.)
In practice, these calculations are highly sensitive to assumptions, especially regarding:
- the investment returns to cash transfers;
- how much confidence one places in the developmental impacts of deworming based on limited evidence; and
- the subjective assessment of the relative value of averting child mortality and improving incomes.
We guess that in purely programmatic terms, and given our values, bednet distributions are more cost-effective than deworming, which is more cost-effective than cash transfers. However, we think there are plausible values for these assumptions that would permit any ordering of the three programs.
Details of our cost-effectiveness analysis are discussed in a 2012 blog post
. The general picture is that deworming appears to be between 2 and 5 times as cost-effective as cash-transfers, in financial terms. We encourage readers who find formal cost-effectiveness analysis important to examine the details of our calculations and assumptions, and to try putting in their own. To the extent that we have intuitive preferences and biases, these could easily be creeping into the assumption- and judgment-call-laden work we’ve done in generating our cost-effectiveness figures, and we’re not entirely confident that the figures themselves are adding substantial information beyond the intuitions we have from examining the details of them.
More, including links to our spreadsheets, at our 2012 discussion of the cost-effectiveness of cash transfers and other interventions
Room for more funding
GiveDirectly provided the following statement regarding its room for more funding
Our target for additional revenue in FY 2013 is at least $2.85M and at most $6.85M.
We think about the first number as follows. With our current staff (one COO) we have, conservatively estimated, the capacity to move $5M per year to new recipients. We also wish to increase transfer commitments to current recipients by a total of $250K. Against this total capacity of $5.25M we have already received a $2.4M grant, leaving room for $2.85M.
As part of the grant agreement we expect to hire a second full-time field manager. This manager will ultimately provide an additional $5M / year in throughput capacity but will first participate in a two-month apprenticeship, leaving time to manage $4M during his/her first year. This yields a total capacity of $6.85M.
To move amounts larger than this we would hire additional full-time field managers. We are open to this in principle but would want a frank conversation with interested donors about the costs and benefits of scaling up at that pace, as opposed to staging donations over multiple years.
Note that the above incorporates the fact that GiveDirectly has recently received a $2.4 million grant (half of which it is now working to distribute as part of the Siaya II transfer round).
While these funding goals constitute a significant expansion in its capacity, and that there exists some risk that as a relatively young organization GiveDirectly will struggle to transfer these funds, we believe that the risk does not seem too large and the benefit of enabling GiveDirectly to expand quickly is significant.
As of November 2012, GiveDirectly is in the process of enrolling 1,000 households to which it will transfer $1 million. Once this round is complete, we would expect GiveDirectly to be able to double its capacity for transfers and be in a position to move ~$2 million in early 2013. Assuming that goes well, GiveDirectly would be in a position to double again (~$4 million) in another transfer round in mid-to-late 2013. Because the vast majority of donors give near the end of the calendar year, we believe it is reasonable for donors to provide GiveDirectly with the funds it needs to maximize its potential for transfers in 2013.
Given that the primary capacity needs GiveDirectly has are surveyors to conduct the census, enroll recipients and back check eligibility and that GiveDirectly has successfully hired these employees in the past, we would guess that they would not struggle to hire many of these types of employees in the future. We also have the informal impression (from our site visit) that these types of employees are hired in greater quantities by Innovations for Poverty Action
, which GiveDirectly has worked with in Kenya, so presumably there is sufficient supply of the types of employees GiveDirectly would hire to scale up
In our view, the biggest risk is that GiveDirectly is unable to find additional capacity at the executive-level to manage the expansion. Currently, GiveDirectly employs only one executive in the field, and adding capacity at this position might prove difficult.
Flexible vs. typical funds
GiveDirectly has stated to us that by default, it intends to use all funds for cash transfers along the same basic lines (in terms of size, structure and targeting) as its past programs, aiming to spend 90%+ of such donations directly on transfers. However, it has also stated to us that it is interested in making cash transfers in ways that differ from earlier protocols in order to continue experimenting and learning, such as (a) experimenting with different structures or different targeting approaches (including giving transfers to all members of a village, rather than targeting specific people within a village); (b) conducting more intensive studies, such as the one discussed above
Accordingly, on our "Donate" page, we offer donors the option to have their funds be treated as "typical" donations (in which case the "default" approach applies) or as "flexible" donations (in which case GiveDirectly may use funds for other purposes, such as those discussed above). Consistent with our general policy of recommending unrestricted support, we recommend the latter and have marked it as the default option for our website.
GiveDirectly as an organization
We believe GiveDirectly to be an exceptionally strong and effective organization:
- Self-evaluation: GiveDirectly has invested heavily in self-evaluation from the start, and furthermore it has pre-registered its study design for additional accountability and credibility.
- Track record: Although it is very young, we feel that GiveDirectly's first year went well: GiveDirectly successfully accomplished its goal of transferring cash to extremely low-income people at a 90% ratio.
- Communication: GiveDirectly has always communicated extremely clearly and directly with us and given thoughtful answers to our critical questions. Generally, GiveDirectly seems to come to conclusions that we find reasonable on key questions.
- Transparency: GiveDirectly appears to value transparency as much as any organization we’ve encountered. We have not seen it hesitate to share information publicly (unless it had what we consider a good reason).
However, we continue to see potential room for improvement, particularly related to the fact that Paul Niehaus, the CEO, works only part-time (unpaid) for GiveDirectly. (GiveDirectly's COO is full-time.) It has been argued to us that having a part-time CEO is acceptable given the relative simplicity of the intervention GiveDirectly is carrying out, and that it has advantages since it creates a situation where the CEO's job security is not tied to the continuity of the organization. Despite these arguments, on balance we think it is a negative that the CEO is only part-time. We think that there is substantial potential for GiveDirectly to experiment with different approaches to its intervention and to engage in public conversation around its work, and that the CEO's undivided attention would be beneficial in those areas.
More on how we think about evaluating organizations at our 2012 blog post
- Diwan, Faizan. Innovations for Poverty Project Associate. Conversation with GiveWell, November 8, 2012.
- GiveDirectly. Analysis plan for randomized controlled trial (PPT).
- GiveDirectly. Annotated budget projections (May 11, 2012) (PDF).
- GiveDirectly. Audit log (XLS).
- GiveDirectly. Back check instrument (PDF).
- GiveDirectly. Balance sheet. GiveDirectly asked us to keep this document confidential temporarily.
- GiveDirectly. Board spending breakdown (PDF).
- GiveDirectly. Capacity note (DOC).
- GiveDirectly. Clarifications on GiveWell's draft review of GiveDirectly (DOC).
- GiveDirectly. Contextualizing transfer size (DOC).
- GiveDirectly. Enrollment speed of distributions (PDF).
- GiveDirectly. FAQs. http://givedirectly.org/faqs.php (accessed January 12, 2012). Archived by WebCite® at http://www.webcitation.org/64e05vHkD.
- GiveDirectly. Financials. http://givedirectly.org/financials.php (accessed January 12, 2012). Archived by WebCite® at http://www.webcitation.org/64e07SFAl.
- GiveDirectly. GiveWell clarifications (March 28, 2012) (DOC).
- GiveDirectly. Income and expense. GiveDirectly asked us to keep this document confidential temporarily.
- GiveDirectly. IRS form 990 (2010) (PDF).
- GiveDirectly. M-PESA transfer history (XLSX).
- GiveDirectly. Name collection instrument (PDF).
- GiveDirectly. Nike instrument (PDF).
- GiveDirectly. Notes on verification data (November 17, 2011) (PDF).
- GiveDirectly. Offering memorandum (January 2012). GiveDirectly asked us to keep this document confidential.
- GiveDirectly. Notes to financial statements. GiveDirectly asked us to keep this document confidential temporarily.
- GiveDirectly. Operating efficiency as of 31 August 2012 (PDF).
- GiveDirectly. Operational process overview (PDF).
- GiveDirectly. Overview of targeting process (PDF).
- GiveDirectly. Overview of update documents (PDF).
- GiveDirectly. Post-transfer audit (PDF).
- GiveDirectly. Rarieda verification. GiveDirectly asked us to keep this document confidential temporarily.
- GiveDirectly. Rarieda verification stats (PDF).
- GiveDirectly. Room for more funding summary (PDF).
- GiveDirectly. RCT mid-line results overview. GiveDirectly asked us to keep this document confidential.
- GiveDirectly. RCT midline variables. GiveDirectly asked us to keep this document confidential.
- GiveDirectly. Siaya I enrollment database (XLS).
- GiveDirectly. Siaya II transfer schedule (PDF).
- GiveDirectly. Siaya poverty data by location (PDF).
- GiveDirectly. Siaya verification. GiveDirectly asked us to keep this document confidential temporarily.
- GiveDirectly. Siaya verification stats (PDF).
- GiveDirectly. Siaya village index (XLSX).
- GiveDirectly. Survey for randomized controlled trial (XLS).
- GiveDirectly. Team. http://givedirectly.org/team.php (accessed on January 12, 2012). Archived by WebCite® at http://www.webcitation.org/64e1DISuU.
- GiveDirectly. Updated data (March 31, 2012) (XLS).
- GiveDirectly. Values. http://www.givedirectly.org/values.php (accessed November 19, 2012). Archived by WebCite® at http://www.webcitation.org/6CIH90bWJ.
- GiveDirectly. Verification data (November 17, 2011) (XLS).
- GiveDirectly. Verification template (November 7, 2011) (DOC).
- GiveDirectly. Verification template (October 1, 2012) (PDF).
- GiveDirectly. Village targeting regression (PDF).
- GiveWell. Household size analysis (XLS).
- GiveWell. Site visit notes (DOC).
- Mukhopadhyay, Piali. GiveDirectly COO. Email to GiveWell, January 25, 2012.
- Mukhopadhyay, Piali. GiveDirectly COO. Conversation with GiveWell, November 7, 2012.
- Mukhopadhyay, Piali. GiveDirectly COO. Conversation with GiveWell, November 8, 2012.
- Mukhopadhyay, Piali. GiveDirectly COO. Email to GiveWell, November 23, 2012.
- Niehaus, Paul, and Jeremy Shapiro. GiveDirectly Founders. Phone conversation with GiveWell, October 6, 2012 (DOC).
- Niehaus, Paul. GiveDirectly Founder. Conversation with GiveWell, April 1, 2012.
- Niehaus, Paul. GiveDirectly Founder. Conversation with GiveWell, October 22 2012.
- Niehaus, Paul. GiveDirectly Founder. Conversation with GiveWell, November 5, 2012.
- Niehaus, Paul. GiveDirectly Founder. Email to GiveWell, April 23, 2012.
- Niehaus, Paul. GiveDirectly Founder. Email to GiveWell, November 20, 2012.
- Niehaus, Paul. GiveDirectly Founder. Email to GiveWell, November 24, 2012.
- Tala, Lydia. GiveDirectly Field Assistant. Conversation with GiveWell, November 6, 2012.
- Tala, Lydia. GiveDirectly Field Assistant. Conversation with GiveWell, November 7, 2012.