One Acre Fund — Tree Program RCT and Data Collection (January 2023)

Note: This page summarizes the rationale behind a GiveWell grant to One Acre Fund. One Acre Fund staff reviewed this page prior to publication.

Summary

In January 2023, GiveWell made a $1.3 million grant to One Acre Fund to implement a randomized controlled trial (RCT) of, and collect additional data related to, its tree program in Rwanda. The grant was funded by donations to the All Grants Fund.

Our primary reasons for recommending this grant include:

  • We think that One Acre Fund’s tree program could meet GiveWell’s cost-effectiveness threshold in certain contexts, but we have a large degree of uncertainty about a number of drivers of cost-effectiveness. This grant could address some of our uncertainties by estimating the causal effect of One Acre Fund’s tree program on the number of seedlings planted and surviving past the first dry season.
  • It could also allow us to benchmark the estimates from One Acre Fund’s routine monitoring and evaluation to those estimated by an external evaluator. This would allow us to have more confidence when generalizing any results to other contexts.
  • Updates on the impact of One Acre Fund’s tree program could change our view on the overall cost-effectiveness of the program, and in turn, affect our funding decisions regarding this and similar programs in the future. Given the relatively low costs of this grant and the resulting data’s potential to influence the allocation of a moderate amount of funding in the future, we estimate this is a cost-effective use of funding.

Our main reservations are:

  • We will have substantial uncertainty about the benefits of One Acre Fund’s tree program even after receiving results from this study because of the long time period over which trees are expected to grow and be used.
  • If One Acre Fund changes its routine monitoring methodology in the future, it may no longer be possible to benchmark results from monitoring to the results from the RCT. We think this could occur as One Acre Fund continues to improve its methodologies and modeling assumptions. This could mean that we may lose the value of benchmarking the results.

GiveWell spoke with staff from the One Acre Fund on July 11, 2023, about updates on this grant.

Published: June 2023

Table of Contents

The organization

One Acre Fund is a non-profit which provides agricultural services and products to smallholder farmers.1 It currently operates in nine countries (Burundi, Ethiopia, Kenya, Malawi, Nigeria, Rwanda, Tanzania, Uganda, and Zambia) and the bundle of services it provides varies from country to country.2 In September 2022 GiveWell recommended a $77,523 grant to support the planning of data collection activities to evaluate One Acre Fund’s tree program.

The intervention

One Acre Fund’s tree program aims to encourage farmers to grow more trees by providing seedlings and training on growing practices.3 In low-income countries, employment is concentrated in the agricultural sector,4 and the poorest households are more likely than others to work in agriculture, according to data compiled and reported by the World Bank.5 Timber trees may generate income for households who can sell timber products,6 so tree planting may have the potential to raise incomes among poor households.

There is evidence from one RCT in Kenya that One Acre Fund’s tree program increases the number of trees planted after one year.7 This study tested a “tree kit” model,8 as opposed to One Acre Fund’s current “seedlings” model,9 and we are unsure how representative the context studied in this RCT was of One Acre Fund’s typical programs.10 We are not aware of any direct causal evidence showing that additional trees increase farmers’ profits and household income over the long term. Given the lack of high-quality direct evidence on farmers’ incomes, we are highly uncertain about the evidence for effectiveness, and we rely on projections from One Acre Fund, which estimate increases in farmer income over the long term based on increases seen in the number of trees planted. See our report on One Acre Fund's tree program for our review of the program and its evidence.

We estimate that in most contexts this program is slightly below or around the level of cost-effectiveness of programs we expect to direct marginal donations to (which is ten or more times as cost-effective as unconditional cash transfers as of April 2023).11 However, we think that our bottom line cost-effectiveness estimate could change if we get more information. See more about our key uncertainties here.

The grant

GiveWell is making a grant of approximately $1.3 million to One Acre Fund to conduct an RCT of its tree program in Rwanda and collect additional monitoring data. The grant would fund tree planting at the end of 2023, planting surveys in early 2024, and endline surveys at the end of 2024, with the final report expected to be completed in early 2025. One Acre Fund has contracted an external evaluator—Laterite—to conduct the RCT. Laterite is an East African firm that specializes in research for social impact.12

Collection of visual tree count data is planned to measure the following primary outcomes:13

  • Tree survival rate
  • Total trees planted
  • Incremental trees
  • Substitution effect (by comparing the total number/value of trees planted by species)

Additional survey data would focus on the following parameters:14

  • Farmer use of trees (the share of farmers using trees for particular uses, estimated units, prices, and volumes of various tree products that they have recently sold, wait period between harvests, etc.)
  • Farmer costs per year (input costs, time and labor costs, and the opportunity cost of time and land use)
  • Tree prices (the value of tree products, by age and species of tree).

Laterite will use these data to estimate the possible impact of One Acre Fund’s program on farmers’ incomes from trees.

Other than direct RCT and survey design and implementation costs, the grant will also cover additional monitoring by One Acre Fund, to ensure that its routine data collection will be able to be benchmarked to the findings from the RCT.15

Budget for grant activities

The budget for the grant is $1,333,388, broken down as follows:16

  • $352,162 for One Acre Fund, of which
    • $143,795 is for project leadership, support costs, internal evaluation for benchmarking, and office-based mapping
    • $74,012 is for field-based activities (including management, scouting, additional trees distributed, transportation for incentive delivery, and planting and survival surveys)
    • $134,355 is for field incentives, to compensate control group farmers, nursery operators and field promoters
  • $981,226 for Laterite which covers research and survey costs.

The case for the grant

We are recommending this grant for the following reasons:

  • We expect that the grant will allow us to learn more about the cost-effectiveness of One Acre Fund’s tree program, which may influence our funding decisions. Updates on the impact of One Acre Fund’s tree program could change our view on the overall cost-effectiveness of the program, and in turn, affect our funding decisions regarding this and similar programs in the future. Given the relatively low costs of this grant and the resulting data’s potential to influence the allocation of a moderate amount of funding in the future, we estimate this is a cost-effective use of funding. More below.
  • We expect the grant will provide GiveWell with an opportunity to work on and learn more about income-generating programs in general, which may put GiveWell in a better position to evaluate income-generating programs in the future. The majority of grants GiveWell investigates and recommends are related to health programs. We believe that there is value in deepening our knowledge of income-generating programs through working on programs of this type, because it may allow us to improve how we evaluate income-generating programs in the future.
  • We have a strong qualitative impression of the One Acre Fund team. We have been impressed with One Acre Fund’s value alignment and detailed responses during discussions. It has also been transparent with us about potential limitations of its program and factors that could worsen cost-effectiveness.

Improving our cost-effectiveness estimate for the tree program

Our best guess is that One Acre Fund’s tree program (assuming at-scale costs) in Rwanda and Burundi is around 10 times as cost-effective as unconditional cash transfers,17 and that together these two countries could absorb around $5 million per year. We also think that the tree program in Kenya is around 9 times as cost-effective as unconditional cash transfers, and could absorb almost $6 million per year.18 However, we have substantial uncertainty about these cost-effectiveness estimates because they are based on non-causal data from One Acre Fund’s routine monitoring and evaluation.19

This grant could address some of our uncertainties by generating a causal estimate of the effect of One Acre Fund’s tree program on the number of seedlings planted and surviving past the first dry season. It could also allow us to benchmark the estimates from One Acre Fund’s routine monitoring and evaluation to those estimated by an external evaluator in an RCT.20 We will compare the RCT results (for parameters including the number of surviving seedlings, tree usage, wood product prices, and opportunity costs of growing trees) to the non-causal estimates generated by One Acre Fund’s routine monitoring practices in the same geographic area.

We think this learning could cause us to update our best guess of the cost-effectiveness of the tree program either up or down. We roughly estimate the value of information from this grant to be around 22 times as cost-effective as unconditional cash transfers.21 If this work shows high alignment between One Acre Fund’s internal monitoring and those of an external evaluator, we would also have more confidence in One Acre Fund’s internal monitoring in the future.

Risks and reservations

We have the following primary reservations about this grant:

  • We will continue to have substantial uncertainty about program impacts over the time horizon of the modeled benefits. Trees grow and are used over long time horizons (we model a 15 year timespan). We have substantial uncertainty about how medium and long-term trends (such as technological innovation, urbanization, climate change, etc.) might affect the cost-effectiveness of this program. We have not explicitly incorporated potential long-term trends in our cost-effectiveness model. Rethink Priorities investigated this uncertainty, but the possible long-term impact of the tree program remains speculative.22 This grant will not reduce our long-term uncertainties about the program.
  • It may not be possible to benchmark the RCT results to One Acre Fund’s internal M&E results if it changes its methodology in the future. It seems possible that One Acre Fund’s monitoring methodologies and modeling assumptions may change over time. This means that we may decide in future that we are not confident in our conclusion of how One Acre Fund’s monitoring results compare to the RCT results (and therefore may lose the value of benchmarking the results).

We also have reservations about the following issues, though we consider them less important than those above:

  • Intra-organizational funging may reduce the cost-effectiveness of any future GiveWell funding to the tree program. It is possible that if GiveWell were to fund One Acre Fund’s tree program in the future, One Acre Fund could shift unrestricted funding it would otherwise have used to support the tree program to other related programs. If these other programs were less cost-effective, this could reduce the overall cost-effectiveness of GiveWell funding. We have already begun to communicate with One Acre Fund about GiveWell’s conceptualization of intra-organizational funging and plan to continue these conversations to try to mitigate this risk.
  • It is possible that One Acre Fund could have funded this study using unrestricted organizational funds. However, One Acre Fund has already commissioned an evaluation of its tree program,23 and we believe that GiveWell is the primary potential user of results of this study and data collection. We think these factors make it less likely that One Acre Fund would conduct similar data gathering without this grant.
  • This RCT requires foregoing impact for the control group farmers for the duration of the trial. One Acre Fund has high penetration throughout most of Rwanda, so the control group for the purposes of this trial would otherwise have been treated.24 The control group farmers will be compensated.25 We believe this is worth doing because the data that are generated may allow us to direct funding to support or expand the program.

Plans for follow up

We plan to have periodic check-ins with the One Acre Fund team during the planning phase of the RCT. We expect to receive a final report of the results of the RCT in early 2025, following completion of the endline survey in late 2024.26

Internal forecasts

For this grant, we are recording the following forecasts:

Confidence Prediction By time​​
70% We will think the program conducted in Rwanda is more than 8 times as cost-effective as cash transfers at the end of the grant. April 2025
70% We will think the program conducted in Burundi is more than 8 times as cost-effective as cash transfers at the end of the grant. April 2025
60% We will think the program conducted in Kenya is more than 8 times as cost-effective as cash transfers at the end of the grant. April 2025
55% We will think the program conducted in Rwanda is more than 10 times as cost-effective as cash transfers at the end of the grant. April 2025
55% We will think the program conducted in Burundi is more than 10 times as cost-effective as cash transfers at the end of the grant. April 2025
40% We will think the program conducted in Kenya is more than 10 times as cost-effective as cash transfers at the end of the grant. April 2025
20% We will think the program conducted in Rwanda is more than 12 times as cost-effective as cash transfers at the end of the grant. April 2025
20% We will think the program conducted in Burundi is more than 12 times as cost-effective as cash transfers at the end of the grant. April 2025
10% We will think the program conducted in Kenya is more than 12 times as cost-effective as cash transfers at the end of the grant. April 2025

Our process

We previously recommended a grant to One Acre Fund to support the planning of data collection activities to evaluate its tree program. As part of this work, One Acre Fund selected an external evaluator (Laterite) and the organizations jointly prepared a proposal for an RCT and additional monitoring activities. We reviewed this proposal.

We have had several calls and email exchanges with One Acre Fund to understand its tree program, routine monitoring and evaluation activities, and the proposed RCT.27

Using our cost-effectiveness analysis of One Acre Fund's tree program as a base, we created a value of information analysis of One Acre Fund’s proposed RCT in Rwanda.

Sources

Document Source
Castañeda et al. 2018 Source
GiveWell, "One Acre Fund – Planning Grant for Research on Tree Program (September 2022)" Source
GiveWell, "One Acre Fund Tree Program," 2022 Source
GiveWell, One Acre Fund seedlings BOTEC, 2022 Source
GiveWell, value of information analysis of One Acre Fund's Rwanda seedlings RCT grant [2023] Source
GiveWell's non-verbatim summary of a conversation with One Acre Fund, March 13, 2018 Source
GiveWell's non-verbatim summary of conversations with One Acre Fund on August 5, 2022; September 19, 2022; October 11, 2022; November 18, 2022; and January 20, 2023 Source
GiveWell’s Cost-Effectiveness Analyses webpage, 2023 Source
Laterite 2021 Source (archive)
Laterite, "About Laterite" Source (archive)
Laterite, "Evaluation of the One Acre Fund Rwanda tree program impact: Technical proposal prepared for One Acre Fund," 2022 Source
Laterite, One Acre Fund Tree Program: Midline evaluation report, 2020 Source
Miller et al. 2020 Source (archive)
One Acre Fund, "About Us" Source (archive)
One Acre Fund, "Countries we serve" Source (archive)
One Acre Fund, "Training" Source (archive)
One Acre Fund, Cultivating New Frontiers: 2021 Annual Report Source (archive)
One Acre Fund, Summary of 2019-2021 tree seedling randomized trial Source (archive)
Rethink Priorities, "Wood Commodities," 2022 Source
World Bank, "Employment in agriculture as percentage of total employment in low-income countries, International Labour Organization, ILOSTAT database," 2021 Source (archive)
  • 1

    “For smallholder farmers, the line between flourishing and struggling often comes down to small differences in planting technique and the quality of seeds and supplies. Providing access to better farm products — such as naturally-produced hybrid seed and a small amount of fertilizer — and education on farming techniques became the driving mission behind One Acre Fund.” One Acre Fund, "About Us"

  • 2

  • 3
    • “Tree-planting programs have been central to our impact strategy, and in 2021 we expanded our investment in them dramatically by supporting over 2 million farmers to plant more than 40 million trees, of which we project 20 million will reach maturity." One Acre Fund, Cultivating New Frontiers: 2021 Annual Report, p. 10.
    • "One Acre Fund’s team of field officers provides expert, tailored training directly to farmers throughout the year. This includes training on . . . tree planting." One Acre Fund, "Training"
    • "1AF also launched an agroforestry platform ten years ago, and as of 2021, supports 1.9 million farmers in planting ~40 million trees each year.

  • 4

    In 2019, 59% of employment in low-income countries was in agriculture, based on International Labour Organization estimates. World Bank, "Employment in agriculture as percentage of total employment in low-income countries, International Labour Organization, ILOSTAT database," 2021.

  • 5

    See Castañeda et al. 2018, p. 256, figure 4. We have not vetted the data presented in this figure or in the analysis as a whole.

  • 6

    “Trees can enable economic gains through sale and consumption of timber and non-timber products, as well as the environmental services they provide." Miller et al. 2020, p. 2.

  • 7
    • "We find that after one year of the 1AF Tree Program, farmers in the treatment group had on average 7 more grevillea trees and were 15.6 percentage points more likely to have planted grevillea trees compared to control group farmers." Laterite 2021, p. 5.
    • The intention-to-treat (ITT) effect for the number of all trees planted as a result of One Acre Fund’s program in the RCT population was 13.8 trees. See Laterite, One Acre Fund Tree Program: Midline evaluation report, 2020, p. 26, Table 13.

  • 8
    • "Farmers in the treatment training groups were scheduled to receive the “Tree Program”. This consists of tree kits, and tree training as part of their regular base package of inputs (improved seeds and fertilizer for maize and beans plus any add-on products) for the 2019 long rains season. Control training groups were scheduled to receive a base package that did not include the tree kits and tree-specific training." Laterite 2021, p. 9
    • “The Tree Program contains grevillea seeds, planting fertilizer, large planting the seeds, smaller planting bags for raising individual seedlings once they are big enough, and a set of trainings specifically on tree planting and maintenance.” Laterite 2021, p. 1.

  • 9

    "1AF also launched an agroforestry platform ten years ago, and as of 2021, supports 1.9 million farmers in planting ~40 million trees each year.

  • 10
    • "These findings [from the RCT] are substantially greater than 1AF’s internal findings for tree-planting in Kenya in 2019. 1AF believes the two districts in which the Laterite study took place are overall a more hospitable area for tree-planting (e.g., local market prices per tree are substantially higher) compared to the 40 districts in which 1AF’s internal measurement took place." One Acre Fund, Summary of 2019-2021 tree seedling randomized trial, p. 1.
    • Beginning in 2022, One Acre Fund’s preferred model is to provide seedlings (rather than seeds) to farmers, who will then receive training to transplant them into the ground: “Additionally, in focus group discussions, both farmers and 1AF field officers preferred a program model that offered tree seedlings (rather than tree seeds), as the former are easier to plant and more likely to survive. After considering all facets (farmer demand, impact, cost, etc.), 1AF has decided to shift efforts in Kenya towards a model in which 1AF grows (or contracts local entrepreneurs to grow) tree seedlings, and delivers those seedlings directly to farmers, which should increase both the adoption rate among farmers and overall number of trees planted per adopter, albeit with higher program cost.” One Acre Fund, Summary of 2019-2021 tree seedling randomized trial, p. 10.

  • 11
    • As of April 2023, we are primarily looking to recommend grants for funding opportunities that we estimate are ten or more times as cost-effective as GiveDirectly's unconditional cash transfer program.
    • The values provided by One Acre Fund that are used in our cost-effectiveness analysis are current as of October 2022, but are continuously being updated as One Acre Fund conducts monitoring and data collection.
    • Our cost-effectiveness analysis estimates that this program will range from 0.1 to 10.0 times as cost-effective as unconditional cash transfers, depending on the country modeled (and associated costs).
    • See our cost-effectiveness analysis of One Acre Fund’s tree program, "Cost-effectiveness analysis" tab, "Cost-effectiveness in multiples of cash transfers, after all adjustments" row.
    • We estimate that the program will be marginally more cost-effective when using estimated costs at scale. See our Value of information analysis of One Acre Fund's Rwanda seedlings RCT grant [2023], "Cost-effectiveness analysis at scale" tab, "Cost-effectiveness in multiples of cash transfers, after all adjustments" row.

  • 12

    “Laterite is an African firm specialized in research for social impact. We strive to carry out impactful research that helps decision-makers find solutions to complex development problems. Our approach is structured, data intensive and embedded in the local context.” Laterite, "About Laterite"

  • 13

  • 14

    See Table 5, p. 27: Laterite, "Evaluation of the One Acre Fund Rwanda tree program impact: Technical proposal prepared for One Acre Fund," 2022.

  • 15

    “1AF will modify its strategy for the tree planting and survival surveys to benchmark 1AF’s internal measurement results to Laterite’s RCT results with precision in the following manner:

    1. Sample sizes in the 6 RCT districts: 1AF typically follows 2000 farmers across all districts of the tree program that allows us to measure the impact of incremental trees surviving for 1AF adopters versus controls…For the year 2024 when the RCT will be conducted, 1AF will follow 3000 farmers specifically in the 6 RCT districts. This is in addition to the 2000 samples we require for the total program, where usually only 467 samples would have been covered in the 6 districts. This means that 1AF will increase its sample size by approximately 2533 farmers in the 6 districts in the internal evaluation specifically for the benchmarking exercise.
    2. Selection of Villages for Internal 1AF Study: A cell typically consists of 5-6 villages. For the RCT, Laterite will randomly pick 2 villages in each cell to draw their samples. To ensure we select farmers for the internal evaluation that are as geographically close to the RCT farmers, we will draw our sample from 2 of the remaining villages in the cell where the RCT will not survey…
    3. Samples per Cell/Village: In total, 1AF will select 5 cells per district which will be the exact same as the treatment cells in the RCT. In each cell, 1AF will select 2 villages for the survey. In each village, 1AF will randomly select 50 farmers for the planting and tree survival surveys using random walk selection across each village. Using our regular approach, we should expect some of these randomly selected farmers to be either 1AF tree adopters or non-adopters.” Laterite, "Evaluation of the One Acre Fund Rwanda tree program impact: Technical proposal prepared for One Acre Fund," 2022, p. 31.

  • 16

  • 17

    Note that a) our cost-effectiveness analyses are simplified models that are highly uncertain, and b) our cost-effectiveness threshold for directing funding to particular programs changes periodically. As of early 2023, our bar for directing funding is about 10 times as cost-effective as unconditional cash transfers. See GiveWell’s Cost-Effectiveness Analyses webpage for more information about how we use cost-effectiveness estimates in our grantmaking.

  • 18

    See our Value of information analysis of One Acre Fund's Rwanda seedlings RCT grant [2023], “Program costs and room for more funding” tab, "Summary for VOI - using "blue sky" costs at scale" section.

  • 19

    One Acre Fund models the effect of the tree program on farmers’ incomes by combining assumptions about how many marginal trees are planted, what farmers use or sell trees for, when farmers use or sell those trees, and at what price. These assumptions are based on data routinely collected by One Acre Fund. Although we do not have any explicit concerns from the modeling that we have seen, there is room for judgment in these assumptions, and we are unsure whether an external evaluator would come to the same conclusions as One Acre Fund. See our intervention report on One Acre Fund’s tree program for more information.

  • 20

    We expect that there may be some variance in the two sets of estimates because of differences in how the estimates are generated and increased counterfactual demand for trees in a context where One Acre Fund has worked for some time. For a discussion of these limitations, see section 9.2 “Important notes about the benchmarking exercise,” Laterite, "Evaluation of the One Acre Fund Rwanda tree program impact: Technical proposal prepared for One Acre Fund," 2022, p. 31.

  • 21

    See our Value of information analysis of One Acre Fund's Rwanda seedlings RCT grant [2023], “Updated VOI” tab, "CE of RCT" row.

  • 22

    “Our best guess (with low confidence) is that prices for high-quality wood commodities (e.g., timber) will increase more or less proportionately to rural incomes in the next 1-2 decades. We are moderately confident that prices for low-quality wood commodities [such as those used for cooking] will decline as incomes increase and technology improves, though we are unsure over what timeline.” Rethink Priorities, "Wood Commodities," 2022.

  • 23

    One Acre Fund contracted Laterite in 2018 to conduct an RCT of its tree program in Kenya. See Laterite 2021 for more information on the evaluation.

  • 24

    “Since the 1AF tree program already has nearly country-wide coverage, there are no suitable control areas outside of the program areas. Only regions that are particularly hard to reach or otherwise differ significantly from the existing program areas have not yet been reached by the program. Thus, we have decided to use cells from within the existing program areas as controls. To create a valid counterfactual the program will have to be paused for one year in control cells (see Table 1). With this design the impact estimates will capture the incremental effect of delivering the tree program for one additional year.” Laterite, "Evaluation of the One Acre Fund Rwanda tree program impact: Technical proposal prepared for One Acre Fund," 2022, p. 17.

  • 25

    “Since in control cells the registration will be a mock exercise, it will be necessary to compensate implementation partners, community nursery operators and farmers to maintain their goodwill and trust in 1AF. The compensation for implementation partners should be approximately equivalent to the forgone benefits, while farmers will be compensated by receiving the forgone trees as well as additional high value trees in next year’s seedling distribution.” Laterite, "Evaluation of the One Acre Fund Rwanda tree program impact: Technical proposal prepared for One Acre Fund," 2022, p. 15.

  • 26

    See "Work plan" figure, Laterite, "Evaluation of the One Acre Fund Rwanda tree program impact: Technical proposal prepared for One Acre Fund," 2022, p. 38.

  • 27

    GiveWell's non-verbatim summary of conversations with One Acre Fund on August 5, 2022; September 19, 2022; October 11, 2022; November 18, 2022; and January 20, 2023