You are here

Precision Agriculture for Development - November 2020 Version

We discontinued the "standout charity" designation

Precision Agriculture for Development (PAD) was designated a GiveWell standout charity, but we stopped publishing a list of standout charities in October 2021. More information is available in this blog post.

Standout charities were organizations that did not meet all of our criteria to be GiveWell top charities, but stood out from the vast majority of organizations we considered. However, we prioritized directing funding to our top charities. More information about standout charities is linked here.

We are no longer maintaining the review of PAD below.

Published: November 2020

Summary

What do they do? Precision Agriculture for Development (PAD) provides locally-customized agricultural advice to farmers through their mobile phones (e.g., recommendations to use specific farming practices or inputs, such as seeds and fertilizers, or answers to specific questions posed by farmers). (More)

Does it work? We have completed a preliminary review of the evidence on programs that are implemented by PAD or similar to those implemented by PAD and have relied primarily on a meta-analysis of three randomized controlled trials (RCTs), which finds that mobile-based agricultural advice leads to a 4% increase in farmers’ yields, though the estimated effect has a wide confidence interval. We are highly uncertain about the extent to which the findings will generalize to settings where PAD plans to operate in the future and whether PAD would be able to provide sufficient monitoring and evaluation data to reduce this uncertainty. Our limited understanding is that PAD has some experience with a variety of different monitoring tools and has not yet applied these tools comprehensively across its programs. (More)

What do you get for your dollar? We roughly estimate that it costs PAD $1-2 per year to reach a household with its messaging. Based on our initial cost-effectiveness model, our guess is that PAD's cost-effectiveness is below the range of programs we would consider recommending funding in the near future, but near enough to that range that this conclusion could change as PAD's work evolves, as we get additional information, or due to changes in the amount of highly cost-effective room for more funding that our top charities collectively have available. (More)

Is there room for more funding? PAD reports that it has significant levels of room for more funding to expand its existing services, support new programs that are similar to those studied in the existing evidence, and undertake research and product testing. (More)

PAD is one of our standout charities because of its:

  • Unusually strong self-analysis, particularly in supporting RCTs on its program.
  • Standout transparency. It has shared significant, detailed information about its program with us.
  • Work on a potentially cost-effective program. We have completed an initial cost-effectiveness analysis and provisionally concluded that PAD is less cost-effective than the programs we currently recommend that donors support, but not by a large margin. New information could plausibly lead us to believe that this program is as cost-effective as our top charities.

Major unresolved questions include:

  • To date, we have done a limited amount of work to model PAD's cost-effectiveness. Further work or new information could change our conclusion. One potential area for exploration is to focus on a particular form of the program or geographic location that PAD could expand to, in order to determine whether we would estimate this to have higher cost-effectiveness.
  • We have not conducted an in-depth review of the information PAD collects from its users on engagement, knowledge, and reported behavior change. If we were to consider funding PAD in the future, we would want to learn more about how programs funded with additional donations would be monitored.
  • We have not sought out a strong understanding of how PAD would spend additional funding. We have a general understanding that PAD could absorb more funding than it expects to receive.

Our review process

To date, our investigation process has consisted of:

  • Several conversations with PAD.1
  • Reviewing available evidence on mobile-based agricultural advice programs to develop our interim intervention report.
  • Reviewing documents PAD sent us after our conversations and in response to our queries.

What do they do?

PAD provides personalized agricultural advice to farmers through their mobile phones.2 PAD provides advice through a combination of short message service (SMS) messages (i.e., short, text-based messages) and automated phone calls.3 This advice includes recommending farmers use or not use specific agricultural inputs (e.g., seeds and fertilizers),4 providing decision-support tools to farmers (e.g., on how to deal with pest infestations or which seeds to select given the farmer's land use and agroclimatic zone),5 and answering specific questions posed by farmers.6

PAD’s programs vary in intensity and comprehensiveness across settings. “Light-touch” programs generally provide messages to farmers about a single topic. “High-touch” programs provide advice on a wider set of agronomic issues and—in addition to SMS and voice messages “pushed” to farmers in the "light-touch" model—may include providing a hotline that farmers can call to receive advice from agronomists, access market prices for agricultural commodities, and listen to questions asked by other farmers. Most of PAD's programs fall somewhere in between its "light-touch" and "high-touch" classifications.7

PAD’s general process when starting a new program is to:

  • source agricultural advice from agronomic experts on specific crops in the targeted country,8
  • gather farmers’ mobile phone numbers, generally from partner organizations,9
  • conduct focus groups with farmers to learn more about what content would be most helpful,10
  • put the advice in a format that is easy to understand,11
  • share advice with farmers via SMS or voice messages at intervals designed to coincide with key decisions points in a cropping cycle,12
  • test the effectiveness of variations of the messaging and format,13 and
  • conduct monitoring and evaluation activities to measure impact (more below).

In 2019, PAD’s programs reached an estimated 3.5 million farmers across eight countries: India, Kenya, Rwanda, Ethiopia, Pakistan, Uganda, Bangladesh, and Zambia.14 Approximately 55% of these farmers were reached by services that PAD operates for, or in partnership with, governments and other organizations. Another 40% of farmers were reached by services provided by organizations that PAD advises (in order to increase the impact of the existing services). For about 5% of the farmers it served in 2019, PAD operated the service independently.15

Breakdown of spending

PAD’s expenses were $3.9 million in 2019.16 In 2020, it plans to spend 42% of its budget on program staff, 20% on research staff, 20% on global overhead, 13% on operational costs other than staff, 4% on telecommunications, and 1% on user acquisition.17

Does it work?

What is the evidence that the program leads to higher incomes for farmers?

We have completed an interim intervention report in which we discuss the available evidence on the effectiveness of PAD’s program. We have spent limited time to form the initial views presented in our interim intervention report and, at this point, our views are preliminary. Here we briefly summarize the evidence we considered as part of our interim intervention report.

A meta-analysis by Fabregas, Kremer, and Schilbach 2019 of three studies of programs implemented by PAD and similar to those implemented by PAD18 finds that mobile-based agricultural advice leads to a 4% increase in farmers’ yields, though the estimated effect has a wide confidence interval (the range of plausible values).19 We are highly uncertain about the extent to which these findings will generalize to settings where PAD plans to operate in the future and whether PAD would be able to provide sufficient monitoring and evaluation data to reduce this uncertainty.

We view Fabregas, Kremer, and Schilbach 2019 as providing moderate quality evidence for the effect of programs like those implemented by PAD on farmer yields, though we have not conducted a thorough review of the meta-analysis or the studies included. We have a skeptical view that mobile-based agricultural advice programs have a limited effect on farmer behavior and income; overall, we view the evidence from Fabregas, Kremer, and Schilbach 2019 as a modest positive update on this view.

How does PAD monitor its programs on an ongoing basis?

To date, we have done a limited review of the monitoring and evaluation activities PAD uses to test the effectiveness of its programs. We haven't yet sought the level of understanding of PAD's monitoring that we have for our top charities. Our understanding based on our review so far is that PAD has some experience with a variety of different monitoring tools but has not yet applied these tools comprehensively across its programs.20 Based on this, if we were to consider funding PAD in the future, we would want to learn more about how programs funded with additional donations would be monitored.

PAD provided a written summary of its monitoring and evaluation activities and also described its monitoring and evaluation activities in two conversations with GiveWell.21 Here we summarize our current understanding of the monitoring and evaluation activities to measure each of the following:22

  1. User engagement with PAD's services (e.g. picking up and listening to calls, reading SMS messages). PAD shared data from programs in five states in India on the percentage of calls users picked up, the average proportion of calls listened to (calculated as the duration of the call that the farmer listened to divided by the total message length), the percentage of calls users rated, and what the average rating was. It also shared data on inbound calls from farmers to PAD's hotline (e.g. number of calls received, average call duration). This data, which PAD shared in September 2020, covers December 2019 to August 2020.23 We have not yet asked PAD for details of how this data was collected and analyzed. For other programs, our understanding is that PAD has not collected data regularly on user engagement.24
  2. Improved knowledge and comprehension of recommended agronomic practices and self-reported behavior change. PAD told us that it has surveyed farmers to measure knowledge, comprehension, and reported behavior change in select programs: on an ongoing basis in one of its programs in Kenya,25 in all of its programs in India,26 and in 2018 in Rwanda.27 It has shared select results from its surveys in Kenya, Rwanda, and India and a training guide for its surveys in India.28 As of September 2020, it was analyzing the first survey data from India that was based on this guide.29 We have not yet asked PAD for more detail on the survey data it has collected. For other programs, our understanding is that PAD has not surveyed its users.30
  3. Beneficial outcomes for farmers, including increases in yields and net income. PAD has reported results on agricultural yield for its program in Gujarat based on an RCT that was conducted there.31 It expects to collect data on yields and/or net income in the future through an RCT in Odisha,32 and through administrative data on milk production in Kenya.33

What do you get for your dollar?

We estimate that the cost per farmer of PAD’s programs is between $1 and $2 per year.34 Note that our cost-effectiveness analyses are simplified models that do not take into account a number of factors. There are limitations to this kind of cost-effectiveness analysis, and we believe that cost-effectiveness estimates such as these should not be taken literally, due to the significant uncertainty around them. We provide these estimates (a) for comparative purposes and (b) because working on them helps us ensure that we are thinking through as many of the relevant issues as possible.

More details on this estimate and on our preliminary cost-effectiveness analysis for PAD are in our interim intervention report.

Is there room for more funding?

In early 2020, PAD told us that if it had additional funding, it would be able to increase the number of farmers it reaches substantially. It estimated that it could absorb up to $2 million per year for current programs and $9 million per year for new programs that we believe would be similar to those that have been studied with RCTs.35 At the time, its top priorities included programs in Brazil, India, Nigeria, the Democratic Republic of Congo, Pakistan and Rwanda.36 PAD also noted that it is constrained in funding for conducting additional research and development.37 As of November 2020, PAD’s research priorities included "(i) strengthening the evidence base on yields, profitability, averted crop loss, and cost-effectiveness, (ii) understanding barriers to efficient social learning and testing digital interventions to overcome those barriers, (iii) digitizing interventions proven in small-scale studies to increase farmer yields and test their cost-effectiveness at a large scale."38

Sources

Document Source
Fabregas, Kremer, and Schilbach 2019 Source
Faull, "The Journey to One Million: PAD in India," 2020 Source (archive)
GiveWell, Cost-effectiveness analysis of PAD, 2020 Source
GiveWell, Interim intervention report on PAD (mobile-based agricultural advice), 2020 Source
GiveWell's non-verbatim summary of a conversation with Precision Agriculture for Development, August 9, 2017 Source
GiveWell's non-verbatim summary of a conversation with Precision Agriculture for Development, February 12, 2020 Source
GiveWell's non-verbatim summary of conversations with Precision Agriculture for Development, February 25, 2020 and March 5, 2020 Source
PAD, "India: Programs" Source (archive)
PAD, "Kenya: Programs" Source (archive)
PAD, "Our model" Source (archive)
PAD, 2019 Annual Report: Finances Source (archive)
PAD, 2020 budget breakdown for GiveWell Source
PAD, Appendix table: Trials conducted by PAD and affiliated researchers, 2020 Source
PAD, Polling training guide: India, 2020 Source
PAD, Report on evidence base and monitoring data, 2020 Source
PAD, Sample usage tracker: India, 2020 Source
PAD, Slide presentation, 2017 Source (archive)