You are here

Living Goods

Donate

Living Goods does not meet all of our criteria to be a GiveWell top charity but is a standout charity. Although we don't recommend these organizations as strongly as we do our top charities, they stand out from the vast majority of organizations we have considered.

More information: What is our evaluation process?

Published: November 2014

Since publishing this review, we have a mid-2016 update on Living Goods and notes from our conversations with Living Goods in May 2017. In September 2016, the authors of the randomized-controlled trial of Living Goods program posted a discussion paper on the study (archived version).

Summary

What do they do? Living Goods runs a network of Community Health Promoters (CHPs) who sell health and household goods door-to-door in their communities in Uganda and Kenya and provide basic health counseling. Living Goods also provides consulting and funding to BRAC to run a similar network in Uganda and to other organizations to run similar networks in other locations.

Does it work? The main evidence for Living Goods’ impact is a randomized controlled trial that found that the program caused a 27% reduction in under-5 mortality. The full details of this study are not yet publicly available. We have some questions about this study that we cannot yet discuss because the authors are in the process of submitting the final report for publication. We plan to publish a review of the study once the authors have published the full report. Living Goods collects ongoing monitoring data on various aspects of its programs; we have limited confidence in the quality of this data, which raises questions about our ability to learn about the impact of future work. We do not know how effective Living Goods’ work with its partners has been.

What do you get for your dollar? We estimate that Living Goods' cost per life saved will be roughly $10,000 in 2014-2016. Making assumptions that we would guess are particularly optimistic about Living Goods, we estimate the cost per life saved at about $4,400. Pessimistic assumptions lead to an estimate of about $37,000 per life saved.

Is there room for more funding? Living Goods is seeking about $10 million per year for the next 4 years to scale up its program in Uganda five fold, up from $3.3 million raised in 2013. It is currently in discussions with a number of its core funders who have expressed interest in backing the scale up. No funds have been committed at this stage. If all proposals to core funders are successful, Living Goods will have a gap of approximately $2 to 3 million per year. Living Goods is also seeking additional funding to co-fund partners to start networks in other countries.

Living Goods is a standout because of its:

  • Randomized-controlled trial, which found that Living Goods' program reduced child mortality by a very large amount.
  • Standout transparency - it has shared significant, detailed information about its programs with us.

Major unresolved issues include:

  • The main evidence for Living Goods' program is a single RCT, whereas other programs we recommend are generally supported by multiple sources of evidence. We have a number of potential concerns about Living Goods' RCT.
  • As a result of our limited confidence in how the Living Goods and BRAC programs are monitored, we do not know how much we will be able to learn about the success of the programs in the future.
  • The size of Living Goods' funding gap for the next year is highly uncertain because major funders are considering supporting the program. If Living Goods raises enough funds to scale up the program studied with the RCT, it may allocate additional funds to programs with less of a track record.

Our review process

We first considered Living Goods in 2009 and noted that it stood out for subjecting its program to a rigorous evaluation. In 2012, we spoke with Living Goods to get an update on its work and its randomized controlled trial (RCT; conversation notes). Living Goods told us at that time that the results from the RCT were not going to be reliable due to spillover of the intervention into control locations and incorrect matching of the treatment area and the treatment survey area – issues that reduce the statistical power of the study.1

In 2014, Living Goods informed us that the results of the RCT were available and that the effect on mortality was large enough to overcome the problems with statistical power. This year, in order to publish an interim review of Living Goods, we spoke with Living Goods twice (notes from May 2014 here and forthcoming for July 2014) and reviewed documents that it sent us in response to our questions about the RCT, financial statements and budgets, product offerings and sales records, metrics reports, accuracy checks on CHP reports, CHP selection process, CHP dropout rates and profitability, and program performance.

Since finalizing our interim review and determining that Living Goods was a contender for a recommendation, we met with Living Goods in October 2014, requested additional documents, conducted a site visit to Living Goods' program in Uganda in October 2014 (see notes and photos), and spoke once more with Living Goods in November 2014. We incorporated what we learned from discussions, documents, and the site visit into this review.

Previous versions of this page:

What do they do?

Living Goods has two main activities:

  1. Running a network of agents who sell health and other products door-to-door in Uganda and Kenya and who provide some basic health services.
  2. Consulting with other organizations on issues related to running similar agent-based systems in Uganda and other locations and funding some of these projects.

We discuss each of these more below.

Running a network of Community Health Promoters

Living Goods describes its network of agents as follows:2

Living Goods operates a network of franchised community health entrepreneurs, called Community Health Promoters (CHPs). CHPs provide health education and earn a living selling health and other “life-changing” and money-saving products home-to-home in their communities. Living Goods provides a loan to each CHP to start and operate her own business. The CHP makes an income by selling products related to healthy living and increased productivity such as treatments for malaria, diarrhea and pneumonia, soap, feminine hygiene items, solar lanterns, high efficiency cookstoves and more. CHPs are also trained in Integrated Community Case Management (ICCM), providing diagnosis and treatment for diarrhea, malaria and pneumonia. They aim to reduce under-five mortality by focusing on treatments for pneumonia, diarrhea, malaria, follow ups and referrals, providing pre-natal care, encouraging delivery in a facility, and improving newborn health through post-natal visits.

Recruiting and training


Living Goods recruits new CHPs through referrals from existing CHPs, community groups, other NGOs, and religious and local leaders.3 Criteria for new CHPs include: ability to spend two hours per day on CHP activities, 25-45 years old with 7 or more years of schooling, active in her community, fluent in local language and able to read and communicate in basic English.4 Candidates are asked to complete an interview to test their communication skills and engagement,5 attend a two week training course in health and business, and pass a test at the end of the course.

Training covers diagnosis, treatment, and recognizing danger signs that require referral to health facilities, as well as business and sales skills.6 After a CHP passes training, she spends her first two weeks conducting a census of all of the households in her designated area and collecting phone numbers and other key information.7 During this census, CHPs note which households have children under-5 and pregnant women.8

Those who pass the test are required to invest about $30 in initial supplies; Living Goods offers 6-month loans to cover part of the investment.9 Living Goods provides CHPs with a “business-in-a-bag,” which includes uniforms and promotional materials, and a phone.10 Living Goods also provides ongoing in-service training for CHPs.11

Living Goods told us that it interviews about five applicants for every applicant that advances to the training course and that about 5-10% of trainees do not pass the test at the end of the training course.12

Products


In Uganda, CHPs sell preventative health goods (fortified foods, mosquito nets, malaria prophylaxis, vitamins and minerals), treatments (malaria treatments, ORS and zinc, antibiotics, deworming pills, medicines for pain, coughing, and colds), reproductive health goods (delivery kits, menstrual pads, contraceptives), hygiene goods (soap, diapers, toothpaste, toilet paper), and household goods that Living Goods has selected because, in Living Goods’ words, they “have a proven economic benefit and may contribute to improved health” (cookstoves, solar lighting and power, water treatment), among other products.13 CHPs are expected to have medicines for diarrhea (ORS), pneumonia (amoxicillin), and malaria (ACTs) in stock at all times.14

The sales data Living Goods has shared with us is sales to CHPs, from Living Goods branches, rather than sales by CHPs to clients.15 Living Goods notes that both types of products offered and margins can change quickly16 – margins given are from July 2014 and may not reflect long-term trends. It is our understanding that "CHP margins" in the table below are those set by Living Goods; we don't know if they are followed in practice. Living Goods told us that it strongly discourages CHPs from selling at prices other than those set by Living Goods and that CHPs typically follow the price guidelines.17

Sales by product in Uganda between January 2011 and August 2014 (quantities/size per unit vary)18

Product type Sales (USD) % of total sales Units sold Living Goods margins CHP margins
Soap $229,379 24.9% 324,381 1-13% 4-24%
Cookstoves $204,434 22.2% 51,843 0-25% 11-31%
Fortified food $89,109 9.7% 243,718 4-19% 6-11%
Malaria treatments $65,760 7.1% 93,284 23% 33%
Solar lighting and power $62,455 6.8% 3,041 -40-30% 2-46%
Delivery kits $57,137 6.2% 18,115 20% 10%
Diapers $36,027 3.9% 16,771 10% 9%
Pain, cough & cold $34,579 3.8% 35,442 9-37% 8-66%
Menstrual pads $27,826 3.0% 20,652 7-17% 8-12%
Contraception $25,673 2.8% 42,327 20-33% 28-44%
ORS and zinc $13,829 1.5% 74,419 10-26% 7-33%
Deworming $10,874 1.2% 6,588 36% 48%
Mosquito nets $8,397 0.9% 2,579 12-24% 7-23%
Antibiotics $8,190 0.9% 6,990 25% 44%
Other (e.g. vitamins and minerals, fuel, water treatment, toothpaste) $47,817 5.2% 97,303 0-75% 5-100%
Total $921,486 100.0% 1,037,453

Living Goods’ program in Kenya is newer,19 smaller, and offers fewer products. CHPs in Kenya do not currently offer treatments, which are key to Living Goods’ goal of reducing child mortality. They do offer mosquito nets, fortified foods, contraceptives, and delivery kits, though these account for a small portion of sales to date. Living Goods told us that its current focus in Kenya is on selling “durables that deliver proven economic impact” such as high-efficiency cookstoves and solar lights.20

Sales by product in Kenya between July 2013 and July 201421

Product type Sales (USD) % of total sales Units solds
Cookstoves $29,780 56.5% 2,155
Solar lighting and power $11,064 21.0% 476
Water filters $3,196 6.1% 598
Menstrual pads $1,578 3.0% 1,095
Business materials $1,494 2.8% 963
Diapers $1,447 2.7% 386
Fortified food $1,349 2.6% 15,384
Other (e.g. fuel, contraception, delivery kits, mosquito nets) $2,807 5.3% 3,786
Total $52,715 100.0% 24,843

Health services


In addition to going door-to-door selling products, CHPs are asked to:

  • Provide households with advice on topics including nutrition, family planning, hygiene, and use of preventative tools such as bed nets, water treatment, clean stoves, and solar lights.22
  • Evaluate the health of children in the household and diagnose illnesses, using the Integrated Community Case Management (ICCM) system.23
  • Conduct follow-ups with children they have treated and, if necessary, refer the children to other healthcare providers.24
  • Give their phone number to clients and respond to calls when asked to provide treatment.25
  • Track pregnancies and make 2-3 visits during pregnancy to provide education and vitamins, check for risk factors, sell delivery kits, and encourage women to use health facilities for antenatal and delivery care. CHPs receive small financial incentives for registering pregnancies and making pre- and post-natal visits.26
  • Visit mothers shortly after they give birth and once more in the first week to “ensure proper newborn care practices.”27
  • Hold community events to provide health education, at least twice per month.28

Oversight


CHPs are overseen by Branch Managers. Branch Managers are expected to visit or talk to each CHP at least once per month29 and collect data from CHPs on pregnancy registrations, treatments provided to children under 5, and other activities.30 Living Goods told us that during their field visits with CHPs, Branch Managers are expected to evaluate CHPs on several criteria:31

  • Appearance: for example, checking that CHPs have a sign on their home indicating that they work for Living Goods and that they are wearing their uniform.
  • Inventory: checking that CHPs have a sufficient amount of key treatments.
  • Health knowledge: evaluating CHPs’ competency in diagnosing key illnesses and providing antenatal and postnatal care.
  • Product knowledge: evaluating CHPs’ knowledge of key benefits of products.
  • Selling skills.

We do not have a strong sense of the quality of Branch Managers’ evaluations of CHPs, and we have not seen documentation to support that Branch Managers carry out these evaluations once per month with all CHPs as planned.

Based on these reports and sales records, Branch Managers are asked to rank CHPs, occasionally give bonuses to top performers, provide coaching to medium and low performers, and dismiss the lowest performers.32 Criteria for high performance includes over $63 of product purchases, 7 or more registered pregnancies, 13 or more malaria treatments and 6 or more diarrhea treatments per month.33

Field Supervisors are responsible for supervising Branch Staff and monitoring the performance of Living Goods branches. They are expected to be in touch with branch management daily and visit branches frequently to answer questions, listen to challenges, and help find solutions to problems. Living Goods looks for Field Supervisors that have backgrounds in sales, health education, and community outreach.34

Planned smartphone system


In the future, Living Goods plans to require all of its CHPs to use Android-based smartphones during visits with customers.35

This system will:

  • Allow CHPs to keep electronic records of their customers and their past activities.36
  • Assist CHPs with accurate diagnosis and treatment by prompting them with questions for the patient and then providing a diagnosis and treatment recommendation based on the answers.37
  • Enable branches to collect data on CHPs’ sales and treatment activities in real time and linked to GPS coordinates.38

Living Goods has begun using its smartphone system in four of its eight branches, training 100% of the agents from three of them on the Android platform in October 2014 (as of this writing), with plans to move the rest to the system in the coming months.39

Living Goods told us that it expects this new smartphone system to reduce its costs of data collection and provide better monitoring and customer data, as well as make it easier for Living Goods to follow up and spot check agent-registered treatments and pregnancies.40

Partnerships

Living Goods has worked intensively with BRAC in Uganda and provided consulting services to PSI, The Clinton Foundation, and Marie Stopes International.41

BRAC partnership


Living Goods began, in 2008, as a partnership with BRAC to operate a network of CHPs in Uganda, and in 2009 launched a directly-managed network of CHPs using the same model.42 Living Goods has provided both technical and financial support, totaling over $2 million, to BRAC for the CHP program.43 BRAC has 128 branches with active CHPs in Uganda, but only 24 of these branches currently receive significant funding from Living Goods and have additional features, such as incentive payments for CHPs and a higher number of CHPs per branch.44

BRAC- and Living Goods-managed programs generally operate under the same model, but there are several differences between the programs, including:

  • BRAC usually works in poorer, more rural areas.45
  • BRAC sells a slightly different mix of drugs and products. For example, BRAC does not sell antibiotics to treat pneumonia because it does not yet have permission to sell antibiotics in Uganda.46
  • BRAC and Living Goods have slightly different CHP recruitment, training, and certification processes.47
  • BRAC provides a somewhat broader range of health services. For example, its CHPs also focus on immunizations and family planning.48
  • Living Goods has a more selective process for hiring branch staff and compensates its branch staff more highly.49
  • BRAC and Living Goods have slightly different incentive payments for CHPs.50
  • BRAC has invested more staff time in antenatal care.51
  • During the RCT, BRAC’s cost per CHP was roughly 50% lower because:52
    • BRAC operates at a larger scale, so its management costs per CHP were lower.
    • BRAC did not provide incentive payments to CHPs for some of the RCT period.
    • Other differences in BRAC’s CHP program (listed above) generally reduced its costs relative to Living Goods’ program.

The RCT discussed below includes both BRAC and Living Goods branches.

Other partnerships


As of 2014, Living Goods is actively working with one additional organization: PSI. It previously worked with PSI in Mozambique and is now working with PSI in Myanmar. In 2014, Living Goods began working with PSI Myanmar to provide “high-touch technical assistance to PSI to design and implement a replication of our Uganda model beginning in two townships.” PSI Myanmar will pay Living Goods over three years for its work, which Living Goods expects to cover most of its costs for the project.53

Living Goods has also provided technical assistance to Marie Stopes International, which was working to create a community health worker network in Kenya,54 and to the Clinton Foundation, to help design a pilot in Peru for an entrepreneurship program.55

Living Goods has three full-time staff members who focus on partnerships. A couple additional staff allocate a percentage of their time to this as well.56

Spending breakdown

We have seen expenditure data for Living Goods for 2012 and 2013 and a budget for 2014.57 Both of the tables below exclude the cost of goods. In both 2012 and 2013, Living Goods fully recovered the cost of goods (and made a margin – see third table below), and it expects to in 2014 as well. When the cost of goods are included in the total for 2012 and 2013, it accounts for about 7% of expenditures in those years.

Expenditures by type (in millions)

2012 2013 2014 % of expenditures 2012-2014
US-based expenses $1.17 $1.71 $1.27 41%
Country-level expenses $0.31 $1.06 $2.07 34%
Branch Offices and Transport $0.11 $0.20 $0.23 5%
Marketing and Promotion $0.12 $0.11 $0.13 4%
Training $0.05 $0.04 $0.03 1%
Sub-Grant to BRAC $0.60 $0.15 $0.36 11%
Capital Expenditures $0.08 $0.20 $0.14 4%
Total $2.43 $3.47 $4.22

Expenditures by program (in millions)

2012 2013 2014 % of expenditures 2012-2014 | excluding unallocated
LG Uganda $0.59 $0.79 $1.26 26% | 44%
LG Kenya - $0.50 $0.79 13% | 22%
BRAC Uganda $0.60 $0.15 $0.36 11% | 18%
Other partnerships $0.09 $0.32 $0.53 9% | 16%
Unallocated (primarily US-based operations) $1.16 $1.71 $1.28 41% | N/A
TOTAL $2.43 $3.47 $4.22

Margin on goods sold

2012 2013 Total
Cost of goods $172,726 $285,808 $458,534
Wholesale sales revenue $261,586 $393,663 $655,249
Damages, expired, and write downs $13,394 $39,966 $53,360
Final margin 29% 17% 22%

Living Goods organizational structure

Living Goods’ staff consists of about 5 US-based staff, about 35 staff on country-level teams in Uganda and Kenya, 2 field supervisors, and a Branch Manager and Assistant Branch Manager at each of eight branches in Uganda and one Branch Manager in Kenya.58 Country-level staff include staff specializing in training, product development, sales, health impact, logistics, technology, finance, talent, monitoring, and partnerships.59

Does it work?

To evaluate Living Goods’ impact, we considered:

  1. Do the health products that CHPs sell improve health and save lives?
  2. Does Living Goods increase access to care and/or quality of care?
  3. What does the randomized controlled trial of Living Goods’ program say about its impact?
  4. What does Living Goods’ ongoing monitoring say about its impact?
  5. Has Living Goods’ work with partners had an impact on those projects and have those projects improved or saved lives?
  6. Do the products that CHPs sell provide economic benefits?

In short:

  • The main evidence for Living Goods’ impact is a randomized controlled trial that found that the program caused a 27% reduction in under-5 mortality. The full details of this study are not yet publicly available. We have some questions about this study that we cannot yet discuss. We plan to publish a review of the study once the authors have published the full report. (More)
  • Living Goods collects ongoing monitoring data on various aspects of its programs. We have not seen reports on many aspects of its monitoring, and we have limited confidence in the quality of the monitoring data that has been shared with us, which raises questions about our ability to learn about the impact of future work. (More)
  • We have some information about the impact of Living Goods’ partnership with BRAC (mostly from the randomized controlled trial of its program). We do not know how effective Living Goods’ work with its other partners has been. (More)

Details follow.

Do the products that CHPs sell improve health and save lives?

Living Goods says that it “aim[s] to reduce under-five mortality by focusing on an ICCM+ approach, [which] includes quality diagnosis and treatments for pneumonia, diarrhea, malaria, encouraging prevention and healthy behaviors, and improving maternal and newborn health, especially in the perinatal period.”60 We have not completed recent evidence reviews for artemisinin-combination therapy (ACT) for malaria (older review of the program here), antibiotics to treat pneumonia or oral rehydration salts to treat diarrhea, but we believe that there is relatively strong evidence to support these programs. We have not researched the efficacy of interventions for newborn conditions. Living Goods told us that it focuses on newborn conditions in particular because it has found that a large portion of child deaths occur in the first month of life and that there are interventions that can significantly reduce this mortality.61 Living Goods’ RCT supported this; it found that 75% of deaths recorded in the RCT were in the first year and 52% in the first month of life.62

CHPs also sell deworming medications and mosquito nets, on which we have completed evidence reviews and believe to be effective. However, the manner in which these products are distributed (one-off purchases rather than community-wide distributions) may affect their impact. In addition, these products account for a small portion of Living Goods’ sales.

Other products and services may also contribute to health, but we have not reviewed the evidence on their impact. Such effects might include improved maternal health from pregnancy support and post-natal visits, reduced indoor air pollution and injuries resulting from use of cookstoves and solar lighting, improved nutrition from the use of fortified foods and vitamins and minerals, and reduced infections from the use of delivery kits and soap for handwashing.

Note that evidence that the products CHPs supply improve health would not be sufficient to demonstrate Living Goods’ impact, because households might obtain the products from other sources in the absence of CHPs. However, this evidence helps provide context on whether the RCT results are plausible.

Does Living Goods increase access to care and quality of care?

Living Goods notes that the drugs it sells are free in public facilities, but that clients may prefer to buy from CHPs because of the cost of traveling to facilities and possibility that the facility will be out of the medicine. According to Living Goods, clients may also be reluctant to buy drugs from other private providers because of the risk of getting a counterfeit medicine.63 Living Goods sent us a study conducted at the midline of its RCT that claims that both availability of counterfeit drugs and drug prices decreased at private retailers in areas where CHPs worked.64 According to the study, about 37% of private drug shops in the areas it studied sold fake ACT drugs,65 and availabilty of fake ACTs was about 50% lower among non-Living Goods sellers in the areas where Living Goods worked.66 Additional results on these potential effects will be made available when the full RCT is published. We have not yet reviewed the study or vetted other claims about how and why consumers choose to obtain medicine.

Living Goods also told us that it recently developed a five-step quality management process for all of its medicines that will be implemented in 2015.67 We do not yet know the details of this process.

Anecdotal evidence from our site visit


Note that areas that we visited and CHPs and customers we interacted with were not selected randomly (more details in footnote).68

Does Living Goods increase access to care and/or quality of care? Where would clients obtain health products from in the absence of CHPs?

Overall, the evidence from our site visit seemed broadly consistent with the possibility that Living Goods is improving access to healthcare and quality of healthcare in the areas where it works. For example, it may be increasing access to essential medication by having lower prices than competitors and a convenient delivery model, and it may be improving quality of care by building trust with its customers. However, we put little weight on this evidence because it is based on limited and potentially non-representative information.

On our site visit, we learned about some customers’ reasons for using Living Goods’ services. For example:

  • Some customers told us that Living Goods’ prices are lower than its competitors’ prices.69
  • It seemed that some customers buy products from Living Goods because it is convenient.70
  • Some customers seem to use Living Goods’ services because they have strong relationships with their CHPs.71

We also visited some private drug sellers in order to try to understand the other healthcare options that were available to people in areas that we visited. These shops seemed to have key medicines and products in stock, such as pneumonia medications, rapid diagnostic tests (RDTs) for malaria, and antimalarials. We were not able to determine anything about the quality of these drugs. It generally seemed that private drug sellers were reasonably accessible in the areas we visited.72

Living Goods staff gave us more information about other healthcare options that are available to people near the Living Goods Tula branch. They told us:73

  • The area is supposed to have Village Health Team members (government community health workers, called “VHTs”), but in practice they rarely work because they are not paid.
  • Government health centers provide free treatment, but they are often stocked out of essential medication such as antimalarials. Since travel to these clinics is expensive, some people have stopped attempting to go to them.
  • Pharmacies usually have essential medicine in stock but can be expensive. Also, it is often hard to find pharmacies far away from town centers. The cost of a typical course of Artemisinin Combination Therapy (ACT, antimalarials) would be about 5,000 UGX (~$1.83) at a pharmacy.
  • Private clinics are known for overcharging for treatment, but they are often the only option far away from town centers. The cost of a typical course of ACT at a private clinic would be 12,000 UGX (~$4.40).
  • The cost of a typical course of ACT at a hospital would be 6,000 UGX (~$2.20).
  • It would cost about 6,000 UGX (~$2.20) to travel to the nearest government hospital from the area – a significant amount of money for poor members of this community.
  • Living Goods sells a treatment course of ACT for 3,000 UGX (~$1.10).

We have not independently vetted this information.

Among the limited sample of people we encountered, it seemed that if Living Goods were not an option they would most likely go to private clinics or pharmacies to purchase treatments.74

Are Living Goods products consistently high quality?

The two Living Goods branches that we visited seemed to be following proper protocols as we understood them. In particular, they seemed to be carefully monitoring the supply, condition, and expiration dates of their medications.75

This generally supports the case that Living Goods offers high quality products, but because our experience on the site visit may have been non-representative and there are many other potential factors that may affect the effectiveness of drug treatments aside from following storage procedures for drugs properly, we consider it to be only weak evidence in support of the quality of Living Goods’ drug treatments.

What does the randomized controlled trial of Living Goods’ program say about its impact?

Researchers conducted a cluster-randomized controlled trial (RCT) of the CHP program in Uganda between 2011 and 2013. The study included both villages served by CHPs managed by Living Goods and villages served by CHPs managed by BRAC.76 The researchers have published an abstract on the study,77 and shared a more in-depth report with us. The more in-depth report is not yet cleared for publication because the authors are seeking publication in an academic journal.78 We have some questions based on the in-depth report that we cannot yet discuss. We plan to publish a review of the study once the authors have published the full report.

The study included 214 rural villages and surveyed 8119 households. It found that the program reduced under-five mortality by 27%.79 The under-five mortality effect was smaller (though still statistically significant) in BRAC branches than in Living Goods branches, but the cost per CHP was lower in BRAC branches as well.80 The authors note:81

The Community Health Promoter program could affect mortality through a number of channels, including improved access to treatment and health services, improved quality of treatment and health services, better access and knowledge of prevention, and by influencing other actors to improve the quality of services and products that they provide/sell. We find evidence supporting all these channels, including a 16% increase in treatment of diarrhea with ORS and zinc; an 54% increase, albeit starting from low levels, in follow-up visits for under-five children falling sick with malaria, ARI or diarrhea and an 71% increase, again starting from a low level, in home visits in the first seven postnatal days. While the likelihood of treatment with ACTs and antibiotics are similar across assignment arms, households in treatment villages are significantly more likely to purchase ACTs, antibiotics, and ORS/zinc from CHPs. Earlier research has shown that the quality of health products such as ACTs is low in many local markets, so the changed consumption pattern, and/or the increased competition as a result of the entry of the CHPs, most likely increased the quality of curative treatment of treatment households.

We find these results highly promising, however we have not fully vetted the results and note that the strongest programs we have considered are backed by multiple studies, while there has only been one rigorous study of the Living Goods program. In addition, from the information we have seen, we have the following concerns:

  • It appears that villages may have been carefully selected for inclusion in the sample (i.e. the list that was then divided into treatment and control) and that only a selection of Living Goods and BRAC branches were considered for the list. While this shouldn’t bias the results of the study, it could affect how replicable the results are in other locations.
  • With such a program it is not possible to hide from participants whether or not they received the intervention and outcome measures rely on self-reports of events that may have occurred a few years in the past. Those who knew they were part of the program (because they had interacted with a CHP) may have been influenced to give more positive answers. However, this effect is less likely to impact mortality reports than questions on more subjective or less memorable events.

What does Living Goods’ ongoing monitoring say about its impact?

Performance indicators

Living Goods reports on such metrics as number of malaria and diarrhea treatments provided, % of malaria and diarrhea cases followed up within 2 days, number of pregnancy registrations, % of pregnant women delivering in a facility or with trained help, % of women visited in the first 48 hours after giving birth, and % of referral cases followed up. It reports these metrics for both its directly-managed network and for the BRAC network. We have seen data for January 2012 through June 2014.82

We put limited weight on these metrics because:

  • The data for these metrics are self-reported by CHPs, who are evaluated on their performance on such metrics and provided financial incentives for certain behaviors,83 which may incentivize them to inflate their reports.
  • There is some evidence that CHPs are inflating the data. Living Goods told us that two main ways it checks the accuracy of these reports are to (a) compare the number of malaria and diarrhea treatments it sells to CHPs, to the number of malaria and diarrhea cases that CHPs report treating,84 and (b) conduct monthly analyses (called “outlier reports”) to identify CHPs who report unusually high numbers of malaria or diarrhea treatments, active pregnancies, and/or pregnancies registered.85 In January to August 2014, CHPs reported about 4% more malaria treatments than Living Goods sold to CHPs and about 32% more diarrhea treatments than Living Goods sold.86 Living Goods notes that because dosages vary by the age of the patient and CHPs may choose to stock up on key treatments, over a short time frame, these figures may not fully match.87 For example, with malaria treatments, the package sold equates to 1 adult dosage, 2 doses for an older child, or 4 doses for a younger child.88 This makes it difficult to correlate sales to actual treatments without knowing the exact age of every treatment (which Living Goods expects to be more feasible with the new Android system).89 Living Goods also shared an example of an “outlier report” with us.90 Living Goods told us that the example report includes data from all CHPs who sell health treatments.91 Of this portion of CHPs, it seems that roughly 20% were determined to have reported an unusually high amount of activity for at least one of the outcomes (malaria treatments, diarrhea treatments, active pregnancies, or pregnancies registered).92
  • It appears that Living Goods calculates the % of cases followed up as the total number of follow up visits reported divided by the total number of treatments provided. As a result, this rate can exceed 100%. The rate of cases of malaria and diarrhea followed up was over 100% in every month from February to June 2014, reaching a maximum of 133% in April.93

Randomized follow-ups


Living Goods has stated that it “uses randomized follow-ups to improve quality control and monitoring of treatments."94 Living Goods told us that it conducts or has conducted a few types of randomized follow-ups:

  • Sometimes, on their field visits, Branch Managers randomly choose and visit a few households in a CHP’s customer register to verify the accuracy of a CHP’s reported activities.95
  • Branch Managers visit a sample of customers who received goods or services from CHPs who were identified by an “outliers” report (mentioned above).96
  • In the past, it conducted some randomized follow-ups in order to verify the accuracy of CHPs’ pregnancy registrations and newborn visits because CHPs receive an incentive payment for such activity.97

We have not seen any results from such follow-ups. Living Goods told us that it may collect records from these follow-ups but that the records are not readily available because they are not reviewed by staff at the U.S. office.98

Field reports


Branch Managers began recording their evaluations of CHPs during field visits in August 2014.99 The content of these evaluations is described in the oversight section above. We have not seen data from these evaluations.

Stakeholder reports


Living Goods publishes annual “stakeholder reports” which report, at a high level, on organization and program progress. A few interesting notes from the 2013 report:

  • In BRAC’s network "under five diarrhea and malaria treatments and sales per CHP fell due to procurement challenges in the second half of the year that affected pricing."100
  • Living Goods "developed credit products for both consumers and agents, with an impressively high repayment rate of over 99%."101 We do not know how the repayment rate was calculated.
  • Living Goods conducted surveys of CHPs in both Uganda and Kenya, and reported a selection of positive results.102 Living Goods sent us additional information about the CHP surveys, but we have not yet reviewed it.103

Future monitoring plans


As of November 2014, Living Goods was working to revise and strengthen its monitoring and evaluation strategy. It expects to finalize its strategy in early 2015 and plans to implement any monitoring changes shortly after the strategy is finalized.104 Living Goods told us that its new strategy:105

  • Will continue to monitor performance against key performance indicators including prompt treatments, pregnancies supported, newborn visits, in stock rates, and more.
  • Will rely on increased use of Android smartphones by CHPs in order to track CHP performance real-time and verify the accuracy of CHPs’ reported activity.106 Living Goods’ goal is to eliminate use of paper reporting and convert all agents to 100% mobile data collection.107
  • Will continue to rely on reports that Living Goods currently compiles about outlier cases, discrepancies between treatments and sales, and CHP sales performance.
  • Will require CHPs to be retested on their health knowledge and ability to properly diagnose and treat annually. Living Goods may collect reports on the percentage of CHPs who do not pass this “recertification” test.
  • Will add monitoring tools to verify the accuracy of CHPs’ reported activity.108
  • May add monitoring tools to check the quality of CHPs’ diagnoses and prescribed treatments. The new Android platform includes a step-by-step diagnostic program, which Living Goods expects to improve accuracy and consistency of diagnosis and treatment.109
  • May involve tracking new performance metrics, such as percentage of treatments diagnosed within 48 hours of contracting illness, nutrition indicators, and geographic coverage (using mobile GPS data).
  • May involve conducting quasi-experimental research, such as by implementing a pre-treatment and post-treatment household survey. This aspect of the monitoring and evaluation strategy is least certain.

Living Goods expects that BRAC branches will also implement any new monitoring that is implemented at Living Goods branches, though it may be a significant amount of time before BRAC branches implement smartphone monitoring.110

Has Living Goods’ work with partners had an impact on those projects and have those projects improved or saved lives?

As discussed above, the RCT on Living Goods’ program includes an evaluation of the BRAC program that it has provided technical and financial assistance to. We have not seen information on whether Living Goods’ work with other partners has been impactful (about 9% of Living Goods’ total budget from 2012-2014 was spent on non-BRAC partnerships).111

Living Goods’ monitoring of BRAC


Living Goods told us that it conducts some monitoring of BRAC’s programs. For example:112

  • Each month, BRAC shares reports on its key performance indicators (i.e., number of malaria and diarrhea treatments provided, number of pregnancy registrations, etc.) with Living Goods. Living Goods reviews the reports and follows up with BRAC about any potential issues.
  • In the past, Living Goods staff has carried out randomized checks of BRAC branches.

We have not seen any reports on follow ups with BRAC or randomized checks.

Living Goods told us that it is still in the process of determining how it will monitor BRAC's programs in the future.113

BRAC's programs have independent leadership, so Living Goods may have limited ability to affect BRAC's policies and practices.

BRAC’s monitoring


BRAC told us that it invests a significant amount of staff time in verifying the accuracy of its CHPs’ reported activities and to evaluate the quality of the care that CHPs provide.114 However, we have not seen any reports on details of these evaluations. We summarize the details of the monitoring system BRAC described to us in this footnote.115

BRAC told us that all monitoring reports are collected and summarized at each level of the organization and flow up all the way to the BRAC country office. The BRAC country office summarizes findings from these reports and discusses them at a monthly meeting.116

We learned about BRAC's monitoring shortly before publishing this report. It is our impression from our conversations with Living Goods that it does not regularly request BRAC’s detailed field-level monitoring results.117 We have not yet asked BRAC to share documentation from its monitoring.

Anecdotal evidence from our site visit


During our 2014 site visit to a BRAC branch, we noticed a few potential, likely minor issues:

  • CHPs told us that some CHPs sell drugs to drug shop owners, which is against BRAC-LG rules.118
  • The branch that we visited seemed to have very few deworming pills (albendazole) left in stock.119
  • CHPs reported sometimes running out of medicines and restocking seemed to require paying reasonably expensive travel fees.120

We put limited weight on these concerns because they are based on anecdotal evidence.

BRAC also told us that it is working to strengthen its procurement supply chain because its branches sometimes experience stock-outs.121

Do the products that CHPs sell provide economic benefits?

Some Living Goods products, such as cookstoves and solar energy products, may provide economic benefits to customers by, for example, saving them money on fuel. Living Goods sent us estimates of how much customers who buy cookstoves and solar energy products save.122 Living Goods told us that it estimates that its customers save 50% of their monthly fuel costs on average when using an efficient cookstove, and that typical fuel spending in Uganda tends to be in the range of $20-25 USD per month pre-stove purchase.123 We have not yet reviewed these estimates.

Possible negative or offsetting impact

  • Are clients’ problems sometimes misdiagnosed and/or overdiagnosed? We do not know how difficult it is to properly diagnose the various problems that CHPs may come across with a CHP’s level of training. For example, it seems possible that malaria may be difficult to distinguish from pneumonia. Misdiagnosis may cause worse outcomes for clients if they would have sought healthcare providers with better diagnosis skills in the absence of Living Goods’ program. Additionally, since CHPs have a financial incentive to sell more treatments, it seems possible that the Living Goods program could cause CHPs to overdiagnose clients and overprescribe treatments. As discussed in the monitoring section above, Living Goods has told us that it recognizes this concern and has some systems in place to attempt to reduce the likelihood that this occurs, but we are not confident that Living Goods' systems would identify many cases of misdiagnosis or overdiagnosis.
  • Are some CHPs harmed by the program? CHPs take on financial risk by joining the program. Living Goods estimates that its CHP turnover rate is about 22% per year, and that 70-80% of CHPs who leave chose to drop out of the program, while the rest are asked to leave.124 We do not know how common it is for CHPs to lose money as a result of their participation.
  • Does Living Goods divert skilled labor from other areas? CHPs are fairly well-educated and well-connected members of their communities. Would they have been adding value to their communities in other ways if they were not spending time as a CHP? Given the part-time nature of their work, we would guess that this is not a major concern.

What do you get for your dollar?

Living Goods' estimates

Living Goods has provided estimates of its future cost-per-life saved based on the lives saved per CHP in the RCT and modeling of future costs and numbers of CHPs. These estimates are based on Living Goods Uganda-specific budgets for 2015-2018 and the number of Living Goods-managed agents it expects to have each year. The "mid case" uses the blended mortality reduction estimated by the RCT (25.5%), which includes both Living Goods and BRAC branches, while the "low case" seeks to account for possible reductions in impact during scaling (using 20% for the impact on mortality), and the "best case" uses the mortality reduction observed for Living Goods branches only in 2013 (42%).125

Our adjustments to Living Goods' estimates

Living Goods shared the details of how it calculated its estimates, and we think they are largely reasonable with a few caveats. There are a few additional assumptions we make in our estimates of Living Goods cost effectiveness:126

  • Living Goods has included only those costs that it incurs within Uganda or that are allocated to the Uganda program.127 We have included Living Goods' full budget, minus expected program revenue.128 These additional costs support the program directly and indirectly and include US-based costs (e.g., business development, communications and advocacy) and Living Goods’ partnership team. Living Goods told us that it does not include its partnerships team costs in its model of its cost-effectiveness because that team currently has costs but no basis for modeling future associated impact of its spending.129 The partnerships costs represent roughly 17% of Living Goods’ total budget for 2015-2018 and other non-Uganda and non-partnership costs (primarily US-based costs) represent about 10% of Living Goods’ total funding need for those years.130
  • The budget Living Goods provided does not include costs of the Kenya program, so these are missing from our model.
  • Living Goods' estimates rely on an estimation of the lives saved per CHP based on the 2012 under-5 mortality rate in Uganda. Under-5 mortality fell by about 4-6% per year in Uganda from 2009 to 2012.131 If this trend continues, lives saved per CHP will likely be lower in the future. In some of our scenarios, we have assumed that mortality, in the absence of Living Goods' program, will continue to fall over the next four years.
  • In one of our scenarios, we have added an adjustment for contamination of the RCT due to CHPs visiting control group households.132 This contamination could cause the RCT to underestimate the effect of the intervention.
  • We have added an adjustment for our subjective guess on whether the RCT results would be replicated if another RCT was done on the scale up program. This adjustment aims to capture both the strength of our belief in the accuracy of the RCT results as well as in the likelihood that Living Goods and BRAC will be able to replicate the results at a greater scale and in somewhat different locations (though still within Uganda). This adjustment is varied across scenarios. However, the average replicability adjustment in our scenarios effectively reduces the mortality reduction in the RCT by about 50%.
  • We have varied, across scenarios, the number of years we include in the estimate. Since Living Goods expects its cost per CHP to decline as it scales, it is more conservative to only include costs and benefits for the next 1-3 years than it is to include all 5 years.

Two GiveWell staffers have entered parameters for the various assumptions to derive "pessimistic," "base case," and "optimistic" estimates of Living Goods' cost per life saved (similar in concept to Living Goods' "low case," "mid case," and "best case"). The two staffers used somewhat different assumptions but generated similar estimates:133

  • Pessimistic: About $37,000 per life saved
  • Base case: About $10,000 per life saved
  • Optimistic: About $4,400 per life saved

Details in this spreadsheet.134 In the future, we may ask Living Goods for some additional information that will help us improve the model (details in footnote).135 In the future, we may also conduct further research to help us improve the model (details in footnote).136

Note that the estimate does not take into account any other potential benefits of the program, such as improved health in non-fatal cases or economic benefits from the purchase of money-saving durables such as solar lights, more efficient cook stoves, or water filters (discussed above). The estimate focuses solely on under-5 mortality because this was the primary outcome examined by Living Goods’ RCT. We do not have enough information about Living Goods’ effects on other health outcomes to confidently include them in our analysis.

Average cost per life saved of Living Goods’ Uganda program

The figures discussed above estimate the average cost per life saved by the Living Goods program. This approach is consistent with our approach to estimating cost effectiveness for our top charities and seeks to deal with issues of "leverage" by treating all donor funds the same.

Another approach, and one that Living Goods favors relative to the approach above, would be to attempt to calculate an average cost per life saved for Living Goods’ Uganda program only. This excludes many costs that Living Goods expects to incur regardless of the number of CHPs it supports, namely, U.S. headquarters costs and the costs of supporting the partnership team.

With these assumptions, we estimate a base case of about $7,000 per life saved.137 As noted above, this estimate is not directly comparable with cost effectiveness estimates we have created for other charities.

Is there room for more funding?

Scaling up in Uganda

Scale up plans

Having recently completed the RCT, Living Goods is now aiming to rapidly scale up the networks of CHPs directly managed by Living Goods and managed by BRAC, from about 1,000 CHPs (400 Living Goods, 600 BRAC) to 6,500 CHPs (2,500 Living Goods, 4,000 BRAC).138 Living Goods shared a detailed breakdown of its scale-up plan for networks directly managed by Living Goods with us.139

Living Goods expects its scale-up in Uganda to be similar to its past work in the country.140 Differences may include:

  • A greater number of agents per branch.141
  • An increased focus on postnatal visits.142
  • An increased focus on nutrition products and nutrition education.143
  • Work by BRAC on improving the quality of local health facilities, particularly for delivery care, through training, checklists, and new protocols.144
  • For the first time, Living Goods will have some male CHPs.145

Living Goods has also noted a couple of ways in which the setting it is working in has changed since the start of the project: bednet coverage is 2-3 times higher and the market price of malaria treatment has been reduced.146 Under-5 mortality in Uganda, according to the World Bank, decreased from 83 per 1,000 live birth in 2009 to 69 in 2012.147

Funding need

In total, Living Goods is seeking funding to support a budget of about $9.6 million per year, including costs incurred by BRAC. Since Living Goods’ overall budget is increasing, it also aims to raise an additional $3 million in reserves (about $2 million in 2015 and about $1 million in 2016).148 It expects to raise some funds from partnership consulting fees and margins on goods sold to CHPs, leaving about $10 million per year that would need to be supported by donor funding.149 In 2012 Living Goods raised $2.8 million, and in 2013 it raised $3.3 million from donors.150 Living Goods told us that it believes there is a decent chance it will reach two-thirds of its funding target for the first year through agreements with funders who have supported its work in the past, but the money has not yet been secured and the funding need will grow each year.151 The Children’s Investment Fund Foundation (CIFF), one of Living Goods’ major core funders historically, will be deciding in Q1 2015 whether to fund Living Goods' scale-up. CIFF has indicated a significant interest in funding up to 50% of Living Goods’ costs for 2015-2018, with the condition that Living Goods raises approximately $1 for every $1 CIFF contributes.152 Regardless of CIFF’s decision, Living Goods is actively seeking funding from new and existing donors to fill its funding gap and plans to use these funds to support scaling up in Uganda and its partnerships and advocacy team’s efforts to help organizations replicate the model in other places.153

Living Goods told us that BRAC is largely relying on Living Goods to secure funding for BRAC's portion of the Uganda scale-up.154

Uganda scale-up budget 2015-2018 (in millions)155

Expenditures Program revenue Donor funding needed
Living Goods Uganda $11.29 $1.02 $10.27
BRAC Uganda $14.98 $0.93 $14.05
Partnerships $6.52 $3.25 $3.27
Other (primarily US-based) $5.68 - $5.68
TOTAL $38.48 $5.21 $33.27

Operations in Kenya

Living Goods told us that in the past it has used a mix of unrestricted and restricted funding to support its operations in Kenya.156 In 2015, Living Goods told us that it plans to spend about $700,000 in Kenya,157 which is similar to the roughly $750,000 it spent in 2014.158 As far as we know, Living Goods does not have plans to use further unrestricted funds to scale up in Kenya. Living Goods told us that if it decided to significantly expand or change its operations in Kenya, it would find additional independent funding to support this.159

Replication fund

Living Goods has told us that scaling up in Uganda is its top priority, but in order to have the transformational impact it seeks, it wants to demonstrate that the model can work in other markets. To this end, Living Goods and CIFF are discussing creating a fund for replications of its program by partners. If it received sufficient funding for its scale-up, it may allocate unrestricted funds to the replication fund.160 Living Goods has proposed creating a fund of $10-15 million to provide co-funding for programs in three countries that would reach 3-5 million people per country; it estimates that the full cost of a new country program would be $10-15 million,161 so its support would cover 33% of the total. It expects that having such a fund would improve its ability to negotiate and launch new projects with partners.162

It plans to target countries based on:163

  1. Need/impact: high child mortality rates driven by malaria, diarrhea, respiratory infections, and neonatal conditions; high rates of counterfeit drugs; high drug prices; low rates of use of water filters, clean cookstoves, and solar lights.
  2. Economics/sustainability: reasonable household purchasing power, and large (over 15 million) populations with high population density.
  3. Governance: low security risk and a government that will allow the program to sell medicines.
  4. Current partner presence: Living Goods would prefer to work with partners that already have relationships in country.

Based on these criteria, Living Goods is most strongly considering partnerships in Nigeria, India, Ghana, Zambia, Tanzania, Kenya, and Myanmar.164 Possible partners include PSI, Management Sciences for Health, Marie Stopes International, CARE, BRAC, Save the Children, and Plan International.165

Living Goods as an organization

We have limited observations on which to base an assessment of Living Goods as an organization. However, our sense is that Living Goods is a strong organization:

  • Track record: Living Goods is a relatively young organization; it was founded in 2007. Its RCT, though not yet fully vetted, appears to have found strong effects of its program. However, we feel that we have limited information on its track record because we have not seen detailed, high-quality monitoring of its program. We expect to learn more about Living Goods as an organization as we see how it handles the challenges of scaling up its program and tracking the quality of its program as it scales.
  • Self-evaluation: Living Goods has invested unusually heavily in understanding the impact of its program by completing a RCT of its program. It has also demonstrated a commitment to collecting information on its program on an ongoing basis, though we believe that this information is less rigorous than that of some of our other top charities to date.
  • Communication: Living Goods has communicated clearly and directly with us and given thoughtful answers to our critical questions.
  • Transparency: Living Goods has consistently been strong in its commitment to transparency. We have not seen it hesitate to share information publicly (unless it had what we felt was a good reason).

More on how we think about evaluating organizations at our 2012 blog post.

Sources

Document Source
Bjorkman-Nykvist et al. 2013 Source (archive)
Bjorkman-Nykvist et al. 2014 abstract Source (archive)
Chuck Slaughter, Brad Presner, Lisa McCandless, conversation with GiveWell, October 1, 2014 Unpublished
Chuck Slaughter, conversation with GiveWell, July 29, 2014 Unpublished
Email from Chuck Slaughter, Founder and President, Brad Presner, Director of Analytics, and Lisa McCandless, Director of Development, Living Goods, November 24, 2014 Unpublished
Email from Chuck Slaughter, Founder and President, Brad Presner, Director of Analytics, and Lisa McCandless, Director of Development, Living Goods, October 6, 2014 Unpublished
GiveWell estimate of Living Goods cost effectiveness (November 2014) Source
GiveWell spending analysis for Living Goods (November 2014) Source
GiveWell's non-verbatim summary of a conversation with Chuck Slaughter, July 20, 2012 Source
GiveWell's non-verbatim summary of a conversation with Chuck Slaughter, May 5, 2014 Source
GiveWell’s non-verbatim summary of a conversation with Jakob Svensson and Chuck Slaughter, July 31, 2014 Source
Lisa McCandless, conversation with GiveWell, November 7, 2014 Unpublished
Lisa McCandless, Living Goods Director of Business Development, email to GiveWell, July 24, 2014 Unpublished
Living Goods BRAC Partnership Source (archive)
Living Goods budget (2014) Source
Living Goods Business-In-A-Bag Source (archive)
Living Goods CHP Interview Guide Source
Living Goods CHP Interview Test Source
Living Goods CHP Profile and Expectations Source
Living Goods Customer Savings Survey (February 2014) Source
Living Goods Draft Branch Management Handbook Unpublished
Living Goods draft budget (2015-2018) Unpublished
Living Goods Draft Cost per Capita Served Methodology Source
Living Goods Draft Cost per Life Saved Methodology Source
Living Goods Draft Replication Strategy (September 2014) Source
Living Goods GiveWell Document Request Guide Source
Living Goods Health Dashboard (June 2014) Source
Living Goods Health Reporting Outliers (August 2014) Source
Living Goods Kenya Monthly Sales Unpublished
Living Goods Monthly CHP Survey (May 2013) Unpublished
Living Goods Org Chart (August 2014) Source
Living Goods Partner Organizations Source (archive)
Living Goods Product Impact Strategy (2014) Source
Living Goods Product List (July 2014) Source
Living Goods Quarterly Dashboard Source
Living Goods Sales Dashboard (June 2014) Source
Living Goods Scaling Plan (2015-19) Source
Living Goods Stakeholder Report (2012) Source
Living Goods Stakeholder Report (2013) Source
Living Goods Training & Coaching Source (archive)
Living Goods Treatment vs. Sales Summary (January to August 2014) Source
Living Goods Ugandan Monthly Sales Unpublished
Living Goods unaudited financials (2012-2013) Unpublished
Living Goods Where We Work Source (archive)
Living Goods-BRAC Draft Scale Up Concept Note (August 2014) Source
Molly Christiansen, conversation with GiveWell, November 3, 2014 Unpublished
Notes from GiveWell site visit to Living Goods and BRAC, October 2014 Source
WHO/UNICEF Integrated Community Case Management Source (archive)
World Bank under-5 mortality rate data Source (archive)