Living Goods - 2014 Interim Review | GiveWell

You are here

Living Goods - 2014 Interim Review

This page is an interim review. We have published a more recent version of this review here.

Living Goods is applying to be a 2014 top-rated charity. Here we discuss what we have learned so far and our major outstanding questions.

More information: What is our evaluation process?

Published: October 2014

Summary

What do they do? Living Goods runs a network of Community Health Promoters (CHPs) who sell health and household goods door-to-door in their communities in Uganda and Kenya and provide basic health counseling. Living Goods also provides consulting and funding to other organizations to run similar networks in other locations.

Does it work? The main evidence for Living Goods’ impact is a randomized controlled trial that found that the program caused a 25% reduction in under-5 mortality. The full details of this study are not yet publicly available. Living Goods collects ongoing monitoring data on various aspects of its programs; we have limited confidence in the quality of this data, which raises questions about our ability to learn about the impact of future work. We do not know how effective Living Goods’ work with its partners has been.

What do you get for your dollar? We estimate that Living Goods' cost per life saved will be roughly $11,000 in 2014-2016. Making assumptions that we would guess are particularly optimistic about Living Goods, we estimate the cost per life saved at about $3,300. Pessimistic assumptions lead to an estimate of $28,000 per life saved. Our work on this model is ongoing.

Is there room for more funds? Living Goods is seeking about $9 million per year for the next 4 years to scale up its program in Uganda, up from $3.3 million raised in 2013. It is currently in discussions with a number of its core funders who have expressed interest in backing the scale up. No funds have been committed at this stage. If all proposals to core funders are successful, Living Goods will have a gap of approximately $2 to 3 million per year. Living Goods is also seeking additional funding to co-fund partners to start networks in other countries.

What are GiveWell’s next steps? Living Goods is a contender for a GiveWell recommendation. We plan to continue our review process of Living Goods to try to answer our remaining questions.

Why we are publishing this page

As we mentioned in our plan for 2014, a key research priority for this year is actively pursuing investigations of promising charities. This page is intended to update our followers on Living Goods’ application and what we've learned so far.

Our investigation process

We first considered Living Goods in 2009 and noted that it stood out for subjecting its program to a rigorous evaluation. In 2012, we spoke with Living Goods to get an update on its work and its RCT (conversation notes). Living Goods told us at that time that the results from the RCT were not going to be reliable due to spillover of the intervention into control locations and incorrect matching of the treatment area and the treatment survey area – issues that reduce the statistical power of the study.1

In 2014, Living Goods informed us that the results of the RCT were available and that the effect on mortality was large enough to overcome the problems with statistical power. This year, we have spoken with Living Goods twice (notes from May 2014 here and forthcoming for July 2014) and reviewed documents that it sent us in response to our questions about the RCT, financial statements and budgets, product offerings and sales records, metrics reports, accuracy checks on CHP reports, CHP selection process, CHP dropout rates and profitability, and program performance.

We also met with Living Goods in September 2014 and have not yet incorporated what we learned during that meeting into this review. We plan to conduct a site visit to Living Goods' program in Uganda in October 2014 and then publish a final review for 2014 that incorporates additional information from the meeting, site visit, and documents.

Previous version of this page here.

What do they do?

Living Goods has two main activities:

  1. Running a network of agents who sell health and other products door-to-door in Uganda and Kenya and who provide some basic health services.
  2. Consulting with other organizations on issues related to running similar agent-based systems in other locations and funding some of these projects.

We discuss each of these more below.

Running a network of Community Health Promoters

Living Goods describes its network of agents as follows:2

Living Goods operates a network of franchised community health entrepreneurs, called Community Health Promoters (CHPs). CHPs provide health education and earn a living selling health and other “life-changing” and money-saving products home-to-home in their communities. Living Goods provides a loan to each CHP to start and operate her own business. The CHP makes an income by selling products related to healthy living and increased productivity such as treatments for malaria, diarrhea and pneumonia, soap, feminine hygiene items, solar lanterns, high efficiency cookstoves and more. CHPs are also trained in the Integrated Community Case Management (ICCM) model to provide basic health counseling on a variety of topics. They aim to reduce under-five mortality by focusing on treatments for pneumonia, diarrhea, malaria, follow ups and referrals, providing pre-natal care, encouraging delivery in a facility, and improving newborn health through post-natal visits.

Recruiting and training


Living Goods recruits new CHPs through referrals from existing CHPs, community groups, other NGOs, and religious and local leaders.3 Criteria for new CHPs include: ability to spend two hours per day on CHP activities, 25-45 years old with 7 or more years of schooling, active in her community, fluent in local language and able to read and communicate in basic English.4 Candidates are asked to complete an interview to test their communication skills and engagement,5 attend a two week training course in health and business, and pass a test at the end of the course. Training covers diagnosis, treatment, and recognizing danger signs that require referral to health facilities, as well as business and sales skills.6

Those who pass the test are required to invest about $30 in initial supplies; Living Goods offers 6-month loans to cover part of the investment.7 Living Goods provides CHPs with a “business-in-a-bag,” which includes uniforms and promotional materials, and a phone.8 Living Goods also provides ongoing in-service training for CHPs.9

Products


In Uganda, CHPs sell preventative health goods (fortified foods, mosquito nets, malaria prophylaxis, vitamins and minerals), treatments (malaria treatments, ORS and zinc, antibiotics, deworming pills, medicines for pain, coughing, and colds), reproductive health goods (delivery kits, menstrual pads, contraceptives), hygiene goods (soap, diapers, toothpaste, toilet paper), and household goods that Living Goods has selected because, in Living Goods’ words, they “have a proven economic benefit and may contribute to improved health” (cookstoves, solar lighting and power, water treatment), among other products.10 CHPs are expected to have medicines for diarrhea (ORS), pneumonia (amoxicillin), and malaria (ACTs) in stock at all times.11

The sales data Living Goods has shared with us is sales to CHPs, from Living Goods branches, rather than sales by CHPs to clients.12 Living Goods notes that both types of products offered and margins can change quickly13 – margins given are from July 2014 and may not reflect long-term trends. It is our understanding that "CHP margins" in the table below are those set by Living Goods; we don't know if they are followed in practice. Living Goods told us that it strongly discourages CHPs from selling at prices other than those set by Living Goods and that CHPs typically follow the price guidelines.14

Sales by product in Uganda between January 2011 and August 2014 (quantities/size per unit vary)15

Product type Sales (USD) % of total sales Units sold Living Goods margins CHP margins
Soap $229,379 24.9% 324,381 1-13% 4-24%
Cookstoves $204,434 22.2% 51,843 0-25% 11-31%
Fortified food $89,109 9.7% 243,718 4-19% 6-11%
Malaria treatments $65,760 7.1% 93,284 23% 33%
Solar lighting and power $62,455 6.8% 3,041 -40-30% 2-46%
Delivery kits $57,137 6.2% 18,115 20% 10%
Diapers $36,027 3.9% 16,771 10% 9%
Pain, cough & cold $34,579 3.8% 35,442 9-37% 8-66%
Menstrual pads $27,826 3.0% 20,652 7-17% 8-12%
Contraception $25,673 2.8% 42,327 20-33% 28-44%
ORS and zinc $13,829 1.5% 74,419 10-26% 7-33%
Deworming $10,874 1.2% 6,588 36% 48%
Mosquito nets $8,397 0.9% 2,579 12-24% 7-23%
Antibiotics $8,190 0.9% 6,990 25% 44%
Other (e.g. vitamins and minerals, fuel, water treatment, toothpaste) $47,817 5.2% 97,303 0-75% 5-100%
Total $921,486 100.0% 1,037,453

Living Goods’ program in Kenya is newer,16 smaller, and offers fewer products. CHPs in Kenya do not currently offer treatments, which are key to Living Goods’ goal of reducing child mortality. They do offer mosquito nets, fortified foods, contraceptives, and delivery kits, though these account for a small portion of sales to date. Living Goods told us that its focus in Kenya is on selling “durables that deliver proven economic impact” such as high-efficiency cookstoves and solar lights.17

Sales by product in Kenya between July 2013 and July 201418

Product type Sales (USD) % of total sales Units solds
Cookstoves $29,780 56.5% 2,155
Solar lighting and power $11,064 21.0% 476
Water filters $3,196 6.1% 598
Menstrual pads $1,578 3.0% 1,095
Business materials $1,494 2.8% 963
Diapers $1,447 2.7% 386
Fortified food $1,349 2.6% 15,384
Other (e.g. fuel, contraception, delivery kits, mosquito nets) $2,807 5.3% 3,786
Total $52,715 100.0% 24,843

Health services


In addition to going door-to-door selling products, CHPs are asked to:

  • Provide households with advice on topics including nutrition, family planning, hygiene, and use of preventative tools such as bed nets, water treatment, clean stoves, and solar lights.19
  • Evaluate the health of children in the household and diagnose illnesses, using the Integrated Community Case Management (ICCM) system.20
  • Give their phone number to clients and respond to calls when asked to provide treatment.21
  • Track pregnancies and make 2-3 visits during pregnancy to provide education and vitamins, check for risk factors, sell delivery kits, and encourage women to use health facilities for antenatal and delivery care. CHPs receive small financial incentives for registering pregnancies and making pre- and post-natal visits.22
  • Visit mothers shortly after they give birth and once more in the first week to “ensure proper newborn care practices.”23
  • Hold community events to provide health education, at least twice per month.24

Oversight


CHPs are overseen by Branch Managers. Branch Managers are expected to visit or talk to each CHP at least once per month25 and collect data from CHPs on pregnancy registrations, treatments provided to children under 5, and other activities.26 Based on these reports and sales records, Branch Managers are asked to rank CHPs, occasionally give bonuses to top performers, provide coaching to medium and low performers, and dismiss the lowest performers.27 Criteria for high performance includes over $63 of product purchases, 7 or more registered pregnancies, 13 or more malaria treatments and 6 or more diarrhea treatments per month.28

Field Supervisors are responsible for supervising Branch Staff and monitoring the performance of Living Goods branches. They are expected to be in touch with branch management daily and visit branches frequently to answer questions, listen to challenges, and help find solutions to problems. Living Goods looks for Field Supervisors that have backgrounds in sales, health education, and community outreach.29

Living Goods has also stated that it “uses randomized follow-ups to improve quality control and monitoring of treatments."30 We have not seen more information on this and Living Goods told us that it does not maintain records from these follow-ups.31

Partnerships

Living Goods has worked intensively with BRAC in Uganda and provided consulting services to PSI, The Clinton Foundation, and Marie Stopes International.32

BRAC partnership


Living Goods began, in 2008, as a partnership with BRAC to operate a network of CHPs in Uganda, and in 2009 launched a directly-managed network of CHPs using the same model.33 Living Goods has provided both technical and financial support, totaling over $2 million, to BRAC for the CHP program.34 It is our understanding that the BRAC- and Living Goods-managed programs operate similarly,35 though Living Goods notes some differences in focus (see footnote for details).36 The impact study discussed below includes both BRAC and Living Goods branches.

Other partnerships


As of 2014, Living Goods is actively working with one additional organization: PSI. It previously worked with PSI in Mozambique and is now working with PSI in Myanmar. In 2014, Living Goods began working with PSI Myanmar to provide “high-touch technical assistance to PSI to design and implement a replication of our Uganda model beginning in two townships.” PSI Myanmar will pay Living Goods $200,000 over three years for its work, which Living Goods expects to cover most of its costs for the project.37

Living Goods has also provided technical assistance to Marie Stopes International, which was working to create a community health worker network in Kenya,38 and to the Clinton Foundation, to help design a pilot in Peru for an entrepreneurship program.39

Living Goods has three full-time staff members who focus on partnerships. A couple additional staff allocate a percentage of their time to this as well.40

Spending breakdown

We have seen expenditure data for Living Goods for 2012 and 2013 and a budget for 2014.41 Both of the tables below exclude the cost of goods. In both 2012 and 2013, Living Goods fully recovered the cost of goods (and made a margin – see third table below), and it expects to in 2014 as well. When the cost of goods are included in the total for 2012 and 2013, it accounts for 7% of expenditures in those years.

Expenditures by type (in millions)

2012 2013 2014 % of expenditures 2012-2014
US-based expenses $1.17 $1.71 $1.27 41%
Country-level expenses $0.90 $1.20 $2.07 41%
Branch Offices and Transport $0.11 $0.20 $0.23 5%
Marketing and Promotion $0.12 $0.11 $0.13 4%
Training $0.05 $0.04 $0.03 1%
Sub-Grant to BRAC - - $0.36 4%
Capital Expenditures $0.08 $0.20 $0.14 4%
Total $2.43 $3.46 $4.22 100%

Expenditures by program (in millions)

2012 2013 2014 % of expenditures 2012-2014 | excluding unallocated
Uganda $0.59 $0.78 $1.26 26% | 45%
Kenya $0.00 $0.49 $0.79 13% | 22%
Partnerships $0.69 $0.47 $0.81 19% | 33%
Unallocated (primarily US-based operations) $1.16 $2.14 $1.36 42% | N/A
Total $2.43 $3.87 $4.22

Margin on goods sold

2012 2013 Total
Cost of goods $172,726 $264,263 $436,989
Wholesale sales revenue $261,586 $373,017 $634,603
Damages, expired, and write downs $13,394 $34,274 $47,668
Final margin 29% 20% 24%

Living Goods organizational structure

Living Goods’ staff consists of about 5 US-based staff, about 35 staff on country-level teams in Uganda and Kenya, 2 field supervisors, and a Branch Manager and Assistant Branch Manager at each of eight branches in Uganda and one Branch Manager in Kenya.42 Country-level staff include staff specializing in training, product development, sales, health impact, logistics, technology, finance, talent, monitoring, and partnerships.43

Does it work?

To evaluate Living Goods’ impact, we considered:

  1. Do the health products that CHPs sell improve health and save lives?
  2. What does the randomized controlled trial of Living Goods’ program say about its impact?
  3. What does Living Goods’ ongoing monitoring say about its impact?
  4. Has Living Goods’ work with partners had an impact on those projects and have those projects improved or saved lives?
  5. Do the products that CHPs sell provide economic benefits?

Do the products that CHPs sell improve health and save lives?

Living Goods says that it “aim[s] to reduce under-five mortality by focusing on an ICCM+ approach, [which] includes quality treatments for pneumonia, diarrhea, malaria, encouraging prevention and healthy behaviors, and improving maternal and newborn health, especially in the perinatal period.”44 We have not completed recent evidence reviews for artemisinin-combination therapy (ACT) for malaria (older review of the program here), antibiotics to treat pneumonia or oral rehydration salts to treat diarrhea, but we believe that there is relatively strong evidence to support these programs. We have not researched the efficacy of interventions for newborn conditions. Living Goods told us that it focuses on newborn conditions in particular because it has found that a large portion of child deaths occur in the first month of life and that there are interventions that can significantly reduce this mortality.45

CHPs also sell deworming medications and mosquito nets, on which we have completed evidence reviews and believe to be effective. However, the manner in which these products are distributed (one-off purchases rather than community-wide distributions) may affect their impact. In addition, these products account for a small portion of Living Goods’ sales.

Other products and services may also contribute to health, but we have not reviewed the evidence on their impact. Such effects might include improved maternal health from pregnancy support and post-natal visits, reduced indoor air pollution and injuries resulting from use of cookstoves and solar lighting, improved nutrition from the use of fortified foods and vitamins and minerals, and reduced infections from the use of delivery kits and soap for handwashing.

Note that evidence that the products CHPs supply improve health would not be sufficient to demonstrate Living Goods’ impact, because households might obtain the products from other sources in the absence of CHPs. However, this evidence helps provide context on whether the results discussed in the next section are plausible.

Living Goods notes that the drugs it sells are free in public facilities, but that clients may prefer to buy from CHPs because of the cost of traveling to facilities and possibility that the facility will be out of the medicine. According to Living Goods, clients may also be reluctant to buy drugs from other private providers because of the risk of getting a counterfeit medicine.46 Living Goods sent us a study conducted at the midline of its RCT that claims that both availability of counterfeit drugs and drug prices decreased at private retailers in areas where CHPs worked.47 Additional results on these potential effects will be made available when the full RCT is published. We have not yet reviewed the midline study or vetted other claims about how and why consumers choose to obtain medicine.

What does the randomized controlled trial of Living Goods’ program say about its impact?

Researchers conducted a cluster-randomized controlled trial (RCT) of the CHP program in Uganda between 2011 and 2013. The study included both villages served by CHPs managed by Living Goods and villages served by CHPs managed by BRAC.48 The researchers have published an abstract on the study, and shared a more in-depth report with us. The more in-depth report is not yet cleared for publication because the authors are seeking publication in an academic journal.49 The report we have seen is not a full write-up; it lacks details such as a description of the baseline survey, analysis of sample attrition, and confidence intervals for results. We plan to publish a review of the study once the authors have published the full report.

The study included 214 rural villages and surveyed 8119 households. It found that the program reduced under-five mortality by 25%. The authors note:50

The Community Health Promoter program could affect mortality through a number of channels, including improved access to treatment and health services, improved quality of treatment and health services, better access and knowledge of prevention, and by influencing other actors to improve the quality of services and products that they provide/sell. We find evidence supporting all these channels, including a 17% increase in treatment of diarrhea with ORS and zinc; an 54% increase, albeit starting from low levels, in follow-up visits for under-five children falling sick with malaria, ARI or diarrhea and an 72% increase, again starting from a low level, in home visits in the first seven postnatal days. While the likelihood of treatment with ACTs and antibiotics are similar across assignment arms, households in treatment villages are significantly more likely to purchase ACTs, antibiotics, and ORS/zinc from CHPs. Earlier research has shown that the quality of health products such as ACTs is low in many local markets, so the changed consumption pattern, and/or the increased competition as a result of the entry of the CHPs, most likely increased the quality of curative treatment of treatment households.

We find these results highly promising, however we have not fully vetted the results and note that the strongest programs we have considered are backed by multiple studies, while there has only been one rigorous study of the Living Goods program. In addition, from the information we have seen, we have the following concerns:

  • It is our understanding that child mortality rates were not measured at baseline. It is possible that there were differences in mortality rates between treatment and control groups from the start. While randomization and the sample size makes this unlikely, researchers often conduct baseline surveys for RCTs to compare key variables, in order to check that randomization successfully created comparable groups.
  • It appears that villages may have been carefully selected for inclusion in the sample (i.e. the list that was then divided into treatment and control) and that only a selection of Living Goods and BRAC branches were considered for the list. While this shouldn’t bias the results of the study, it could affect how replicable the results are in other locations.
  • With such a program it is not possible to hide from participants whether or not they received the intervention and outcome measures rely on self-reports of events that may have occurred a few years in the past. Those who knew they were part of the program (because they had interacted with a CHP) may have been influenced to give more positive answers. However, this effect is less likely to impact mortality reports than questions on more subjective or less memorable events.
  • We don’t know whether surveyors knew which villages were treatment villages. We would guess that it was not possible to keep this knowledge from the surveyors since questions included whether the household had received services from a CHP.

What does Living Goods’ ongoing monitoring say about its impact?

Living Goods reports on such metrics as number of malaria and diarrhea treatments provided, % of malaria and diarrhea cases followed up within 2 days, number of pregnancy registrations, % of pregnant women delivering in a facility or with trained help, % of women visited in the first 48 hours after giving birth, and % of referral cases followed up. It reports these metrics for both its directly-managed network and for the BRAC network. We have seen data for January 2012 through June 2014.51

We put limited weight on these metrics because:

  1. The data for these metrics are self-reported by CHPs, who are evaluated on their performance on such metrics and provided financial incentives for certain behaviors,52 which may incentivize them to inflate their reports.
  2. There is some evidence that CHPs are inflating the data. Living Goods told us that the main way it checks the accuracy of these reports is to compare the number of malaria and diarrhea treatments it sells to CHPs, to the number of malaria and diarrhea cases that CHPs report treating.53 In January to June 2014, CHPs reported 12% more malaria treatments than Living Goods sold to CHPs and 29% more diarrhea treatments than Living Goods sold.54 Living Goods notes that because dosages vary by the age of the patient and CHPs may choose to stock up on key treatments, over a short time frame, these figures may not fully match.55
  3. It appears that Living Goods calculates the % of cases followed up as the total number of follow up visits reported divided by the total number of treatments provided. As a result, this rate can exceed 100%. The rate of cases of malaria and diarrhea followed up was over 100% in every month from February to June 2014, reaching a maximum of 133% in April.56

In addition, Living Goods publishes annual “stakeholder reports” which report, at a high level, on organization and program progress. A few interesting notes from the 2013 report:

  • In BRAC’s network "under five diarrhea and malaria treatments and sales per CHP fell due to procurement challenges in the second half of the year that affected pricing."57
  • Living Goods "developed credit products for both consumers and agents, with an impressively high repayment rate of over 99%."58 We do not know how the repayment rate was calculated.
  • Living Goods conducted surveys of CHPs in both Uganda and Kenya, and reported a selection of positive results.59 Living Goods sent us additional information about the CHP surveys, but we have not yet reviewed it.

Has Living Goods’ work with partners had an impact on those projects and have those projects improved or saved lives?

As discussed above, the RCT on Living Goods’ program includes an evaluation of the BRAC program that it has provided technical and financial assistance to. We have not seen information on whether Living Goods’ work with other partners has been impactful (nor do we know what portion of its total budget is spent on non-BRAC partnerships, though the total partnerships budget was 28% of programs in 2012-2014).60

Do the products that CHPs sell provide economic benefits?

Some Living Goods products, such as cookstoves and solar energy products, may provide economic benefits to customers by, for example, saving them money on fuel. Living Goods sent us estimates of how much customers who buy cookstoves and solar energy products save, but we have not yet reviewed these estimates.

Possible negative or offsetting impact

  • Are some CHPs harmed by the program? CHPs take on financial risk by joining the program. Living Goods estimates that its CHP turnover rate is about 20-30% per year, and that 70-80% of CHPs who leave chose to drop out of the program, while the rest are asked to leave.61
  • Does Living Goods divert skilled labor from other areas? CHPs are fairly well-educated and well-connected members of their communities. Would they have been adding value to their communities in other ways if they were not spending time as a CHP? Given the part-time nature of their work, we would guess that this is not a major concern.

What do you get for your dollar?

Living Goods' estimates

Living Goods has provided estimates of its future cost-per-life saved based on the lives saved per CHP in the RCT and modeling of future costs and numbers of CHPs. These estimates are based on Living Goods Uganda-specific budgets for 2015-2018 and the number of Living Goods-managed agents it expects to have each year. The "mid case" uses the overall mortality reduction estimated by the RCT (25.5%), which includes both Living Goods and BRAC branches, while the "low case" seeks to account for possible reductions in impact during scaling (using 20% for the impact on mortality), and the "best case" uses the mortality reduction observed for 2013 (42%).62

Our adjustments to Living Goods' estimates

Living Goods shared the details of how it calculated its estimates, and we think they are largely reasonable with a few caveats. There are a few additional assumptions we make in our estimates of Living Goods cost effectiveness:63

  • Living Goods has included only those costs that it incurs within Uganda.64 We have included Living Goods' full budget, minus expected program revenue.65 These additional costs support the program directly and indirectly. The budget Living Goods provided does not include costs of the Kenya program, so these are missing from our model.
  • Living Goods' estimates rely on an estimation of the lives saved per CHP based on the 2012 under-5 mortality rate in Uganda. Under-5 mortality fell quickly in Uganda from 2009 to 2012.66 If this trend continues, lives saved per CHP will likely be lower in the future. In some of our scenarios, we have assumed that mortality, in the absence of Living Goods' program, will continue to fall over the next four years.
  • We have added an adjustment for contamination of the RCT due to CHPs visiting control group households. This contamination could cause the RCT to underestimate the effect of the intervention.
  • We have added an adjustment for our subjective guess on whether the RCT results would be replicated if another RCT was done on the scale up program. This adjustment aims to capture both the strength of our belief in the accuracy of the RCT results as well as in the likelihood that Living Goods and BRAC will be able to replicate the results at a greater scale and in somewhat different locations (though still within Uganda).
  • We have varied, across scenarios, the number of years we include in the estimate. Since Living Goods expects its cost per CHP to decline as it scales, it is more conservative to only include costs and benefits for the next 1-2 years than it is to include all 4 years.

Two GiveWell staffers have entered parameters for the various assumptions to derive "pessimistic," "base case," and "optimistic" estimates of Living Goods' cost per life saved (similar in concept to Living Goods' "low case," "mid case," and "best case"). The two staffers used somewhat different assumptions but generated similar estimates:

  • Pessimistic: About $28,000 per life saved
  • Base case: About $11,000 per life saved
  • Optimistic: About $3,300 per life saved

Details in this spreadsheet. Our work on this model is ongoing, and we plan to ask Living Goods for some additional information that will help us improve the model (details in footnote).67

Note that the estimate does not take into account any other potential benefits of the program, such as improved health in non-fatal cases or economic benefits. The estimate focuses on under-5 mortality because this was the primary outcome examined by Living Goods’ RCT. We do not have enough information about Living Goods’ effects on other health outcomes to confidently include them in our analysis.

Marginal cost per life saved

The figures discussed above estimate the average cost per life saved by the Living Goods program. This approach is consistent with our approach to estimating cost effectiveness for our top charities (though is at an earlier stage) and seeks to deal with issues of "leverage" by treating all donor funds the same.

Another approach, and the one that Living Goods favors, would be to attempt to calculate a marginal cost per life saved by excluding costs that Living Goods expects to incur regardless of the number of CHPs it supports, namely, headquarters costs and the costs of supporting the partnership team. For simplicity, we assume that all costs incurred within Uganda are variable and that they increase linearly with the number of CHPs supported. In reality, there are likely some in-country costs that increase non-linearly (e.g. a single country director may be required regardless of the number of CHPs, or additional branches may be further from Living Goods' main office, increasing transportation costs for field managers to oversee each branch as the number of CHPs grows).

With these assumptions, we estimate a base case of $7,500-8,500 per life saved.68 As noted above, this estimate is not directly comparable with cost effectiveness estimates we have created for other charities.

Room for more funding?

Scaling up in Uganda

Having recently completed the RCT, Living Goods is now aiming to rapidly scale-up the networks of CHPs directly managed by Living Goods and managed by BRAC, from about 1,000 CHPs (400 Living Goods, 600 BRAC) to 6,500 CHPs (2,500 Living Goods, 4,000 BRAC).69

In total, it is seeking funding to support a budget of about $10.3 million per year, including costs incurred by BRAC. It expects to raise some of this from partnership consulting fees and margins on goods sold to CHPs, leaving an estimated $8.9 million per year that would need to be supported by donor funding.70 In 2012 Living Goods raised $2.8 million, and in 2013 it raised $3.3 million from donors.71 Living Goods told us that it believes there is a decent chance it will reach two-thirds of its funding target for the first year through agreements with funders who have supported its work in the past, but the money has not yet been secured and the funding need will grow each year.72 It is actively seeking funding from new donors to fill the gap.73

Uganda scale-up budget 2015-2018 (in millions)74

Expenditures Program revenue Donor funding needed
Living Goods Uganda $10.25 $1.01 $9.24
BRAC Uganda $18.37 $1.27 $17.10
Partnerships $5.60 $3.25 $2.35
Other (primarily US-based) $7.03 $0.00 $7.03
TOTAL $41.25 $5.53 $35.73

Living Goods expects its scale-up in Uganda to be similar to its past work in the country.75 Differences may include:

  • A greater number of agents per branch.76
  • An increased focus on nutrition products and nutrition education.77
  • Work by BRAC on improving the quality of local health facilities, particularly for delivery care, through training, checklists, and new protocols.78

Living Goods has also noted a couple of ways in which the setting it is working in has changed since the start of the project: bednet coverage is 2-3 times higher and the market price of malaria treatment has been reduced.79 Under-5 mortality in Uganda, according to the World Bank, decreased from 83 per 1,000 live birth in 2009 to 69 in 2012.80

Replication seed fund

Living Goods has told us that scaling up in Uganda is its top priority, but that it is also interested in creating a “seed fund” for replications of its program by partners. If it received sufficient funding for its scale-up, it may allocate unrestricted funds to the replication seed fund.81 Living Goods has proposed creating a seed fund of $15 million to provide co-funding for programs in three countries that would reach 3-5 million people per country; it estimates that the full cost of a new country program would be $10-15 million,82 so its support would cover 33-50% of the total. It expects that having a seed fund would improve its ability to negotiate and launch new projects.83

It plans to target countries based on:84

  1. Need/impact: high child mortality rates driven by malaria, diarrhea, respiratory infections, and neonatal conditions; high rates of counterfeit drugs; high drug prices; low rates of use of water filters, clean cookstoves, and solar lights.
  2. Economics/sustainability: reasonable household purchasing power, and large (over 15 million) populations with high population density.
  3. Governance: low security risk and a government that will allow the program to sell medicines.
  4. Current partner presence: Living Goods would prefer to work with partners it already has relationships with in other countries.

Based on these criteria, Living Goods is most strongly considering partnerships in Nigeria, India, Ghana, Zambia, Kenya, and Myanmar.85 Possible partners include PSI, Society for Family Health Nigeria (an organization founded by PSI), Management Sciences for Health, CARE, BRAC, Save the Children, and Plan International.86

Major questions for further investigation

  • Can we increase our confidence in the impact of Living Goods’ past work? In order to improve our understanding of and confidence in Living Goods’ track record, we would be interested in better understanding:
    • Are Living Goods staff carrying out randomized follow-ups with clients, and if so, what are they finding?
    • Does the RCT have methodological problems that undermine the results?
    • Where would clients obtain health products from in the absence of CHPs? What would the quality of these products be?87 Are there products they would not have had access to?
    • Are Living Goods products consistently high quality?
    • How effective are cookstoves, fortified food, solar lighting, and delivery kits at improving health? Do cookstoves and solar lighting provide economic benefits?
    • What have been the outcomes of past partnerships, both in terms of Living Goods’ effect on the projects and the projects’ effects on the populations served?
    • How much money do CHPs lose on average when they leave the program and how does the loss affect them?
  • Will future work be as impactful as past work? How has child health and access to health goods and services changed in Uganda since the start of the RCT? How similar will replications outside of Uganda be to the program that was studied with the RCT and how similar will the new contexts be to that of Uganda?
  • Will we be able to learn about the impact of future work? How will Living Goods evaluate the CHP program going forward? Do CHP reports provide accurate information on program progress? How will partner projects, including BRAC in Uganda and PSI in Myanmar, be monitored and evaluated, and will Living Goods be able to share this information with us?
  • How cost effective is the CHP program? We have a a few detailed questions for Living Goods that will help us improve our cost effectiveness model.
  • Will Living Goods’ funding gap be filled by other funders? How likely is it that Living Goods receives full funding for its scale-up from other sources? Will BRAC raise funds for its portion of the scale-up?
  • We also have a few specific questions about how the program works.
    • What portion of past spending on partnerships was on the BRAC partnership versus other partnerships?
    • How are CHP incentive payments structured?
    • Why is Living Goods working in Kenya? Will it be able to sell medicines there in the future? Is it planning to scale-up that program?
    • What are the “credit products” that Living Goods offers?88

Sources

Document Source
Bjorkman-Nykvist et al. 2013 Source (archive)
Bjorkman-Nykvist et al. 2014 abstract Source (archive)
Chuck Slaughter, conversation with GiveWell, July 29, 2014 Unpublished
Email from Chuck Slaughter, Founder and President, Brad Presner, Director of Analytics, and Lisa McCandless, Director of Development, Living Goods, October 6, 2014 Unpublished
GiveWell spending analysis for Living Goods Source
GiveWell estimates of Living Goods cost effectiveness Source
GiveWell's non-verbatim summary of a conversation with Chuck Slaughter, July 20, 2012 Source
GiveWell's non-verbatim summary of a conversation with Chuck Slaughter, May 5, 2014 Source
Lisa McCandless, Living Goods Director of Business Development, email to GiveWell, July 24, 2014 Unpublished
Living Goods BRAC Partnership Source (archive)
Living Goods Draft Branch Management Handbook Unpublished
Living Goods budget (2014) Source
Living Goods draft budget (2015-2018) Unpublished
Living Goods Business-In-A-Bag Source (archive)
Living Goods CHP Interview Guide Source
Living Goods CHP Interview Test Source
Living Goods CHP Profile and Expectations Source
Living Goods Draft Cost per Capita Served Methodology Source
Living Goods Draft Cost per Life Saved Methodology Source
Living Goods GiveWell Document Request Guide Source
Living Goods Health Dashboard (June 2014) Source
Living Goods Kenya Monthly Sales Unpublished
Living Goods Org Chart (August 2014) Source
Living Goods Partner Organizations Source (archive)
Living Goods Product List (July 2014) Source
Living Goods Quarterly Dashboard Source
Living Goods Draft Replication Strategy (September 2014) Source
Living Goods Sales Dashboard (June 2014) Source
Living Goods Stakeholder Report (2012) Source
Living Goods Stakeholder Report (2013) Source
Living Goods Training & Coaching Source (archive)
Living Goods Treatment vs. Sales Summary (January to June 2014) Source
Living Goods Ugandan Monthly Sales Unpublished
Living Goods unaudited financials (2013) Unpublished
Living Goods Where We Work Source (archive)
Living Goods-BRAC Draft Scale Up Concept Note (August 2014) Source
WHO/UNICEF Integrated Community Case Management Source (archive)
World Bank under-5 mortality rate data Source (archive)